writing test items:

MULTIPLE CHOICE QUESTIONS

CIMC
w w w. o k c a r e e r t e c h . o r g / c i m c 800-654-4502

WRITING TEST ITEMS: Multiple-Choice Questions

Developed by Oklahoma Department of Career and Technology Education Testing Division 2000

INTRODUCTION

Multiple-choice questions (MCQs) play an important role in achievement testing and are among the most widely used of selection-type items on examinations today. The popularity of the MCQ stems from the fact that they can be designed to measure a variety of learning outcomes. It takes a great deal of time, skill, and adherence to a set of well-recognized rules for item construction to develop a good MCQ item. The purpose of this guide is to present and explain to test item writers the rules for developing good multiple-choice questions.

NATURE OF THE MULTIPLE-CHOICE QUESTION (MCQ) Multiple-choice questions consist of a stem, options (distractors), and a key. What is the leading cause of death in the 15-24 age group? stem key *A. accidents B. cancer options (distractors) C. heart disease D. respirator disease

1. Stem - a question or an incomplete statement which presents the problem. The stem of a MCQ may be presented either as a question or as an incomplete statement. The choice of form makes no difference in the overall effectiveness of the stem, as long as the item writer presents a clear and specific problem. An example of the two different types of stem forms are found below: Sample MCQ Stem Written as a Question: Which of the basic food groups is richest in calcium? A. B. C. D. breads and grains dairy products* fruits and vegetables meat and poultry

Sample MCQ Stem Written as an Incomplete Statement: An example of a supply-type test item is the: A. B. C. D. multiple-choice item true-false item matching item short-answer item*

2. Options (distractors) - the answer choices or options, referred to as distractors, given to provide possible solutions to the problem. They function to distract those individuals who are uncertain of the answer. Typically, there are four options: a. b. one correct answer (the key), and three distractors.

3. Key - the correct or best answer. The key is typically denoted with an *.

RULES FOR WRITING MULTIPLE-CHOICE QUESTIONS The goal of the multiple-choice (MCQ) item format is to present students with a task that is both important and clearly understood, and that can be answered correctly by anyone who has achieved the intended learning outcome. There should be nothing in the content or structure of the item that would prevent an informed student from responding correctly. Similarly, nothing in the content or structure of the item should enable an uninformed student to select the correct answer. The following rules for item writing are intended to guide the item developer to design MCQs that function as intended. The rules are divided into two categories: (1) rules for developing the stem and (2) rules for developing the options. The Stem 1. Present a single, clearly formulated problem in the stem of the item. The stem should contain only information that is pertinent to the question or problem and should be worded so that the examinee can understand it without reading the options.

Poor: The cell islets of the pancreas: A. B. C. D. are located around the edge of the pancreas. contain ducts. disappear as one grows older. produce insulin.*

Better: The cell islets of the pancreas secrete the substance called: A. B. C. D. trypsin. insulin.* tryptophan. adrenaline.

The first example is merely a collection of true-false statements with a common stem. However, the stem in the second example presents a single problem. A good test to check the clarity and completeness of a multiple-choice stem is to cover the alternatives and determine whether it could be answered without the choices. 2. State the stem of the item in simple, clear language. The problem in the stem of a MCQ should be stated clearly, using straightforward vocabulary and should be free of unnecessary complex wording and sentence structure. Poor: Cells of one kind belong to a particular group performing a specialized duty. We call this group of cells a tissue. All of us have different kinds of tissues in our bodies. Which of the following would be classified as epithelial tissue? A. B. C. D. adenoids and tonsils cartilage mucous membranes* tendons

Better: Which of the following would be classified as epithelial tissue? A. B. C. D. adenoids and tonsils cartilage mucous membranes* tendons

The stem in the first example contains unnecessary material which increases the amount of reading needed for an item, making it more difficult for the examinee to distinguish between relevant and irrelevant material. 3. Put as much of the wording as possible in the stem of the item. The stem should include as much of the item as possible. If the same words or phrases are repeated in all or most of the options, rewrite the stem to include the repetitious material. Note the following examples: Poor: In objective testing, the term objective: A. B. C. D. refers to the method of identifying the learning outcomes. refers to the method of presenting the problem. refers to the method of selecting the test content. refers to the method of scoring the answers.*

Better: In objective testing, the term objective refers to the method of: A. B. C. D. 4. identifying the learning outcomes. presenting the problem. selecting the test content. scoring the answers.* State the stem of the item in positive form, wherever possible. Avoid using negatives such as NO, NOT, and EXCEPT. If you must use a negative word, write it in capital letters and underline it so that the test-taker will not miss it. Poor: Which of the following structures of the ear is NOT concerned with hearing? A. B. C. D. cochlea eardrum oval window semicircular canals*

Better: Which one of the following structures of the ear helps to maintain balance? A. B. C. D. Poor: Which one of the following is not a desirable practice when preparing multiple-choice items? A. B. C. D. Shortening the stem by lengthening the alternatives.* Stating the stem in positive form. Underlining certain words in the stem for emphasis. Using a stem that could function as a short-answer item. cochlea eardrum oval window semicircular canals*

Better: All of the following are desirable practices when preparing multiple choice items EXCEPT: A. B. C. D. shortening the stem by lengthening the alternatives.* stating the item in positive form. underlining certain words in the stem for emphasis. using a stem that could function as a short-answer item.

Please note that when negative wording is used in the stem of an item, not only should it be emphasized by placing it in capital letters and underlining it, but also placed near the end of the statement. 5. Make certain that the intended answer is correct or clearly best. When the correct-answer form of a MCQ item is used, there should be only one unquestionably correct answer. With the best-answer form, the intended answer should be one that content experts would agree is clearly the best.

Poor: Which of the following types of cattle is a dairy breed? A. B. C. D. Angus Guernsey* Hereford Holstein*

Better: Which of the following types of cattle is a dairy breed? A. B. C. D. Angus Brahman Hereford Holstein*

In the first example, both B and D are correct answers. They both are dairy breeds of cattle. In the second example, the only correct answer is D. All the other options are breeds of meat cattle.

The Options 1. Make all options grammatically consistent with the stem of the item and parallel in form. Be sure to check the options against the wording in the stem to make sure they are grammatically consistent. This will help to avoid easy elimination of options. Poor: Penicillin is obtained from a: A. B. C. D. bacterium. coal-tars. mold.* tropical tree

Better: Penicillin is obtained from: A. B. C. D. bacteria. coal-tars. mold* tropical tree

In first example, a coal-tars is not grammatically correct when the stem and the option is read as a complete sentence. 2. Avoid giving clues in the correct answer by providing common verbal associations to words that are in the stem. Poor: What is the best way to locate a lawyer for assistance? A. B. C. D. Contact a lawyer referral service.* Contact a police officer. Look for an ad in the newspaper. Wait until you go to jail.

Better: What is the best way to locate a lawyer for assistance? A. B. C. D. Ask a police officer. Call a bail bonding agency. Contact a legal referral service.* Look for an ad in the phone book.

Note that in the first example, the word lawyer is in both the stem and the correct answer. 3. Be sure that the wrong answers are plausible. The distractors must be attractive to examinees who are lacking in knowledge about the material the item is intended to assess. Therefore, the incorrect answer choices should be logically consistent with the stem and should represent common errors made by students at a particular ability level.

Poor: Subtract 8032 - 5743

A. B. C. D.

2289* 2288 2378 3378

Better: Subtract 8032 -5743

A. B. C. D. 4.

2289* 2389 [failing to change 0 to 9] 3399 [failing to decrease two digits borrowed from] 3711 [subtracting the big number from the small one] Vary the relative length of the correct answer to eliminate length as a clue. All options should be of approximately the same length. Avoid any tendency to make the correct answer consistently longer or shorter than the distractors.

Poor: One advantage of multiple-choice items over essay questions is that they: A. B. C. D. depend more on recall. measure more complex outcomes. provide for a more extensive sampling of course content.* require less time to score.

Better: One advantage of multiple-choice items over essay questions is that they: A. B. C. D. place greater emphasis on the recall of factual information. provide for a more extensive sampling of course content.* provide for the measurement of more complex learning outcomes. require less time for test preparation and scoring.

5.

Use the option "none of the above" only when the keyed answer can be classified unequivocally as correct or incorrect. The "none of the above" option works better when the stem is stated as a question rather than a sentence to be completed. An incomplete sentence rarely works because "none of the above" seldom completes the stem grammatically. Nevertheless, the use of the "none of the above" option should be avoided.

6.

Avoid the use of "all of the above". The "all of the above" option makes it possible to answer the item on the basis of partial information. Since the examinee is to select only one answer, they can detect "all of the above" as the correct choice simply by noting that two of the alternatives are correct. They can also detect it as a wrong answer by recognizing that at least one of the alternatives is incorrect, thus enhancing their chance of guessing the correct answer from the remaining choices.

Poor: Which of the following factors must be considered in computing basal energy requirements? A. B. C. D. age height weight all of the above*

Better: Which of the following factors must be considered in computing basal energy requirements? A. B. C. D. 7. age only weight only height and weight only age, height, and weight* Alphabetize the distractors and arrange numerical options in ascending order. To ensure that the placement of the correct answer, or key, does not follow a pattern that may be apparent to the examinee, the options should be listed in alphabetical order. If the options are numerical, then the options should be listed in ascending order.

Poor: A communication system for the blind that includes raised dots that are arranged to represent letters of the alphabet is called: A. B. C. D. Braille* Sign language Morse code Arabic lettering

Better: A communication system for the blind that includes raised dots that are arranged to represent letters of the alphabet is called: A. B. C. D. Poor: How long should an orphan calf be fed warm whole milk? A. B. C. D. 14 days 21 days 10 days* 28 days Arabic lettering Braille* Morse code Sign language

Better: How long should an orphan calf be fed warm whole milk? A. B. C. D. 10 days* 14 days 21 days 28 days

MULTIPLE-CHOICE QUESTION EVALUATION CHECKLIST Stem Does the item present a single, clearly formulated problem? Is the item stated in simple, clear language? Is the stem worded so that there is no repetition of material in the options (distractors)? Is the stem stated in the positive wherever possible? If negative wording is used, is it emphasized (by underlining and caps)? Is the intended answer correct or clearly best? Options Are all the options grammatically consistent with the stem of the item and parallel in form? Are the options free from verbal clues to the correct answer? Are the distractors plausible? Is the relative length of the correct answer similar to that of the distractors? Has the option "none of the above" been used in its appropriate fashion? Has the option "all of the above" been avoided? Have all the options been arranged in alphabetical order? Have all numerical options been arranged in ascending order?

CIMC
Curriculum and Instructional Materials Center 1500 West 7th Avenue Stillwater, OK 74074-4364 800.654.4502 Fax 405.743.5154 www.okcareertech.org/cimc