Binary-Choice Test Items
Binary-Choice Test Items
Binary-Choice Test Items
When students select an answer from only two options, they are completing
a binary-choice item, also called alternative response. The most common
binary-choice item is the true/false question. Other types of options can be
right/wrong, correct/incorrect, yes/no, fact/opinion, agree/disagree, and so on. In
each case, the student selects one of two options. In this section, we will use the
terms binary-choice, alternative response, and true/false (TF) interchangeably.
Miller et al. (2009) notes that binary-choice items are popular probably
because they are quick and easy to write, or at least they seem to be. It is true that
these items do take less time to write than good objective items of any other
format, but good binary-choice items are not that easy to write.
Before we proceed to discussing good practices in writing binary-choice
items, test your prior knowledge by accomplishing the following exercise. Use your
common sense to help you determine which are good items and which are poor.
Exercise: Put a G in the space next to the items you believe are good binary-choice
items and a P next to the items you feel are poor.
_____ 1. High-IQ children always get high grades in school.
_____ 2. Cognitive theorists believe that motivation to learn comes from extrinsic
factors.
_____ 3. If a plane crashed on the North and South Korean border, half of the
survivors would be buried in North Korea and half in South Korea.
_____ 4. The use of double negatives is not an altogether undesirable characteristic
of diplomats and academicians.
_____ 5. Prayer should not be outlawed in schools.
_____ 6. Of the objective items, true-false items are the least time-consuming to
construct.
_____ 7. The trend toward competency testing of high school graduates began in
the late 1970s and represents a big step forward for underachieving
learners.
In your notebooks, try to provide a brief explanation for each of your answers
in the exercise. To maximize learning, prepare you better for the discussions that
follow, and help you in the item writing activity later, make sure that you
accomplish the exercise before you read the answers and explanations provided
next.
Of the items presented in the exercise, only numbers two and six are
appropriately written. All the rest are poor items. Let’s discuss each below.
In item 1, the word always is an absolute. To some extent, alternative
response items depend on absolute judgments. However, statements of facts are
seldom completely true or completely false. Thus, an alert student will usually
answer “false” to items that include always, all, never, or only.
Remember that in classroom assessment, what we want to measure during
or after a learning experience is added knowledge or skill in students, not their test
wiseness. So to avoid this problem, reduce the effects of guessing by avoiding the
Item 2 is a good one. To answer the item correctly, the students would have
to know the perspectives of cognitive theorists about learning and motivation.
Item 3 is a trick question. “Survivors” of a plane crash are not buried! Chances
are that you never even noticed the word survivors and probably assumed the item
referred to fatalities. Trick items may have a place in tests of critical reading and
visual discrimination (in which case they would no longer be trick questions), but
seldom are they appropriate in the average classroom test. Rewritten, item 3 might
read:
If a plane crashes on the North and South Korean border, half the fatalities
would be buried in North Korea and half in South Korea.
Item 4 is also poor. First of all, it includes a double negative ⎼ not and
undesirable. Items with a single negative are confusing enough. Negating the first
negative with a second wastes space and test-taking time and also confuses most
students. If you want to say something, say it positively. The following revision
makes this item slightly more acceptable.
We said slightly more acceptable because the item is still troublesome. The
word altogether is an absolute, and we now know we should avoid absolutes since
there usually are exceptions to the rules they imply. When we eliminate altogether,
the item reads:
However, the item is still flawed because it states an opinion, not a fact. Is
the item true or false? The answer depends on who you ask. To most of us, the use
of double negatives is probably undesirable, for the reasons already stated. To some
diplomats, the use of double negatives may seem highly desirable. In short,
binary-choice statements should be factual. If you must use a binary-choice item to
measure knowledge of an opinionated position or statement, state the referent (the
person or group that made the statement or took the position), as illustrated in the
following revision:
Notice the word NOT in Item 5. When you include a negative in a test item,
highlight it in italics, underlining, or uppercase letters so that the reader will not
overlook it. Remember that you intend to determine whether your students have
mastered your objective, not to ensure low test scores.
Item 6 represents a good item. It measures factual information, and the
phrase “Of the objective items” qualifies the item and limits it to a specific frame of
reference.
The last item is deficient because it is double barreled: It is actually two items
in one. When do you mark true for a double-barreled item? When both parts of the
item are true? When one part is true? Or only when the most important part is true?
The point is that items should measure a single idea. Double barreled items take too
much time to read and comprehend. To avoid this problem, simply construct two
items, as we have done here:
The trend toward competency testing of high school graduates began in the
late 1970s.
The trend toward competency testing represents a big step forward for
underachieving learners.
According to DepEd, the trend toward competency testing represents a big step
forward for underachieving learners.
1. Write the item so that the answer options are consistent with the logic in
the sentence. The way the item is written will suggest a certain logic for
what type of response is most appropriate. For example, if you want to test
spelling knowledge, it doesn’t make much sense to use true/false questions;
it would be better to use correct/incorrect as options.
3. Avoid long, complex sentences. As noted earlier, a test item should indicate
whether a student has achieved the knowledge or understanding being
measured. Long, complex sentences tend to also measure reading
comprehension and therefore should be avoided in tests designed to
measure achievement.
Examples
Poor: T F A cup with hot water that has a spoon in it will cool more quickly
than a similar cup with the same amount of hot water that does not have a
spoon in it.
Improved: T F Hot water in a cup will cool more quickly if a spoon is
placed in a cup.
4. Avoid broad general statements if they are to be judged true or false. Most
broad generalizations are false unless qualified, and the use of qualifiers
provides clues to the answer.
Examples
Poor: T F The president of the Philippines is elected to that office.
Poor: T F The president of the Philippines is usually elected to that office.
In this example, the first version is generally true but must be marked false
because there are exceptions, such as when the vice president takes office in
event of the president’s death or impeachment. In the second version, the
qualifier usually makes the statement true but provides a definite clue.
Words such as usually, generally, often, and sometimes, are more likely to
appear in true statements, and absolute terms such as always, never, all,
none, and only are more likely to appear in false statements.
Examples
Poor: T F Philippine senators are not elected to six-year terms.
Improved: T F Philippine senators are elected to six-year terms.
10. Do not try to trick students. Items that are written to “trick” students by
including a word that changes the meaning of an idea or by inserting some
trivial fact should be avoided. Trick items undermine your credibility,
frustrate students, and provide less valid measures of knowledge.
Examples
T F Other things being equal, an electric stove with greater resistance will
be hotter than a stove with less resistance.
T F Jon is building a new electric motor. His decision to use thicker wire
results in less resistance.
Examples
If the statement is a fact, circle F; if it is an opinion, circle O.
F O Literature is ancient Rome’s most important legacy.
F O The word Mississippi has 11 letters.
F O The best way to wash a car is with a sponge.
If people are to think critically about a topic, they must first be able to
distinguish fact from opinion.
Additional reasoning skills can be assessed using the same approach by
developing some statements that are examples of the skill and some statements
that are not examples. This can be done with many of the critical thinking skills (e.g.,
identifying stereotypes, biased statements, emotional language, relevant data, and
verifiable data.)
Examples
If the statement is an example of a stereotype, circle S; if it is not a
stereotype, circle N.
Examples
If the second part of the sentence explains why the first part is true, circle T
for true; if it does not explain why the first part is true, circle F for false.
T F Food is essential because it tastes good.
T F Plants are essential because they provide oxygen.
T F Alex is tall because he has blue eyes.
Checklist for Writing and Reviewing Binary-Choice Items (McMillan, 2018; Miller
et al., 2009)
✓ Is this the most appropriate type of item to use?
✓ Is the type of answer logically consistent with the statement?
✓ Does the item contain a single idea?
✓ Are the statements briefly and clearly expressed?
✓ Is trivial knowledge being tested?
✓ Is the item stated positively?
✓ Are opinion statements attributed to some source?
✓ Have specific determiners (e.g., usually, always) been avoided?
✓ Are the true and false (or other binary) items approximately equal in length?
✓ Has a detectable pattern of answers been avoided?
✓ Is there an approximately equal number of true and false (or other binary)
items?
✓ Does the item try to trick students?
✓ If revised, are the items still relevant to the intended learning outcomes?
✓ Have the items been set aside for a time before reviewing them?
1
T/F types of items are not suitable for some types of subject matter. T/F
statements require course material that can be phrased so that the statements are
true or false without qualification or exception. There are areas in which such
absolutely true or false statements cannot be made. In some fields, such as the
social sciences, practically all significant statements require some qualification. In
some subject areas, only relatively trivial statements can be reduced to absolute
terms.
2
Miller et al., (2009) notes that the perceived ease of construction of T/F items has
probably resulted from the common practice of taking statements from textbooks,
changing half of them to false statements, and submitting the product to students
as a T/F test. Such test items are often too obvious that everyone gets them correct
or so ambiguous that even the better prepared students are confused. In short, it is
easy to construct poor T/F items. To construct clearly stated T/F items that measure
significant learning outcomes, however, requires much skill.
Activity 5D
Task Description: This activity is designed to test your ability to construct binary
choice items according to guidelines presented.
Task Instructions:
1. Based on the test blueprint constructed for Activity 5A, select two or three
instructional objectives that can be measured using binary-choice items.
2. Write a total of 10 binary-choice items. Each objective selected should have
3-5 items that appropriately measure the learning outcome targeted.
3. Indicate the level of thinking, the objective, and the directions for every set
of items.
4. Review the test items using the checklist provided in this module.
5. Make necessary revisions.