MCQ Advanced
Français
GUIDE · WRITING A GOOD TEST
A good question
cannot be guessed.
A multiple-choice test measures what the class understood only if the correct answer does not give itself away, if every distractor matches a real mistake, and if the correction teaches something. The fourteen pieces of advice below are the ones the editor checks as you type: every warning links here.
They come from published item-writing guidelines (Haladyna, Downing and Rodriguez, 2002) and from classroom use. They are advice: they never block publishing.
MISSING_QUESTION VAGUE_STEM
Really ask the question.
A statement that asks nothing makes students guess what is expected from the options. They then answer the question they imagined, not yours. The statement should stand alone: one should be able to answer it in one's head before reading the options.
Same trap with code: if some options are about compiling and others about running, say which one you ask.
Photosynthesis.
Which gas does photosynthesis release?
NEGATIVE_STEM
Make negatives visible.
"Which one is not..." or "all except..." are read too fast: a student who understood answers the opposite because they skipped the "not". If the question really is about an exception, write the word in capitals or bold. Otherwise, turn it into a positive question.
Which of these numbers is not prime?
Which of these numbers is NOT prime?
TOO_MANY_OPTIONS
Four options are enough.
Beyond four, the extra options are rarely chosen: they lengthen reading without making the question fairer. Three good distractors beat five of which two fool nobody. Keep the ones that match a mistake you have already seen in class.
META_OPTION
Neither "all" nor "none".
"All of the above" is found by partial elimination: recognising two correct options is enough to tick it. "None of the above" does not say which misconception the student had in mind. Either way, the distractor analysis loses what it looks for. If several options are correct, make it a question with several answers.
Which animals are mammals? A. the whale, B. the bat, C. all of the above
Several answers: the whale, the bat, the shark, the penguin.
LENGTH_BIAS LENGTH_RANK_BIAS
Length gives nothing away.
To be exact, the correct answer gets qualified, and grows longer. Students know it: "when in doubt, pick the longest". The editor flags a correct answer at least 1.6 times longer than the longest distractor, then, from ten questions, a paper where it is the longest more than one time in three. Shorten the correct answer or flesh out the distractors, with the same precision.
A. Paris · B. Lyon · C. Marseille, the country's second city by population
A. Paris · B. Lyon · C. Marseille
ANSWER_POSITION_BIAS
The answer moves around.
When writing, one often puts the correct answer first, or second. From ten questions, the editor flags a paper where more than 60 % of the correct answers sit at the same position. Shuffling the options at sitting time fixes the test, not a printed paper or proofreading: vary positions as you write.
NO_EXPLANATION PARTIAL_EXPLANATION
Explain every option.
The correction is when the student learns. Explaining only the correct answer does not tell them why their choice was wrong. One line under each distractor is enough: it names the mistake that leads there. It is also what makes the class analysis readable: "40 % chose B, that is, they added the denominators".
- [ ] 2/5
- [x] 5/6
> Over the same denominator, 3/6 + 2/6.
- [ ] 2/5
> Denominators are never added.
- [x] 5/6
> Over the same denominator, 3/6 + 2/6.
NO_RULE NO_COURSE_REF
A rule, a place to revise.
A RULE: line gives the sentence to remember, the one you would like the student to tell themselves next time. A COURSE: on the theme, or a REVIEW: on the question, says where to revise: the correction and the revision sheet show them, and the "Review first" list points to the right chapter.
RULE: To add two fractions, first put them over the same denominator.
REVIEW: Chapter 3, fractions
DUPLICATE_QUESTION
Never the same question twice.
Two nearly identical questions count the same idea twice and skew the theme's success rate. The editor flags those sharing at least 72 % of their words, in the paper or in another paper of the same folder. Keep the better one, or make the other test a different aspect.
LEVEL_MISSING TOO_MUCH_RECALL NOT_ENOUGH_APPLY
Understand, not only recall.
Each question carries a level from 1 (remember) to 5 (judge, create). A paper made mostly of definitions measures vocabulary, not understanding. The editor flags more than 15 % of level 1 questions, or fewer than half at levels 3 to 5. To go up a level, give a situation and ask what happens.
What is the formula for average speed?
A train covers 120 km in 1 h 30 min. What is its average speed?
EMPTY_THEME THIN_THEME
Themes with enough questions.
The analysis tells the class which themes to go over again. With one or two questions, a success rate means nothing: a single misread question brings the whole theme down. Aim for at least three questions per theme, and remove empty themes.
TOO_LONG DURATION_MISMATCH
A bearable duration.
The editor estimates the time from each question's planned time, with a 20 % margin. A shorter announced duration puts the class under pressure, and the last questions measure speed. Beyond 90 minutes, it is fatigue that gets measured: remove questions rather than lengthen the test.
MEDIA_WITHOUT_ALT HEAVY_MEDIA HEAVY_PAPER
Accessible, light media.
An image without a text alternative is silent for a student using a screen reader. Describe what one needs to see to answer, without giving the answer away. As for weight, a file above 5 MB, or more than 25 MB in all, opens slowly on a phone or a busy network: compress the image, shorten the video.
STRAY_DOLLAR
The dollar sign.
In a statement, $...$ writes a formula. A lone dollar that opens no formula shows as is, but it is often a formula left unclosed. For a price, write \$: the editor then knows it is meant.