Alice (in Wonderland) has nibbles a bit.
Did she grow tall or small?
How to design multiple-choice questions.
Since taking the plunge and adding multiple-choice
questions to my assessment repertoire,
I’ve found they have refreshing and unexpected advantages.
They make assessment more reliable, marking less
labour-intensive,
pupil understanding and misconceptions more visible, and
allow a wider breadth of knowledge
to be assessed across a unit than just using essays or
complex, holistic end-of-unit assessments.
They save countless hours of marking downstream,
and get pupils thinking deeply about subject content.
Both Alex Quigley and Cristina Milos have written perceptively about how
tricky they are to create. How can we ensure that the advantages outweigh the
limitations?
Research from Little, Bjork, Bjork and Angello (that Alex cites) suggests not only
that they are as effective as short-answer tests for
retention, but they also have
an important advantage over them – that pupils have to
think through incorrect alternatives.
The key insight is that these alternatives must be plausible enough to enable pupils
to retrieve why correct alternatives are correct and
incorrect options are incorrect.
I’ve found these seven principles helpful in
multiple-choice design:
1. The proximity of options
increases the rigour of the question
For instance, the question is, what year was the battle
of Hastings?
Options 1065, 1066, 1067, 1068 or 1069 are more rigorous
than
options 1066, 1166, 1266, 1366 or 1466. Of course, the
question itself also determines the rigour: ‘80 is what percentage of 200?’ is
much easier than ‘79 is what percentage of 316?’
2. The number of incorrect
options increases rigour
Three options give pupils a 33% chance of guessing the
correct answer;
five options reduces the chances of guessing to 20%;
always create five rather than three or four options for
multiple choice questions.
A ‘don’t know’ option prevents pupils from blindly guessing,
allowing them to flag up questions they’re unsure about
rather than getting
lucky with a correct guess.
3. Incorrect options should be
plausible but unambiguously wrong
If options are too implausible, this reduces rigour as
pupils can too quickly dismiss them.
For instance, in the question: what do Charles Dickens
and Oliver Twist have in common,
an implausible option would be that they were both bank
robbers.
However, if answers are too ambiguously similar, this
creates problems. For instance,
in the question, ‘What happens in the plot of Oliver
Twist?’, these options are too ambiguous:
a) A young boy runs away to
London
b) An orphan falls in with a
street gang of street urchins
c) A poor orphan is adopted by a
wealthy gentleman
d) A criminal murders a young
woman and is pursued by a mob
e) A gang of pickpockets abduct a
young boy
4. Incorrect options should be
frequent misconceptions where possible
For example, if you know pupils often confuse how
autobiographical ‘Oliver Twist’ is,
create options as common confusions.
These distractors flag up what pupils are thinking if
they select an incorrect option:
a). Both were born in a workhouse
b). Both were separated from their parents and family
c). Both were put in prison for debt
d). Both had families who were put in prison for debt
e). Both were orphans
5. Multiple correct options
make a question more rigorous.
Not stating how many correct options there are makes
pupils think harder. For example:
Which characteristics of “Elegy Written in a Country
Churchyard” can be seen as Romantic?
A. It celebrates the supernatural.
B. It is written in iambic pentameter.
C. It emphasises emotion over reason.
D. It deals with the lives of common people.
E. It aspires to nature and the sublime.
6. The occasional negative
question encourages kids to read the questions more carefully.
Once they get a question like ‘Which of these is NOT a
cause of World War 1?‘ wrong, and realise why, they’ll work out they need to
read questions again to double-check on what it is they’re asking.
7. Stretch questions can be
created with comparisons or connections between topics.
What was common to both the USA and Germany during the
Great Depression?
a) Jewish immigration increased
b) Membership of Ku Klux Klan
increased
c) Public works projects were
implemented
d) Government social programs
were reduced
Good, Better, Best
Here is an example of honing a question to take into
account the key insight that multiple-choice options must be plausible, but unambiguously
distinctive. Pupils aren’t able to work out the good, better,
or best options from context alone. They’d need to carefully think through the
nuances.
What does the word ‘resilient’
mean in this sentence?
Her resilient attitude toward life enabled her to
overcome difficult situations.
Bad multiple-choice options are
not plausible
A. depressing
B. dishonest
C. flexible
D. anxious
Good multiple-choice options
are plausible and not too ambiguous
A. flexible
B. energetic
C. positive
D. enthusiastic
Better options are plausible
and less ambiguous
Her resilient attitude toward life enabled her to
overcome difficult situations.
A. flexible and durable
B. energetic and enthusiastic
C. positive and creative
D. logical and calm
Best multiple-choice options
plausible yet distinctive
Her resilient attitude toward life enabled her to
overcome difficult situations.
A. flexible and durable, with the ability to bounce
back from setbacks
B. energetic and enthusiastic, with the ability to
turn negatives into positives
C. positive and creative, with the ability to
make something out of nothing
D. logical and calm, with the ability to solve
complex problems
So, multiple-choice questions require us to design
options carefully
if they are to become a valuable part of our assessment mix.
Deep roots produce sweet fruits
If Heads of Department invest the up-front time to create
them for their subject,
their teachers stand to benefit from a strong return on
that investment in terms of time saved
and pupil thinking – for years to come.
You can TCR specialist and language dictionaries
that are spontaneously accessed.
I can Turbo Charge Read a novel 6-7 times faster and remember what
I’ve read.
I can TCR an instructional/academic book around 20 times faster and remember what I’ve
read.
A
practical overview of
Turbo Charged Reading YouTube
How to choose a
book. A Turbo Charged
Reading YouTube
Emotions
when Turbo Charged Reading YouTube
Advanced Reading Skills Perhaps you’d like
to join my FaceBook group ?
Perhaps you’d like to check
out my sister blogs:
www.innermindworking.blogspot.com gives many ways for
you to work with the stresses of life
www.ourinnerminds.blogspot.com
take advantage of business experience and expertise.
www.happyartaccidents.blogspot.com
just for fun.
To quote the Dr Seuss
himself, “The more that you read, the more things you will know.
The more that you learn; the
more places you'll go.”
No comments:
Post a Comment
Your opinions, experience and questions are welcome. M'reen