Why use multiple-choice questions?

Why use multiple-choice questions?

Blind justice: rebalancing the weighing scales 

There is some scepticism as to whether multiple-choice questions are useful.

A while ago, I shared this scepticism. I assumed they weren’t very rigorous.

Now that I’ve been using multiple-choice questions for some time, inspired by Daisy Christodoulou, 

I’m beginning to realise why they are so useful. Daisy argues that they can require

higher order thinking, if the distractors home in on important, frequent misconceptions.

Daisy went through all of the literature and history multiple choice questions

on British Columbia leaving exam, and says ‘all of them made me think hard about deep issues.’

This fits with Professor Rob Coe’s simple theory of learning:

learning happens when you have to think hard about subject content.

Phil Stock has also come on a similar journey to me in his thinking on multiple-choice questions.

Here are some of the reasons why I think they can benefit teaching:

1. They make assessment more reliable

2. They make marking far less labour-intensive

3. They make pupil understanding more visible to teachers

Let’s take each in turn.

1. They make assessment more reliable 

Reliability is best understood with the weighing scale analogy:

the chance that if the object has not changed weight, the weighing scale will tell us the same weight. Similarly, you want different assessors to give the same answer the same score.

Extended essays are easy to set but hard to assess reliably.

Anyone who has been in an English or humanities moderation meeting,

with different teachers giving the same essay different scores on the same rubric, will know that.

Multiple-choice questions are hard to set but are always reliable to assess.

My initial assumption that they weren’t rigorous was wrong: it depends on how they are set.

Paul Bambrick Santoyo makes the following case:

    In an open-ended question, the rubric defines the rigour

    In a multiple choice question, the options define the rigour

    Effective assessment combines them: both are useful

As a demonstration, take two example option sets:

When did Charles Dickens write Oliver Twist?

a). 1537

b). 1637

c). 1737

d). 1837

e). 1937

or

a). 1835

b). 1836

c). 1837

d). 1838

e). 1839

The closer the options, the tougher the question. The first option set is not very rigorous;

all you have to know is that Dickens wrote in the 19^th century.

The second is more rigorous; you have to know the exact year it was written*.

Of course, the nature of the question changes the rigour as well as the complexity

of the options: What is 17% of 200? is a less rigorous question than What is 79% of 158? 

This goes for English, too:

What do the author Charles Dickens and his character Oliver Twist have in common?

a). Both were born in a workhouse

b). Both were separated from their parents and family

c). Both were put in prison for debt

d). Both had families who were put in prison for debt

e). Both were orphans

To answer that, you need a clear understanding of plot, character, author and context,

and you have to think hard about them.

2. They make marking far less labour-intensive

 For a start, they can be digitized and marked automatically by computer.

You can give a 50 question quiz to 30 pupils and the 1500 answers are marked instantaneously.

This is not true of extended, open questions at all: marking 30 paragraphs takes not 30 seconds

but 30-60 minutes; marking 30 essays takes 120+. What’s more, instead of every teacher

in the department, and every new teacher, slogging away marking the same assessment each year, the multiple choice quiz (for a novel like Oliver Twist or play like The Crucible –and many other topics in English, Maths and other subjects) can be created with an upfront workload, shared between

a few subject experts, then used for years to come. Essay questions will still be important.

But they can be combined with multiple-choice questions.

Once created, MCQs save huge amounts of teacher time downstream.

3. They make pupil understanding more visible to teachers

 The granularity of those 50 questions gives precise visibility into who understands what,

and which pupils lack understanding. It’s a very precise diagnosis. If Jay gets 20 out of 50,

it’s crystal clear that 40% isn’t as good as Izzy, who got 45 out of 50, and 90%.

You can target support to Jay based on what he still hasn’t understood,

instead of trying to reteach the lot. And if there were 5 questions that everyone got wrong,

even Izzy, then you can reteach and revisit those in lessons. 

Multiple-choice questions are a formidably powerful diagnostic and formative tool for teaching.

[Update from Kevin Cooper’s comment below] Multiple choice questions also allow the full range

of a course to be tested. An essay tests depth of understanding focused on a narrow selection

of content; multiple choice questions test breadth of understanding across a much wider range

of content. Both are important. But the middle option of open questions without options 

would ratchet up the labour-intensity for teachers, 

and not realise the benefits of multiple choice on reliability either.

All this challenges assumptions that are easy to make about such questions:

that they aren’t rigorous or valid measures of learning. Rigour and validity depend on their design, which I’ll write about in another post: how to create effective multiple-choice questions.

Try using multiple-choice questions as an assessment vehicle. They are a useful part of the mix.

 ***

*But what if pupils guess the right answer?

This can be mitigated in at least four ways, which can be combined:

increase the number of questions (not just 1 or 2 but 10 or 20)

increase the number of options (not just 4, a 25% chance of correct guessing,

but 5, a 20% chance, or 6, a 16% chance of guessing)

have a high pass mark: increase it from 70% to 80% or 90%

have a penalty for incorrect answers: lose a mark for every wrong answer

https://pragmaticreform.wordpress.com/2014/03/08/whymcqs/

You can TCR software and engineering manuals for spontaneously recall – or pass that exam.

I can Turbo Charge Read a novel 6-7 times faster and remember what I’ve read.

I can TCR an instructional/academic book around 20 times faster and remember what I’ve read.

Introduction to Turbo Charged Reading YouTube

A practical overview of Turbo Charged Reading YouTube  

How to choose a book. A Turbo Charged Reading YouTube

Emotions when Turbo Charged Reading YouTube

Advanced Reading Skills Perhaps you’d like to join my FaceBook group ?

www.facebook.com/mreenhunt

www.pinterest.com/mreenhunt

Perhaps you’d like to check out my sister blogs:

www.innermindworking.blogspot.com         gives many ways for you to work with the stresses of life

www.ourinnerminds.blogspot.com              take advantage of business experience and expertise.

www.happyartaccidents.blogspot.com        just for fun.

To quote the Dr Seuss himself, “The more that you read, the more things you will know.

The more that you learn; the more places you'll go.”