Homework: What does the Hattie research actually say?

Is this dock, sorrel or something else?

Homework: What does the Hattie research actually say?

http://headguruteacher.com/2012/10/21/homework-what-does-the-hattie-research-actually-say/

This is an excellent book.  It is an attempt to distil the key messages from the vast array of studies

that have been undertaken across the world into all the different factors 

that lead to Homework.  

As you would hope and expect, the book contains details of the statistical methodology underpinning 

a meta-analysis and the whole notion of ‘effect size’ that drives the thinking in the book.  

There is a discussion about what is measurable and how effect size can be interpreted in different ways. 

The key outcomes are interesting, suggesting a number of key factors that are likely to make

 the greatest impact in classrooms and more widely in the lives of learners.

My main interest here is to explore what Hattie says about homework.  

This stems from a difficulty I have when I hear or read, fairly often, that ‘research shows that homework makes no difference’. It is cited as a hard fact in articles such as this one by Tim Lott in the Guardian: 

Why do we torment kids with homework?   Even though Tim is talking about his 6 year old, 

and cites research that refers to ‘younger kids’, too often the sweeping generalisation is applied 

to all homework for all students.  It bugs me and I think it is wrong.

Research cited for younger children

I have written about my views on homework under the heading ‘Homework Matters:

Great Teachers set Great Homework’ . I’ve said that all my instincts as a teacher

(and a parent) tell me that homework is a vital element in the learning process;

reinforcing the interaction between teacher and student; between home and school

and paving the way to students being independent autonomous learners. 

Am I biased? Yes.  Is this based on hunches and personal experience? Of course. 

Is it backed up by research……?  Well that is the question.

So, what does Hattie say about homework?

Helpfully he uses Homework studies as an example of the overall process of meta-analyses,

so there is plenty of material. In a key example, he describes a study of five meta-analyses

that capture 161 separate studies involving over 100,000 students

as having an effect size d= 0.29.  What does this mean? 

This is the best typical effect size across all the studies, suggesting:

improving the rate of learning by 15% – or advancing children’s learning by about a year

65% of effects were positive

35% of effects were negative

average achievement exceeded 62% of the levels of students not given homework.

However, there are other approaches such as  the ‘common language effect’ (CLE)

that compares effects from different  distributions. For homework a d= 0.29 

effect translates into a 21% chance that homework will make a positive difference. 

Or, from two classes, 21 times out of a 100, using homework will be more effective.  

Hattie then says that terms such as ‘small, medium and large’ need to be used with caution

in respect of effect size.  He is ambitious and won’t accept comparison with 0.0

as a sign of a good strategy.   He cites Cohen as suggesting with reason that 0.2 is small,

0.4 is medium and 0.6 is large and later argues himself that we need a hinge-point

where d > 0.4 is needed for an effect to be above average and d > 0.6 to be considered excellent.

OK.  So what is this all saying. Homework, taken as an aggregated whole, shows an effect size 

of d= 0.29 that is between small and medium?  Oh.. but wait… here comes an important detail. 

Turn the page:  The studies show that the effect size at Primary Age is d = 0.15

and for Secondary students it is d = 0.64!  Well, now we are starting to make some sense. 

On this basis, homework for secondary students has an ‘excellent’ effect.  I am left thinking that, 

with a difference so marked, surely it is pure nonsense to aggregate these measures in the first place?

Hattie goes on to report that other factors make a difference to the results: 

eg when what is measured is very precise (eg improving addition or phonics), a bigger effect is seen compared to when the outcome is more ephemeral. So, we need to be clear:  what is measured 

has an impact on the scale of the effect.  This means that we have to throw in all kinds of caveats about the validity of the process.  There will be some forms of homework more likely to show an effect than others;  

it is not really sensible to lump all work that might be done in between lessons into the catch-all ‘homework’ and then to talk about an absolute measure of impact.  Hattie is at pains to point out that there will be great variations across the different studies that simply average out to the effect size on his barometers.  

Again, in truth, each study really needs to be looked at in detail.  

What kind of homework? What measure of attainment?  What type of students?  

And so on…. so many variables that aggregating them together is more or less made meaningless? 

Well, I’d say so.

Nevertheless, d= 0.64!  That matches my predisposed bias so I should be happy.  q.e.d. 

Case closed.  I’m right and all the nay-sayers are wrong. Maybe, but the detail, as always,

is worth looking at.  Hattie suggests that the reason for the difference between the  d=0.15

at primary level at d=0.64 at secondary is that younger students can’t under take unsupported study as well, they can’t filter out irrelevant information or avoid environmental distractions –

and if they struggle, the overall effect can be negative.

At secondary level he suggests there is no evidence that prescribing homework develops time management skills and that the highest effects in secondary are associated with rote learning, practice or rehearsal of subject matter; more task-orientated homework has higher effects that deep learning and problem solving.  Overall, the more complex, open-ended and unstructured tasks are, the lower the effect sizes.  

Short, frequent homework closely monitored by teachers has more impact that their converse forms 

and effects are higher for higher ability students than lower ability students,

higher for older rather than younger students. 

Finally, the evidence is that teacher involvement in homework is key to its success.

So, what Hattie actually says about homework is complex. 

There is no meaningful sense in which it could be stated that

“the research says X about homework” in a simple soundbite. 

There are some lessons to learn:

The more specific and precise the task is, the more likely it is to make an impact for all learners.  

Homework that is more open, more complex is more appropriate for able and older students.

Teacher monitoring and involvement is key – so putting students in a position where their learning

 is too complex, extended or unstructured to be done unsupervised is not healthy. 

This is more likely for young children, hence the very low effect size for primary age students.

All of this makes sense to me and none of it challenges my predisposition to be a massive advocate for homework.  The key is to think about the micro- level issues, not to lose all of that in a ridiculous averaging process.  Even at primary level, students are not all the same.  Older, more able students in Year 5/6 may well benefit from homework where kids in Year 2 may not.  Let’s not lose the trees for the wood!  

Also, what Hattie shows is that educational inputs, processes and outcomes are all highly subjective human interactions.  Expecting these things to be reduced sensibly into scientifically absolute measured truths is absurd.  Ultimately, education is about values and attitudes and we need to see all research in that context.

PS. If you are reading this from Sweden, Tack för läsning. Låt mig veta era tankar om denna fråga.

Introduction to Turbo Charged Reading YouTube

A practical overview of Turbo Charged Reading YouTube 

How to choose a book. A Turbo Charged Reading YouTube

Emotions when Turbo Charged Reading YouTube

Advanced Reading Skills Perhaps you’d like to join my FaceBook group ?

www.facebook.com/mreenhunt

www.pinterest.com/mreenhunt

Perhaps you’d like to check out my sister blogs:

www.innermindworking.blogspot.com         gives many ways for you to work with the stresses of life

www.ourinnerminds.blogspot.com              which takes advantage of the experience and expertise of others.

www.happyartaccidents.blogspot.com       just for fun.

To quote the Dr Seuss himself, “The more that you read, the more things you will know.

The more that you learn; the more places you'll go.”