Debunking Hattie: Evaluating the Contribution of Academic Studies to Policy Development and Implementation in Australia

Abstract

John Hattie’s book ‘Visible Learning’ published in 2009 can only be described as an amazing feat of scientific inquiry. He and his team created a mega-analysis, a meta-analysis of 800 meta-analyses, to see what the visible effect sizes were of different elements on education. To generalise, pretty much everything had an effect, mainly positive, with the mean effect size being 0.4. This has then been pretty much adopted into practice as anything being above 0.4 as having an above average impact, and hence being ‘good’, and anything below 0.4 having a below average impact and hence being ‘not so good’. While such a sweeping generalisation ‘makes sense’ in terms of rational logic, I argue here that it has been incredibly damaging in limiting the thinking on educational progress and change in the Australian policy context. I stage this argument in 5 sections: methodology, context, assumptions, leadership and currency – and offer this as a framework for evaluating academia’s contribution to policy agendas. I conclude by reflecting on Hattie’s later papers which move his conclusions forwards academically but have yet to do so in practice.