Yes, DI did it: the impact of Direct Instruction on literacy outcomes for Very Remote Indigenous schools

Abstract In the journal article Did DI do it? The impact of a programme designed to improve literacy for Aboriginal and Torres Strait Islander students in remote schools, Guenther and Osborne (2020) compare schoolwide NAPLAN reading scale scores for 25 Very Remote Indigenous schools implementing Direct Instruction through the Flexible Literacy for Remote Primary Schools Program (‘Flexible Literacy’ or ‘the program’) with those for 118 Very Remote Indigenous schools not involved with the program, to assert the program has not improved literacy outcomes. Good to Great Schools Australia (GGSA) undertook an analysis of the same school data for Reading, Writing, Spelling and Grammar and Punctuation scores. Our findings contradict theirs. In all areas, schools participating in the program show significant growth compared with all Australian and all Very Remote Indigenous schools. In Reading, schools involved in the program from 2015 to 2017 averaged 124% growth, while the average growth for comparable ages was 19 and 34% for Australian and Very Remote Indigenous schools, respectively. In Grammar and Punctuation schools involved in the program in the same period grew 180%, whilst growth for Australian schools was 15%, and for Very Remote Indigenous schools, 28%. These contrasting results illustrate the importance of evaluating growth to assess the impact of educational programs, rather than achievement alone, particularly in the case of Very Remote Indigenous schools where achievement levels are far below Australian grade levels. Guenther and Osborne's comparison of achievement across schools rather than measuring growth within schools obscures real gains and is misleading.