An introduction to the difference-in-differences design in education policy research

Abstract

Education researchers often encounter scenarios where an abrupt policy change occurs within or across jurisdictions or populations that affect key student outcomes. Difference-in-differences is a research design analysts can use to estimate causal effects of these “natural experiments.” This article introduces education researchers to the difference-in-differences approach. We provide an overview of the standard two-by-two design, which examines changes in outcomes over time between a treated and an unaffected group. Then, we discuss the key assumptions and threats to validity that researchers must address to have maximum confidence in their results. We also discuss commonly used alternative specifications that provide researchers with a more nuanced approach to estimating policy effects. Throughout the paper, we illustrate the difference-in-differences approach using the example of the implementation of an after-school tutoring program to increase economically disadvantaged students’ academic performance. In supplementary material, we provide Stata code to help assist researchers adapt the approach to their own scholarship.