Simulation Study of Nonequivalent Group-Matching and Regression-Discontinuity Designs: Evaluations of Gifted and Talented Programs

Abstract

Abstract In a typical evaluation of gifted and talented (G&T) programs, previously identified G&T students are matched with control students not participating in the program and group differences at the end of the program are used to infer program effects. Eight variations of this general-matching design, based on the initial selection of G&T students at Time 1 and the matching G&T and control students at Time 2, were evaluated with simulated data. Even when true program effects were zero, Time 3 G&T scores were substantially higher than control-group scores for all 8 design variations. In some cases, this bias, a special case of a regression-to-the-mean effect, was larger than effects typically found in meta-analyses of research in this area. When scores for selected and unselected students were available for Time 1 selection criteria, the regression-discontinuity design offered a better approach. Statistical and graphical analyses based on this alternative design provided unbiased estimates of program e...