The file drawer problem in social science survey experiments

Abstract

The file drawer problem—often operationalized in terms of statistically significant results being published and statistically insignificant not being published—is widely documented in the social sciences. We extend Franco’s et al. [ Science 345 , 1502–1505(2014)] seminal study of the file drawer problem in survey experiments submitted to the Time-sharing Experiments for the Social Sciences (TESS) data collection program. We examine projects begun after Franco et al. The updated period coincides with the contemporary open science movement. We find evidence of the problem, stemming from scholars opting to not write up insignificant results. However, that tendency is substantially smaller than it was in the prior decade. This suggests increased recognition of the importance of null results, even if the problem remains in the domain of survey experiments.