Causal Inference Challenges with Interrupted Time Series Designs: An Evaluation of an Assault Weapons Ban in California

Abstract

The interrupted time series design was introduced to social scientists in 1963 by Campbell and Stanley, analysis methods were proposed by Box and Tiao in 1975, and more recent treatments are easily found (Box et al., 2016). Despite its popularity, current results in statistics reveal fundamental oversights in the standard statistical methods employed. Adaptive model selection built into recommended practice causes challenging problems for post-model-selection-inference. What one might call model cherry picking can invalidate conventional statistical inference, statistical tests and confidence intervals with damaging consequences for causal inference. There are technical developments that can correct for these problems, but these remedies raise conceptual difficulties for causal inference when proper estimands are defined. The issues are illustrated with an analysis of the impact of an assault weapons ban on daily handgun sales in California from 1996 through 2018. Statistically valid regression functionals are obtained, but their causal meaning is unclear. Researchers might be best served by interpreting only the sign of such functionals.