Comparing methods for estimating causal treatment effects of administrative health data: A plasmode simulation study

Abstract

Estimating the causal effects of health policy interventions is crucial for policymaking but is challenging when using real-world administrative health care data due to a lack of methodological guidance. To help fill this gap, we conducted a plasmode simulation using such data from a recent policy initiative launched in a deprived urban area in Germany. Our aim was to evaluate and compare the following methods for estimating causal effects: propensity score matching, inverse probability of treatment weighting, and entropy balancing, all combined with difference-in-differences analysis, augmented inverse probability weighting, and targeted maximum likelihood estimation. Additionally, we estimated nuisance parameters using regression models and an ensemble learner called superlearner. We focused on treatment effects related to the number of physician visits, total health care cost, and hospitalization. While each approach has its strengths and weaknesses, our results demonstrate that the superlearner generally worked well for handling nuisance terms in large covariate sets when combined with doubly robust estimation methods to estimate the causal contrast of interest. In contrast, regression-based nuisance parameter estimation worked best in small covariate sets when combined with singly robust methods.