Re-evaluating causal inference: Bias reduction in confounder-effect modifier scenarios

Abstract

Propensity Score Matching (PSM) is a widely used method for estimating causal treatment effects, but its performance can be limited in complex scenarios. This paper examines cases where a confounder also serves as an effect modifier and compares the bias-reduction performance of PSM with Inverse Probability Weighting (IPW). Using the University of California, Berkeley graduate admission data as an illustrative example, we show that PSM can produce biased estimates of the Average Treatment Effect (ATE) in such contexts. Through a simulation study, we demonstrate that PSM generally fails to adequately reduce bias for the ATE when a confounder is also an effect modifier, while IPW yields less biased estimates with lower Mean Squared Error (MSE). To validate these findings in a more real-world setting, we analyse data generated from a well-known matched-pairs experimental study of Mexico's Seguro Popular de Salud (Universal Health Insurance) Program. From this experiment we derive observational data that incorporates confounders and effect modifiers and compare the performance of PSM and IPW estimators. Our results confirm that IPW consistently provides more accurate and reliable estimates of the ATE, with smaller bias, compared to PSM.