Higher expectations: a systematic review of reporting the science of propensity score modeling in criminal justice studies

Objectives

This systematic review examined the reporting practices of propensity score modeling (PSM) in criminology and criminal justice studies.

Methods

There were 229 articles (with 527 individual sets of PSM analyses) published between 2000 and 2022 in the top 19 ranked criminology and criminal justice journals that were identified for study inclusion. These articles were systematically coded to assess the presence of key PSM reporting elements, including covariates, estimation methods, balancing strategy, matching type, sample size, balance assessment, sensitivity analysis, and post-PSM analyses.

Results

The use of PSM has increased dramatically over the last two decades. Study findings suggest that while there are many areas where the discipline has improved reporting over the last 22 years, there are still others that are sorely lacking. Our review reveals that only 28.9% of studies reported the propensity score technique used, 21.8% did not report pre- or post-PSM sample information, and 26.6% of all studies examined either did not report a balance assessment or reported using only one assessment which were largely null hypothesis significance testing.

Conclusions

Our findings provide a baseline assessment of PSM reporting and provide some guidance for conducting, reviewing, and publishing PSM research in criminology and criminal justice journals.