How generalizable are findings from police surveys? A review of multi-agency studies

ABSTRACT Policing scholars frequently use surveys to understand officer attitudes and behavioral intentions. Yet, it is difficult to gain access to one – let alone multiple – agencies. Thus, officer surveys often reflect views in a single department, making it unclear how generalizable the findings are. For the present study, we conducted an exploratory review of articles published in 16 criminology and policing journals from 2000 to 2017. We identified 600 studies that involved surveying one or more samples of police officers. From this list, we set out to determine: (1) how often authors administered their surveys to more than one sample, and (2) when surveys were administered to more than one sample, how often were results consistent across samples? We found eighty-seven (14.5%) articles that involved collecting survey data from multiple agencies, though only 29 (4.8% overall, 33.3% of multi-agencies studies) met our inclusion criteria. Importantly, only 15 studies could be analyzed as some authors no longer had data, could not share data, or did not response to our emails. Results were fully consistent across samples in just one published study. In the other studies, findings partially replicated – though sometimes results were in the opposite direction across departments. Thus, replication is critical before policy is created from single-agency surveys.