Screening screeners: calculating classification indices using correlations and cut-points

Abstract

Given the recent push for universal screening, it is important to take into account how well a screener identifies children at risk for reading problems as well as how screener and sample information contribute to this classification. Picking the best cut-point for a particular sample and screening goal can be challenging given that test manuals often report classification information for a specific cut-point and sample base rate which may not generalize to other samples. By assuming a bivariate normal distribution, it is possible to calculate all of the classification information for a screener based on the correlation between the screener and outcome, the cut-point on the outcome (i.e., the base rate in the sample), and the cut-point on the screener. We provide an example with empirical data to validate these estimation procedures. This information is the basis for a free online tool that provides classification information for a given correlation between screener and outcome and cut-points on each. Results show that the correlation between screener and outcome needs to be greater than .9 (higher than observed in practice) to obtain good classification. These findings are important for researchers, administrators, and practitioners because current screeners do not meet these requirements. Since a correlation is dependent on the reliability of the measures involved, we need screeners with better reliability and/or multiple measures to increase reliability. Additionally, we demonstrate the impact of base rate on positive predictive power and discuss how gated screening can be useful in samples with low base rates.