Revisiting Sackett et al.'s (2022) rationale behind their recommendation against correcting for range restriction in concurrent validation studies.

Abstract

Sackett et al. (2022) recommend against correcting for range restriction (RR) in concurrent validation studies. The main rationale behind their recommendation is that unless "rzx" (an unrestricted true-score correlation between the third variable Z where actual selection occurred in a top-down manner [a.k.a., suitability] and the predictor of interest, X) is as high as .90 and selection ratios are as low as .30-both unlikely events in their view, the degree of RR (ux) in concurrent validation studies is unlikely to be low enough (i.e., lower than .90) to warrant RR correction. That is, (a) the "rzx" ≥ .90 and (b) the selection ratio ≤ .30 are two critical conditions for the third condition, (c) ux ≤ .90, a need for RR correction. In this study, we revisit each of these conditions that constitute the rationale behind their recommendation: (a) whether "rzx" is unlikely to be as high as .90; (b) whether selection ratios of .30 or lower are "extreme"; and (c) whether the degree of RR is "little to no" (i.e., ux ≥ .90) in concurrent validation studies, thus no need for correcting for RR in concurrent validation studies. First, our reanalysis of their Table 1 indicates that it is not implausible that "rzx" is as high as .90. Second, several studies report that selection ratios of .30 or lower are not extreme. Finally, our reanalysis of their Table 5 indicates that Sackett et al. substantially underestimate the severity of RR and its biasing effect on operational validity in concurrent validation studies due to their use of a particular RR correction method (Case IV). We believe these findings suggest that there is not sufficient support for the rationale behind Sackett et al.'s recommendation and, thus, their recommendation itself should be reconsidered. (PsycInfo Database Record (c) 2023 APA, all rights reserved).