Validity Stability Across Entering College Cohorts: Exploring the Temporal Generalizability of Local Validity Estimates

Abstract

Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.