International Journal of Selection and Assessment最新文献

The release of new generative artificial intelligence (AI) tools, including new large language models (LLMs), continues at a rapid pace. Upon the release of OpenAI's new o1 models, I reconducted Hickman et al.'s (2024) analyses examining how well LLMs perform on a quantitative ability (number series) test. GPT-4 scored below the 20th percentile (compared to thousands of human test takers), but o1 scored at the 95th percentile. In response to these updated findings and Lievens and Dunlop's (2025) article about the effects of LLMs on the validity of pre-employment assessments, I make an urgent call to action for selection and assessment researchers and practitioners. A recent survey suggests that a large proportion of applicants are already using generative AI tools to complete high-stakes assessments, and it seems that no current assessments will be safe for long. Thus, I offer possibilities for the future of testing, detail their benefits and drawbacks, and provide recommendations. These possibilities are: increased use of proctoring, adding strict time limits, using LLM detection software, using think-aloud (or similar) protocols, collecting and analyzing trace data, emphasizing samples over signs, and redesigning assessments to allow LLM use during completion. Several of these possibilities inspire future research to modernize assessment. Future research should seek to improve our understanding of how to design valid assessments that allow LLM use, how to effectively use trace test-taker data, and whether think-aloud protocols can help differentiate experts and novices.

Cybervetting, assessing social media in personnel selection, is widely used. However, individuals concerned often perceive this practice negatively. We propose that attitudes toward cybervetting may depend on the platform used and the cultural context. Thus, we transfer the attitudes toward cybervetting scale to a context with strict data regulations: Germany. In an online between-subjects experiment with platform users and non-users (N = 100 working professionals and students), we examined attitudes toward cybervetting on different social media platforms (professional: LinkedIn vs. personal: Instagram) and their relationship with organizational attractiveness. We found that German participants viewed cybervetting on professional platforms with more skepticism than American participants. Hierarchical regression analysis revealed higher perceived fairness, lower invasion of privacy, and higher organizational attractiveness when cybervetting was done on professional platforms.

Game-based assessment (GBA) has garnered attention in the personnel selection and assessment context owing to its postulated potential to improve applicant reactions. However, GBAs can differ considerably depending on their specific design. Therefore, we sought to determine whether test taker reactions to GBAs vary owing to the different manifestations that GBAs may take on, and to test takers' individual preferences for such assessments. In an experimental study, each of N = 147 participants was shown six different GBAs and asked to rate several applicant reaction variables concerning these assessments. We found that reactions to GBAs were not inherently positive even though GBAs were generally perceived as enjoyable. However, perceptions of fairness and organizational attractiveness varied considerably between GBAs. Participants' age and experience with video games were related to reactions but had less impact than the different GBAs. Our results suggest that a technology-as-designed approach, which considers GBAs as a combination of multiple components (e.g., game elements), is crucial in GBA research to provide generalizable results for theory and practice.

To address concerns of subtle discrimination against stigmatized groups, we meta-analyze the magnitude and moderators of bias against non-standard accents in employment interview evaluations. Results from a multi-level random-effects meta-analysis (unique effects: k = 41, N = 7,596; multi-level effects accounting for dependencies: k = 120, N = 20,873) demonstrate that standard-accented (SA) interviewees are consistently favored over non-standard-accented (NSA) interviewees (d = 0.46). Accent bias is stronger against women compared to men, particularly when evaluator samples are predominantly female, and was strongly predicted by interviewers' stereotypes of NSA interviewees as less competent and, to a lesser extent, as less warm. Accent bias was not significantly impacted by perceptions of comprehensibility, accentedness, accent type, interview modality, study rigor, or job speaking skill requirements.

Personality assessments are commonly used in hiring, but concerns about faking have raised doubts about their effectiveness. Qualitative reviews show mixed and inconsistent impacts of faking on criterion-related validity. To address this, a series of meta-analyses were conducted using matched samples of honest and motivated respondents (i.e., instructed to fake, applicants). In 80 paired samples, the average difference in validity coefficients between honest and motivated samples across five-factor model traits ranged from 0.05 to 0.08 (largest for conscientiousness and emotional stability), with the validity ratio ranging from 64% to 72%. Validity was attenuated when candidates faked regardless of sample type, trait relevance, or the importance of impression management, though variation existed across criterion types. Both real applicant samples (k = 25) and instructed response conditions (k = 55) showed a reduction in validity across honest and motivated conditions, including when managerial ratings of job performance were the criterion. Thus, faking impacted the validity in operational samples. This suggests that practitioners should be cautious relying upon concurrent validation evidence (for personality inventories) and expect attenuated validity in operational applicant settings, particularly for conscientiousness and emotional stability scales. That said, it is important to highlight that personality assessments generally maintained useful validity even under-motivated conditions.