International Journal of Selection and Assessment最新文献

Bobko et al. (2024) responded to Sackett et al.'s (2022) compilation of meta-analytic evidence for the validity of a wide variety of measures used as predictors of overall job performance, offering a set of alternative methodological choices which they term “considered estimation” to counter the Sackett et al. approach of “conservative estimation.” Here we offer a rebuttal to Bobko et al. A primary concern is that Bobko et al. apply the label “conservative estimation” to the full range of methodological choices made by Sackett et al. Yet, we clarify the narrow and specific meaning of “conservative estimation,” and note that that the bulk of Bobko et al.'s concerns are independent of the principle of conservative estimation. We also respond to Bobko et al.'s two key concerns, namely, comparing validity estimates when one is corrected for range restriction and one is not and comparing validity estimates for predictors reflecting psychological constructs and those reflecting measurement methods, and also briefly address a range of other critiques offered by Bobko et al.

Large language models (LLMs), such as ChatGPT, have reshaped opportunities and challenges across various fields, including human resources (HR). Concerns have arisen about the potential for personality assessment manipulation using LLMs, posing a risk to the validity of these tools. This threat is a reality: recent research suggests that many candidates are using AI to complete pre-hire assessments. This study addresses this problem by examining whether ChatGPT can outperform humans in faking personality assessments while avoiding detection. To explore this, two experiments were conducted focusing on assessing job-relevant traits, with and without coaching, and with two methods of identifying faking, specifically using an impression management (IM) measure and an overclaiming questionnaire (OCQ). For each study, we used responses from 100 working adults recruited via the Prolific platform, which were compared to 100 replications from ChatGPT. The results revealed that while ChatGPT showed some ability to manipulate assessments, without coaching it did not consistently outperform humans. Coaching had a minimal impact on reducing IM scores for either humans or ChatGPT, but reduced OCQ bias scores for ChatGPT. These findings highlight the limitations of current faking detection measures and emphasize the need for further research to refine methods for ensuring the integrity of personality assessments in HR, particularly as artificial intelligence becomes more available to candidates.

Artificial intelligence (AI) is increasingly used to score employment interviews in the early stages of the hiring process, but AI algorithms may be particularly prone to interviewee faking. Our study compared the extent to which people can improve their scores on self-report scales, structured and less structured human interview ratings, and AI interview ratings. Further, we replicate and extend prior research by examining how interviewee abilities and impression management tactics influence score inflation across scoring methods. Participants (N = 152) completed simulated, asynchronous interviews in honest and applicant-like conditions in a within-subjects design. The AI algorithms in the study were trained to replicate question-level structured interview ratings. Participants' scores increased most on self-reports (overall Cohen's d = 0.62) and least on AI interview ratings (overall Cohen's d = 0.14), although AI score increases were similar to those observed for human interview ratings (overall Cohen's d = 0.22). On average, across conditions, AI interview ratings converged more strongly with structured human ratings based on behaviorally anchored rating scales than with less structured human ratings. Verbal ability only predicted score improvement on self-reports, while increased use of honest defensive impression management tactics predicted improvement in AI and less structured human interview scores. Ability to identify criteria did not predict score improvement. Overall, these AI interview scores behaved similarly to structured human ratings. We discuss future possibilities for investigating faking in AI interviews, given that interviewees may try to “game” the system when aware that they are being evaluated by AI.

In laboratory faking research, participants are often instructed to respond honestly (generic instructions [GIs], control condition) or to fake (personnel-selection scenario [PSS], faking condition). Considering the research on instruction-level contextualization, a PSS might not only motivate participants to fake but might also promote the adoption of a work frame of reference (FOR). Thus, differences in responses between faking and control conditions could partly result from FOR effects. (Full) item-level contextualization can also be used to promote the adoption of a work FOR, and the adoption through this route is stronger than through instruction manipulation. We combined the two approaches to disentangle FOR and faking, conducted a 4-wave longitudinal study with a 2 (instructions: GIs vs. PSS) × 2 (full item-level work contextualization absent vs. present) repeated-measures design (N = 309), and compared the effects of these conditions on three HEXACO-PI-R scales (Conscientiousness, Emotionality, Honesty-Humility). Irrespective of the investigated personality trait, the ANOVAs revealed significant main effects. As expected, compared with GIs, the PSS increased the adoption of a work FOR, and the effects were smaller than the effects of full item-level work contextualization present (vs. absent). Also, as expected, the PSS (vs. GIs) and full item-level work contextualization present (vs. absent) changed participants' scale mean scores. However, importantly, there were no interaction effects. Exploratory mediation analyses indicated direct rather than indirect (mediator: adoption of a work FOR) effects of instructions on participants' scale mean scores. In conclusion, the internal validity of faking research is not threatened by confounding FOR effects.

This paper presents research that both replicates and extends previous findings relating to applicant fairness perceptions of various selection methods. Using a working population (N = 281), applicant perceptions of nine ‘traditional’ selection methods were explored, alongside eight ‘newer’ selection methods, including game-based assessment, online interviews, and situational judgement tests. Findings showed that work sample tests, knowledge tests and interviews in person were rated most positively, whilst asynchronous video interviews, personal contacts and professional social media were rated least positively. Some differences were found based on whether participants had previous experience completing the selection method, the mode of delivery for the selection method, and the country in which the participant worked. In line with previous research, selection methods appeared more acceptable and fairer to applicants when they are job-related, offer candidates the opportunity to demonstrate their skills and abilities and are based on sound scientific research. The results are discussed in terms of theoretical and practical implications and future research.

Deploying Artificial Intelligence (AI) for job interview evaluations, while a potential signal of high innovativeness, may risk suggesting poor people orientation on the part of the organisation. This study utilizes an experimental methodology to investigate whether AI evaluation (AIE) is interpreted as a positive (high innovativeness) or negative (low people orientation) signal by the job applicant, and whether the ensuing effects on attitudes towards the organisation depend on the type of organization implementing the technology. Results indicate that AIE is interpreted more strongly as a signal of how the organisation treats people rather than of how innovative it is. Additionally, removing humans from the selection process appears to be a ‘bridge too far’, when it comes to technological advances in the selection process.