Internationally Educated Nurses and the Canadian English Language Benchmark Assessment for Nurses: A Qualitative Test Validation Study of Test-Taker Accounts

Abstract

This qualitative validation study examines sixteen Internationally Educated Nurses’ (IENs’) accounts of the Canadian English Language Benchmark Assessment for Nurses (CELBAN) at two testing centres (Toronto and Hamilton). This study adopts both focus groups and one-on-one interviews to investigate the inferences drawn from the test, and its consequences. Focus groups and interviews were conducted using an adapted interview guide utilized in the TOEFL iBT investigation of test-taker accounts of construct representation and construct irrelevant variance (DeLuca et al., 2013). While construct representation describes the degree of authenticity in the presentation of Canadian English language nursing tasks, construct irrelevant variance refers to potential factors impacting the test-taking experience which might contribute to a score variance that was not reflective of test-taker knowledge of the testing constructs (Messick, 1989, 1991, 1996). In this study, test-taker accounts of construct representation and construct irrelevant variance constituted the data which were coded and analyzed abductively via the sensitizing concepts derived from DeLuca et al., and Cheng and DeLuca (2011) on examining test-takers’ experience and their contribution to validity. Seven themes emerged, answering four research questions: How do IENs characterize their test experience? How do IENs describe the assessment constructs? What, if any, sources of Construct Irrelevant Variance (CIV) do IENs describe? Do IENs feel the language tasks are authentic? Overall, participants reported positive experiences with the CELBAN, while identifying some possible sources of CIV. Given the CELBAN’s widespread use for high-stakes decisions (a component of nursing certification and licensure), further research of IEN-test-taker responses to construct representation and construct irrelevant variance will remain critical to our understanding of the role of language competency testing for IENs.