Artificial Intelligence in Computerized Adaptive Testing

Abstract

Artificial intelligence (AI) is increasingly used to provide customized and efficient e-learning, job search, and career development assistance to students and workers. Both students and jobseekers encounter assessments several times throughout their career and during job searches. Organizations now employ computerized adaptive testing (CAT), a computer-administered assessment that serves questions based upon the ability of a test taker. CAT aims to provide personalized assessments to test takers to accurately estimate their proficiency with respect to a latent trait (e.g., general intelligence and personality characteristic) that is not directly observable. There are several challenges in CAT, such as estimating the latent traits of an individual, generating questions, and question selection. Furthermore, these challenges become more complex as the number of latent trait dimensions being measured increases or if item responses are categorical rather than binary (e.g., using a 1 to 5 scale versus true or false). Traditional approaches employ psychometric and statistical models to make estimations. However, many approaches using machine learning, deep learning, and other AI techniques have emerged to provide better performance. In this paper, we provide a technique-oriented review of AI applications in CAT, and highlight the advantages, limitations, and future challenges in this problem area. We also reconcile different terms and notations used across psychometrics and AI to assist future research and development.