Robot-assisted language assessment: development and evaluation of feasibility and usability

Abstract

Many studies have shown that robots can provide medical help to patients, such as supporting physical movements, managing mood, or simulating cognitive function. However, robotic cognitive/language assessment, which is vital for mental health care, has not been fully explored and is limited to only a few types of assessment. The aim of this study is to present and evaluate a social robot equipped with a web-based language assessment for sentence comprehension test (SCT) with a dialogue system involving yes/no questions. A total of 50 participants took the test with 36 items conducted by a robot (robot-SCT), while a total of 55 participants took the same test but conducted by a human examiner (human-SCT). Comparative analyses were performed to evaluate the validity of the robot-SCT in terms of test scores and time-related measures. Usability was evaluated through the system usability score and interview feedback. With regard to the validity of the robot-SCT, the test scores indicated no significant differences between the robot-SCT and human-SCT. In addition, conditional differences in reaction time for the test items were observed, similar to the previous paper-and-pencil researches. The high system usability scores (i.e., mean = 78.5, SD = 11) demonstrated the high usability of the robot-SCT. This study demonstrates the validity and usability of robotic language assessment among normal adults. However, further evaluation is required for people with dementia or mild cognitive impairment.