Bridging computer and education sciences: A systematic review of automated emotion recognition in online learning environments

Abstract

Emotions play an important role in the learning process. With intelligent technology support, identification and intervention of learners’ cognition have made great achievement, but the care of emotion has been in the absence for a long time. In recent years, the use of affective computing technology to solve affective loss in online education has become a key research topic. To date, a growing number of studies have investigated automated emotion recognition (AER) in online environments. However, AER has been mainly studied from the perspective of computer science focusing on technical characteristics of developing AI technology while its pedagogical value and educational application has been overlooked. Therefore, this systematic literature review aimed to bring together educational and technical aspects of AER. Following PRISMA methodology, a comprehensive search of AER research from 2010 to 2024 in three databases (Web of Science, Science Direct and IEEE Xplore) identified 117 studies that met inclusion criteria. The articles were coded for report characteristics, educational characteristics (tech platform, pedagogy, assessment, content), technical characteristics (emotion model, emotion category, emotion measurement channel, database, algorithm model) and outcome characteristics (technical result, educational application). We found that the primary purpose of these studies was to develop and evaluate systems for AER, rather than implementing these systems in real online learning environments. Furthermore, our findings indicated a lack of integration between computer science and educational science in the realm of AER. Despite the fact that most algorithm models demonstrated high accuracy in AER, the interpretability of the results was significantly constrained by the quality of the databases used, along with the scarcity of studies focusing on the effective and real-time application of AER results. These findings provide essential guidance for shaping future research and development pathways in this field.