Empowering Affect-Aware Systems by Monitoring Mouse Speed and Acceleration

Abstract

Mouse tracking can be used as a non-obtrusive data-collection method to identify in real time the users' cognitive and emotional states. Despite the advances in the field, most studies focus on measuring decision conflict processes in typical choice-making tasks, while a framework for emotion prediction in different contexts of web interactions is missing. The present study investigates the potential of measuring a person's negative emotional state through solely mouse cursor data of speed and acceleration. A two study experiment was designed to monitor the mouse behavior of 79 participants in three different types of gaming apps: two gamified campaigns (a puzzle and a hidden-items game), and one Game-based Learning (GBL) quiz task. The collected dataset comprised 123 valid records of mouse features and self-reported emotional statements. A set of different classifiers were trained and tested, where we achieved a maximum accuracy of 81% and 83% for frustration and confusion, respectively. We also achieved higher accuracy, namely 85%, in the case of gamified tasks, excluding the GBL task, implying that further research should be conducted in this field. Our findings indicate that by analyzing speed and acceleration data, it is possible to make efficient predictions of a user's emotional state in different web activities.