Estimation of subtle facial emotion changes from expressionless images

Abstract

Subtle changes in emotional expressions occur more frequently compared to rich ones, which makes the evaluation of the emotional response of an individual challenging. In this study, we focus on the near-expressionless facial images, indicated with low arousal and valence value. We investigated the facial landmarks which are crucial in estimating subtle emotion through a novel feature selection method named Random Combination Selection with Iterative Step (RACSIS)1 . By combining appearance and geometrical features, while reducing the feature points up to 93.8%, the Mean Absolute Error (MAE) for Arousal = [-4 8], Valence = [-7 6], was reduced to 54.95% and 46.39% for the full emotional spectrum and the subtle emotion, respectively. We then tested the performance of the RACSIS to estimate the emotional response of participants undertaking audio-visual activities. We conclude that: 1. Appearance features played a greater role in reducing the MAE. 2. Feature selection (FS) by RACSIS achieved lower MAE values compared to correlation.