An Efficient Illumination-Invariant Dynamic Facial Expression Recognition for Driving Scenarios

Abstract

Facial expression recognition (FER) is significant in many application scenarios, such as driving scenarios with very different lighting conditions between day and night. Existing methods primarily focus on eliminating the negative effects of pose and identity information on FER, but overlook the challenges posed by lighting variations. So, this work proposes an efficient illumination-invariant dynamic FER method. To augment the robustness of FER methods to illumination variance, contrast normalisation is introduced to form a low-level illumination-invariant expression features learningmodule. In addition, to extract dynamic and salient expression features, a two-stage temporal attention mechanism is introduced to form a high-level dynamic salient expression features learning module deciphering dynamic facial expression patterns. Furthermore, additive angular margin loss is incorporated into the training of the proposed model to increase the distances between samples of different categories while reducing the distances between samples belonging to the same category. We conducted comprehensive experiments using the Oulu-CASIA and DFEW datasets. On the Oulu-CASIA VIS and NIR subsets in the normal illumination, the proposed method achieved accuracies of 92.08% and 91.46%, which are 1.01 and 7.06 percentage points higher than the SOTA results from the DCBLSTM and CELDL method, respectively. Based on the Oulu-CASIA NIR subset in the dark illumination, the proposed method achieved an accuracies of 91.25%, which are 4.54 percentage points higher than the SOTA result from the CDLLNet method. On the DFEW dataset, the proposed method achieved a UAR of 60.67% and a WAR of 71.48% with 12M parameters, approaching the SOTA result from the VideoMAE model with 86M parameters. The outcomes of our experiments validate the effectiveness of the proposed dynamic FER method, affirming its ability in addressing the challenges posed by diverse illumination conditions in driving scenarios.