Emrnet: enhanced micro-expression recognition network with attention and distance correlation

Abstract

Micro-expression recognition (MER) is inherently challenging due to the difficulty of extracting subtle, localized changes in micro-expressions (MEs). Various optical flow-based methods have been proposed for MER, as optical flow can effectively suppress facial identity information while capturing the movement patterns of MEs. However, these methods, characterized by simple architectures, often fail to extract discriminative features, resulting in suboptimal performance. In this paper, we propose an Enhanced Micro-expression Recognition Network with attention and distance correlation (EMRNet) for MER. EMRNet consists of three key phases: First, we introduce a novel channel-wise region-aware attention mechanism within two identical Inception networks, designed to extract global and local expression features in parallel, based on the optical flow input of the same ME. Second, to enhance ME representations, we propose a regularized dilated loss function incorporating distance correlation, which improves the information entropy transferred between the two branches. Last, emotion categories are predicted by fusing the expression-dilated features in the classification branch. Extensive experiments conducted on the composite database from the MEGC 2019 challenge demonstrate the effectiveness of EMRNet under both leave-one-subject-out (LOSO) cross-validation and the composite database evaluation (CDE) protocol. The results show that our approach successfully generates discriminative features, achieving substantial performance gains. Furthermore, EMRNet outperforms existing single-stream and dual-stream models, delivering superior results in MER.