Study on multidimensional emotion recognition fusing dynamic brain network features in EEG signals

Abstract

Accurate emotion recognition is crucial in scientific research and has widespread applications in medical and educational fields. Existing studies have explored to some extent the effects of space, time, and frequency domain features of EEG signals on emotion recognition but have neglected the extensive spatial features contained in the complex information interactions reflected in the synergistic work between different brain regions, as well as the effects of the interactions between the time–frequency-space features on the fusion of the dynamic features of the captured emotional state. To tackle these challenges, this paper presents a multi-dimensional emotion recognition method that incorporates dynamic brain functional network features of EEG signals. The method analyzes spatial connectivity patterns associated with emotional representations by constructing a dynamic brain functional network, aiming to capture time–space features in the EEG signals; Simultaneously, time–frequency feature extraction is achieved by using time–frequency map transformation to fine-tune the pre-trained ResNet18 model. DE and PSD of each frequency band are extracted as complementary frequency domain features through frequency band segmentation. This paper also proposes a bidirectional long and short-term memory network that incorporates an improved attention mechanism to fuse time–frequency-spatial features and consider interactions between multidimensional features. The recognition accuracies on the arousal dimension and valence dimension on the DEAP dataset reached 97.01% and 94.92%, respectively. The recognition accuracy on the SEED dataset reached 92.97%. This fully proves that the emotion recognition method described in this paper effectively extracts multidimensional features, leading to a significant improvement in emotion recognition accuracy.