DEMA: Deep EEG-first multi-physiological affect model for emotion recognition

Abstract

In the field of electroencephalogram (EEG) emotion recognition, existing studies often focus solely on EEG-specific features, neglecting valuable emotional cues present in other physiological signals. To address this gap, we introduce the Deep EEG-first Multi-Physiological Affect (DEMA) model. DEMA leverages a Deep Multi-View Convolutional Neural Network (DMCNN) to extract comprehensive features from multi-domain EEG signals. Additionally, it integrates other physiological signals through an EEG-first Multi-Physiological Fusion (EFMF) approach. Integrating multi-physiological signals while eliminating disparate emotional characteristics, an affective influence matrix (AIM) is introduced to assess the influence of EEG signals on other physiological signals, thereby unifying different levels of representation. Our method achieves significant advancements, with DEMA scoring 97.55% in valence and 97.61% in arousal on the DEAP dataset. Ablation studies confirm that combining EEG signals with blood pressure and respiration belts optimally enhances emotion recognition performance.