PulseEmoNet: Pulse emotion network for speech emotion recognition

Abstract

In recent years, Speech Emotion Recognition (SER) has garnered significant attention due to its potential applications in human–computer interaction, healthcare, and affective computing. However, existing approaches often face challenges in handling the complex, multimodal nature of speech data and the variability in emotional expressions across different contexts. In this paper, we propose PulseEmoNet, a novel deep learning-based framework designed to enhance the robustness of SER systems by integrating pulse signal information with acoustic features. The key innovation of our approach lies in the development of a PulseEmoNet that effectively captures the temporal and physiological correlates of emotional states from speech signals. Experimental results on multiple benchmark datasets demonstrate the superiority of PulseEmoNet over existing models. On EMODB, SAVEE, and CASIA, PulseEmoNet achieved accuracies of 91.11 %, 78.75 %, and 93.08 %, respectively, outperforming previous methods like 3DRNN + Attention and GM-TCN. Additionally, it achieved 88.70 % on BodEMODB, 61.40 % on IEMOCAP, and 95.98 % on ESD. These results highlight the effectiveness of PulseEmoNet in diverse emotional recognition tasks, providing a promising solution for real-time, cross-domain SER applications.