Dynamic Emotion-Dependent Network With Relational Subgraph Interaction for Multimodal Emotion Recognition

Abstract

Multimodal Emotion Recognition in Conversations (MERC) is an important topic in human-computer interaction. In the MERC task, conversations exhibit dynamic emotional dependency, including inter-speaker and intra-speaker emotional dependency, both are vital in understanding the content. However, current research primarily integrates these two emotional dependencies into one unified module, limiting the accuracy of MERC. In this paper, we propose a dynamic emotion-dependent network with relational subgraph interaction named DEDNet. DEDNet introduces relational subgraphs to separately model two emotional dependencies, enabling structured learning paths for utterances based on distinct emotional dependency types. Specifically, nodes indicate the utterances at different moments in the conversation, while edges define the emotional dependency and temporal relationships between nodes. To explicitly capture the differences between these two emotional dependencies, distinct subgraphs are designed for comprehensive representations. Furthermore, we propose an incremental interactive strategy, sequentially leveraging two emotional dependencies to learn the changes in dependency relationships. We find that modeling inter-speaker emotional dependency can better identify negative emotions and modeling intra-speaker emotional dependency can better recognize positive emotions. Experimental results demonstrate that our model outperforms current state-of-the-art methods on three benchmark datasets, IEMOCAP, MELD and DailyDialog.