Graph Reconstruction Attention Fusion Network for Multimodal Sentiment Analysis

Abstract

Multimodal sentiment analysis (MSA) has become increasingly popular due to the exponential surge of user comments on social media. The MSA aims to efficiently integrate various modalities through a superior fusion framework. However, previous studies have primarily focused on the integration of sequence data while neglecting its structural information. In addition, effectively modeling the continuous expression of human sentiment polarity remains a significant challenge. Therefore, we propose the graph reconstruction attention fusion network, which availably promotes the multimodal fusion process by combining sequence learning with graph learning. First, we design a graph reconstruction learning module to obtain multimodal graph embeddings. Second, a text-guided cross-modal enhancement architecture is adopted to acquire multimodal representations, where a sentiment attenuation factor is introduced to promote emotional continuity modeling. Finally, we propose a feature-wised attention structure adapted for the classifier, it dynamically adjusts weights of multimodal features that are beneficial for downstream tasks. Extensive experiments on three challenging datasets, CMU-MOSI, CMU-MOSEI, and CH-SIMS demonstrate that our model significantly outperforms existing state-of-the-art methods.