Emotional boundaries and intensity aware model for incomplete multimodal sentiment analysis

Multimodal sentiment analysis aims to obtain comprehensive emotional features from multiple data sources when all modalities are accessible. However, in real-world scenarios, it is often impossible for all modalities to be available all the time. This issue leads to a significant degradation in the performance of multimodal sentiment analysis. There are two major challenges in this field: accurately identifying samples near the classification boundaries is difficult, and the recognition performance varies significantly among different modality combinations. In this article, we propose an Emotional Boundaries and Emotional Intensity Aware (EBIA) model to enhance the robustness of incomplete multimodal sentiment analysis. Specifically, we design a Boundary Fuzzy Aware (BFA) module to learn the intra-class and inter-class consistency of samples, transferring full modalities integrity information to the missing modality environment, and prompting the model to focus on samples near the class boundaries. Additionally, we introduce a Weak Modality Aware (WMA) module that calculates additional predictions for each modality, guiding the model to focus on weak modality combinations. Extensive experiments and analyses conducted on three popular benchmark datasets demonstrate the effectiveness of our proposed method compared with several baseline methods.