MCCI: A multi-channel collaborative interaction framework for multimodal knowledge graph completion

Abstract

Multimodal knowledge graph completion (MKGC) aims to leverage multimodal information to predict missing fact triplets. However, existing MKGC approaches largely ignore the heterogeneity and interaction complexity between modal details, resulting in a lack of balance in the intra- and inter-modal expression. To address the above challenges, we propose a novel multi-channel collaborative interaction (MCCI) framework for MKGC, which is composed of feature encoding, dual-flow alignment, and decision fusion modules. Specifically, in the encoding stage, information filtering and visual enhancement-based methods are used to capture high-quality multimodal features. Furthermore, the dual-flow alignment module expands the potential correlations between different modalities, thereby facilitating the interaction frequency of the information. In the fusion stage, dynamically allocate modality weights and generate prediction outcomes. Experimental results show that compared with the state-of-the-art approaches, the proposed MCCI framework has an improvement of 5.7% and 19.8% in Hits@10 and MR, respectively.