Knowledge-aware multimodal pre-training for fake news detection

Abstract

Amidst the rapid propagation of multimodal fake news across various social media platforms, the identification and filtering of disinformation have emerged as critical areas of academic research. A salient characteristic of fake news lies in its diversity, encompassing text–image inconsistency, content–knowledge inconsistency, and content fabrication. However, existing endeavors are generally tailored to a specific subset of fake news, leading to limited universality. Moreover, these models primarily rely on scarce and exorbitant manually labeled annotations, which is incapable of providing sufficient learning signals to detect a variety of fake news. To address these challenges, we propose a novel knowledge-aware multimodal pre-training paradigm for fake news detection, dubbed KAMP. Our motivation lies in incorporating unsupervised correlations through pre-training tasks as complementary to alleviate the dependency on annotations. KAMP consists of a novel multimodal learning model and various delicate pre-training tasks to simultaneously capture valuable knowledge from single modality, multiple modalities, and background knowledge graphs. Our proposal undergoes comprehensive evaluation across two widely utilized datasets, and experimental results demonstrate the superiority of our proposal.