Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities

Abstract

As social media platforms evolve from text-based forums into multi-modal environments, the nature of misinformation in social media is also transforming accordingly. Taking advantage of the fact that visual modalities such as images and videos are more favorable and attractive to users, and textual content is sometimes skimmed carelessly, misinformation spreaders have recently targeted contextual connections between the modalities, e.g., text and image. Hence, many researchers have developed automatic techniques for detecting possible cross-modal discordance in web-based content. We analyze, categorize, and identify existing approaches in addition to the challenges and shortcomings they face in order to unearth new research opportunities in the field of multi-modal misinformation detection.