Beyond Text: Multimodal Credibility Assessment Approaches for Online User-Generated Content

Abstract

User-Generated Content (UGC) is increasingly becoming prevalent on various digital platforms. The content generated on social media, review forums, and question-answer platforms impacts a larger audience and influences their political, social, and other cognitive abilities. Traditional credibility assessment mechanisms involve assessing the credibility of the source and the text. However, with the increase in how user content can be generated and shared (audio, video, images), multimodal representation of User-Generated Content has become increasingly popular. This paper reviews the credibility assessment of UGC in various domains, particularly identifying fake news, suspicious profiles, and fake reviews and testimonials, focusing on both textual content and the source of the content creator. Next, the concept of multimodal credibility assessment is presented, which also includes audio, video, and images in addition to text. After that, the paper presents a systematic review and comprehensive analysis of work done in the credibility assessment of UGC considering multimodal features. Additionally, the paper provides extensive details on the publicly available multimodal datasets for the credibility assessment of UGC. In the end, the research gaps, challenges, and future directions in assessing the credibility of multimodal user-generated content are presented.