Identifying fact-checkable microblogs during disasters: a classification-ranking approach

Microblogging sites are increasingly playing an important role in real-time disaster management. However, rumors and fake news often spread on such platforms, which if not detected, can derail the rescue operations. Therefore, it becomes imperative to verify some of the information posted on social media during disaster situations. To this end, it is necessary to correctly identify fact-checkable posts, so that their information content can be verified. In the present work, we address the problem of identifying fact-checkable posts on the Twitter microblogging site. We organized a shared task in the FIRE 2018 conference to study the problem of identification of fact-checkable tweets posted during a particular disaster event (the 2015 Nepal earthquake). This paper describes the dataset used in the shared task, and compares the performance of different methodologies for identifying fact-checkable tweets. We primarily experiment with two different types of approaches - classification-based and ranking-based. Our experiments show that a hybrid methodology involving both classification and ranking performs well and outperforms the methodologies that employ only classification or only ranking.