Tweet question classification for enhancing Tweet Question Answering System

Abstract

In the evolving landscape of social media, effective Question Answering (QA) systems are crucial for enhancing user engagement and satisfaction. Question classification (QC) is vital for improving the efficiency and accuracy of QA systems. Given the informal and noisy nature of social media texts, which differ significantly from general domain QC datasets, there is a strong need for a specialized tweet QC system for social media QA. In this study, we annotated questions in the Tweet QA dataset, performed tweet question classification, and developed the TweetQC dataset, comprising tweet questions with associated labels. We fine-tuned both general and domain-specific pre-trained language models (PTLMs) on the tweet questions. Experimental results show that TweetRoBERTa achieves the highest F1-score of 91.98, outperforming other PTLMs. Additionally, PTLMs trained on the TREC dataset and evaluated on the TweetQC dataset exhibited an accuracy decline of over 35% compared to models trained and evaluated on the TweetQC dataset. Furthermore, incorporating the expected answer type as an additional feature significantly enhances the performance of tweet QA models. Experimental results proves that TweetRoBERTa achieved the maximum ROUGEL score when compared with existing models for Tweet QA system.