Rumor Detection Based on Improved Transformer

Abstract

In the field of rumor detection, the existing Transformer-based methods ignore the location information and fail to effectively use the potential information of the text. Therefore, we propose a social media rumor detection method based on improved Transformer that improves the standard Transformer through two novel techniques. First, learnable relative positional encoding is used to endow the Transformer with the ability of direction- and distance-awareness. Second, absolute positional encoding is used, through which each word with different absolute positions is mapped to its corresponding representation space. The experimental results show that, compared with the current best benchmark method, the accuracy of this method on the three data sets of Twitter15, Twitter16 and Weibo has increased by 0.9%, 0.6%, and 1.4%, respectively. The improved Transformer is effective and can significantly improve the effect of social media rumor detection.