Detecting Offensive Content on Twitter During Proud Boys Riots

Abstract

Hateful and offensive speech on online social media platforms has seen a rise in the recent years. Often used to convey humor through sarcasm or to emphasize a point, offensive speech may also be employed to insult, deride and mock alternate points of view. In turbulent and chaotic circumstances, insults and mockery can lead to violence and unrest, and hence, such speech must be identified and tagged to limit its damage. This paper presents an application of machine learning to detect hateful and offensive content from Twitter feeds shared after the protests by Proud Boys, an extremist, ideological and violent hate group. A comprehensive coding guide, consolidating definitions of what constitutes offensive content based on the potential to trigger and incite people is developed and used to label the tweets. Linguistic, auxiliary and social features extracted from these labeled tweets were used to train machine learning classifiers, which detect offensive content with an accuracy of about 92%. An analysis of the importance scores reveals that offensiveness is pre-dominantly a function of words and their combinations, rather than meta features such as punctuations and quotes. This observation can form the foundation of pre-trained classifiers that can be deployed to automatically detect offensive speech in new and unforeseen circumstances.