Understanding Facial Expression Expressing Hate from Online Short-form Videos

Abstract

The impact of hate speech is not only detrimental to an individual's human rights; but also, a grave threat to social stability and democracy. Through social media, the spread of hate speech has alarmingly increased across the globe. Various social media platform's goal is to eliminate hateful content and this challenge poses the need for automatic and accurate hate speech detection. Presently, known techniques in this research primarily made use of either text or audio features. However, the use of the facial expression in hate speech detection is not that explored. Thus, for this study, the use of facial expressions to understand hate speech has been thoroughly investigated. The dataset used is image data generated from Filipino Tiktok videos with a frame size of 1080 x 1920 pixels and divided into 5 frames per second. Two approaches namely conventional and deep learning-based frameworks have been implemented in building the Facial Expression Recognition (FER) model to understand hate speech. Based on the experimentation, the conventional approach using the Random Forest approach has achieved the best performance with 86.9% training accuracy and 84.8% validation accuracy, outperforming the other conventional classifiers and the DL-based approach significantly. For future direction, facial expression features combined with text or audio input type will be implemented to examine whether the use of facial expression can complement or improve hate speech detection models.