Machine Learning Performance Comparison for Toxic Speech Classification : Online Payday Loan Scams in Indonesia

Abstract

The recent advancement of Machine Learning (ML) has brought us to many implementations. Online payday loan scam is a phenomenon which interestingly containing toxic speech in conversation. Toxic speech means implying threat toxic speech, offensive language, and hate speech. toxic speech would ultimately trigger such responses, namely loss of work ethic, alienation from the social, even suicidal thought. Despite the unnerving impact of toxic speech, there is still little known research regarding toxic speech, one of them is how to classify toxic speech. This research aims to make a comparison of various ML techniques with the means of classifying toxic speech found in the online payday loan scam phenomenon. For this experiment, we employed Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Random Forest (RF), and k-Nearest Neighbour (k-NN). All data were taken, filtered, and normalized manually from YouTube. Many reported the incident of online payday loan scam via YouTube in the form of two-way call communication. In total, there are 79 fraud report records converted into *.wav files, followed by the feature extraction process using openSMILE, and are classified using machine learning. We get the MLP result which has an acquisition value of 97.9%, below that received SVM 97.2%.