Deep-Learning-Based Automated Scoring for the Severity of Toxic Comments Using Electra

With the increasing popularity of the Internet, social media plays a crucial role in people's daily communication. However, due to the anonymity of Internet, toxic comments emerge in an endless stream on the Internet, which seriously affects the health of online social environment. To effectively reduce the impact of toxic comments, automated scoring methods for the severity of toxic comments are in great demand. For that purpose, a deep-learning-based natural language processing technique is proposed using ELECTRA to automatically score the toxicity of a comment in this work. The backbone of our model is the ELECTRA discriminator, and the downstream regression task is accomplished by the following head layer. Three head layers are implemented separately: multi-layer perceptron, convolutional neural network, and attention. The dataset used for model training is from the Kaggle competition Toxic Comment Classification Challenge, and the model performance is evaluated through another Kaggle competition Jigsaw Rate Severity of Toxic Comments. By a boost from the K-Fold cross validation and an ensemble of three models with different head layers, our method can reach a competition score 0.80343. Such score ranks 71/2301 (top 3.1%) in the leaderboard and can get a silver medal in the competition. The results in this work would help filter the toxic comments and harmful text information automatically and effectively on the Internet, and could greatly reduce the cost of manual review and help build a healthier Internet environment.