Automatic Detection of Insulting Sentences in Conversation

Abstract

An overall goal of our work is to use machine-learning based solutions to assist children with communication difficulties in their communication task. In this paper, we concentrate on the problem of recognizing insulting sentences the child says, or insulting sentences that are told to him. An automated agent that is able to recognize such sentences can alert the child in real time situations, and can suggest how to respond to the resulting social situation. We composed a dataset of 1241 non-insulting and 1255 insulting sentences. We trained different machine learning methods on 90% randomly chosen sentences from the dataset and tested it on the remaining. We used the following machine learning methods: Multi-Layer Neural Network, SVM, Naive Bayes, Decision Tree, and Tree Bagger for the task. We found that the best predictors of the insulting sentences, were the SVM method, with 80% recall and over 75%precision, and the Multi-Layer Neural Network and the Tree Bagger, with precision and recall exceeding 75%, We also found that adding additional data to the learning process, such as 9500 labeled sentences from twitter, or adding the word “positive” and the word “negative” to sentences including positive or negative words, respectively, slightly improves the results in most of the cases. Our results provide the cornerstones for an automated system that would enable on-line assistance and consultation for children with communication disabilities, and also for other persons with communication problems, in a way that will enable them to function better in society through this assistance.