Context in abusive language detection: On the interdependence of context and annotation of user comments

Abstract

One of the challenges for automated abusive language detection is combating unintended bias, which can be easily introduced through the annotation process, especially when what is (not) considered abusive is subjective and heavily context dependent. Our study incorporates a fine-grained, socio-pragmatic perspective to data modeling by taking into consideration contextual elements that impact the quality of abusive language corpora. We use a fine-grained annotation scheme that distinguishes between different types of non-abuse along with explicit and implicit abuse. We include the following non-abusive categories: meta, casual profanity, argumentative language, irony, and non-abusive language. Experts and minimally trained annotators use this scheme to manually re-annotate instances originally considered abusive by crowdsourced annotators in a standard corpus. After re-annotation, we investigate discrepancies between experts and minimally trained annotators. Our investigation shows that minimally trained annotators have difficulty interpreting contextual aspects and distinguishing between content performing abuse and content about abuse or instances of casual profanity. It also demonstrates how missing information or contextualization cues are often a source of disagreement across all types of annotators and poses a significant challenge for developing robust, nuanced corpora and annotation guidelines for abusive language detection.