The importance of centering harm in data infrastructures for ‘soft moderation’: X’s Community Notes as a case study

Abstract

This article critically examines the social implications of data infrastructures designed to moderate contested content categories such as disinformation. It does so in the context of new online safety regulation (e.g. the EU Digital Services Act) that pushes digital platforms to improve how they tackle both illegal and ‘legal but harmful’ content. In particular, we investigate and conceptualise X’s Community Notes, a tool that uses ‘human-AI cooperation’ to add context to tweets, as a data infrastructure for ‘soft moderation’. We find that Community Notes is limited when dealing with under-acknowledged online harms, such as those derived from the intersection between disinformation and humour. While research points to the potential of content moderation solutions that combine automation with humans-in-the-loop, we show how this approach can fail when disinformation is poorly defined in policy and practice.