Extracting and classifying exceptional COVID-19 measures from multilingual legal texts: The merits and limitations of automated approaches

Abstract

This paper contributes to ongoing scholarly debates on the merits and limitations of computational legal text analysis by reflecting on the results of a research project documenting exceptional COVID-19 management measures in Europe. The variety of exceptional measures adopted in countries characterized by different legal systems and natural languages, as well as the rapid evolution of such measures, pose considerable challenges to manual textual analysis methods traditionally used in the social sciences. To address these challenges, we develop a supervised classifier to support the manual coding of exceptional policies by a multinational team of human coders. After presenting the results of various natural language processing (NLP) experiments, we show that human-in-the-loop approaches to computational text analysis outperform unsupervised approaches in accurately extracting policy events from legal texts. We draw lessons from our experience to ensure the successful integration of NLP methods into social science research agendas.