Evaluating the effectiveness of machine translation of audio description: the results of two pilot studies in the English-Dutch language pair

Abstract

The field of translation is undergoing various profound changes. On the one hand it is being thoroughly reshaped by the advent and constant improvement of new technologies. On the other hand, new forms of translation are starting to see the light of day in the wake of social and legal developments that require that products and content that are created, are accessible for everybody. One of these new forms of translation, is audio description (AD), a service that is aimed at making audiovisual content accessible to people with sight loss. New legislation requires that this content is accessible by 2025, which constitutes a tremendous task given the limited number of people that are at present trained as audio describers. A possible solution would be to use machine translation to translate existing audio descriptions into different languages. Since AD is characterized by short sentences and simple, concrete language, it could be a good candidate for machine translation. In the present study, we want to test this hypothesis for the English-Dutch language pair. Three 30 minute AD excerpts of different Dutch movies that were originally audio described in English, were translated into Dutch using DeepL. The translations were analysed using the harmonized DQF-MQM error typology and taking into account the specific multimodal nature of the source text and the intersemiotic dimension of the original audio description process. The analysis showed that the MT output had a relatively high error rate, particularly in the categories of Accuracy – mistranslation and Fluency – grammar. This seems to indicate that extensive post-editing will be needed, before the text can be used in a professional context.