Audiovisual translation as orchestration of multimodal synergies

Abstract

This study explores the coordination between subtitles and other semiotic resources in films, from within a systemic functional semiotic framework. The paper focuses on the subtitling of interjections (e.g., ‘oh’, ‘wow’, ‘yay’) to examine how subtitles are synthesized in films with respect to the presence of facial expressions and/or bodily gestures which might be perceived as conveying similar meanings. The analysis of multiple versions of intralingual subtitles of two English-language films shows that the seemingly random omission of soundtrack interjections from subtitles is patterned to a considerable degree – those interjections which are concurrent with semiotically correlated actions were frequently omitted. The findings suggest that subtitling is a complex process involving synergy between subtitles and other semiotic resources, which calls for interdisciplinary research integrating translation, multimodality, and linguistics.