Navigating a multimodal ensemble: Learners mediating verbal and non-verbal turns in online interaction tasks

Abstract Research into the multimodal aspects of language is increasingly important as communication through a screen plays a greater role in modern society than ever before (Liou, 2011). Multimodality has been explored from a number of angles relating to computer-mediated communication (CMC), such as its affordances and impact on language learners, highlighting its relevance and importance in the field of second language acquisition (SLA). Because CMC scenarios require attending to both peers and the screen, learners can be seen as positioned as “semiotic initiators and responders” (Coffin & Donohue, 2014). Increasingly, researchers are highlighting a need for a methodological “turn” to analyse this scenario from a “language” focus to a more holistic understanding of the interactions (Flewitt, 2008; Hampel & Hauck, 2006; Kress & van Leeuwen, 2001; Lamy, 2006). Along these lines, this case study explores how the action of task completion is mediated between six dyads (and individuals within the dyads) during an online peer-to-peer audioconferencing event. Drawing on notions from multimodal (inter)actional analysis (Norris, 2004, 2006) and the notion of “semiotic initiators and responders”, it investigates semiotic mediation with screen-based resources through analysis of audio recordings, screenshots, log files, task simulation and reconstruction. Results highlight oral and screen-based initiations and responses that take place during task completion, which is presented as a framework.