Your body tells how you engage in collaboration: Machine-detected body movements as indicators of engagement in collaborative math knowledge building

Collaborative learning, driven by knowledge co-construction and meaning negotiation, is a pivotal aspect of educational contexts. While gesture's importance in conveying shared meaning is recognized, its role in collaborative group settings remains understudied. This gap hinders accurate and equitable assessment and instruction, particularly for linguistically diverse students. Advancements in multimodal learning analytics, leveraging sensor technologies, offer innovative solutions for capturing and analysing body movements. This study employs these novel approaches to demonstrate how learners' machine-detected body movements during the learning process relate to their verbal and nonverbal contributions to the co-construction of embodied math knowledge. These findings substantiate the feasibility of utilizing learners' machine-detected body movements as a valid indicator for inferring their engagement with the collaborative knowledge construction process. In addition, we empirically validate that these inferred different levels of learner engagement indeed impact the desired learning outcomes of the intervention. This study contributes to our scientific understanding of multimodal approaches to knowledge expression and assessment in learning, teaching, and collaboration.