Student profiles based on in-game performance and help-seeking behaviours in an online mathematics game

Abstract

Well-designed online educational games can improve students' math knowledge, skills and engagement; however, more research is needed to understand how to formatively assess components of students' mathematical understanding and learning as students solve problems in online educational games. In this study, we examined how 7th-grade students' (N = 760) in-game performance (ie, strategy efficiency, mathematical validity of the first action, mathematical errors) and help-seeking behaviours (ie, hint requests) correlated with their algebraic knowledge in an online mathematics game. The k-means cluster analysis identified four groups of students based on their in-game metrics, and some variabilities in their in-game performance were found. Although hint requests were available, only a few students showed a high percentage of hint requests during their gameplay. The regression analysis revealed that students' in-game performance and help-seeking behavioural patterns explained a significant amount of variance in students' algebraic knowledge above and beyond their prior knowledge. Our results suggest that how students interact with the online learning game is related to their later math performance, and qualitative aspects of student behaviours captured using log data provide meaningful information to students' algebraic understanding.

Practitioner notes

What is already known about this topic

• Well-designed online educational games can improve students' math knowledge, skills, and engagement.
• Several studies found that some in-game metrics, such as in-game progress or correctness of tasks, are positively associated with learning outcomes.
• However, many studies used simple metrics like the correctness of each task or the number of problems completed to measure students' performance in the game.
• Few studies measured more in the moment or qualitative aspects of student in-game performance, for example, how efficiently students solved problems in the game.

What this paper adds

• In-game metrics measuring more in the moment and qualitative aspects of students' problem-solving explained a significant amount of variance in students' algebraic knowledge above and beyond students' prior knowledge.
• Students' strategy efficiency and strategic planning were positively associated with students' math problem-solving performance.
• Although hints were available for almost all problems, only a few students exhibited a high level of help-seeking behaviours, and the students with the lowest in-game performance who needed help the most showed low help-seeking behaviours.

Implications for practice and/or policy

• Classifying students into profiles could provide ways to identify students with different levels of problem-solving skills and help teachers provide more personalized instruction.
• Teachers could use this information as a formative assessment to identify the core areas of intervention needed, particularly for students struggling with math concepts taught in the game.
• Instructional strategies that make hints more appealing or engaging to students might be needed.