The effectiveness of intelligent tutoring on training in a video game

Abstract

In this paper we evaluate the effectiveness of intelligent tutoring approaches on mastery and learning in a serious 3D immersive game called Heuristica. Heuristica teaches students to recognize and mitigate cognitive biases using a set of scenarios on a space station to perform tasks such as diagnosing and repairing problems or observing and evaluating game characters performing tasks. The student is evaluated on interactions in the 3D environment and on answers to questions provided by text or audio. We tested two types of tailoring: a) Student Model guided gameplay based on performance and b) Student Model guided gameplay with added worked-out examples (WOEs) whenever the student displays specific misconceptions or bugs in reasoning. We expected that customizing a player's game experience based on his or her pre-test knowledge scores and in-game behavior would tailor the learning experience and improve the effectiveness of the training. Ninety-four participants played one of three versions of the game, and the experiment evaluated and compared the efficacy of the game using either a fixed-order version of the game (no tailoring) or either of the two tailoring approaches. Differences in the mastery scores captured during gameplay provided additional insight into these results. Implications for intelligent tutoring use in games are discussed.