Studying Retrieval Practice in an Intelligent Tutoring System

Retrieval practice (also known as testing effect or test-enhanced learning) is a well-studied and established technique for improving the retention of knowledge. Many previous works have confirmed the benefits of retrieval practice in laboratory experiments involving the memorization of words or facts. In this study, we build on these works and analyze retrieval practice in an intelligent tutoring system. Using a large data set composed of the actions of almost 4 million students studying math and chemistry, we look at the possible benefits of retrieval practice in the ALEKS adaptive learning and assessment system. We compare two different types of retrieval practice---one involving the assessment of learned material, and another involving the learning of closely related content that builds on the learned material---leveraging the scale of the available data to control for several confounding variables. Finally, we look at the timing of retrieval practice within the system and the possible effect it has on forgetting. The results indicate that a delay in retrieval practice is associated with better retention and that, while being assessed on learned material is beneficial, the learning of closely related content is associated with an even higher rate of retention.