Analysis and design of mastery learning criteria

ABSTRACT A common personalisation approach in educational systems is mastery learning. A key step in this approach is a criterion that determines whether a learner has already achieved mastery. We thoroughly analyse several mastery criteria for the basic case of a single well-specified knowledge component. For the analysis we use experiments with both simulated and real data. The results show that the choice of data sources used for mastery decision and the setting of thresholds are more important than the choice of a learner modelling technique. We argue that a simple exponential moving average method is a suitable technique for mastery criterion and discuss techniques for the choice of a mastery threshold. We also propose an extension of the exponential moving average method that takes into account practical aspects like time intensity of items and we report on a practical application of this mastery criterion in a widely used educational system.