Large-scale evaluation of cold-start mitigation in adaptive fact learning: Knowing “what” matters more than knowing “who”

Abstract

Adaptive learning systems offer a personalised digital environment that continually adjusts to the learner and the material, with the goal of maximising learning gains. Whenever such a system encounters a new learner, or when a returning learner starts studying new material, the system first has to determine the difficulty of the material for that specific learner. Failing to address this “cold-start” problem leads to suboptimal learning and potential disengagement from the system, as the system may present problems of an inappropriate difficulty or provide unhelpful feedback. In a simulation study conducted on a large educational data set from an adaptive fact learning system (about 100 million trials from almost 140 thousand learners), we predicted individual learning parameters from response data. Using these predicted parameters as starting estimates for the adaptive learning system yielded a more accurate model of learners’ memory performance than using default values. We found that predictions based on the difficulty of the fact (“what”) generally outperformed predictions based on the ability of the learner (“who”), though both contributed to better model estimates. This work extends a previous smaller-scale laboratory-based experiment in which using fact-specific predictions in a cold-start scenario improved learning outcomes. The current findings suggest that similar cold-start alleviation may be possible in real-world educational settings. The improved predictions can be harnessed to increase the efficiency of the learning system, mitigate the negative effects of a cold start, and potentially improve learning outcomes.