Should We Give Learners Control Over Item Difficulty?

Personalized educational systems are able to provide learners questions of specified difficulty. Since learners differ, the appropriate level of difficulty may vary and it may be impossible to find an universal setting. We implemented a version of an adaptive educational system for geography practice that allows learners to adjust difficulty of questions. We evaluated this feature using a randomized control experiment. The overall results show only a small effect of the adjustment. A more detailed analysis, however, shows that for some groups of learners the effect can be important, although not necessarily advantageous. The collected data from the experiment provide insight into how to tune question difficulty automatically.