Fairer but Not Fair Enough On the Equitability of Knowledge Tracing

Abstract

Adaptive educational technologies have the capacity to meet the needs of individual students in theory, but in some cases, the degree of personalization might be less than desired, which could lead to inequitable outcomes for students. In this paper, we use simulations to demonstrate that while knowledge tracing algorithms are substantially more equitable than giving all students the same amount of practice, such algorithms can still be inequitable when they rely on inaccurate models. This can arise as a result of two factors: (1) using student models that are fit to aggregate populations of students, and (2) using student models that make incorrect assumptions about student learning. In particular, we demonstrate that both the Bayesian knowledge tracing algorithm and the N-Consecutive Correct Responses heuristic are susceptible to these concerns, but that knowledge tracing with the additive factor model may be more equitable. The broader message of this paper is that when designing learning analytics algorithms, we need to explicitly consider whether the algorithms act fairly with respect to different populations of students, and if not, how we can make our algorithms more equitable.