Investigating Problem Solving on Calculator Items in a Large-Scale Digitally Based Assessment: A Data Mining Approach

Abstract

This exploratory study investigated the behaviors and content of onscreen calculator usage by a nationally representative sample of eighth-grade students who responded to items from the 2017 National Assessment of Educational Progress mathematics assessment. Meaningful features were generated from the process data to infer whether students spontaneously used calculators for mathematical problem solving, how frequently and when they used them, and the nature of the operations performed on calculators. Sequential pattern mining was applied on sequences of calculator keystrokes to obtain patterns of operations that were representative of students’ problem-solving strategies or processes. Results indicated that higher scoring students not only were more likely to use calculators, but also used them in a more goal-driven manner than lower scoring students.