Scoring Essays on an iPad Versus a Desktop Computer: An Exploratory Study

Abstract

Recognizing the appealing features of a tablet (e.g., an iPad), including size, mobility, touch screen display, and virtual keyboard, more educational professionals are moving away from larger laptop and desktop computers and turning to the iPad for their daily work, such as reading and writing. Following the results of a recent survey of individuals who serve as ETS raters, more than 40% reported that they would prefer to use an iPad or other type of tablet to score essays. However, iPad-based essay scoring could affect scoring accuracy and scoring time because the smaller screen and other features of an iPad may also affect raters' reading comprehension and score assigning processes. To address this issue, we invited 10 experienced raters to score holistically 40 essays for a graduate admission test using a desktop computer and an iPad following a counterbalanced design. We compared the raters' scores against the criterion scores and analyzed scoring times, scoring behaviors, and raters' answers to a structured interview after the scoring experiment. The results reveal no obvious differences between the two devices in the scoring accuracy or average scoring time per essay, which suggests that scoring on an iPad may not reduce scoring quality or scoring productivity for essays that are holistically scored as compared to scoring the essays on a desktop computer. We also found a few iPad-specific issues that raters reported, including issues associated with the invisible scrolling bar and the extra scrolling needed to reach the score-assignment panel, difficulty navigating between the prompt and the essay response, and oversensitivity of the touch screen.