Revealing Test Answer Behavior Patterns Through Quartile Analysis

Abstract

Characterizing a dataset by the mean value homogenizes the data to lose the integrity of the highs and lows, however, a quartile analysis quantifies the tendencies of both high- and low-performing participants for comparison. This study analyzed the grammar assessment responses of 8th grade students to determine patterns of response between the lowest and highest quartile. Using Peng’s Learning Portrait Model, each assessment cell was coded to show the accuracy of prior and subsequent answers. Analysis of these codes revealed that learners in the lowest quartile were significantly likely to respond inconsistently (variable accuracy, such as correct-incorrect-correct) and that learners in the highest quartile were significantly likely to respond consistently, whether correct or incorrect. Further, the baseline score increased over the course of seven months by 25% on unrelated content, suggesting that familiarity with the application software can account for that much of a student’s assessment score. Future explorations on the dynamics of online assessment and the persistence of students in resolving inaccuracies on digital assessments are encouraged.