Student rating consistency in online peer assessment from the perspectives of individual and class

Abstract

Most of previous studies examined the consistency among two or three student raters giving scores to different peers (i.e. ratees), whereas few of them researched the consistency among more student raters giving scores to different peers (i.e. ratees). The present study aimed to see whether there is a consistency among student raters giving scores to peers in an online assessment activity. Participants of the study were 266 graduate students, with 137 males and 129 females, from 27 classes in a department at some university. Each student was required to give a presentation graded by the peers anonymously. From to the perspective of class, the result of the Kendall coefficient of concordance showed that a proportion of more than half-classes presented a significant correlation with a moderate or strong consistency among student raters giving scores to peers, and thus the number of raters or peers should be more for a higher reliability. The result of Chi-square test from the perspective of individual revealed that students held a consistent standard when giving scores to peers' presentations. Finally the implications for theories and practices are proposed.