Framing Feedback: Choosing Review Environment Features that Support High Quality Peer Assessment

Abstract

Peer assessment is rapidly growing in online learning, as it presents a method to address scalability challenges. However, research suggests that the benefits of peer review are obtained inconsistently. This paper explores why, introducing three ways that framing task goals significantly changes reviews. Three experiments manipulated features in the review environment. First, adding a numeric scale to open text reviews was found to elicit more explanatory, but lower quality reviews. Second, structuring a review task into short, chunked stages elicited more diverse feedback. Finally, showing reviewers a draft along with finished work elicited reviews that focused more on the work's goals than aesthetic details. These findings demonstrate the importance of carefully structuring online learning environments to ensure high quality peer reviews.