Grade Language Heterogeneity in Simulation Models of Peer Review

Abstract

Simulation models have proven to be valuable tools for studying peer review processes. However, the e ects of some of thesemodels’ assumptions have not been tested, nor have thesemodels been examined in comparative contexts. In this paper, we address two of these assumptions which go in tandem: (1) on the granularity of the evaluation scale, and (2) on the homogeneity of the grade language (i.e. whether reviewers interpret evaluation grades in the same fashion). We test the consequences of these assumptions by extending awell-known agent-basedmodel of author and reviewer behaviourwith discrete evaluation scales and reviewers’ interpretation of the grade language. In this way, we compare a peer review model with a homogeneous grade language, as assumed inmost models of peer review, with amore psychologically realistic model where reviewers interpret the grades of the evaluation scale heterogeneously. We find that grade language heterogeneity can indeed a ect the predictions of a model of peer review.