A Preliminary Investigation of the Effects of Open Review on Citation Impact

Abstract

ABSTRACT This study investigates the impact of open review on citation counts of academic articles. By collecting articles published in the material science area of Nature Communications from 2014 to 2018, we exploit the adoption of voluntary open review in 2016 to examine how article citation counts are associated with open review using propensity score matching and regression analysis. OLS regression results show that open review comes at the expense of fewer citations received by articles. Specifically, open‐reviewed articles received 44% fewer citations than their non‐open‐reviewed counterparts. An additional mediation analysis reveals that this decrease in citations may be partially attributed to the lengthening of the review cycle of articles that were open reviewed–open‐reviewed articles experienced 16% longer review cycles; every 16% extension of the review cycle led to 5.8% fewer citations. Results remained qualitatively similar when we restricted the citation windows to three years since publication. Contrary to previous findings concerning the positive effects on articles’ citation impact, our preliminary results call for additional efforts in identifying the costs and benefits associated with open review.