Characterization of the Xen Project Code Review Process: an Experience Report

Abstract

Many software development projects have introduced manda-tory code review for every change to the code. This meansthat the project needs to devote a significant effort to re-view all proposed changes, and that their merging into thecode base may get considerably delayed. Therefore, all thoseprojects need to understand how code review is working, andthe delays it is causing in time to merge.This is the case in the Xen project, which performs peerreview using mailing lists. During the first half of 2015, somepeople in the project observed a large and sustained increasein the number of messages related to code review, which hadstarted some years before. This observation led to concernson whether the code review process was having some trouble,and too large an impact on the overall development process.Those concerns were addressed with a quantitative study,which is presented in this paper. Based on the informa-tion in code review messages, some metrics were defined toinfer delays imposed by code review. The study producedquantitative data suitable for informed discussion, which theproject is using to understand its code review process, andto take decisions to improve it.