A New Code Review Method based on Human Errors

Abstract

Modern code reviews tend to take a lightweight process, in which the accuracy and efficiency of identifying defects rely heavily on code reviewers’ experience. The human errors of developers, as a significant cause of software defects, is a key to identifying defects. However, there is a lack of understanding of the human error mechanisms underlying defects in code. This paper proposes an innovative code review method for identifying defects by pinpointing the scenarios that developers tend to commit errors. The method was validated by a comprehensive experimental study that involved 49 code reviewers organized in two independent groups, i.e. experimental group vs. controlled group for each other. Forty reviewers have completed the whole experiment and provided the data for statistical analysis on the effects of the approach. The experiment shows that the proposed method has significantly improved True Positives and Sensitivity by about 400%, improved Precision by approximately 200%, and reduced around one-third of False Positives. The effects were consistent across different tasks and different code reviewers.