SoReady: An Extension of the Test and Defect Coverage-Based Analytics Model for Pull-Based Software Development

Abstract

Pull-based software development is a distributed development model that offers an opportunity to review a pull request before it gets merged into the main repository. A pull request addresses new features, bug fixing, and maintenance issues submitted by both integrators or contributors. It appears that many empirical studies are conducted to discover how pull request evaluation is done, and to our knowledge, limited research exists for assessing release readiness of pull requests. Studies also reported that the failure rate of pull-requests rapidly increases when there are many forks created. It is therefore, questions worth exploring are whether the code review really contributing to the code quality, and how to determine the release readiness of pull requests? In our previous work, test and defect coverage-based analytics model (TDCAM) has been proven to be suitable to determine the readiness of releases for software that is rapidly evolving, in which this is also a characteristic of pull-based software development. In this paper, the TDCAM has been extended to include pull request coverage indicators. The proposed model, namely as SoReady and the visualization analysis presented herein has enabled five developers in a commercial setting to make informed and evidence-based decisions regarding the test status of each pull request and overall reliability of an open source software through a prototype dashboard.