Using Continuous Integration Techniques in Open Source Projects—An Exploratory Study

For a growing number of software projects, continuous integration (CI) techniques are becoming an essential part of the process. However, the maturity of their adoption in open source projects varies. In this paper, we present an empirical study on GitHub repositories to explore the use of continuous integration techniques in open source projects. Following the Goal-Question-Metric (GQM) approach, 3 research questions and 7 metrics were defined for such a goal. We mined 10 repositories of open source projects with 101,149 pull requests, 399,671 commits from 20,432 developers. This was followed by exploratory data analysis for each metric. In summary, our results indicate that (RQ1) most failed CI builds required a small change in the pull request to fix the code; (RQ2) CI builds of smaller pull requests are more likely to succeed than larger ones; (RQ3) there was no correlation found between developer experience in committing to the project and the success rate of CI builds. Most of the projects studied have not yet developed a mature approach to using continuous integration techniques. In these cases, developers do not thoroughly test code before submitting pull requests. Furthermore, the results of the study confirmed that developers tend to submit pull requests with small amounts of new or modified code.