An Empirical Study on Continuous Integration Trends, Topics and Challenges in Stack Overflow

Abstract

During the last few years, Continuous Integration (CI) has become a common practice in open-source and industrial environments to reduce the scope for errors and increase the speed to market through the automated build and test processes. However, despite this wide adoption throughout the years, little is known about the challenges developers discuss. Analyzing the discussions of developers is required to understand what researchers, educators and practitioners should focus on, and how discussion communities can be helpful to shed the light on CI challenges. In this study, we examine Stack Overflow (SO), the most popular crowd-sourced forum, to understand the challenges developers face in the CI context. We collect a corpus of 27,728 CI-related developers posts from SO and analyze those posts through a mixed method with quantitative and qualitative analyzes. To study the trends of CI discussions, we investigated the metadata of CI questions, users and tags. Then, we extract the CI main topics using Latent Dirichlet Allocation (LDA) tuned with Genetic Algorithm (GA). Finally, we investigate the most popular and difficult topics faced by developers based on unanswered questions to get further insights into CI challenges. The LDA clustering reveals that developers face challenges with six main topics namely Build, Testing, Version Control, Configuration, Deployment, and CI Culture. Particularly, we found that the build topic is the most popular among the studied topics and that version control and testing topics are the most difficult for the SO community. Our study uncovers insights about CI challenges and adds evidence to existing knowledge about CI issues related especially to software build.