An evaluation of continuous integration and delivery frameworks for classroom use

Abstract

Continuous integration and delivery (CI/CD) frameworks are a core element of DevOps-based software development. A PHP-based case study assessed the suitability of five such frameworks---JFrog Arti-factory, Bitbucket Pipelines, Jenkins, Azure DevOps, and TeamCity---for instructional use. The five were found to be roughly equivalent in terms of their usability for simple configurations. The effort needed to implement CI/CD substantially increased for more realistic production scenarios, like deployments to cloud and load-balanced platforms. These results suggest a need to limit CI/CD-based academic projects to simple infrastructure and technology stacks: e.g., a web application on a single instance web server.