An Industrial Study on the Differences between Pre-Release and Post-Release Bugs

Abstract

Software bugs constitute a frequent and common issue of software development. To deal with this problem, modern software development methodologies introduce dedicated quality assurance procedures. At the same time researchers aim at developing techniques capable of supporting the early discovery and fix of bugs. One important factor that guides such research attempts is the characteristics of software bugs and bug fixes. In this paper, we present an industrial study on the characteristics and differences between pre-release bugs, i.e. bugs detected during software development, and post-release bugs, i.e. bugs that escaped to production. Understanding such differences is of paramount importance as it will improve our understanding on the testing and debugging support that practitioners require from the research community, on the validity of the assumptions of several research techniques, and, most importantly, on the reasons why bugs escape to production. To this end, we analyze 37 industrial projects from BGL BNP Paribas and document the differences between pre-release bugs and post-release bugs. Our findings suggest that post-release bugs are more complex to fix, requiring developers to modify several source code files, written in different programming languages, and configuration files, as well. We also find that approximately 82% of the post-release bugs involve code additions and can be characterized as 'omission' bugs. Finally, we conclude the paper with a discussion on the implications of our study and provide guidance to future research directions.