Estimating the Return on Investment of Defect Taxonomy Supported System Testing in Industrial Projects

Abstract

Defect taxonomies collect and organize the domain knowledge and project experience of experts and are a valuable instrument of system testing for several reasons. They provide systematic backup for the design of tests, support decisions for the allocation of testing resources and provide a suitable basis for measuring the product and test quality. In this paper, we present a method of system testing based on defect taxonomies and an appropriate estimation procedure for its return on investment depending on several parameters like the average test design time or the number of test cycles and experience values of a test organization. The estimated return on investment provides decision support whether to apply defect taxonomy supported system testing for a specific product or not. We develop the estimation procedure in the context of an industrial project from a public health insurance institution where the return on investment was positive after the first test cycle. From the experience of this project we extract guidelines and heuristics for precise estimation and interpretation of the return on investment of defect taxonomy supported system testing in the context of other projects.