Investigating Bugs in AI-Infused Systems: Analysis and Proposed Taxonomy

Abstract

Testing for critical AI systems is non-trivial as these systems are prone to a new breed of sophisticated software defects. The admissibility of these systems and their fundamental social acceptance is tightly coupled with assuring whether the potential hazards to humans, animals, and property posed by the prospect defects can be minimized and limited to an acceptable level. In this work, we address the problem of assurance for critical AI systems by firstly, analyzing the nature of defects that occur in AI -infused systems in general and how to combat these within a testing strategy. Secondly, developing a focused taxon-omy of prospect defects in critical AI systems. This taxonomy enables the development of the non-critical proxy (i.e., stand-in) equivalent by reproducing defects with similar characteristics.