Software and System Reliability Engineering for Autonomous Systems Incorporating Machine Learning

Abstract

Artificial intelligence and machine learning have attracted significant interest as enablers of autonomous systems. However, these techniques are susceptible to a variety of failures as well as adversarial attacks, suggesting the need for formal reliability and resilience engineering methods. Tempered by the knowledge that machine learning is not a panacea and that private industry, infrastructure management, and defense systems are regularly subject to external attack, it is essential to assess the possible failures and corresponding consequences that these technologies may inadvertently introduce. This paper seeks to bridge the gap between traditional and emerging methods to support the engineering of autonomous systems incorporating machine learning. Toward this end we seek to synthesize methods from established fields such as system and reliability engineering as well as software testing with recent trends in the design and test of machine learning algorithms. The proposed approach should provide organizations with additional structure to comprehend and allocate their risk mitigation efforts in order to address issues that will inevitably arise from these less well understood technologies.