Enhancing Learning-Enabled Software Systems to Address Environmental Uncertainty

Abstract

An overarching problem with Learning-Enabled Systems (LES) is determining whether training data is sufficient to ensure the LES is resilient to environmental uncertainty and how to obtain better training data to improve the system's performance when it is not. Automated methods can ease the burden for developers by augmenting real-world data with synthetically generated data. We propose an evolution-based method to assist developers with the assessment of learning-enabled systems in environments not covered by available datasets. We have developed Enki, a tool that can generate various conditions of the environment in order to discover properties that lead to diverse and unique system behaviors. These environmental properties are then used to construct synthetic data for two purposes: (1) to assess a system's performance in an uncertain environment and (2) to improve system resilience in the presence of uncertainty. We show that our technique outperforms a random generation method when assessing the effect of multiple adverse environmental conditions on a Deep Neural Network (DNN) trained for the commonly-used CIFAR-10 benchmark.