ISTA+: Test case generation and optimization for intelligent systems based on coverage analysis

With the increasing use of intelligent systems in various domains such as self-driving cars, robotics, and smart cities, it is crucial to ensure the quality of intelligent systems for their reliable and effective use in various domains. However, testing intelligent systems poses unique challenges due to their complex structure, low efficiency, and the high cost associated with manually collecting a large number of test cases. Hence, it is crucial to design tools that can adequately test intelligent systems while overcoming these obstacles.

We propose an intelligent system test tool called ISTA+. This tool implements automatic generation and optimization of test cases based on coverage analysis, resulting in improved test adequacy for intelligent systems. To evaluate the effectiveness of ISTA+, we applied it to two different models (fully-connected DNN and the Rambo model) and two datasets of different data types (i.e., image and text). The evaluation results demonstrate that ISTA+ successfully improves the test dataset quality and ensures comprehensive testing for both text and image data types.

• •

  Link to source code: https://github.com/wuxiaoxue/ISTAplus

• •

  Link to video demonstration: https://youtu.be/6CkzMJ0ghq8