An intelligent test management system for optimizing decision making during software testing

Abstract

To ensure the proper testing of any software product, it is imperative to cover various functional and non-functional requirements at different testing levels (e.g., unit or integration testing). Ensuring appropriate testing requires making a series of decisions—e.g., assigning features to distinct Continuous Integration (CI) configurations or determining which test specifications to automate. Such decisions are generally made manually and require in-depth domain knowledge. This study introduces, implements, and evaluates ITMOS (Intelligent Test Management Optimization System), an intelligent test management system designed to optimize decision-making during the software testing process. ITMOS efficiently processes new requirements presented in natural language, segregating each requirement into appropriate CI configurations based on predefined quality criteria. Additionally, ITMOS has the capability to suggest a set of test specifications for test automation. The feasibility and potential applicability of the proposed solution were empirically evaluated in an industrial telecommunications project at Ericsson. In this context, ITMOS achieved accurate results for decision-making tasks, exceeding the requirements set by domain experts.