A similarity-based multi-objective test optimization technique using search algorithm

Abstract

Context:

Software undergoes a constant evolution driven by ongoing changes in customer requirements, which enhances the competitive advantage. Regression testing plays a pivotal role by ensuring that modifications have not introduced detrimental effects on the system under test.

Problem:

However, regression testing becomes prohibitively expensive as the software grows in complexity and the size of the test suite also expands. Moreover, keeping the test cases up-to-date and managing the relevant test data can become a laborious and challenging task. Hence, it is required to optimize the test suite by finding a diverse subset of test cases having high code coverage, fault-detection rate, and minimal execution time.

Objective:

To solve the regression test optimization problem, the researchers have proposed various approaches including greedy algorithms, search-based algorithms, and clustering algorithms. However, existing approaches lack in finding the global optimal solution and are mostly focused on the single-objective test optimization problem. Inspired by this, we propose a Similarity-based Multi-Objective Optimization Technique (SMOOT) for test suite reduction using a Grey Wolf Optimizer (GWO) algorithm. The proposed technique employs different similarity metrics, including Cosine Similarity, Euclidean Distance, Jaccard Similarity, Manhattan Distance, and Minkowski Distance, to evaluate the similarity score of the tests. This ensures a comprehensive assessment of test diversity to achieve high code coverage and fault-detection rate while minimizing the test execution cost.

Method:

We evaluated the performance of GWO with state-of-the-art search-based algorithms using three varying types of case studies. Similarly, to evaluate the similarity score of the considered search algorithms, we employed state-of-the-art similarity measures.

Results:

The experimental results revealed that GWO significantly outperformed the considered search algorithms by attaining high code coverage and fault-detection rate while minimizing the test execution time. Moreover, we found that GWO attained a higher similarity score than the other considered search algorithms using the employed similarity measures.

Conclusion:

Based on the attained results, we believe that the proposed technique could be useful for the researchers and practitioners by effectively handling multi-objective regression test optimization problem.