A systematic review on search-based test suite reduction: State-of-the-art, taxonomy, and future directions

Abstract

Regression testing remains a promising research area for the last few decades. It is a type of testing that aims at ensuring that recent modifications have not adversely affected the software product. After the introduction of a new change in the system under test, the number of test cases significantly increases to handle the modification. Consequently, it becomes prohibitively expensive to execute all of the generated test cases within the allocated testing time and budget. To address this situation, the test suite reduction (TSR) technique is widely used that focusses on finding a representative test suite without compromising its effectiveness such as fault-detection capability. In this work, a systematic review study is conducted that intends to provide an unbiased viewpoint about TSR based on various types of search algorithms. The study's main objective is to examine and classify the current state-of-the-art approaches used in search-based TSR contexts. To achieve this, a systematic review protocol is adopted and, the most relevant primary studies (57 out of 210) published between 2007 and 2022 are selected. Existing search-based TSR approaches are classified into five main categories, including evolutionary-based, swarm intelligence-based, human-based, physics-based, and hybrid, grounded on the type of employed search algorithm. Moreover, the current work reports the parameter settings according to their category, the type of considered operator(s), and the probabilistic rate that significantly impacts on the quality of the obtained solution. Furthermore, this study describes the comparison baseline techniques that support the empirical comparison regarding the cost-effectiveness of a search-based TSR approach. Finally, it isconcluded that search-based TSR has great potential to optimally solve the TSR problem. In this regard, several potential research directions are outlined as useful for future researchers interested in conducting research in the TSR domain.