Enhancing logic-based testing with EvoDomain: A search-based domain-oriented test suite generation approach

Context

Effective software testing requires test adequacy criteria. MC/DC, a widely used logic-based testing criterion, struggles to detect domain errors caused by incorrect arithmetic operations. Domain errors occur when test requirement boundaries shift or tilt, causing unpredictable behavior and system crashes.

Objective

To address the inadequacy of MC/DC in detecting domain errors, we present EvoDomain, a search-based testing technique.

Method

EvoDomain uses a memetic algorithm combining genetic and hill-climbing algorithms, along with the DBSCAN clustering algorithm to select diversified boundary test data. The memetic algorithm is designed to efficiently enhance the search process for covering boundary test data. We compared EvoDomain with two logic-based testing approaches, a domain-oriented test suite generation approach, and random testing.

Results

Evaluations on 30 case studies show EvoDomain increases fault detection by 74.44% over MC/DC and 65.06% over RoRG. Additionally, EvoDomain improves support for different fault types by up to 68.89% for MC/DC and 66.33% for RoRG. Compared to COSMOS, which uses static analysis, EvoDomain improves the convergence effectiveness of identifying feasible subdomains by 32%. It offers high accuracy (0.99-1) and F1-score (0.99-1). EvoDomain finds the subdomains in less than 1/3 the time of Random search.

Conclusion

EvoDomain effectively generates domain-oriented test suites, enhancing the accuracy and effectiveness of fault detection.