Component-based architectural regression test selection for modularized software systems

Abstract

Regression testing is an essential part of software development, but it can be costly and require significant computational resources. Regression Test Selection (RTS) improves regression testing efficiency by only re-executing the tests that have been affected by code changes. Recently, dynamic and static RTS techniques for Java projects showed that selecting tests at a coarser granularity, class-level, is more effective than selecting tests at a finer granularity, method- or statement-level. However, prior techniques are mainly considering Java object-oriented projects but not modularized Java projects. Given the explicit support of architectural constructs introduced by the Java Platform Module System (JPMS) in the ninth edition of Java, these research efforts are not customized for component-based Java projects. To that end, we propose two static component-based RTS approaches called CORTS and its variant C2RTS tailored for component-based Java software systems. CORTS leverages the architectural information such as components and ports, specified in the module descriptor files, to construct module-level dependency graph and identify relevant tests. The variant, C2RTS, is a hybrid approach in which it integrates analysis at both the module and class levels, employing module descriptor files and compile-time information to construct the dependency graph and identify relevant tests.

We evaluated CORTS and C2RTS on 1200 revisions of 12 real-world open source software systems, and compared the results with those of class-level dynamic (Ekstazi) and static (STARTS) RTS approaches. The results showed that CORTS and C2RTS outperformed the static class-level RTS in terms of safety violation that measures to what extent an RTS technique misses test cases that should be selected. Using Ekstazi as the baseline, the average safety violation with respect to Ekstazi was 1.14% for CORTS, 2.21% for C2RTS, and 3.19% for STARTS. On the other hand, the results showed that CORTS and C2RTS selected more test cases than Ekstazi and STARTS. The average reduction in test suite size was 22.78% for CORTS and 43.47% for C2RTS comparing to the 68.48% for STARTS and 84.21% for Ekstazi. For all the studied subjects, CORTS and C2RTS reduced the size of the static dependency graphs compared to those generated by static class-level RTS, leading to faster graph construction and analysis for test case selection. Additionally, CORTS and C2RTS achieved reductions in overall end-to-end regression testing time compared to the retest-all strategy.