Ekstazi: Lightweight Test Selection

Abstract

Regression testing is a crucial, but potentially time-consuming, part of software development. Regression test selection (RTS), which runs only a subset of tests, was proposed over three decades ago as a promising way to speed up regression testing. However, RTS has not been widely adopted in practice. We propose EKSTAZI , a lightweight RTS tool, that can integrate well with testing frameworks and build systems, increasing the chance for adoption. EKSTAZI tracks dynamic dependencies of tests on files and requires no integration with version-control systems. We implemented EKSTAZI for Java+JUnit and Scala+ScalaTest, and evaluated it on 615 revisions of 32 open-source projects (totaling almost 5M LOC). The results show that EKSTAZI reduced the end-to-end testing time by 32% on average compared to executing all tests. EKSTAZI has been adopted for day-to-day use by several Apache developers. The demo video for EKSTAZI can be found at http://www.youtube.com/watch?v=jE8K5_UCP28.