Using Delta Debugging to Minimize Stress Tests for Concurrent Data Structures

Abstract

Concurrent data structures are often tested under stress to detect bugs that can only be exposed by some rare interleavings of instructions. A typical stress test for a concurrent data structure creates a number of threads that repeatedly invoke methods of the target data structure. After a failure is detected by a stress test, developers need to localize the fault causing the failure. However, the execution trace of a failed stress test may be very long, making it time-consuming to replay the failure and localize the fault. In this paper, we present an approach to minimizing stress tests for concurrent data structures. Our approach is to create a smaller test that still produces the same failure by removing some of the threads and/or method invocations in the original stress test. We apply delta debugging to identify the threads and method invocations that are essential for causing the failure. Other threads and method invocations are removed to create a smaller stress test. To increase the chance of triggering the original failure during the execution of the new stress test, we force the new execution to replay the original failed execution trace when possible, and try to guide the execution back to the failed trace when the execution diverges. We describe a tool called TestMinimizer and report the results of an empirical study in which TestMinimizer was applied to 16 real-life concurrent data structures. The results of our evaluation showed that TestMinimizer can effectively and efficiently minimize the stress tests for these concurrent data structures.