Random Mutant Selection and Equivalent Mutants Revisited

Abstract

Mutation Testing offers a powerful approach to assessing unit test set quality; however, software developers are often reluctant to embrace the technique because of the tremendous number of mutants it generates, including redundant and equivalent mutants. In spite of their large numbers, redundant mutants are relatively innocuous. Recent research indicates that redundant mutants affect a test engineer’s work effort only slightly, whereas equivalent mutants have a direct linear impact. Moreover, the time invested analyzing equivalent mutants produces no unit tests. Dominator mutants address the redundancy problem by allowing the tester to write a minimal number of tests. This paper demonstrates empirically that when using randomly selected mutants as test requirements, the probability of drawing a dominator or quasi-dominator is initially double that of a non-dominator, and progressively increases. It also demonstrates that even non-dominator mutants are highly likely to elicit dominator killing tests. Finally it demonstrates that the probability of selecting an equivalent mutant quickly overwhelms all other selections. These observations provide new insight into the effectiveness of random mutant selection, and into the magnitude of the problem posed by equivalent mutants.