MutantBench: an Equivalent Mutant Problem Comparison Framework

The Equivalent Mutant Problem is an ongoing research problem that led to the creation of multiple reusable non-standardized mutant datasets. However, cross-study evaluation is still tedious. To tackle this problem, we propose MutantBench, a novel open-source comparison framework that is designed with a focus on the FAIR data principles and adoptability by the community. With this, tools within the Equivalent Mutant Problem can be empirically evaluated using the same dataset which then allows for improved cross-study evaluation. We also combine existing datasets resulting in a mutant dataset containing 4400 mutants with 1416 equivalent mutants. This increases the previously largest mutant dataset by more than a thousand equivalent mutants.