SFIDMT-ART: A metamorphic group generation method based on Adaptive Random Testing applied to source and follow-up input domains

Context:

The performance of metamorphic testing relates strongly to the quality of test cases. However, most related research has only focused on source test cases, ignoring follow-up test cases to some extent. In this paper, we identify a potential problem that may be encountered with existing metamorphic group generation algorithms. We then propose a possible solution to address this problem. Based on this solution, we design a new algorithm for generating effective source and follow-up test cases.

Objective:

To improve the performance (test effectiveness and efficiency) of metamorphic testing.

Methods:

We introduce the concept of the input-domain difference problem, which is likely to affect the performance of metamorphic group generation algorithms. We propose a new test-case distribution criterion for metamorphic testing to address this problem. Based on our proposed criterion, we further present a new metamorphic group generation algorithm, from a black-box perspective, with new distance metrics to facilitate this algorithm.

Results:

Our algorithm performs significantly better than existing algorithms, in terms of test effectiveness, efficiency and test-case diversity.

Conclusions:

Through experiments, we find that the input-domain difference problem is likely to affect the performance of metamorphic group generation algorithms. The experimental results demonstrate that our algorithm can achieve good test efficiency, effectiveness, and test-case diversity.