A Novelty Search and Metamorphic Testing Approach to Automatic Test Generation

Abstract

A common task in search-based testing is automatically identifying valuable test cases for software systems. However, existing approaches tend to either search for unique tests with regard to inputs or outputs (i.e., novelty search) or search for inputs that invalidate some expected proposition regarding the software (i.e., metamorphic testing). Problematically, verifying unique tests induces the oracle problem while an invalidated proposition results in a single test case. In this paper we utilize novelty search and metamorphic testing to discover a broad range of unique test cases that are directly verifiable via a metamorphic relation and invalidate such an expected proposition in fewer generations of an evolutionary algorithm than direct search. We apply this novelty search and metamorphic testing combination to discover errors in identifying the midpoint of a geodesic as a proof-of-concept.