New ideas: automated engineering of metamorphic testing environments for domain-specific languages

Abstract

Two crucial aspects for the trustworthy utilization of domain-specific languages (DSLs) are their semantic correctness, and proper testing support for their users. Testing is frequently used to verify correctness, but is often done informally -- which may yield unreliable results -- and requires substantial effort for creating suitable test cases and oracles. To alleviate this situation, we propose an automated technique for building metamorphic testing environments for DSLs. Metamorphic testing identifies expected relationships between the outputs of two consecutive tests, reducing the effort in specifying oracles and creating test cases manually. This new ideas paper presents the overarching concepts, the architecture and a prototype implementation. We illustrate our proposal using a DSL to model and simulate data centres.