Metamorphic Testing for Reliability in System of Systems

Abstract

System of Systems (SoS), refers to a class of systems that are comprised of independent constituent systems (CS) interacting with one another to form a larger system, thus resulting in a common goal being achieved through the set of interactions between the CSs. In conventional systems software testing, the problem of identifying correct behaviour given a set of test inputs is called the oracle problem. In terms of SoS, the oracle problem is exacerbated due to each CS in an SoS being sufficiently complex systems themselves. We address the oracle problem in the domain of the software testing in System of Systems (SoS) by applying Metamorphic Testing (MT). In order to derive concrete Metamorphic Relations (MR) for SoS, we first borrow the concept of creating MRs from abstract Metamorphic Relation Patterns (MRPs), where the represented MRPs themselves describe characteristics that are used in the modeling and analysis of SoS. They are then applied to two SoS simulator types, the Smart Grid SoS and Mass Casualty Incident (MCI) Response SoS in order to test for reliability in SoS and to also determine the viability of MT in SoS. It is shown through the experiments conducted that the derived concrete MRs are able to find faults in both the systems under test.