Jon Ayerdi;Valerio Terragni;Gunel Jahangirova;Aitor Arrieta;Paolo Tonella
{"title":"GenMorph:通过遗传编程自动生成变形关系","authors":"Jon Ayerdi;Valerio Terragni;Gunel Jahangirova;Aitor Arrieta;Paolo Tonella","doi":"10.1109/TSE.2024.3407840","DOIUrl":null,"url":null,"abstract":"Metamorphic testing is a popular approach that aims to alleviate the oracle problem in software testing. At the core of this approach are Metamorphic Relations (MRs), specifying properties that hold among multiple test inputs and corresponding outputs. Deriving MRs is mostly a manual activity, since their automated generation is a challenging and largely unexplored problem. This paper presents \n<sc>GenMorph</small>\n, a technique to automatically generate MRs for Java methods that involve inputs and outputs that are boolean, numerical, or ordered sequences. \n<sc>GenMorph</small>\n uses an evolutionary algorithm to search for \n<italic>effective</i>\n test oracles, i.e., oracles that trigger no false alarms and expose software faults in the method under test. The proposed search algorithm is guided by two fitness functions that measure the number of false alarms and the number of missed faults for the generated MRs. Our results show that \n<sc>GenMorph</small>\n generates effective MRs for 18 out of 23 methods (mutation score > 20%). Furthermore, it can increase \n<sc>Randoop</small>\n's fault detection capability in 7 out of 23 methods, and \n<sc>Evosuite</small>\n's in 14 out of 23 methods. When compared with \n<sc>AutoMR</small>\n, a state-of-the-art MR generator, \n<sc>GenMorph</small>\n also outperformed its fault detection capability in 9 out of 10 methods.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":null,"pages":null},"PeriodicalIF":6.5000,"publicationDate":"2024-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"GenMorph: Automatically Generating Metamorphic Relations via Genetic Programming\",\"authors\":\"Jon Ayerdi;Valerio Terragni;Gunel Jahangirova;Aitor Arrieta;Paolo Tonella\",\"doi\":\"10.1109/TSE.2024.3407840\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Metamorphic testing is a popular approach that aims to alleviate the oracle problem in software testing. At the core of this approach are Metamorphic Relations (MRs), specifying properties that hold among multiple test inputs and corresponding outputs. Deriving MRs is mostly a manual activity, since their automated generation is a challenging and largely unexplored problem. This paper presents \\n<sc>GenMorph</small>\\n, a technique to automatically generate MRs for Java methods that involve inputs and outputs that are boolean, numerical, or ordered sequences. \\n<sc>GenMorph</small>\\n uses an evolutionary algorithm to search for \\n<italic>effective</i>\\n test oracles, i.e., oracles that trigger no false alarms and expose software faults in the method under test. The proposed search algorithm is guided by two fitness functions that measure the number of false alarms and the number of missed faults for the generated MRs. Our results show that \\n<sc>GenMorph</small>\\n generates effective MRs for 18 out of 23 methods (mutation score > 20%). Furthermore, it can increase \\n<sc>Randoop</small>\\n's fault detection capability in 7 out of 23 methods, and \\n<sc>Evosuite</small>\\n's in 14 out of 23 methods. When compared with \\n<sc>AutoMR</small>\\n, a state-of-the-art MR generator, \\n<sc>GenMorph</small>\\n also outperformed its fault detection capability in 9 out of 10 methods.\",\"PeriodicalId\":13324,\"journal\":{\"name\":\"IEEE Transactions on Software Engineering\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":6.5000,\"publicationDate\":\"2024-03-31\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Software Engineering\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10542726/\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, SOFTWARE ENGINEERING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Software Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10542726/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
GenMorph: Automatically Generating Metamorphic Relations via Genetic Programming
Metamorphic testing is a popular approach that aims to alleviate the oracle problem in software testing. At the core of this approach are Metamorphic Relations (MRs), specifying properties that hold among multiple test inputs and corresponding outputs. Deriving MRs is mostly a manual activity, since their automated generation is a challenging and largely unexplored problem. This paper presents
GenMorph
, a technique to automatically generate MRs for Java methods that involve inputs and outputs that are boolean, numerical, or ordered sequences.
GenMorph
uses an evolutionary algorithm to search for
effective
test oracles, i.e., oracles that trigger no false alarms and expose software faults in the method under test. The proposed search algorithm is guided by two fitness functions that measure the number of false alarms and the number of missed faults for the generated MRs. Our results show that
GenMorph
generates effective MRs for 18 out of 23 methods (mutation score > 20%). Furthermore, it can increase
Randoop
's fault detection capability in 7 out of 23 methods, and
Evosuite
's in 14 out of 23 methods. When compared with
AutoMR
, a state-of-the-art MR generator,
GenMorph
also outperformed its fault detection capability in 9 out of 10 methods.
期刊介绍:
IEEE Transactions on Software Engineering seeks contributions comprising well-defined theoretical results and empirical studies with potential impacts on software construction, analysis, or management. The scope of this Transactions extends from fundamental mechanisms to the development of principles and their application in specific environments. Specific topic areas include:
a) Development and maintenance methods and models: Techniques and principles for specifying, designing, and implementing software systems, encompassing notations and process models.
b) Assessment methods: Software tests, validation, reliability models, test and diagnosis procedures, software redundancy, design for error control, and measurements and evaluation of process and product aspects.
c) Software project management: Productivity factors, cost models, schedule and organizational issues, and standards.
d) Tools and environments: Specific tools, integrated tool environments, associated architectures, databases, and parallel and distributed processing issues.
e) System issues: Hardware-software trade-offs.
f) State-of-the-art surveys: Syntheses and comprehensive reviews of the historical development within specific areas of interest.