J. T. de Souza, C. Maia, Fabricio Gomes de Freitas, Daniel Coutinho
This paper reports a comprehensive experimental study regarding the human competitiveness of search based software engineering (SBSE). The experiments were performed over four well-known SBSE problem formulations: next release problem, multi-objective next release problem, workgroup formation problem and the multi-objective test case selection problem. For each of these problems, two instances, with increasing sizes, were synthetically generated and solved by both metaheuristics and human subjects. A total of 63 professional software engineers participated in the experiment by solving some or all problem instances, producing together 128 responses. The comparison analysis strongly suggests that the results generated by search based software engineering can be said to be human competitive.
{"title":"The Human Competitiveness of Search Based Software Engineering","authors":"J. T. de Souza, C. Maia, Fabricio Gomes de Freitas, Daniel Coutinho","doi":"10.1109/SSBSE.2010.25","DOIUrl":"https://doi.org/10.1109/SSBSE.2010.25","url":null,"abstract":"This paper reports a comprehensive experimental study regarding the human competitiveness of search based software engineering (SBSE). The experiments were performed over four well-known SBSE problem formulations: next release problem, multi-objective next release problem, workgroup formation problem and the multi-objective test case selection problem. For each of these problems, two instances, with increasing sizes, were synthetically generated and solved by both metaheuristics and human subjects. A total of 63 professional software engineers participated in the experiment by solving some or all problem instances, producing together 128 responses. The comparison analysis strongly suggests that the results generated by search based software engineering can be said to be human competitive.","PeriodicalId":309806,"journal":{"name":"2nd International Symposium on Search Based Software Engineering","volume":"18 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132273658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recent results in Search-Based Testing show that the relatively simple Alternating Variable hill climbing method outperforms Evolutionary Testing (ET) for many programs. For ET to perform well in covering an individual branch, a program must have a certain structure that gives rise to a fitness landscape that the crossover operator can exploit. This paper presents theoretical and empirical investigations into the types of program structure that result in such landscapes. The studies show that crossover lends itself to programs that process large data structures or have an internal state that is reached over a series of repeated function or method calls. The empirical study also investigates the type of crossover which works most efficiently for different program structures. It further compares the results obtained by ET with those obtained for different variants of hill climbing algorithm, which are found to be effective for many structures considered favourable to crossover, with the exception of structures with landscapes containing entrapping local optima.
{"title":"How Does Program Structure Impact the Effectiveness of the Crossover Operator in Evolutionary Testing?","authors":"Phil McMinn","doi":"10.1109/SSBSE.2010.11","DOIUrl":"https://doi.org/10.1109/SSBSE.2010.11","url":null,"abstract":"Recent results in Search-Based Testing show that the relatively simple Alternating Variable hill climbing method outperforms Evolutionary Testing (ET) for many programs. For ET to perform well in covering an individual branch, a program must have a certain structure that gives rise to a fitness landscape that the crossover operator can exploit. This paper presents theoretical and empirical investigations into the types of program structure that result in such landscapes. The studies show that crossover lends itself to programs that process large data structures or have an internal state that is reached over a series of repeated function or method calls. The empirical study also investigates the type of crossover which works most efficiently for different program structures. It further compares the results obtained by ET with those obtained for different variants of hill climbing algorithm, which are found to be effective for many structures considered favourable to crossover, with the exception of structures with landscapes containing entrapping local optima.","PeriodicalId":309806,"journal":{"name":"2nd International Symposium on Search Based Software Engineering","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124250396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Software testing is an expensive task that significantly contributes to the total cost of a software development project. Among the many strategies available to test a software project, the creation of automated test cases that can be enacted after building a release or resolving a defect is increasingly used in the industry. However, certain defects found in the system operation may block major business operations. These critical defects are sometimes resolved directly in the production environment under such a restricted deadline that there is not enough time to run the complete set of automated test cases upon the patched version of the software. Declining to run the test case suite allows a quicker release of the software to production, but also allows other defects to be introduced into the system. This paper presents a heuristic approach to select test cases that might support emergency changes aiming to maximize the coverage and diversity of the testing activity under a strict time constraint and given the priority of the features that were changed.
{"title":"Test Case Selection Method for Emergency Changes","authors":"Fábio de Almeida Farzat","doi":"10.1109/SSBSE.2010.13","DOIUrl":"https://doi.org/10.1109/SSBSE.2010.13","url":null,"abstract":"Software testing is an expensive task that significantly contributes to the total cost of a software development project. Among the many strategies available to test a software project, the creation of automated test cases that can be enacted after building a release or resolving a defect is increasingly used in the industry. However, certain defects found in the system operation may block major business operations. These critical defects are sometimes resolved directly in the production environment under such a restricted deadline that there is not enough time to run the complete set of automated test cases upon the patched version of the software. Declining to run the test case suite allows a quicker release of the software to production, but also allows other defects to be introduced into the system. This paper presents a heuristic approach to select test cases that might support emergency changes aiming to maximize the coverage and diversity of the testing activity under a strict time constraint and given the priority of the features that were changed.","PeriodicalId":309806,"journal":{"name":"2nd International Symposium on Search Based Software Engineering","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122787212","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The order in which requirements are implemented in a system affects the value delivered to the final users in the successive releases of the system. Requirements prioritization aims at ranking the requirements so as to trade off user priorities and implementation constraints, such as technical dependencies among requirements and necessarily limited resources allocated to the project. Requirement analysts possess relevant knowledge about the relative importance of requirements. We use an Interactive Genetic Algorithm to produce a requirement ordering which complies with the existing priorities, satisfies the technical constraints and takes into account the relative preferences elicited from the user. On a real case study, we show that this approach improves non interactive optimization, ignoring the elicited preferences, and that it can handle a number of requirements which is otherwise problematic for state of the art techniques.
{"title":"Using Interactive GA for Requirements Prioritization","authors":"P. Tonella, A. Susi, Francis Palma","doi":"10.1109/SSBSE.2010.17","DOIUrl":"https://doi.org/10.1109/SSBSE.2010.17","url":null,"abstract":"The order in which requirements are implemented in a system affects the value delivered to the final users in the successive releases of the system. Requirements prioritization aims at ranking the requirements so as to trade off user priorities and implementation constraints, such as technical dependencies among requirements and necessarily limited resources allocated to the project. Requirement analysts possess relevant knowledge about the relative importance of requirements. We use an Interactive Genetic Algorithm to produce a requirement ordering which complies with the existing priorities, satisfies the technical constraints and takes into account the relative preferences elicited from the user. On a real case study, we show that this approach improves non interactive optimization, ignoring the elicited preferences, and that it can handle a number of requirements which is otherwise problematic for state of the art techniques.","PeriodicalId":309806,"journal":{"name":"2nd International Symposium on Search Based Software Engineering","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129361511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Multi-Objective Set Cover problem forms the basis of many optimisation problems in software testing because the concept of code coverage is based on the set theory. This paper presents Mask-Coding, a novel representation of solutions for set cover optimisation problems that explores the problem space rather than the solution space. The new representation is empirically evaluated with set cover problems formulated from real code coverage data. The results show that Mask-Coding representation can improve both the convergence and diversity of the Pareto-efficient solution set of the multi-objective set cover optimisation.
{"title":"A Novel Mask-Coding Representation for Set Cover Problems with Applications in Test Suite Minimisation","authors":"S. Yoo","doi":"10.1109/SSBSE.2010.12","DOIUrl":"https://doi.org/10.1109/SSBSE.2010.12","url":null,"abstract":"Multi-Objective Set Cover problem forms the basis of many optimisation problems in software testing because the concept of code coverage is based on the set theory. This paper presents Mask-Coding, a novel representation of solutions for set cover optimisation problems that explores the problem space rather than the solution space. The new representation is empirically evaluated with set cover problems formulated from real code coverage data. The results show that Mask-Coding representation can improve both the convergence and diversity of the Pareto-efficient solution set of the multi-objective set cover optimisation.","PeriodicalId":309806,"journal":{"name":"2nd International Symposium on Search Based Software Engineering","volume":"134 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132159990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Despite the large number of publications on Search--Based Software Testing (SBST), there remain few publicly available tools. This paper introduces AUSTIN, a publicly available SBST tool for the C language. The paper validates the tool with an empirical study of its effectiveness and efficiency in achieving branch coverage compared to random testing and the Evolutionary Testing Framework (ETF), which is used in-house by Daimler and others for Evolutionary Testing. The programs used in the study consist of eight non--trivial, real-world C functions drawn from three embedded automotive software modules. For the majority of the functions, AUSTIN is at least as effective (in terms of achieved branch coverage) as the ETF, and is considerably more efficient.
{"title":"AUSTIN: A Tool for Search Based Software Testing for the C Language and Its Evaluation on Deployed Automotive Systems","authors":"Kiran Lakhotia, M. Harman, Hamilton Gross","doi":"10.1109/SSBSE.2010.21","DOIUrl":"https://doi.org/10.1109/SSBSE.2010.21","url":null,"abstract":"Despite the large number of publications on Search--Based Software Testing (SBST), there remain few publicly available tools. This paper introduces AUSTIN, a publicly available SBST tool for the C language. The paper validates the tool with an empirical study of its effectiveness and efficiency in achieving branch coverage compared to random testing and the Evolutionary Testing Framework (ETF), which is used in-house by Daimler and others for Evolutionary Testing. The programs used in the study consist of eight non--trivial, real-world C functions drawn from three embedded automotive software modules. For the majority of the functions, AUSTIN is at least as effective (in terms of achieved branch coverage) as the ETF, and is considerably more efficient.","PeriodicalId":309806,"journal":{"name":"2nd International Symposium on Search Based Software Engineering","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115019471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A large percentage of the cost of rework can be avoided by finding more faults earlier in a software testing process. Therefore, determination of which software testing phases to focus improvements work on, has considerable industrial interest. This paper evaluates the use of five different techniques, namely particle swarm optimization based artificial neural networks (PSO-ANN), artificial immune recognition systems (AIRS), gene expression programming (GEP), genetic programming (GP) and multiple regression (MR), for predicting the number of faults slipping through unit, function, integration and system testing phases. The objective is to quantify improvement potential in different testing phases by striving towards finding the right faults in the right phase. We have conducted an empirical study of two large projects from a telecommunication company developing mobile platforms and wireless semiconductors. The results are compared using simple residuals, goodness of fit and absolute relative error measures. They indicate that the four search-based techniques (PSO-ANN, AIRS, GEP, GP) perform better than multiple regression for predicting the fault-slip-through for each of the four testing phases. At the unit and function testing phases, AIRS and PSO-ANN performed better while GP performed better at integration and system testing phases. The study concludes that a variety of search-based techniques are applicable for predicting the improvement potential in different testing phases with GP showing more consistent performance across two of the four test phases.
{"title":"Search-based Prediction of Fault-slip-through in Large Software Projects","authors":"W. Afzal, R. Torkar, R. Feldt, Greger Wikstrand","doi":"10.1109/SSBSE.2010.19","DOIUrl":"https://doi.org/10.1109/SSBSE.2010.19","url":null,"abstract":"A large percentage of the cost of rework can be avoided by finding more faults earlier in a software testing process. Therefore, determination of which software testing phases to focus improvements work on, has considerable industrial interest. This paper evaluates the use of five different techniques, namely particle swarm optimization based artificial neural networks (PSO-ANN), artificial immune recognition systems (AIRS), gene expression programming (GEP), genetic programming (GP) and multiple regression (MR), for predicting the number of faults slipping through unit, function, integration and system testing phases. The objective is to quantify improvement potential in different testing phases by striving towards finding the right faults in the right phase. We have conducted an empirical study of two large projects from a telecommunication company developing mobile platforms and wireless semiconductors. The results are compared using simple residuals, goodness of fit and absolute relative error measures. They indicate that the four search-based techniques (PSO-ANN, AIRS, GEP, GP) perform better than multiple regression for predicting the fault-slip-through for each of the four testing phases. At the unit and function testing phases, AIRS and PSO-ANN performed better while GP performed better at integration and system testing phases. The study concludes that a variety of search-based techniques are applicable for predicting the improvement potential in different testing phases with GP showing more consistent performance across two of the four test phases.","PeriodicalId":309806,"journal":{"name":"2nd International Symposium on Search Based Software Engineering","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115892904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：