Yves Le Traon, T. Mouelhi, Franck Fleurey, B. Baudry
in this paper, we study an issue related to the abstraction level of a meta-model through the example of a model-driven approach for specifying, deploying and testing security policies in Java applications. The issue we focus on is the balance between a "generic" meta-model and the semantics we want to attach to it, which ahs to be precise enough. The goal of the original work was to present a full MDE process to check the consistency of a security policy and generate qualification criteria for the test cases testing the security mechanisms in the final code. The most original idea is that security policy is specified independently of the underlying access control language (OrBAC, RBAC, DAC or MAC). It is based on a generic security meta-model which can be used for early consistency checks in the security policy. We qualify the test cases that validate the security policy in the application with a fault injection technique, mutation applied to access control policies. In the empirical results on 3 case studies, we explore the advantages and limitations of the mutation operators and verification checks whose semantics is defined on the meta-model. The overall question we address is not the feasibility of the approach as shown in our previous work but the quality of a metamodel for test and verification purpose.
{"title":"Language-Specific vs. Language-Independent Approaches: Embedding Semantics on a Metamodel for Testing and Verifying Access Control Policies","authors":"Yves Le Traon, T. Mouelhi, Franck Fleurey, B. Baudry","doi":"10.1109/ICSTW.2010.67","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.67","url":null,"abstract":"in this paper, we study an issue related to the abstraction level of a meta-model through the example of a model-driven approach for specifying, deploying and testing security policies in Java applications. The issue we focus on is the balance between a \"generic\" meta-model and the semantics we want to attach to it, which ahs to be precise enough. The goal of the original work was to present a full MDE process to check the consistency of a security policy and generate qualification criteria for the test cases testing the security mechanisms in the final code. The most original idea is that security policy is specified independently of the underlying access control language (OrBAC, RBAC, DAC or MAC). It is based on a generic security meta-model which can be used for early consistency checks in the security policy. We qualify the test cases that validate the security policy in the application with a fault injection technique, mutation applied to access control policies. In the empirical results on 3 case studies, we explore the advantages and limitations of the mutation operators and verification checks whose semantics is defined on the meta-model. The overall question we address is not the feasibility of the approach as shown in our previous work but the quality of a metamodel for test and verification purpose.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128306198","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Antonia Estero-Botaro, F. Palomo-Lozano, I. Medina-Bulo
This work presents a quantitative evaluation of the quality of a set of mutation operators for WS-BPEL 2.0 (Web Services Business Process Execution Language), an OASIS standardized language for the composition of Web Services. A series of experiments is conducted, aiming to determine how selective operators are for the qualification of test cases and, more generally, the quality of the operators themselves. This includes a formalization of mutation testing defining the key concepts and measures employed. Three different compositions are considered and special attention is paid to the properties that suitable test-suites should enjoy.
{"title":"Quantitative Evaluation of Mutation Operators for WS-BPEL Compositions","authors":"Antonia Estero-Botaro, F. Palomo-Lozano, I. Medina-Bulo","doi":"10.1109/ICSTW.2010.36","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.36","url":null,"abstract":"This work presents a quantitative evaluation of the quality of a set of mutation operators for WS-BPEL 2.0 (Web Services Business Process Execution Language), an OASIS standardized language for the composition of Web Services. A series of experiments is conducted, aiming to determine how selective operators are for the qualification of test cases and, more generally, the quality of the operators themselves. This includes a formalization of mutation testing defining the key concepts and measures employed. Three different compositions are considered and special attention is paid to the properties that suitable test-suites should enjoy.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129366663","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
It is commonly accepted that strong typing is useful for revealing programmer errors and so the use of dynamically typed languages increases the importance of software testing. Mutation analysis is a demanding software testing criterion. Although mutation analysis has been applied to procedural languages, and object oriented languages, little work has been done on the mutation analysis of programs written in dynamically typed languages. Mutation analysis depends on the substitution and modification of program elements. In a strongly typed language, the declared type of the mutated element, a variable or operator, can be used to avoid generating type-incorrect substitutions or modifications. Ina dynamically typed language, this type information is not available and so a much greater range of mutations are potentially applicable but many of the resulting mutants are likely to be incompetent (too easily killed). This paper describes a mutation analysis method in which the definition of mutants is performed at run-time when type information is available. The type information can be used to avoid generating incompetent mutants.
{"title":"Type Sensitive Application of Mutation Operators for Dynamically Typed Programs","authors":"L. Bottaci","doi":"10.1109/ICSTW.2010.56","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.56","url":null,"abstract":"It is commonly accepted that strong typing is useful for revealing programmer errors and so the use of dynamically typed languages increases the importance of software testing. Mutation analysis is a demanding software testing criterion. Although mutation analysis has been applied to procedural languages, and object oriented languages, little work has been done on the mutation analysis of programs written in dynamically typed languages. Mutation analysis depends on the substitution and modification of program elements. In a strongly typed language, the declared type of the mutated element, a variable or operator, can be used to avoid generating type-incorrect substitutions or modifications. Ina dynamically typed language, this type information is not available and so a much greater range of mutations are potentially applicable but many of the resulting mutants are likely to be incompetent (too easily killed). This paper describes a mutation analysis method in which the definition of mutants is performed at run-time when type information is available. The type information can be used to avoid generating incompetent mutants.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129466301","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mutation testing is a powerful and flexible test technique. Traditional mutation testing makes a small change to the syntax of a description (usually a program) in order to create a mutant. A test set is considered to be good if it distinguishes between the original description and all of the (functionally nonequivalent) mutants. These mutants can be seen as representing potential small slips and thus mutation testing aims to produce a test set that is good at finding such slips. It has also been argued that a test set that finds such small changes is likely to find larger changes. This paper introduces a new approach to mutation testing, called semantic mutation testing. Rather than mutate the description, semantic mutation testing mutates the semantics of the language in which the description is written. The mutations of the semantics of the language represent possible misunderstandings of the description language and thus capture a different class of faults. Since the likely misunderstandings are highly context dependent, this context should be used to determine which semantic mutants should be produced. The approach is illustrated through examples with state charts and C code. In addition, a semantic mutation testing tool for C is proposed.
{"title":"Semantic Mutation Testing","authors":"J. A. Clark, Haitao Dan, R. Hierons","doi":"10.1109/icstw.2010.8","DOIUrl":"https://doi.org/10.1109/icstw.2010.8","url":null,"abstract":"Mutation testing is a powerful and flexible test technique. Traditional mutation testing makes a small change to the syntax of a description (usually a program) in order to create a mutant. A test set is considered to be good if it distinguishes between the original description and all of the (functionally nonequivalent) mutants. These mutants can be seen as representing potential small slips and thus mutation testing aims to produce a test set that is good at finding such slips. It has also been argued that a test set that finds such small changes is likely to find larger changes. This paper introduces a new approach to mutation testing, called semantic mutation testing. Rather than mutate the description, semantic mutation testing mutates the semantics of the language in which the description is written. The mutations of the semantics of the language represent possible misunderstandings of the description language and thus capture a different class of faults. Since the likely misunderstandings are highly context dependent, this context should be used to determine which semantic mutants should be produced. The approach is illustrated through examples with state charts and C code. In addition, a semantic mutation testing tool for C is proposed.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"9 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133906010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Test-driven development (TDD) is a software development practice that supposedly leads to better quality and fewer defects in code. TDD is a simple practice, but developers sometimes do not apply all the required steps correctly. This article presents some of the most common mistakes that programmers make when practicing TDD, identified by an online survey with 218 volunteer programmers. Some mistakes identified were: to forget the refactoring step, building complex test scenarios, and refactor another piece of code while working on a test. Some mistakes are frequently made by around 25% of programmers.
{"title":"Most Common Mistakes in Test-Driven Development Practice: Results from an Online Survey with Developers","authors":"M. Aniche, M. Gerosa","doi":"10.1109/ICSTW.2010.16","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.16","url":null,"abstract":"Test-driven development (TDD) is a software development practice that supposedly leads to better quality and fewer defects in code. TDD is a simple practice, but developers sometimes do not apply all the required steps correctly. This article presents some of the most common mistakes that programmers make when practicing TDD, identified by an online survey with 218 volunteer programmers. Some mistakes identified were: to forget the refactoring step, building complex test scenarios, and refactor another piece of code while working on a test. Some mistakes are frequently made by around 25% of programmers.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130695611","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ethar Elsaka, W. E. Moustafa, Bao-Ngoc Nguyen, A. Memon
Graphical user interfaces (GUIs) for today's applications are extremely large. Moreover, they provide many degrees of freedom to the end-user, thus allowing the user to perform a very large number of event sequences on the GUI. The large sizes and degrees of freedom create severe problems for GUI quality assurance, including GUI testing. In this paper, we leverage methods and measures from network analysis to analyze and study GUIs, with the goal of aiding GUI testing activities. We apply these methods and measures on the event-flow graph model of GUIs. Results of a case study show that "network centrality measures" are able to identify the most important events in the GUI as well as the most important sequences of events. These events and sequences are good candidates for test prioritization. In addition, the "betweenness clustering" method is able to partition the GUI into regions that can be tested separately.
{"title":"Using Methods & Measures from Network Analysis for GUI Testing","authors":"Ethar Elsaka, W. E. Moustafa, Bao-Ngoc Nguyen, A. Memon","doi":"10.1109/ICSTW.2010.61","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.61","url":null,"abstract":"Graphical user interfaces (GUIs) for today's applications are extremely large. Moreover, they provide many degrees of freedom to the end-user, thus allowing the user to perform a very large number of event sequences on the GUI. The large sizes and degrees of freedom create severe problems for GUI quality assurance, including GUI testing. In this paper, we leverage methods and measures from network analysis to analyze and study GUIs, with the goal of aiding GUI testing activities. We apply these methods and measures on the event-flow graph model of GUIs. Results of a case study show that \"network centrality measures\" are able to identify the most important events in the GUI as well as the most important sequences of events. These events and sequences are good candidates for test prioritization. In addition, the \"betweenness clustering\" method is able to partition the GUI into regions that can be tested separately.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116400058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The objective of this qualitative study was to explore and understand the conditions that influence software testing as an online service and elicit important research issues. Interviews were conducted with managers from eleven organizations. The study used qualitative grounded theory as its research method. The results indicate that the demand for software testing as an online service is on the rise and is influenced by conditions such as the level of domain knowledge needed to effectively test an application, flexibility and cost effectiveness as benefits, security and pricing as top requirements, cloud computing as the delivery mode and the need for software testers to hone their skills. Potential research areas suggested include application areas best suited for online software testing, pricing and handling of test data among others.
{"title":"Software Testing as an Online Service: Observations from Practice","authors":"L. Riungu, Ossi Taipale, K. Smolander","doi":"10.1109/ICSTW.2010.62","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.62","url":null,"abstract":"The objective of this qualitative study was to explore and understand the conditions that influence software testing as an online service and elicit important research issues. Interviews were conducted with managers from eleven organizations. The study used qualitative grounded theory as its research method. The results indicate that the demand for software testing as an online service is on the rise and is influenced by conditions such as the level of domain knowledge needed to effectively test an application, flexibility and cost effectiveness as benefits, security and pricing as top requirements, cloud computing as the delivery mode and the need for software testers to hone their skills. Potential research areas suggested include application areas best suited for online software testing, pricing and handling of test data among others.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129827121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents a technique to consider constraints on input signals of continuous systems when applying search-based testing. The signal constraints are described using a logic based on Signal Temporal Logic. We developed a distance-oriented evaluation technique for these constraints that provides an exact rating of the amount of constraint violation, thus allowing a ranking of the generated solutions in terms of constraint violation. An adaptive penalty function is then used to incorporate the evaluation results into the optimization. Finally, the overall method is shown to be capable of considering signal constraints appropriately when experimentally applied to search-based black-box testing of a MATLAB SIMULINK model of an automatic transmission controller.
{"title":"Considering Signal Constraints in Search-Based Testing of Continuous Systems","authors":"Benjamin Wilmes, Andreas Windisch","doi":"10.1109/ICSTW.2010.22","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.22","url":null,"abstract":"This paper presents a technique to consider constraints on input signals of continuous systems when applying search-based testing. The signal constraints are described using a logic based on Signal Temporal Logic. We developed a distance-oriented evaluation technique for these constraints that provides an exact rating of the amount of constraint violation, thus allowing a ranking of the generated solutions in terms of constraint violation. An adaptive penalty function is then used to incorporate the evaluation results into the optimization. Finally, the overall method is shown to be capable of considering signal constraints appropriately when experimentally applied to search-based black-box testing of a MATLAB SIMULINK model of an automatic transmission controller.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127228480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Modern attacks are being made against client side applications, such as web browsers, which most users use to surf and communicate on the internet. Client honeypots visit and interact with suspect web sites in order to detect and collect information about malware to protect users from malicious websites or to allow security professionals to investigate malicious content. This paper will present the idea of using web-based technology and integrating it with a client honeypot by building a low interaction client honeypot tool called Honeyware. It describes the benefits of Honeyware as well as the challenges of a low interaction client honeypot and provides some ideas for how these challenges could be overcome.
{"title":"Honeyware: A Web-Based Low Interaction Client Honeypot","authors":"Yaser Alosefer, O. Rana","doi":"10.1109/ICSTW.2010.41","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.41","url":null,"abstract":"Modern attacks are being made against client side applications, such as web browsers, which most users use to surf and communicate on the internet. Client honeypots visit and interact with suspect web sites in order to detect and collect information about malware to protect users from malicious websites or to allow security professionals to investigate malicious content. This paper will present the idea of using web-based technology and integrating it with a client honeypot by building a low interaction client honeypot tool called Honeyware. It describes the benefits of Honeyware as well as the challenges of a low interaction client honeypot and provides some ideas for how these challenges could be overcome.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130168483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Various mutation approximation techniques have been proposed in the literature in order to reduce the expenses of mutation. This paper presents results from an empirical study conducted for first and second order mutation testing strategies. Its scope is to evaluate the relative application cost and effectiveness of the different mutation strategies. The application cost was based: on the number of mutants, the equivalent ones and on the number of test cases needed to expose them by each strategy. Each strategy's effectiveness was evaluated by its ability to expose a set of seeded faults. The results indicate that on the one hand the first order mutation testing strategies can be in general more effective than the second order ones. On the other hand, the second order strategies can drastically decrease the number of the introduced equivalent mutants, generally forming a valid cost effective alternative to mutation testing.
{"title":"An Empirical Evaluation of the First and Second Order Mutation Testing Strategies","authors":"Mike Papadakis, N. Malevris","doi":"10.1109/ICSTW.2010.50","DOIUrl":"https://doi.org/10.1109/ICSTW.2010.50","url":null,"abstract":"Various mutation approximation techniques have been proposed in the literature in order to reduce the expenses of mutation. This paper presents results from an empirical study conducted for first and second order mutation testing strategies. Its scope is to evaluate the relative application cost and effectiveness of the different mutation strategies. The application cost was based: on the number of mutants, the equivalent ones and on the number of test cases needed to expose them by each strategy. Each strategy's effectiveness was evaluated by its ability to expose a set of seeded faults. The results indicate that on the one hand the first order mutation testing strategies can be in general more effective than the second order ones. On the other hand, the second order strategies can drastically decrease the number of the introduced equivalent mutants, generally forming a valid cost effective alternative to mutation testing.","PeriodicalId":117410,"journal":{"name":"2010 Third International Conference on Software Testing, Verification, and Validation Workshops","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125386072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：