Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069055
D. Talby
One approach to applying keyword driven testing in a model-driven development environment is by defining a domain specific language for test cases. The toolset then provides test editors, versioning, validation, reporting and hyperlinks across models - in addition to enabling automated test execution. This case study evaluates the effectiveness of such a solution as perceived by two teams of professional testers, who used it to test several products over a two year period. The results suggest that in addition to the expected benefits of automation, the solution reduces the time and effort required to write tests, maintain tests and plan the test authoring and execution efforts - at the expense of requiring longer training and a higher bar for recruiting testers.
{"title":"The perceived value of authoring and automating acceptance tests using a model driven development toolset","authors":"D. Talby","doi":"10.1109/IWAST.2009.5069055","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069055","url":null,"abstract":"One approach to applying keyword driven testing in a model-driven development environment is by defining a domain specific language for test cases. The toolset then provides test editors, versioning, validation, reporting and hyperlinks across models - in addition to enabling automated test execution. This case study evaluates the effectiveness of such a solution as perceived by two teams of professional testers, who used it to test several products over a two year period. The results suggest that in addition to the expected benefits of automation, the solution reduces the time and effort required to write tests, maintain tests and plan the test authoring and execution efforts - at the expense of requiring longer training and a higher bar for recruiting testers.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129963371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069035
A. C. Dias-Neto, G. Travassos
The technical literature regarding Model-based Testing (MBT) has several techniques with different characteristics and goals available to be applied in software projects. Besides the lack of information regarding these techniques, they could be applied together in a software project aiming at improving the testing coverage. However, this decision needs to be carefully analyzed to avoid loss of resources in a software project. Based on this scenario, this paper proposes an approach with the purpose of supporting the unique or combined selection of MBT techniques for a given software project considering two aspects: the adequacy level between MBT techniques and the software project characteristics and impact of more than one MBT technique in some testing process variables. At the end, preliminary results of an experimental evaluation are presented.
{"title":"Porantim: An approach to support the combination and selection of Model-based Testing techniques","authors":"A. C. Dias-Neto, G. Travassos","doi":"10.1109/IWAST.2009.5069035","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069035","url":null,"abstract":"The technical literature regarding Model-based Testing (MBT) has several techniques with different characteristics and goals available to be applied in software projects. Besides the lack of information regarding these techniques, they could be applied together in a software project aiming at improving the testing coverage. However, this decision needs to be carefully analyzed to avoid loss of resources in a software project. Based on this scenario, this paper proposes an approach with the purpose of supporting the unique or combined selection of MBT techniques for a given software project considering two aspects: the adequacy level between MBT techniques and the software project characteristics and impact of more than one MBT technique in some testing process variables. At the end, preliminary results of an experimental evaluation are presented.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122791875","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069046
L. Jin
This paper describes the construction of an automated test framework for search engine and reports our current practice in establishing a process for test automation. The paper presents the technical solutions that overcome the difficulties in search engine testing, which involve large volume of data, complicated ranking rules and randomization in ordering search results. One reason that many test automation efforts failed in industry is because automated scripts are not developed and structured for future maintenance and evolution. The test framework enables effective and efficient reuse and maintenance of test scripts. The paper also discusses issues related to the process of test automation. They are equally as important as overcoming technical barriers. All of these discussed above have been applied to the automated functional testing of a commercial search engine.
{"title":"Automated functional testing of search engine","authors":"L. Jin","doi":"10.1109/IWAST.2009.5069046","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069046","url":null,"abstract":"This paper describes the construction of an automated test framework for search engine and reports our current practice in establishing a process for test automation. The paper presents the technical solutions that overcome the difficulties in search engine testing, which involve large volume of data, complicated ranking rules and randomization in ordering search results. One reason that many test automation efforts failed in industry is because automated scripts are not developed and structured for future maintenance and evolution. The test framework enables effective and efficient reuse and maintenance of test scripts. The paper also discusses issues related to the process of test automation. They are equally as important as overcoming technical barriers. All of these discussed above have been applied to the automated functional testing of a commercial search engine.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126768047","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069041
T. Clark
Model driven functional system testing generates test scenarios from behavioural and structural models. In order to autmatically generate tests, conditions such as invariants and pre-/post-conditions must be precisely defined. UML provides the Object Constraint Language (OCL) for this purpose; however OCL expressions can become very complex. This paper describes an approach that allows many commonly found OCL patterns to be expressed as snapshot patterns that correspond directly to the information model diagrams. Behaviour is constructed as chains of snapshots, or filmstrips. Snapshots and filmstrips are as expressive as UML behaviour models and OCL but it is argued that they are more accessible and more modular.
{"title":"Model based functional testing using pattern directed filmstrips","authors":"T. Clark","doi":"10.1109/IWAST.2009.5069041","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069041","url":null,"abstract":"Model driven functional system testing generates test scenarios from behavioural and structural models. In order to autmatically generate tests, conditions such as invariants and pre-/post-conditions must be precisely defined. UML provides the Object Constraint Language (OCL) for this purpose; however OCL expressions can become very complex. This paper describes an approach that allows many commonly found OCL patterns to be expressed as snapshot patterns that correspond directly to the information model diagrams. Behaviour is constructed as chains of snapshots, or filmstrips. Snapshots and filmstrips are as expressive as UML behaviour models and OCL but it is argued that they are more accessible and more modular.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128650003","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069048
Zhong Jie Li, Jun Zhu, Liang-Jie Zhang, Naomi M. Mitsumori
This paper proposes a method for Web services test case generation, which is centered on a practical test data generation framework that has higher probability to penetrate the service implementation logic. This presented framework leverages both the information contained in the WSDL/XSD files and the information provided by testers as well as enables testers to customize the fields to be tested, field constraints and data generation rules. The proposed method can generate test cases that are effective in detecting defects and efficient in reducing test time. This paper also presents preliminary empirical evidence to illustrate the value of the method.
{"title":"Towards a practical and effective method for Web services test case generation","authors":"Zhong Jie Li, Jun Zhu, Liang-Jie Zhang, Naomi M. Mitsumori","doi":"10.1109/IWAST.2009.5069048","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069048","url":null,"abstract":"This paper proposes a method for Web services test case generation, which is centered on a practical test data generation framework that has higher probability to penetrate the service implementation logic. This presented framework leverages both the information contained in the WSDL/XSD files and the information provided by testers as well as enables testers to customize the fields to be tested, field constraints and data generation rules. The proposed method can generate test cases that are effective in detecting defects and efficient in reducing test time. This paper also presents preliminary empirical evidence to illustrate the value of the method.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127161958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069052
P. Bunyakiati, A. Finkelstein
In ICSE'08 we demonstrated the Java UML Lightweight Enumerator (JULE) tool, which supports compliance test generation from modeling standards specifications. When employed in our framework for software tool certification, JULE provides a powerful technology to enumerate a set of test cases that exhaustively test a modeling tool. JULE avoids combinatorial explosion by generating test cases only up to non-isomorphism. In this paper, a case study presented is an experiment on the use of a test suite generated from JULE to assess the compliance of an open source software tool - ArgoUML. This case study illustrates how ArgoUML is tested and reveals some previously unknown non-compliance issues. The case study highlights how software modeling tools can be tested for standards compliance and how test results can be analyzed to diagnose the causes of non-compliance in a software tool.
{"title":"The compliance testing of software tools with respect to the UML standards specification - The ArgoUML case study","authors":"P. Bunyakiati, A. Finkelstein","doi":"10.1109/IWAST.2009.5069052","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069052","url":null,"abstract":"In ICSE'08 we demonstrated the Java UML Lightweight Enumerator (JULE) tool, which supports compliance test generation from modeling standards specifications. When employed in our framework for software tool certification, JULE provides a powerful technology to enumerate a set of test cases that exhaustively test a modeling tool. JULE avoids combinatorial explosion by generating test cases only up to non-isomorphism. In this paper, a case study presented is an experiment on the use of a test suite generated from JULE to assess the compliance of an open source software tool - ArgoUML. This case study illustrates how ArgoUML is tested and reveals some previously unknown non-compliance issues. The case study highlights how software modeling tools can be tested for standards compliance and how test results can be analyzed to diagnose the causes of non-compliance in a software tool.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114612314","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069045
Nicky Williams, M. Roger
Under certain conditions, the worst-case execution time (WCET) of a function can be found by measuring the effective execution time for each feasible execution path. Automatic generation of test inputs can help make this approach more feasible. To reduce the number of tests, we define two partial orders on the execution paths of the program under test. Under further conditions, these partial orders represent the relation between the execution times of the paths. We explain how we modified the strategy of the PathCrawler structural test-case generation tool to generate as few tests as possible for paths which are not maximal in these partial orders, whilst ensuring that the WCET is exhibited by at least one case in the set. The techniques used could also serve in the implementation of other test generation strategies which have nothing to do with WCET.
{"title":"Test generation strategies to measure worst-case execution time","authors":"Nicky Williams, M. Roger","doi":"10.1109/IWAST.2009.5069045","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069045","url":null,"abstract":"Under certain conditions, the worst-case execution time (WCET) of a function can be found by measuring the effective execution time for each feasible execution path. Automatic generation of test inputs can help make this approach more feasible. To reduce the number of tests, we define two partial orders on the execution paths of the program under test. Under further conditions, these partial orders represent the relation between the execution times of the paths. We explain how we modified the strategy of the PathCrawler structural test-case generation tool to generate as few tests as possible for paths which are not maximal in these partial orders, whilst ensuring that the WCET is exhibited by at least one case in the set. The techniques used could also serve in the implementation of other test generation strategies which have nothing to do with WCET.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117118207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069043
Bernard Botella, Mickaël Delahaye, S. Ha, N. Kosmatov, Patricia Mouy, M. Roger, Nicky Williams
Structural testing is widely used in industrial verification processes of critical software. This report presents PathCrawler, a structural test generation tool that may be used to automate this activity, and several evaluation criteria of automatic test generation tools for C programs. These criteria correspond to the issues identified during our ongoing experience in the development of PathCrawler and its application to industrial software. They include issues arising for some specific types of software. Some of them are still difficult open problems. Others are (partially) solved, and the solution adopted in PathCrawler is discussed. We believe that these criteria must be satisfied in order for the automation of structural testing to become an industrial reality.
{"title":"Automating structural testing of C programs: Experience with PathCrawler","authors":"Bernard Botella, Mickaël Delahaye, S. Ha, N. Kosmatov, Patricia Mouy, M. Roger, Nicky Williams","doi":"10.1109/IWAST.2009.5069043","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069043","url":null,"abstract":"Structural testing is widely used in industrial verification processes of critical software. This report presents PathCrawler, a structural test generation tool that may be used to automate this activity, and several evaluation criteria of automatic test generation tools for C programs. These criteria correspond to the issues identified during our ongoing experience in the development of PathCrawler and its application to industrial software. They include issues arising for some specific types of software. Some of them are still difficult open problems. Others are (partially) solved, and the solution adopted in PathCrawler is discussed. We believe that these criteria must be satisfied in order for the automation of structural testing to become an industrial reality.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122824218","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069042
Aaron Marback, Hyunsook Do, Ke He, Samuel Kondamarri, Dianxiang Xu
Software security issues have been a major concern to the cyberspace community, so a great deal of research on security testing has been performed, and various security testing techniques have been developed. Most of these techniques, however, have focused on testing software systems after their implementation is completed. To build secure and dependable software systems in a cost-effective way, however, it is necessary to put more effort upfront during the software development life cycle. In this paper, we provided a security testing approach that derives test cases from design-level artifacts. The security testing approach we consider consists of four activities: building threat trees from threat modeling; generating security tests from threat trees; generating test inputs including valid and invalid inputs; and assigning input values to parameters. We also conducted an empirical study to show feasibility of our approach.
{"title":"Security test generation using threat trees","authors":"Aaron Marback, Hyunsook Do, Ke He, Samuel Kondamarri, Dianxiang Xu","doi":"10.1109/IWAST.2009.5069042","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069042","url":null,"abstract":"Software security issues have been a major concern to the cyberspace community, so a great deal of research on security testing has been performed, and various security testing techniques have been developed. Most of these techniques, however, have focused on testing software systems after their implementation is completed. To build secure and dependable software systems in a cost-effective way, however, it is necessary to put more effort upfront during the software development life cycle. In this paper, we provided a security testing approach that derives test cases from design-level artifacts. The security testing approach we consider consists of four activities: building threat trees from threat modeling; generating security tests from threat trees; generating test inputs including valid and invalid inputs; and assigning input values to parameters. We also conducted an empirical study to show feasibility of our approach.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"107 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121820562","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-05-18DOI: 10.1109/IWAST.2009.5069054
Madhuri R. Marri, Tao Xie, N. Tillmann, J. D. Halleux, Wolfram Schulte
Unit testing is a technique of testing a single unit of a program in isolation. The testability of the unit under test can be reduced when the unit interacts with its environment. The construction of high-covering unit tests and their execution require appropriate interactions with the environment such as a file system or database. To help set up the required environment, developers can use mock objects to simulate the behavior of the environment. In this paper, we present an empirical study to analyze the use of mock objects to test file-system-dependent software. We use a mock object of the FileSystem API provided with the Pex automatic testing tool in our study. We share our insights gained on the benefits of using mock objects in unit testing and discuss the faced challenges.
{"title":"An empirical study of testing file-system-dependent software with mock objects","authors":"Madhuri R. Marri, Tao Xie, N. Tillmann, J. D. Halleux, Wolfram Schulte","doi":"10.1109/IWAST.2009.5069054","DOIUrl":"https://doi.org/10.1109/IWAST.2009.5069054","url":null,"abstract":"Unit testing is a technique of testing a single unit of a program in isolation. The testability of the unit under test can be reduced when the unit interacts with its environment. The construction of high-covering unit tests and their execution require appropriate interactions with the environment such as a file system or database. To help set up the required environment, developers can use mock objects to simulate the behavior of the environment. In this paper, we present an empirical study to analyze the use of mock objects to test file-system-dependent software. We use a mock object of the FileSystem API provided with the Pex automatic testing tool in our study. We share our insights gained on the benefits of using mock objects in unit testing and discuss the faced challenges.","PeriodicalId":401585,"journal":{"name":"2009 ICSE Workshop on Automation of Software Test","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130636193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：