Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.25
T. Hartmann
In this paper, the author explains a new approach of model based end-to-end chain testing using scenarios with original and simulated equipment. The first goal is to automatically derive test data and test cases from the model, which is defined by a domain specific language. Several solvers can be attached to the conversion to quickly create a wide variety of stimuli for the system(s) under test. Furthermore, the system under test can be stimulated by either original equipment - which is connected to the test bench - or the test bench can simulate equipment and create inputs for the tested systems. Any mixture of simulated and original equipment is possible and can be changed on the fly. In the end, the results from the system under test are collected. These results can then be displayed back in the model. This method is currently used and improved in the project "E-Cab" in which the author is involved. Passengers travelling by plane are in the focus of this project. Complete services and service chains - from the booking at home up to leaving the destination airport - are created and used by many systems communicating with each other. The author expects advantages from testing these end-to-end chains with this approach.
{"title":"Model Based Testing of End-to-End Chains Using Domain Specific Languages","authors":"T. Hartmann","doi":"10.1109/TAICPART.2009.25","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.25","url":null,"abstract":"In this paper, the author explains a new approach of model based end-to-end chain testing using scenarios with original and simulated equipment. The first goal is to automatically derive test data and test cases from the model, which is defined by a domain specific language. Several solvers can be attached to the conversion to quickly create a wide variety of stimuli for the system(s) under test. Furthermore, the system under test can be stimulated by either original equipment - which is connected to the test bench - or the test bench can simulate equipment and create inputs for the tested systems. Any mixture of simulated and original equipment is possible and can be changed on the fly. In the end, the results from the system under test are collected. These results can then be displayed back in the model. This method is currently used and improved in the project \"E-Cab\" in which the author is involved. Passengers travelling by plane are in the focus of this project. Complete services and service chains - from the booking at home up to leaving the destination airport - are created and used by many systems communicating with each other. The author expects advantages from testing these end-to-end chains with this approach.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117138048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.37
B. Peischl, N. Riaz, F. Wotawa
In this article we briefly state the idea behind modelbased diagnosis and its application to debugging RTL (Register Transfer Level) Verilog designs. In providing a debugging model for the Verilog HDL (Hardware Description Language) we rely on a specific abstraction (trace semantics) that captures solely quiescent states of the design. In this vein we manage to overcome the inherent complexity issues of event-based Verilog without relying on specific fault models. To leverage test patterns for design error localization we propose the filtering approach and relate it to the concept of Ackermann constraints. Notably, our empirical results demonstrate that our novel technique considerably increases the diagnosis resolution even under presence of only a couple of test cases. The article outlines a case study comprising several circuits, where the proposed technique allowed one for excluding 95 per cent of the Verilog code from being faulty by merely considering a couple of test cases.
在本文中，我们简要介绍了基于模型的诊断及其在调试RTL (Register Transfer Level) Verilog设计中的应用。在为Verilog HDL(硬件描述语言)提供调试模型时，我们依赖于一个特定的抽象(跟踪语义)，它只捕获设计的静态状态。在这种情况下，我们设法克服了基于事件的Verilog固有的复杂性问题，而不依赖于特定的故障模型。为了利用测试模式进行设计错误定位，我们提出了过滤方法，并将其与Ackermann约束的概念联系起来。值得注意的是，我们的经验结果表明，即使在只有几个测试用例的情况下，我们的新技术也大大提高了诊断分辨率。这篇文章概述了一个包含几个电路的案例研究，其中所提出的技术允许通过仅仅考虑几个测试用例来排除95%的Verilog代码的错误。
{"title":"Test Patterns for Verilog Design Error Localization","authors":"B. Peischl, N. Riaz, F. Wotawa","doi":"10.1109/TAICPART.2009.37","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.37","url":null,"abstract":"In this article we briefly state the idea behind modelbased diagnosis and its application to debugging RTL (Register Transfer Level) Verilog designs. In providing a debugging model for the Verilog HDL (Hardware Description Language) we rely on a specific abstraction (trace semantics) that captures solely quiescent states of the design. In this vein we manage to overcome the inherent complexity issues of event-based Verilog without relying on specific fault models. To leverage test patterns for design error localization we propose the filtering approach and relate it to the concept of Ackermann constraints. Notably, our empirical results demonstrate that our novel technique considerably increases the diagnosis resolution even under presence of only a couple of test cases. The article outlines a case study comprising several circuits, where the proposed technique allowed one for excluding 95 per cent of the Verilog code from being faulty by merely considering a couple of test cases.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"138 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123282876","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.27
Dae S. Kim-Park, C. Riva, J. Tuya
A partial test oracle is proposed to verify the actual outputs in access testing on XML data. The considered software under test is a query program which receives as input an XML document obtained from an XML repository of any kind, and produces XML data as output. To deal with the actual outputs from this testing process, the partial oracle evaluates the correctness of the test executions according to: (1) a loose specification provided by the tester, and (2) a set of predefined constraints that describe invariant properties of the expected outputs. By means of the loose specification, the oracle can particularize the constraints to the concrete behaviour of the query program to test. This approach enables the oracle to give automatically a response about the correctness of the program under test with a certain precision at feasible cost. To illustrate the usefulness of the approach a case study is presented.
{"title":"A Partial Test Oracle for XML Query Testing","authors":"Dae S. Kim-Park, C. Riva, J. Tuya","doi":"10.1109/TAICPART.2009.27","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.27","url":null,"abstract":"A partial test oracle is proposed to verify the actual outputs in access testing on XML data. The considered software under test is a query program which receives as input an XML document obtained from an XML repository of any kind, and produces XML data as output. To deal with the actual outputs from this testing process, the partial oracle evaluates the correctness of the test executions according to: (1) a loose specification provided by the tester, and (2) a set of predefined constraints that describe invariant properties of the expected outputs. By means of the loose specification, the oracle can particularize the constraints to the concrete behaviour of the query program to test. This approach enables the oracle to give automatically a response about the correctness of the program under test with a certain precision at feasible cost. To illustrate the usefulness of the approach a case study is presented.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122961276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.29
N. Hashish, L. Bottaci
Unit tests are a valuable part of the testing phase but they can be repetitious and difficult to maintain. This paper argues that a generalised unit test can concisely express a functional requirement that would otherwise be expressed using several similar unit tests. Initial ideas are presented for how large sets of existing unit tests can be used to inform the design of language constructs for writing generalised unit tests and for how automatic test data generation can be used in conjunction with generalised unit tests.
{"title":"Language Constructs for Generalising Unit Tests: Research Proposal","authors":"N. Hashish, L. Bottaci","doi":"10.1109/TAICPART.2009.29","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.29","url":null,"abstract":"Unit tests are a valuable part of the testing phase but they can be repetitious and difficult to maintain. This paper argues that a generalised unit test can concisely express a functional requirement that would otherwise be expressed using several similar unit tests. Initial ideas are presented for how large sets of existing unit tests can be used to inform the design of language constructs for writing generalised unit tests and for how automatic test data generation can be used in conjunction with generalised unit tests.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129744809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.35
D. Hoffman, Lewis Sobotkiewicz, Hong-Yi Wang, P. Strooper, Gary Bazdell, B. Stevens
Covering arrays and context-free grammars have seen extensive use in software test generation. A covering-array algorithm takes a list of domains and generates a subset of the cartesian product of the domains. A grammar-based test generation (GBTG) algorithm takes a grammar G and generates a subset of the language accepted by G. Covering arrays and GBTG are usually applied independently. We show that CFG rules and covering-array specifications can be freely intermixed, with precise, intuitive semantics and efficient generation. We present a novel approach for ``tagging'' grammars with specifications for mixed-strength covering arrays, a generalization of conventional covering arrays. We have developed algorithms for test generation and implemented a tool for generating test cases from tagged grammars.
{"title":"Test Generation with Context Free Grammars and Covering Arrays","authors":"D. Hoffman, Lewis Sobotkiewicz, Hong-Yi Wang, P. Strooper, Gary Bazdell, B. Stevens","doi":"10.1109/TAICPART.2009.35","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.35","url":null,"abstract":"Covering arrays and context-free grammars have seen extensive use in software test generation. A covering-array algorithm takes a list of domains and generates a subset of the cartesian product of the domains. A grammar-based test generation (GBTG) algorithm takes a grammar G and generates a subset of the language accepted by G. Covering arrays and GBTG are usually applied independently. We show that CFG rules and covering-array specifications can be freely intermixed, with precise, intuitive semantics and efficient generation. We present a novel approach for ``tagging'' grammars with specifications for mixed-strength covering arrays, a generalization of conventional covering arrays. We have developed algorithms for test generation and implemented a tool for generating test cases from tagged grammars.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123709579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.20
G. Fraser, A. Gargantini
Test case generation can be represented as a model checking problem, such that model checking tools automatically generate test cases. This has previously been applied to testing techniques such as coverage criteria, combinatorial testing, mutation testing, or requirements testing, and further criteria can be used similarly. However, little comparison between the existing techniques has been done to date, making it difficult to choose a technique. In this paper we define existing and new criteria in a common framework, and evaluate and compare them on a set of realistic specifications. Part of our findings is that because testing with model checkers represents the test case generation problem in a very flexible way best results can be achieved by combining several techniques from different categories. A best effort approach where test cases are only created for uncovered test requirements can create relatively small test suites that cover many or all different test techniques.
{"title":"An Evaluation of Specification Based Test Generation Techniques Using Model Checkers","authors":"G. Fraser, A. Gargantini","doi":"10.1109/TAICPART.2009.20","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.20","url":null,"abstract":"Test case generation can be represented as a model checking problem, such that model checking tools automatically generate test cases. This has previously been applied to testing techniques such as coverage criteria, combinatorial testing, mutation testing, or requirements testing, and further criteria can be used similarly. However, little comparison between the existing techniques has been done to date, making it difficult to choose a technique. In this paper we define existing and new criteria in a common framework, and evaluate and compare them on a set of realistic specifications. Part of our findings is that because testing with model checkers represents the test case generation problem in a very flexible way best results can be achieved by combining several techniques from different categories. A best effort approach where test cases are only created for uncovered test requirements can create relatively small test suites that cover many or all different test techniques.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116258053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.32
M. Irfan
In order to reduce the cost and provide rapid development, most of the modern and complex systems are built integrating prefabricated third party components COTS. We have been investigating techniques to build formal models for black box components. The integration testing framework developed by our team leaves several open strategies; we will be investigating variations of these open strategies to enhance applicability. We are investigating the heuristics to improve the existing methodologies for learning black boxes and integration testing. We are addressing the counter-example part of the learning algorithm for improvements and are examining different techniques to identify the counterexamples in a more efficient way.
{"title":"Heuristics for Improving Model Learning Based Software Testing","authors":"M. Irfan","doi":"10.1109/TAICPART.2009.32","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.32","url":null,"abstract":"In order to reduce the cost and provide rapid development, most of the modern and complex systems are built integrating prefabricated third party components COTS. We have been investigating techniques to build formal models for black box components. The integration testing framework developed by our team leaves several open strategies; we will be investigating variations of these open strategies to enhance applicability. We are investigating the heuristics to improve the existing methodologies for learning black boxes and integration testing. We are addressing the counter-example part of the learning algorithm for improvements and are examining different techniques to identify the counterexamples in a more efficient way.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134127710","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.34
D. Hoffman, Hong-Yi Wang, Mitch Chang, David Ly-Gagnon
Grammar based test generation (GBTG) has seen extensive study and considerable practical use since the 1970s. GBTG was introduced to generate source code for testing compilers from context-free grammars specifying language syntax. More recently, GBTG has been applied to many other testing problems, including the generation of eXtensible Markup Language (XML) documents. Recent research has shown how to integrate covering-array generation into GBTG tools. While the integration offers considerable power to the tester, there are few practical demonstrations in the literature. We present a case study showing how to use grammars and covering arrays for effective input generation. The generated tests have been used to systematically evaluate an RSS feed parser for HTML injection vulnerabilities.
{"title":"Grammar Based Testing of HTML Injection Vulnerabilities in RSS Feeds","authors":"D. Hoffman, Hong-Yi Wang, Mitch Chang, David Ly-Gagnon","doi":"10.1109/TAICPART.2009.34","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.34","url":null,"abstract":"Grammar based test generation (GBTG) has seen extensive study and considerable practical use since the 1970s. GBTG was introduced to generate source code for testing compilers from context-free grammars specifying language syntax. More recently, GBTG has been applied to many other testing problems, including the generation of eXtensible Markup Language (XML) documents. Recent research has shown how to integrate covering-array generation into GBTG tools. While the integration offers considerable power to the tester, there are few practical demonstrations in the literature. We present a case study showing how to use grammars and covering arrays for effective input generation. The generated tests have been used to systematically evaluate an RSS feed parser for HTML injection vulnerabilities.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130247755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.19
A. Kalaji, R. Hierons, S. Swift
The extended finite state machine is a powerful model that can capture almost all the aspects of a system. However, testing from an EFSM is yet a challenging task due to two main problems: path feasibility and path test data generation. Although optimization algorithms are efficient, their applications to EFSM testing have received very little attention. The aim of this paper is to develop a novel approach that utilizes optimization algorithms to test from EFSM models.
{"title":"A Search-Based Approach for Automatic Test Generation from Extended Finite State Machine (EFSM)","authors":"A. Kalaji, R. Hierons, S. Swift","doi":"10.1109/TAICPART.2009.19","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.19","url":null,"abstract":"The extended finite state machine is a powerful model that can capture almost all the aspects of a system. However, testing from an EFSM is yet a challenging task due to two main problems: path feasibility and path test data generation. Although optimization algorithms are efficient, their applications to EFSM testing have received very little attention. The aim of this paper is to develop a novel approach that utilizes optimization algorithms to test from EFSM models.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124077013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2009-09-04DOI: 10.1109/TAICPART.2009.14
L. Borner, B. Paech
Existing software systems consist of thousands of software components realizing countless requirements. To fulfill these requirements, components have to interact with or depend on each other. The goal of the integration testing process is to test that the interactions between these components are correctly realized. However, it is impossible to test all dependencies because of time and budget constraints. Therefore, error-prone dependencies have to be selected as tests. This paper presents an approach to select the test focus in the integration test process. It uses dependency and error information of previous versions of the system under test. Error-prone dependency properties are identified by statistical approaches and used to select dependencies in the current version of the system. The results of two case studies with real software systems are presented.
{"title":"Using Dependency Information to Select the Test Focus in the Integration Testing Process","authors":"L. Borner, B. Paech","doi":"10.1109/TAICPART.2009.14","DOIUrl":"https://doi.org/10.1109/TAICPART.2009.14","url":null,"abstract":"Existing software systems consist of thousands of software components realizing countless requirements. To fulfill these requirements, components have to interact with or depend on each other. The goal of the integration testing process is to test that the interactions between these components are correctly realized. However, it is impossible to test all dependencies because of time and budget constraints. Therefore, error-prone dependencies have to be selected as tests. This paper presents an approach to select the test focus in the integration test process. It uses dependency and error information of previous versions of the system under test. Error-prone dependency properties are identified by statistical approaches and used to select dependencies in the current version of the system. The results of two case studies with real software systems are presented.","PeriodicalId":339626,"journal":{"name":"2009 Testing: Academic and Industrial Conference - Practice and Research Techniques","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133817514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：