Growing complexity of today’s software development requires new and better techniques in software testing. A promising one seems to be model-based testing. The goal is to automatically generate test artefacts from models, improve test coverage and guarantee traceability. Typical problems are missing reuse of design models and test case explosion. Our research work aims to find a solution for the mentioned problems in the area of UML and Business Information Systems. We use model transformations to automatically generate test models from manually annotated design models using a holistic view. In this paper we define and justify the research problem and present first results.
{"title":"Holistic Model-Based Testing for Business Information Systems","authors":"M. Mlynarski","doi":"10.1109/ICST.2010.35","DOIUrl":"https://doi.org/10.1109/ICST.2010.35","url":null,"abstract":"Growing complexity of today’s software development requires new and better techniques in software testing. A promising one seems to be model-based testing. The goal is to automatically generate test artefacts from models, improve test coverage and guarantee traceability. Typical problems are missing reuse of design models and test case explosion. Our research work aims to find a solution for the mentioned problems in the area of UML and Business Information Systems. We use model transformations to automatically generate test models from manually annotated design models using a holistic view. In this paper we define and justify the research problem and present first results.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"141 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114170921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The purpose of our research is to evaluate and compare different approaches for the specification, verification and implementation of model transformations, and to make recommendations for a transformation specification language which is modular, verifiable, and supports reuse and implementation. In this paper we survey existing approaches to model transformations and propose a new specification and implementation approach for transformations. We describe case studies, of state machine slicing, and re-architecting systems for achieving quality of service in service-oriented architectures, which are used to evaluate model transformation specification approaches and languages.
{"title":"Specification of UML Model Transformations","authors":"Shekoufeh Kolahdouz Rahimi","doi":"10.1109/ICST.2010.31","DOIUrl":"https://doi.org/10.1109/ICST.2010.31","url":null,"abstract":"The purpose of our research is to evaluate and compare different approaches for the specification, verification and implementation of model transformations, and to make recommendations for a transformation specification language which is modular, verifiable, and supports reuse and implementation. In this paper we survey existing approaches to model transformations and propose a new specification and implementation approach for transformations. We describe case studies, of state machine slicing, and re-architecting systems for achieving quality of service in service-oriented architectures, which are used to evaluate model transformation specification approaches and languages.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125251488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Developers often make multiple changes to software. These changes are introduced to work cooperatively or to accomplish separate goals. However, changes might not interact as expected or may produce undesired side effects. Thus, it is crucial for software-development tasks to know exactly which changes interact. For example, testers need this information to ensure that regression test suites test the combined behaviors of changes. For another example, teams of developers must determine whether it is safe to merge variants of a program modified in parallel. Existing techniques can be used to detect at runtime potential interactions among changes, but these reports tend to be coarse and imprecise. To address this problem, in this paper, we first present a formal model of change interactions at the code level, and then describe a new technique, based on this model, for detecting at runtime such interactions with accuracy. We also present the results of a comparison of our technique with other techniques on a set of Java subjects. Our results clearly suggest that existing techniques are too inaccurate and only our technique, of all those studied, provides acceptable confidence in detecting real change interactions occurring at runtime.
{"title":"Precisely Detecting Runtime Change Interactions for Evolving Software","authors":"Raúl A. Santelices, M. J. Harrold, A. Orso","doi":"10.1109/ICST.2010.29","DOIUrl":"https://doi.org/10.1109/ICST.2010.29","url":null,"abstract":"Developers often make multiple changes to software. These changes are introduced to work cooperatively or to accomplish separate goals. However, changes might not interact as expected or may produce undesired side effects. Thus, it is crucial for software-development tasks to know exactly which changes interact. For example, testers need this information to ensure that regression test suites test the combined behaviors of changes. For another example, teams of developers must determine whether it is safe to merge variants of a program modified in parallel. Existing techniques can be used to detect at runtime potential interactions among changes, but these reports tend to be coarse and imprecise. To address this problem, in this paper, we first present a formal model of change interactions at the code level, and then describe a new technique, based on this model, for detecting at runtime such interactions with accuracy. We also present the results of a comparison of our technique with other techniques on a set of Java subjects. Our results clearly suggest that existing techniques are too inaccurate and only our technique, of all those studied, provides acceptable confidence in detecting real change interactions occurring at runtime.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"38 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130811423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Intermittent failures and nondeterministic behavior complicate and compromise the effectiveness of software testing and debugging. To increase the observability of software faults, we explore the effect hardware configurations and processor load have on intermittent failures and the nondeterministic behavior of software systems. We conducted a case study on Mozilla Firefox with a selected set of reported field failures. We replicated the conditions that caused the reported failures ten times on each of nine hardware configurations by varying processor speed, memory, hard drive capacity, and processor load. Using several observability tools, we found that hardware configurations that had less processor speed and memory observed more failures than others. Our results also show that by manipulating processor load, we can influence the observability of some faults.
{"title":"Does Hardware Configuration and Processor Load Impact Software Fault Observability?","authors":"R. Syed, Brian P. Robinson, L. Williams","doi":"10.1109/ICST.2010.55","DOIUrl":"https://doi.org/10.1109/ICST.2010.55","url":null,"abstract":"Intermittent failures and nondeterministic behavior complicate and compromise the effectiveness of software testing and debugging. To increase the observability of software faults, we explore the effect hardware configurations and processor load have on intermittent failures and the nondeterministic behavior of software systems. We conducted a case study on Mozilla Firefox with a selected set of reported field failures. We replicated the conditions that caused the reported failures ten times on each of nine hardware configurations by varying processor speed, memory, hard drive capacity, and processor load. Using several observability tools, we found that hardware configurations that had less processor speed and memory observed more failures than others. Our results also show that by manipulating processor load, we can influence the observability of some faults.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130777746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Planning and scheduling of testing activities play an important role for any independent test team that performs tests for different software systems, developed by different development teams. This work studies the application of machine learning tools and variable selection tools to solve the problem of estimating the execution effort of functional tests. An analysis of the test execution process is developed and experiments are performed on two real databases. The main contributions of this paper are the approach of selecting the significant variables for database synthesis and the use of an artificial neural network trained with an asymmetric cost function.
{"title":"Machine Learning Methods and Asymmetric Cost Function to Estimate Execution Effort of Software Testing","authors":"Daniel Guerreiro e Silva, M. Jino, B. T. D. Abreu","doi":"10.1109/ICST.2010.46","DOIUrl":"https://doi.org/10.1109/ICST.2010.46","url":null,"abstract":"Planning and scheduling of testing activities play an important role for any independent test team that performs tests for different software systems, developed by different development teams. This work studies the application of machine learning tools and variable selection tools to solve the problem of estimating the execution effort of functional tests. An analysis of the test execution process is developed and experiments are performed on two real databases. The main contributions of this paper are the approach of selecting the significant variables for database synthesis and the use of an artificial neural network trained with an asymmetric cost function.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123681140","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
There is a growing trend to move desktop applications towards the web using advances made in web technologies such as Ajax. One common way to provide assurance about the correctness of such complex and evolving systems is through regression testing. Regression testing classical web applications has already been a notoriously daunting task because of the dynamism in web interfaces. Ajax applications pose an even greater challenge since the test case fragility degree is higher due to extensive run-time manipulation of the DOM tree and asynchronous client/server interactions. In this paper, we propose a technique, in which we automatically generate test cases and apply pipelined oracle comparators along with generated DOM templates, to deal with dynamic non-deterministic behavior in Ajax user interfaces. Our approach, implemented in Crawljax, is open source and provides a set of generic oracle comparators, template generators, and visualizations of test failure output. We describe two case studies evaluating the effectiveness, scalability, and required manual effort of the approach.
{"title":"Regression Testing Ajax Applications: Coping with Dynamism","authors":"D. Roest, A. Mesbah, A. Deursen","doi":"10.1109/ICST.2010.59","DOIUrl":"https://doi.org/10.1109/ICST.2010.59","url":null,"abstract":"There is a growing trend to move desktop applications towards the web using advances made in web technologies such as Ajax. One common way to provide assurance about the correctness of such complex and evolving systems is through regression testing. Regression testing classical web applications has already been a notoriously daunting task because of the dynamism in web interfaces. Ajax applications pose an even greater challenge since the test case fragility degree is higher due to extensive run-time manipulation of the DOM tree and asynchronous client/server interactions. In this paper, we propose a technique, in which we automatically generate test cases and apply pipelined oracle comparators along with generated DOM templates, to deal with dynamic non-deterministic behavior in Ajax user interfaces. Our approach, implemented in Crawljax, is open source and provides a set of generic oracle comparators, template generators, and visualizations of test failure output. We describe two case studies evaluating the effectiveness, scalability, and required manual effort of the approach.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"381 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123955705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sigrid Eldh, Joachim Brandt, M. Street, H. Hansson, S. Punnekkat
Development of large and complex software intensive systems with continuous builds typically generates large volumes of information with complex patterns and relations. Systematic and automated approaches are needed for efficient handling of such large quantities of data in a comprehensible way. In this paper we present an approach and tool enabling autonomous behavior in an automated test management tool to gain efficiency in concurrent software development and test. By capturing the required quality criteria in the test specifications and automating the test execution, test management can potentially be performed to a great extent without manual intervention. This work contributes towards a more autonomous behavior within a distributed remote test strategy based on metrics for decision making in automated testing. These metrics optimize management of fault corrections and retest, giving consideration to the impact of the identified weaknesses, such as fault-prone areas in software.
{"title":"Towards Fully Automated Test Management for Large Complex Systems","authors":"Sigrid Eldh, Joachim Brandt, M. Street, H. Hansson, S. Punnekkat","doi":"10.1109/ICST.2010.58","DOIUrl":"https://doi.org/10.1109/ICST.2010.58","url":null,"abstract":"Development of large and complex software intensive systems with continuous builds typically generates large volumes of information with complex patterns and relations. Systematic and automated approaches are needed for efficient handling of such large quantities of data in a comprehensible way. In this paper we present an approach and tool enabling autonomous behavior in an automated test management tool to gain efficiency in concurrent software development and test. By capturing the required quality criteria in the test specifications and automating the test execution, test management can potentially be performed to a great extent without manual intervention. This work contributes towards a more autonomous behavior within a distributed remote test strategy based on metrics for decision making in automated testing. These metrics optimize management of fault corrections and retest, giving consideration to the impact of the identified weaknesses, such as fault-prone areas in software.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124425695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Black box testing focuses on the core functionality of the system, while user interface testing is concerned with details of user interactions. Functional and user interface test cases are usually generated from two distinct system models, one for the functionality and one for the user interface. As a result, test cases derived from either model capture only partial system behavior and as such, are inadequate for testing full system behavior. We propose a method for formally integrating the model for the system functionality and the model for the user interface. The resulting composite model is then used to generate more complete test cases, capturing detailed user interactions as well as secondary system interactions. In this paper we employ use cases for modeling system functionality, and task models for describing user interfaces.
{"title":"A Formal Model for Generating Integrated Functional and User Interface Test Cases","authors":"D. Sinnig, F. Khendek, Patrice Chalin","doi":"10.1109/ICST.2010.56","DOIUrl":"https://doi.org/10.1109/ICST.2010.56","url":null,"abstract":"Black box testing focuses on the core functionality of the system, while user interface testing is concerned with details of user interactions. Functional and user interface test cases are usually generated from two distinct system models, one for the functionality and one for the user interface. As a result, test cases derived from either model capture only partial system behavior and as such, are inadequate for testing full system behavior. We propose a method for formally integrating the model for the system functionality and the model for the user interface. The resulting composite model is then used to generate more complete test cases, capturing detailed user interactions as well as secondary system interactions. In this paper we employ use cases for modeling system functionality, and task models for describing user interfaces.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126015520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this case study we test a landing gear control system of a military aircraft with the new version of Lutess, a tool for testing automatically synchronous software. Lutess requires the tester to specify the environment of the software under test by means of invariant properties in order to guide the test data generation. This specification can be enriched by operational profile specification in order to obtain more realistic scenarios. Moreover, test generation guided by safety properties makes possible to test more thoroughly the key features of the software, possibly under hypotheses on the software behavior. In this case, the generator chooses input data which are able to violate the properties. The new version of Lutess is based on constraint logic programming and provides some additional features (numeric inputs and outputs, hypotheses for safety guided testing, more powerful operational profiles). In this paper, we present the necessary steps for building the test model for Lutess on a real case study from the avionics. This makes possible to better understand the applicability of the approach and to assess the difficulty and the effectiveness of such a process in real-world applications.
{"title":"Towards a Testing Methodology for Reactive Systems: A Case Study of a Landing Gear Controller","authors":"L. Madani, V. Papailiopoulou, I. Parissis","doi":"10.1109/ICST.2010.21","DOIUrl":"https://doi.org/10.1109/ICST.2010.21","url":null,"abstract":"In this case study we test a landing gear control system of a military aircraft with the new version of Lutess, a tool for testing automatically synchronous software. Lutess requires the tester to specify the environment of the software under test by means of invariant properties in order to guide the test data generation. This specification can be enriched by operational profile specification in order to obtain more realistic scenarios. Moreover, test generation guided by safety properties makes possible to test more thoroughly the key features of the software, possibly under hypotheses on the software behavior. In this case, the generator chooses input data which are able to violate the properties. The new version of Lutess is based on constraint logic programming and provides some additional features (numeric inputs and outputs, hypotheses for safety guided testing, more powerful operational profiles). In this paper, we present the necessary steps for building the test model for Lutess on a real case study from the avionics. This makes possible to better understand the applicability of the approach and to assess the difficulty and the effectiveness of such a process in real-world applications.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132447642","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A random testing strategy can be effective at finding faults, but may leave some routines entirely untested if it never gets to call them on objects satisfying their preconditions. This limitation is particularly frustrating if the object pool does contain some precondition-satisfying objects but the strategy, which selects objects at random, does not use them. The extension of random testing described in this article addresses the problem. Experimentally, the resulting strategy succeeds in testing 56% of the routines that the pure random strategy missed; it tests hard routines 3.6 times more often; although it misses some of the faults detected by the original strategy, it finds 9.5% more faults overall; and it causes negligible overhead.
{"title":"Satisfying Test Preconditions through Guided Object Selection","authors":"Yi Wei, S. Gebhardt, B. Meyer, M. Oriol","doi":"10.1109/ICST.2010.34","DOIUrl":"https://doi.org/10.1109/ICST.2010.34","url":null,"abstract":"A random testing strategy can be effective at finding faults, but may leave some routines entirely untested if it never gets to call them on objects satisfying their preconditions. This limitation is particularly frustrating if the object pool does contain some precondition-satisfying objects but the strategy, which selects objects at random, does not use them. The extension of random testing described in this article addresses the problem. Experimentally, the resulting strategy succeeds in testing 56% of the routines that the pure random strategy missed; it tests hard routines 3.6 times more often; although it misses some of the faults detected by the original strategy, it finds 9.5% more faults overall; and it causes negligible overhead.","PeriodicalId":192678,"journal":{"name":"2010 Third International Conference on Software Testing, Verification and Validation","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131885957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：