首页 > 最新文献

2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)最新文献

英文 中文
Applying a Pairwise Coverage Criterion to Scenario-Based Testing 对基于场景的测试应用成对覆盖标准
L. D. Bousquet, Mickaël Delahaye, Catherine Oriat
Some scenario-based testing approaches propose to express a test suite as a regular expression (called scenario or pattern). It specifies a set of valid sequences of operations in a abstract way. When the regular expression is "unfold", test sequences are obtained. Usually, the unfolding is done in an exhaustive way, which can result in a combinatorial explosion. In this article, we explore a pairwise coverage criterion to select a subset of test sequences satisfying the pattern, in order to decrease the number of test sequences. The originality of the approach lies in the fact that the pairwise criterion is applied to the instantiated method calls (and not on the parameters). We applied this strategy to generate unit tests for Java classes. The quality of the test suites is evaluated with a mutation analysis and compared to test suites randomly generated.
一些基于场景的测试方法建议将测试套件表示为正则表达式(称为场景或模式)。它以抽象的方式指定一组有效的操作序列。当正则表达式为“展开”时,得到测试序列。通常,展开是以穷尽的方式完成的,这可能导致组合爆炸。在本文中,我们探索了一个成对覆盖准则来选择满足模式的测试序列子集,以减少测试序列的数量。该方法的独创性在于将成对标准应用于实例化的方法调用(而不是参数)。我们将此策略应用于为Java类生成单元测试。通过突变分析评估测试套件的质量,并与随机生成的测试套件进行比较。
{"title":"Applying a Pairwise Coverage Criterion to Scenario-Based Testing","authors":"L. D. Bousquet, Mickaël Delahaye, Catherine Oriat","doi":"10.1109/ICSTW.2016.23","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.23","url":null,"abstract":"Some scenario-based testing approaches propose to express a test suite as a regular expression (called scenario or pattern). It specifies a set of valid sequences of operations in a abstract way. When the regular expression is \"unfold\", test sequences are obtained. Usually, the unfolding is done in an exhaustive way, which can result in a combinatorial explosion. In this article, we explore a pairwise coverage criterion to select a subset of test sequences satisfying the pattern, in order to decrease the number of test sequences. The originality of the approach lies in the fact that the pairwise criterion is applied to the instantiated method calls (and not on the parameters). We applied this strategy to generate unit tests for Java classes. The quality of the test suites is evaluated with a mutation analysis and compared to test suites randomly generated.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131866829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
A Tool for Automated Inspection of Software Design Documents and Its Empirical Evaluation in an Aviation Industry Setting 航空工业背景下软件设计文档自动检测工具及其实证评价
M. E. Coskun, M. Ceylan, Kadir Yigitozu, V. Garousi
While software inspection is an effective activity to detect defects early in the software development lifecycle, it is an effort-intensive and error-prone activity. Motivated by a real need in the context of the Turkish Aerospace Industries Inc. (TAI), a tool named AutoInspect was developed to (semi-) automate the inspection of software design documents and, as a result, to increase the efficiency and effectiveness of the inspection process. We present in this paper the features of the tool, its development details and its initial evaluation for inspecting the design documents of three real systems in the company. The results of the initial case-study reveal that the tool is indeed able to increase the inspections efficiency and effectiveness. In terms of efficiency, inspection engineers who used AutoInspect performed 41-50% more efficiently, for the three design documents under study, compared to the case when the tool was not used (i.e., manual inspections). In terms of effectiveness, compared to manual inspections, the automated approach found between 23-33% more defects in the three design documents under study. As the tool currently only provides partial automation, our efforts are currently underway to increase its automation level even further.
虽然软件检查是在软件开发生命周期早期检测缺陷的有效活动,但它是一项工作量大且容易出错的活动。在土耳其航空航天工业公司(TAI)的实际需求的推动下,开发了一种名为AutoInspect的工具,用于(半)自动化软件设计文档的检查,从而提高了检查过程的效率和有效性。本文介绍了该工具的特点、开发细节以及对公司三个实际系统设计文件的初步评价。初步的案例研究结果表明,该工具确实能够提高检查的效率和效果。在效率方面,与不使用AutoInspect工具(即人工检查)的情况相比,使用AutoInspect的检查工程师对所研究的三个设计文档的效率提高了41-50%。就有效性而言,与人工检查相比,自动化方法在研究的三个设计文档中发现了23-33%的缺陷。由于该工具目前只提供部分自动化,我们目前正在努力进一步提高其自动化水平。
{"title":"A Tool for Automated Inspection of Software Design Documents and Its Empirical Evaluation in an Aviation Industry Setting","authors":"M. E. Coskun, M. Ceylan, Kadir Yigitozu, V. Garousi","doi":"10.1109/ICSTW.2016.12","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.12","url":null,"abstract":"While software inspection is an effective activity to detect defects early in the software development lifecycle, it is an effort-intensive and error-prone activity. Motivated by a real need in the context of the Turkish Aerospace Industries Inc. (TAI), a tool named AutoInspect was developed to (semi-) automate the inspection of software design documents and, as a result, to increase the efficiency and effectiveness of the inspection process. We present in this paper the features of the tool, its development details and its initial evaluation for inspecting the design documents of three real systems in the company. The results of the initial case-study reveal that the tool is indeed able to increase the inspections efficiency and effectiveness. In terms of efficiency, inspection engineers who used AutoInspect performed 41-50% more efficiently, for the three design documents under study, compared to the case when the tool was not used (i.e., manual inspections). In terms of effectiveness, compared to manual inspections, the automated approach found between 23-33% more defects in the three design documents under study. As the tool currently only provides partial automation, our efforts are currently underway to increase its automation level even further.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114617110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Combinatorial Testing: From Algorithms to Applications 组合测试:从算法到应用
A. Gargantini, Rachel Tzoref
IWCT introduction.
IWCT介绍。
{"title":"Combinatorial Testing: From Algorithms to Applications","authors":"A. Gargantini, Rachel Tzoref","doi":"10.1109/ICSTW.2016.40","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.40","url":null,"abstract":"IWCT introduction.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129755216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Dynamic Integration Test Selection Based on Test Case Dependencies 基于测试用例依赖性的动态集成测试选择
S. Tahvili, Mehrdad Saadatmand, S. Larsson, W. Afzal, M. Bohlin, Daniel Sundmark
Prioritization, selection and minimization of test cases are well-known problems in software testing. Test case prioritization deals with the problem of ordering an existing set of test cases, typically with respect to the estimated likelihood of detecting faults. Test case selection addresses the problem of selecting a subset of an existing set of test cases, typically by discarding test cases that do not add any value in improving the quality of the software under test. Most existing approaches for test case prioritization and selection suffer from one or several drawbacks. For example, they to a large extent utilize static analysis of code for that purpose, making them unfit for higher levels of testing such as integration testing. Moreover, they do not exploit the possibility of dynamically changing the prioritization or selection of test cases based on the execution results of prior test cases. Such dynamic analysis allows for discarding test cases that do not need to be executed and are thus redundant. This paper proposes a generic method for prioritization and selection of test cases in integration testing that addresses the above issues. We also present the results of an industrial case study where initial evidence suggests the potential usefulness of our approach in testing a safety-critical train control management subsystem.
测试用例的优先级、选择和最小化是软件测试中众所周知的问题。测试用例优先级处理对现有测试用例集排序的问题,通常是根据检测错误的估计可能性。测试用例选择解决了选择现有测试用例集合的子集的问题,通常是通过丢弃在改进被测软件质量方面没有增加任何价值的测试用例。大多数现有的测试用例优先级和选择方法都有一个或几个缺点。例如,它们在很大程度上利用代码的静态分析来达到这个目的,使得它们不适合更高级别的测试,比如集成测试。此外,它们没有利用基于先前测试用例的执行结果动态改变优先级或选择测试用例的可能性。这样的动态分析允许丢弃不需要执行的测试用例,因此是冗余的。本文提出了一种在集成测试中对测试用例进行优先级排序和选择的通用方法,以解决上述问题。我们还提出了一个工业案例研究的结果,其中初步证据表明,我们的方法在测试安全关键列车控制管理子系统方面可能有用。
{"title":"Dynamic Integration Test Selection Based on Test Case Dependencies","authors":"S. Tahvili, Mehrdad Saadatmand, S. Larsson, W. Afzal, M. Bohlin, Daniel Sundmark","doi":"10.1109/ICSTW.2016.14","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.14","url":null,"abstract":"Prioritization, selection and minimization of test cases are well-known problems in software testing. Test case prioritization deals with the problem of ordering an existing set of test cases, typically with respect to the estimated likelihood of detecting faults. Test case selection addresses the problem of selecting a subset of an existing set of test cases, typically by discarding test cases that do not add any value in improving the quality of the software under test. Most existing approaches for test case prioritization and selection suffer from one or several drawbacks. For example, they to a large extent utilize static analysis of code for that purpose, making them unfit for higher levels of testing such as integration testing. Moreover, they do not exploit the possibility of dynamically changing the prioritization or selection of test cases based on the execution results of prior test cases. Such dynamic analysis allows for discarding test cases that do not need to be executed and are thus redundant. This paper proposes a generic method for prioritization and selection of test cases in integration testing that addresses the above issues. We also present the results of an industrial case study where initial evidence suggests the potential usefulness of our approach in testing a safety-critical train control management subsystem.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"128 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127393134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 23
Code Defenders: A Mutation Testing Game 代码捍卫者:一款变异测试游戏
J. Rojas, G. Fraser
Mutation testing is endorsed by software testing researchers for its unique capability of providing pragmatic estimates of a test suite's fault detection capability, and for guiding testers in improving their test suites. In practice, however, wide-spread adoption of mutation testing is hampered because any non-trivial program results in huge numbers of mutants, many of which are either trivial or equivalent, and thus useless. Trivial mutants reduce the motivation of developers in trusting and using the technique, while equivalent mutants are frustratingly difficult to handle. These problems are exacerbated by insufficient education on testing, which often means that mutation testing is not well understood in practice. These are examples of the types of problems that gamification aims to overcome by making such tedious activities competitive and entertaining. In this paper, we introduce the first steps towards building Code Defenders, a mutation testing game where players take the role of an attacker, who aims to create the most subtle non-equivalent mutants, or a defender, who aims to create strong tests to kill these mutants. The benefits of such an approach are manifold: The game can serve an educational role by engaging learners in mutation testing activities in a fun way. Experienced players will produce strong test suites, capable of detecting even the most subtle bugs that other players can conceive. Equivalent mutants are handled by making them a special part of the gameplay, where points are at stake in duels between attackers and defenders.
突变测试得到了软件测试研究者的认可,因为它提供了对测试套件的故障检测能力进行实用评估的独特能力,并指导测试人员改进他们的测试套件。然而,在实践中,突变测试的广泛采用受到阻碍,因为任何非平凡的程序都会导致大量的突变,其中许多要么是平凡的,要么是等价的,因此是无用的。微不足道的突变降低了开发人员信任和使用该技术的动机,而等效的突变则令人沮丧地难以处理。这些问题由于测试教育的不足而加剧,这通常意味着在实践中对突变测试没有很好的理解。这些都是游戏化旨在通过将乏味的活动变得具有竞争性和娱乐性来克服的问题类型。在本文中,我们将介绍构建Code Defenders的第一步,这是一款突变测试游戏,玩家将扮演攻击者的角色,其目标是创造最微妙的非等效突变,或者是防御者的角色,其目标是创建强大的测试来杀死这些突变。这种方法的好处是多方面的:游戏可以通过以有趣的方式吸引学习者参与突变测试活动来发挥教育作用。有经验的玩家会制作强大的测试套件,能够发现其他玩家可能想到的最细微的漏洞。等效突变体的处理方式是将其作为游戏玩法的一个特殊部分,即在攻击者和防御者之间的决斗中,得分是至关重要的。
{"title":"Code Defenders: A Mutation Testing Game","authors":"J. Rojas, G. Fraser","doi":"10.1109/ICSTW.2016.43","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.43","url":null,"abstract":"Mutation testing is endorsed by software testing researchers for its unique capability of providing pragmatic estimates of a test suite's fault detection capability, and for guiding testers in improving their test suites. In practice, however, wide-spread adoption of mutation testing is hampered because any non-trivial program results in huge numbers of mutants, many of which are either trivial or equivalent, and thus useless. Trivial mutants reduce the motivation of developers in trusting and using the technique, while equivalent mutants are frustratingly difficult to handle. These problems are exacerbated by insufficient education on testing, which often means that mutation testing is not well understood in practice. These are examples of the types of problems that gamification aims to overcome by making such tedious activities competitive and entertaining. In this paper, we introduce the first steps towards building Code Defenders, a mutation testing game where players take the role of an attacker, who aims to create the most subtle non-equivalent mutants, or a defender, who aims to create strong tests to kill these mutants. The benefits of such an approach are manifold: The game can serve an educational role by engaging learners in mutation testing activities in a fun way. Experienced players will produce strong test suites, capable of detecting even the most subtle bugs that other players can conceive. Equivalent mutants are handled by making them a special part of the gameplay, where points are at stake in duels between attackers and defenders.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132143971","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 51
Successive Refinement of Models for Model-Based Testing to Increase System Test Effectiveness 基于模型的测试模型的连续细化以提高系统测试效率
Ceren Sahin Gebizli, Hasan Sözer, A. Ercan
Model-based testing is used for automatically generating test cases based on models of the system under test. The effectiveness of tests depends on the contents of these models. Therefore, we introduce a novel three-step model refinement approach. We represent test models in the form of Markov chains. First, we update state transition probabilities in these models based on usage profile. Second, we perform an update based on fault likelihood that is estimated with static code analysis. Our third update is based on error likelihood that is estimated with dynamic analysis. We generate and execute test cases after each refinement. We applied our approach for model-based testing of a Smart TV system and new faults were revealed after each refinement.
基于模型的测试用于根据被测系统的模型自动生成测试用例。测试的有效性取决于这些模型的内容。因此,我们引入了一种新的三步模型改进方法。我们用马尔可夫链的形式表示测试模型。首先,我们根据使用情况更新这些模型中的状态转移概率。其次,我们根据静态代码分析估计的故障可能性执行更新。我们的第三个更新是基于动态分析估计的错误可能性。我们在每次细化之后生成并执行测试用例。我们将我们的方法应用于智能电视系统的基于模型的测试,每次改进后都会发现新的故障。
{"title":"Successive Refinement of Models for Model-Based Testing to Increase System Test Effectiveness","authors":"Ceren Sahin Gebizli, Hasan Sözer, A. Ercan","doi":"10.1109/ICSTW.2016.10","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.10","url":null,"abstract":"Model-based testing is used for automatically generating test cases based on models of the system under test. The effectiveness of tests depends on the contents of these models. Therefore, we introduce a novel three-step model refinement approach. We represent test models in the form of Markov chains. First, we update state transition probabilities in these models based on usage profile. Second, we perform an update based on fault likelihood that is estimated with static code analysis. Our third update is based on error likelihood that is estimated with dynamic analysis. We generate and execute test cases after each refinement. We applied our approach for model-based testing of a Smart TV system and new faults were revealed after each refinement.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122212019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
Pseudo-Exhaustive Testing of Attribute Based Access Control Rules 基于属性的访问控制规则的伪穷举测试
D. Kuhn, Vincent C. Hu, David F. Ferraiolo, R. Kacker, Yu Lei
Access control typically requires translating policies or rules given in natural language into a form such as a programming language or decision table, which can be processed by an access control system. Once rules have been described in machine-processable form, testing is necessary to ensure that the rules are implemented correctly. This paper describes an approach based on combinatorial test methods for efficiently testing access control rules, using the structure of attribute based access control (ABAC) to detect a large class of faults without a conventional test oracle.
访问控制通常需要将以自然语言给出的策略或规则转换为可由访问控制系统处理的编程语言或决策表等形式。一旦以机器可处理的形式描述了规则,就需要进行测试以确保规则得到正确实现。本文提出了一种基于组合测试方法的有效测试访问控制规则的方法,利用基于属性的访问控制(ABAC)结构来检测大类故障,而无需传统的测试oracle。
{"title":"Pseudo-Exhaustive Testing of Attribute Based Access Control Rules","authors":"D. Kuhn, Vincent C. Hu, David F. Ferraiolo, R. Kacker, Yu Lei","doi":"10.1109/ICSTW.2016.35","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.35","url":null,"abstract":"Access control typically requires translating policies or rules given in natural language into a form such as a programming language or decision table, which can be processed by an access control system. Once rules have been described in machine-processable form, testing is necessary to ensure that the rules are implemented correctly. This paper describes an approach based on combinatorial test methods for efficiently testing access control rules, using the structure of attribute based access control (ABAC) to detect a large class of faults without a conventional test oracle.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122394881","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 15
Automatic Generation of UTP Models from Requirements in Natural Language 基于自然语言的需求自动生成UTP模型
Satoshi Masuda, T. Matsuodani, K. Tsuda
Unified Modeling Language (UML) is the language of modeling from requirements for software design. UML Testing Profile (UTP) is the definition of the modeling test from requirements analysis for software testing. UTP has Test Architecture, Test Behavior, Test Data, and Time Concepts as the test models. Requirements are described in natural language, and engineers who have modeling skills then manually generate test models. Hence the generation of test models depends upon the engineer's skills, leaving the quality of test models unstable. In this paper, we present automatic generation test models from requirements in natural language by focusing on descriptions of test cases in UTP test behavior. We develop three rules to generate test models from requirements by using natural language processing techniques and experiment with our approach on requirements in language that is considered natural English. Our results in three case studies show the promise of our approach.
统一建模语言(UML)是从软件设计需求出发进行建模的语言。UML测试概要文件(UTP)是从软件测试的需求分析中对建模测试的定义。UTP将测试架构、测试行为、测试数据和时间概念作为测试模型。需求用自然语言描述,拥有建模技能的工程师然后手动生成测试模型。因此,测试模型的生成取决于工程师的技能,从而导致测试模型的质量不稳定。在本文中,我们通过关注UTP测试行为中测试用例的描述,提出了用自然语言从需求中自动生成测试模型。我们开发了三条规则,通过使用自然语言处理技术从需求中生成测试模型,并用我们的方法在被认为是自然英语的语言中进行需求实验。我们在三个案例研究中的结果显示了我们的方法的前景。
{"title":"Automatic Generation of UTP Models from Requirements in Natural Language","authors":"Satoshi Masuda, T. Matsuodani, K. Tsuda","doi":"10.1109/ICSTW.2016.27","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.27","url":null,"abstract":"Unified Modeling Language (UML) is the language of modeling from requirements for software design. UML Testing Profile (UTP) is the definition of the modeling test from requirements analysis for software testing. UTP has Test Architecture, Test Behavior, Test Data, and Time Concepts as the test models. Requirements are described in natural language, and engineers who have modeling skills then manually generate test models. Hence the generation of test models depends upon the engineer's skills, leaving the quality of test models unstable. In this paper, we present automatic generation test models from requirements in natural language by focusing on descriptions of test cases in UTP test behavior. We develop three rules to generate test models from requirements by using natural language processing techniques and experiment with our approach on requirements in language that is considered natural English. Our results in three case studies show the promise of our approach.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132782020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Nequivack: Assessing Mutation Score Confidence Nequivack:评估突变评分置信度
Dominik Holling, Sebastian Banescu, Marco Probst, A. Petrovska, A. Pretschner
The mutation score is defined as the number of killed mutants divided by the number of non-equivalent mutants. However, whether a mutant is equivalent to the original program is undecidable in general. Thus, even when improving a test suite, a mutant score assessing this test suite may become worse during the development of a system, because of equivalent mutants introduced during mutant creation. This is a fundamental problem. Using static analysis and symbolic execution, we show how to establish non-equivalence or "don't know" among mutants. If the number of don't knows is small, this is a good indicator that a computed mutation score actually reflects its above definition. We can therefore have an increased confidence that mutation score trends correspond to actual improvements of a test suite's quality, and are not overly polluted by equivalent mutants. Using a set of 14 representative unit size programs, we show that for some, but not all, of these programs, the above confidence can indeed be established. We also evaluate the reproducibility, efficiency and effectiveness of our Nequivack tool. Our findings are that reproducibility is completely given. A single mutant analysis can be performed within 3 seconds on average, which is efficient for practical and industrial applications.
突变得分定义为被杀突变体的数量除以非等效突变体的数量。然而,突变体是否等同于原始程序通常是无法确定的。因此,即使在改进测试套件时,评估该测试套件的突变分数可能在系统开发期间变得更糟,因为在突变创建期间引入了等效的突变。这是一个根本性的问题。使用静态分析和符号执行,我们展示了如何在突变体之间建立不等价或“不知道”。如果不知道的数量很少,这是一个很好的指标,表明计算的突变得分实际上反映了它的上述定义。因此,我们可以增加对突变得分趋势与测试套件质量的实际改进相对应的信心,并且不会被等效的突变过度污染。使用一组14个具有代表性的单位规模方案,我们表明,对于这些方案中的一些,但不是全部,上述置信度确实可以建立。我们还评估了Nequivack工具的重复性、效率和有效性。我们的发现是可重复性是完全给定的。单突变体分析平均可在3秒内完成,这对于实际和工业应用是有效的。
{"title":"Nequivack: Assessing Mutation Score Confidence","authors":"Dominik Holling, Sebastian Banescu, Marco Probst, A. Petrovska, A. Pretschner","doi":"10.1109/ICSTW.2016.29","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.29","url":null,"abstract":"The mutation score is defined as the number of killed mutants divided by the number of non-equivalent mutants. However, whether a mutant is equivalent to the original program is undecidable in general. Thus, even when improving a test suite, a mutant score assessing this test suite may become worse during the development of a system, because of equivalent mutants introduced during mutant creation. This is a fundamental problem. Using static analysis and symbolic execution, we show how to establish non-equivalence or \"don't know\" among mutants. If the number of don't knows is small, this is a good indicator that a computed mutation score actually reflects its above definition. We can therefore have an increased confidence that mutation score trends correspond to actual improvements of a test suite's quality, and are not overly polluted by equivalent mutants. Using a set of 14 representative unit size programs, we show that for some, but not all, of these programs, the above confidence can indeed be established. We also evaluate the reproducibility, efficiency and effectiveness of our Nequivack tool. Our findings are that reproducibility is completely given. A single mutant analysis can be performed within 3 seconds on average, which is efficient for practical and industrial applications.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114420092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
Pairwise Coverage-Based Testing with Selected Elements in a Query for Database Applications 对数据库应用程序查询中的选定元素进行基于覆盖率的成对测试
Koji Tsumura, H. Washizaki, Y. Fukazawa, Keishi Oshima, Ryota Mibe
Because program behaviors of database applications depend on the data used, code coverages do not effectively test database applications. Additionally, test coverages for database applications that focus on predicates in Structured Query Language (SQL) queries are not useful if the necessary predicates are omitted. In this paper, we present two new database applications using Plain Pairwise Coverage (PPC) and Selected Pairwise Coverage (SPC) for SQL queries called Plain Pairwise Coverage Testing (PPCT) and Selected Pairwise Coverage Testing (SPCT), respectively. These coverages are based on pairwise testing coverage, which employs selected elements in the SQL SELECT query as parameters. We also implement a coverage calculation tool and conduct case studies on two open source software systems. PPCT and SPCT can detect many faults, which are not detected by existing test methods based on predicates in the query. Furthermore, the case study suggests that SPCT can detect faults more efficiently than PPCT and the costs of SPCT can be further reduced by ignoring records filtered out by the conditions of the query.
因为数据库应用程序的程序行为依赖于所使用的数据,代码覆盖率不能有效地测试数据库应用程序。此外,如果省略了必要的谓词,那么关注结构化查询语言(Structured Query Language, SQL)查询中的谓词的数据库应用程序的测试覆盖率将是无用的。在本文中,我们提出了两个新的数据库应用程序,分别使用Plain Pairwise Coverage (PPC)和Selected Pairwise Coverage (SPC)进行SQL查询,称为Plain Pairwise Coverage Testing (PPCT)和Selected Pairwise Coverage Testing (SPCT)。这些覆盖率是基于成对测试覆盖率的,它使用SQL SELECT查询中的选定元素作为参数。我们还实现了覆盖率计算工具,并对两个开放源码软件系统进行了案例研究。基于查询中的谓词,PPCT和SPCT可以检测到许多现有测试方法无法检测到的故障。此外,案例研究表明,SPCT可以比PPCT更有效地检测故障,并且通过忽略被查询条件过滤掉的记录可以进一步降低SPCT的成本。
{"title":"Pairwise Coverage-Based Testing with Selected Elements in a Query for Database Applications","authors":"Koji Tsumura, H. Washizaki, Y. Fukazawa, Keishi Oshima, Ryota Mibe","doi":"10.1109/ICSTW.2016.19","DOIUrl":"https://doi.org/10.1109/ICSTW.2016.19","url":null,"abstract":"Because program behaviors of database applications depend on the data used, code coverages do not effectively test database applications. Additionally, test coverages for database applications that focus on predicates in Structured Query Language (SQL) queries are not useful if the necessary predicates are omitted. In this paper, we present two new database applications using Plain Pairwise Coverage (PPC) and Selected Pairwise Coverage (SPC) for SQL queries called Plain Pairwise Coverage Testing (PPCT) and Selected Pairwise Coverage Testing (SPCT), respectively. These coverages are based on pairwise testing coverage, which employs selected elements in the SQL SELECT query as parameters. We also implement a coverage calculation tool and conduct case studies on two open source software systems. PPCT and SPCT can detect many faults, which are not detected by existing test methods based on predicates in the query. Furthermore, the case study suggests that SPCT can detect faults more efficiently than PPCT and the costs of SPCT can be further reduced by ignoring records filtered out by the conditions of the query.","PeriodicalId":335145,"journal":{"name":"2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"129 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124246457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 9
期刊
2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1