Software requirement specification (SRS) document is the most crucial document in software development process. All subsequent steps in software development are influenced by this document. However, issues in requirement, such as ambiguity or incomplete specification may lead to misinterpretation of requirements which consequently, influence the testing activities and higher the risk of time and cost overrun of the project. Finding defects in the initial development phase is crucial since the defect that found late is more expensive than if it was found early. This study describes an automated approach for detecting ambiguous software requirement specification. To this end, we propose the combination of text mining and machine learning. Since the dataset is derived from Malaysian industrial SRS documents, this study only focuses on the Malaysian context. We used text mining for feature extraction and for preparing the training set. Based on this training set, the method ‘learns’ to detect the ambiguous requirement specification. In this paper, we study a set of nine (9) classification algorithms from the machine learning community and evaluate which algorithm performs best to detect the ambiguous software requirement specification. Based on the experiment’s result, we develop a working prototype which later is used for our initial validation of our approach. The initial validation shows that the result produced by the classification model is reasonably acceptable. Even though this study is an experimental benchmark, we optimist that this approach may contributes to enhance the quality of SRS.
{"title":"Ambiguous Software Requirement Specification Detection: An Automated Approach","authors":"Mohd Hafeez Osman, M. Zaharin","doi":"10.1145/3195538.3195545","DOIUrl":"https://doi.org/10.1145/3195538.3195545","url":null,"abstract":"Software requirement specification (SRS) document is the most crucial document in software development process. All subsequent steps in software development are influenced by this document. However, issues in requirement, such as ambiguity or incomplete specification may lead to misinterpretation of requirements which consequently, influence the testing activities and higher the risk of time and cost overrun of the project. Finding defects in the initial development phase is crucial since the defect that found late is more expensive than if it was found early. This study describes an automated approach for detecting ambiguous software requirement specification. To this end, we propose the combination of text mining and machine learning. Since the dataset is derived from Malaysian industrial SRS documents, this study only focuses on the Malaysian context. We used text mining for feature extraction and for preparing the training set. Based on this training set, the method ‘learns’ to detect the ambiguous requirement specification. In this paper, we study a set of nine (9) classification algorithms from the machine learning community and evaluate which algorithm performs best to detect the ambiguous software requirement specification. Based on the experiment’s result, we develop a working prototype which later is used for our initial validation of our approach. The initial validation shows that the result produced by the classification model is reasonably acceptable. Even though this study is an experimental benchmark, we optimist that this approach may contributes to enhance the quality of SRS.","PeriodicalId":121144,"journal":{"name":"2018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124639927","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dietmar Freudenstein, Jeannette Radduenz, Maximilian Junker, S. Eder, B. Hauptmann
Designing a small set of tests that nonetheless cover the requirements sufficiently is tantamount to keep costs for testing at bay while still maintaining the necessary quality. Engineering an opti- mal test-suite requires both, insight into the domain and the system under test, but also carefully examining the combinatorics inherent in the requirements. Especially the second part is a cognitive chal- lenge and systematic methods are cumbersome when performed by hand. In this paper, we present Specmate, a tool that supports and partly automates the design of tests from requirements. It pro- vides light-weight modeling techniques to capture requirements, test generation facilities to create test specifications and further supporting functions to derive test procedures or test-scripts from specifications. Specmate has been developed and evaluated in the context of one of the core business systems of Allianz Deutsch- land, a large insurance company. The source code is freely avail- able at GitHub and an online-demo of Specmate is available at http://specmate.in.tum.de.
{"title":"Automated Test-Design from Requirements - The Specmate Tool","authors":"Dietmar Freudenstein, Jeannette Radduenz, Maximilian Junker, S. Eder, B. Hauptmann","doi":"10.1145/3195538.3195543","DOIUrl":"https://doi.org/10.1145/3195538.3195543","url":null,"abstract":"Designing a small set of tests that nonetheless cover the requirements sufficiently is tantamount to keep costs for testing at bay while still maintaining the necessary quality. Engineering an opti- mal test-suite requires both, insight into the domain and the system under test, but also carefully examining the combinatorics inherent in the requirements. Especially the second part is a cognitive chal- lenge and systematic methods are cumbersome when performed by hand. In this paper, we present Specmate, a tool that supports and partly automates the design of tests from requirements. It pro- vides light-weight modeling techniques to capture requirements, test generation facilities to create test specifications and further supporting functions to derive test procedures or test-scripts from specifications. Specmate has been developed and evaluated in the context of one of the core business systems of Allianz Deutsch- land, a large insurance company. The source code is freely avail- able at GitHub and an online-demo of Specmate is available at http://specmate.in.tum.de.","PeriodicalId":121144,"journal":{"name":"2018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET)","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131831818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Testing in the active sense is the most common way to perform verification and validation of systems, but testing in the passive sense has one compelling property: independence. Independence from test stimuli and other passive tests opens up for parallel testing and off-line analysis. However, the tests can be difficult to develop since the complete testable state must be expressed using some formalism. We argue that a carefully chosen language together with an interactive work flow, providing immediate feedback, can enable testers to approach passive testing. We have conducted a case study in the automotive domain, interviewing experienced testers. The testers have been introduced to, and had hands-on practice with a tool. The tool is based on Easy Approach to Requirements Syntax (EARS) and provides an interactive work flow for developing and evaluating test results. The case study shows that i) the testers believe passive testing is useful for many of their tests, ii) they see benefits in parallelism and off-line analysis, iii) the interactive work flow is necessary for writing the testable state expression, but iv) when the testable state becomes too complex, then the proposed language is a limitation. However, the language contributes to concise tests, resembling executable requirements.
{"title":"A Case Study of Interactive Development of Passive Tests","authors":"Daniel Flemström, T. Gustafsson, A. Kobetski","doi":"10.1145/3195538.3195544","DOIUrl":"https://doi.org/10.1145/3195538.3195544","url":null,"abstract":"Testing in the active sense is the most common way to perform verification and validation of systems, but testing in the passive sense has one compelling property: independence. Independence from test stimuli and other passive tests opens up for parallel testing and off-line analysis. However, the tests can be difficult to develop since the complete testable state must be expressed using some formalism. We argue that a carefully chosen language together with an interactive work flow, providing immediate feedback, can enable testers to approach passive testing. We have conducted a case study in the automotive domain, interviewing experienced testers. The testers have been introduced to, and had hands-on practice with a tool. The tool is based on Easy Approach to Requirements Syntax (EARS) and provides an interactive work flow for developing and evaluating test results. The case study shows that i) the testers believe passive testing is useful for many of their tests, ii) they see benefits in parallelism and off-line analysis, iii) the interactive work flow is necessary for writing the testable state expression, but iv) when the testable state becomes too complex, then the proposed language is a limitation. However, the language contributes to concise tests, resembling executable requirements.","PeriodicalId":121144,"journal":{"name":"2018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127194996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Tahvili, L. Hatvani, M. Felderer, W. Afzal, Mehrdad Saadatmand, M. Bohlin
One of the challenging issues in improving the test efficiency is that of achieving a balance between testing goals and testing resources. Test execution scheduling is one way of saving time and budget, where a set of test cases are grouped and tested at the same time. To have an optimal test execution schedule, all related information of a test case (e.g. execution time, functionality to be tested, dependency and similarity with other test cases) need to be analyzed. Test scheduling problem becomes more complicated at high-level testing, such as integration testing and especially in manual testing procedure. Test specifications are generally written in natural text by humans and usually contain ambiguity and uncertainty. Therefore, analyzing a test specification demands a strong learning algorithm. In this position paper, we propose a natural language processing-based approach that, given test specifications at the integration level, allows automatic detection of test cases semantic dependencies. The proposed approach utilizes the Doc2Vec algorithm and converts each test case into a vector in n-dimensional space. These vectors are then grouped using the HDBSCAN clustering algorithm into semantic clusters. Finally, a set of cluster-based test scheduling strategies are proposed for execution. The proposed approach has been applied in a sub-system from the railway domain by analyzing an ongoing testing project at Bombardier Transportation AB, Sweden.
{"title":"Cluster-Based Test Scheduling Strategies Using Semantic Relationships between Test Specifications","authors":"S. Tahvili, L. Hatvani, M. Felderer, W. Afzal, Mehrdad Saadatmand, M. Bohlin","doi":"10.1145/3195538.3195540","DOIUrl":"https://doi.org/10.1145/3195538.3195540","url":null,"abstract":"One of the challenging issues in improving the test efficiency is that of achieving a balance between testing goals and testing resources. Test execution scheduling is one way of saving time and budget, where a set of test cases are grouped and tested at the same time. To have an optimal test execution schedule, all related information of a test case (e.g. execution time, functionality to be tested, dependency and similarity with other test cases) need to be analyzed. Test scheduling problem becomes more complicated at high-level testing, such as integration testing and especially in manual testing procedure. Test specifications are generally written in natural text by humans and usually contain ambiguity and uncertainty. Therefore, analyzing a test specification demands a strong learning algorithm. In this position paper, we propose a natural language processing-based approach that, given test specifications at the integration level, allows automatic detection of test cases semantic dependencies. The proposed approach utilizes the Doc2Vec algorithm and converts each test case into a vector in n-dimensional space. These vectors are then grouped using the HDBSCAN clustering algorithm into semantic clusters. Finally, a set of cluster-based test scheduling strategies are proposed for execution. The proposed approach has been applied in a sub-system from the railway domain by analyzing an ongoing testing project at Bombardier Transportation AB, Sweden.","PeriodicalId":121144,"journal":{"name":"2018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126350368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In order to reach an acceptable level of confidence in the quality of a software product, testing of the software is paramount. To obtain "good" quality software it is essential to rely on "good" test cases. To define the criteria for what make up for a "good" test case is not a trivial task. Over the past 15 years, a short list of publications have presented criteria for "good" test cases but without ranking them based on their importance. This paper presents a non-exhaustive and non-authoritative tentative list of 15 criteria and a ranking of their relative importance. A number of the criteria come from previous publications but also from discussions with our industrial partners. The ranking is based on results collected via a questionnaire that was sent out to a limited number of randomly chosen respondents in the Swedish software industry. This means that the results are more indicative than conclusive.
{"title":"Test Case Quality as Perceived in Sweden","authors":"A. Adlemo, He Tan, Vladimir Tarasov","doi":"10.1145/3195538.3195541","DOIUrl":"https://doi.org/10.1145/3195538.3195541","url":null,"abstract":"In order to reach an acceptable level of confidence in the quality of a software product, testing of the software is paramount. To obtain \"good\" quality software it is essential to rely on \"good\" test cases. To define the criteria for what make up for a \"good\" test case is not a trivial task. Over the past 15 years, a short list of publications have presented criteria for \"good\" test cases but without ranking them based on their importance. This paper presents a non-exhaustive and non-authoritative tentative list of 15 criteria and a ranking of their relative importance. A number of the criteria come from previous publications but also from discussions with our industrial partners. The ranking is based on results collected via a questionnaire that was sent out to a limited number of randomly chosen respondents in the Swedish software industry. This means that the results are more indicative than conclusive.","PeriodicalId":121144,"journal":{"name":"2018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116895391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nelly Condori-Fernández, Maria Fernanda Granda, T. Vos
Researchers have proposed a number of prioritization techniques to help decision makers select an optimal combination of (non-) functional requirements to implement. [Problem] However, most of them are defined based on an ordinal or nominal scale, which are not reliable because they are limited to simple operations of ranked or ordered requirements. [Principal ideas] We argue that the importance of certain requirements could be determined by their criticality level, which can be assessed using a ratio scale. [Contribution]The main contribution of the paper is the new strategy proposed for prioritizing functional requirements, using early mutation testing and dependency analysis.
{"title":"Toward a Functional Requirements Prioritization with Early Mutation Testing","authors":"Nelly Condori-Fernández, Maria Fernanda Granda, T. Vos","doi":"10.1145/3195538.3195539","DOIUrl":"https://doi.org/10.1145/3195538.3195539","url":null,"abstract":"Researchers have proposed a number of prioritization techniques to help decision makers select an optimal combination of (non-) functional requirements to implement. [Problem] However, most of them are defined based on an ordinal or nominal scale, which are not reliable because they are limited to simple operations of ranked or ordered requirements. [Principal ideas] We argue that the importance of certain requirements could be determined by their criticality level, which can be assessed using a ratio scale. [Contribution]The main contribution of the paper is the new strategy proposed for prioritizing functional requirements, using early mutation testing and dependency analysis.","PeriodicalId":121144,"journal":{"name":"2018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116503388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Testing is subject to two basic constraints, namely cost and quality. The cost depends on the efficiency of the testing activities as well as their quality and testability. The author’s practical experience in large-scale systems shows that if the requirements are adapted iteratively or the architecture is altered, testability decreases. However, what is often lacking is a root cause analysis of the testability degradations and the introduction of improvement measures during software development. In order to introduce agile practices in the rigid strategy of the V-model, good testability of software artifacts is vital. So testability is also the bridgehead towards agility. In this paper, we report on a case study in which we measure and improve testability on the basis of the Goal Question Metric Approach.
{"title":"Measuring and Improving Testability of System Requirements in an Industrial Context by Applying the Goal Question Metric Approach","authors":"Armin Beer, M. Felderer","doi":"10.1145/3195538.3195542","DOIUrl":"https://doi.org/10.1145/3195538.3195542","url":null,"abstract":"Testing is subject to two basic constraints, namely cost and quality. The cost depends on the efficiency of the testing activities as well as their quality and testability. The author’s practical experience in large-scale systems shows that if the requirements are adapted iteratively or the architecture is altered, testability decreases. However, what is often lacking is a root cause analysis of the testability degradations and the introduction of improvement measures during software development. In order to introduce agile practices in the rigid strategy of the V-model, good testability of software artifacts is vital. So testability is also the bridgehead towards agility. In this paper, we report on a case study in which we measure and improve testability on the basis of the Goal Question Metric Approach.","PeriodicalId":121144,"journal":{"name":"2018 IEEE/ACM 5th International Workshop on Requirements Engineering and Testing (RET)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131605405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：