Model-checkers are powerful tools that can find individual traces through models to satisfy desired properties. These traces provide solutions to a number of problems. Instead of individual traces, software testing needs sets of traces that satisfy coverage criteria. Finding a trace set in a large model is difficult because model checkers generate single traces and use a lot of memory. Space and time requirements of modelchecking algorithms grow exponentially with respect to the number of variables and parallel automata of the model being analyzed. We present a method that generates a set of traces by iteratively invoking a model checker. The method mitigates the memory consumption problem by dynamically building partitions along the traces. This method was applied to a testability case study, and it generated the complete trace set, while ordinary model-checking could only generate 26%.
{"title":"Generating Trace-Sets for Model-based Testing","authors":"B. Lindström, P. Pettersson, A. Offutt","doi":"10.1109/ISSRE.2007.15","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.15","url":null,"abstract":"Model-checkers are powerful tools that can find individual traces through models to satisfy desired properties. These traces provide solutions to a number of problems. Instead of individual traces, software testing needs sets of traces that satisfy coverage criteria. Finding a trace set in a large model is difficult because model checkers generate single traces and use a lot of memory. Space and time requirements of modelchecking algorithms grow exponentially with respect to the number of variables and parallel automata of the model being analyzed. We present a method that generates a set of traces by iteratively invoking a model checker. The method mitigates the memory consumption problem by dynamically building partitions along the traces. This method was applied to a testability case study, and it generated the complete trace set, while ordinary model-checking could only generate 26%.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133824252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
During development and testing, changes made to a system to repair a detected fault can often inject a new fault into the code base. These injected faults may not be in the same files that were just changed, since the effects of a change in the code base can have ramifications in other parts of the system. We propose a methodology for determining the effect of a change and then prioritizing regression test cases by gathering software change records and analyzing them through singular value decomposition. This methodology generates clusters of files that historically tend to change together. Combining these clusters with test case information yields a matrix that can be multiplied by a vector representing a new system modification to create a prioritized list of test cases. We performed a post hoc case study using this technique with three minor releases of a software product at IBM. We found that our methodology suggested additional regression tests in 50% of test runs and that the highest-priority suggested test found an additional fault 60% of the time.
{"title":"Prioritization of Regression Tests using Singular Value Decomposition with Empirical Change Records","authors":"Mark Sherriff, Mike Lake, L. Williams","doi":"10.1109/ISSRE.2007.25","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.25","url":null,"abstract":"During development and testing, changes made to a system to repair a detected fault can often inject a new fault into the code base. These injected faults may not be in the same files that were just changed, since the effects of a change in the code base can have ramifications in other parts of the system. We propose a methodology for determining the effect of a change and then prioritizing regression test cases by gathering software change records and analyzing them through singular value decomposition. This methodology generates clusters of files that historically tend to change together. Combining these clusters with test case information yields a matrix that can be multiplied by a vector representing a new system modification to create a prioritized list of test cases. We performed a post hoc case study using this technique with three minor releases of a software product at IBM. We found that our methodology suggested additional regression tests in 50% of test runs and that the highest-priority suggested test found an additional fault 60% of the time.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126645031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we develop a novel approach to estimate the optimal preventive rejuvenation schedule which maximizes the steady-state system availability. In the case with unknown system failure time distribution, the preventive rejuvenation is triggered for the purpose of preventive maintenance of software system. We formulate the upper and lower bounds of the predictive system availability using the one-look ahead predictive survivor function, and derive the pessimistic and optimistic rejuvenation policies. In the real data analysis we focus on a real Web server system and show the usefulness of the non-parametric predictive inference approach proposed in this paper.
{"title":"Non-parametric Predictive Inference of Preventive Rejuvenation Schedule in Operational Software Systems","authors":"K. Rinsaka, T. Dohi","doi":"10.1109/ISSRE.2007.13","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.13","url":null,"abstract":"In this paper we develop a novel approach to estimate the optimal preventive rejuvenation schedule which maximizes the steady-state system availability. In the case with unknown system failure time distribution, the preventive rejuvenation is triggered for the purpose of preventive maintenance of software system. We formulate the upper and lower bounds of the predictive system availability using the one-look ahead predictive survivor function, and derive the pessimistic and optimistic rejuvenation policies. In the real data analysis we focus on a real Web server system and show the usefulness of the non-parametric predictive inference approach proposed in this paper.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115647442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nachiappan Nagappan, L. Williams, M. Vouk, J. Osborne
In industrial practice, information on the software field quality of a product is available too late in the software lifecycle to guide affordable corrective action. An important step towards remediation of this problem lies in the ability to provide an early estimation of post-release field quality. This paper evaluates the Software Testing and Reliability Early Warning for Java (STREW-J) metric suite leveraging the software testing effort to predict post-release field quality early in the software development phases. The metric suite is applicable for software products implemented in Java for which an extensive suite of automated unit test cases are incrementally created as development proceeds. We validated the prediction model using the STREW-J metrics via a two-phase case study approach which involved 27 medium-sized open source projects, and five industrial projects. The error in estimation and the sensitivity of the predictions indicate the STREW-J metric suite can be used effectively to predict post-release software field quality.
{"title":"Using In-Process Testing Metrics to Estimate Post-Release Field Quality","authors":"Nachiappan Nagappan, L. Williams, M. Vouk, J. Osborne","doi":"10.1109/ISSRE.2007.18","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.18","url":null,"abstract":"In industrial practice, information on the software field quality of a product is available too late in the software lifecycle to guide affordable corrective action. An important step towards remediation of this problem lies in the ability to provide an early estimation of post-release field quality. This paper evaluates the Software Testing and Reliability Early Warning for Java (STREW-J) metric suite leveraging the software testing effort to predict post-release field quality early in the software development phases. The metric suite is applicable for software products implemented in Java for which an extensive suite of automated unit test cases are incrementally created as development proceeds. We validated the prediction model using the STREW-J metrics via a two-phase case study approach which involved 27 medium-sized open source projects, and five industrial projects. The error in estimation and the sensitivity of the predictions indicate the STREW-J metric suite can be used effectively to predict post-release software field quality.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123759832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sara Sprenkle, L. Pollock, H. Esquivel, Barbara Hazelwood, Stacey Ecott
Software developers need automated techniques to maintain the correctness of complex, evolving Web applications. While there has been success in automating some of the testing process for this domain, there exists little automated support for verifying that the executed test cases produce expected results. We assist in this tedious task by presenting a suite of automated oracle comparators for testing Web applications. To effectively identify failures, each comparator is specialized to particular characteristics of the possibly nondeterministic Web applications' output in the form of HTML responses. We also describe combinations of comparators designed to achieve both high precision and recall in failure detection and a tool for helping testers to analyze the output of multiple oracles in detail. We present results from an evaluation of the effectiveness and costs of the oracle comparators. We also provide recommendations to testers on applying effective oracle comparators based on their application's characteristics.
{"title":"Automated Oracle Comparators for TestingWeb Applications","authors":"Sara Sprenkle, L. Pollock, H. Esquivel, Barbara Hazelwood, Stacey Ecott","doi":"10.1109/ISSRE.2007.26","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.26","url":null,"abstract":"Software developers need automated techniques to maintain the correctness of complex, evolving Web applications. While there has been success in automating some of the testing process for this domain, there exists little automated support for verifying that the executed test cases produce expected results. We assist in this tedious task by presenting a suite of automated oracle comparators for testing Web applications. To effectively identify failures, each comparator is specialized to particular characteristics of the possibly nondeterministic Web applications' output in the form of HTML responses. We also describe combinations of comparators designed to achieve both high precision and recall in failure detection and a tool for helping testers to analyze the output of multiple oracles in detail. We present results from an evaluation of the effectiveness and costs of the oracle comparators. We also provide recommendations to testers on applying effective oracle comparators based on their application's characteristics.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"04 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127376091","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Coordinated Atomic Actions (CAAs) have been introduced about ten years ago as a conceptual framework for developing fault-tolerant concurrent systems. All the work done since then extended the CAA framework with the capabilities to model, verify, and implement concurrent distributed systems following pre-defined development methodologies. As a result, CAAs, compared to other approaches available, offer a rich set of means for engineering dependable systems. Nevertheless, it is sometimes difficult to have a global and analytical view of all the features available as this concept provides a number of features which need to be applied in combination. The main contribution of this paper is in presenting a complete state-of-the-art overview of the work done around CAAs from the three perspectives: the definitions of the fundamental concepts, their various semantics and the means supporting formal verification. This paper is useful for the potential CAAs users in helping them to avoid misinterpretation when employing all the available features. Finally, our paper should contribute in better understanding of the likely directions in which the CAA framework may evolve in the near future.
{"title":"Coordinated Atomic Actions for Dependable Distributed Systems: the Current State in Concepts, Semantics and Verification Means","authors":"B. Gallina, N. Guelfi, A. Romanovsky","doi":"10.1109/ISSRE.2007.10","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.10","url":null,"abstract":"Coordinated Atomic Actions (CAAs) have been introduced about ten years ago as a conceptual framework for developing fault-tolerant concurrent systems. All the work done since then extended the CAA framework with the capabilities to model, verify, and implement concurrent distributed systems following pre-defined development methodologies. As a result, CAAs, compared to other approaches available, offer a rich set of means for engineering dependable systems. Nevertheless, it is sometimes difficult to have a global and analytical view of all the features available as this concept provides a number of features which need to be applied in combination. The main contribution of this paper is in presenting a complete state-of-the-art overview of the work done around CAAs from the three perspectives: the definitions of the fundamental concepts, their various semantics and the means supporting formal verification. This paper is useful for the potential CAAs users in helping them to avoid misinterpretation when employing all the available features. Finally, our paper should contribute in better understanding of the likely directions in which the CAA framework may evolve in the near future.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"128 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124223075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper is the second in a series of empirical studies about requirement error abstraction and classification as a quality improvement approach. The Requirement error abstraction and classification method supports the developers' effort in efficiently identifying the root cause of requirements faults. By uncovering the source of faults, the developers can locate and remove additional related faults that may have been overlooked, thereby improving the quality and reliability of the resulting system. This study is a replication of an earlier study that adds a control group to address a major validity threat. The approach studied includes a process for abstracting errors from faults and provides a requirement error taxonomy for organizing those errors. A unique aspect of this work is the use of research from human cognition to improve the process. The results of the replication are presented and compared with the results from the original study. Overall, the results from this study indicate that the error abstraction and classification approach improves the effectiveness and efficiency of inspectors. The requirement error taxonomy is viewed favorably and provides useful insights into the source of faults. In addition, human cognition research is shown to be an important factor that affects the performance of the inspectors. This study also provides additional evidence to motivate further research.
{"title":"Requirement Error Abstraction and Classification: A Control Group Replicated Study","authors":"G. Walia, Jeffrey C. Carver, T. Philip","doi":"10.1109/ISSRE.2007.14","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.14","url":null,"abstract":"This paper is the second in a series of empirical studies about requirement error abstraction and classification as a quality improvement approach. The Requirement error abstraction and classification method supports the developers' effort in efficiently identifying the root cause of requirements faults. By uncovering the source of faults, the developers can locate and remove additional related faults that may have been overlooked, thereby improving the quality and reliability of the resulting system. This study is a replication of an earlier study that adds a control group to address a major validity threat. The approach studied includes a process for abstracting errors from faults and provides a requirement error taxonomy for organizing those errors. A unique aspect of this work is the use of research from human cognition to improve the process. The results of the replication are presented and compared with the results from the original study. Overall, the results from this study indicate that the error abstraction and classification approach improves the effectiveness and efficiency of inspectors. The requirement error taxonomy is viewed favorably and provides useful insights into the source of faults. In addition, human cognition research is shown to be an important factor that affects the performance of the inspectors. This study also provides additional evidence to motivate further research.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123541785","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper empirically compares malicious traffic originating inside an organization (i.e., internal traffic) with malicious traffic originating outside an organization (i.e., external traffic). Two honeypot target computers were deployed to collect malicious traffic data over a period of fifteen weeks. In the first study we showed that there was a weak correlation between internal and external traffic based on the number of malicious connections. Since the type of malicious activity is linked to the port that was targeted, we focused on the most frequently targeted ports. We observed that internal malicious traffic often contained different malicious content compared to that of external traffic. In the third study, we discovered that the volume of malicious traffic was linked to the day of the week. We showed that internal and external malicious activities differ: where the external malicious activity is quite stable over the week, the internal traffic varied as a function of the users' activity profile.
{"title":"A Comparison between Internal and External Malicious Traffic","authors":"M. Cukier, S. Panjwani","doi":"10.1109/ISSRE.2007.32","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.32","url":null,"abstract":"This paper empirically compares malicious traffic originating inside an organization (i.e., internal traffic) with malicious traffic originating outside an organization (i.e., external traffic). Two honeypot target computers were deployed to collect malicious traffic data over a period of fifteen weeks. In the first study we showed that there was a weak correlation between internal and external traffic based on the number of malicious connections. Since the type of malicious activity is linked to the port that was targeted, we focused on the most frequently targeted ports. We observed that internal malicious traffic often contained different malicious content compared to that of external traffic. In the third study, we discovered that the volume of malicious traffic was linked to the day of the week. We showed that internal and external malicious activities differ: where the external malicious activity is quite stable over the week, the internal traffic varied as a function of the users' activity profile.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"14 9","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131879967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Software reliability modeling techniques have been touted as way of measuring and tracking software systems reliability. However a number of issues make it difficult to use and apply these models in practice. In this paper we show some of the challenges and issues that we have encountered in applying these techniques to track and predict a networking software system reliability behavior at two different stages of its life cycle. Through our case study we show some of the practical solutions we have adopted to overcome these challenges. We also try to establish a relationship between the software testing phase based reliability prediction and field software reliability measurement in order to derive a systematic tracking approach.
{"title":"Measuring Software Reliability in Practice: An Industry Case Study","authors":"S. Benlarbi, David Stortz","doi":"10.1109/ISSRE.2007.33","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.33","url":null,"abstract":"Software reliability modeling techniques have been touted as way of measuring and tracking software systems reliability. However a number of issues make it difficult to use and apply these models in practice. In this paper we show some of the challenges and issues that we have encountered in applying these techniques to track and predict a networking software system reliability behavior at two different stages of its life cycle. Through our case study we show some of the practical solutions we have adopted to overcome these challenges. We also try to establish a relationship between the software testing phase based reliability prediction and field software reliability measurement in order to derive a systematic tracking approach.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120991389","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computer and network security incidents have increasing financial consequences as demand for network accessibility and connectivity to resources continues to rise. These security incidents can lead to direct financial losses either through data theft of personal and/or proprietary information as well as a reputational damage which may negatively impact stock prices or consumer confidence in a company. This paper examines a large set of security incident data using tools from the software reliability community. We look at applying Non-Homogenous Poisson Process (NHPP) models as a method for describing the reliability growth process. We examine the full set of incidents as well as subsets of the data based on incident types. We look at using the Laplace test to guide selection of the appropriate models. Then, based on the trend results, we apply various NHPP models (i.e., Goel-Okumutu, S-Shaped, Duane, and K-Stage Curve) to illustrate the relevance of using these models to fit the incident data and to predict future incidents.
{"title":"Applying Software Reliability Models on Security Incidents","authors":"Edward M. Condon, M. Cukier, T. He","doi":"10.1109/ISSRE.2007.29","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.29","url":null,"abstract":"Computer and network security incidents have increasing financial consequences as demand for network accessibility and connectivity to resources continues to rise. These security incidents can lead to direct financial losses either through data theft of personal and/or proprietary information as well as a reputational damage which may negatively impact stock prices or consumer confidence in a company. This paper examines a large set of security incident data using tools from the software reliability community. We look at applying Non-Homogenous Poisson Process (NHPP) models as a method for describing the reliability growth process. We examine the full set of incidents as well as subsets of the data based on incident types. We look at using the Laplace test to guide selection of the appropriate models. Then, based on the trend results, we apply various NHPP models (i.e., Goel-Okumutu, S-Shaped, Duane, and K-Stage Curve) to illustrate the relevance of using these models to fit the incident data and to predict future incidents.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116868717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：