Software metrics are often supposed to give valuable information for the development of software. In this paper we focus on several common internal metrics: Lines of Code, number of comments, Halstead Volume and McCabe's Cyclomatic Complexity. We try to find relations between these internal software metrics and metrics of software dependability: Probability of Failure on Demand and number of defects. The research is done using 59 specifications from a programming competition---The Online Judge--on the internet. Each specification provides us between 111 and 11,495programs for our analysis; the total number of programs used is 71,917. We excluded those programs that consist of a look-up table. The results for the Online Judge programs are: (1) there is a very strong correlation between Lines of Code and Hal- stead Volume; (2) there is an even stronger correlation between Lines of Code and McCabe's Cyclomatic Complexity; (3) none of the internal software metrics makes it possible to discern correct programs from incorrect ones; (4) given a specification, there is no correlation between any of the internal software metrics and the software dependability metrics.
{"title":"Correlations between Internal Software Metrics and Software Dependability in a Large Population of Small C/C++ Programs","authors":"M. V. D. Meulen, M. Revilla","doi":"10.1109/ISSRE.2007.12","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.12","url":null,"abstract":"Software metrics are often supposed to give valuable information for the development of software. In this paper we focus on several common internal metrics: Lines of Code, number of comments, Halstead Volume and McCabe's Cyclomatic Complexity. We try to find relations between these internal software metrics and metrics of software dependability: Probability of Failure on Demand and number of defects. The research is done using 59 specifications from a programming competition---The Online Judge--on the internet. Each specification provides us between 111 and 11,495programs for our analysis; the total number of programs used is 71,917. We excluded those programs that consist of a look-up table. The results for the Online Judge programs are: (1) there is a very strong correlation between Lines of Code and Hal- stead Volume; (2) there is an even stronger correlation between Lines of Code and McCabe's Cyclomatic Complexity; (3) none of the internal software metrics makes it possible to discern correct programs from incorrect ones; (4) given a specification, there is no correlation between any of the internal software metrics and the software dependability metrics.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116434536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Using a specific machine learning technique, this paper proposes a way to identify suspicious statements during debugging. The technique is based on principles similar to Tarantula but addresses its main flaw: its difficulty to deal with the presence of multiple faults as it assumes that failing test cases execute the same fault(s). The improvement we present in this paper results from the use of C4.5 decision trees to identify various failure conditions based on information regarding the test cases' inputs and outputs. Failing test cases executing under similar conditions are then assumed to fail due to the same fault(s). Statements are then considered suspicious if they are covered by a large proportion of failing test cases that execute under similar conditions. We report on a case study that demonstrates improvement over the original Tarantula technique in terms of statement ranking. Another contribution of this paper is to show that failure conditions as modeled by a C4.5 decision tree accurately predict failures and can therefore be used as well to help debugging.
{"title":"Using Machine Learning to Support Debugging with Tarantula","authors":"L. Briand, Y. Labiche, Xuetao Liu","doi":"10.1109/ISSRE.2007.31","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.31","url":null,"abstract":"Using a specific machine learning technique, this paper proposes a way to identify suspicious statements during debugging. The technique is based on principles similar to Tarantula but addresses its main flaw: its difficulty to deal with the presence of multiple faults as it assumes that failing test cases execute the same fault(s). The improvement we present in this paper results from the use of C4.5 decision trees to identify various failure conditions based on information regarding the test cases' inputs and outputs. Failing test cases executing under similar conditions are then assumed to fail due to the same fault(s). Statements are then considered suspicious if they are covered by a large proportion of failing test cases that execute under similar conditions. We report on a case study that demonstrates improvement over the original Tarantula technique in terms of statement ranking. Another contribution of this paper is to show that failure conditions as modeled by a C4.5 decision tree accurately predict failures and can therefore be used as well to help debugging.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134364624","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
P. Bishop, Ilir Gashi, B. Littlewood, David Wright
Fault tolerance via design diversity is often the only viable way of achieving sufficient dependability levels when using off-the-shelf components. We have reported previously on studies with bug reports of four open-source and commercial off-the-shelf database servers and later release of two of them. The results were very promising for designers of fault-tolerant solutions that wish to employ diverse servers: very few bugs caused failures in more than one server and none caused failure in more than two. In this paper we offer details of two approaches we have studied to construct reliability growth models for a 1-out-of-2 fault-tolerant server which utilize the bug reports. The models presented are of practical significance to system designers wishing to employ diversity with off-the-shelf components since often the bug reports are the only direct dependability evidence available to them.
{"title":"Reliability Modeling of a 1-Out-Of-2 System: Research with Diverse Off-The-Shelf SQL Database Servers","authors":"P. Bishop, Ilir Gashi, B. Littlewood, David Wright","doi":"10.1109/ISSRE.2007.16","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.16","url":null,"abstract":"Fault tolerance via design diversity is often the only viable way of achieving sufficient dependability levels when using off-the-shelf components. We have reported previously on studies with bug reports of four open-source and commercial off-the-shelf database servers and later release of two of them. The results were very promising for designers of fault-tolerant solutions that wish to employ diverse servers: very few bugs caused failures in more than one server and none caused failure in more than two. In this paper we offer details of two approaches we have studied to construct reliability growth models for a 1-out-of-2 fault-tolerant server which utilize the bug reports. The models presented are of practical significance to system designers wishing to employ diversity with off-the-shelf components since often the bug reports are the only direct dependability evidence available to them.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133422482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Enterprise systems must guarantee high availability and reliability to provide 24/7 services without interruptions and failures. Mechanisms for handling exceptional cases and implementing fault tolerance techniques can reduce failure occurrences, and increase dependability. Most of such mechanisms address major problems that lead to unexpected service termination or crashes, but do not deal with many subtle domain dependent failures that do not necessarily cause service termination or crashes, but result in incorrect results. In this paper, we propose a technique for developing selfprotecting systems. The technique proposed in this paper observes values at relevant program points. When the technique detects a software failure, it uses the collected information to identify the execution contexts that lead to the failure, and automatically enables mechanisms for preventing future occurrences of failures of the same type. Thus, failures do not occur again after the first detection of a failure of the same type.
{"title":"Towards Self-Protecting Enterprise Applications","authors":"Davide Lorenzoli, L. Mariani, M. Pezzè","doi":"10.1109/ISSRE.2007.21","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.21","url":null,"abstract":"Enterprise systems must guarantee high availability and reliability to provide 24/7 services without interruptions and failures. Mechanisms for handling exceptional cases and implementing fault tolerance techniques can reduce failure occurrences, and increase dependability. Most of such mechanisms address major problems that lead to unexpected service termination or crashes, but do not deal with many subtle domain dependent failures that do not necessarily cause service termination or crashes, but result in incorrect results. In this paper, we propose a technique for developing selfprotecting systems. The technique proposed in this paper observes values at relevant program points. When the technique detects a software failure, it uses the collected information to identify the execution contexts that lead to the failure, and automatically enables mechanisms for preventing future occurrences of failures of the same type. Thus, failures do not occur again after the first detection of a failure of the same type.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"191 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123382621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Operational or "beta" testing of software has a number of benefits for software vendors and has become common industry practice. However, ordinary users are more likely to overlook or misreport software problems than experienced software testers are. To compensate for this shortcoming, we present a technique called corroboration-based filtering for corroborating user assessments of individual operational executions for which audit information has been captured for possible offline review. Independent assessments concerning similar executions are pooled by automatically clustering together executions with similar execution profiles. Executions are chosen for review based on their user assessments, the size of the cluster each execution belongs to, and whether the cluster has already been confirmed by developers to contain an actual failure. We explain the rationale for this technique, analyze it probabilistically, and present the results of empirically comparing it to alternative techniques.
{"title":"Corroborating User Assessments of Software Behavior to Facilitate Operational Testing","authors":"Vinay Augustine, Andy Podgurski","doi":"10.1109/ISSRE.2007.30","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.30","url":null,"abstract":"Operational or \"beta\" testing of software has a number of benefits for software vendors and has become common industry practice. However, ordinary users are more likely to overlook or misreport software problems than experienced software testers are. To compensate for this shortcoming, we present a technique called corroboration-based filtering for corroborating user assessments of individual operational executions for which audit information has been captured for possible offline review. Independent assessments concerning similar executions are pooled by automatically clustering together executions with similar execution profiles. Executions are chosen for review based on their user assessments, the size of the cluster each execution belongs to, and whether the cluster has already been confirmed by developers to contain an actual failure. We explain the rationale for this technique, analyze it probabilistically, and present the results of empirically comparing it to alternative techniques.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125594972","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper describes a study performed in an industrial setting that attempts to build predictive models to identify parts of a Java system with a high fault probability. The system under consideration is constantly evolving as several releases a year are shipped to customers. Developers usually have limited resources for their testing and inspections and would like to be able to devote extra resources to faulty system parts. The main research focus of this paper is two-fold: (1) use and compare many data mining and machine learning techniques to build fault-proneness models based mostly on source code measures and change/fault history data, and (2) demonstrate that the usual classification evaluation criteria based on confusion matrices may not be fully appropriate to compare and evaluate models.
{"title":"Data Mining Techniques for Building Fault-proneness Models in Telecom Java Software","authors":"E. Arisholm, L. Briand, M. Fuglerud","doi":"10.1109/ISSRE.2007.22","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.22","url":null,"abstract":"This paper describes a study performed in an industrial setting that attempts to build predictive models to identify parts of a Java system with a high fault probability. The system under consideration is constantly evolving as several releases a year are shipped to customers. Developers usually have limited resources for their testing and inspections and would like to be able to devote extra resources to faulty system parts. The main research focus of this paper is two-fold: (1) use and compare many data mining and machine learning techniques to build fault-proneness models based mostly on source code measures and change/fault history data, and (2) demonstrate that the usual classification evaluation criteria based on confusion matrices may not be fully appropriate to compare and evaluate models.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"2019 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126850156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
To measure the reliability of a website from a user's point of view, the uncertainly on the usage of the website has to be taken into account. In this paper we investigate the influence of this uncertainly on the reliability estimate for a web server. For this purpose a session based Markov model is used to model the usage extracted from the server's logfiles. From these logfiles a complete user profile can be extracted together with an estimate of the uncertainty on this user profile. This paper investigates the applicability of this kind of Markov model on web server reliability and discusses the difficulties with data extraction from the logfiles. Advantages and disadvantages of this approach are discussed and the approach is applied to data from a university department's web server to demonstrate its applicability.
{"title":"Sensitivity of Website Reliability to Usage Profile Changes","authors":"Kim Weyns, Martin Höst","doi":"10.1109/ISSRE.2007.9","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.9","url":null,"abstract":"To measure the reliability of a website from a user's point of view, the uncertainly on the usage of the website has to be taken into account. In this paper we investigate the influence of this uncertainly on the reliability estimate for a web server. For this purpose a session based Markov model is used to model the usage extracted from the server's logfiles. From these logfiles a complete user profile can be extracted together with an estimate of the uncertainty on this user profile. This paper investigates the applicability of this kind of Markov model on web server reliability and discusses the difficulties with data extraction from the logfiles. Advantages and disadvantages of this approach are discussed and the approach is applied to data from a university department's web server to demonstrate its applicability.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129820499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents statistical inference of computer virus propagation using non-homogeneous Poisson processes (NHPPs). Under some mathematical assumptions, the number of infected hosts can be modeled by an NHPP In particular, this paper applies a framework of mixed-type NHPPs to the statistical inference of periodic virus propagation. The mixed-type NHPP is defined by a superposition of NHPPs. In numerical experiments, we examine a goodness-of-fit criterion of NHPPs on fitting to real virus infection data, and discuss the effectiveness of the model-based prediction approach for computer virus propagation.
{"title":"Statistical Inference of Computer Virus Propagation Using Non-Homogeneous Poisson Processes","authors":"H. Okamura, K. Tateishi, T. Dohi","doi":"10.1109/ISSRE.2007.28","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.28","url":null,"abstract":"This paper presents statistical inference of computer virus propagation using non-homogeneous Poisson processes (NHPPs). Under some mathematical assumptions, the number of infected hosts can be modeled by an NHPP In particular, this paper applies a framework of mixed-type NHPPs to the statistical inference of periodic virus propagation. The mixed-type NHPP is defined by a superposition of NHPPs. In numerical experiments, we examine a goodness-of-fit criterion of NHPPs on fitting to real virus infection data, and discuss the effectiveness of the model-based prediction approach for computer virus propagation.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"150 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125885401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peter J. Clarke, Djuradj Babich, Tariq M. King, James F. Power
One of the characteristics of the increasingly widespread use of object-oriented libraries and the resulting intensive use of inheritance is the proliferation of dependencies on abstract classes. Such classes defer the implementation of some features, and are typically used as a specification or design tool. However, since their features are not fully implemented, abstract classes cannot be instantiated, and thus pose challenges for execution-based testing strategies. This paper presents a structured approach that supports the testing of features in abstract classes. Core to the approach is a series of static analysis steps that build a comprehensive view of the inter-class dependencies in the system under test. We then leveraged this information to define a test order for the methods in an abstract class that minimizes the number of stubs required during testing, and clearly identifies the required functionality of these stubs. Our approach is based on a comprehensive taxonomy of object-oriented classes that provides a framework for our analysis. First we describe the algorithms to calculate the inter-class dependencies and the test-order that minimizes stub creation. Then we give an overview of our tool, AbstractTestJ that implements our approach by generating a test order for the methods in an abstract Java class. Finally, we harness this tool to provide an analysis of 12 substantial Java applications that demonstrates both the feasibility of our approach and the importance of this technique.
{"title":"Intra-Class Testing of Abstract Class Features","authors":"Peter J. Clarke, Djuradj Babich, Tariq M. King, James F. Power","doi":"10.1109/ISSRE.2007.11","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.11","url":null,"abstract":"One of the characteristics of the increasingly widespread use of object-oriented libraries and the resulting intensive use of inheritance is the proliferation of dependencies on abstract classes. Such classes defer the implementation of some features, and are typically used as a specification or design tool. However, since their features are not fully implemented, abstract classes cannot be instantiated, and thus pose challenges for execution-based testing strategies. This paper presents a structured approach that supports the testing of features in abstract classes. Core to the approach is a series of static analysis steps that build a comprehensive view of the inter-class dependencies in the system under test. We then leveraged this information to define a test order for the methods in an abstract class that minimizes the number of stubs required during testing, and clearly identifies the required functionality of these stubs. Our approach is based on a comprehensive taxonomy of object-oriented classes that provides a framework for our analysis. First we describe the algorithms to calculate the inter-class dependencies and the test-order that minimizes stub creation. Then we give an overview of our tool, AbstractTestJ that implements our approach by generating a test order for the methods in an abstract Java class. Finally, we harness this tool to provide an analysis of 12 substantial Java applications that demonstrates both the feasibility of our approach and the importance of this technique.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"2512 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131333114","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiangrong Wang, Hang Shi, Tze-Yau William Huang, F. C. Lin
Product security is an on-going challenge for network equipment vendors. In this paper, we present a systematic methodology for some software vulnerability assessment and security function verification. Based on this approach, a scalable and adaptable automatic test system was implemented to test over a hundred production software releases over the past year. This paper describes the methodology, the framework, and the results.
{"title":"Integrated Software Vulnerability and Security Functionality Assessment","authors":"Xiangrong Wang, Hang Shi, Tze-Yau William Huang, F. C. Lin","doi":"10.1109/ISSRE.2007.20","DOIUrl":"https://doi.org/10.1109/ISSRE.2007.20","url":null,"abstract":"Product security is an on-going challenge for network equipment vendors. In this paper, we present a systematic methodology for some software vulnerability assessment and security function verification. Based on this approach, a scalable and adaptable automatic test system was implemented to test over a hundred production software releases over the past year. This paper describes the methodology, the framework, and the results.","PeriodicalId":193805,"journal":{"name":"The 18th IEEE International Symposium on Software Reliability (ISSRE '07)","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116169617","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：