Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173244
P. Netisopakul, L. White, John Morris, D. Hoffman
For the testing of container classes and the algorithms or programs that operate on the data in a container, these data have the property of being homogeneous throughout the container. We have developed an approach for this situation called data coverage testing, where automated test generation can systematically generate increasing test data size. Given a program and a test model, it can be theoretically shown that there exists a sufficiently large test data set size N, such that testing with a data set size larger than N does not detect more faults. A number of experiments have been conducted using a set of C++ STL programs, comparing data coverage testing with two other testing strategies: statement coverage and random generation. These experiments validate the theoretical analysis for data coverage, confirming the predicted sufficiently large N for each program.
{"title":"Data coverage testing of programs for container classes","authors":"P. Netisopakul, L. White, John Morris, D. Hoffman","doi":"10.1109/ISSRE.2002.1173244","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173244","url":null,"abstract":"For the testing of container classes and the algorithms or programs that operate on the data in a container, these data have the property of being homogeneous throughout the container. We have developed an approach for this situation called data coverage testing, where automated test generation can systematically generate increasing test data size. Given a program and a test model, it can be theoretically shown that there exists a sufficiently large test data set size N, such that testing with a data set size larger than N does not detect more faults. A number of experiments have been conducted using a set of C++ STL programs, comparing data coverage testing with two other testing strategies: statement coverage and random generation. These experiments validate the theoretical analysis for data coverage, confirming the predicted sufficiently large N for each program.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128452628","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173211
Wen-li Wang, Mei-Hwa Chen
A number of Markov-based software reliability models have been developed for measuring software reliability. However, the application of these models is strictly limited to software that satisfies the Markov properties. The objective of our work is to expand the application domain of the Markov-based models, so that most software can be modeled and software reliability can be measured at the architecture level. To overcome the limitations of Markov properties, our model takes execution history into account and addresses both deterministic and probabilistic software behaviors. Each state represents the executions of one or more components depending on the architectural styles. In addition, the executions of one component are depicted by using distinctive states, when such executions are influenced by past states. Furthermore, we construct loops to eliminate the likelihood of unlimited state expansion and utilize a binomial tree structure to account for all the different execution paths. We show that Markov models are applicable even to software that does not fully satisfy the Markov properties. Therefore, we significantly improve the state of the art in architecture-based software reliability modeling.
{"title":"Heterogeneous software reliability modeling","authors":"Wen-li Wang, Mei-Hwa Chen","doi":"10.1109/ISSRE.2002.1173211","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173211","url":null,"abstract":"A number of Markov-based software reliability models have been developed for measuring software reliability. However, the application of these models is strictly limited to software that satisfies the Markov properties. The objective of our work is to expand the application domain of the Markov-based models, so that most software can be modeled and software reliability can be measured at the architecture level. To overcome the limitations of Markov properties, our model takes execution history into account and addresses both deterministic and probabilistic software behaviors. Each state represents the executions of one or more components depending on the architectural styles. In addition, the executions of one component are depicted by using distinctive states, when such executions are influenced by past states. Furthermore, we construct loops to eliminate the likelihood of unlimited state expansion and utilize a binomial tree structure to account for all the different execution paths. We show that Markov models are applicable even to software that does not fully satisfy the Markov properties. Therefore, we significantly improve the state of the art in architecture-based software reliability modeling.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"102 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116118276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173221
C. Fetzer, Zhen Xiao
Improving the dependability of computer systems is increasingly important as more and more of our lives depend on the availability of such systems. Wrapping dynamic link libraries is an effective approach for improving the reliability and security of computer software without source code access. We describe a flexible framework to generate a rich set of software wrappers for shared libraries. We describe the architecture of the wrapper generator, the problems of how to generate wrappers efficiently, and our solutions to these problems. Based on a set of properties declared for a function, the generator can create a variety of wrappers to suit the diverse requirements of application programs. Performance measurements indicate that the overhead of the generated wrappers is small.
{"title":"A flexible generator architecture for improving software dependability","authors":"C. Fetzer, Zhen Xiao","doi":"10.1109/ISSRE.2002.1173221","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173221","url":null,"abstract":"Improving the dependability of computer systems is increasingly important as more and more of our lives depend on the availability of such systems. Wrapping dynamic link libraries is an effective approach for improving the reliability and security of computer software without source code access. We describe a flexible framework to generate a rich set of software wrappers for shared libraries. We describe the architecture of the wrapper generator, the problems of how to generate wrappers efficiently, and our solutions to these problems. Based on a set of properties declared for a function, the generator can create a variety of wrappers to suit the diverse requirements of application programs. Performance measurements indicate that the overhead of the generated wrappers is small.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127599638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173274
P. Bishop, R. Bloomfield
In this paper we extend an earlier worst case bound reliability theory to derive a worst case reliability function R(t), which gives the worst case probability of surviving a further time t given an estimate of residual defects in the software N and a prior test time T. The earlier theory and its extension are presented and the paper also considers the case where there is a low probability of any defect existing in the program. For the "fractional defect" case, there can be a high probability of surviving any subsequent time t. The implications of the theory are discussed and compared with alternative reliability models.
{"title":"Worst case reliability prediction based on a prior estimate of residual defects","authors":"P. Bishop, R. Bloomfield","doi":"10.1109/ISSRE.2002.1173274","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173274","url":null,"abstract":"In this paper we extend an earlier worst case bound reliability theory to derive a worst case reliability function R(t), which gives the worst case probability of surviving a further time t given an estimate of residual defects in the software N and a prior test time T. The earlier theory and its extension are presented and the paper also considers the case where there is a low probability of any defect existing in the program. For the \"fractional defect\" case, there can be a high probability of surviving any subsequent time t. The implications of the theory are discussed and compared with alternative reliability models.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131209693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173225
J. Michael, Bernard J. Bossuyt, Byron B. Snyder
The levels of quality, maintainability, testability, and stability of software can be improved and measured through the use of automated testing tools throughout the software development process. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects of the software-testing task Automated testing tools vary in their underlying approach, quality, and ease-of-use, among other characteristics. In this paper we propose a suite of objective metrics for measuring tool characteristics, as an aid in systematically evaluating and selecting automated testing tools.
{"title":"Metrics for measuring the effectiveness of software-testing tools","authors":"J. Michael, Bernard J. Bossuyt, Byron B. Snyder","doi":"10.1109/ISSRE.2002.1173225","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173225","url":null,"abstract":"The levels of quality, maintainability, testability, and stability of software can be improved and measured through the use of automated testing tools throughout the software development process. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects of the software-testing task Automated testing tools vary in their underlying approach, quality, and ease-of-use, among other characteristics. In this paper we propose a suite of objective metrics for measuring tool characteristics, as an aid in systematically evaluating and selecting automated testing tools.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121902590","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173206
Grégory Lestiennes, M. Gaudel
Deriving test cases from formal specifications of communicating processes has been studied for awhile. Several methods have been proposed for specifications based on FSM (Finite State Machines), LTS (Labelled Transition Systems), IOTS (Input Output Transition Systems), etc. However, most approaches are limited to a finite set of actions, excluding the possibility of communicating typed values between processes. This article presents a test derivation and selection method based on a model of communicating processes with inputs, outputs and data types, which is closer to actual implementations of communication protocols.
{"title":"Testing processes from formal specifications with inputs, outputs and data types","authors":"Grégory Lestiennes, M. Gaudel","doi":"10.1109/ISSRE.2002.1173206","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173206","url":null,"abstract":"Deriving test cases from formal specifications of communicating processes has been studied for awhile. Several methods have been proposed for specifications based on FSM (Finite State Machines), LTS (Labelled Transition Systems), IOTS (Input Output Transition Systems), etc. However, most approaches are limited to a finite set of actions, excluding the possibility of communicating typed values between processes. This article presents a test derivation and selection method based on a model of communicating processes with inputs, outputs and data types, which is closer to actual implementations of communication protocols.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129419102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173217
R. Klemm, Navjot Singh
Many systems and techniques exist for detecting application failures. However, previously known generic failure detection solutions are only of limited use for Java applications because they do not take into consideration the specifics of the Java language and the Java execution environment. In this article, we present the application-independent Java Application Supervisor (JAS). JAS can automatically detect, log, and resolve a variety of execution problems and failures in Java applications. In most cases, JAS requires neither modifications nor access to the source code of the supervised application. A set of simple user-specified policies guides the failure detection, logging, and recovery process in JAS. A JAS configuration manager automatically generates default policies from the bytecode of an application. The user can modify these default policies as needed. Our experimental studies show that JAS typically incurs little execution time and memory overhead for the target application. We describe an experiment with a Web proxy that exhibits reliability and performance problems under heavy load and demonstrate an increase in the rate of successful requests to the server by almost 33% and a decrease in the average request processing time by approximately 22% when using JAS.
{"title":"Automatic failure detection, logging, and recovery for high-availability Java servers","authors":"R. Klemm, Navjot Singh","doi":"10.1109/ISSRE.2002.1173217","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173217","url":null,"abstract":"Many systems and techniques exist for detecting application failures. However, previously known generic failure detection solutions are only of limited use for Java applications because they do not take into consideration the specifics of the Java language and the Java execution environment. In this article, we present the application-independent Java Application Supervisor (JAS). JAS can automatically detect, log, and resolve a variety of execution problems and failures in Java applications. In most cases, JAS requires neither modifications nor access to the source code of the supervised application. A set of simple user-specified policies guides the failure detection, logging, and recovery process in JAS. A JAS configuration manager automatically generates default policies from the bytecode of an application. The user can modify these default policies as needed. Our experimental studies show that JAS typically incurs little execution time and memory overhead for the target application. We describe an experiment with a Web proxy that exhibits reliability and performance problems under heavy load and demonstrate an increase in the rate of successful requests to the server by almost 33% and a decrease in the average request processing time by approximately 22% when using JAS.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125723910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173299
J. Munson, A. Nikora
An important aspect of developing models relating the number and type of faults in a software system to a set of structural measurement is defining what constitutes a fault. By definition, a fault is a structural imperfection in a software system that may lead to the system's eventually failing. A measurable and precise definition of what faults are makes it possible to accurately identify and count them, which in turn allows the formulation of models relating fault counts and types to other measurable attributes of a software system. Unfortunately, the most widely-used definitions are not measurable; there is no guarantee that two different individuals looking at the same set of failure reports and the same set of fault definitions will count the same number of underlying faults. The incomplete and ambiguous nature of current fault definitions adds a noise component to the inputs used in modeling fault content. If this noise component is sufficiently large, any attempt to develop a fault model will produce invalid results. As part of our on-going work in modeling software faults, we have developed a method of unambiguously identifying and counting faults. Specifically, we base our recognition and enumeration of software faults on the grammar of the language of the software system. By tokenizing the differences between a version of the system exhibiting a particular failure behavior, and the version in which changes were made to eliminate that behavior, we are able to unambiguously count the number of faults associated with that failure. With modern configuration management tools, the identification and counting of software faults can be automated.
{"title":"Toward a quantifiable definition of software faults","authors":"J. Munson, A. Nikora","doi":"10.1109/ISSRE.2002.1173299","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173299","url":null,"abstract":"An important aspect of developing models relating the number and type of faults in a software system to a set of structural measurement is defining what constitutes a fault. By definition, a fault is a structural imperfection in a software system that may lead to the system's eventually failing. A measurable and precise definition of what faults are makes it possible to accurately identify and count them, which in turn allows the formulation of models relating fault counts and types to other measurable attributes of a software system. Unfortunately, the most widely-used definitions are not measurable; there is no guarantee that two different individuals looking at the same set of failure reports and the same set of fault definitions will count the same number of underlying faults. The incomplete and ambiguous nature of current fault definitions adds a noise component to the inputs used in modeling fault content. If this noise component is sufficiently large, any attempt to develop a fault model will produce invalid results. As part of our on-going work in modeling software faults, we have developed a method of unambiguously identifying and counting faults. Specifically, we base our recognition and enumeration of software faults on the grammar of the language of the software system. By tokenizing the differences between a version of the system exhibiting a particular failure behavior, and the version in which changes were made to eliminate that behavior, we are able to unambiguously count the number of faults associated with that failure. With modern configuration management tools, the identification and counting of software faults can be automated.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116562488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173241
H. Okamura, Satoshi Miyahara, T. Dohi
Long running software systems are known to experience an aging phenomenon called software aging, one in which the accumulation of errors during the execution of software leads to performance degradation and eventually results in failure. To counteract this phenomenon an active fault management approach, called software rejuvenation, is particularly useful. It essentially involves gracefully terminating an application or a system and restarting it in a clean internal state. We deal with dependability analysis of a client/server software system with rejuvenation. Three dependability measures in the server process, steady-state availability, loss probability of requests and mean response time on tasks, are derived from the well-known hidden Markovian analysis under the time-based software rejuvenation scheme. In numerical examples, we investigate the sensitivity of some model parameters to the dependability measures.
{"title":"Dependability analysis of a client/server software system with rejuvenation","authors":"H. Okamura, Satoshi Miyahara, T. Dohi","doi":"10.1109/ISSRE.2002.1173241","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173241","url":null,"abstract":"Long running software systems are known to experience an aging phenomenon called software aging, one in which the accumulation of errors during the execution of software leads to performance degradation and eventually results in failure. To counteract this phenomenon an active fault management approach, called software rejuvenation, is particularly useful. It essentially involves gracefully terminating an application or a system and restarting it in a clean internal state. We deal with dependability analysis of a client/server software system with rejuvenation. Three dependability measures in the server process, steady-state availability, loss probability of requests and mean response time on tasks, are derived from the well-known hidden Markovian analysis under the time-based software rejuvenation scheme. In numerical examples, we investigate the sensitivity of some model parameters to the dependability measures.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130040747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2002-11-12DOI: 10.1109/ISSRE.2002.1173279
Yu Lei, K. Tai
Existing reachability analysis techniques for asynchronous message-passing programs assume causal communication, which means that messages sent to a destination are received in the order they are sent. In this paper, we present a new reachability analysis approach, called blocking-based simultaneous reachability analysis (BSRA). BSRA can be applied to asynchronous message-passing programs based on any communication scheme. From a global state g, BSRA allows processes to proceed simultaneously until each of them terminates or is ready to execute a receive operation. Global states reached by such executions from g are called next blocking points of g. For each next blocking point of g, waiting messages and receive operations are matched to produce immediate BSRA-based successor states of g. Intermediate global states from g to each of g's immediate BSRA-based successors are not saved. We describe an algorithm for generating BSRA-based reachability, graphs and show that this algorithm guarantees the detection of deadlocks. Our empirical results indicate that BSRA significantly reduces the number of states in reachability graphs. Extensions of BSRA for partial order reduction and model checking are discussed.
{"title":"Blocking-based simultaneous reachability analysis of asynchronous message-passing programs","authors":"Yu Lei, K. Tai","doi":"10.1109/ISSRE.2002.1173279","DOIUrl":"https://doi.org/10.1109/ISSRE.2002.1173279","url":null,"abstract":"Existing reachability analysis techniques for asynchronous message-passing programs assume causal communication, which means that messages sent to a destination are received in the order they are sent. In this paper, we present a new reachability analysis approach, called blocking-based simultaneous reachability analysis (BSRA). BSRA can be applied to asynchronous message-passing programs based on any communication scheme. From a global state g, BSRA allows processes to proceed simultaneously until each of them terminates or is ready to execute a receive operation. Global states reached by such executions from g are called next blocking points of g. For each next blocking point of g, waiting messages and receive operations are matched to produce immediate BSRA-based successor states of g. Intermediate global states from g to each of g's immediate BSRA-based successors are not saved. We describe an algorithm for generating BSRA-based reachability, graphs and show that this algorithm guarantees the detection of deadlocks. Our empirical results indicate that BSRA significantly reduces the number of states in reachability graphs. Extensions of BSRA for partial order reduction and model checking are discussed.","PeriodicalId":159160,"journal":{"name":"13th International Symposium on Software Reliability Engineering, 2002. Proceedings.","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2002-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133701640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}