Pub Date : 2013-11-04DOI: 10.1109/ISSREW.2013.6688924
E. Denney, Ganesh J. Pai
We describe a generic approach for automatically integrating the output generated from a formal method/tool into a software safety assurance case, as an evidence argument, by (a) encoding the underlying reasoning as a safety case pattern, and (b) instantiating it using the data produced from the method/tool. We believe this approach not only improves the trustworthiness of the evidence generated from a formal method/tool, by explicitly presenting the reasoning and mechanisms underlying its genesis, but also provides a way to gauge the suitability of the evidence in the context of the wider assurance case. We illustrate our work by application to a real example-an unmanned aircraft system - where we invoke a formal code analysis tool from its autopilot software safety case, automatically transform the verification output into an evidence argument, and then integrate it into the former.
{"title":"Evidence arguments for using formal methods in software certification","authors":"E. Denney, Ganesh J. Pai","doi":"10.1109/ISSREW.2013.6688924","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688924","url":null,"abstract":"We describe a generic approach for automatically integrating the output generated from a formal method/tool into a software safety assurance case, as an evidence argument, by (a) encoding the underlying reasoning as a safety case pattern, and (b) instantiating it using the data produced from the method/tool. We believe this approach not only improves the trustworthiness of the evidence generated from a formal method/tool, by explicitly presenting the reasoning and mechanisms underlying its genesis, but also provides a way to gauge the suitability of the evidence in the context of the wider assurance case. We illustrate our work by application to a real example-an unmanned aircraft system - where we invoke a formal code analysis tool from its autopilot software safety case, automatically transform the verification output into an evidence argument, and then integrate it into the former.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130334546","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688859
Y. Yokote, Tatsumi Nagayama
This presentation demonstrates an innovative way to build a target system maintaining its dependability in an open system environment, where the boundary of the target system is blurred in the sense that interaction with its surrounding environment is always altered due to several environmental changes such as business objectives, stakeholders' requirements, regulations, and performance requirements. What we call open systems is inherently providing such a nature, and recent IT systems particularly including cloud-based services are categorized in it.
{"title":"Dependability of open systems","authors":"Y. Yokote, Tatsumi Nagayama","doi":"10.1109/ISSREW.2013.6688859","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688859","url":null,"abstract":"This presentation demonstrates an innovative way to build a target system maintaining its dependability in an open system environment, where the boundary of the target system is blurred in the sense that interaction with its surrounding environment is always altered due to several environmental changes such as business objectives, stakeholders' requirements, regulations, and performance requirements. What we call open systems is inherently providing such a nature, and recent IT systems particularly including cloud-based services are categorized in it.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"296 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123117817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688893
Yanqin Huang, Junhua Wu, Yang Feng, Zhenyu Chen, Zhihong Zhao
Spectrum-based Fault Localization (SBFL) techniques use risk evaluation formulas to calculate each statement's likelihood of having a bug based on test results. SBFL can not only be used in statement level, but also can be used with other program entities such as branches, functions and so on. Most previous studies have been conducted under the assumption of a single bug. However, software always contains multi-bugs in practice. A natural idea of debugging is to isolate bugs and then use SBFL techniques to locate one bug for each group. In this paper, we conduct an empirical study on clustering for isolating bugs in fault localization. We analyze the effects of six fault localization techniques and two cluster algorithms. The main observations are: (1) ER5 (Wong1) achieves the best results of fault localization with clustering; (2) K-means outperforms hierarchical clustering for isolating bugs in fault localization.
{"title":"An empirical study on clustering for isolating bugs in fault localization","authors":"Yanqin Huang, Junhua Wu, Yang Feng, Zhenyu Chen, Zhihong Zhao","doi":"10.1109/ISSREW.2013.6688893","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688893","url":null,"abstract":"Spectrum-based Fault Localization (SBFL) techniques use risk evaluation formulas to calculate each statement's likelihood of having a bug based on test results. SBFL can not only be used in statement level, but also can be used with other program entities such as branches, functions and so on. Most previous studies have been conducted under the assumption of a single bug. However, software always contains multi-bugs in practice. A natural idea of debugging is to isolate bugs and then use SBFL techniques to locate one bug for each group. In this paper, we conduct an empirical study on clustering for isolating bugs in fault localization. We analyze the effects of six fault localization techniques and two cluster algorithms. The main observations are: (1) ER5 (Wong1) achieves the best results of fault localization with clustering; (2) K-means outperforms hierarchical clustering for isolating bugs in fault localization.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127102555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688894
Danqing Zhang, Jianhui Jiang, Linbo Chen
Failure clustering is considered as an effective method to alleviate the burden in software development and maintenance stage. However, since the overall software fault space is extremely large, the inherent complexity of the “fault-error-failure” chain becomes an obstacle in failure clustering. In this paper, we present a method of program behavior characterization and clustering which is able to examine and cluster failure behaviors of programs based on their normal executions. We first characterize program executions in order to model runtime behaviors. Then the runtime behaviors are clustered by using a typical fuzzy technique. After that, we evaluate two things: the accuracy of runtime behavior modeling, and the equivalence of a cluster in runtime characterization to that in failure clustering. For the SPEC CPU2000 and SPEC CPU2006 suites of benchmarks, the experimental results and analysis show that our method is effective at clustering similar failure behaviors based on their runtime behavior clustering.
{"title":"Program behavior characterization and clustering: An empirical study for failure clustering","authors":"Danqing Zhang, Jianhui Jiang, Linbo Chen","doi":"10.1109/ISSREW.2013.6688894","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688894","url":null,"abstract":"Failure clustering is considered as an effective method to alleviate the burden in software development and maintenance stage. However, since the overall software fault space is extremely large, the inherent complexity of the “fault-error-failure” chain becomes an obstacle in failure clustering. In this paper, we present a method of program behavior characterization and clustering which is able to examine and cluster failure behaviors of programs based on their normal executions. We first characterize program executions in order to model runtime behaviors. Then the runtime behaviors are clustered by using a typical fuzzy technique. After that, we evaluate two things: the accuracy of runtime behavior modeling, and the equivalence of a cluster in runtime characterization to that in failure clustering. For the SPEC CPU2000 and SPEC CPU2006 suites of benchmarks, the experimental results and analysis show that our method is effective at clustering similar failure behaviors based on their runtime behavior clustering.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125012324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688909
Geetika Goel, A. Roy, R. Ganesan
Field failure analysis is usually driven by a characterization of the different time related properties of failure. This characterization does not help the production support team in understanding the root cause. In order to pinpoint the root cause of failure, one of the most effective techniques used is checking for violations of the system invariants which are the consistent, time invariant correlations that exist in the system. Understanding when and where these violations happen helps in detecting the root cause of the failure. Silent failures, on the other hand are characterized by no evidence of failures either in the console or in the field failure logs. They are unearthed at moments of crisis, either with a customer complaint or other cascading failures. These failures often result in data loss or data corruption, creating many latent errors. Accumulation of these errors over time results in degraded system performance. This represents the problem of software aging and restoration of the system, i.e. its rejuvenation becomes a critical need. Subsequent to the restoration, a rigorous failure detection mechanism is needed to detect them early. What we describe in the paper is a novel method that could be used to detect silent failures using a combination of invariant violation checking and finite state machine based analysis of the system. We use the audit-trail logs of system to extract information about the state and transitions for FSM representation. Currently our research work was limited to proving its efficiency. We applied this approach to our SaaS platform and were able to detect 36 silent failures over a period of 9 months. As next steps, we will implement this as a part of automated failure detection in the operational SaaS platforms.
{"title":"Identifying silent failures of SaaS services using finite state machine based invariant analysis","authors":"Geetika Goel, A. Roy, R. Ganesan","doi":"10.1109/ISSREW.2013.6688909","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688909","url":null,"abstract":"Field failure analysis is usually driven by a characterization of the different time related properties of failure. This characterization does not help the production support team in understanding the root cause. In order to pinpoint the root cause of failure, one of the most effective techniques used is checking for violations of the system invariants which are the consistent, time invariant correlations that exist in the system. Understanding when and where these violations happen helps in detecting the root cause of the failure. Silent failures, on the other hand are characterized by no evidence of failures either in the console or in the field failure logs. They are unearthed at moments of crisis, either with a customer complaint or other cascading failures. These failures often result in data loss or data corruption, creating many latent errors. Accumulation of these errors over time results in degraded system performance. This represents the problem of software aging and restoration of the system, i.e. its rejuvenation becomes a critical need. Subsequent to the restoration, a rigorous failure detection mechanism is needed to detect them early. What we describe in the paper is a novel method that could be used to detect silent failures using a combination of invariant violation checking and finite state machine based analysis of the system. We use the audit-trail logs of system to extract information about the state and transitions for FSM representation. Currently our research work was limited to proving its efficiency. We applied this approach to our SaaS platform and were able to detect 36 silent failures over a period of 9 months. As next steps, we will implement this as a part of automated failure detection in the operational SaaS platforms.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125092343","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688898
Fubin Wu
GessNet™ is the leader in providing risk-driven software solutions and consulting services for medical device risk management, safety assurance case development, FDA submissions, FDA inspections and ISO 13485 quality system audits (http://www.gessnet.com).
{"title":"Integration of software reliability engineering, risk management and safety assurance case","authors":"Fubin Wu","doi":"10.1109/ISSREW.2013.6688898","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688898","url":null,"abstract":"GessNet™ is the leader in providing risk-driven software solutions and consulting services for medical device risk management, safety assurance case development, FDA submissions, FDA inspections and ISO 13485 quality system audits (http://www.gessnet.com).","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"62 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115251874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688889
Zheng Zheng, Yichao Gao, P. Hao, Zhenyu Zhang
In software debugging, statistical fault localization techniques contrast dynamic spectra of program elements to estimate the location of faults in faulty programs. Coincidental correctness may have a negative impact on these techniques because faults can also be triggered in an observed non-failed run and thus disturbs the assessment of fault locations. However, eliminating the confounding relies on the accuracy of recognizing them. This paper makes use of the presence of coincidental correctness as an effective interface to the success of fault localization. We calculate the distribution overlapping of dynamic spectrum in failed runs and in non-failed runs to find out the fault-leading predicates, and further reduce the region by referencing the inter-class distances of the spectra to suppress the less suspicious candidates. Empirical results show that our technique can outperform representative existing predicate-based fault localization techniques.
{"title":"Coincidental correctness: An interference or interface to successful fault localization?","authors":"Zheng Zheng, Yichao Gao, P. Hao, Zhenyu Zhang","doi":"10.1109/ISSREW.2013.6688889","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688889","url":null,"abstract":"In software debugging, statistical fault localization techniques contrast dynamic spectra of program elements to estimate the location of faults in faulty programs. Coincidental correctness may have a negative impact on these techniques because faults can also be triggered in an observed non-failed run and thus disturbs the assessment of fault locations. However, eliminating the confounding relies on the accuracy of recognizing them. This paper makes use of the presence of coincidental correctness as an effective interface to the success of fault localization. We calculate the distribution overlapping of dynamic spectrum in failed runs and in non-failed runs to find out the fault-leading predicates, and further reduce the region by referencing the inter-class distances of the spectra to suppress the less suspicious candidates. Empirical results show that our technique can outperform representative existing predicate-based fault localization techniques.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132278659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688878
A. Tai, C. Walter, L. Fesq, J. Day
Over decades, fault-tree has remained as one of the most widely used fault management (FM) mechanisms by practitioners as a visualization/communication media, as well as a quantitative analysis tool for building dependable systems. Nonetheless, to date fault tree analysis is often unable to provide us with high-confidence results due to potential misrepresentation of the relationships among failure events. In retrospect, catastrophic system failures were almost always caused by a chain of mixed fault manifestation and interaction. Moreover, faults in FM itself may become a critical element in a fault-manifestation chain. The 1996 Ariane-5 rocket self-destruction provides us with a lesson on the devastating consequence of inappropriate FM application. Such prior lessons have inspired us to propose a fault-class-aware and FM-capability-aware fault tree analysis framework.
{"title":"Fault-class-aware fault tree generation and analysis","authors":"A. Tai, C. Walter, L. Fesq, J. Day","doi":"10.1109/ISSREW.2013.6688878","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688878","url":null,"abstract":"Over decades, fault-tree has remained as one of the most widely used fault management (FM) mechanisms by practitioners as a visualization/communication media, as well as a quantitative analysis tool for building dependable systems. Nonetheless, to date fault tree analysis is often unable to provide us with high-confidence results due to potential misrepresentation of the relationships among failure events. In retrospect, catastrophic system failures were almost always caused by a chain of mixed fault manifestation and interaction. Moreover, faults in FM itself may become a critical element in a fault-manifestation chain. The 1996 Ariane-5 rocket self-destruction provides us with a lesson on the devastating consequence of inappropriate FM application. Such prior lessons have inspired us to propose a fault-class-aware and FM-capability-aware fault tree analysis framework.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"114 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123397965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688890
Alexandre Perez, Rui Abreu
Information foraging is a theory to understand how people search for information. In this theory, information scent is the perceived likelihood by the “predator” that a cue will lead to a “prey”. The better the cues, the better the information scent. In automatic debugging, it is the perceived likelihood that the diagnostic report leads to the cause of failures. In this paper, we detail a visualization, offered by the GZoltar toolset, that has the potential to provide better cues. With better we mean providing more information that leads to the fault than, e.g., the source code and code coverage information. The toolset provides a graphical display of the diagnostic reports yielded by well-known debugging techniques. From an information foraging point of view, we argue that the visualization is of added value while debugging. Finally, we report a user study to confirm that GZOLTAR's visualization provides better cues for pinpointing faults.
{"title":"Cues for scent intensification in debugging","authors":"Alexandre Perez, Rui Abreu","doi":"10.1109/ISSREW.2013.6688890","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688890","url":null,"abstract":"Information foraging is a theory to understand how people search for information. In this theory, information scent is the perceived likelihood by the “predator” that a cue will lead to a “prey”. The better the cues, the better the information scent. In automatic debugging, it is the perceived likelihood that the diagnostic report leads to the cause of failures. In this paper, we detail a visualization, offered by the GZoltar toolset, that has the potential to provide better cues. With better we mean providing more information that leads to the fault than, e.g., the source code and code coverage information. The toolset provides a graphical display of the diagnostic reports yielded by well-known debugging techniques. From an information foraging point of view, we argue that the visualization is of added value while debugging. Finally, we report a user study to confirm that GZOLTAR's visualization provides better cues for pinpointing faults.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127216288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-11-01DOI: 10.1109/ISSREW.2013.6688857
Shweta Subramani, M. Vouk, L. Williams
We are studying extension of the classical Software Reliability Engineering (SRE) methodology into the security space. We combine “classical” reliability modeling, when applied to reported vulnerabilities found under “normal” operational profile conditions, with safety oriented fault management processes. We illustrate with open source Fedora software.
{"title":"Non-operational testing of software for security issues","authors":"Shweta Subramani, M. Vouk, L. Williams","doi":"10.1109/ISSREW.2013.6688857","DOIUrl":"https://doi.org/10.1109/ISSREW.2013.6688857","url":null,"abstract":"We are studying extension of the classical Software Reliability Engineering (SRE) methodology into the security space. We combine “classical” reliability modeling, when applied to reported vulnerabilities found under “normal” operational profile conditions, with safety oriented fault management processes. We illustrate with open source Fedora software.","PeriodicalId":332420,"journal":{"name":"2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129162305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：