Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00012
Martin Kang, Ted E. Lee, Sungyong Um
Benchmarking methodology can provide organizations with a way to choose an appropriate information security policy. However, selecting a proper organization as a benchmarking peer is a challenge due to the lack of quantitative methods for benchmarking. This paper proposes methods to select a peer organization by quantitatively measuring the similarity of organizations' InfoSec management systems.
{"title":"Establishment of Methods for Information Security System Policy Using Benchmarking","authors":"Martin Kang, Ted E. Lee, Sungyong Um","doi":"10.1109/ISSREW.2018.00012","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00012","url":null,"abstract":"Benchmarking methodology can provide organizations with a way to choose an appropriate information security policy. However, selecting a proper organization as a benchmarking peer is a challenge due to the lack of quantitative methods for benchmarking. This paper proposes methods to select a peer organization by quantitatively measuring the similarity of organizations' InfoSec management systems.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122203796","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00-30
R. Hanmer, L. Jagadeesan, V. Mendiratta, Heng Zhang
Distributed systems play an increasingly important role in leading-edge networks with high availability requirements, including software-defined networks (SDN), where replicating essential network state information is critical to ensure resilience under failures. Distributed consensus based strong consistency algorithms, such as Raft, are often used to ensure that all components of the distributed system agree on their view of the replicated data, even when a minority of the distributed components crash. Another critical requirement for highly available networks is to gracefully handle overload conditions, where the demands on the network exceed expected levels for a period of time, such as during natural or man-made disasters or popular sporting events. Hence, the strong consistency algorithms used in such networks must also behave gracefully under overload conditions. We show that, in fact, strong consistency algorithms such as Raft may not behave gracefully under overload conditions and can in fact significantly negatively affect SDN control plane availability in these circumstances. We demonstrate that the open-source ONOS SDN controller, which uses the Java-based Atomix implementation of Raft, exhibits such behavior under intent overload, resulting in the loss of requests to the network, and with the entire SDN network eventually crashing. We further demonstrate similar behaviors of the Python-based pysyncobj implementation of Raft. We then propose DynRaft, a dynamic add-on to Raft implementations that continues to ensure the formally proven strong consistency properties of Raft, and demonstrate the effectiveness of DynRaft with the pysyncobj implementation under emulated overload conditions.
{"title":"Friend or Foe: Strong Consistency vs. Overload in High-Availability Distributed Systems and SDN","authors":"R. Hanmer, L. Jagadeesan, V. Mendiratta, Heng Zhang","doi":"10.1109/ISSREW.2018.00-30","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00-30","url":null,"abstract":"Distributed systems play an increasingly important role in leading-edge networks with high availability requirements, including software-defined networks (SDN), where replicating essential network state information is critical to ensure resilience under failures. Distributed consensus based strong consistency algorithms, such as Raft, are often used to ensure that all components of the distributed system agree on their view of the replicated data, even when a minority of the distributed components crash. Another critical requirement for highly available networks is to gracefully handle overload conditions, where the demands on the network exceed expected levels for a period of time, such as during natural or man-made disasters or popular sporting events. Hence, the strong consistency algorithms used in such networks must also behave gracefully under overload conditions. We show that, in fact, strong consistency algorithms such as Raft may not behave gracefully under overload conditions and can in fact significantly negatively affect SDN control plane availability in these circumstances. We demonstrate that the open-source ONOS SDN controller, which uses the Java-based Atomix implementation of Raft, exhibits such behavior under intent overload, resulting in the loss of requests to the network, and with the entire SDN network eventually crashing. We further demonstrate similar behaviors of the Python-based pysyncobj implementation of Raft. We then propose DynRaft, a dynamic add-on to Raft implementations that continues to ensure the formally proven strong consistency properties of Raft, and demonstrate the effectiveness of DynRaft with the pysyncobj implementation under emulated overload conditions.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"184 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123066586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00007
Long Zhang, Zhenyu Zhang
Many software failures are caused by faults in programs. Fault localization is always a difficult task in program debugging, and the spectrum-based fault localization (SBFL in short) is a popular approach. A SBFL technique collects code coverage of program runs, and estimates to what extent individual program entities correlate to the failed runs. We have empirically reported that referencing debugging history can effectively alleviate the impact of program structure on the accuracy of SBFL techniques. However, referencing all test cases indistinguishably may have adverse effects. In this paper, we propose a novel technique SeTCHi, which differentiates test cases according to their coverage and test outputs, and refines SBFL with the means to select supporting test cases with respect to program entities and history program versions. We also conduct an empirical study, which shows that SeTCHi can significantly improve the accuracy of fault localization based on state-of-the-art techniques.
{"title":"SeTCHi: Selecting Test Cases to Improve History-Guided Fault Localization","authors":"Long Zhang, Zhenyu Zhang","doi":"10.1109/ISSREW.2018.00007","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00007","url":null,"abstract":"Many software failures are caused by faults in programs. Fault localization is always a difficult task in program debugging, and the spectrum-based fault localization (SBFL in short) is a popular approach. A SBFL technique collects code coverage of program runs, and estimates to what extent individual program entities correlate to the failed runs. We have empirically reported that referencing debugging history can effectively alleviate the impact of program structure on the accuracy of SBFL techniques. However, referencing all test cases indistinguishably may have adverse effects. In this paper, we propose a novel technique SeTCHi, which differentiates test cases according to their coverage and test outputs, and refines SBFL with the means to select supporting test cases with respect to program entities and history program versions. We also conduct an empirical study, which shows that SeTCHi can significantly improve the accuracy of fault localization based on state-of-the-art techniques.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122836800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00-51
W. Wong, T. Tse, A. Andrzejak
2018 IEEE International Workshop on Debugging and Repair (IDEAR), co-located with the 29th IEEE International Symposium on Software Reliability Engineering (ISSRE 2018)
{"title":"Message from the IDEAR 2018 Workshop Chairs","authors":"W. Wong, T. Tse, A. Andrzejak","doi":"10.1109/ISSREW.2018.00-51","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00-51","url":null,"abstract":"2018 IEEE International Workshop on Debugging and Repair (IDEAR), co-located with the 29th IEEE International Symposium on Software Reliability Engineering (ISSRE 2018)","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117296039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00006
Ingo Pill, F. Wotawa
When obtaining a full-fledged model for diagnostic and debugging purposes is out of reach, abstract logic models might allow us to fall back to abductive reasoning for isolating faults. Such models often only aggregate knowledge about which inputs and faults would have this or that effect on the system. Like in property-based system design or formal verification, we have that the quality of the resulting reasoning process depends heavily on this logic model. Since logic descriptions are not entirely intuitive to formulate and automated processes to derive them are prone to be incomplete, we'd certainly be interested in assessing a model's quality and isolate issues. In this paper, we're proposing to use test cases and spectrum-based fault localization for this task, drawing on the flexibility and ease-of-use of such a spectrum-based concept. Focusing on logic models formulated in propositional Horn-clauses, we provide examples that show the attractiveness of our concept.
{"title":"Spectrum-Based Fault Localization for Logic-Based Reasoning","authors":"Ingo Pill, F. Wotawa","doi":"10.1109/ISSREW.2018.00006","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00006","url":null,"abstract":"When obtaining a full-fledged model for diagnostic and debugging purposes is out of reach, abstract logic models might allow us to fall back to abductive reasoning for isolating faults. Such models often only aggregate knowledge about which inputs and faults would have this or that effect on the system. Like in property-based system design or formal verification, we have that the quality of the resulting reasoning process depends heavily on this logic model. Since logic descriptions are not entirely intuitive to formulate and automated processes to derive them are prone to be incomplete, we'd certainly be interested in assessing a model's quality and isolate issues. In this paper, we're proposing to use test cases and spectrum-based fault localization for this task, drawing on the flexibility and ease-of-use of such a spectrum-based concept. Focusing on logic models formulated in propositional Horn-clauses, we provide examples that show the attractiveness of our concept.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114792561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00-13
Domenico Cotroneo, L. Simone, Alfonso Di Martino, Pietro Liguori, R. Natella
We argue for novel techniques to understand how cloud systems can fail, by enhancing fault injection with distributed tracing and anomaly detection techniques.
我们主张采用新的技术，通过分布式跟踪和异常检测技术增强故障注入，来理解云系统是如何失败的。
{"title":"Enhancing the Analysis of Error Propagation and Failure Modes in Cloud Systems","authors":"Domenico Cotroneo, L. Simone, Alfonso Di Martino, Pietro Liguori, R. Natella","doi":"10.1109/ISSREW.2018.00-13","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00-13","url":null,"abstract":"We argue for novel techniques to understand how cloud systems can fail, by enhancing fault injection with distributed tracing and anomaly detection techniques.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130439730","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00-11
Ailec Wu, A. Rubaiyat, Chris Anton, H. Alemzadeh
We present the preliminary results on developing a weighted N-version programming (NVP) scheme for ensuring resilience of machine learning based steering control algorithms. The proposed scheme is designed based on the fusion of outputs from three redundant Deep Neural Network (DNN) models, independently designed using Udacity's self driving car challenge data. The improvement in reliability compared to single DNN models is evaluated by measuring the steering angle prediction accuracy in the presence of simulated perturbations on input image data caused by various environmental conditions.
{"title":"Model Fusion: Weighted N-Version Programming for Resilient Autonomous Vehicle Steering Control","authors":"Ailec Wu, A. Rubaiyat, Chris Anton, H. Alemzadeh","doi":"10.1109/ISSREW.2018.00-11","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00-11","url":null,"abstract":"We present the preliminary results on developing a weighted N-version programming (NVP) scheme for ensuring resilience of machine learning based steering control algorithms. The proposed scheme is designed based on the fusion of outputs from three redundant Deep Neural Network (DNN) models, independently designed using Udacity's self driving car challenge data. The improvement in reliability compared to single DNN models is evaluated by measuring the steering angle prediction accuracy in the presence of simulated perturbations on input image data caused by various environmental conditions.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132943232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00024
Guanpeng Li, K. Pattabiraman, Nathan Debardeleben
Machine Learning (ML) applications have emerged as the killer applications for next generation hardware and software platforms, and there is a lot of interest in software frameworks to build such applications. TensorFlow is a high-level dataflow framework for building ML applications and has become the most popular one in the recent past. ML applications are also being increasingly used in safety-critical systems such as self-driving cars and home robotics. Therefore, there is a compelling need to evaluate the resilience of ML applications built using frameworks such as TensorFlow. In this paper, we build a high-level fault injection framework for TensorFlow called TensorFI for evaluating the resilience of ML applications. TensorFI is flexible, easy to use, and portable. It also allows ML application programmers to explore the effects of different parameters and algorithms on error resilience.
{"title":"TensorFI: A Configurable Fault Injector for TensorFlow Applications","authors":"Guanpeng Li, K. Pattabiraman, Nathan Debardeleben","doi":"10.1109/ISSREW.2018.00024","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00024","url":null,"abstract":"Machine Learning (ML) applications have emerged as the killer applications for next generation hardware and software platforms, and there is a lot of interest in software frameworks to build such applications. TensorFlow is a high-level dataflow framework for building ML applications and has become the most popular one in the recent past. ML applications are also being increasingly used in safety-critical systems such as self-driving cars and home robotics. Therefore, there is a compelling need to evaluate the resilience of ML applications built using frameworks such as TensorFlow. In this paper, we build a high-level fault injection framework for TensorFlow called TensorFI for evaluating the resilience of ML applications. TensorFI is flexible, easy to use, and portable. It also allows ML application programmers to explore the effects of different parameters and algorithms on error resilience.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133868253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-10-01DOI: 10.1109/ISSREW.2018.00-33
Monika Maidl, Dirk Kröselberg, Jochen Christ, Kristian Beckers
The standards family IEC 62443 represents an international agreement on best practices for securing Industrial Automation Control Systems (IACS). Engineering projects have to address security, but have limits on cost and resources, which makes it particularly challenging to cover all security topics adequately, as prescribed by IEC 62443. We developed a framework that supports engineering projects in addressing the whole range of security aspects more effectively and efficiently. The framework is structured horizontally and vertically. Horizontally, the framework consists of a set artefacts that cover the security aspects as prescribed by IEC-62443, and that need to be filled out by an engineering project. The vertical structure reflects the organizational hierarchy. In each hierarchical layer, the artefact templates are enriched by increasingly detailed and specific guidance in the form of best practices and references. This enables the exchange and reuse of security designs and best practices across the organization. We describe our experiences in applying the framework in large scale industry projects.
{"title":"A Comprehensive Framework for Security in Engineering Projects - Based on IEC 62443","authors":"Monika Maidl, Dirk Kröselberg, Jochen Christ, Kristian Beckers","doi":"10.1109/ISSREW.2018.00-33","DOIUrl":"https://doi.org/10.1109/ISSREW.2018.00-33","url":null,"abstract":"The standards family IEC 62443 represents an international agreement on best practices for securing Industrial Automation Control Systems (IACS). Engineering projects have to address security, but have limits on cost and resources, which makes it particularly challenging to cover all security topics adequately, as prescribed by IEC 62443. We developed a framework that supports engineering projects in addressing the whole range of security aspects more effectively and efficiently. The framework is structured horizontally and vertically. Horizontally, the framework consists of a set artefacts that cover the security aspects as prescribed by IEC-62443, and that need to be filled out by an engineering project. The vertical structure reflects the organizational hierarchy. In each hierarchical layer, the artefact templates are enriched by increasingly detailed and specific guidance in the form of best practices and references. This enables the exchange and reuse of security designs and best practices across the organization. We describe our experiences in applying the framework in large scale industry projects.","PeriodicalId":321448,"journal":{"name":"2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133313791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：