Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00038
A. Morozov, K. Ding, Mikael Steurer, K. Janschek
Increasing complexity and heterogeneity of modern safety-critical systems require advanced tools for quantitative reliability analysis. Most of the available analytical software exploits classical methods such as event trees, static and dynamic fault trees, reliability block diagrams, simple Bayesian networks, and Markov chains. First, these methods fail to adequately model complex interaction of software, hardware, physical components, dynamic feedback loops, propagation of data errors, nontrivial failure scenarios, sophisticated fault tolerance, and resilience mechanisms. Second, these methods are limited to the evaluation of the fixed set of traditional reliability metrics such as the probability of generic system failure, failure rate, MTTF, MTBF, and MTTR. More flexible models, such as the Dual-graph Error Propagation Model (DEPM) can overcome these limitations but have no available tools. This paper introduces the first open-source DEPM-based analytical software tool OpenErrorPro. The DEPM is a formal stochastic model that captures control and data flow structures and reliability-related properties of executable system components. The numerical analysis in OpenErrorPro is based on the automatic generation of Markov chain models and the utilization of modern Probabilistic Model Checking (PMC) techniques. The PMC enables the analysis of highly-customizable resilience metrics, e.g. "the probability of system recovery after a specified system failure during the defined time interval", in addition to the traditional reliability metrics. DEPMs can be automatically generated from Simulink/Stateflow, UML/SysML, and AADL models, as well as source code of software components using LLVM. This allows not only the automated model-based evaluation but also the analysis of systems developed using the combination of several modeling paradigms. The key purpose of the tool is to close the gap between the conventional system design models and advanced analytical methods in order to give system reliability engineers easy and automated access to the full potential of PMC techniques. Finally, OpenErrorPro enables the application of several effective optimizations against the state space explosion of underlying Markov models already in the DEPM level where the system semantics such as control and data flow structures are accessible.
{"title":"OpenErrorPro: A New Tool for Stochastic Model-Based Reliability and Resilience Analysis","authors":"A. Morozov, K. Ding, Mikael Steurer, K. Janschek","doi":"10.1109/ISSRE.2019.00038","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00038","url":null,"abstract":"Increasing complexity and heterogeneity of modern safety-critical systems require advanced tools for quantitative reliability analysis. Most of the available analytical software exploits classical methods such as event trees, static and dynamic fault trees, reliability block diagrams, simple Bayesian networks, and Markov chains. First, these methods fail to adequately model complex interaction of software, hardware, physical components, dynamic feedback loops, propagation of data errors, nontrivial failure scenarios, sophisticated fault tolerance, and resilience mechanisms. Second, these methods are limited to the evaluation of the fixed set of traditional reliability metrics such as the probability of generic system failure, failure rate, MTTF, MTBF, and MTTR. More flexible models, such as the Dual-graph Error Propagation Model (DEPM) can overcome these limitations but have no available tools. This paper introduces the first open-source DEPM-based analytical software tool OpenErrorPro. The DEPM is a formal stochastic model that captures control and data flow structures and reliability-related properties of executable system components. The numerical analysis in OpenErrorPro is based on the automatic generation of Markov chain models and the utilization of modern Probabilistic Model Checking (PMC) techniques. The PMC enables the analysis of highly-customizable resilience metrics, e.g. \"the probability of system recovery after a specified system failure during the defined time interval\", in addition to the traditional reliability metrics. DEPMs can be automatically generated from Simulink/Stateflow, UML/SysML, and AADL models, as well as source code of software components using LLVM. This allows not only the automated model-based evaluation but also the analysis of systems developed using the combination of several modeling paradigms. The key purpose of the tool is to close the gap between the conventional system design models and advanced analytical methods in order to give system reliability engineers easy and automated access to the full potential of PMC techniques. Finally, OpenErrorPro enables the application of several effective optimizations against the state space explosion of underlying Markov models already in the DEPM level where the system semantics such as control and data flow structures are accessible.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124240576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00036
Franck Chauvel, Brice Morin, Enrique Garcia-Ceja
Modern software systems interact with multiple 3rd party dependencies such as the OS file system, libraries, databases or remote services. To verify these interactions, developers write so-called "integration tests" that exercise the software within a specific environment. These tests are not only difficult to write as their environment is complicated, but they are also brittle because changes outside the code (i.e., in the environment) might make them fail unexpectedly. Integration tests are thus underused whereas they could help find many more issues. We hence propose CAMP, a tool that amplifies an existing integration test by exploring variations of the given environment. The tests that CAMP generates alter the services orchestration, the software stacks, the individual components' configuration or any combination thereof. We used CAMP to amplify tests from the Sphinx and Atom open-source projects, and in both cases, we were able to spot undocumented issues related to incompatible environments.
{"title":"Amplifying Integration Tests with CAMP","authors":"Franck Chauvel, Brice Morin, Enrique Garcia-Ceja","doi":"10.1109/ISSRE.2019.00036","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00036","url":null,"abstract":"Modern software systems interact with multiple 3rd party dependencies such as the OS file system, libraries, databases or remote services. To verify these interactions, developers write so-called \"integration tests\" that exercise the software within a specific environment. These tests are not only difficult to write as their environment is complicated, but they are also brittle because changes outside the code (i.e., in the environment) might make them fail unexpectedly. Integration tests are thus underused whereas they could help find many more issues. We hence propose CAMP, a tool that amplifies an existing integration test by exploring variations of the given environment. The tests that CAMP generates alter the services orchestration, the software stacks, the individual components' configuration or any combination thereof. We used CAMP to amplify tests from the Sphinx and Atom open-source projects, and in both cases, we were able to spot undocumented issues related to incompatible environments.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122119679","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00044
Rensong Xie, Xianglong Kong, Lulu Wang, Ying Zhou, Bixin Li
Context-aware API recommendation techniques aim to generate a ranked list of candidate APIs on an editing position during development. The basic context used in traditional API recommendation mainly focuses on the APIs from third-party libraries, limit or even ignore the usage of project-specific code. The limited usage of project-specific code may result in the lack of context information, and degrade the effectiveness of API recommendation. To address this problem, we introduce a novel type of context, i.e., hierarchical context, which can leverage the hidden information of project-specific code by analyzing the call graph. In hierarchical context, a project-specific API is presented as a sequence of low-leveled APIs from third-party libraries. We propose an approach, i.e., HiRec, which builds on the basis of hierarchical context. HiRec is evaluated on 108 projects and the results show that HiRec can obtain much more accurate results than all the other selected approaches in terms of top-5 and top-10 accuracy due to the strong ability of context representation. And HiRec performs closely to the outstanding tools in terms of top-1 accuracy. The average time of recommending execution is less than 1 seconds in most cases, which is acceptable for interaction in an IDE. Unlike current approaches, the effectiveness of HiRec is not impacted much by editing positions. And we can obtain more accurate results from HiRec with larger sizes of training data and hierarchical context.
{"title":"HiRec: API Recommendation using Hierarchical Context","authors":"Rensong Xie, Xianglong Kong, Lulu Wang, Ying Zhou, Bixin Li","doi":"10.1109/ISSRE.2019.00044","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00044","url":null,"abstract":"Context-aware API recommendation techniques aim to generate a ranked list of candidate APIs on an editing position during development. The basic context used in traditional API recommendation mainly focuses on the APIs from third-party libraries, limit or even ignore the usage of project-specific code. The limited usage of project-specific code may result in the lack of context information, and degrade the effectiveness of API recommendation. To address this problem, we introduce a novel type of context, i.e., hierarchical context, which can leverage the hidden information of project-specific code by analyzing the call graph. In hierarchical context, a project-specific API is presented as a sequence of low-leveled APIs from third-party libraries. We propose an approach, i.e., HiRec, which builds on the basis of hierarchical context. HiRec is evaluated on 108 projects and the results show that HiRec can obtain much more accurate results than all the other selected approaches in terms of top-5 and top-10 accuracy due to the strong ability of context representation. And HiRec performs closely to the outstanding tools in terms of top-1 accuracy. The average time of recommending execution is less than 1 seconds in most cases, which is acceptable for interaction in an IDE. Unlike current approaches, the effectiveness of HiRec is not impacted much by editing positions. And we can obtain more accurate results from HiRec with larger sizes of training data and hierarchical context.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"32 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127984137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00027
Zhou Xu, Tao Zhang, Yifeng Zhang, Yutian Tang, Jin Liu, Xiapu Luo, J. Keung, Xiaohui Cui
Analyzing the crash reports recorded upon software crashes is a critical activity for software quality assurance. Predicting whether or not the fault causing the crash (crashing fault for short) resides in the stack traces of crash reports can speed-up the program debugging process and determine the priority of the debugging efforts. Previous work mostly collected label information from bug-fixing logs, and extracted crash features from stack traces and source code to train classification models for the Identification of Crashing Fault Residence (ICFR) of newly-submitted crashes. However, labeled data are not always fully available in real applications. Hence the classifier training is not always feasible. In this work, we make the first attempt to develop a cross project ICFR model to address the data scarcity problem. This is achieved by transferring the knowledge from external projects to the current project via utilizing a state-of-the-art Balanced Distribution Adaptation (BDA) based transfer learning method. BDA not only combines both marginal distribution and conditional distribution across projects but also assigns adaptive weights to the two distributions for better adjusting specific cross project pair. The experiments on 7 software projects show that BDA is superior to 9 baseline methods in terms of 6 indicators overall.
{"title":"Identifying Crashing Fault Residence Based on Cross Project Model","authors":"Zhou Xu, Tao Zhang, Yifeng Zhang, Yutian Tang, Jin Liu, Xiapu Luo, J. Keung, Xiaohui Cui","doi":"10.1109/ISSRE.2019.00027","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00027","url":null,"abstract":"Analyzing the crash reports recorded upon software crashes is a critical activity for software quality assurance. Predicting whether or not the fault causing the crash (crashing fault for short) resides in the stack traces of crash reports can speed-up the program debugging process and determine the priority of the debugging efforts. Previous work mostly collected label information from bug-fixing logs, and extracted crash features from stack traces and source code to train classification models for the Identification of Crashing Fault Residence (ICFR) of newly-submitted crashes. However, labeled data are not always fully available in real applications. Hence the classifier training is not always feasible. In this work, we make the first attempt to develop a cross project ICFR model to address the data scarcity problem. This is achieved by transferring the knowledge from external projects to the current project via utilizing a state-of-the-art Balanced Distribution Adaptation (BDA) based transfer learning method. BDA not only combines both marginal distribution and conditional distribution across projects but also assigns adaptive weights to the two distributions for better adjusting specific cross project pair. The experiments on 7 software projects show that BDA is superior to 9 baseline methods in terms of 6 indicators overall.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130827481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00018
Jie Liang, Yuanliang Chen, Mingzhe Wang, Yu Jiang, Z. Yang, Chengnian Sun, Xun Jiao, Jiaguang Sun
State-of-the-art fuzzers implement various optimizations to enhance their performance. As the optimizations reside in different stages such as input seed selection and mutation, it is tempting to combine the optimizations in different stages. However, our initial attempts demonstrate that naive combination actually worsens the performance, which explains that most optimizations are still isolated by stages and metrics. In this paper, we present InteFuzz, the first framework that synergically integrates multiple fuzzing optimizations. We analyze the root cause for performance degradation in naive combination, and discover optimizations conflict in coverage criteria and optimization granularity. To resolve the conflicts, we propose a novel priority-based scheduling mechanism. The dynamic integration considers both branch-based and block-based coverage feedbacks that are used by most fuzzing optimizations. In our evaluation, we extract four optimizations from popular fuzzers such as AFLFast and FairFuzz and compare InteFuzz against naive combinations. The evaluation results show that InteFuzz outperforms the naive combination by 29% and 26% in path-and branch-coverage. Additionally, InteFuzz triggers 222 more unique crashes, and discovers 33 zero-day vulnerabilities in real-world projects with 12 registered as CVEs.
{"title":"Engineering a Better Fuzzer with Synergically Integrated Optimizations","authors":"Jie Liang, Yuanliang Chen, Mingzhe Wang, Yu Jiang, Z. Yang, Chengnian Sun, Xun Jiao, Jiaguang Sun","doi":"10.1109/ISSRE.2019.00018","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00018","url":null,"abstract":"State-of-the-art fuzzers implement various optimizations to enhance their performance. As the optimizations reside in different stages such as input seed selection and mutation, it is tempting to combine the optimizations in different stages. However, our initial attempts demonstrate that naive combination actually worsens the performance, which explains that most optimizations are still isolated by stages and metrics. In this paper, we present InteFuzz, the first framework that synergically integrates multiple fuzzing optimizations. We analyze the root cause for performance degradation in naive combination, and discover optimizations conflict in coverage criteria and optimization granularity. To resolve the conflicts, we propose a novel priority-based scheduling mechanism. The dynamic integration considers both branch-based and block-based coverage feedbacks that are used by most fuzzing optimizations. In our evaluation, we extract four optimizations from popular fuzzers such as AFLFast and FairFuzz and compare InteFuzz against naive combinations. The evaluation results show that InteFuzz outperforms the naive combination by 29% and 26% in path-and branch-coverage. Additionally, InteFuzz triggers 222 more unique crashes, and discovers 33 zero-day vulnerabilities in real-world projects with 12 registered as CVEs.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117041356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00040
Nasif Imtiaz, Brendan Murphy, L. Williams
Static analysis tools (SATs) often fall short of developer satisfaction despite their many benefits. An understanding of how developers in the real-world act on the alerts detected by SATs can help improve the utility of these tools and determine future research directions. The goal of this paper is to aid researchers and tool makers in improving the utility of static analysis tools through an empirical study of developer action on the alerts detected by Coverity, a state-of-the-art static analysis tool. In this paper, we analyze five open source projects as case studies (Linux, Firefox, Samba, Kodi, and Ovirt-engine) that have been actively using Coverity over a period of at least five years. We investigate the alert occurrences and developer triage of the alerts from the Coverity database; identify the alerts that were fixed through code changes (i.e. actionable) by mining the commit history of the projects; analyze the time an alert remain in the code base (i.e. lifespan) and the complexity of code changes (i.e. fix complexity) in fixing the alert. We find that 27.4% to 49.5% (median: 36.7%) of the alerts are actionable across projects, a rate higher than previously reported. We also find that the fixes of Coverity alerts are generally low in complexity (2 to 7 lines of code changes in the affected file, median: 4). However, developers still take from 36 to 245 days (median: 96) to fix these alerts. Finally, our data suggest that severity and fix complexity may correlate with an alert's lifespan in some of the projects.
{"title":"How Do Developers Act on Static Analysis Alerts? An Empirical Study of Coverity Usage","authors":"Nasif Imtiaz, Brendan Murphy, L. Williams","doi":"10.1109/ISSRE.2019.00040","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00040","url":null,"abstract":"Static analysis tools (SATs) often fall short of developer satisfaction despite their many benefits. An understanding of how developers in the real-world act on the alerts detected by SATs can help improve the utility of these tools and determine future research directions. The goal of this paper is to aid researchers and tool makers in improving the utility of static analysis tools through an empirical study of developer action on the alerts detected by Coverity, a state-of-the-art static analysis tool. In this paper, we analyze five open source projects as case studies (Linux, Firefox, Samba, Kodi, and Ovirt-engine) that have been actively using Coverity over a period of at least five years. We investigate the alert occurrences and developer triage of the alerts from the Coverity database; identify the alerts that were fixed through code changes (i.e. actionable) by mining the commit history of the projects; analyze the time an alert remain in the code base (i.e. lifespan) and the complexity of code changes (i.e. fix complexity) in fixing the alert. We find that 27.4% to 49.5% (median: 36.7%) of the alerts are actionable across projects, a rate higher than previously reported. We also find that the fixes of Coverity alerts are generally low in complexity (2 to 7 lines of code changes in the affected file, median: 4). However, developers still take from 36 to 245 days (median: 96) to fix these alerts. Finally, our data suggest that severity and fix complexity may correlate with an alert's lifespan in some of the projects.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131059415","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00052
William Zhang, Sebastian Banescu, Leodardo Pasos, Steven T. Stewart, Vijay Ganesh
Smart contracts are executable programs that enable the building of a programmable trust mechanism between multiple entities without the need of a trusted third-party. At the time of this writing, there were over 10 million smart contracts deployed on the Ethereum networks and this number continues to grow at a rapid pace. Smart contracts are often written in a Turing-complete programming language called Solidity, which is not easy to audit for subtle errors. Further, since smart contracts are immutable, errors have led to attacks resulting in losses of cryptocurrency worth 100s of millions of USD and reputational damage. Unfortunately, manual security analyses do not scale with size and number of smart contracts. Automated and scalable mechanisms are essential if smart contracts are to gain mainstream acceptance. Researchers have developed several security scanners in the past couple of years. However, many of these analyzer either do not scale well, or if they do, produce many false positives. This issue is exacerbated when bugs are triggered only after a series of interactions with the functions of the contract-under-test. A depth-n vulnerability, refers to a vulnerability that requires invoking a specific sequence of n functions to trigger. Depth-n vulnerabilities are time-consuming to detect by existing automated analyzers, because of the combinatorial explosion of sequences of functions that could be executed on smart contracts. In this paper, we present a technique to analyze depth-n vulnerabilities in an efficient and scalable way by combining symbolic execution and data dependency analysis. A significant advantage of combining symbolic with static analysis is that it scales much better than symbolic alone and does not have the problem of false positive that static analysis tools typically have. We have implemented our technique in a tool called MPro, a scalable and automated smart contract analyzer based on the existing symbolic analysis tool Mythril-Classic and the static analysis tool Slither. We analyzed 100 randomly chosen smart contracts on MPro and our evaluation shows that MPro is about n-times faster than Mythril-Classic for detecting depth-n vulnerabilities, while preserving all the detection capabilities of Mythril-Classic.
{"title":"MPro: Combining Static and Symbolic Analysis for Scalable Testing of Smart Contract","authors":"William Zhang, Sebastian Banescu, Leodardo Pasos, Steven T. Stewart, Vijay Ganesh","doi":"10.1109/ISSRE.2019.00052","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00052","url":null,"abstract":"Smart contracts are executable programs that enable the building of a programmable trust mechanism between multiple entities without the need of a trusted third-party. At the time of this writing, there were over 10 million smart contracts deployed on the Ethereum networks and this number continues to grow at a rapid pace. Smart contracts are often written in a Turing-complete programming language called Solidity, which is not easy to audit for subtle errors. Further, since smart contracts are immutable, errors have led to attacks resulting in losses of cryptocurrency worth 100s of millions of USD and reputational damage. Unfortunately, manual security analyses do not scale with size and number of smart contracts. Automated and scalable mechanisms are essential if smart contracts are to gain mainstream acceptance. Researchers have developed several security scanners in the past couple of years. However, many of these analyzer either do not scale well, or if they do, produce many false positives. This issue is exacerbated when bugs are triggered only after a series of interactions with the functions of the contract-under-test. A depth-n vulnerability, refers to a vulnerability that requires invoking a specific sequence of n functions to trigger. Depth-n vulnerabilities are time-consuming to detect by existing automated analyzers, because of the combinatorial explosion of sequences of functions that could be executed on smart contracts. In this paper, we present a technique to analyze depth-n vulnerabilities in an efficient and scalable way by combining symbolic execution and data dependency analysis. A significant advantage of combining symbolic with static analysis is that it scales much better than symbolic alone and does not have the problem of false positive that static analysis tools typically have. We have implemented our technique in a tool called MPro, a scalable and automated smart contract analyzer based on the existing symbolic analysis tool Mythril-Classic and the static analysis tool Slither. We analyzed 100 randomly chosen smart contracts on MPro and our evaluation shows that MPro is about n-times faster than Mythril-Classic for detecting depth-n vulnerabilities, while preserving all the detection capabilities of Mythril-Classic.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123137750","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00046
Inês Valentim, Nuno Lourenço, Nuno Antunes
Machine learning models are widely adopted in scenarios that directly affect people. The development of software systems based on these models raises societal and legal concerns, as their decisions may lead to the unfair treatment of individuals based on attributes like race or gender. Data preparation is key in any machine learning pipeline, but its effect on fairness is yet to be studied in detail. In this paper, we evaluate how the fairness and effectiveness of the learned models are affected by the removal of the sensitive attribute, the encoding of the categorical attributes, and instance selection methods (including cross-validators and random undersampling). We used the Adult Income and the German Credit Data datasets, which are widely studied and known to have fairness concerns. We applied each data preparation technique individually to analyse the difference in predictive performance and fairness, using statistical parity difference, disparate impact, and the normalised prejudice index. The results show that fairness is affected by transformations made to the training data, particularly in imbalanced datasets. Removing the sensitive attribute is insufficient to eliminate all the unfairness in the predictions, as expected, but it is key to achieve fairer models. Additionally, the standard random undersampling with respect to the true labels is sometimes more prejudicial than performing no random undersampling.
{"title":"The Impact of Data Preparation on the Fairness of Software Systems","authors":"Inês Valentim, Nuno Lourenço, Nuno Antunes","doi":"10.1109/ISSRE.2019.00046","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00046","url":null,"abstract":"Machine learning models are widely adopted in scenarios that directly affect people. The development of software systems based on these models raises societal and legal concerns, as their decisions may lead to the unfair treatment of individuals based on attributes like race or gender. Data preparation is key in any machine learning pipeline, but its effect on fairness is yet to be studied in detail. In this paper, we evaluate how the fairness and effectiveness of the learned models are affected by the removal of the sensitive attribute, the encoding of the categorical attributes, and instance selection methods (including cross-validators and random undersampling). We used the Adult Income and the German Credit Data datasets, which are widely studied and known to have fairness concerns. We applied each data preparation technique individually to analyse the difference in predictive performance and fairness, using statistical parity difference, disparate impact, and the normalised prejudice index. The results show that fairness is affected by transformations made to the training data, particularly in imbalanced datasets. Removing the sensitive attribute is insufficient to eliminate all the unfairness in the predictions, as expected, but it is key to achieve fairer models. Additionally, the standard random undersampling with respect to the true labels is sometimes more prejudicial than performing no random undersampling.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"124 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124884586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00022
Zu-Ming Jiang, Jia-Ju Bai, J. Lawall, Shimin Hu
Device drivers remain a main source of runtime failures in operating systems. To detect bugs in device drivers, fuzzing has been commonly used in practice. However, a main limitation of existing fuzzing approaches is that they cannot effectively test error handling code. Indeed, these fuzzing approaches require effective inputs to cover target code, but much error handling code in drivers is triggered by occasional errors (such as insufficient memory and hardware malfunctions) that are not related to inputs. In this paper, based on software fault injection, we propose a new fuzzing approach named FIZZER, to test error handling code in device drivers. At compile time, FIZZER uses static analysis to recommend possible error sites that can trigger error handling code. During driver execution, by analyzing runtime information, it automatically fuzzes error-site sequences for fault injection to improve code coverage. We evaluate FIZZER on 18 device drivers in Linux 4.19, and in total find 22 real bugs. The code coverage is increased by over 15% compared to normal execution without fuzzing.
{"title":"Fuzzing Error Handling Code in Device Drivers Based on Software Fault Injection","authors":"Zu-Ming Jiang, Jia-Ju Bai, J. Lawall, Shimin Hu","doi":"10.1109/ISSRE.2019.00022","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00022","url":null,"abstract":"Device drivers remain a main source of runtime failures in operating systems. To detect bugs in device drivers, fuzzing has been commonly used in practice. However, a main limitation of existing fuzzing approaches is that they cannot effectively test error handling code. Indeed, these fuzzing approaches require effective inputs to cover target code, but much error handling code in drivers is triggered by occasional errors (such as insufficient memory and hardware malfunctions) that are not related to inputs. In this paper, based on software fault injection, we propose a new fuzzing approach named FIZZER, to test error handling code in device drivers. At compile time, FIZZER uses static analysis to recommend possible error sites that can trigger error handling code. During driver execution, by analyzing runtime information, it automatically fuzzes error-site sequences for fault injection to improve code coverage. We evaluate FIZZER on 18 device drivers in Linux 4.19, and in total find 22 real bugs. The code coverage is increased by over 15% compared to normal execution without fuzzing.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129841107","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：