Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00037
Changwei Zou, Yulei Sui, Hua Yan, Jingling Xue
For performance reasons, C++, albeit unsafe, is often the programming language of choice for developing software infrastructures. A serious type of security vulnerability in C++ programs is type confusion, which may lead to program crashes and control flow hijack attacks. While existing mitigation solutions almost exclusively rely on dynamic analysis techniques, which suffer from low code coverage and high overhead, static analysis has rarely been investigated. This paper presents TCD, a static type confusion detector built on top of a precise demand-driven field-, context-and flow-sensitive pointer analysis. Unlike existing pointer analyses, TCD is type-aware as it not only preserves the type information in the pointed-to objects but also handles complex language features of C++ such as multiple inheritance and placement new, making it therefore possible to reason about type casting in C++ programs. We have implemented TCD in LLVM and evaluated it using seven C++ applications (totaling 526,385 lines of C++ code) from Qt, a widely-adopted C++ toolkit for creating GUIs and cross-platform software. TCD has found five type confusion bugs, including one reported previously in prior work and four new ones, in under 7.3 hours, with a low false positive rate of 28.2%.
{"title":"TCD: Statically Detecting Type Confusion Errors in C++ Programs","authors":"Changwei Zou, Yulei Sui, Hua Yan, Jingling Xue","doi":"10.1109/ISSRE.2019.00037","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00037","url":null,"abstract":"For performance reasons, C++, albeit unsafe, is often the programming language of choice for developing software infrastructures. A serious type of security vulnerability in C++ programs is type confusion, which may lead to program crashes and control flow hijack attacks. While existing mitigation solutions almost exclusively rely on dynamic analysis techniques, which suffer from low code coverage and high overhead, static analysis has rarely been investigated. This paper presents TCD, a static type confusion detector built on top of a precise demand-driven field-, context-and flow-sensitive pointer analysis. Unlike existing pointer analyses, TCD is type-aware as it not only preserves the type information in the pointed-to objects but also handles complex language features of C++ such as multiple inheritance and placement new, making it therefore possible to reason about type casting in C++ programs. We have implemented TCD in LLVM and evaluated it using seven C++ applications (totaling 526,385 lines of C++ code) from Qt, a widely-adopted C++ toolkit for creating GUIs and cross-platform software. TCD has found five type confusion bugs, including one reported previously in prior work and four new ones, in under 7.3 hours, with a low false positive rate of 28.2%.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125531628","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00015
Zeyan Li, Dan Pei, Cheng Luo, Yiwei Zhao, Yongqian Sun, Kaixin Sui, Xiping Wang, Dapeng Liu, Xing Jin, Qi Wang
Operators of online software services periodically collect various measures with many attributes. When a measure becomes abnormal, indicating service problems such as reliability degrade, operators would like to rapidly and accurately localize the root cause attribute combinations within a huge multi-dimensional search space. Unfortunately, previous approaches are not generic or robust in that they all suffer from impractical root cause assumptions, handling only directly collected measures but not derived ones, handling only anomalies with signicant magnitudes but not those insignicant but important ones, requiring manual parameter ne-tuning, or being too slow. This paper proposes a generic and robust multi-dimensional root cause localization approach, Squeeze, that overcomes all above limitations, the first in the literature. Through our novel bottom-up then top-down searching strategy and the techniques based on our proposed generalized ripple effect and generalized potential score, Squeeze is able to reach a good trade off between search speed and accuracy in a generic and robust manner. Case studies in several banks and an Internet company show that Squeeze can localize root causes much more rapidly and accurately than the traditional manual analysis. Furthermore, our extensive experiments on semi-synthetic datasets show that the F1-score of Squeeze outperforms previous approaches by 0.4 on average, while its localization time is only about 10 seconds
{"title":"Generic and Robust Localization of Multi-dimensional Root Causes","authors":"Zeyan Li, Dan Pei, Cheng Luo, Yiwei Zhao, Yongqian Sun, Kaixin Sui, Xiping Wang, Dapeng Liu, Xing Jin, Qi Wang","doi":"10.1109/ISSRE.2019.00015","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00015","url":null,"abstract":"Operators of online software services periodically collect various measures with many attributes. When a measure becomes abnormal, indicating service problems such as reliability degrade, operators would like to rapidly and accurately localize the root cause attribute combinations within a huge multi-dimensional search space. Unfortunately, previous approaches are not generic or robust in that they all suffer from impractical root cause assumptions, handling only directly collected measures but not derived ones, handling only anomalies with signicant magnitudes but not those insignicant but important ones, requiring manual parameter ne-tuning, or being too slow. This paper proposes a generic and robust multi-dimensional root cause localization approach, Squeeze, that overcomes all above limitations, the first in the literature. Through our novel bottom-up then top-down searching strategy and the techniques based on our proposed generalized ripple effect and generalized potential score, Squeeze is able to reach a good trade off between search speed and accuracy in a generic and robust manner. Case studies in several banks and an Internet company show that Squeeze can localize root causes much more rapidly and accurately than the traditional manual analysis. Furthermore, our extensive experiments on semi-synthetic datasets show that the F1-score of Squeeze outperforms previous approaches by 0.4 on average, while its localization time is only about 10 seconds","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130161691","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Software aging, which is caused by Aging-Related Bugs (ARBs), tends to occur in long-running systems and may lead to performance degradation and increasing failure rate during software execution. ARB prediction can help developers discover and remove ARBs, thus alleviating the impact of software aging. However, ARB-prone files occupy a small percentage of all the analyzed files. It is usually difficult to gather sufficient ARB data within a project. To overcome the limited availability of training data, several researchers have recently developed cross-project models for ARB prediction. A key point for cross-project models is to learn a good representation for instances in different projects. Nevertheless, most of the previous approaches neither consider the reconstruction property of new representation nor encode source samples' label information in learning representation. To address these shortcomings, we propose a Supervised Representation Learning Approach (SRLA), which is based on double encoding-layer autoencoder, to perform cross-project ARB prediction. Moreover, we present a transfer cross-validation framework to select the hyper-parameters of cross-project models. Experiments on three large open-source projects demonstrate the effectiveness and superiority of our approach compared with the state-of-the-art approach TLAP.
软件老化是由老化相关bug (aging - related Bugs, arb)引起的，它往往发生在长时间运行的系统中，并可能导致软件执行过程中的性能下降和故障率增加。ARB预测可以帮助开发人员发现并移除ARB，从而减轻软件老化的影响。然而，有arb倾向的文件只占所有分析文件的一小部分。通常很难在项目中收集足够的ARB数据。为了克服训练数据的有限可用性，一些研究人员最近开发了用于ARB预测的跨项目模型。跨项目模型的一个关键点是学习不同项目中实例的良好表示。然而，以往的大多数方法既没有考虑新表示的重构特性，也没有在学习表示中编码源样本的标签信息。为了解决这些缺点，我们提出了一种基于双编码层自编码器的监督表示学习方法(SRLA)来执行跨项目的ARB预测。此外，我们提出了一个转移交叉验证框架来选择跨项目模型的超参数。在三个大型开源项目上的实验表明，与最先进的方法TLAP相比，我们的方法具有有效性和优越性。
{"title":"Supervised Representation Learning Approach for Cross-Project Aging-Related Bug Prediction","authors":"Xiaohui Wan, Zheng Zheng, Fangyun Qin, Yu Qiao, Kishor S. Trivedi","doi":"10.1109/ISSRE.2019.00025","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00025","url":null,"abstract":"Software aging, which is caused by Aging-Related Bugs (ARBs), tends to occur in long-running systems and may lead to performance degradation and increasing failure rate during software execution. ARB prediction can help developers discover and remove ARBs, thus alleviating the impact of software aging. However, ARB-prone files occupy a small percentage of all the analyzed files. It is usually difficult to gather sufficient ARB data within a project. To overcome the limited availability of training data, several researchers have recently developed cross-project models for ARB prediction. A key point for cross-project models is to learn a good representation for instances in different projects. Nevertheless, most of the previous approaches neither consider the reconstruction property of new representation nor encode source samples' label information in learning representation. To address these shortcomings, we propose a Supervised Representation Learning Approach (SRLA), which is based on double encoding-layer autoencoder, to perform cross-project ARB prediction. Moreover, we present a transfer cross-validation framework to select the hyper-parameters of cross-project models. Experiments on three large open-source projects demonstrate the effectiveness and superiority of our approach compared with the state-of-the-art approach TLAP.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128121669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00028
L. Carnevali, Francesco Santoni, E. Vicario
Runtime predictive analysis of quantitative models can support software reliability in various application scenarios. The spread of logging technologies promotes approaches where such models are learned from observed events. We consider a system visiting transient states of a hidden process until reaching a final state and producing observations with stochastic arrival times and types conditioned by visited states, and we abstract it as a marked Markov modulated Poisson Process (MMMPP) with left-to right structure. We present an Expectation-Maximization (EM) algorithm that learns the MMMPP parameters from observation sequences acquired in repeated execution of the transient behavior, and we use the model at runtime to infer the current state of the process from actual observed events and to dynamically evaluate the remaining time to the final state. The approach is illustrated using synthetic datasets generated from a stochastic attack tree of the literature enriched with an observation model associating each state with an expected statistics of observation types and arrival times. Accuracy of prediction is evaluated under different variability of hidden states sojourn durations and of the observations arrival process, and compared against previous literature that mainly exploits either the timing or the types of observed events.
{"title":"Learning Marked Markov Modulated Poisson Processes for Online Predictive Analysis of Attack Scenarios","authors":"L. Carnevali, Francesco Santoni, E. Vicario","doi":"10.1109/ISSRE.2019.00028","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00028","url":null,"abstract":"Runtime predictive analysis of quantitative models can support software reliability in various application scenarios. The spread of logging technologies promotes approaches where such models are learned from observed events. We consider a system visiting transient states of a hidden process until reaching a final state and producing observations with stochastic arrival times and types conditioned by visited states, and we abstract it as a marked Markov modulated Poisson Process (MMMPP) with left-to right structure. We present an Expectation-Maximization (EM) algorithm that learns the MMMPP parameters from observation sequences acquired in repeated execution of the transient behavior, and we use the model at runtime to infer the current state of the process from actual observed events and to dynamically evaluate the remaining time to the final state. The approach is illustrated using synthetic datasets generated from a stochastic attack tree of the literature enriched with an observation model associating each state with an expected statistics of observation types and arrival times. Accuracy of prediction is evaluated under different variability of hidden states sojourn durations and of the observations arrival process, and compared against previous literature that mainly exploits either the timing or the types of observed events.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114830188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00013
Rick Salay, Matt Angus, K. Czarnecki
The use of machine learning (ML) is increasing in many sectors of safety-critical software development and in particular, for the perceptual components of automated driving (AD) functionality. Although some traditional safety engineering techniques such as FTA and FMEA are applicable to ML components, the unique characteristics of ML create challenges. In this paper, we propose a novel safety analysis method called Classification Failure Mode Effects Analysis (CFMEA) which is specialized to assess classification-based perception in AD. Specifically, it defines a systematic way to assess the risk due to classification failure under adversarial attacks or varying degrees of classification uncertainty across the perception-control linkage. We first present the theoretical and methodological foundations for CFMEA, and then demonstrate it by applying it to an AD case study using semantic segmentation perception trained with the Cityscapes driving dataset. Finally, we discuss how CFMEA results could be used to improve an ML-model.
{"title":"A Safety Analysis Method for Perceptual Components in Automated Driving","authors":"Rick Salay, Matt Angus, K. Czarnecki","doi":"10.1109/ISSRE.2019.00013","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00013","url":null,"abstract":"The use of machine learning (ML) is increasing in many sectors of safety-critical software development and in particular, for the perceptual components of automated driving (AD) functionality. Although some traditional safety engineering techniques such as FTA and FMEA are applicable to ML components, the unique characteristics of ML create challenges. In this paper, we propose a novel safety analysis method called Classification Failure Mode Effects Analysis (CFMEA) which is specialized to assess classification-based perception in AD. Specifically, it defines a systematic way to assess the risk due to classification failure under adversarial attacks or varying degrees of classification uncertainty across the perception-control linkage. We first present the theoretical and methodological foundations for CFMEA, and then demonstrate it by applying it to an AD case study using semantic segmentation perception trained with the Cityscapes driving dataset. Finally, we discuss how CFMEA results could be used to improve an ML-model.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"191 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122483428","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00034
Romain Cayre, V. Nicomette, G. Auriol, E. Alata, M. Kaâniche, G. Marconato
Internet of Things (IoT) devices are nowadays widely used in individual homes and factories. Securing these new systems becomes a priority. However, conducting security audits of these connected objects based on experimental evaluation is a challenging task: it requires the use of heterogeneous hardware components leading to a set of specialised software tools, generally incompatible with each other and often complex to use. In this paper, we present a security audit and penetration testing framework called Mirage. This framework, written in Python, is dedicated to the analysis of wireless communications commonly used by IoT devices, and provides a generic, modular, unified and low level audit environment that is easy to adapt to new protocols. The paper describes the software architecture of Mirage, its goals and main features, and presents a concrete example of security audit performed with this framework.
{"title":"Mirage: Towards a Metasploit-Like Framework for IoT","authors":"Romain Cayre, V. Nicomette, G. Auriol, E. Alata, M. Kaâniche, G. Marconato","doi":"10.1109/ISSRE.2019.00034","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00034","url":null,"abstract":"Internet of Things (IoT) devices are nowadays widely used in individual homes and factories. Securing these new systems becomes a priority. However, conducting security audits of these connected objects based on experimental evaluation is a challenging task: it requires the use of heterogeneous hardware components leading to a set of specialised software tools, generally incompatible with each other and often complex to use. In this paper, we present a security audit and penetration testing framework called Mirage. This framework, written in Python, is dedicated to the analysis of wireless communications commonly used by IoT devices, and provides a generic, modular, unified and low level audit environment that is easy to adapt to new protocols. The paper describes the software architecture of Mirage, its goals and main features, and presents a concrete example of security audit performed with this framework.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"80 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129494305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00026
João R. Campos, M. Vieira, E. Costa
The growing complexity of software calls for innovative solutions that support the deployment of reliable and secure software. Machine Learning (ML) has shown its applicability to various complex problems and is frequently used in the dependability domain, both for supporting systems design and verification activities. However, using ML is complex and highly dependent on the problem in hand, increasing the probability of mistakes that compromise the results. In this paper, we introduce Propheticus, a ML framework that can be used to create predictive models for reliable and secure software systems. Propheticus attempts to abstract the complexity of ML whilst being easy to use and accommodating the needs of the users. To demonstrate its use, we present two case studies (vulnerability prediction and online failure prediction) that show how it can considerably ease and expedite a thorough ML workflow.
{"title":"Propheticus: Machine Learning Framework for the Development of Predictive Models for Reliable and Secure Software","authors":"João R. Campos, M. Vieira, E. Costa","doi":"10.1109/ISSRE.2019.00026","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00026","url":null,"abstract":"The growing complexity of software calls for innovative solutions that support the deployment of reliable and secure software. Machine Learning (ML) has shown its applicability to various complex problems and is frequently used in the dependability domain, both for supporting systems design and verification activities. However, using ML is complex and highly dependent on the problem in hand, increasing the probability of mistakes that compromise the results. In this paper, we introduce Propheticus, a ML framework that can be used to create predictive models for reliable and secure software systems. Propheticus attempts to abstract the complexity of ML whilst being easy to use and accommodating the needs of the users. To demonstrate its use, we present two case studies (vulnerability prediction and online failure prediction) that show how it can considerably ease and expedite a thorough ML workflow.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129912062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00016
Jingjing Liang, Yaozong Hou, Shurui Zhou, Junjie Chen, Y. Xiong, Gang Huang
Abstract-Bugs are inevitable in software development and maintenance processes. Recently a lot of research efforts have been devoted to automatic program repair, aiming to reduce the efforts of debugging. However, since it is difficult to ensure that the generated patches meet all quality requirements such as correctness, developers still need to review the patch. In addition, current techniques produce only patches without explanation, making it difficult for the developers to understand the patch. Therefore, we believe a more desirable approach should generate not only the patch but also an explanation of the patch. To generate a patch explanation, it is important to first understand how patches were explained. In this paper, we explored how developers explain their patches by manually analyzing 300 merged bug-fixing pull requests from six projects on GitHub. Our contribution is twofold. First, we build a patch explanation model, which summarizes the elements in a patch explanation, and corresponding expressive forms. Second, we conducted a quantitative analysis to understand the distributions of elements, and the correlation between elements and their expressive forms.
{"title":"How to Explain a Patch: An Empirical Study of Patch Explanations in Open Source Projects","authors":"Jingjing Liang, Yaozong Hou, Shurui Zhou, Junjie Chen, Y. Xiong, Gang Huang","doi":"10.1109/ISSRE.2019.00016","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00016","url":null,"abstract":"Abstract-Bugs are inevitable in software development and maintenance processes. Recently a lot of research efforts have been devoted to automatic program repair, aiming to reduce the efforts of debugging. However, since it is difficult to ensure that the generated patches meet all quality requirements such as correctness, developers still need to review the patch. In addition, current techniques produce only patches without explanation, making it difficult for the developers to understand the patch. Therefore, we believe a more desirable approach should generate not only the patch but also an explanation of the patch. To generate a patch explanation, it is important to first understand how patches were explained. In this paper, we explored how developers explain their patches by manually analyzing 300 merged bug-fixing pull requests from six projects on GitHub. Our contribution is twofold. First, we build a patch explanation model, which summarizes the elements in a patch explanation, and corresponding expressive forms. Second, we conducted a quantitative analysis to understand the distributions of elements, and the correlation between elements and their expressive forms.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134451653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-10-01DOI: 10.1109/ISSRE.2019.00048
K. Goseva-Popstojanova, Thomas Kyanko, Noble Nkwocha
Even though Model-based Software Engineering (MBSwE) techniques and Autogenerated Code (AGC) have been increasingly used to produce complex software systems, there is only anecdotal knowledge about the state-of-the practice. Furthermore, there is a lack of empirical studies that explore the potential quality improvements due to the use of these techniques. This paper presents in-depth qualitative findings about development and Software Assurance (SWA) practices and detailed quantitative analysis of software bug reports of a NASA mission that used MBSwE and AGC. The mission's flight software is a combination of handwritten code and AGC developed by two different approaches: one based on state chart models (AGC-M) and another on specification dictionaries (AGC-D). The empirical analysis of fault proneness is based on 380 closed bug reports created by software developers. Our main findings include: (1) MBSwE and AGC provide some benefits, but also impose challenges. (2) SWA done only at a model level is not sufficient. AGC code should also be tested and the models and AGC should always be kept in-sync. AGC must not be changed manually. (3) Fixes made to address an individual bug report were spread both across multiple modules and across multiple files. On average, for each bug report 1.4 modules, that is, 3.4 files were fixed. (4) Most bug reports led to changes in more than one type of file. The majority of changes to auto-generated source code files were made in conjunction to changes in either file with state chart models or XML files derived from dictionaries. (5) For newly developed files, AGC-M and handwritten code were of similar quality, while AGC-D files were the least fault prone.
{"title":"Benefits and Challenges of Model-Based Software Engineering: Lessons Learned Based on Qualitative and Quantitative Findings","authors":"K. Goseva-Popstojanova, Thomas Kyanko, Noble Nkwocha","doi":"10.1109/ISSRE.2019.00048","DOIUrl":"https://doi.org/10.1109/ISSRE.2019.00048","url":null,"abstract":"Even though Model-based Software Engineering (MBSwE) techniques and Autogenerated Code (AGC) have been increasingly used to produce complex software systems, there is only anecdotal knowledge about the state-of-the practice. Furthermore, there is a lack of empirical studies that explore the potential quality improvements due to the use of these techniques. This paper presents in-depth qualitative findings about development and Software Assurance (SWA) practices and detailed quantitative analysis of software bug reports of a NASA mission that used MBSwE and AGC. The mission's flight software is a combination of handwritten code and AGC developed by two different approaches: one based on state chart models (AGC-M) and another on specification dictionaries (AGC-D). The empirical analysis of fault proneness is based on 380 closed bug reports created by software developers. Our main findings include: (1) MBSwE and AGC provide some benefits, but also impose challenges. (2) SWA done only at a model level is not sufficient. AGC code should also be tested and the models and AGC should always be kept in-sync. AGC must not be changed manually. (3) Fixes made to address an individual bug report were spread both across multiple modules and across multiple files. On average, for each bug report 1.4 modules, that is, 3.4 files were fixed. (4) Most bug reports led to changes in more than one type of file. The majority of changes to auto-generated source code files were made in conjunction to changes in either file with state chart models or XML files derived from dictionaries. (5) For newly developed files, AGC-M and handwritten code were of similar quality, while AGC-D files were the least fault prone.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127711372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：