A relevant objective of software reliability assessment is to get unbiased estimates with an acceptable trade-off between the number of tests required and the variance of the estimate. A low variance is desirable to increase the confidence in the estimate, but too many tests may be required by conventional reliability assessment testing techniques based solely on the operational profile. This article presents probabilistic sampling-based testing, a new technique using unequal probability sampling to exploit auxiliary information about the software under test so as to assess reliability unbiasedly and efficiently. The technique expedites the assessment process assuming the availability of some prior belief about input regions failure proneness. The evaluation by simulation and experimentally shows promising results in terms of estimate accuracy and efficiency.
{"title":"Probabilistic Sampling-Based Testing for Accelerated Reliability Assessment","authors":"R. Pietrantuono, S. Russo","doi":"10.1109/QRS.2018.00017","DOIUrl":"https://doi.org/10.1109/QRS.2018.00017","url":null,"abstract":"A relevant objective of software reliability assessment is to get unbiased estimates with an acceptable trade-off between the number of tests required and the variance of the estimate. A low variance is desirable to increase the confidence in the estimate, but too many tests may be required by conventional reliability assessment testing techniques based solely on the operational profile. This article presents probabilistic sampling-based testing, a new technique using unequal probability sampling to exploit auxiliary information about the software under test so as to assess reliability unbiasedly and efficiently. The technique expedites the assessment process assuming the availability of some prior belief about input regions failure proneness. The evaluation by simulation and experimentally shows promising results in terms of estimate accuracy and efficiency.","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126068263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Naohiko Tsuda, H. Washizaki, Y. Fukazawa, Y. Yasuda, Shunsuke Sugimura
Evolvability defects are non-understandable and non-modifiable states that do not directly produce runtime behavioral failures. Automatic source code evaluation by metrics and thresholds can help reduce the burden of a manual inspection. This study addresses two problems. (1) Evolvability defects are not usually managed in bug tracking systems. (2) Conventional methods cannot fully interpret the relations among the metrics in a given context (e.g., programming language, application domain). The key actions of our method are to (1) gather training-data for machine learning by experts' manual inspection of some of the files in given systems (benchmark) and (2) employ a classification-tree learner algorithm, C5.0, which can deal with non-orthogonal relations between metrics. Furthermore, we experimentally confirm that, even with less training-data, our method provides a more precise evaluation than four conventional methods (the percentile, Alves' method, Bender's method, and the ROC curve-based method).
{"title":"Machine Learning to Evaluate Evolvability Defects: Code Metrics Thresholds for a Given Context","authors":"Naohiko Tsuda, H. Washizaki, Y. Fukazawa, Y. Yasuda, Shunsuke Sugimura","doi":"10.1109/QRS.2018.00022","DOIUrl":"https://doi.org/10.1109/QRS.2018.00022","url":null,"abstract":"Evolvability defects are non-understandable and non-modifiable states that do not directly produce runtime behavioral failures. Automatic source code evaluation by metrics and thresholds can help reduce the burden of a manual inspection. This study addresses two problems. (1) Evolvability defects are not usually managed in bug tracking systems. (2) Conventional methods cannot fully interpret the relations among the metrics in a given context (e.g., programming language, application domain). The key actions of our method are to (1) gather training-data for machine learning by experts' manual inspection of some of the files in given systems (benchmark) and (2) employ a classification-tree learner algorithm, C5.0, which can deal with non-orthogonal relations between metrics. Furthermore, we experimentally confirm that, even with less training-data, our method provides a more precise evaluation than four conventional methods (the percentile, Alves' method, Bender's method, and the ROC curve-based method).","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132586402","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The release of Intel's Software Guard Extensions (SGX) refueled the interest in trusted computing approaches across industry and academia. The corresponding hardware is available, but practical usage patterns and applications are still lacking notable prevalence rates. This paper addresses this gap by approaching trusted computing from the point of view of a software engineer. To help developers in overcoming the initial hurdles of integrating SGX with existing code bases, a small helper library is presented. Furthermore, hardening strategies are identified and applied in a case study based on the simple KISSDB database, demonstrating how SGX can be used in practice.
{"title":"Hardening Application Security Using Intel SGX","authors":"Max Plauth, Fredrik Teschke, D. Richter, A. Polze","doi":"10.1109/QRS.2018.00050","DOIUrl":"https://doi.org/10.1109/QRS.2018.00050","url":null,"abstract":"The release of Intel's Software Guard Extensions (SGX) refueled the interest in trusted computing approaches across industry and academia. The corresponding hardware is available, but practical usage patterns and applications are still lacking notable prevalence rates. This paper addresses this gap by approaching trusted computing from the point of view of a software engineer. To help developers in overcoming the initial hurdles of integrating SGX with existing code bases, a small helper library is presented. Furthermore, hardening strategies are identified and applied in a case study based on the simple KISSDB database, demonstrating how SGX can be used in practice.","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"16 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132794978","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Teng Wang, Xiaodong Liu, Shanshan Li, Xiangke Liao, Wang Li, Qing Liao
As software configurations continue to grow in complexity, misconfiguration has become one of major causes of software failure. Software configuration errors can have catastrophic consequences, seriously affecting the normal use of software and quality of service. And misconfiguration diagnosis faces many challenges, such as path-explosion problems and incomplete statistical data. Our study of the log that is generated in response to misconfigurations by six widely used pieces of software highlights some interesting characteristics. These observations have influenced the design of MisconfDoctor, a misconfiguration diagnosis tool via log-based configuration testing. Through comprehensive misconfiguration testing, MisconfDoctor first extracts log features for every misconfiguration and builds a feature database. When a system misconfiguration occurs, MisconfDoctor suggests potential misconfigurations by calculating the similarity of the new exception log to the feature database. We use manual and real-world error cases from Httpd, MySQL and PostgreSQL in order to evaluate the effectiveness of the tool. Experimental results demonstrate that the tool's accuracy reaches 85% when applied to manual-error cases, and 78% for real-world cases.
{"title":"MisconfDoctor: Diagnosing Misconfiguration via Log-Based Configuration Testing","authors":"Teng Wang, Xiaodong Liu, Shanshan Li, Xiangke Liao, Wang Li, Qing Liao","doi":"10.1109/QRS.2018.00014","DOIUrl":"https://doi.org/10.1109/QRS.2018.00014","url":null,"abstract":"As software configurations continue to grow in complexity, misconfiguration has become one of major causes of software failure. Software configuration errors can have catastrophic consequences, seriously affecting the normal use of software and quality of service. And misconfiguration diagnosis faces many challenges, such as path-explosion problems and incomplete statistical data. Our study of the log that is generated in response to misconfigurations by six widely used pieces of software highlights some interesting characteristics. These observations have influenced the design of MisconfDoctor, a misconfiguration diagnosis tool via log-based configuration testing. Through comprehensive misconfiguration testing, MisconfDoctor first extracts log features for every misconfiguration and builds a feature database. When a system misconfiguration occurs, MisconfDoctor suggests potential misconfigurations by calculating the similarity of the new exception log to the feature database. We use manual and real-world error cases from Httpd, MySQL and PostgreSQL in order to evaluate the effectiveness of the tool. Experimental results demonstrate that the tool's accuracy reaches 85% when applied to manual-error cases, and 78% for real-world cases.","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131062114","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Publisher's Information","authors":"","doi":"10.1109/qrs.2018.00065","DOIUrl":"https://doi.org/10.1109/qrs.2018.00065","url":null,"abstract":"","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128432395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Coordination is important in software development. Socio-Technical Congruence (STC) is proposed to measure the match between coordination requirements and actual coordination activities, and has been proved to have impact on software failures in commercial projects. Continuous defect prediction is aimed to predict defects just in time, which is more meaningful than traditional defect prediction in practice. In this paper, we compute the build-level STC and investigate its usefulness in continuous defect prediction based on 10 GitHub projects. We find that adding STC metrics into logistic regression models can significantly improve both the explanatory power and the predictive power when predicting build failures. Furthermore, we compare the performance of STC and MDL from the aspects of regression and prediction. MDL is short for Missing Developer Links, a deviation of the STC metric. We find that MDL usually performs better than STC. Our work is promising to help detect coordination issues during real time process of software development.
{"title":"Does Socio-Technical Congruence Have an Effect on Continuous Integration Build Failures? An Empirical Study on 10 GitHub Projects","authors":"Weiqiang Zhang, Zhenyu Chen, B. Luo","doi":"10.1109/QRS.2018.00046","DOIUrl":"https://doi.org/10.1109/QRS.2018.00046","url":null,"abstract":"Coordination is important in software development. Socio-Technical Congruence (STC) is proposed to measure the match between coordination requirements and actual coordination activities, and has been proved to have impact on software failures in commercial projects. Continuous defect prediction is aimed to predict defects just in time, which is more meaningful than traditional defect prediction in practice. In this paper, we compute the build-level STC and investigate its usefulness in continuous defect prediction based on 10 GitHub projects. We find that adding STC metrics into logistic regression models can significantly improve both the explanatory power and the predictive power when predicting build failures. Furthermore, we compare the performance of STC and MDL from the aspects of regression and prediction. MDL is short for Missing Developer Links, a deviation of the STC metric. We find that MDL usually performs better than STC. Our work is promising to help detect coordination issues during real time process of software development.","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116912858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We introduce a prototype testing framework as an extension of JUnit for testing actor-based systems. Our framework runs a given JUnit test in various schedules and records the execution trace for each run. In case a failure is observed during one of the test runs, the corresponding execution trace is compared with those traces associated with passed test runs. The sequence of exchanged messages that are different with respect to these runs are extracted. A UML sequence diagram is automatically generated to depict this sequence. Hence, the developer can observe the interleaving of messages that caused the failure. We illustrated the application of our framework on two sample actor-based software systems.
{"title":"Towards a Testing Framework with Visual Feedback for Actor-Based Systems","authors":"Hasan Sözer, Ozan Gürler, Orhan Yilmaz","doi":"10.1109/QRS.2018.00057","DOIUrl":"https://doi.org/10.1109/QRS.2018.00057","url":null,"abstract":"We introduce a prototype testing framework as an extension of JUnit for testing actor-based systems. Our framework runs a given JUnit test in various schedules and records the execution trace for each run. In case a failure is observed during one of the test runs, the corresponding execution trace is compared with those traces associated with passed test runs. The sequence of exchanged messages that are different with respect to these runs are extracted. A UML sequence diagram is automatically generated to depict this sequence. Hence, the developer can observe the interleaving of messages that caused the failure. We illustrated the application of our framework on two sample actor-based software systems.","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129605314","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andreas Schörgenhumer, Mario Kahlhofer, H. Mössenböck, P. Grünbacher
Faults are common in large software systems and must be analyzed to prevent future failures such as system outages. Due to their sheer amount, the observed failures cannot be inspected individually but must be automatically grouped and prioritized. An open challenge is to find similarities in failures across different systems. We propose a novel approach for identifying error-prone software technologies via a cross-system analysis based on monitoring and crash data. Our approach ranks the error-prone software technologies and analyzes the occurred exceptions, thus making it easier for developers to investigate cross-system failures. Finding such failures is highly advantageous as fixing a fault may benefit many affected systems. A preliminary case study on monitoring data of hundreds of different systems demonstrates the feasibility of our approach.
{"title":"Using Crash Frequency Analysis to Identify Error-Prone Software Technologies in Multi-System Monitoring","authors":"Andreas Schörgenhumer, Mario Kahlhofer, H. Mössenböck, P. Grünbacher","doi":"10.1109/QRS.2018.00032","DOIUrl":"https://doi.org/10.1109/QRS.2018.00032","url":null,"abstract":"Faults are common in large software systems and must be analyzed to prevent future failures such as system outages. Due to their sheer amount, the observed failures cannot be inspected individually but must be automatically grouped and prioritized. An open challenge is to find similarities in failures across different systems. We propose a novel approach for identifying error-prone software technologies via a cross-system analysis based on monitoring and crash data. Our approach ranks the error-prone software technologies and analyzes the occurred exceptions, thus making it easier for developers to investigate cross-system failures. Finding such failures is highly advantageous as fixing a fault may benefit many affected systems. A preliminary case study on monitoring data of hundreds of different systems demonstrates the feasibility of our approach.","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"103 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114010565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
James Jerson Ortiz Vega, Gilles Perrouin, Moussa Amrani, Pierre-Yves Schobbens
Mutation testing relies on the principle of artificially injecting faults in systems to create mutants, in order to either assess the sensitivity of existing test suites, or generate test cases that are able to find real faults. Mutation testing has been employed in a variety of application areas and at various levels of abstraction (code and models). In this paper, we focus on model-based mutation testing for timed systems. In order to cartography the field, we provide a taxonomy of mutation operators and discuss their usages on various formalisms, such as timed automata or synchronous languages. We also delineate a research agenda for the field addressing mutation costs, the impact of delays in operators specification and mutation equivalence.
{"title":"Model-Based Mutation Operators for Timed Systems: A Taxonomy and Research Agenda","authors":"James Jerson Ortiz Vega, Gilles Perrouin, Moussa Amrani, Pierre-Yves Schobbens","doi":"10.1109/QRS.2018.00045","DOIUrl":"https://doi.org/10.1109/QRS.2018.00045","url":null,"abstract":"Mutation testing relies on the principle of artificially injecting faults in systems to create mutants, in order to either assess the sensitivity of existing test suites, or generate test cases that are able to find real faults. Mutation testing has been employed in a variety of application areas and at various levels of abstraction (code and models). In this paper, we focus on model-based mutation testing for timed systems. In order to cartography the field, we provide a taxonomy of mutation operators and discuss their usages on various formalisms, such as timed automata or synchronous languages. We also delineate a research agenda for the field addressing mutation costs, the impact of delays in operators specification and mutation equivalence.","PeriodicalId":114973,"journal":{"name":"2018 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130411838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：