During the past decade, much effort has been invested in developing standards to overcome data and software interoperability barriers in the oil and gas industry. Whereas syntactical integration is no longer problem, semantic integration still remains an open challenge. To overcome this problem, standards provide more and more complex structures to capture the semantics of different domains. As standards become more expressive, they may become too complex so that their adaptation is inherently difficult, which may even lead to inconsistencies in specifications. This paper addresses these issues by simplification and verification of correctness properties on an interoperability standard. While the simplification has been achieved by means of multi-level modeling, the accuracy of an industry model has been addressed by verifying the correctness properties using the object-oriented rule-based system Flora-2. Transformation and verification rules were evaluated and their performance has been enhanced by applying algorithmic improvements. The outcomes show that the approach reduces complexity of specifications and supports the verification from a software engineering point of view to simplify the adoption of standards.
{"title":"Simplification and Verification of Software and Data Structure Models in the Engineering Domain","authors":"Muzaffar Igamberdiev, G. Grossmann, M. Stumptner","doi":"10.1109/ASWEC.2015.25","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.25","url":null,"abstract":"During the past decade, much effort has been invested in developing standards to overcome data and software interoperability barriers in the oil and gas industry. Whereas syntactical integration is no longer problem, semantic integration still remains an open challenge. To overcome this problem, standards provide more and more complex structures to capture the semantics of different domains. As standards become more expressive, they may become too complex so that their adaptation is inherently difficult, which may even lead to inconsistencies in specifications. This paper addresses these issues by simplification and verification of correctness properties on an interoperability standard. While the simplification has been achieved by means of multi-level modeling, the accuracy of an industry model has been addressed by verifying the correctness properties using the object-oriented rule-based system Flora-2. Transformation and verification rules were evaluated and their performance has been enhanced by applying algorithmic improvements. The outcomes show that the approach reduces complexity of specifications and supports the verification from a software engineering point of view to simplify the adoption of standards.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117164994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ensuring that mobile device applications are free from bugs is difficult. The difficulty stems, in part, from the unpredictability inherent in the use of multiple mobile networks. Testers typically design and run tests to discover bugs, modify the software and then re-run the test to demonstrate the bug's absence. However, when using wireless networks, it is difficult to recreate the original operating conditions to re-run the test. Other challenges associated with testing software for mobile applications include determining the effects of adding concurrent devices to a system, and the need to manage user interaction during testing. In this paper, we introduce the Odin Test Harness, which is a tool that facilitates a simulated and controlled execution environment for testing middleware and application software involving mobile devices. In particular, the tool provides for fine-grained control over network characteristics including bandwidth, packet loss rates, latency and network interface availability. Moreover the test harness allows mobile apps to be programmatically manipulated, simulating user input, during test execution. Furthermore, the test harness enables testers to simulate load on the system and to help determine its scalability. We have evaluated the test harness' on our middleware platform for hosting mobile services and a healthcare monitoring application. The test harness has been found to be useful in finding bugs that would otherwise remain elusive. In particular, the tool's support for repeatable tests, tests that involve multiple devices, and tests that require user interaction has readied the healthcare application for use in a clinical trial.
确保移动设备应用程序没有漏洞是很困难的。这种困难部分源于使用多种移动网络所固有的不可预测性。测试人员通常设计和运行测试来发现缺陷，修改软件，然后重新运行测试来证明缺陷的存在。然而，当使用无线网络时，很难重现原始的操作条件来重新运行测试。与测试移动应用软件相关的其他挑战包括确定将并发设备添加到系统中的效果，以及在测试期间管理用户交互的需要。在本文中，我们介绍了Odin Test Harness，它是一个工具，为测试涉及移动设备的中间件和应用软件提供了一个模拟和控制的执行环境。特别是，该工具提供了对网络特征的细粒度控制，包括带宽、丢包率、延迟和网络接口可用性。此外，测试工具允许在测试执行期间以编程方式操纵移动应用程序，模拟用户输入。此外，测试工具使测试人员能够模拟系统上的负载并帮助确定其可扩展性。我们已经在中间件平台上评估了用于托管移动服务和医疗监控应用程序的测试工具。人们发现，测试工具在发现bug方面非常有用，否则这些bug将难以捉摸。特别是，该工具支持可重复测试、涉及多个设备的测试以及需要用户交互的测试，这使医疗保健应用程序为临床试验做好了准备。
{"title":"A Test Harness for Networked Mobile Applications and Middleware","authors":"A. Meads, H. Naderi, I. Warren","doi":"10.1109/ASWEC.2015.11","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.11","url":null,"abstract":"Ensuring that mobile device applications are free from bugs is difficult. The difficulty stems, in part, from the unpredictability inherent in the use of multiple mobile networks. Testers typically design and run tests to discover bugs, modify the software and then re-run the test to demonstrate the bug's absence. However, when using wireless networks, it is difficult to recreate the original operating conditions to re-run the test. Other challenges associated with testing software for mobile applications include determining the effects of adding concurrent devices to a system, and the need to manage user interaction during testing. In this paper, we introduce the Odin Test Harness, which is a tool that facilitates a simulated and controlled execution environment for testing middleware and application software involving mobile devices. In particular, the tool provides for fine-grained control over network characteristics including bandwidth, packet loss rates, latency and network interface availability. Moreover the test harness allows mobile apps to be programmatically manipulated, simulating user input, during test execution. Furthermore, the test harness enables testers to simulate load on the system and to help determine its scalability. We have evaluated the test harness' on our middleware platform for hosting mobile services and a healthcare monitoring application. The test harness has been found to be useful in finding bugs that would otherwise remain elusive. In particular, the tool's support for repeatable tests, tests that involve multiple devices, and tests that require user interaction has readied the healthcare application for use in a clinical trial.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115247458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Compositional symbolic execution has been proposed as a way to increase the efficiency of symbolic execution. Essentially, when a function is symbolically executed, a summary of the path that was executed is stored. This summary records the precondition and post condition of the path, and on subsequent calls that satisfy that precondition, the corresponding post condition can be returned instead of executing the function again. However, using functions as the unit of summarisation leaves the symbolic execution tool at the mercy of a program designer, essentially resulting in an arbitrary summarisation strategy. In this paper, we explore the use of fine-grained summaries, in which blocks within functions are summarised. We propose three types of summarisation and demonstrate how to generate these. At such a fine-grained level, symbolic execution of a path effectively becomes the concatenation of the summaries along that path. Using a prototype symbolic execution tool, we perform a preliminary experimental evaluation of our summary approaches, demonstrating that they can improve the speed of symbolic execution by reducing the number of calls sent to the underlying constraint solver.
{"title":"Compositional Symbolic Execution Using Fine-Grained Summaries","authors":"Yude Lin, Tim Miller, H. Søndergaard","doi":"10.1109/ASWEC.2015.32","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.32","url":null,"abstract":"Compositional symbolic execution has been proposed as a way to increase the efficiency of symbolic execution. Essentially, when a function is symbolically executed, a summary of the path that was executed is stored. This summary records the precondition and post condition of the path, and on subsequent calls that satisfy that precondition, the corresponding post condition can be returned instead of executing the function again. However, using functions as the unit of summarisation leaves the symbolic execution tool at the mercy of a program designer, essentially resulting in an arbitrary summarisation strategy. In this paper, we explore the use of fine-grained summaries, in which blocks within functions are summarised. We propose three types of summarisation and demonstrate how to generate these. At such a fine-grained level, symbolic execution of a path effectively becomes the concatenation of the summaries along that path. Using a prototype symbolic execution tool, we perform a preliminary experimental evaluation of our summary approaches, demonstrating that they can improve the speed of symbolic execution by reducing the number of calls sent to the underlying constraint solver.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134061175","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jean-Guy Schneider, Peter Mandile, Steven Versteeg
Assuring quality of contemporary software systems is a very challenging task due to the often large complexity of the deployment environments in which they will operate. Service virtualization is an approach to this challenge where services within the deployment environment are emulated by synthesising service response messages from models or by recording and then replaying service interaction messages with the system. Record-and-replay techniques require an approach where (i) message prototypes can be derived from recorded system interactions (i.e. Request-response sequences), (ii) a scheme to match incoming request messages against message prototypes, and (iii) the synthesis of response messages based on similarities between incoming messages and the recorded system interactions. Previous approaches in service virtualization have required a multiple sequence alignment (MSA) algorithm as a means of finding common patterns of similarities and differences between messages required by all three steps. In this paper, we present a novel MSA algorithm based on Generalized Suffix Trees (GSTs). We evaluated the accuracy and efficiency of the proposed algorithm against six enterprise service message trace datasets, with the proposed algorithm performing up to 50 times faster than standard MSA approaches. Furthermore, the algorithm has applicability to other domains beyond service virtualization.
{"title":"Generalized Suffix Tree Based Multiple Sequence Alignment for Service Virtualization","authors":"Jean-Guy Schneider, Peter Mandile, Steven Versteeg","doi":"10.1109/ASWEC.2015.13","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.13","url":null,"abstract":"Assuring quality of contemporary software systems is a very challenging task due to the often large complexity of the deployment environments in which they will operate. Service virtualization is an approach to this challenge where services within the deployment environment are emulated by synthesising service response messages from models or by recording and then replaying service interaction messages with the system. Record-and-replay techniques require an approach where (i) message prototypes can be derived from recorded system interactions (i.e. Request-response sequences), (ii) a scheme to match incoming request messages against message prototypes, and (iii) the synthesis of response messages based on similarities between incoming messages and the recorded system interactions. Previous approaches in service virtualization have required a multiple sequence alignment (MSA) algorithm as a means of finding common patterns of similarities and differences between messages required by all three steps. In this paper, we present a novel MSA algorithm based on Generalized Suffix Trees (GSTs). We evaluated the accuracy and efficiency of the proposed algorithm against six enterprise service message trace datasets, with the proposed algorithm performing up to 50 times faster than standard MSA approaches. Furthermore, the algorithm has applicability to other domains beyond service virtualization.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"134 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126864754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nowadays, individuals' personal data is at their disposal real-time from any device through cloud storage services (CSS). Such services enable new kinds of collaboration between individuals and have fundamental impact on how we organize and share our data. Nevertheless, only a few studies have been made on the user experience (UX) of such services. This paper reports the user experience of different CSSs (focusing on Drop box, Google Drive, One Drive, and iCloud) based on 19 interviews and 65 survey responses. The results include reasons for the most positive and negative experiences and descriptions of current habits and motivations of the CSS users. As the current use of CSSs is still mostly individual we investigate the potential of taking the UX of CSSs to the next level by integrating different social features to current CSSs. We conclude the paper by explaining the importance of different Cloud UX aspects in CSS context and suggesting design implications improving UX for CSSs.
{"title":"The Next Level of User Experience of Cloud Storage Services: Supporting Collaboration with Social Features","authors":"J. Palviainen, Parisa Pour Rezaei","doi":"10.1109/ASWEC.2015.31","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.31","url":null,"abstract":"Nowadays, individuals' personal data is at their disposal real-time from any device through cloud storage services (CSS). Such services enable new kinds of collaboration between individuals and have fundamental impact on how we organize and share our data. Nevertheless, only a few studies have been made on the user experience (UX) of such services. This paper reports the user experience of different CSSs (focusing on Drop box, Google Drive, One Drive, and iCloud) based on 19 interviews and 65 survey responses. The results include reasons for the most positive and negative experiences and descriptions of current habits and motivations of the CSS users. As the current use of CSSs is still mostly individual we investigate the potential of taking the UX of CSSs to the next level by integrating different social features to current CSSs. We conclude the paper by explaining the importance of different Cloud UX aspects in CSS context and suggesting design implications improving UX for CSSs.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"1360 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126996585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qi Shan, Guoping Rong, He Zhang, Gaoxuan Liu, Dong Shao
Context: Capture-recapture approach has been adopted in software inspection for decades for estimating remaining defects and supporting post-inspection decisions. A number of capture-recapture models and estimators have been borrowed from other disciplines (e.g., Biology and social sciences) and applied in software inspections. These models and estimators were created with different assumptions and the violation to these assumptions may lead to invalid application. Most of the reported empirical evaluations of the capture-recapture models and estimators used relatively small data sets (i.e., Small number of inspectors and/or defects), which leads varying conclusions. Objective: The research reported in this paper aims to better understand this approach for software inspection, and the efficacy of the models and estimators in particular, with large data sets. Method: We carried out an empirical study that evaluated several typical capture-recapture estimators (i.e., M0-ML, Mt-ML, Mt-CH, Mh-CH and Mh-JK). The study employed 57 student inspectors in reviewing software requirement specification and formed 1000 virtual teams at each team size in the post statistic testing. Results: The data analysis indicates that: 1) M0-ML and Mt-ML could not generate accurate estimates even with many inspectors. 2) All estimators suffer high level of failure rate with few inspectors. 3) For a small team, estimator Mh-CH performs the best, while with many inspectors, Mt-CH and Mh-JK perform better regarding the accuracy and the failure rate. Conclusion: Our work reveals that no estimator was superior to others under all situations. With an increased team size, estimates generated by estimator Mt-CH and Mh-JK turn to be more accurate than others. While, M0-ML and Mt-ML are able to offer reference value to support post-inspection because of their convergent underestimates.
{"title":"An Empirical Evaluation of Capture-Recapture Estimators in Software Inspection","authors":"Qi Shan, Guoping Rong, He Zhang, Gaoxuan Liu, Dong Shao","doi":"10.1109/ASWEC.2015.17","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.17","url":null,"abstract":"Context: Capture-recapture approach has been adopted in software inspection for decades for estimating remaining defects and supporting post-inspection decisions. A number of capture-recapture models and estimators have been borrowed from other disciplines (e.g., Biology and social sciences) and applied in software inspections. These models and estimators were created with different assumptions and the violation to these assumptions may lead to invalid application. Most of the reported empirical evaluations of the capture-recapture models and estimators used relatively small data sets (i.e., Small number of inspectors and/or defects), which leads varying conclusions. Objective: The research reported in this paper aims to better understand this approach for software inspection, and the efficacy of the models and estimators in particular, with large data sets. Method: We carried out an empirical study that evaluated several typical capture-recapture estimators (i.e., M0-ML, Mt-ML, Mt-CH, Mh-CH and Mh-JK). The study employed 57 student inspectors in reviewing software requirement specification and formed 1000 virtual teams at each team size in the post statistic testing. Results: The data analysis indicates that: 1) M0-ML and Mt-ML could not generate accurate estimates even with many inspectors. 2) All estimators suffer high level of failure rate with few inspectors. 3) For a small team, estimator Mh-CH performs the best, while with many inspectors, Mt-CH and Mh-JK perform better regarding the accuracy and the failure rate. Conclusion: Our work reveals that no estimator was superior to others under all situations. With an increased team size, estimates generated by estimator Mt-CH and Mh-JK turn to be more accurate than others. While, M0-ML and Mt-ML are able to offer reference value to support post-inspection because of their convergent underestimates.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115727694","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Architectural component models represent high level designs and are frequently used as a central view of architectural descriptions of software systems. They play a crucial role in the whole development process and in achieving the desired software qualities. This paper presents an empirical study that examines the impact of hierarchies on the architecture-level software understand ability. In particular we have studied three different architectural representations of a large-size software system, one with a hierarchical representation where architectural components at all abstraction levels in the hierarchy are shown, and two that do not contain hierarchical abstractions but concentrate only on the lowest level or on the highest level components in the hierarchy. We conducted a controlled experiment in which participants of three groups received one of the three architecture documentations plus the source code of the system and had to answer understand ability related questions. Our results show that using the hierarchical architecture leads to: 1) higher quantity of correctly retrieved elements, 2) lower quantity of incorrectly retrieved elements, and 3) higher overall quality of retrieved elements. The obtained results provide empirical evidence that hierarchies play an important role in the context of architectural component models from the viewpoint of the architecture-level software understandability.
{"title":"The Impact of Hierarchies on the Architecture-Level Software Understandability - A Controlled Experiment","authors":"S. Stevanetic, M. Javed, Uwe Zdun","doi":"10.1109/ASWEC.2015.21","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.21","url":null,"abstract":"Architectural component models represent high level designs and are frequently used as a central view of architectural descriptions of software systems. They play a crucial role in the whole development process and in achieving the desired software qualities. This paper presents an empirical study that examines the impact of hierarchies on the architecture-level software understand ability. In particular we have studied three different architectural representations of a large-size software system, one with a hierarchical representation where architectural components at all abstraction levels in the hierarchy are shown, and two that do not contain hierarchical abstractions but concentrate only on the lowest level or on the highest level components in the hierarchy. We conducted a controlled experiment in which participants of three groups received one of the three architecture documentations plus the source code of the system and had to answer understand ability related questions. Our results show that using the hierarchical architecture leads to: 1) higher quantity of correctly retrieved elements, 2) lower quantity of incorrectly retrieved elements, and 3) higher overall quality of retrieved elements. The obtained results provide empirical evidence that hierarchies play an important role in the context of architectural component models from the viewpoint of the architecture-level software understandability.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"67 9","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132026773","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Human-induced faults play a large role in systems reliability. In cloud platforms, system administrators may inadvertently make catastrophic mistakes, like deleting a virtual disk with important data. Providing rollback for cloud operations can reduce the severity and impact of such mistakes by allowing to revert back to a known, good state. In this paper, we present a scalable approach to rollback operations that change state of a system on proprietary cloud platforms. In our previous work, we provided a system that augments cloud APIs and provides roll-back operation using an AI planner. However, the previous system eventually suffers from the exponential complexity inherent to AI planning tasks. In this paper, we divide and parallelize rollback plan generation, based on characteristics unique to the rollback scenario. Through experimental evaluation, we show that this approach scales better than the previous, naive approach, and effectively avoids the exponential behavior.
{"title":"Scalable Rollback for Cloud Operations Using AI Planning","authors":"S. Satyal, I. Weber, L. Bass, Min Fu","doi":"10.1109/ASWEC.2015.34","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.34","url":null,"abstract":"Human-induced faults play a large role in systems reliability. In cloud platforms, system administrators may inadvertently make catastrophic mistakes, like deleting a virtual disk with important data. Providing rollback for cloud operations can reduce the severity and impact of such mistakes by allowing to revert back to a known, good state. In this paper, we present a scalable approach to rollback operations that change state of a system on proprietary cloud platforms. In our previous work, we provided a system that augments cloud APIs and provides roll-back operation using an AI planner. However, the previous system eventually suffers from the exponential complexity inherent to AI planning tasks. In this paper, we divide and parallelize rollback plan generation, based on characteristics unique to the rollback scenario. Through experimental evaluation, we show that this approach scales better than the previous, naive approach, and effectively avoids the exponential behavior.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130430319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Debugging is crucial for producing reliable software. One of the effective bug localization techniques is Spectral-Based Fault Localization (SBFL). It locates a buggy statement by applying an evaluation metric to program spectra and ranking program components on the basis of the score it computes. Recently, genetic programming has been proposed as a way to find good metrics. We have found that the huge search space for metrics can cause this approach to be slow and unreliable, even for relatively simple data sets. Here we propose a restricted class of "hyperbolic" metrics, with a small number of numeric parameters. This class of functions is based on past theoretical and empirical results. We show that genetic programming can reliably discover effective metrics over a wide range of data sets of program spectra. We evaluate the performance for both real programs and model programs with single bugs, multiple bugs, "deterministic" bugs and nondeterministic bugs.
{"title":"Multiple Bug Spectral Fault Localization Using Genetic Programming","authors":"L. Naish, Neelofar, K. Ramamohanarao","doi":"10.1109/ASWEC.2015.12","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.12","url":null,"abstract":"Debugging is crucial for producing reliable software. One of the effective bug localization techniques is Spectral-Based Fault Localization (SBFL). It locates a buggy statement by applying an evaluation metric to program spectra and ranking program components on the basis of the score it computes. Recently, genetic programming has been proposed as a way to find good metrics. We have found that the huge search space for metrics can cause this approach to be slow and unreliable, even for relatively simple data sets. Here we propose a restricted class of \"hyperbolic\" metrics, with a small number of numeric parameters. This class of functions is based on past theoretical and empirical results. We show that genetic programming can reliably discover effective metrics over a wide range of data sets of program spectra. We evaluate the performance for both real programs and model programs with single bugs, multiple bugs, \"deterministic\" bugs and nondeterministic bugs.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124198894","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Engineering performance-critical systems often requires manual, expensive fine-tuning of critical application parts such as start-up routines, authentication sequences and transactions. It is highly desirable to protect this investment by regression tests that indicate when performance characteristics such as memory usage or thread allocation change. While traditional testing techniques can be used, they are often too coarse, as systems are tested against static thresholds, and therefore important changes that can result in declining system performance will not be detected. To address these limitations, we propose a novel approach to performance regression testing based on automatically generated statistical test oracles. Machine learning methods are used to detect deviations from the profiles shown in these oracles. We present Buto, a proof-of-concept tool tightly integrated into the JUnit testing framework that can be used to test applications executed on the Java virtual machine (JVM). Buto uses data obtained by transparently monitoring applications through Java Management Extensions (JMX). In this paper we describe the Buto framework and demonstrate how to calibrate the tool using an evaluation based on a set of benchmarking examples.
{"title":"Performance Regression Testing on the Java Virtual Machine Using Statistical Test Oracles","authors":"Fergus Hewson, Jens Dietrich, S. Marsland","doi":"10.1109/ASWEC.2015.14","DOIUrl":"https://doi.org/10.1109/ASWEC.2015.14","url":null,"abstract":"Engineering performance-critical systems often requires manual, expensive fine-tuning of critical application parts such as start-up routines, authentication sequences and transactions. It is highly desirable to protect this investment by regression tests that indicate when performance characteristics such as memory usage or thread allocation change. While traditional testing techniques can be used, they are often too coarse, as systems are tested against static thresholds, and therefore important changes that can result in declining system performance will not be detected. To address these limitations, we propose a novel approach to performance regression testing based on automatically generated statistical test oracles. Machine learning methods are used to detect deviations from the profiles shown in these oracles. We present Buto, a proof-of-concept tool tightly integrated into the JUnit testing framework that can be used to test applications executed on the Java virtual machine (JVM). Buto uses data obtained by transparently monitoring applications through Java Management Extensions (JMX). In this paper we describe the Buto framework and demonstrate how to calibrate the tool using an evaluation based on a set of benchmarking examples.","PeriodicalId":310799,"journal":{"name":"2015 24th Australasian Software Engineering Conference","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123951667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：