Software projects can grow very rapidly, reaching hundreds or thousands of files in a relatively short time span. Therefore, manually finding the source code parts that should be changed in order to fix a failure is a difficult task. Static bug localization techniques provide cost-effective means of finding files related to the failure described in a bug report. Recently, structured information retrieval (IR) has been used to improve the effectiveness of static bug localization, being successfully applied by techniques such as BLUiR, BLUiR+, and AmaLgam. However, there are some significant shortcomings on how these techniques were evaluated. First, virtually all evaluations have been limited to very few projects written in only one object-oriented programming language, particularly Java. Therefore, the effectiveness of these techniques in other widely-used object-oriented languages such as C# is still unknown. Second, the experimental setup for most of the evaluations make simplistic assumptions that do not hold on real-world scenarios, thereby raising doubts about the reported effectiveness of these techniques. In this paper, we evaluate BLUiR, BLUiR+, and AmaLgam on 20 C# projects, providing a first assessment of these techniques on a previously untested object-oriented language. Moreover, we set up an experiment that addresses the simplistic assumptions commonly present in bug localization studies, thereby providing evidence on how their findings may be biased. Finally, we extend the algorithms of existing techniques in order to understand if structured information retrieval can benefit from the use of a wider range of program constructs, including C# constructs inexistent in Java.
{"title":"On the Evaluation of Structured Information Retrieval-Based Bug Localization on 20 C# Projects","authors":"Marcelo Garnier, Alessandro F. Garcia","doi":"10.1145/2973839.2973853","DOIUrl":"https://doi.org/10.1145/2973839.2973853","url":null,"abstract":"Software projects can grow very rapidly, reaching hundreds or thousands of files in a relatively short time span. Therefore, manually finding the source code parts that should be changed in order to fix a failure is a difficult task. Static bug localization techniques provide cost-effective means of finding files related to the failure described in a bug report. Recently, structured information retrieval (IR) has been used to improve the effectiveness of static bug localization, being successfully applied by techniques such as BLUiR, BLUiR+, and AmaLgam. However, there are some significant shortcomings on how these techniques were evaluated. First, virtually all evaluations have been limited to very few projects written in only one object-oriented programming language, particularly Java. Therefore, the effectiveness of these techniques in other widely-used object-oriented languages such as C# is still unknown. Second, the experimental setup for most of the evaluations make simplistic assumptions that do not hold on real-world scenarios, thereby raising doubts about the reported effectiveness of these techniques. In this paper, we evaluate BLUiR, BLUiR+, and AmaLgam on 20 C# projects, providing a first assessment of these techniques on a previously untested object-oriented language. Moreover, we set up an experiment that addresses the simplistic assumptions commonly present in bug localization studies, thereby providing evidence on how their findings may be biased. Finally, we extend the algorithms of existing techniques in order to understand if structured information retrieval can benefit from the use of a wider range of program constructs, including C# constructs inexistent in Java.","PeriodicalId":415612,"journal":{"name":"Proceedings of the XXX Brazilian Symposium on Software Engineering","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132537425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E. Nascimento, W. Silva, T. Conte, Igor Steinmacher, Jobson L. Massollar, G. Travassos
Use cases specifications are artifacts employed in all stages of software development, from the requirements elicitation to implementation. During this process, issues related to ambiguity, redundancy, inconsistency, and incompleteness can affect these specifications. These issues can harm software engineers' understanding and, consequently, affect the software quality. Given this context, this paper describes an empirical study to evaluate two different use cases specifications approaches (textual and graphical-based forms). We compared the approaches by assessing the degree of correctness and the time spent to generate the specifications. In addition, we performed an analysis focusing on evaluating the ease of use and usefulness of each approach. The quantitative results showed that textual form and graphical-based specifications presented similar levels of correctness and the time spent to generate them were also similar. The qualitative results indicated that the subjects had difficulties using both approaches; however, subjects stated that graphic-based specifications were easier and more useful to specify use cases.
{"title":"Is a Picture worth a Thousand Words?: A Comparative Analysis of Using Textual and Graphical Approaches to Specify Use Cases","authors":"E. Nascimento, W. Silva, T. Conte, Igor Steinmacher, Jobson L. Massollar, G. Travassos","doi":"10.1145/2973839.2973855","DOIUrl":"https://doi.org/10.1145/2973839.2973855","url":null,"abstract":"Use cases specifications are artifacts employed in all stages of software development, from the requirements elicitation to implementation. During this process, issues related to ambiguity, redundancy, inconsistency, and incompleteness can affect these specifications. These issues can harm software engineers' understanding and, consequently, affect the software quality. Given this context, this paper describes an empirical study to evaluate two different use cases specifications approaches (textual and graphical-based forms). We compared the approaches by assessing the degree of correctness and the time spent to generate the specifications. In addition, we performed an analysis focusing on evaluating the ease of use and usefulness of each approach. The quantitative results showed that textual form and graphical-based specifications presented similar levels of correctness and the time spent to generate them were also similar. The qualitative results indicated that the subjects had difficulties using both approaches; however, subjects stated that graphic-based specifications were easier and more useful to specify use cases.","PeriodicalId":415612,"journal":{"name":"Proceedings of the XXX Brazilian Symposium on Software Engineering","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129826805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Access control mechanisms demand rigorous software testing approaches, otherwise they can end up with security flaws. Finite state machines (FSM) have been used for testing Role-Based Access Control (RBAC) mechanisms and complete, but significantly large, test suites can be obtained. Experimental studies have shown that recent FSM testing methods can reduce the overall test suite length for random FSMs. However, since the similarity between random FSMs and these specifying RBAC mechanisms is unclear, these outcomes cannot be necessarily generalized to RBAC. In this paper, we compare the characteristics and effectiveness of test suites generated by traditional and recent FSM testing methods for RBAC policies specified as FSM models. The methods W, HSI and SPY were applied on RBAC policies specified as FSMs and the test suites obtained were evaluated considering test characteristics (number of resets, average test case length, and test suite length) and effectiveness on the RBAC fault domain. Our results corroborate some outcomes of previous investigations in which test suites presented different characteristics. On average, the SPY method generated test suites with 32% less resets, average test case length 78% greater than W and HSI, and overall length 46% lower. There were no differences among FSM testing methods for RBAC regarding effectiveness. However, the SPY method significantly reduced the overall test suite length and the number of resets.
{"title":"Evaluating test characteristics and effectiveness of FSM-based testing methods on RBAC systems","authors":"C. Damasceno, P. Masiero, A. Simão","doi":"10.1145/2973839.2973849","DOIUrl":"https://doi.org/10.1145/2973839.2973849","url":null,"abstract":"Access control mechanisms demand rigorous software testing approaches, otherwise they can end up with security flaws. Finite state machines (FSM) have been used for testing Role-Based Access Control (RBAC) mechanisms and complete, but significantly large, test suites can be obtained. Experimental studies have shown that recent FSM testing methods can reduce the overall test suite length for random FSMs. However, since the similarity between random FSMs and these specifying RBAC mechanisms is unclear, these outcomes cannot be necessarily generalized to RBAC. In this paper, we compare the characteristics and effectiveness of test suites generated by traditional and recent FSM testing methods for RBAC policies specified as FSM models. The methods W, HSI and SPY were applied on RBAC policies specified as FSMs and the test suites obtained were evaluated considering test characteristics (number of resets, average test case length, and test suite length) and effectiveness on the RBAC fault domain. Our results corroborate some outcomes of previous investigations in which test suites presented different characteristics. On average, the SPY method generated test suites with 32% less resets, average test case length 78% greater than W and HSI, and overall length 46% lower. There were no differences among FSM testing methods for RBAC regarding effectiveness. However, the SPY method significantly reduced the overall test suite length and the number of resets.","PeriodicalId":415612,"journal":{"name":"Proceedings of the XXX Brazilian Symposium on Software Engineering","volume":"XCIX 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131385969","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
B. B. N. França, Helvio Jeronimo Junior, G. Travassos
Recently, DevOps has emerged as an alternative for software organizations inserted into a dynamic market to handle daily software demands. As claimed, it intends to make the software development and operations teams to work collaboratively. However, it is hard to observe a shared understanding of DevOps, what potentially hinders the discussions in the literature and can confound observations when conducting empirical studies. Therefore, we performed a Multivocal Literature Review aiming at characterizing DevOps in multiple perspectives, including data sources from technical and gray literature. Grounded Theory procedures were used to rigorous analyze the collected data. It allowed us to achieve a grounded definition for DevOps, as well as to identify its recurrent principles, practices, required skills, potential benefits, challenges and what motivates the organizations to adopt it. Finally, we understand the DevOps movement has identified relevant issues in the state-of-the-practice. However, we advocate for the scientific investigations concerning the potential benefits and drawbacks as a consequence of adopting the suggested principles and practices.
{"title":"Characterizing DevOps by Hearing Multiple Voices","authors":"B. B. N. França, Helvio Jeronimo Junior, G. Travassos","doi":"10.1145/2973839.2973845","DOIUrl":"https://doi.org/10.1145/2973839.2973845","url":null,"abstract":"Recently, DevOps has emerged as an alternative for software organizations inserted into a dynamic market to handle daily software demands. As claimed, it intends to make the software development and operations teams to work collaboratively. However, it is hard to observe a shared understanding of DevOps, what potentially hinders the discussions in the literature and can confound observations when conducting empirical studies. Therefore, we performed a Multivocal Literature Review aiming at characterizing DevOps in multiple perspectives, including data sources from technical and gray literature. Grounded Theory procedures were used to rigorous analyze the collected data. It allowed us to achieve a grounded definition for DevOps, as well as to identify its recurrent principles, practices, required skills, potential benefits, challenges and what motivates the organizations to adopt it. Finally, we understand the DevOps movement has identified relevant issues in the state-of-the-practice. However, we advocate for the scientific investigations concerning the potential benefits and drawbacks as a consequence of adopting the suggested principles and practices.","PeriodicalId":415612,"journal":{"name":"Proceedings of the XXX Brazilian Symposium on Software Engineering","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117068143","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Software developers commonly rely on well-known software architecture patterns, such as MVC, to build their applications. In many of these patterns, classes play specific roles in the system, such as Controllers or Entities, which means that each of these classes has specific characteristics in terms of object-oriented class design and implementation. Indeed, as we have shown in a previous study, architectural roles are different from each other in terms of code metrics. In this paper, we present a study in a software development company in which we captured developers' perceptions on object-oriented design aspects of the architectural roles in their system and whether these perceptions match the source code metric analysis. We found that their developers do not have a common perception of how their architectural roles behave in terms of object-oriented design aspects, and that their perceptions also do not match the results of the source code metric analysis. This phenomenon also does not seem to be related to developers' experience. We find these results alarming, and thus, we suggest software development teams to invest in education and knowledge sharing about how their system's architectural roles behave.
{"title":"Developers' Perceptions on Object-Oriented Design and Architectural Roles","authors":"M. Aniche, M. Gerosa, Christoph Treude","doi":"10.1145/2973839.2973846","DOIUrl":"https://doi.org/10.1145/2973839.2973846","url":null,"abstract":"Software developers commonly rely on well-known software architecture patterns, such as MVC, to build their applications. In many of these patterns, classes play specific roles in the system, such as Controllers or Entities, which means that each of these classes has specific characteristics in terms of object-oriented class design and implementation. Indeed, as we have shown in a previous study, architectural roles are different from each other in terms of code metrics. In this paper, we present a study in a software development company in which we captured developers' perceptions on object-oriented design aspects of the architectural roles in their system and whether these perceptions match the source code metric analysis. We found that their developers do not have a common perception of how their architectural roles behave in terms of object-oriented design aspects, and that their perceptions also do not match the results of the source code metric analysis. This phenomenon also does not seem to be related to developers' experience. We find these results alarming, and thus, we suggest software development teams to invest in education and knowledge sharing about how their system's architectural roles behave.","PeriodicalId":415612,"journal":{"name":"Proceedings of the XXX Brazilian Symposium on Software Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132190280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
H. Rocha, G. D. Oliveira, M. T. Valente, H. T. Marques-Neto
Bug handling represents a major effort in most software projects. To improve this relevant task, software organizations must first understand the current status of their bug resolution process. Although there are plentiful research in bug reports, few of them address the bug handling workflow to better understand and reason about the maintenance process. To this purpose, we report a characterization study focused on the typical workflow followed by Mozilla Firefox developers when resolving bugs. We propose the concept of Bug Flow Graphs (BFG) to help understand the characterization. We analyze 13,564 bugs reported for Firefox in 2015 and we discovered some interesting characteristics of Firefox's bug workflow: (a) when a bug is not formally assigned to a developer it requires ten more days to be resolved; (b) approximately 94% of duplicate bugs are closed within two days or less after they appear in the tracking system (which reveals the efficiency of Firefox's duplicate bug detection procedures); (c) incomplete bugs, which are never assigned to developers, usually require 70 days to be closed; (d) more skilled developers show a faster resolution time than less skilled ones; (e) for less skilled developers a bug usually spends more time waiting to be assigned than being fixed.
{"title":"Characterizing Bug Workflows in Mozilla Firefox","authors":"H. Rocha, G. D. Oliveira, M. T. Valente, H. T. Marques-Neto","doi":"10.1145/2973839.2973844","DOIUrl":"https://doi.org/10.1145/2973839.2973844","url":null,"abstract":"Bug handling represents a major effort in most software projects. To improve this relevant task, software organizations must first understand the current status of their bug resolution process. Although there are plentiful research in bug reports, few of them address the bug handling workflow to better understand and reason about the maintenance process. To this purpose, we report a characterization study focused on the typical workflow followed by Mozilla Firefox developers when resolving bugs. We propose the concept of Bug Flow Graphs (BFG) to help understand the characterization. We analyze 13,564 bugs reported for Firefox in 2015 and we discovered some interesting characteristics of Firefox's bug workflow: (a) when a bug is not formally assigned to a developer it requires ten more days to be resolved; (b) approximately 94% of duplicate bugs are closed within two days or less after they appear in the tracking system (which reveals the efficiency of Firefox's duplicate bug detection procedures); (c) incomplete bugs, which are never assigned to developers, usually require 70 days to be closed; (d) more skilled developers show a faster resolution time than less skilled ones; (e) for less skilled developers a bug usually spends more time waiting to be assigned than being fixed.","PeriodicalId":415612,"journal":{"name":"Proceedings of the XXX Brazilian Symposium on Software Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129810769","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：