Daniel Honsel, V. Honsel, Marlon Welter, S. Waack, J. Grabowski
The evolution of software projects is driven by developers who are in control of the developed artifacts and the quality of software projects depends on the work of participating developers. Thus, a simulation tool requires a suitable model of the commit behavior of different developer types. In this paper, we present an agent-based model for software processes containing the commit behavior for different developer types. The description of these types results from mining software repositories. Since relationships between software entities, e.g., files, classes, modules, axe represented as dependency graphs, simulation results can be assessed automatically by Conditional Random Fields (CRFs). By adjusting simulation parameters for one project we are able to give a quality trend of other projects similar in size and duration only by changing the effort and the size of other projects to simulate.
{"title":"Monitoring Software Quality by Means of Simulation Methods","authors":"Daniel Honsel, V. Honsel, Marlon Welter, S. Waack, J. Grabowski","doi":"10.1145/2961111.2962617","DOIUrl":"https://doi.org/10.1145/2961111.2962617","url":null,"abstract":"The evolution of software projects is driven by developers who are in control of the developed artifacts and the quality of software projects depends on the work of participating developers. Thus, a simulation tool requires a suitable model of the commit behavior of different developer types. In this paper, we present an agent-based model for software processes containing the commit behavior for different developer types. The description of these types results from mining software repositories. Since relationships between software entities, e.g., files, classes, modules, axe represented as dependency graphs, simulation results can be assessed automatically by Conditional Random Fields (CRFs). By adjusting simulation parameters for one project we are able to give a quality trend of other projects similar in size and duration only by changing the effort and the size of other projects to simulate.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127717266","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Static analysis and penetration testing are common techniques used to discover security bugs in implementation code. Penetration testing is often performed in black-box way by probing the attack surface of a running system and discovering its security holes. Static analysis techniques operate in a white-box way by analyzing the source code of a system and identifying security weaknesses. Because of their different nature, the two techniques report their findings in two different ways. This paper presents an exploratory study meant to determine whether a vulnerability report generated by a security tool based on static analysis is more or less useful than a report generated by a security tool based on penetration testing. The usefulness is judged from the perspective of the developers that have to devise a vulnerability-fixing patch. The initial results show an advantage when using penetration testing in one of the two cases we investigated.
{"title":"Static Analysis and Penetration Testing from the Perspective of Maintenance Teams","authors":"M. Ceccato, R. Scandariato","doi":"10.1145/2961111.2962611","DOIUrl":"https://doi.org/10.1145/2961111.2962611","url":null,"abstract":"Static analysis and penetration testing are common techniques used to discover security bugs in implementation code. Penetration testing is often performed in black-box way by probing the attack surface of a running system and discovering its security holes. Static analysis techniques operate in a white-box way by analyzing the source code of a system and identifying security weaknesses. Because of their different nature, the two techniques report their findings in two different ways. This paper presents an exploratory study meant to determine whether a vulnerability report generated by a security tool based on static analysis is more or less useful than a report generated by a security tool based on penetration testing. The usefulness is judged from the perspective of the developers that have to devise a vulnerability-fixing patch. The initial results show an advantage when using penetration testing in one of the two cases we investigated.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126437401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yu Zhao, Feng Zhang, Emad Shihab, Ying Zou, A. Hassan
Background: Bug fixing is one major activity in software maintenance to solve unexpected errors or crashes of software systems. However, a bug fix can also be incomplete and even introduce new bugs. In such cases, extra effort is needed to rework the bug fix. The reworking requires to inspect the problem again, and perform the code change and verification when necessary. Discussions throughout the bug fixing process are important to clarify the reported problem and reach a solution. Aims: In this paper, we explore how discussions during the initial bug fix period (i.e., before the bug reworking occurs) associate with future bug reworking. We focus on two types of "reworked bug fixes": 1) the initial bug fix made in a re-opened bug report; and 2) the initially submitted patch if multiple patches are submitted for a single bug report. Method: We perform a case study using five open source projects (i.e., Linux, Firefox, PDE, Ant and HTTP). The discussions are studied from six perspectives (i.e., duration, number of comments, dispersion, frequency, number of developers and experience of developers). Furthermore, we extract topics of discussions using Latent Dirichlet Allocation (LDA). Results: We find that the occurrence of bug reworking is associated with various perspectives of discussions. Moreover, discussions on some topics (e.g., code inspection and code testing) can decrease the frequency of bug reworking. Conclusions: The discussions during the initial bug fix period may serve as an early indicator of what bug fixes are more likely to be reworked.
{"title":"How Are Discussions Associated with Bug Reworking?: An Empirical Study on Open Source Projects","authors":"Yu Zhao, Feng Zhang, Emad Shihab, Ying Zou, A. Hassan","doi":"10.1145/2961111.2962591","DOIUrl":"https://doi.org/10.1145/2961111.2962591","url":null,"abstract":"Background: Bug fixing is one major activity in software maintenance to solve unexpected errors or crashes of software systems. However, a bug fix can also be incomplete and even introduce new bugs. In such cases, extra effort is needed to rework the bug fix. The reworking requires to inspect the problem again, and perform the code change and verification when necessary. Discussions throughout the bug fixing process are important to clarify the reported problem and reach a solution. Aims: In this paper, we explore how discussions during the initial bug fix period (i.e., before the bug reworking occurs) associate with future bug reworking. We focus on two types of \"reworked bug fixes\": 1) the initial bug fix made in a re-opened bug report; and 2) the initially submitted patch if multiple patches are submitted for a single bug report. Method: We perform a case study using five open source projects (i.e., Linux, Firefox, PDE, Ant and HTTP). The discussions are studied from six perspectives (i.e., duration, number of comments, dispersion, frequency, number of developers and experience of developers). Furthermore, we extract topics of discussions using Latent Dirichlet Allocation (LDA). Results: We find that the occurrence of bug reworking is associated with various perspectives of discussions. Moreover, discussions on some topics (e.g., code inspection and code testing) can decrease the frequency of bug reworking. Conclusions: The discussions during the initial bug fix period may serve as an early indicator of what bug fixes are more likely to be reworked.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130676034","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Context: The Mission Design and Navigation Software Group at the Jet Propulsion Laboratory (JPL) maintains mission critical software systems. We have good empirical data and models for maintenance demand---when defects will occur, how many and how severe they will be, and how much effort is needed to address them. However, determining the level of staffing needed to address maintenance issues is an ongoing challenge and is often done ad-hoc. There are two common strategies are (1) reactive - add/remove staff as needed to respond to maintenance issues, and (2) capacitive - retain a given staff size to address issues as they occur, proactively address issues and prevent defects. Goal: To use our empirical models for maintenance demand to address the issue of staffing from a risk perspective. Method: We develop a staffing model that allows us to simulate large numbers of maintenance histories. From these histories we examine the risks posed to the institution as a function of the staffing available to address issues as they arise. Results: We find that the model developed matches our intuition. There is a "sweet spot" in staffing levels that allows issues to be addressed in a timely fashion. Below that level the institution experiences substantial risk; staffing above that level does little to improve the institutions risk exposure. Conclusion: The models developed provide tools that, for the first time, allow us to quantitatively discuss the level of staffing needed to ensure that we can meet the time constrained demands for maintenance on mission critical systems and thereby determine staffing budgets that ensure mission success.
{"title":"Staffing Strategies for Maintenance of Critical Software Systems at the Jet Propulsion Laboratory","authors":"W. Taber, D. Port","doi":"10.1145/2961111.2962640","DOIUrl":"https://doi.org/10.1145/2961111.2962640","url":null,"abstract":"Context: The Mission Design and Navigation Software Group at the Jet Propulsion Laboratory (JPL) maintains mission critical software systems. We have good empirical data and models for maintenance demand---when defects will occur, how many and how severe they will be, and how much effort is needed to address them. However, determining the level of staffing needed to address maintenance issues is an ongoing challenge and is often done ad-hoc. There are two common strategies are (1) reactive - add/remove staff as needed to respond to maintenance issues, and (2) capacitive - retain a given staff size to address issues as they occur, proactively address issues and prevent defects. Goal: To use our empirical models for maintenance demand to address the issue of staffing from a risk perspective. Method: We develop a staffing model that allows us to simulate large numbers of maintenance histories. From these histories we examine the risks posed to the institution as a function of the staffing available to address issues as they arise. Results: We find that the model developed matches our intuition. There is a \"sweet spot\" in staffing levels that allows issues to be addressed in a timely fashion. Below that level the institution experiences substantial risk; staffing above that level does little to improve the institutions risk exposure. Conclusion: The models developed provide tools that, for the first time, allow us to quantitatively discuss the level of staffing needed to ensure that we can meet the time constrained demands for maintenance on mission critical systems and thereby determine staffing budgets that ensure mission success.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131633187","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wenhua Hu, Jeffrey C. Carver, Vaibhav Anu, G. Walia, Gary L. Bradshaw
Background: Developing correct software requirements is important for overall software quality. Most existing quality improvement approaches focus on detection and removal of faults (i.e. problems recorded in a document) as opposed identifying the underlying errors that produced those faults. Accordingly, developers are likely to make the same errors in the future and fail to recognize other existing faults with the same origins. Therefore, we have created a Human Error Taxonomy (HET) to help software engineers improve their software requirement specification (SRS) documents. Aims: The goal of this paper is to analyze whether the HET is useful for classifying errors and for guiding developers to find additional faults. Methods: We conducted a empirical study in a classroom setting to evaluate the usefulness and feasibility of the HET. Results: First, software developers were able to employ error categories in the HET to identify and classify the underlying sources of faults identified during the inspection of SRS documents. Second, developers were able to use that information to detect additional faults that had gone unnoticed during the initial inspection. Finally, the participants had a positive impression about the usefulness of the HET. Conclusions: The HET is effective for identifying and classifying requirements errors and faults, thereby helping to improve the overall quality of the SRS and the software.
{"title":"Detection of Requirement Errors and Faults via a Human Error Taxonomy: A Feasibility Study","authors":"Wenhua Hu, Jeffrey C. Carver, Vaibhav Anu, G. Walia, Gary L. Bradshaw","doi":"10.1145/2961111.2962596","DOIUrl":"https://doi.org/10.1145/2961111.2962596","url":null,"abstract":"Background: Developing correct software requirements is important for overall software quality. Most existing quality improvement approaches focus on detection and removal of faults (i.e. problems recorded in a document) as opposed identifying the underlying errors that produced those faults. Accordingly, developers are likely to make the same errors in the future and fail to recognize other existing faults with the same origins. Therefore, we have created a Human Error Taxonomy (HET) to help software engineers improve their software requirement specification (SRS) documents. Aims: The goal of this paper is to analyze whether the HET is useful for classifying errors and for guiding developers to find additional faults. Methods: We conducted a empirical study in a classroom setting to evaluate the usefulness and feasibility of the HET. Results: First, software developers were able to employ error categories in the HET to identify and classify the underlying sources of faults identified during the inspection of SRS documents. Second, developers were able to use that information to detect additional faults that had gone unnoticed during the initial inspection. Finally, the participants had a positive impression about the usefulness of the HET. Conclusions: The HET is effective for identifying and classifying requirements errors and faults, thereby helping to improve the overall quality of the SRS and the software.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130431478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Context: Software Engineering (SE) research with a scientific foundation aims to influence SE practice to enable and sustain efficient delivery of high quality software. Goal: To improve the impact of SE research, one objective is to facilitate practitioners in choosing empirically vetted interventions. Method: Literature from evidence-based medicine, economic evaluations in SE and software economics is reviewed. Results: In empirical SE research, the emphasis has been on substantiating the claims about the benefits of proposed interventions. However, to support informed decision making by practitioners regarding technology adoption, we must present a business case for these interventions, which should comprise not just effectiveness, but also the evidence of cost-effectiveness. Conclusions: This paper highlights the need to investigate and report the resources required to adopt an intervention. It also provides some guidelines and examples to improve support for practitioners in decisions regarding technology adoption.
{"title":"Is effectiveness sufficient to choose an intervention?: Considering resource use in empirical software engineering","authors":"N. Ali","doi":"10.1145/2961111.2962631","DOIUrl":"https://doi.org/10.1145/2961111.2962631","url":null,"abstract":"Context: Software Engineering (SE) research with a scientific foundation aims to influence SE practice to enable and sustain efficient delivery of high quality software. Goal: To improve the impact of SE research, one objective is to facilitate practitioners in choosing empirically vetted interventions. Method: Literature from evidence-based medicine, economic evaluations in SE and software economics is reviewed. Results: In empirical SE research, the emphasis has been on substantiating the claims about the benefits of proposed interventions. However, to support informed decision making by practitioners regarding technology adoption, we must present a business case for these interventions, which should comprise not just effectiveness, but also the evidence of cost-effectiveness. Conclusions: This paper highlights the need to investigate and report the resources required to adopt an intervention. It also provides some guidelines and examples to improve support for practitioners in decisions regarding technology adoption.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132022725","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fabio Q. B. da Silva, C. França, Cleyton V. C. de Magalhães, Ronnie E. S. Santos
Context: Work Design refers to the different ways in which a given work or task can be designed and performed. The study of work design is important because every decision related to how the work is performed can affect the outcomes of individuals and the effectiveness of teamwork. Goal: To investigate work design characteristics of software engineering work and identify areas for future research. Method: We performed a survey with professional software engineers working in different commercial software companies in Brazil. We sent invitations to just over 150 professionals and received 77 valid answers from participants working in 35 distinct software companies. We measured 21 work design constructs, as well as job burnout, role conflict, role ambiguity, and two constructs related to job rotation. Results: Comparisons of our sample with other sample consisting of a diversity of professions showed that software engineering work has distinct characteristics, but some results require further investigation. Conclusion: We identified relevant characteristics of software engineering work and areas for further research. In particular, longitudinal studies are needed to address the temporal variations impossible to identify in cross sectional studies.
{"title":"Preliminary Findings about the Nature of Work in Software Engineering: An Exploratory Survey","authors":"Fabio Q. B. da Silva, C. França, Cleyton V. C. de Magalhães, Ronnie E. S. Santos","doi":"10.1145/2961111.2962625","DOIUrl":"https://doi.org/10.1145/2961111.2962625","url":null,"abstract":"Context: Work Design refers to the different ways in which a given work or task can be designed and performed. The study of work design is important because every decision related to how the work is performed can affect the outcomes of individuals and the effectiveness of teamwork. Goal: To investigate work design characteristics of software engineering work and identify areas for future research. Method: We performed a survey with professional software engineers working in different commercial software companies in Brazil. We sent invitations to just over 150 professionals and received 77 valid answers from participants working in 35 distinct software companies. We measured 21 work design constructs, as well as job burnout, role conflict, role ambiguity, and two constructs related to job rotation. Results: Comparisons of our sample with other sample consisting of a diversity of professions showed that software engineering work has distinct characteristics, but some results require further investigation. Conclusion: We identified relevant characteristics of software engineering work and areas for further research. In particular, longitudinal studies are needed to address the temporal variations impossible to identify in cross sectional studies.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130793332","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Context: The success of crowdsourced software development (CSD) depends on a large crowd of trustworthy software workers who are registering and submitting for their interested tasks in exchange of financial gains. Preliminary analysis on software worker behaviors reveals an alarming task-quitting rate of 82.9%. Goal: The objective of this study is to empirically investigate worker decision factors and provide better decision support in order to improve the success and efficiency of CSD. Method: We propose a novel problem formulation, DCW-DS, and an analytics-based decision support methodology to guide workers in acceptance of offered development tasks. DCS-DS is evaluated using more than one year's real-world data from TopCoder, the leading CSD platform. Results: Applying Random Forest based machine learning with dynamic updates, we can predict a worker as being a likely quitter with 99% average precision and 99% average recall accuracy. Similarly, we achieved 78% average precision and 88% average recall for the worker winner class. For workers just following the top three task recommendations, we have shown that the average quitting rate goes down below 6%. Conclusions: In total, the proposed method can be used to improve total success rate as well as reduce quitting rate of tasks performed.
{"title":"Who Should Take This Task?: Dynamic Decision Support for Crowd Workers","authors":"Ye Yang, M. R. Karim, R. Saremi, G. Ruhe","doi":"10.1145/2961111.2962594","DOIUrl":"https://doi.org/10.1145/2961111.2962594","url":null,"abstract":"Context: The success of crowdsourced software development (CSD) depends on a large crowd of trustworthy software workers who are registering and submitting for their interested tasks in exchange of financial gains. Preliminary analysis on software worker behaviors reveals an alarming task-quitting rate of 82.9%. Goal: The objective of this study is to empirically investigate worker decision factors and provide better decision support in order to improve the success and efficiency of CSD. Method: We propose a novel problem formulation, DCW-DS, and an analytics-based decision support methodology to guide workers in acceptance of offered development tasks. DCS-DS is evaluated using more than one year's real-world data from TopCoder, the leading CSD platform. Results: Applying Random Forest based machine learning with dynamic updates, we can predict a worker as being a likely quitter with 99% average precision and 99% average recall accuracy. Similarly, we achieved 78% average precision and 88% average recall for the worker winner class. For workers just following the top three task recommendations, we have shown that the average quitting rate goes down below 6%. Conclusions: In total, the proposed method can be used to improve total success rate as well as reduce quitting rate of tasks performed.","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131182932","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Genero, Andreas Jedlitschka, M. Jørgensen, G. Scanniello, S. Sampath, D. Caivano, D. Port
{"title":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","authors":"M. Genero, Andreas Jedlitschka, M. Jørgensen, G. Scanniello, S. Sampath, D. Caivano, D. Port","doi":"10.1145/2961111","DOIUrl":"https://doi.org/10.1145/2961111","url":null,"abstract":"","PeriodicalId":208212,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"100 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123028035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：