S. Rafi, M. Akbar, Mary-Luz Sánchez-Gordón, Ricardo Colomo Palacios
Background: DevOps is currently one of the main trends in software development. Low-Code is also an emerging tendency that, combined with DevOps, may offer significant value to software businesses by improving the process. However, how DevOps practices and low-code are combined is little known. Aim: This study aims to understand the practitioner's perspectives on low-code trends. Method: Twelve interviews with IT professionals who deal with low-code in the context of DevOps were conducted. Then, a grounded theory approach was used to theme the interview quotes into emergent categories. Results: The main result of this exploratory study reveals that such an approach is the most common response to the skill shortages of software professionals. Conclusion: This study suggests the emergence of DevOps and low-code could significantly contribute to the development of quality products with low-cost and time.
{"title":"DevOps Practitioners’ Perceptions of the Low-code Trend","authors":"S. Rafi, M. Akbar, Mary-Luz Sánchez-Gordón, Ricardo Colomo Palacios","doi":"10.1145/3544902.3546635","DOIUrl":"https://doi.org/10.1145/3544902.3546635","url":null,"abstract":"Background: DevOps is currently one of the main trends in software development. Low-Code is also an emerging tendency that, combined with DevOps, may offer significant value to software businesses by improving the process. However, how DevOps practices and low-code are combined is little known. Aim: This study aims to understand the practitioner's perspectives on low-code trends. Method: Twelve interviews with IT professionals who deal with low-code in the context of DevOps were conducted. Then, a grounded theory approach was used to theme the interview quotes into emergent categories. Results: The main result of this exploratory study reveals that such an approach is the most common response to the skill shortages of software professionals. Conclusion: This study suggests the emergence of DevOps and low-code could significantly contribute to the development of quality products with low-cost and time.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123986837","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Soft skills of software professionals (e.g., communication, interpersonal skills) significantly contribute to project and product success. Aims: We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to types of software engineering positions, and (c) how soft skills relate to characteristics of hiring organizations. We focus on organizations in New Zealand, a country with a relatively small but growing software sector characterized by a skills shortage and embedded in a bi-cultural context. Method: We used a qualitative research method and manually analyzed 530 job adverts from New Zealand’s largest job portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without a pre-defined set of soft skills. Results: We found explicit references to soft skills in 82% of adverts. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related soft skills are most in demand, regardless of the type of position. Soft skills related to broader human or societal values (e.g., empathy or cultural awareness) or distributed development are not frequently requested. Soft skills do not depend on company size or core business. Conclusions: Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.
{"title":"What Soft Skills Does the Software Industry *Really* Want? An Exploratory Study of Software Positions in New Zealand","authors":"M. Galster, A. Mitrovic, S. Malinen, Jay Holland","doi":"10.1145/3544902.3546247","DOIUrl":"https://doi.org/10.1145/3544902.3546247","url":null,"abstract":"Background: Soft skills of software professionals (e.g., communication, interpersonal skills) significantly contribute to project and product success. Aims: We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to types of software engineering positions, and (c) how soft skills relate to characteristics of hiring organizations. We focus on organizations in New Zealand, a country with a relatively small but growing software sector characterized by a skills shortage and embedded in a bi-cultural context. Method: We used a qualitative research method and manually analyzed 530 job adverts from New Zealand’s largest job portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without a pre-defined set of soft skills. Results: We found explicit references to soft skills in 82% of adverts. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related soft skills are most in demand, regardless of the type of position. Soft skills related to broader human or societal values (e.g., empathy or cultural awareness) or distributed development are not frequently requested. Soft skills do not depend on company size or core business. Conclusions: Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127819088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Di Cui, Xingyu Li, Feiyang Liu, Siqi Wang, Jie Dai, Lu Wang, Qingshan Li
Background: Many safety-critical industrial applications have turned to deep learning systems as a fundamental component. Most of these systems rely on deep learning libraries, and bugs of such libraries can have irreparable consequences. Aims: Over the years, dependency structure has shown to be a practical indicator of software quality, widely used in numerous bug prediction techniques. The problem is that when analyzing bugs in deep learning libraries, researchers are unclear whether dependency structures still have a high correlation and which forms of dependency structures perform the best. Method: In this paper, we present a systematic investigation of the above question and implement a dependency structure-centric bug analysis tool: Depend4BL, capturing the interaction between dependency structures and bug locations in deep learning libraries. Results: We employ Depend4BL to analyze the top 5 open-source deep learning libraries on Github in terms of stars and forks, with 279,788 revision commits and 8,715 bug fixes. The results demonstrate the significant differences among syntactic, history, and semantic structures, and their vastly different impacts on bug locations. Their combinations have the potential to further improve bug prediction for deep learning libraries. Conclusions: In summary, our work provides a new perspective regarding to the correlation between dependency structures and bug locations in deep learning libraries. We release a large set of benchmarks and a prototype toolkit to automatically detect various forms of dependency structures for deep learning libraries. Our study also unveils useful findings based on quantitative and qualitative analysis that benefit bug prediction techniques for deep learning libraries.
{"title":"Towards Demystifying the Impact of Dependency Structures on Bug Locations in Deep Learning Libraries","authors":"Di Cui, Xingyu Li, Feiyang Liu, Siqi Wang, Jie Dai, Lu Wang, Qingshan Li","doi":"10.1145/3544902.3546246","DOIUrl":"https://doi.org/10.1145/3544902.3546246","url":null,"abstract":"Background: Many safety-critical industrial applications have turned to deep learning systems as a fundamental component. Most of these systems rely on deep learning libraries, and bugs of such libraries can have irreparable consequences. Aims: Over the years, dependency structure has shown to be a practical indicator of software quality, widely used in numerous bug prediction techniques. The problem is that when analyzing bugs in deep learning libraries, researchers are unclear whether dependency structures still have a high correlation and which forms of dependency structures perform the best. Method: In this paper, we present a systematic investigation of the above question and implement a dependency structure-centric bug analysis tool: Depend4BL, capturing the interaction between dependency structures and bug locations in deep learning libraries. Results: We employ Depend4BL to analyze the top 5 open-source deep learning libraries on Github in terms of stars and forks, with 279,788 revision commits and 8,715 bug fixes. The results demonstrate the significant differences among syntactic, history, and semantic structures, and their vastly different impacts on bug locations. Their combinations have the potential to further improve bug prediction for deep learning libraries. Conclusions: In summary, our work provides a new perspective regarding to the correlation between dependency structures and bug locations in deep learning libraries. We release a large set of benchmarks and a prototype toolkit to automatically detect various forms of dependency structures for deep learning libraries. Our study also unveils useful findings based on quantitative and qualitative analysis that benefit bug prediction techniques for deep learning libraries.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126398868","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Code summarization automatically generates the corresponding natural language descriptions according to the input code to characterize the function implemented by source code. Comprehensiveness of code representation is critical to code summarization task. However, most existing approaches typically use coarse-grained fusion methods to integrate multi-modal features. They generally represent different modalities of a piece of code, such as an Abstract Syntax Tree (AST) and a token sequence, as two embeddings and then fuse the two ones at the AST/code levels. Such a coarse integration makes it difficult to learn the correlations between fine-grained code elements across modalities effectively. Aims: This study intends to improve the model’s prediction performance for high-quality code summarization by accurately aligning and fully fusing semantic and syntactic structure information of source code at node/token levels. Method: This paper proposes a Multi-Modal Fine-grained Feature Fusion approach (MMF3) for neural code summarization. The method uses the Transformer architecture. In particular, we introduce a novel fine-grained fusion method, which allows fine-grained fusion of multiple code modalities at the token and node levels. Specifically, we use this method to fuse information from both token and AST modalities and apply the fused features to code summarization. Results: We conduct experiments on one Java and one Python datasets, and evaluate generated summaries using four metrics. The results show that: 1) the performance of our model outperforms the current state-of-the-art models, and 2) the ablation experiments show that our proposed fine-grained fusion method can effectively improve the accuracy of generated summaries. Conclusion: MMF3 can mine the relationships between cross-modal elements and perform accurate fine-grained element-level alignment fusion accordingly. As a result, more clues can be provided to improve the accuracy of the generated code summaries.
{"title":"MMF3: Neural Code Summarization Based on Multi-Modal Fine-Grained Feature Fusion","authors":"Zheng Ma, Yuexiu Gao, Lei Lyu, Chen Lyu","doi":"10.1145/3544902.3546251","DOIUrl":"https://doi.org/10.1145/3544902.3546251","url":null,"abstract":"Background: Code summarization automatically generates the corresponding natural language descriptions according to the input code to characterize the function implemented by source code. Comprehensiveness of code representation is critical to code summarization task. However, most existing approaches typically use coarse-grained fusion methods to integrate multi-modal features. They generally represent different modalities of a piece of code, such as an Abstract Syntax Tree (AST) and a token sequence, as two embeddings and then fuse the two ones at the AST/code levels. Such a coarse integration makes it difficult to learn the correlations between fine-grained code elements across modalities effectively. Aims: This study intends to improve the model’s prediction performance for high-quality code summarization by accurately aligning and fully fusing semantic and syntactic structure information of source code at node/token levels. Method: This paper proposes a Multi-Modal Fine-grained Feature Fusion approach (MMF3) for neural code summarization. The method uses the Transformer architecture. In particular, we introduce a novel fine-grained fusion method, which allows fine-grained fusion of multiple code modalities at the token and node levels. Specifically, we use this method to fuse information from both token and AST modalities and apply the fused features to code summarization. Results: We conduct experiments on one Java and one Python datasets, and evaluate generated summaries using four metrics. The results show that: 1) the performance of our model outperforms the current state-of-the-art models, and 2) the ablation experiments show that our proposed fine-grained fusion method can effectively improve the accuracy of generated summaries. Conclusion: MMF3 can mine the relationships between cross-modal elements and perform accurate fine-grained element-level alignment fusion accordingly. As a result, more clues can be provided to improve the accuracy of the generated code summaries.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132926873","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Q&A sites allow to study how users reference and request support on technical debt. To date only few studies, focusing on narrow aspects, investigate technical debt on Stack Overflow. Aims: We aim at gaining an in-depth understanding on the characteristics of technical debt questions on Stack Overflow. In addition, we assess if identification strategies based on machine learning can be used to automatically identify and classify technical debt questions. Method: We use automated and manual processes to identify technical debt questions on Stack Overflow. The final set of 415 questions is analyzed to study (i) technical debt types, (ii) question length, (iii) perceived urgency, (iv) sentiment, and (v) themes. Natural language processing and machine learning techniques are used to assess if questions can be identified and classified automatically. Results: Architecture debt is the most recurring debt type, followed by code and design debt. Most questions display mild urgency, with frequency of higher urgency steadily declining as urgency rises. Question length varies across debt types. Sentiment is mostly neutral. 29 recurrent themes emerge. Machine learning can be used to identify technical debt questions and binary urgency, but not debt types. Conclusions: Different patterns emerge from the analysis of technical debt questions on Stack Overflow. The results provide further insights on the phenomenon, and support the adoption of a more comprehensive strategy to identify technical debt questions.
{"title":"Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow","authors":"Nicholas Kozanidis, R. Verdecchia, Emitzá Guzmán","doi":"10.1145/3544902.3546245","DOIUrl":"https://doi.org/10.1145/3544902.3546245","url":null,"abstract":"Background: Q&A sites allow to study how users reference and request support on technical debt. To date only few studies, focusing on narrow aspects, investigate technical debt on Stack Overflow. Aims: We aim at gaining an in-depth understanding on the characteristics of technical debt questions on Stack Overflow. In addition, we assess if identification strategies based on machine learning can be used to automatically identify and classify technical debt questions. Method: We use automated and manual processes to identify technical debt questions on Stack Overflow. The final set of 415 questions is analyzed to study (i) technical debt types, (ii) question length, (iii) perceived urgency, (iv) sentiment, and (v) themes. Natural language processing and machine learning techniques are used to assess if questions can be identified and classified automatically. Results: Architecture debt is the most recurring debt type, followed by code and design debt. Most questions display mild urgency, with frequency of higher urgency steadily declining as urgency rises. Question length varies across debt types. Sentiment is mostly neutral. 29 recurrent themes emerge. Machine learning can be used to identify technical debt questions and binary urgency, but not debt types. Conclusions: Different patterns emerge from the analysis of technical debt questions on Stack Overflow. The results provide further insights on the phenomenon, and support the adoption of a more comprehensive strategy to identify technical debt questions.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126273235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: With the boosting development of IoT technology, the supply chains of IoT devices become more powerful and sophisticated, and the security issues introduced by code reuse are becoming more prominent. Therefore, the detection and management of vulnerabilities through code similarity detection technology is of great significance for protecting the security of IoT devices. Aim: We aim to propose a more accurate, parallel-friendly, and realistic software supply chain vulnerability detection solution for IoT devices. Method: This paper presents PG-VulNet, standing for Vulnerability-detection Network based on Pseudo-code Graphs. It is a ”multi-model” cross-architecture vulnerability detection solution based on pseudo-code and Graph Matching Network (GMN). PG-VulNet extracts both behavioral and structural features of pseudo-code to build customized feature graphs and then uses GMN to detect supply chain vulnerabilities based on these graphs. Results: The experiments show that PG-VulNet achieves an average detection accuracy of 99.14%, significantly higher than existing approaches like Gemini, VulSeeker, FIT, and Asteria. In addition to this, PG-VulNet also excels in detection overhead and false alarms. In the real-world evaluation, PG-VulNet detected 690 known vulnerabilities in 1,611 firmwares. Conclusions: PG-VulNet can effectively detect the vulnerabilities introduced by software supply chain in IoT firmwares and is well suited for large-scale detection. Compared with existing approaches, PG-VulNet has significant advantages.
{"title":"PG-VulNet: Detect Supply Chain Vulnerabilities in IoT Devices using Pseudo-code and Graphs","authors":"Xin Liu, Yixiong Wu, Qingchen Yu, Shangru Song, Yue Liu, Qingguo Zhou, Jianwei Zhuge","doi":"10.1145/3544902.3546240","DOIUrl":"https://doi.org/10.1145/3544902.3546240","url":null,"abstract":"Background: With the boosting development of IoT technology, the supply chains of IoT devices become more powerful and sophisticated, and the security issues introduced by code reuse are becoming more prominent. Therefore, the detection and management of vulnerabilities through code similarity detection technology is of great significance for protecting the security of IoT devices. Aim: We aim to propose a more accurate, parallel-friendly, and realistic software supply chain vulnerability detection solution for IoT devices. Method: This paper presents PG-VulNet, standing for Vulnerability-detection Network based on Pseudo-code Graphs. It is a ”multi-model” cross-architecture vulnerability detection solution based on pseudo-code and Graph Matching Network (GMN). PG-VulNet extracts both behavioral and structural features of pseudo-code to build customized feature graphs and then uses GMN to detect supply chain vulnerabilities based on these graphs. Results: The experiments show that PG-VulNet achieves an average detection accuracy of 99.14%, significantly higher than existing approaches like Gemini, VulSeeker, FIT, and Asteria. In addition to this, PG-VulNet also excels in detection overhead and false alarms. In the real-world evaluation, PG-VulNet detected 690 known vulnerabilities in 1,611 firmwares. Conclusions: PG-VulNet can effectively detect the vulnerabilities introduced by software supply chain in IoT firmwares and is well suited for large-scale detection. Compared with existing approaches, PG-VulNet has significant advantages.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"86 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116525652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Makayla Moster, Aarav Chandra, Chris W. L. Chu, Weiyi Liu, Paige Rodeghero
Background: Listening to music is a common practice among software developers. Music listening after work can help release work-related stress; while listening to music at work can improve work efficiency and make tedious work more enjoyable. The working environment of developers in the past few years has changed dramatically due to the vast adoption of remote and hybrid work policies. Aims: We aim to understand how listening to music at work affects remote developers’ perceived productivity and creativity. Method: We investigated 130 software developers and collected their music listening habits during remote work in a questionnaire. We studied the impact of listening to music on developers’ creativity and productivity while working remotely during the COVID-19 pandemic. Results: Our survey data suggests that developers generally feel more productive and creative when listening to music during remote working conditions. Conclusion: We found that developers who listen to music feel more productive and creative while working remotely due to reducing environment distractions.
{"title":"In the Zone: An Analysis of the Music Practices of Remote Software Developers","authors":"Makayla Moster, Aarav Chandra, Chris W. L. Chu, Weiyi Liu, Paige Rodeghero","doi":"10.1145/3544902.3546638","DOIUrl":"https://doi.org/10.1145/3544902.3546638","url":null,"abstract":"Background: Listening to music is a common practice among software developers. Music listening after work can help release work-related stress; while listening to music at work can improve work efficiency and make tedious work more enjoyable. The working environment of developers in the past few years has changed dramatically due to the vast adoption of remote and hybrid work policies. Aims: We aim to understand how listening to music at work affects remote developers’ perceived productivity and creativity. Method: We investigated 130 software developers and collected their music listening habits during remote work in a questionnaire. We studied the impact of listening to music on developers’ creativity and productivity while working remotely during the COVID-19 pandemic. Results: Our survey data suggests that developers generally feel more productive and creative when listening to music during remote working conditions. Conclusion: We found that developers who listen to music feel more productive and creative while working remotely due to reducing environment distractions.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129942911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Defect Prediction research aims at assisting software engineers in the early identification of software defect during the development process. A variety of automated approaches, ranging from traditional classification models to more sophisticated learning approaches, have been explored to this end. Among these, recent studies have proposed the use of ensemble prediction models (i.e., aggregation of multiple base classifiers) to build more robust defect prediction models. Aims: In this paper, we introduce a novel approach based on multi-objective evolutionary search to automatically generate defect prediction ensembles. Our proposal is not only novel with respect to the more general area of evolutionary generation of ensembles, but it also advances the state-of-the-art in the use of ensemble in defect prediction. Method: We assess the effectiveness of our approach, dubbed as Multi-objectiveEnsembleGeneration (MEG), by empirically benchmarking it with respect to the most related proposals we found in the literature on defect prediction ensembles and on multi-objective evolutionary ensembles (which, to the best of our knowledge, had never been previously applied to tackle defect prediction). Result: Our results show that MEG is able to generate ensembles which produce similar or more accurate predictions than those achieved by all the other approaches considered in 73% of the cases (with favourable large effect sizes in 80% of them). Conclusions: MEG is not only able to generate ensembles that yield more accurate defect predictions with respect to the benchmarks considered, but it also does it automatically, thus relieving the engineers from the burden of manual design and experimentation.
{"title":"MEG: Multi-objective Ensemble Generation for Software Defect Prediction","authors":"Rebecca Moussa, Giovani Guizzo, Federica Sarro","doi":"10.1145/3544902.3546255","DOIUrl":"https://doi.org/10.1145/3544902.3546255","url":null,"abstract":"Background: Defect Prediction research aims at assisting software engineers in the early identification of software defect during the development process. A variety of automated approaches, ranging from traditional classification models to more sophisticated learning approaches, have been explored to this end. Among these, recent studies have proposed the use of ensemble prediction models (i.e., aggregation of multiple base classifiers) to build more robust defect prediction models. Aims: In this paper, we introduce a novel approach based on multi-objective evolutionary search to automatically generate defect prediction ensembles. Our proposal is not only novel with respect to the more general area of evolutionary generation of ensembles, but it also advances the state-of-the-art in the use of ensemble in defect prediction. Method: We assess the effectiveness of our approach, dubbed as Multi-objectiveEnsembleGeneration (MEG), by empirically benchmarking it with respect to the most related proposals we found in the literature on defect prediction ensembles and on multi-objective evolutionary ensembles (which, to the best of our knowledge, had never been previously applied to tackle defect prediction). Result: Our results show that MEG is able to generate ensembles which produce similar or more accurate predictions than those achieved by all the other approaches considered in 73% of the cases (with favourable large effect sizes in 80% of them). Conclusions: MEG is not only able to generate ensembles that yield more accurate defect predictions with respect to the benchmarks considered, but it also does it automatically, thus relieving the engineers from the burden of manual design and experimentation.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126410706","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
F. Santos, Bianca Trinkenreich, J. F. Pimentel, I. Wiese, Igor Steinmacher, A. Sarma, M. Gerosa
[Background] Selecting an appropriate task is challenging for Open Source Software (OSS) project newcomers and a variety of strategies can help them in this process. [Aims] In this research, we compare the perspective of maintainers, newcomers, and existing contributors about the importance of strategies to support this process. Our goal is to identify possible gulfs of expectations between newcomers who are meant to be helped and contributors who have to put effort into these strategies, which can create friction and impede the usefulness of the strategies. [Method] We interviewed maintainers (n=17) and applied inductive qualitative analysis to derive a model of strategies meant to be adopted by newcomers and communities. Next, we sent a questionnaire (n=64) to maintainers, frequent contributors, and newcomers, asking them to rank these strategies based on their importance. We used the Schulze method to compare the different rankings from the different types of contributors. [Results] Maintainers and contributors diverged in their opinions about the relative importance of various strategies. The results suggest that newcomers want a better contribution process and more support to onboard, while maintainers expect to solve questions using the available communication channels. [Conclusions] The gaps in perspectives between newcomers and existing contributors create a gulf of expectation. OSS communities can leverage our results to prioritize the strategies considered the most important by newcomers.
{"title":"How to Choose a Task? Mismatches in Perspectives of Newcomers and Existing Contributors","authors":"F. Santos, Bianca Trinkenreich, J. F. Pimentel, I. Wiese, Igor Steinmacher, A. Sarma, M. Gerosa","doi":"10.1145/3544902.3546236","DOIUrl":"https://doi.org/10.1145/3544902.3546236","url":null,"abstract":"[Background] Selecting an appropriate task is challenging for Open Source Software (OSS) project newcomers and a variety of strategies can help them in this process. [Aims] In this research, we compare the perspective of maintainers, newcomers, and existing contributors about the importance of strategies to support this process. Our goal is to identify possible gulfs of expectations between newcomers who are meant to be helped and contributors who have to put effort into these strategies, which can create friction and impede the usefulness of the strategies. [Method] We interviewed maintainers (n=17) and applied inductive qualitative analysis to derive a model of strategies meant to be adopted by newcomers and communities. Next, we sent a questionnaire (n=64) to maintainers, frequent contributors, and newcomers, asking them to rank these strategies based on their importance. We used the Schulze method to compare the different rankings from the different types of contributors. [Results] Maintainers and contributors diverged in their opinions about the relative importance of various strategies. The results suggest that newcomers want a better contribution process and more support to onboard, while maintainers expect to solve questions using the available communication channels. [Conclusions] The gaps in perspectives between newcomers and existing contributors create a gulf of expectation. OSS communities can leverage our results to prioritize the strategies considered the most important by newcomers.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"234 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131667354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yingling Li, Xing Che, Yuekai Huang, Junjie Wang, Song Wang, Yawen Wang, Qing Wang
Background. Issues are prevalent, and identifying the correct priority of the reported issues is crucial to reduce the maintenance effort and ensure higher software quality. There are several approaches for the automatic priority prediction, yet they do not fully utilize the related information that might influence the priority assignment. Our observation reveals that there are noticeable correlations between an issue’s priority and its category, e.g., an issue of bug category tends to be assigned with higher priority than an issue of document category. This correlation motivates us to employ multi-task learning to share the knowledge about issue’s category prediction and facilitating priority prediction. Aims. This paper aims at providing an automatic approach for effective issue’s priority prediction, to reduce the burden of the project members and better manage the issues. Method. We propose issue priority prediction approach PRIMA with deep multi-task learning, which takes the issue category prediction as another task to facilitate the information sharing and learning. It consists of three main phases: 1) data preparation and augmentation phase, which allows data sharing beyond single task learning; 2) model construction phase, which designs shared layers to encode the semantics of textual descriptions, and task-specific layers to model two tasks in parallel; it also includes the indicative attributes to better capture an issue’s inherent meaning; 3) model training phase, which enables eavesdropping by shared loss function between two tasks. Results. Evaluations with four large-scale open-source projects show that PRIMA outperforms commonly-used and state-of-the-art baselines, with 32% -55% higher precision, and 28% - 56% higher recall. Compared with single task learning, the performance improvement reaches 18% in precision and 19% in recall. Results from our user study further prove its potential practical value. Conclusions. The proposed approach provides a novel and effective way for issue priority prediction, and sheds light on jointly exploring other issue-management tasks.
{"title":"A Tale of Two Tasks: Automated Issue Priority Prediction with Deep Multi-task Learning","authors":"Yingling Li, Xing Che, Yuekai Huang, Junjie Wang, Song Wang, Yawen Wang, Qing Wang","doi":"10.1145/3544902.3546257","DOIUrl":"https://doi.org/10.1145/3544902.3546257","url":null,"abstract":"Background. Issues are prevalent, and identifying the correct priority of the reported issues is crucial to reduce the maintenance effort and ensure higher software quality. There are several approaches for the automatic priority prediction, yet they do not fully utilize the related information that might influence the priority assignment. Our observation reveals that there are noticeable correlations between an issue’s priority and its category, e.g., an issue of bug category tends to be assigned with higher priority than an issue of document category. This correlation motivates us to employ multi-task learning to share the knowledge about issue’s category prediction and facilitating priority prediction. Aims. This paper aims at providing an automatic approach for effective issue’s priority prediction, to reduce the burden of the project members and better manage the issues. Method. We propose issue priority prediction approach PRIMA with deep multi-task learning, which takes the issue category prediction as another task to facilitate the information sharing and learning. It consists of three main phases: 1) data preparation and augmentation phase, which allows data sharing beyond single task learning; 2) model construction phase, which designs shared layers to encode the semantics of textual descriptions, and task-specific layers to model two tasks in parallel; it also includes the indicative attributes to better capture an issue’s inherent meaning; 3) model training phase, which enables eavesdropping by shared loss function between two tasks. Results. Evaluations with four large-scale open-source projects show that PRIMA outperforms commonly-used and state-of-the-art baselines, with 32% -55% higher precision, and 28% - 56% higher recall. Compared with single task learning, the performance improvement reaches 18% in precision and 19% in recall. Results from our user study further prove its potential practical value. Conclusions. The proposed approach provides a novel and effective way for issue priority prediction, and sheds light on jointly exploring other issue-management tasks.","PeriodicalId":220679,"journal":{"name":"Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132146654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：