Pub Date : 2023-05-01DOI: 10.1109/TechDebt59074.2023.00017
Daniel Skryseth, Karthik Shivashankar, I. Pilán, A. Martini
Background: Technical Debt (TD) needs to be controlled and tracked during software development. Support to automatically track TD in issue trackers is limited. Aim: We explore the usage of a large dataset of developer-labeled TD issues in combination with cutting-edge Natural Language Processing (NLP) approaches to automatically classify TD in issue trackers. Method: We mine and analyze more than 160GB of textual data from GitHub projects, collecting over 55,600 TD issues and consolidating them into a large dataset (GTD dataset). We use such datasets to train and test Transformer ML models. Then we test the model’s generalization ability by testing them on six unseen projects. Finally, we re-train the models including part of the TD issues from the target project to test their adaptability. Results and conclusion: (i) We create and release the GTD dataset, a comprehensive dataset including TD issues from 6,401 public repositories with various contexts; (ii) By training Transformers using the GTD dataset, we achieve performance metrics that are promising; (iii) Our results are a significant step forward towards supporting the automatic classification of TD in issue trackers, especially when the models are adapted to the context of unseen projects after fine-tuning.
{"title":"Technical Debt Classification in Issue Trackers using Natural Language Processing based on Transformers","authors":"Daniel Skryseth, Karthik Shivashankar, I. Pilán, A. Martini","doi":"10.1109/TechDebt59074.2023.00017","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00017","url":null,"abstract":"Background: Technical Debt (TD) needs to be controlled and tracked during software development. Support to automatically track TD in issue trackers is limited. Aim: We explore the usage of a large dataset of developer-labeled TD issues in combination with cutting-edge Natural Language Processing (NLP) approaches to automatically classify TD in issue trackers. Method: We mine and analyze more than 160GB of textual data from GitHub projects, collecting over 55,600 TD issues and consolidating them into a large dataset (GTD dataset). We use such datasets to train and test Transformer ML models. Then we test the model’s generalization ability by testing them on six unseen projects. Finally, we re-train the models including part of the TD issues from the target project to test their adaptability. Results and conclusion: (i) We create and release the GTD dataset, a comprehensive dataset including TD issues from 6,401 public repositories with various contexts; (ii) By training Transformers using the GTD dataset, we achieve performance metrics that are promising; (iii) Our results are a significant step forward towards supporting the automatic classification of TD in issue trackers, especially when the models are adapted to the context of unseen projects after fine-tuning.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127439552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/TechDebt59074.2023.00015
Lakmal Silva, M. Unterkalmsteiner, K. Wnuk
Background: Software documentation often struggles to catch up with the pace of software evolution. The lack of correct, complete, and up-to-date documentation results in an increasing number of documentation defects which could introduce delays in integrating software systems. In our previous study on a bug analysis tool called MultiDimEr, we provided evidence that documentation-related defects contribute to a significant number of bug reports. Aims: First, we want to identify documentation defect types contributing to documentation defects and thereby identifying documentation debt. Secondly, we aim to find pragmatic solutions to minimize most common documentation defects to pay off the documentation debt in the long run. Method: We investigated documentation defects related to an industrial software system. First, we looked at the types of different documentation and associated bug reports. We categorized the defects according to an existing documentation defect taxonomy. Results: Based on a sample of 101 defects, we found that a majority of defects are caused by documentation defects falling into the Information Content (What) category (86). Within this category, the documentation defect types Erroneous code examples (23), Missing documentation (35), and Outdated content (19) contributed to most of the documentation defects. We propose to adapt two solutions to mitigate these types of documentation defects. Conclusions: In practice, documentation debt can easily go undetected since a large share of resources and focus is dedicated to deliver high-quality software. This study provides evidence that documentation debt can contribute to increase in maintenance costs due to the number of documentation defects. We suggest to adapt two main solutions to tackle documentation debt by implementing (i) Dynamic Documentation Generation (DDG) and/or (ii) Automated Documentation Testing (ADT), which are both based on defining a single and robust information source for documentation.
{"title":"Towards identifying and minimizing customer-facing documentation debt","authors":"Lakmal Silva, M. Unterkalmsteiner, K. Wnuk","doi":"10.1109/TechDebt59074.2023.00015","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00015","url":null,"abstract":"Background: Software documentation often struggles to catch up with the pace of software evolution. The lack of correct, complete, and up-to-date documentation results in an increasing number of documentation defects which could introduce delays in integrating software systems. In our previous study on a bug analysis tool called MultiDimEr, we provided evidence that documentation-related defects contribute to a significant number of bug reports. Aims: First, we want to identify documentation defect types contributing to documentation defects and thereby identifying documentation debt. Secondly, we aim to find pragmatic solutions to minimize most common documentation defects to pay off the documentation debt in the long run. Method: We investigated documentation defects related to an industrial software system. First, we looked at the types of different documentation and associated bug reports. We categorized the defects according to an existing documentation defect taxonomy. Results: Based on a sample of 101 defects, we found that a majority of defects are caused by documentation defects falling into the Information Content (What) category (86). Within this category, the documentation defect types Erroneous code examples (23), Missing documentation (35), and Outdated content (19) contributed to most of the documentation defects. We propose to adapt two solutions to mitigate these types of documentation defects. Conclusions: In practice, documentation debt can easily go undetected since a large share of resources and focus is dedicated to deliver high-quality software. This study provides evidence that documentation debt can contribute to increase in maintenance costs due to the number of documentation defects. We suggest to adapt two main solutions to tackle documentation debt by implementing (i) Dynamic Documentation Generation (DDG) and/or (ii) Automated Documentation Testing (ADT), which are both based on defining a single and robust information source for documentation.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130410202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/TechDebt59074.2023.00013
Markus Finke, Thomas J. Neff, Tobias Reichl
This paper presents a process for management of technical debt (TD) and how it can be integrated sustainably into the existing software development process in an industrial context. A holistic approach is pursued where the development team and the management team agree on a common understanding of TD. From this, requirements for the process are derived, which should support the medium- and long-term planning of the roadmap. By iteratively evaluating the technical debt in the context of the feature roadmap, an internal development benefit (debt repayment) can be combined with an external benefit (new features), while optimizing development costs at the same time. The first results illustrate the positive effect of the continuous repayment of technical debt.
{"title":"How to introduce TD Management into a Software Development Process – A Practical Approach","authors":"Markus Finke, Thomas J. Neff, Tobias Reichl","doi":"10.1109/TechDebt59074.2023.00013","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00013","url":null,"abstract":"This paper presents a process for management of technical debt (TD) and how it can be integrated sustainably into the existing software development process in an industrial context. A holistic approach is pursued where the development team and the management team agree on a common understanding of TD. From this, requirements for the process are derived, which should support the medium- and long-term planning of the roadmap. By iteratively evaluating the technical debt in the context of the feature roadmap, an internal development benefit (debt repayment) can be combined with an external benefit (new features), while optimizing development costs at the same time. The first results illustrate the positive effect of the continuous repayment of technical debt.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"80 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126327813","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/TechDebt59074.2023.00016
Domenico Gigante, Fabiano Pecorelli, Vita Santa Barletta, Andrea Janes, Valentina Lenarduzzi, D. Taibi, M. T. Baldassarre
Software quality is crucial in software development: if not addressed in early phases of the software development life cycle, it may even lead to technical bankruptcy, i.e., a situation in which modifications cost more than redeveloping the application from scratch. In addition, code security must also be addressed to reduce software vulnerabilities and to comply with legal requirements. In this work, we aim to investigate the relationship between refactoring code quality and software security, with the purpose of understanding whether and to what extent improving software quality could have a positive impact on software security as well. Specifically, we investigate to what extent rule violations of a software quality tool such as SonarQube overlap with rule violations of a software vulnerability tool like Fortify Static Code Analyzer. We first compared the rules encoded in the quality models of both tools, to discover possible overlapping cases. Later, we compared the issues raised by both tools on a set of open source Java projects; we also investigated the cases in which a quality refactoring process impacts over software security (thus removing one or more vulnerabilities). We furthermore validated our results statistically. Our results show that resolving software quality issues might also resolve security issues but only in part: many security issues still persist in the source code; also, some quality aspects are more likely to be improved in respect to others. In addition, this empirical study uncovers rule co-occurrences between the two tools. This study confirms the need for using a security-oriented static analysis tool to enforce software security instead of relying only on a quality-oriented one. Results have highlighted important insights for practitioners.
{"title":"Resolving Security Issues via Quality-Oriented Refactoring: A User Study","authors":"Domenico Gigante, Fabiano Pecorelli, Vita Santa Barletta, Andrea Janes, Valentina Lenarduzzi, D. Taibi, M. T. Baldassarre","doi":"10.1109/TechDebt59074.2023.00016","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00016","url":null,"abstract":"Software quality is crucial in software development: if not addressed in early phases of the software development life cycle, it may even lead to technical bankruptcy, i.e., a situation in which modifications cost more than redeveloping the application from scratch. In addition, code security must also be addressed to reduce software vulnerabilities and to comply with legal requirements. In this work, we aim to investigate the relationship between refactoring code quality and software security, with the purpose of understanding whether and to what extent improving software quality could have a positive impact on software security as well. Specifically, we investigate to what extent rule violations of a software quality tool such as SonarQube overlap with rule violations of a software vulnerability tool like Fortify Static Code Analyzer. We first compared the rules encoded in the quality models of both tools, to discover possible overlapping cases. Later, we compared the issues raised by both tools on a set of open source Java projects; we also investigated the cases in which a quality refactoring process impacts over software security (thus removing one or more vulnerabilities). We furthermore validated our results statistically. Our results show that resolving software quality issues might also resolve security issues but only in part: many security issues still persist in the source code; also, some quality aspects are more likely to be improved in respect to others. In addition, this empirical study uncovers rule co-occurrences between the two tools. This study confirms the need for using a security-oriented static analysis tool to enforce software security instead of relying only on a quality-oriented one. Results have highlighted important insights for practitioners.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133089863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/TechDebt59074.2023.00014
Wuxia Jin, Yuyun Zhang, Jiaowei Shang, Yi Hou, Ming Fan, Ting Liu
During long-term software evolution, it is inevitable that an accumulation of changes leads to architectural erosion and debt. Diverse metric-based methods have been developed to identify architectural problems that violate design principles and degrade software maintainability. However, there still exists a gap between the implementation-level metrics and architecture-level metrics. Consequently, it hinders the comprehensibility, interpretability, and indicative(-bility) of the measurement results. To fill this gap, we propose the dbMIT to identify potential code changes that make architecture decay. Our dbMIT first integrates popular metrics such as the CK suite. Then dbMIT constructs a forest structure of metrics, serving as a knowledge base to relate the metrics across levels together. The forest aims to link the architecture-level metrics and implementation-level metrics. Via pre-defined rules using the forest structure, our dbMIT identifies code changes that potentially cause the architecture decay. The usage of forest structure of metrics makes it easy for developers to understand detection results, explain why the detected code changes are potential contributors to the decay, and indicate the code scope for resolution. We also contribute a web-based tool to integrate our dbMIT. Our experiments on the collected projects demonstrate the effectiveness of dbMIT against a history-based ground-truth.
{"title":"Identifying Code Changes for Architecture Decay via a Metric Forest Structure","authors":"Wuxia Jin, Yuyun Zhang, Jiaowei Shang, Yi Hou, Ming Fan, Ting Liu","doi":"10.1109/TechDebt59074.2023.00014","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00014","url":null,"abstract":"During long-term software evolution, it is inevitable that an accumulation of changes leads to architectural erosion and debt. Diverse metric-based methods have been developed to identify architectural problems that violate design principles and degrade software maintainability. However, there still exists a gap between the implementation-level metrics and architecture-level metrics. Consequently, it hinders the comprehensibility, interpretability, and indicative(-bility) of the measurement results. To fill this gap, we propose the dbMIT to identify potential code changes that make architecture decay. Our dbMIT first integrates popular metrics such as the CK suite. Then dbMIT constructs a forest structure of metrics, serving as a knowledge base to relate the metrics across levels together. The forest aims to link the architecture-level metrics and implementation-level metrics. Via pre-defined rules using the forest structure, our dbMIT identifies code changes that potentially cause the architecture decay. The usage of forest structure of metrics makes it easy for developers to understand detection results, explain why the detected code changes are potential contributors to the decay, and indicate the code scope for resolution. We also contribute a web-based tool to integrate our dbMIT. Our experiments on the collected projects demonstrate the effectiveness of dbMIT against a history-based ground-truth.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"136 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121110729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/TechDebt59074.2023.00012
Fandi Bi, Birgit Vogel-Heuser, Fengmin Du, Nils Hanich, Ennuri Cho
Underlying and undiscovered technical debt (TD) that burdens the system and makes future changes more costly or impossible poses significant risks to mechatronic systems. Multi-disciplinary collaboration and cooperation lead to interdisciplinary interfaces and new life cycle phases that cause greater ripple effects to the TD distribution. When quantifying TD contagiousness in interdisciplinary engineering, only a few metrics, methods, or tools prove applicable. In this work, we propose a method containing two key metrics to quantify TD contagiousness across product life cycles and disciplines. Furthermore, we suggest a matrix multiplication method to quantify the adverse impact on each discipline and the system. By applying the methods to the data of three comparable companies in the industrial automation domain, the results enable us to measure and prioritize the TD incidents’ contagiousness. This method provides a first step towards the systematic quantification of TD in the interdisciplinary environment and provides metrics to compare systems based on objective factors.
{"title":"Technical Debt Contagiousness Metrics for Measurement and Prioritization in Mechatronics","authors":"Fandi Bi, Birgit Vogel-Heuser, Fengmin Du, Nils Hanich, Ennuri Cho","doi":"10.1109/TechDebt59074.2023.00012","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00012","url":null,"abstract":"Underlying and undiscovered technical debt (TD) that burdens the system and makes future changes more costly or impossible poses significant risks to mechatronic systems. Multi-disciplinary collaboration and cooperation lead to interdisciplinary interfaces and new life cycle phases that cause greater ripple effects to the TD distribution. When quantifying TD contagiousness in interdisciplinary engineering, only a few metrics, methods, or tools prove applicable. In this work, we propose a method containing two key metrics to quantify TD contagiousness across product life cycles and disciplines. Furthermore, we suggest a matrix multiplication method to quantify the adverse impact on each discipline and the system. By applying the methods to the data of three comparable companies in the industrial automation domain, the results enable us to measure and prioritize the TD incidents’ contagiousness. This method provides a first step towards the systematic quantification of TD in the interdisciplinary environment and provides metrics to compare systems based on objective factors.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116841972","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/TechDebt59074.2023.00018
Nikolaos Nikolaidis, Apostolos Ampatzoglou, A. Chatzigeorgiou, N. Mittas, E. Konstantinidis, P. Bamidis
One of the most well-known laws of software evolution suggests that code quality deteriorates over time. Following this law, recent empirical studies have brought evidence that Technical Debt (TD) Principal tends to increase (in absolute value) as the system grows, since more technical debt issues are added than resolved over time. To shed light into how technical debt accumulation occurs in practice, in this paper we explore specific maintenance activities (i.e., feature addition, bug fixing, and refactoring) and explore the balance between the technical debt that they introduce or resolve. To achieve this goal, we rely on studying Pull Requests (PR), which are the most established way to contribute code to an open-source project. A Pull Request is usually comprised by more than one commits, corresponding to a specific development / maintenance activity. In our study, we categorized Pull Requests, based on their labels, to find the effect that the different maintenance activities have on the accumulation of technical debt across evolution. In particular, we have analysed more than 13.5K pull requests (mined from 10 OSS projects), by calculating the TD Principal (calculated through SonarQube) before and after the Pull Requests. The results of the study suggested that several labels are used for tagging Pull Requests, out of which the most prevalent ones are new features, bug fixing, and refactoring. The effect of these activities on TD Principal accumulation is statistically different, and: (a) the addition of features tends to increase TD Principal; (b) refactoring is having an almost consistent positive effect (reducing TD Principal); and (c) bug fixing activity has undecisive impact on TD Principal. These results are compared to existing studies, interpreted, and various useful implications for researchers and practitioners have been drawn.
{"title":"Exploring the Effect of Various Maintenance Activities on the Accumulation of TD Principal","authors":"Nikolaos Nikolaidis, Apostolos Ampatzoglou, A. Chatzigeorgiou, N. Mittas, E. Konstantinidis, P. Bamidis","doi":"10.1109/TechDebt59074.2023.00018","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00018","url":null,"abstract":"One of the most well-known laws of software evolution suggests that code quality deteriorates over time. Following this law, recent empirical studies have brought evidence that Technical Debt (TD) Principal tends to increase (in absolute value) as the system grows, since more technical debt issues are added than resolved over time. To shed light into how technical debt accumulation occurs in practice, in this paper we explore specific maintenance activities (i.e., feature addition, bug fixing, and refactoring) and explore the balance between the technical debt that they introduce or resolve. To achieve this goal, we rely on studying Pull Requests (PR), which are the most established way to contribute code to an open-source project. A Pull Request is usually comprised by more than one commits, corresponding to a specific development / maintenance activity. In our study, we categorized Pull Requests, based on their labels, to find the effect that the different maintenance activities have on the accumulation of technical debt across evolution. In particular, we have analysed more than 13.5K pull requests (mined from 10 OSS projects), by calculating the TD Principal (calculated through SonarQube) before and after the Pull Requests. The results of the study suggested that several labels are used for tagging Pull Requests, out of which the most prevalent ones are new features, bug fixing, and refactoring. The effect of these activities on TD Principal accumulation is statistically different, and: (a) the addition of features tends to increase TD Principal; (b) refactoring is having an almost consistent positive effect (reducing TD Principal); and (c) bug fixing activity has undecisive impact on TD Principal. These results are compared to existing studies, interpreted, and various useful implications for researchers and practitioners have been drawn.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133606371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-16DOI: 10.1109/TechDebt59074.2023.00009
M. Sheikhaei, Yuan Tian
Software and systems traceability is essential for downstream tasks such as data-driven software analysis and intelligent tool development. However, despite the increasing attention to mining and understanding technical debt in software systems, specific tools for supporting the track of technical debts are rarely available. In this work, we propose the first programming language-independent tracking tool for self-admitted technical debt (SATD) – a sub-optimal solution that is explicitly annotated by developers in software systems. Our approach takes a git repository as input and returns a list of SATDs with their evolution actions (created, deleted, updated) at the commit-level. Our approach also returns a line number indicating the latest starting position of the corresponding SATD in the system. Our SATD tracking approach first identifies an initial set of raw SATDs (which only have created and deleted actions) by detecting and tracking SATDs in commits’ hunks, leveraging a state-of-the-art language-independent SATD detection approach. Then it calculates a context-based matching score between pairs of deleted and created raw SATDs in the same commits to identify SATD update actions. The results of our preliminary study on Apache Tomcat and Apache Ant show that our tracking tool can achieve a F1 score of 92.8% and 96.7% respectively.
{"title":"Automated Self-Admitted Technical Debt Tracking at Commit-Level: A Language-independent Approach","authors":"M. Sheikhaei, Yuan Tian","doi":"10.1109/TechDebt59074.2023.00009","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00009","url":null,"abstract":"Software and systems traceability is essential for downstream tasks such as data-driven software analysis and intelligent tool development. However, despite the increasing attention to mining and understanding technical debt in software systems, specific tools for supporting the track of technical debts are rarely available. In this work, we propose the first programming language-independent tracking tool for self-admitted technical debt (SATD) – a sub-optimal solution that is explicitly annotated by developers in software systems. Our approach takes a git repository as input and returns a list of SATDs with their evolution actions (created, deleted, updated) at the commit-level. Our approach also returns a line number indicating the latest starting position of the corresponding SATD in the system. Our SATD tracking approach first identifies an initial set of raw SATDs (which only have created and deleted actions) by detecting and tracking SATDs in commits’ hunks, leveraging a state-of-the-art language-independent SATD detection approach. Then it calculates a context-based matching score between pairs of deleted and created raw SATDs in the same commits to identify SATD update actions. The results of our preliminary study on Apache Tomcat and Apache Ant show that our tracking tool can achieve a F1 score of 92.8% and 96.7% respectively.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130389236","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-03-16DOI: 10.1109/TechDebt59074.2023.00011
William Aiken, Paul K. Mvula, Paula Branco, Guy-Vincent Jourdan, M. Sabetzadeh, H. Viktor
Artificial Intelligence and Machine Learning have witnessed rapid, significant improvements in Natural Language Processing (NLP) tasks. Utilizing Deep Learning, researchers have taken advantage of repository comments in Software Engineering to produce accurate methods for detecting Self-Admitted Technical Debt (SATD) from 20 open-source Java projects’ code. In this work, we improve SATD detection with a novel approach that leverages the Bidirectional Encoder Representations from Transformers (BERT) architecture. For comparison, we re-evaluated previous deep learning methods and applied stratified 10-fold cross-validation to report reliable F1-scores. We examine our model in both cross-project and intra-project contexts. For each context, we use re-sampling and duplication as augmentation strategies to account for data imbalance. We find that our trained BERT model improves over the best performance of all previous methods in 19 of the 20 projects in cross-project scenarios. However, the data augmentation techniques were not sufficient to overcome the lack of data present in the intra-project scenarios, and existing methods still perform better. Future research will look into ways to diversify SATD datasets in order to maximize the latent power in large BERT models.
{"title":"Measuring Improvement of F1-Scores in Detection of Self-Admitted Technical Debt","authors":"William Aiken, Paul K. Mvula, Paula Branco, Guy-Vincent Jourdan, M. Sabetzadeh, H. Viktor","doi":"10.1109/TechDebt59074.2023.00011","DOIUrl":"https://doi.org/10.1109/TechDebt59074.2023.00011","url":null,"abstract":"Artificial Intelligence and Machine Learning have witnessed rapid, significant improvements in Natural Language Processing (NLP) tasks. Utilizing Deep Learning, researchers have taken advantage of repository comments in Software Engineering to produce accurate methods for detecting Self-Admitted Technical Debt (SATD) from 20 open-source Java projects’ code. In this work, we improve SATD detection with a novel approach that leverages the Bidirectional Encoder Representations from Transformers (BERT) architecture. For comparison, we re-evaluated previous deep learning methods and applied stratified 10-fold cross-validation to report reliable F1-scores. We examine our model in both cross-project and intra-project contexts. For each context, we use re-sampling and duplication as augmentation strategies to account for data imbalance. We find that our trained BERT model improves over the best performance of all previous methods in 19 of the 20 projects in cross-project scenarios. However, the data augmentation techniques were not sufficient to overcome the lack of data present in the intra-project scenarios, and existing methods still perform better. Future research will look into ways to diversify SATD datasets in order to maximize the latent power in large BERT models.","PeriodicalId":131882,"journal":{"name":"2023 ACM/IEEE International Conference on Technical Debt (TechDebt)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121861403","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：