Despite the exponential growth of deep learning software during the last decade, there is a lack of tools to test and debug issues in deep learning programs. Current static analysis tools do not address challenges specific to deep learning as observed by past research on bugs specific to this area. Existing deep learning bug detection tools focus on specific issues like shape mismatches. In this paper, we present a vision for an abstraction-based approach to detect deep learning bugs and the plan to evaluate our approach. The motivation behind the abstraction-based approach is to be able to build an intermediate version of the neural network that can be analyzed in development time to provide live feedback programmers are used to with other kind of bugs.
{"title":"NerdBug: automated bug detection in neural networks","authors":"Foad Jafarinejad, Krishna Narasimhan, M. Mezini","doi":"10.1145/3464968.3468409","DOIUrl":"https://doi.org/10.1145/3464968.3468409","url":null,"abstract":"Despite the exponential growth of deep learning software during the last decade, there is a lack of tools to test and debug issues in deep learning programs. Current static analysis tools do not address challenges specific to deep learning as observed by past research on bugs specific to this area. Existing deep learning bug detection tools focus on specific issues like shape mismatches. In this paper, we present a vision for an abstraction-based approach to detect deep learning bugs and the plan to evaluate our approach. The motivation behind the abstraction-based approach is to be able to build an intermediate version of the neural network that can be analyzed in development time to provide live feedback programmers are used to with other kind of bugs.","PeriodicalId":295937,"journal":{"name":"Proceedings of the 1st ACM International Workshop on AI and Software Testing/Analysis","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122649415","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Vescan, Camelia Chisalita-Cretu, C. Serban, L. Dioşan
Nowadays, software systems encounter repeated modifications in order to satisfy any requirement regarding a business change. To assure that these changes do not affect systems' proper functioning, those parts affected by the changes need to be retested, minimizing the negative impact of performed modifications on another part of the software. In this research, we investigate how different optimization techniques (with various criteria) could improve the effectiveness of the testing activity, in particular the effectiveness of test case prioritization. The most efficient test schedules are identified by using either a Greedy algorithm or a Genetic Algorithm, optimizing a quality function that incorporates single or multiple criteria. Both functional requirements (with the existing dependencies between them) and non-functional requirements (i.e. quality attributes for test cases) are integrated with the quality assessment of a test order. Therefore, the conducted experiments considered various criteria combinations (faults, costs, and number of test cases), being applied to both theoretical case studies and a real-world benchmark. The conclusion of the experiments shows that the Genetic Algorithm outperforms the Greedy on all considered criteria.
{"title":"On the use of evolutionary algorithms for test case prioritization in regression testing considering requirements dependencies","authors":"A. Vescan, Camelia Chisalita-Cretu, C. Serban, L. Dioşan","doi":"10.1145/3464968.3468407","DOIUrl":"https://doi.org/10.1145/3464968.3468407","url":null,"abstract":"Nowadays, software systems encounter repeated modifications in order to satisfy any requirement regarding a business change. To assure that these changes do not affect systems' proper functioning, those parts affected by the changes need to be retested, minimizing the negative impact of performed modifications on another part of the software. In this research, we investigate how different optimization techniques (with various criteria) could improve the effectiveness of the testing activity, in particular the effectiveness of test case prioritization. The most efficient test schedules are identified by using either a Greedy algorithm or a Genetic Algorithm, optimizing a quality function that incorporates single or multiple criteria. Both functional requirements (with the existing dependencies between them) and non-functional requirements (i.e. quality attributes for test cases) are integrated with the quality assessment of a test order. Therefore, the conducted experiments considered various criteria combinations (faults, costs, and number of test cases), being applied to both theoretical case studies and a real-world benchmark. The conclusion of the experiments shows that the Genetic Algorithm outperforms the Greedy on all considered criteria.","PeriodicalId":295937,"journal":{"name":"Proceedings of the 1st ACM International Workshop on AI and Software Testing/Analysis","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128026366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sebastian Sztwiertnia, Maximilian Grübel, Amine Chouchane, Daniel Sokolowski, Krishna Narasimhan, M. Mezini
Machine learning (ML) is on the rise to be ubiquitous in modern software. Still, its use is challenging for software developers. So far, research has focused on the ML libraries to find and mitigate these challenges. However, there is initial evidence that programming languages also add to the challenges, identifiable in different distributions of bugs in ML programs. To fill this research gap, we propose the first empirical study on the impact of programming languages on bugs in ML programs. We plan to analyze software from GitHub and related discussions in GitHub issues and Stack Overflow for bug distributions in ML programs, aiming to identify correlations with the chosen programming language, its features and the application domain. This study's results enable better-targeted use of available programming language technology in ML programs, preventing bugs, reducing errors and speeding up development.
{"title":"Impact of programming languages on machine learning bugs","authors":"Sebastian Sztwiertnia, Maximilian Grübel, Amine Chouchane, Daniel Sokolowski, Krishna Narasimhan, M. Mezini","doi":"10.1145/3464968.3468408","DOIUrl":"https://doi.org/10.1145/3464968.3468408","url":null,"abstract":"Machine learning (ML) is on the rise to be ubiquitous in modern software. Still, its use is challenging for software developers. So far, research has focused on the ML libraries to find and mitigate these challenges. However, there is initial evidence that programming languages also add to the challenges, identifiable in different distributions of bugs in ML programs. To fill this research gap, we propose the first empirical study on the impact of programming languages on bugs in ML programs. We plan to analyze software from GitHub and related discussions in GitHub issues and Stack Overflow for bug distributions in ML programs, aiming to identify correlations with the chosen programming language, its features and the application domain. This study's results enable better-targeted use of available programming language technology in ML programs, preventing bugs, reducing errors and speeding up development.","PeriodicalId":295937,"journal":{"name":"Proceedings of the 1st ACM International Workshop on AI and Software Testing/Analysis","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129971118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jupyter notebooks are now widely adopted by data analysts as they provide a convenient environment for presenting computational results in a literate-programming document that combines code snippets, rich text, and inline visualizations. Literate-programming documents are intended to be computational narratives that are supplemented with self-explanatory text, but, recent studies have shown that this is lacking in practice. Efforts in the software engineering community to increase code comprehension in literate programming are limited. To address this, as a first step, this paper presents a prototype Jupyter notebook annotator, HeaderGen, that automatically creates a narrative structure in notebooks by classifying and annotating code cells based on the machine learning workflow. HeaderGen generates a markdown cell header for each code cell by statically analyzing the notebook, and in addition, associates these cell headers with a clickable table of contents for easier navigation. Further, we discuss our vision and opportunities based on this prototype.
{"title":"Automated cell header generator for Jupyter notebooks","authors":"Ashwini Venkatesh, E. Bodden","doi":"10.1145/3464968.3468410","DOIUrl":"https://doi.org/10.1145/3464968.3468410","url":null,"abstract":"Jupyter notebooks are now widely adopted by data analysts as they provide a convenient environment for presenting computational results in a literate-programming document that combines code snippets, rich text, and inline visualizations. Literate-programming documents are intended to be computational narratives that are supplemented with self-explanatory text, but, recent studies have shown that this is lacking in practice. Efforts in the software engineering community to increase code comprehension in literate programming are limited. To address this, as a first step, this paper presents a prototype Jupyter notebook annotator, HeaderGen, that automatically creates a narrative structure in notebooks by classifying and annotating code cells based on the machine learning workflow. HeaderGen generates a markdown cell header for each code cell by statically analyzing the notebook, and in addition, associates these cell headers with a clickable table of contents for easier navigation. Further, we discuss our vision and opportunities based on this prototype.","PeriodicalId":295937,"journal":{"name":"Proceedings of the 1st ACM International Workshop on AI and Software Testing/Analysis","volume":"111 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121801078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：