Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00014
Sagar Sen, Simon Myklebust Nielsen, E. J. Husom, Arda Goknil, Simeon Tverdal, Leonardo Sastoque Pinilla
The Industrial Internet of Things (IIoT) leverages thousands of interconnected sensors and computing devices to monitor and control large and complex industrial processes. Machine learning (ML) applications in IIoT use data acquired from multiple sensors to perform tasks such as predictive maintenance. While remembering useful learning from the past, these applications need to adapt learning for evolving sensor data stemming from changes in industrial processes and environmental conditions. This paper presents a continual learning pipeline to learn from the evolving data while replaying selected parts of the old data. The pipeline is configured to produce ML experiences (e.g., training a baseline neural network model), improve the baseline model with the new data while replaying part of the old data, and infer/predict using a specific model version given a stream of IIoT sensor data. We have evaluated our approach from an AI Engineering perspective using three industrial case studies, i.e., predicting tool wear, remaining useful lifetime, and anomalies from sensor data acquired from CNC machining and broaching operations. Our results show that configuring experiences for replay-driven continual learning allows dynamic maintenance of ML performance on evolving data while minimizing the excessive accumulation of legacy sensor data.
{"title":"Replay-Driven Continual Learning for the Industrial Internet of Things","authors":"Sagar Sen, Simon Myklebust Nielsen, E. J. Husom, Arda Goknil, Simeon Tverdal, Leonardo Sastoque Pinilla","doi":"10.1109/CAIN58948.2023.00014","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00014","url":null,"abstract":"The Industrial Internet of Things (IIoT) leverages thousands of interconnected sensors and computing devices to monitor and control large and complex industrial processes. Machine learning (ML) applications in IIoT use data acquired from multiple sensors to perform tasks such as predictive maintenance. While remembering useful learning from the past, these applications need to adapt learning for evolving sensor data stemming from changes in industrial processes and environmental conditions. This paper presents a continual learning pipeline to learn from the evolving data while replaying selected parts of the old data. The pipeline is configured to produce ML experiences (e.g., training a baseline neural network model), improve the baseline model with the new data while replaying part of the old data, and infer/predict using a specific model version given a stream of IIoT sensor data. We have evaluated our approach from an AI Engineering perspective using three industrial case studies, i.e., predicting tool wear, remaining useful lifetime, and anomalies from sensor data acquired from CNC machining and broaching operations. Our results show that configuring experiences for replay-driven continual learning allows dynamic maintenance of ML performance on evolving data while minimizing the excessive accumulation of legacy sensor data.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126628048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00031
Leon Chemnitz, David Reichenbach, Hani Aldebes, Mariam Naveed, Krishna Narasimhan, M. Mezini
Automatic code generation has recently attracted large attention and is becoming more significant to the software development process. Solutions based on Machine Learning and Artificial Intelligence are being used to increase human and software efficiency in potent and innovative ways. In this paper, we aim to leverage these developments and introduce a novel approach to generating frontend component code for the popular Angular framework. We propose to do this using behavior-driven development test specifications as input to a transformer-based machine learning model; however, we do not provide any proof-of-concept solution in this work. Our approach aims to drastically reduce the development time needed for web applications while potentially increasing software quality and introducing new research ideas toward automatic code generation.
{"title":"Towards Code Generation from BDD Test Case Specifications: A Vision","authors":"Leon Chemnitz, David Reichenbach, Hani Aldebes, Mariam Naveed, Krishna Narasimhan, M. Mezini","doi":"10.1109/CAIN58948.2023.00031","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00031","url":null,"abstract":"Automatic code generation has recently attracted large attention and is becoming more significant to the software development process. Solutions based on Machine Learning and Artificial Intelligence are being used to increase human and software efficiency in potent and innovative ways. In this paper, we aim to leverage these developments and introduce a novel approach to generating frontend component code for the popular Angular framework. We propose to do this using behavior-driven development test specifications as input to a transformer-based machine learning model; however, we do not provide any proof-of-concept solution in this work. Our approach aims to drastically reduce the development time needed for web applications while potentially increasing software quality and introducing new research ideas toward automatic code generation.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130416523","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00025
Iordanis Fostiropoulos, Bowman Brown, L. Itti
Machine learning is facing a ‘reproducibility crisis’ where a significant number of works report failures when attempting to reproduce previously published results. We evaluate the sources of reproducibility failures using a meta-analysis of 142 replication studies from ReScience C and 204 code repositories. We find that missing experiment details such as hyperparameters are potential causes of unreproducibility. We experimentally show the bias of different hyperparameter selection strategies and conclude that consolidated artifacts with a unified framework can help support reproducibility.
{"title":"Reproducibility Requires Consolidated Artifacts","authors":"Iordanis Fostiropoulos, Bowman Brown, L. Itti","doi":"10.1109/CAIN58948.2023.00025","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00025","url":null,"abstract":"Machine learning is facing a ‘reproducibility crisis’ where a significant number of works report failures when attempting to reproduce previously published results. We evaluate the sources of reproducibility failures using a meta-analysis of 142 replication studies from ReScience C and 204 code repositories. We find that missing experiment details such as hyperparameters are potential causes of unreproducibility. We experimentally show the bias of different hyperparameter selection strategies and conclude that consolidated artifacts with a unified framework can help support reproducibility.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"320 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123692519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00028
Arumoy Shome, Luís Cruz, A. Deursen
Visualisations drive all aspects of the Machine Learning (ML) Development Cycle but remain a vastly untapped resource by the research community. ML testing is a highly interactive and cognitive process which demands a human-in-the-loop approach. Besides writing tests for the code base, bulk of the evaluation requires application of domain expertise to generate and interpret visualisations. To gain a deeper insight into the process of testing ML systems, we propose to study visualisations of ML pipelines by mining Jupyter notebooks. We propose a two prong approach in conducting the analysis. First, gather general insights and trends using a qualitative study of a smaller sample of notebooks. And then use the knowledge gained from the qualitative study to design an empirical study using a larger sample of notebooks. Computational notebooks provide a rich source of information in three formats—text, code and images. We hope to utilise existing work in image analysis and Natural Language Processing for text and code, to analyse the information present in notebooks. We hope to gain a new perspective into program comprehension and debugging in the context of ML testing.
{"title":"Towards Understanding Machine Learning Testing in Practise","authors":"Arumoy Shome, Luís Cruz, A. Deursen","doi":"10.1109/CAIN58948.2023.00028","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00028","url":null,"abstract":"Visualisations drive all aspects of the Machine Learning (ML) Development Cycle but remain a vastly untapped resource by the research community. ML testing is a highly interactive and cognitive process which demands a human-in-the-loop approach. Besides writing tests for the code base, bulk of the evaluation requires application of domain expertise to generate and interpret visualisations. To gain a deeper insight into the process of testing ML systems, we propose to study visualisations of ML pipelines by mining Jupyter notebooks. We propose a two prong approach in conducting the analysis. First, gather general insights and trends using a qualitative study of a smaller sample of notebooks. And then use the knowledge gained from the qualitative study to design an empirical study using a larger sample of notebooks. Computational notebooks provide a rich source of information in three formats—text, code and images. We hope to utilise existing work in image analysis and Natural Language Processing for text and code, to analyse the information present in notebooks. We hope to gain a new perspective into program comprehension and debugging in the context of ML testing.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134599940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00026
Eduard Pinconschi, Sofia Reis, Chi Zhang, Rui Abreu, H. Erdogmus, C. Pasareanu, Limin Jia
Software vulnerability detection (SVD) aims to identify potential security weaknesses in software. SVD systems have been rapidly evolving from those being based on testing, static analysis, and dynamic analysis to those based on machine learning (ML). Many ML-based approaches have been proposed, but challenges remain: training and testing datasets contain duplicates, and building customized end-to-end pipelines for SVD is time-consuming. We present Tenet, a modular framework for building end-to-end, customizable, reusable, and automated pipelines through a plugin-based architecture that supports SVD for several deep learning (DL) and basic ML models. We demonstrate the applicability of Tenet by building practical pipelines performing SVD on real-world vulnerabilities.
{"title":"Tenet: A Flexible Framework for Machine-Learning-based Vulnerability Detection","authors":"Eduard Pinconschi, Sofia Reis, Chi Zhang, Rui Abreu, H. Erdogmus, C. Pasareanu, Limin Jia","doi":"10.1109/CAIN58948.2023.00026","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00026","url":null,"abstract":"Software vulnerability detection (SVD) aims to identify potential security weaknesses in software. SVD systems have been rapidly evolving from those being based on testing, static analysis, and dynamic analysis to those based on machine learning (ML). Many ML-based approaches have been proposed, but challenges remain: training and testing datasets contain duplicates, and building customized end-to-end pipelines for SVD is time-consuming. We present Tenet, a modular framework for building end-to-end, customizable, reusable, and automated pipelines through a plugin-based architecture that supports SVD for several deep learning (DL) and basic ML models. We demonstrate the applicability of Tenet by building practical pipelines performing SVD on real-world vulnerabilities.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"87 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124034519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00018
Valentina Lenarduzzi, M. Isomursu
The main goal of this project is to develop an AI Living Lab providing methods and software tools for AI trustworthiness analysis, running digital twins to simulate Digital Health solutions (Hardware and Software) integrated with AI elements in vitro for early-stage validation experiments. In this paper, we present the motivation beyond the need of a AI Living Lab methods for researchers and companies, our idea in practice, and the scheduled roadmap. The insights of the AI Living Lab can enable researchers to understand possible problems on the quality of AI-enabled systems opening new research topics and allows companies to understand how to better address quality issues in their systems.
{"title":"AI Living Lab: Quality Assurance for AI-based Health systems","authors":"Valentina Lenarduzzi, M. Isomursu","doi":"10.1109/CAIN58948.2023.00018","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00018","url":null,"abstract":"The main goal of this project is to develop an AI Living Lab providing methods and software tools for AI trustworthiness analysis, running digital twins to simulate Digital Health solutions (Hardware and Software) integrated with AI elements in vitro for early-stage validation experiments. In this paper, we present the motivation beyond the need of a AI Living Lab methods for researchers and companies, our idea in practice, and the scheduled roadmap. The insights of the AI Living Lab can enable researchers to understand possible problems on the quality of AI-enabled systems opening new research topics and allows companies to understand how to better address quality issues in their systems.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124591816","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00022
Jati H. Husen, H. Washizaki, H. Tun, Nobukazu Yoshioka, Y. Fukazawa, Hironori Takeuchi, Hiroshi Tanaka, Kazuki Munakata
Machine learning system analysis requires different approaches for each different task and domain. Selecting a proper set of analytic models can be challenging for a specific problem. This paper discusses the extensibility of the Multi-View Modeling Framework for ML Systems approach using process mapping and extensible metamodel. We conducted a case study to evaluate the feasibility of such extensibility by extending the approach to facilitate an activity-driven analysis for an optical character recognition system. Based on the result of the case study, we found that Multi-View Modeling Framework for ML Systems is likely to be extensible.
{"title":"Extensible Modeling Framework for Reliable Machine Learning System Analysis","authors":"Jati H. Husen, H. Washizaki, H. Tun, Nobukazu Yoshioka, Y. Fukazawa, Hironori Takeuchi, Hiroshi Tanaka, Kazuki Munakata","doi":"10.1109/CAIN58948.2023.00022","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00022","url":null,"abstract":"Machine learning system analysis requires different approaches for each different task and domain. Selecting a proper set of analytic models can be challenging for a specific problem. This paper discusses the extensibility of the Multi-View Modeling Framework for ML Systems approach using process mapping and extensible metamodel. We conducted a case study to evaluate the feasibility of such extensibility by extending the approach to facilitate an activity-driven analysis for an optical character recognition system. Based on the result of the case study, we found that Multi-View Modeling Framework for ML Systems is likely to be extensible.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115902975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00036
M. Zeller, M. Rothfelder, C. Klein
Traditional automation technologies alone are not sufficient to enable the fully automated operation of trains. However, Artificial Intelligence (AI) and Machine Learning (ML) offers great potential to realize the mandatory novel functions to replace the tasks of a human train driver, such as obstacle detection on the tracks. The problem, which still remains unresolved, is to find a practical way to link AI/ML techniques with the requirements and approval processes that are applied in the railway domain. The safe.trAIn project aims to lay the foundation for the safe use of AI/ML to achieve the driverless operation of rail vehicles and thus addresses this key technological challenge hindering the adoption of unmanned rail transport. The project goals are to develop guidelines and methods for the reliable engineering and safety assurance of ML in the railway domain. Therefore, the project investigates methods to reliable design ML models and to prove the trustworthiness of AI-based functions taking robustness, uncertainty, and transparency aspects of the AI/ML model into account.
{"title":"safe.trAIn – Engineering and Assurance of a Driverless Regional Train","authors":"M. Zeller, M. Rothfelder, C. Klein","doi":"10.1109/CAIN58948.2023.00036","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00036","url":null,"abstract":"Traditional automation technologies alone are not sufficient to enable the fully automated operation of trains. However, Artificial Intelligence (AI) and Machine Learning (ML) offers great potential to realize the mandatory novel functions to replace the tasks of a human train driver, such as obstacle detection on the tracks. The problem, which still remains unresolved, is to find a practical way to link AI/ML techniques with the requirements and approval processes that are applied in the railway domain. The safe.trAIn project aims to lay the foundation for the safe use of AI/ML to achieve the driverless operation of rail vehicles and thus addresses this key technological challenge hindering the adoption of unmanned rail transport. The project goals are to develop guidelines and methods for the reliable engineering and safety assurance of ML in the railway domain. Therefore, the project investigates methods to reliable design ML models and to prove the trustworthiness of AI-based functions taking robustness, uncertainty, and transparency aspects of the AI/ML model into account.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114928547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00030
András Schmelczer, Joost Visser
Trustworthy and robust deployment of AI applications requires adherence to a range of AI engineering best practices. But, while professionals already have access to frameworks for deploying AI, case studies and developer surveys have found that many deployments do not follow best practices.We hypothesize that the adoption of AI deployment best practices can be improved by finding less complex framework designs that combine ease of use with built-in support for best practices. To investigate this hypothesis, we applied a design science approach to develop a new framework, called GreatAI, and evaluate its ease of use and best practice support.The initial design focusses on the domain of natural language processing (NLP), but with generalisation in mind. To assess applicability and generalisability, we conducted interviews with ten practitioners. We also assessed best practice coverage.We found that our framework helps implement 33 best practices through an accessible interface. These target the transition from prototype to production phase in the AI development lifecycle. Feedback from professional data scientists and software engineers showed that ease of use and functionality are equally important in deciding to adopt deployment technologies, and the proposed framework was rated positively in both dimensions.
{"title":"Trustworthy and Robust AI Deployment by Design: A framework to inject best practice support into AI deployment pipelines","authors":"András Schmelczer, Joost Visser","doi":"10.1109/CAIN58948.2023.00030","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00030","url":null,"abstract":"Trustworthy and robust deployment of AI applications requires adherence to a range of AI engineering best practices. But, while professionals already have access to frameworks for deploying AI, case studies and developer surveys have found that many deployments do not follow best practices.We hypothesize that the adoption of AI deployment best practices can be improved by finding less complex framework designs that combine ease of use with built-in support for best practices. To investigate this hypothesis, we applied a design science approach to develop a new framework, called GreatAI, and evaluate its ease of use and best practice support.The initial design focusses on the domain of natural language processing (NLP), but with generalisation in mind. To assess applicability and generalisability, we conducted interviews with ten practitioners. We also assessed best practice coverage.We found that our framework helps implement 33 best practices through an accessible interface. These target the transition from prototype to production phase in the AI development lifecycle. Feedback from professional data scientists and software engineers showed that ease of use and functionality are equally important in deciding to adopt deployment technologies, and the proposed framework was rated positively in both dimensions.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131869154","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-05-01DOI: 10.1109/CAIN58948.2023.00009
L. Boué, Pratap Kunireddy, Pavle Subotic
Dependency hell is a well-known pain point in the development of large software projects and machine learning (ML) code bases are not immune from it. In fact, ML applications suffer from an additional form of dependency hell, namely, data source dependency hell. This term refers to the central role played by data and its unique quirks that often lead to unexpected failures of ML models which cannot be explained by code changes. In this paper, we present an automated data source dependency mapping framework that allows MLOps engineers to monitor the whole dependency map of their models in a fast paced engineering environment and thus mitigate ahead of time the consequences of any data source changes. Our system is based on a unified and generic approach, employing techniques from static analysis, from which data sources can be identified on a wide range of source artifacts. Our framework is currently deployed within Microsoft and used by Microsoft MLOps engineers in production.
{"title":"Automatically Resolving Data Source Dependency Hell in Large Scale Data Science Projects","authors":"L. Boué, Pratap Kunireddy, Pavle Subotic","doi":"10.1109/CAIN58948.2023.00009","DOIUrl":"https://doi.org/10.1109/CAIN58948.2023.00009","url":null,"abstract":"Dependency hell is a well-known pain point in the development of large software projects and machine learning (ML) code bases are not immune from it. In fact, ML applications suffer from an additional form of dependency hell, namely, data source dependency hell. This term refers to the central role played by data and its unique quirks that often lead to unexpected failures of ML models which cannot be explained by code changes. In this paper, we present an automated data source dependency mapping framework that allows MLOps engineers to monitor the whole dependency map of their models in a fast paced engineering environment and thus mitigate ahead of time the consequences of any data source changes. Our system is based on a unified and generic approach, employing techniques from static analysis, from which data sources can be identified on a wide range of source artifacts. Our framework is currently deployed within Microsoft and used by Microsoft MLOps engineers in production.","PeriodicalId":175580,"journal":{"name":"2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)","volume":"85 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133749862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：