An Empirical Study of Common Challenges in Developing Deep Learning Applications

Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael R. Lyu, Miryung Kim
{"title":"An Empirical Study of Common Challenges in Developing Deep Learning Applications","authors":"Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael R. Lyu, Miryung Kim","doi":"10.1109/ISSRE.2019.00020","DOIUrl":null,"url":null,"abstract":"Recent advances in deep learning promote the innovation of many intelligent systems and applications such as autonomous driving and image recognition. Despite enormous efforts and investments in this field, a fundamental question remains under-investigated—what challenges do developers commonly face when building deep learning applications? To seek an answer, this paper presents a large-scale empirical study of deep learning questions in a popular Q&A website, Stack Overflow. We manually inspect a sample of 715 questions and identify seven kinds of frequently asked questions. We further build a classification model to quantify the distribution of different kinds of deep learning questions in the entire set of 39,628 deep learning questions. We find that program crashes, model migration, and implementation questions are the top three most frequently asked questions. After carefully examining accepted answers of these questions, we summarize five main root causes that may deserve attention from the research community, including API misuse, incorrect hyperparameter selection, GPU computation, static graph computation, and limited debugging and profiling support. Our results highlight the need for new techniques such as cross-framework differential testing to improve software development productivity and software reliability in deep learning.","PeriodicalId":254749,"journal":{"name":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"111","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ISSRE.2019.00020","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 111

Abstract

Recent advances in deep learning promote the innovation of many intelligent systems and applications such as autonomous driving and image recognition. Despite enormous efforts and investments in this field, a fundamental question remains under-investigated—what challenges do developers commonly face when building deep learning applications? To seek an answer, this paper presents a large-scale empirical study of deep learning questions in a popular Q&A website, Stack Overflow. We manually inspect a sample of 715 questions and identify seven kinds of frequently asked questions. We further build a classification model to quantify the distribution of different kinds of deep learning questions in the entire set of 39,628 deep learning questions. We find that program crashes, model migration, and implementation questions are the top three most frequently asked questions. After carefully examining accepted answers of these questions, we summarize five main root causes that may deserve attention from the research community, including API misuse, incorrect hyperparameter selection, GPU computation, static graph computation, and limited debugging and profiling support. Our results highlight the need for new techniques such as cross-framework differential testing to improve software development productivity and software reliability in deep learning.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
深度学习应用开发中常见挑战的实证研究
深度学习的最新进展推动了许多智能系统和应用的创新,如自动驾驶和图像识别。尽管在这个领域付出了巨大的努力和投资,但一个基本的问题仍然没有得到充分的研究——开发人员在构建深度学习应用程序时通常面临哪些挑战?为了寻找答案,本文对一个流行的问答网站Stack Overflow上的深度学习问题进行了大规模的实证研究。我们手动检查了715个问题的样本,并确定了7种常见问题。我们进一步建立了一个分类模型来量化不同类型的深度学习问题在整个39,628个深度学习问题集中的分布。我们发现程序崩溃、模型迁移和实现问题是最常被问到的三个问题。在仔细检查了这些问题的公认答案后,我们总结了可能值得研究界关注的五个主要根本原因,包括API滥用,不正确的超参数选择,GPU计算,静态图形计算以及有限的调试和分析支持。我们的研究结果强调了对跨框架差分测试等新技术的需求,以提高深度学习中的软件开发效率和软件可靠性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Inferring Performance Bug Patterns from Developer Commits Learning Marked Markov Modulated Poisson Processes for Online Predictive Analysis of Attack Scenarios An Empirical Study of Common Challenges in Developing Deep Learning Applications Evaluation of Anomaly Detection Algorithms Made Easy with RELOAD ISSRE 2019 External Reviewers
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1