Structured pruning of neural networks for constraints learning

IF 0.8 4区 管理学 Q4 OPERATIONS RESEARCH & MANAGEMENT SCIENCE Operations Research Letters Pub Date : 2024-10-15 DOI:10.1016/j.orl.2024.107194
{"title":"Structured pruning of neural networks for constraints learning","authors":"","doi":"10.1016/j.orl.2024.107194","DOIUrl":null,"url":null,"abstract":"<div><div>In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity in applications such as cancer treatment, algorithmic configuration, and chemical process optimization. This integration often uses Mixed Integer Programming (MIP) formulations to represent the chosen ML model, that is often an Artificial Neural Networks (ANNs) due to their widespread use. However, ANNs frequently contain a large number of parameters, resulting in MIP formulations impractical to solve. In this paper we showcase the effectiveness of a ANN pruning, when applied to models prior to their integration into MIPs. We discuss why pruning is more suitable in this context than other ML compression techniques, and we highlight the potential of appropriate pruning strategies via experiments on MIPs used to construct adversarial examples to ANNs. Our results demonstrate that pruning offers remarkable reductions in solution times without hindering the quality of the final decision, enabling the resolution of previously unsolvable instances.</div></div>","PeriodicalId":54682,"journal":{"name":"Operations Research Letters","volume":null,"pages":null},"PeriodicalIF":0.8000,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Operations Research Letters","FirstCategoryId":"91","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167637724001305","RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"OPERATIONS RESEARCH & MANAGEMENT SCIENCE","Score":null,"Total":0}
引用次数: 0

Abstract

In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity in applications such as cancer treatment, algorithmic configuration, and chemical process optimization. This integration often uses Mixed Integer Programming (MIP) formulations to represent the chosen ML model, that is often an Artificial Neural Networks (ANNs) due to their widespread use. However, ANNs frequently contain a large number of parameters, resulting in MIP formulations impractical to solve. In this paper we showcase the effectiveness of a ANN pruning, when applied to models prior to their integration into MIPs. We discuss why pruning is more suitable in this context than other ML compression techniques, and we highlight the potential of appropriate pruning strategies via experiments on MIPs used to construct adversarial examples to ANNs. Our results demonstrate that pruning offers remarkable reductions in solution times without hindering the quality of the final decision, enabling the resolution of previously unsolvable instances.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
对神经网络进行结构化修剪以促进约束学习
近年来,机器学习(ML)模型与运筹学(OR)工具的整合在癌症治疗、算法配置和化学过程优化等应用中越来越受欢迎。这种整合通常使用混合整数编程(MIP)公式来表示所选的 ML 模型,由于人工神经网络(ANN)的广泛使用,这种模型通常是人工神经网络。然而,ANN 经常包含大量参数,导致 MIP 公式难以解决。在本文中,我们展示了在将模型集成到 MIPs 之前对其进行剪枝处理的有效性。我们讨论了为什么剪枝在这种情况下比其他 ML 压缩技术更合适,并通过对用于为 ANN 构建对抗示例的 MIP 进行实验,强调了适当剪枝策略的潜力。我们的结果表明,剪枝能显著缩短求解时间,同时不影响最终决策的质量,从而解决以前无法求解的实例。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Operations Research Letters
Operations Research Letters 管理科学-运筹学与管理科学
CiteScore
2.10
自引率
9.10%
发文量
111
审稿时长
83 days
期刊介绍: Operations Research Letters is committed to the rapid review and fast publication of short articles on all aspects of operations research and analytics. Apart from a limitation to eight journal pages, quality, originality, relevance and clarity are the only criteria for selecting the papers to be published. ORL covers the broad field of optimization, stochastic models and game theory. Specific areas of interest include networks, routing, location, queueing, scheduling, inventory, reliability, and financial engineering. We wish to explore interfaces with other fields such as life sciences and health care, artificial intelligence and machine learning, energy distribution, and computational social sciences and humanities. Our traditional strength is in methodology, including theory, modelling, algorithms and computational studies. We also welcome novel applications and concise literature reviews.
期刊最新文献
Break maximization for round-robin tournaments without consecutive breaks Anchored rescheduling problem with non-availability periods The circle packing problem: A theoretical comparison of various convexification techniques Structured pruning of neural networks for constraints learning Worst-case analysis of restarted primal-dual hybrid gradient on totally unimodular linear programs
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1