{"title":"Structured pruning of neural networks for constraints learning","authors":"","doi":"10.1016/j.orl.2024.107194","DOIUrl":null,"url":null,"abstract":"<div><div>In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity in applications such as cancer treatment, algorithmic configuration, and chemical process optimization. This integration often uses Mixed Integer Programming (MIP) formulations to represent the chosen ML model, that is often an Artificial Neural Networks (ANNs) due to their widespread use. However, ANNs frequently contain a large number of parameters, resulting in MIP formulations impractical to solve. In this paper we showcase the effectiveness of a ANN pruning, when applied to models prior to their integration into MIPs. We discuss why pruning is more suitable in this context than other ML compression techniques, and we highlight the potential of appropriate pruning strategies via experiments on MIPs used to construct adversarial examples to ANNs. Our results demonstrate that pruning offers remarkable reductions in solution times without hindering the quality of the final decision, enabling the resolution of previously unsolvable instances.</div></div>","PeriodicalId":54682,"journal":{"name":"Operations Research Letters","volume":null,"pages":null},"PeriodicalIF":0.8000,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Operations Research Letters","FirstCategoryId":"91","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167637724001305","RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"OPERATIONS RESEARCH & MANAGEMENT SCIENCE","Score":null,"Total":0}
引用次数: 0
Abstract
In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity in applications such as cancer treatment, algorithmic configuration, and chemical process optimization. This integration often uses Mixed Integer Programming (MIP) formulations to represent the chosen ML model, that is often an Artificial Neural Networks (ANNs) due to their widespread use. However, ANNs frequently contain a large number of parameters, resulting in MIP formulations impractical to solve. In this paper we showcase the effectiveness of a ANN pruning, when applied to models prior to their integration into MIPs. We discuss why pruning is more suitable in this context than other ML compression techniques, and we highlight the potential of appropriate pruning strategies via experiments on MIPs used to construct adversarial examples to ANNs. Our results demonstrate that pruning offers remarkable reductions in solution times without hindering the quality of the final decision, enabling the resolution of previously unsolvable instances.
近年来,机器学习(ML)模型与运筹学(OR)工具的整合在癌症治疗、算法配置和化学过程优化等应用中越来越受欢迎。这种整合通常使用混合整数编程(MIP)公式来表示所选的 ML 模型,由于人工神经网络(ANN)的广泛使用,这种模型通常是人工神经网络。然而,ANN 经常包含大量参数,导致 MIP 公式难以解决。在本文中,我们展示了在将模型集成到 MIPs 之前对其进行剪枝处理的有效性。我们讨论了为什么剪枝在这种情况下比其他 ML 压缩技术更合适,并通过对用于为 ANN 构建对抗示例的 MIP 进行实验,强调了适当剪枝策略的潜力。我们的结果表明,剪枝能显著缩短求解时间,同时不影响最终决策的质量,从而解决以前无法求解的实例。
期刊介绍:
Operations Research Letters is committed to the rapid review and fast publication of short articles on all aspects of operations research and analytics. Apart from a limitation to eight journal pages, quality, originality, relevance and clarity are the only criteria for selecting the papers to be published. ORL covers the broad field of optimization, stochastic models and game theory. Specific areas of interest include networks, routing, location, queueing, scheduling, inventory, reliability, and financial engineering. We wish to explore interfaces with other fields such as life sciences and health care, artificial intelligence and machine learning, energy distribution, and computational social sciences and humanities. Our traditional strength is in methodology, including theory, modelling, algorithms and computational studies. We also welcome novel applications and concise literature reviews.