一种基于GA-SMSM的CNN各层剪枝率优化方法

IF 3.3 2区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Memetic Computing Pub Date : 2023-11-17 DOI:10.1007/s12293-023-00402-2

Xiaoyu Dong, Pinshuai Yan, Mengfei Wang, Binqi Li, Yuantao Song

{"title":"一种基于GA-SMSM的CNN各层剪枝率优化方法","authors":"Xiaoyu Dong, Pinshuai Yan, Mengfei Wang, Binqi Li, Yuantao Song","doi":"10.1007/s12293-023-00402-2","DOIUrl":null,"url":null,"abstract":"<p>Parameter pruning is one of the primary methods for compressing CNN models, aiming to reduce redundant parameters, the complexity of time and space, and the calculation resources of the network, all while ensuring minimal loss in the network’s performance. Currently, most existing parameter pruning methods adopt equal pruning rates across all layers. Different from previous methods, this paper focuses on the optimal combination of each layer’s pruning rates within a given pruning rate of the whole model. Genetic algorithm is used to determine the pruning rate for each layer. It’s worth noting that while the pruning rate for individual layers may vary, the average pruning rate across all layers does not exceed the given pruning rate. Experimental validation is conducted on CIFAR10 and ImageNet ILSVRC2012 datasets using VGGNet and ResNet architectures. The results show that the accuracy loss and the FLOPs of the pruned model using our method are superior to those pruned using previous methods.</p>","PeriodicalId":48780,"journal":{"name":"Memetic Computing","volume":"11 2","pages":""},"PeriodicalIF":3.3000,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"An optimization method for pruning rates of each layer in CNN based on the GA-SMSM\",\"authors\":\"Xiaoyu Dong, Pinshuai Yan, Mengfei Wang, Binqi Li, Yuantao Song\",\"doi\":\"10.1007/s12293-023-00402-2\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Parameter pruning is one of the primary methods for compressing CNN models, aiming to reduce redundant parameters, the complexity of time and space, and the calculation resources of the network, all while ensuring minimal loss in the network’s performance. Currently, most existing parameter pruning methods adopt equal pruning rates across all layers. Different from previous methods, this paper focuses on the optimal combination of each layer’s pruning rates within a given pruning rate of the whole model. Genetic algorithm is used to determine the pruning rate for each layer. It’s worth noting that while the pruning rate for individual layers may vary, the average pruning rate across all layers does not exceed the given pruning rate. Experimental validation is conducted on CIFAR10 and ImageNet ILSVRC2012 datasets using VGGNet and ResNet architectures. The results show that the accuracy loss and the FLOPs of the pruned model using our method are superior to those pruned using previous methods.</p>\",\"PeriodicalId\":48780,\"journal\":{\"name\":\"Memetic Computing\",\"volume\":\"11 2\",\"pages\":\"\"},\"PeriodicalIF\":3.3000,\"publicationDate\":\"2023-11-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Memetic Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1007/s12293-023-00402-2\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Memetic Computing","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s12293-023-00402-2","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

摘要

参数剪枝是压缩CNN模型的主要方法之一，旨在减少冗余参数，减少时间和空间的复杂性，减少网络的计算资源，同时保证网络的性能损失最小。目前，现有的参数剪枝方法大多采用各层相等的剪枝率。与以往的方法不同，本文关注的是在给定的整个模型剪枝率范围内，各层剪枝率的最优组合。采用遗传算法确定每一层的剪枝率。值得注意的是，虽然各个层的剪枝率可能不同，但所有层的平均剪枝率不会超过给定的剪枝率。采用VGGNet和ResNet架构在CIFAR10和ImageNet ILSVRC2012数据集上进行了实验验证。结果表明，采用该方法修剪后的模型精度损失和FLOPs均优于以往的方法。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

摘要图片

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

An optimization method for pruning rates of each layer in CNN based on the GA-SMSM

Parameter pruning is one of the primary methods for compressing CNN models, aiming to reduce redundant parameters, the complexity of time and space, and the calculation resources of the network, all while ensuring minimal loss in the network’s performance. Currently, most existing parameter pruning methods adopt equal pruning rates across all layers. Different from previous methods, this paper focuses on the optimal combination of each layer’s pruning rates within a given pruning rate of the whole model. Genetic algorithm is used to determine the pruning rate for each layer. It’s worth noting that while the pruning rate for individual layers may vary, the average pruning rate across all layers does not exceed the given pruning rate. Experimental validation is conducted on CIFAR10 and ImageNet ILSVRC2012 datasets using VGGNet and ResNet architectures. The results show that the accuracy loss and the FLOPs of the pruned model using our method are superior to those pruned using previous methods.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Memetic Computing COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE-OPERATIONS RESEARCH & MANAGEMENT SCIENCE

CiteScore

6.80

自引率

12.80%

发文量

期刊介绍： Memes have been defined as basic units of transferrable information that reside in the brain and are propagated across populations through the process of imitation. From an algorithmic point of view, memes have come to be regarded as building-blocks of prior knowledge, expressed in arbitrary computational representations (e.g., local search heuristics, fuzzy rules, neural models, etc.), that have been acquired through experience by a human or machine, and can be imitated (i.e., reused) across problems. The Memetic Computing journal welcomes papers incorporating the aforementioned socio-cultural notion of memes into artificial systems, with particular emphasis on enhancing the efficacy of computational and artificial intelligence techniques for search, optimization, and machine learning through explicit prior knowledge incorporation. The goal of the journal is to thus be an outlet for high quality theoretical and applied research on hybrid, knowledge-driven computational approaches that may be characterized under any of the following categories of memetics: Type 1: General-purpose algorithms integrated with human-crafted heuristics that capture some form of prior domain knowledge; e.g., traditional memetic algorithms hybridizing evolutionary global search with a problem-specific local search. Type 2: Algorithms with the ability to automatically select, adapt, and reuse the most appropriate heuristics from a diverse pool of available choices; e.g., learning a mapping between global search operators and multiple local search schemes, given an optimization problem at hand. Type 3: Algorithms that autonomously learn with experience, adaptively reusing data and/or machine learning models drawn from related problems as prior knowledge in new target tasks of interest; examples include, but are not limited to, transfer learning and optimization, multi-task learning and optimization, or any other multi-X evolutionary learning and optimization methodologies.