PDSR: Optimization of SNIP Pre-Pruning Algorithm Based On Dynamic Sparsity Rate

Abstract

As a key parameter of network pruning, the sparsity rate determines the sparsity effect after network pruning, and is closely related to the complexity, accuracy and application of neural network. Therefore, the determination of neural network sparsity rate has become one of the research hotspots. After reading a large number of relevant literatures, it is found that in the process of model training, the value of sparsity rate is often set artificially according to experience. So that the sparsity rate cannot change dynamically with the change of experimental environment and data, and the accuracy of its value is difficult to determine. To solve the above problems, this paper introduces dynamic sparsity rate, optimizes the SNIP pre-pruning algorithm, and proposes the PDSR algorithm. It calculates the sparsity rate dynamically according to the connection sensitivity of weights and realizes the pre-pruning of neural networks. Experimental results on various convolutional neural networks show that compared with SNIP algorithm, the PDSR algorithm has obvious improvement in accuracy rate and operation efficiency.