PRF: deep neural network compression by systematic pruning of redundant filters

Abstract

In deep neural networks, the filters of convolutional layers play an important role in extracting the features from the input. Redundant filters often extract similar features, leading to increased computational overhead and larger model size. To address this issue, a two-step approach is proposed in this paper. First, the clusters of redundant filters are identified based on the cosine distance between them using hierarchical agglomerative clustering (HAC). Next, instead of pruning all the redundant filters from every cluster in single-shot, we propose to prune the filters in a systematic manner. To prune the filters, the cluster importance among all clusters and filter importance within each cluster are identified using the \(\ell _1\)-norm based criterion. Then, based on the pruning ratio filters from the least important cluster to the most important ones are pruned systematically. The proposed method showed better results compared to other clustering-based works. The benchmark datasets CIFAR-10 and ImageNet are used in the experiments. After pruning 83.92% parameters from VGG-16 architecture, an improvement over the baseline is observed. After pruning 54.59% and 49.33% of the FLOPs from ResNet-56 and ResNet-110, respectively, both showed an improvement in accuracy. After pruning 52.97% of the FLOPs, the top-5 accuracy of ResNet-50 drops by only 0.56 over ImageNet.