SFP: Similarity-based filter pruning for deep neural networks

Abstract

Convolutional neural networks have exhibited exceptional performance in various artificial intelligence domains, particularly in large-scale image processing tasks. However, the proliferation of network parameters and computational requirements has emerged as a significant bottleneck for the practical deployment of CNNs. In this paper, we propose a novel similarity-based filter pruning (SFP) approach for compressing convolutional neural networks, which is different from the traditional pruning method. The existing pruning methods eliminate the unimportant parameters but ignore the duplication of the reserved convolutional kernels. In the proposed SFP, kernels are clustered first according to their similarity, then the unimportant and redundant kernels are pruned in each class, which is more efficient than traditional pruning methods only based on the importance criterion. Furthermore, this paper introduces the concept of Kernel Dispersion to evaluate sparsity across distinct network layers, and proposes Distillation Fine-Tuning with Variable Temperature Coefficient to expedite convergence and enhance accuracy. The performance of the proposed similarity-based filter pruning approach is evaluated on different datasets, including CIFAR10, CIFAR100, ImageNet, and VOC. The experimental results indicate that the proposed SFP achieves approximately 1% higher accuracy at a comparable pruning rate compared to traditional state-of-the-art pruning methods.