An Efficient Mixed Bit-width Searching Strategy for CNN Quantization based on BN Scale Factors

Abstract

In recent years, the rapid development of mixed-precision quantification technology has greatly reduced the scale of the model and the amount of calculation. However, the previous mixed bit-width strategies are too complicated, such as reinforcement learning strategies and Hessian matrix strategies. This paper proposes an efficient mixed bit-width searching strategy, which measures the sensitivity of the convolutional layer by the scale factors of the BN layer. The advantage of this strategy is that the parameters of the pre-trained model are used and no extra computation is introduced, which greatly simplifies the complexity of the bit-width selection strategy. In this paper, Resnet18 and Resnet50 models are used to conduct comparative experiments, and the differences between the proposed strategy and several previous algorithms are compared in terms of accuracy, model size and computation amount. It is verified that the accuracy of quantization in this paper is reduced within 2% compared with FP32 baseline, and the accuracy is reduced with about 0.5% compared with HAWQ. Overall, the performance is similar to that of HAWQ. This paper also compares the calculation complexity of the quantized bit-width of HAWQ-V3 with the calculation complexity of the quantized bit-width of this paper, which proves that the computational complexity of the strategy in this paper is far less than that of HAWQ-V3.