A Theoretical CNN Compression Framework for Resource-Restricted Environments

Abstract

Convolutional Neural Network (CNN) is considered as one of the most significant algorithms of deep learning that has made impressive achievements in many areas of computer vision and natural language processing. In the current times of big data, input data dimensions keep on increasing which leads to the development of complex CNN models for processing such big data. This has made CNN computationally intensive and limits its practical application to some extent. To address the aforementioned issue, this paper presents a detailed review of various network compression methods existing in literature. Two most commonly deployed network compression methods have been discussed including pruning and quantization which can be coupled with CNN to increase its applicability. The main goal of presenting this comprehensive review of the state-of-the-art pruning and quantization-based network compression schemes is to significantly improve trade-off between CNN architectural size and computational cost versus its performance in resource restricted environments. Additionally, this paper also exploits the challenges posed by these techniques when implemented for large-scale CNNs. In this context, paper also presents a novel framework to perform network compression of CNN to meet the requirements of resource-restricted devices.