Efficient LUT-based FPGA Accelerator Design for Universal Quantized CNN Inference

Abstract

Deep learning has achieved remarkable success in a variety of tasks in real life, such as speech and vision. However, the vast computational complexity of convolution neural networks (CNN) has limited the speed of the network running in hardware. In recent years, network quantization technology has made it possible to quantize network into the 16-bit fixed point, 8-bit integer, and even binary, maintaining the original performance, while the computational complexity of the network inference is still considerable. Therefore, exploring high-performance and efficient hardware architecture designed for quantized neural networks (QNN) is necessary to eliminate the bottleneck of high-density computing requirements. FPGA is a highly parallelized hardware computing platform. The outstanding advantage is that it contains a large number of primary configurable logic resources. We explore the possibility of implementation for convolution calculations based on LUTs, introduce the integer multipliers and addition trees based on FPGAs, and propose an efficient computing architecture for QNN. With the optimization of Winograd convolution algorithm for QNN, we demonstrate that our scheme could significantly reduce the number of multipliers without using DSP resources, saving the usage of LUT resources by 2.25× at least. In the end, our LUT-based architecture for QNN will shorten the latency up to 19.3× and represent more effective performance compared other methods.