Context-Preserving Filter Reorganization for VDSR-Based Super-resolution

Abstract

This paper presents a hardware design to process a CNN for single image super-resolution (SISR). Very deep convolutional network for image super-resolution (VDSR) is a promising algorithm for SISR but it is too complex to be implemented in hardware for commercial products. The proposed design aims to implement VDSR with relatively small hardware resources while minimizing a degradation of image quality. To this end, 1D reorganization of a convolution filter is proposed to reduce the number of multipliers. In addition, the 1D vertical filter is changed to reduce the internal SRAM to store the input feature map. For the implementation with a reasonable hardware cost, the numbers of layers and channels per layer, as well as the parameter resolution, are decreased without a significant reduction of image quality which is observed from simulation results. The 1D reorganization reduces the number of multiplies to 55.6% whereas the size reduction of 1D vertical filter halves the buffer size. As a result, the proposed design processes a full-HD video in real time with 8,143.5k gates and 333.1kB SRAM while the image quality is degraded by 1.06dB when compared with VDSR.