High-Frequency Feature Learning in Image Super-Resolution with Sub-Pixel Convolutional Neural Network

Abstract

Sub-pixel convolutional neural network is efficient for image super-resolution. However, the images generated are relatively smooth. Improving the learning ability of high-frequency features is of great significance for sub-pixel convolutional neural network to get better performance. In the paper, we propose an improved algorithm of sub-pixel convolutional neural network based on high-frequency feature learning for image super-resolution, it optimizes the traditional sub-pixel convolutional structure. Firstly we introduce a residual convolutional layer in the generation net. it assigns the residual factor to each sub-pixel feature map and forces each pixel feature map to adaptively use the input information. Furthermore, a method for high frequency feature mapping is proposed. During image super-resolution training stage, the multi-task learning function, combining the pixel-level loss function with high-frequency contrast loss function, make the generation images getting closer to the target super-resolution images in high-frequency domain. The experiments on CelebA dataset show that our proposed method can effectively improve the quality of super-resolution images by contrast to the traditional sub-pixel convolutional neural network.