An Image Restoration Architecture using Abstract Features and Generative Models

Image restoration and its different variations are important topics in low-level image processing. One of the main challenges in image restoration is dependency of current methods to the corruption characteristics. In this paper, we have proposed an image restoration architecture that enables us to address different types of corruption, regardless of type, amount and location. The main intuition behind our approach is restoring original images from abstracted perceptual features. Using an encoder-decoder architecture, image restoration can be defined as an image transformation task. Abstraction of perceptual features is done in the encoder part of the model and determines the sampling point within original images' Probability Density Function (PDF). The PDF of original images is learned in the decoder section by using a Generative Adversarial Network (GAN) that receives the sampling point from the encoder part. Concretely, sampling from the learned PDF restores original image from its corrupted version. Pretrained network extracts perceptual features and Restricted Boltzmann Machine (RBM) makes the abstraction over them in the encoder section. By developing a new algorithm for training the RBM, the features of the corrupted images have been refined. In the decoder, the Generator network restores original images from abstracted perceptual features while Discriminator determines how good the restoration result is. The proposed approach has been compared with both traditional approaches like BM3D and with modern deep models like IRCNN and NCSR. We have also considered three different categories of corruption including denoising, inpainting and deblurring. Experimental results confirm performance of the model.