Preliminary Investigation on Single Remote Sensing Image Inpainting Through a Modified GAN

Abstract

Because of impacts resulted from sensor malfunction and clouds, there is usually a great deal of missing regions (pixels) in remotely sensed imagery. To make full use of the remotely sensed imagery affected, different algorithms for remote sensing images inpainting have been proposed. In this paper, an unsupervised convolutional neural network (CNN) context generate model was modified to recover the affected (or un-recorded) pixels in a single image without auxiliary information. Unlike existing nonparametric algorithms in which pixels located in surrounding region are used to estimate the unrecorded pixel, the proposed method directly generates content based on a neural network. To ensure recovered results with high quality, a modified reconstruction loss was used in training the model, which included structural similarity index (SSIM) loss and Ll loss. Comparison of the proposed model with bilinear interpolation was indicated through relative error. The performances of two methods in scenes with different complexity were discussed further. Results show that the proposed model performed better in simple scenes (i.e., with relative homogeneity), compared to the traditional method. Meanwhile, the corrupted images of channel blue were recovered more accurately, compared to the corrupted images of other channels (i.e., channel green and channel red). The relationship between scene complexities and channels shows that same scene has different complexities in different channels. The scene complexity presents significant correlation with recovered results, high complexity images are always accompanied by poor recovered results. It suggests that the recovering accuracy depends on scene complexity.