Enhancing Visual Data Completion With Pseudo Side Information Regularization

Abstract

Unsupervised image restoration methods relying on a single data source often face challenges in achieving high-quality visual data completion due to the absence of additional supplementary information. This paper presents a novel optimization framework to address this limitation and further enhance the performance of image restoration. The framework generates pseudo side information (PSI) and utilizes it to guide the process of visual data completion. We introduce a pseudo side information regularizer (PSIR) tailored specifically for visual data completion tasks. The PSIR comprises two components: the PSI generator and updater, responsible for generating and refining the PSI, and the neural self-expressive prior (NSEP), which identifies a prior matching the desired result and PSI during optimization. Notably, our method achieves comprehensive visual data completion across various data types without the need for additional reference side information or training data. Extensive experimental evaluations conducted on spectral data (including color images, multispectral images, and hyperspectral images), video data (including gray video, color video, and hyperspectral video), magnetic resonance image, and real cloud data demonstrate the superiority of our approach over other state-of-the-art completion methods under different missing rate scenarios.