Self-Supervised Pretraining for Stereoscopic Image Super-Resolution With Parallax-Aware Masking

Abstract

Most existing learning-based methods for stereoscopic image super-resolution rely on a great number of high-resolution stereoscopic images as labels. To alleviate the problem of data dependency, this paper proposes a self-supervised pretraining-based method for stereoscopic image super-resolution (SelfSSR). Specifically, to develop a self-supervised pretext task for stereoscopic images, a parallax-aware masking strategy (PAMS) is designed to adaptively mask matching areas of the left and right views. With PAMS, the network is encouraged to effectively predict missing information of input images. Besides, a cross-view Transformer module (CVTM) is presented to aggregate the intra-view and inter-view information simultaneously for stereoscopic image reconstruction. Meanwhile, the cross-attention map learned by CVTM is utilized to guide the masking process in PAMS. Comparative results on four datasets show that the proposed SelfSSR achieves state-of-the-art performance by using only 10% of labeled training data.