EdgeStereoSR: A multi-task network with transformers for stereo image super-resolution considering edge prior

Abstract

Recently, stereo image super-resolution methods focusing on exploring cross-view information have been widely studied and achieved good performance. However, it is still challenging for them to reconstruct high-quality high-frequency details. In addition, they mainly focus on improving quantitative metrics, neglecting the perceptual quality of reconstructed images. In this paper, to improve the accuracy of high-frequency reconstruction, we propose a multi-task network with Transformers considering edge prior, named EdgeStereoSR, which achieves better stereo image SR under the guidance of edge detection. Basically, edge priors have two contributions. First, we propose a cross-view Transformer (CVT), which utilizes edge priors to guide the correspondence search, thus more accurate cross-view information can be captured. Second, we propose a cross-task Transformer (CTT), which exploits edge priors to guide the high-frequency reconstruction, thus images with more details and sharper edges can be reconstructed. To further improve the visual quality, we propose EdgeStereoSR-G, integrating the generative adversarial network into EdgeStereoSR. Specially, a spatial-view discriminator is designed to learn the stereo image distribution so as to make the reconstructed stereo image more photo-realistic and avoid parallax inconsistency. Extensive experiments show that the proposed methods are superior to other state-of-the-art methods in terms of both quantitative metrics and visual quality.