A feature reuse framework with texture-adaptive aggregation for reference-based super-resolution

Abstract

Reference-based super-resolution (RefSR), significant success has been achieved in the field of super-resolution. It reconstructs low-resolution (LR) inputs using high-resolution reference images, obtaining more high-frequency details and alleviating the ill-posed problem of single-image super-resolution (SISR). Previous research in the RefSR has mainly focused on finding correlations, transferring, and aggregating similar texture information from LR reference (Ref) the LR. However, an essential detail of perceptual loss and adversarial loss has been underestimated, impacting texture transfer and reconstruction negatively. In this paper, we propose a feature reuse framework, FRFSR, which divides the model training into two steps. Firstly, the first model is trained using reconstruction loss to enhance its texture transfer and aggregation abilities. Secondly, using all losses for training, the feature output of the first model is reintroduced into the training process to supplement texture, generating visually appealing images. The feature reuse framework is applicable to any RefSR model, and experiments show that several RefSR methods exhibit improved performance when retrained with our reuse framework. Considering that the textures in the reference are not entirely consistent with those in the LR, this naturally leads to the problem of texture misuse. Therefore, we design a Dynamic Residual Block (DRB). The DRB utilizes the feature perception capability of decoupled dynamic filters to dynamically aggregate texture information between LR input and Ref images, reducing instances of texture misuse. The source code can be obtained from https://github.com/Yi-Yang355/FRFSR.