See as You Desire: Scale-Adaptive Face Super-Resolution for Varying Low Resolutions

Abstract

Face super-resolution (FSR) is critical for bolstering intelligent security in Internet of Things (IoT) systems. Recent deep learning-driven FSR algorithms have attained remarkable progress. However, they always require separate model training and optimization for each scaling factor or input resolution, leading to inefficiency and impracticality. To overcome these limitations, we propose SAFNet, an innovative framework tailored for scale-adaptive FSR with arbitrary input resolution. SAFNet integrates scale information into representation learning to enable adaptive feature extraction and introduces dual-embedding attention to boost adaptive feature reconstruction. It leverages facial self-similarity and spatial-frequency collaboration to achieve precise scale-aware SR representations. This is attained through three key modules: 1) the scale adaption guidance unit (SAGU); 2) the scale-aware nonlocal self-similarity (SNLS) module; and 3) the spatial-frequency interactive modulation (SFIM) module. SAGU imports scaling factors using frequency encoding, SNLS exploits self-similarity to enrich feature representations, and SFIM incorporates spatial and frequency information to predict target pixel values adaptively. Comprehensive evaluations across four benchmark datasets reveal that SAFNet outperforms the second-best compared state-of-the-art (SOTA) method by about 0.2 dB/0.007 in PSNR/SSIM ( $\times 4$ on CelebA) with reduced 18.68%/42.64% computational complexity/time cost. This demonstrates SAFNet’s effectiveness and superiority, showcasing its potential as a promising solution for scale and input resolution adaptation challenges in FSR. The code will be available at https://github.com/ICVIPLab/SAFNet.