Spectral recovery-guided hyperspectral super-resolution using transfer learning

Single hyperspectral image (HSI) super-resolution (SR) has attracted researcher’s attention; however, most existing methods directly model the mapping between low- and high-resolution images from an external training dataset, which requires large memory and com-puting resources. Moreover, there are few such available training datasets in real cases, which prevent deep-learning-based methods from further improving performance. Here, a novel single HSI SR method based on transfer learning is proposed. The proposed method is composed of two stages: spectral down-sampled image SR reconstruction based on transfer learning and HSI reconstruction via spectral recovery module. Instead of directly applying the learned knowledge from the colour image domain to HSI SR, the spectral down-sampled image is fed into a spatial SR model to obtain a high-resolution image, which acts as a bridge between the colour image and HSI. The spectral recovery network is used to restore the HSI from the bridge image. In addition, pre-training and collaborative fine-tuning are proposed to promote the performance of SR and spectral recovery. Experiments on two public HSI datasets show that the proposed method achieves promising SR performance with a small paired HSI dataset.