A Multistage Semi-Supervised Network for Hyperspectral Super-Resolution

Abstract

Hyperspectral imaging can capture abundant spectral information and reveal the spectral absorption properties of surface materials. Nevertheless, the tradeoff in spatial resolution reduces its capacity to represent surface object textures and structures; hyperspectral super-resolution (SR) technology is a viable solution to this problem. Yet, mainstream supervised methods depend on low-scale training data and specific data distributions, restricting their generalization capability and practicality in real scenarios. Although unsupervised methods remove the reliance on training data, they still face suboptimal reconstruction quality due to the absence of reference images and inaccuracies in degradation process estimation. Furthermore, bridging the performance gap between simulated datasets and real-world applications remains challenging. To this end, we propose a semi-supervised network that effectively couples unsupervised learning and supervised pretraining in multistage architecture, MCS-Net for short. The network consists of three key components: degradation information estimation (DIE), supervised fusion pretraining (SFP) at low resolution, and unsupervised image generation (UIG) at full resolution. The MCS-Net first estimates deep degradation information from input image pairs using DIE. It then applies supervised learning in SFP to construct a pretrained fusion function and its parameters from the input low-resolution data pairs and their fused outputs. Finally, the pretrained parameters from the previous stage are used to initialize the fusion network of UIG, which is then fine-tuned under the guidance of degradation parameters estimated by DIE, enabling the network to process full-resolution images effectively. Ablation experiments validated the effectiveness of each component. Moreover, the proposed MCS-Net outperforms the existing state of the art (SOTA) methods across six evaluation metrics in the simulation experiments, and the experimental results on real satellite data further validate the outstanding image fusion performance of MCS-Net and its potential for practical applications.