Bidomain uncertainty gated recursive network for pan-sharpening

Abstract

Pan-sharpening aims to integrate the complementary information of different modalities of satellite images, e.g., texture-rich PAN images and multi-spectral (MS) images, to produce more informative fusion images for various practical tasks. Currently, most deep learning based pan-sharpening techniques primarily concentrate on developing various elaborate architectures to enhance their representation capabilities. Despite significant advancements, these heuristic and intricate network designs result in models that lack interpretability and exhibit poor generalization ability in real-world scenarios. To alleviate these issues, we propose a simple yet effective spatial-frequency (bidomain) uncertainty gated progressive fusion framework for pan-sharpening, termed BUGPan. Specifically, the main body of BUGPan consists of multiple uncertainty gated recursive modules (UGRM) which are responsible for the cross-modal representation learning at different spatial resolutions. In contrast to the prior recursive designs that perform a fixed and manually set number of recursions, the UGRM introduces an innovative spatial-frequency uncertainty gated recursive mechanism featuring two key designs, i.e., Bidomain Uncertainty Estimation (BUE) and Uncertainty-Aware Gating (UAG). This mechanism strategically orchestrates the recursive embedding of features, and tailors the process to specific outcome contexts, enabling more transparent feature representation learning. To the best of our knowledge, this is not only the first attempt to introduce both spatial and frequency uncertainty in pan-sharpening, but we also extend the role of uncertainty in a novel functional mechanism design. Extensive experimental results highlight the superiority of the proposed BUGPan, surpassing other state-of-the-art methods both qualitatively and quantitatively across multiple satellite datasets. Particularly noteworthy is its ability to generalize effectively to real-world scenarios and new satellite data. The code is available at https://github.com/coder-JMHou/BUGPan.