Two-Level Semantic-Driven Diffusion Based Hyperspectral Pansharpening

Abstract

Over recent years, denoising diffusion probabilistic models (DDPMs) have received many attentions due to their powerful ability to infer data distribution. However, most of existing DDPM-based hyperspectral (HS) pansharpening methods over rely on local processing to perform recovery, which usually fails to reconcile global contextual semantics and local details in data. To address the issue, we propose a two-level semantic-driven diffusion method for HS pansharpening. In our method, we first extract semantics in two levels, where the low-level semantic not only leads the extraction of conditional details, but also supports the further semantic extraction while the high-level semantic is related to scene cognition. Then, the features from both the low-level and high-level semantics are conditionally injected to the denoising network to guide the high-resolution HS recovery. Experiments on multiple datasets verify the effectiveness of our method.