A deep unfolding network based on intrinsic image decomposition for pansharpening

Abstract

Pansharpening aims to obtain high-resolution multispectral images by fusing panchromatic and low-resolution multispectral images. However, current deep learning-based methods lack reasonable interpretability and suffer from certain spectral and spatial distortions. To address these issues, we propose an interpretable deep unfolding network based on intrinsic image decomposition, called DUN-IID. IID decomposes the multispectral image into a reflectance component and a shading component to formulate a novel variational optimization function. The reflectance and shading components effectively reflect spectral and spatial information, respectively. This decoupling strategy enhances feature fidelity by preventing interference between spectral and spatial information, enabling independent optimization of spatial reconstruction and spectral correction during fusion. The optimization function is solved by the half-quadratic splitting method and unfolded into the end-to-end DUN-IID, which consists of two primal update blocks for prior learning and two dual update blocks for reconstruction. To alleviate the effects of information loss across intermediate stages, we introduce the source images into two primal update blocks for information enhancement. Therefore, the reflectance and shading primal update blocks are customized as a multi-scale structure and a band-aware construction, respectively. Besides, the multi-dimension attention mechanism is adopted to improve feature representation. Extensive experiments validate that our method is superior to other state-of-the-art methods.