Hyperspectral image (HSI) is extensively used in classification, detection, tracking, and other tasks attributed to its capacity to capture extensive spectral information and comprehensively characterize the spectral signatures unique to distinct materials. Nevertheless, the acquisition of high-resolution HSI (HR-HSI) is fundamentally challenged by the intrinsic technical constraints of HSI sensors. Therefore, the fusion of low-resolution HSI and high-resolution multispectral image (MSI) to obtain HR-HSI has become a research hotspot. Existing fusion algorithms often do not make full use of the correlation between spatial and spectral information, which makes the fusion model lack interpretability. Therefore, this paper builds a spectral-assisted multi-receptive field fusion network (SMF2Net) for HSI and MSI fusion. Specifically, for spatial information, this paper designs a spectral feature-based spatial partition convolution block (SFSPB) and a multi-receptive field interaction fusion block (MFIFB) to capture the spatial information in the source images. Among them, the SFSPB extracts spatial information in different regions according to the spectral features of the image; meanwhile, the MFIFB extracts the spatial information of different receptive fields and performs feature fusion so that SMF2Net can obtain rich semantic information. For spectral information, this paper constructs a HybridFormer block based on spectral multi-head self-attention. It models the long-distance dependence of the spectrum through multi-head self-attention and enhances the spectral information in the fusion result through channel attention. In this paper, experiments are carried out on one real dataset and two simulated datasets. The experimental results indicate that the proposed algorithm can achieve advanced fusion effectiveness subjectively and objectively. The source code is available at: https://github.com/cvmdsp/SMF2Net.
扫码关注我们
求助内容:
应助结果提醒方式:
