Feature Reconstruction Guided Fusion Network for Hyperspectral and LiDAR Classification

Abstract

Deep learning has become increasingly popular in hyperspectral image (HSI) and light detection and ranging (LiDAR) data classification, thanks to its powerful feature learning and representation capabilities. However, HSI often contains substantial redundant information, which can hinder efficient data utilization. Furthermore, the significant disparity in information content between HSI and LiDAR data poses a major challenge in representing and aligning semantic information across these two modalities. To address these challenges, we propose a fusion network structure guided by feature reconstruction embedding (FRE). This approach employs feature decomposition to reconstruct HSI features and incorporates weight embedding to seamlessly integrate the reconstructed information into classification features. Furthermore, we introduce a cross-modal attention fusion module designed to merge extracted HSI and LiDAR features. This module fully exploits the complementary nature of these two type of feature, facilitating effective information exchange and semantic alignment across multimodal data. We evaluated our method on three widely used HSI and LiDAR datasets: Houston 2013, Augsburg, and MUUFL. Experimental results demonstrate that our proposed FRGFNet significantly outperforms traditional probabilistic methods and state-of-the-art deep learning networks, showcasing its effectiveness in multisource data fusion.