LRDTN: Spectral–Spatial Convolutional Fusion Long-Range Dependence Transformer Network for Hyperspectral Image Classification

Abstract

Recently, deep learning has achieved remarkable breakthroughs in hyperspectral image (HSI) classification tasks, particularly with methods based on convolutional neural networks (CNNs) and transformers. However, these methods have several limitations: 1) the limited receptive field inherent in the convolutional layer greatly hampers capturing feature contextual information on a large scale and 2) transformers cannot establish strong local relationships, making it challenging to characterize complex dependencies between distant pixels and different bands in HSIs. Moreover, as the network complexity increases, so does the number of network parameters. We propose a novel network called the spectral-spatial convolutional fusion long-range dependence transformer network (LRDTN) for HSI classification to address these challenges. LRDTN comprises three key components: dynamic-dependent convolutional (DDC) module, the multiscale enhanced fusion (MsEF) module, and the local–global perception transformer (LGPT). Specifically, the DDC dynamically models local features, while the MsEF integrates information from different scales to capture contextual relationships in HSI features effectively. Additionally, the ability to mine and utilize HSI local-global features and complex long-range dependencies is enhanced by the proposed transformer variant, LGPT. Ultimately, through the ingeniously designed structure of the LRDTN, the model effectively maintains its performance while reducing the number of network parameters. Extensive experiments conducted on four typical HSI datasets, including urban areas, agricultural areas, and swamps, demonstrate the superiority of LRDTN over other state-of-the-art networks. The code is available at https://github.com/ybyangjing/LRDTN .