E2TNet: Efficient enhancement Transformer network for hyperspectral image classification

Abstract

Recently, Convolutional Transformer-based models have become popular in hyperspectral image (HSI) classification tasks and gained competitive classification performance. However, some Convolutional Transformer-based models fail to effectively mine the global correlations of coarse-grained and fine-grained features, which is adverse to recognizing the refined scale variation information of land-cover. The combination of convolution operations and multihead self-attention mechanisms also increases the computational cost, leading to low classification efficiency. In addition, shallow spectral–spatial features are directly input into the encoder, which inevitably incurs redundant spectral information. Therefore, this paper proposes an efficient enhancement Transformer network (E2TNet) for HSI classification. Specifically, this paper first designs a spectral–spatial feature fusion module to extract spectral and spatial features from HSI cubes and fuse them. Second, considering that redundant spectral information has a negative impact on classification performance, this paper designs a spectral–spatial feature weighted module to improve the feature representation of critical spectral information. Finally, to explore the global correlations of coarse-grained and fine-grained features and improve classification efficiency, an efficient multigranularity information fusion module is embedded in the encoder of E2TNet. The experiment is conducted on four benchmark hyperspectral datasets, and the experimental results demonstrate that the proposed E2TNet is better than several Convolutional Transformer-based classification models in terms of classification accuracy and classification efficiency.