Contour-Enhanced Visual State-Space Model for Remote Sensing Image Classification

Abstract

The accurate classification of remote sensing (RS) images can quickly identify various geographical features, which is important for planning, utilizing, and protecting natural resources. Recently, the visual Mamba model, as an extension of the vision transformer (ViT), is attracting widespread attention due to its global receptive field and linear complexity. However, the self-attention mechanism of visual transformers can lead to feature collapse in the deep layers, resulting in the disappearance of low-level visual features. In RS images, low-level features, and especially luminance gradient features, can help discern object boundaries and contour information. This is beneficial for the accurate classification of images but has not been fully leveraged. To make full use of contour information and explore the impact of using handcrafted low-level features on the deep layers of the model, in this study, a contour-enhanced Mamba model based on Vision Mamba (VMamba), is proposed, named G-VMamba. The core novelty of G-VMamba lies in its contour enhancement module (ConEM). First, two separate paths are used to extract adaptive luminance gradients and multidimensional convolutional features at each network layer. Subsequently, the features are combined to impose the constraints of low-level features onto the deeper networks. RS image classification experiments were conducted to evaluate the model’s performance, and the results demonstrate the superior performance of G-VMamba in classification tasks. An analysis of class activation maps (CAMs) across different categories shows that G-VMamba focuses more on color (or luminance) change significantly regions in images than models like VMamba, highlighting the efficacy of contour enhancement. The code will be available at: https://github.com/yanliyue/Contour-enhanced-Visual-State-Space-Model .