Attention based network for fusion of polarimetric and contextual features for polarimetric synthetic aperture radar image classification

Abstract

Polarimetric synthetic aperture radar (PolSAR) images containing polarimetric, scattering and contextual features are useful radar data for ground surface classification. Appropriate feature extraction and fusion by using a small set of available labeled samples is an important and challenging task. Several transformers with self-attention mechanism have recently achieved great success for PolSAR image classification. While almost all methods just exploit the self-attention features from the PolSAR cube, the feature fusion method proposed in this work, which is called attention based scattering and contextual (ASC) network, utilizes the polarimetric self-attention beside two cross-attention blocks. The cross-attention blocks extract the polarimetric-scattering dependencies and polarimetric-contextual interactions, individually. The proposed ASC network uses three inputs: the PolSAR cube, the scattering feature maps obtained by clustering of the entropy-alpha features, and the segmentation maps obtained by a super-pixel generation algorithm. The features extracted by self- and cross-attention blocks are fused together, and the residual learning improves the feature learning. While transformers and attention-based networks usually need large training sets, the proposed ASC network shows high efficiency with relatively low number of training samples in various real and synthetic PolSAR images. For example, in the Flevoland PolSAR image containing 15 classes acquired by AIRSAR in L-band, with using 100 training samples per class (less than 1% of labeled samples), the ASC network achieves the overall accuracy of 99.51, which is statistically preferred than the self-attention-based network according to the McNemars test.