Cross-Scene Hyperspectral Image Classification With Consistency-Aware Customized Learning

Abstract

Recently, unsupervised domain adaptation (UDA) techniques have been introduced for cross-scene hyperspectral image (HSI) classification tasks. These techniques aim to transfer knowledge from labeled source scenes to unlabeled target scenes, addressing the issue of limited supervisory information. However, most UDA methods fail to analyze the variability of domain shifts from different source samples to target ones, thus limiting the domain adaptation effect. To this end, this paper develops a consistency-aware customized learning (CACL) approach for cross-scene HSI classification. Overall, domain-level and class-level distribution alignment are designed separately. The former is implemented by adversarial training between the feature extractor and the domain discriminator. For the latter, the spectral-spatial prototypes of the source and target domains are first dynamically extracted, respectively. Then the prototype-based labels are assigned to the target domain samples, according to the cosine similarity-based cross-domain category prototype matching strategy. Considering that the consistency of the prototype-based labels with the predicted pseudo-labels reflects the degree of domain shifts of the target samples, a customized learning strategy is developed via inter-/intra-domain contrastive learning. With the joint domain-level and fine-grained class-level distribution alignment, the supervised information from the source domain is better migrated to the target domain, improving classification performance. Comprehensive experiments on two single-modal and one multi-modal cross-scene datasets demonstrate the effectiveness of the proposed algorithm.