Few-Shot Open-Set Collaborative Classification of Multispectral and Hyperspectral Images With Adaptive Joint Similarity Metric

Abstract

Hyperspectral images (HSIs) have higher spectral resolution than multispectral (MS) images, but they have a narrower swath than MS images. The limited spectral resolution of MS images constrains their classification capabilities, and annotating remote sensing data is time-consuming and laborious. In addition, large-scale MS images may contain unknown classes not present in the training data. This article attempts to use partially overlapping HS images with limited labels to assist in the classification of large-scene MS images. It can correctly distinguish known classes and simultaneously identify unknown classes, thereby achieving better classification results for MS images. To address this challenge, a few-shot open-set HS–MS image collaborative classification method is proposed. Specifically, a spectral–spatial feature interactive enhancement (SSFIE) module is designed for richer feature extraction and enhanced classification capabilities in the feature extraction stage. In the few-shot learning (FSL) stage, an adaptive joint similarity metric criterion is proposed to improve feature mapping between the source and target domains. Discriminative joint probability adaptation (DJPA) is used for domain adaptation and to enhance feature discriminability, while batch nuclear-norm maximization (BNM) is employed to increase the feature diversity. In the testing phase, the open-set classification module is designed to correctly classify samples of known classes while simultaneously distinguishing unknown classes. The experimental results on four cross-domain HS–MS data pairs demonstrate that our proposed method outperforms state-of-the-art methods.