Prototypical Metric Segment Anything Model for Data-Free Few-Shot Semantic Segmentation

Abstract

Few-shot semantic segmentation (FSS) is crucial for image interpretation, yet it is constrained by requirements for extensive base data and a narrow focus on foreground-background differentiation. This work introduces Data-free Few-shot Semantic Segmentation (DFSS), a task that requires limited labeled images and forgoes the need for extensive base data, allowing for comprehensive image segmentation. The proposed method utilizes the Segment Anything Model (SAM) for its generalization capabilities. The Prototypical Metric Segment Anything Model is introduced, featuring an initial segmentation phase followed by prototype matching, effectively addressing the learning challenges posed by limited data. To enhance discrimination in multi-class segmentation, the Supervised Prototypical Contrastive Loss (SPCL) is designed to refine prototype features, ensuring intra-class cohesion and inter-class separation. To further accommodate intra-class variability, the Adaptive Prototype Update (APU) strategy dynamically refines prototypes, adapting the model to class heterogeneity. The method's effectiveness is demonstrated through superior performance over existing techniques on the DFSS task, marking a significant advancement in UAV image segmentation.