Uncertainty guided semi-supervised few-shot segmentation with prototype level fusion

Abstract

Few-Shot Semantic Segmentation (FSS) aims to tackle the challenge of segmenting novel categories with limited annotated data. However, given the diversity among support-query pairs, transferring meta-knowledge to unseen categories poses a significant challenge, particularly in scenarios featuring substantial intra-class variance within an episode task. To alleviate this issue, we propose the Uncertainty Guided Adaptive Prototype Network (UGAPNet) for semi-supervised few-shot semantic segmentation. The key innovation lies in the generation of reliable pseudo-prototypes as an additional supplement to alleviate intra-class semantic bias. Specifically, we employ a shared meta-learner to produce segmentation results for unlabeled images in the pseudo-label prediction module. Subsequently, we incorporate an uncertainty estimation module to quantify the difference between prototypes extracted from query and support images, facilitating pseudo-label denoising. Utilizing these refined pseudo-label samples, we introduce a prototype rectification module to obtain effective pseudo-prototypes and generate a generalized adaptive prototype for the segmentation of query images. Furthermore, generalized few-shot semantic segmentation extends the paradigm of few-shot semantic segmentation by simultaneously segmenting both unseen and seen classes during evaluation. To address the challenge of confusion region prediction between these two categories, we further propose a novel Prototype-Level Fusion Strategy in the prototypical contrastive space. Extensive experiments conducted on two benchmarks demonstrate the effectiveness of the proposed UGAPNet and prototype-level fusion strategy. Our source code will be available on https://github.com/WHL182/UGAPNet.