Efficient Sampling-based Gaussian Processes for few-shot semantic segmentation

Abstract

Few-shot segmentation (FSS) is a longstanding challenge in computer vision. Previous methods adopting Gaussian Processes (GPs) aggregate detailed information and manage complex distributions from small support sets, thereby modeling uncertainty of features and handling wide variations in context. However, the exact GP-based FSS methods struggle with computational burden and information redundancy. To tackle the issues, we propose ESGP, an Efficient Sampling-based Gaussian Process framework for few-shot segmentation. The model decouples the GP into a two-step process: weight space approximation for the prior and function space update for the posterior. Additionally, we adopt Deep Kernel Learning to enhance ESGP’s performance. This combination results in a faster, more accurate FSS model that effectively concentrates support sample information. Moreover, GP’s inherent ability to model uncertainty provides robust predictions and valuable insights into segmentation reliability. Experimental results demonstrate that ESGP outperforms previous GP-based methods and achieves competitive performance with state-of-the-art techniques.