A Graph-Convolutional-Network based Prototype Mixing Model for Few-shot Segmentation

Abstract

Over the past few years, deep convolutional neural networks (CNNs) based semantic segmentation methods reached the state-of-the-art performance. To train a model with the ability to know a concept, a lot of pixel level annotated images are required, which is time consuming and hard to cover unseen object categories. Thus, few-shot semantic segmentation has been developed to implement segmentation with a few annotation images. In this paper, we proposed a novel prototype mixing model for few shot segmentation. Different with other works which only produce prototypes form support set, our proposed model learn a group of concept-specific prototypes from support set and then generate prototypes from query set. With prototypes from both query set and support set, we proposed a GCN(Graphic Convolutional Network) module to generate mixing prototypes for better utilizing of informations from different categories. We also proposed a clustering module to produce multi-prototypes for representing different parts of a single semantic class, which reach better performance than single prototype. Our model achieve 48.8% and 55.9%mIoU score on PASCAL-5i for 1-shot and 5-shot settings respectively.