Co-Graph Convolution for Instance Segmentation

Abstract

Segmenting various instances in various contexts with a common model is a challenge for instance segmentation. In this paper, we address this problem by capturing rich relationship information and propose our Co-Graph Convolution Network (CGC-Net). Based on Mask R-CNN, we propose our co-graph convolution mask head. Specifically, we decouple the mask head into two mask heads. For each mask head, we append a graph convolution layer to capture the corresponding relationship information. One focuses on the relationship information between appearance features for each position of the instance itself, while the other pays more attention to the semantic relationship between each channel for the corresponding instance's features. In addition, we add a co-relationship module to each graph convolution layer to share similar relationships between instances with the same category in an image. We integrate the outputs of two mask heads by element-wise multiplication to improve feature representation for final instance segmentation prediction. Compared with other state-of-the-art instance segmentation methods, experiments on MS COCO and Cityscapes datasets demonstrate our method's competitiveness. Besides, in order to verify the generalization of our CGC-Net, we also add our CGC-Net to other instance segmentation networks, and the experiment results show our method still can obtain stable gains in performance.