Class agnostic and specific consistency learning for weakly-supervised point cloud semantic segmentation

Abstract

This paper focuses on Weakly Supervised 3D Point Cloud Semantic Segmentation (WS3DSS), which involves annotating only a few points while leaving a large number of points unlabeled in the training sample. Existing methods roughly force point-to-point predictions across different augmented versions of inputs close to each other. While this paper introduces a carefully-designed approach for learning class agnostic and specific consistency, based on the teacher–student framework. The proposed class-agnostic consistency learning, to bring the features of student and teacher models closer together, enhances the model robustness by replacing the traditional point-to-point prediction consistency with the group-to-group consistency based on the perturbed local neighboring points’ features. Furthermore, to facilitate learning under class-wise supervisions, we propose a class-specific consistency learning method, pulling the feature of the unlabeled point towards its corresponding class-specific memory bank feature. Such a class of the unlabeled point is determined as the one with the highest probability predicted by the classifier. Extensive experimental results demonstrate that our proposed method surpasses the SOTA method SQN (Huet al., 2022) by 2.5% and 8.3% on S3DIS dataset, and 4.4% and 13.9% on ScanNetV2 dataset, on the 0.1% and 0.01% settings, respectively. Code is available at https://github.com/jasonwjw/CASC.