Multi-view Network with Transformer for Point Cloud Semantic Segmentation

Abstract

The input of most point cloud semantic segmentation networks is the reconstructed complete point cloud, but in practical application scenarios, the vision devices often capture single frame point cloud data. In order to better adapt to the actual segmentation requirements in dynamic scenes, this paper proposes an online incremental point cloud semantic segmentation method, which inputs the existing saved point cloud and the currently captured point cloud into the network to make up for the lack of information in the single frame point cloud. The Transformer structure is added to the network to strengthen the fusion of contextual information. Triple Loss is introduced in the feature space to distinguish different types of point clouds in a fine-grained manner. The experimental results show that compared with the benchmark MCPNet model, the proposed semantic segmentation model improves mIoU by 2.8% and mAcc by 7% on the S3DIS Area5 dataset, further improving the accuracy of point cloud semantic segmentation.