Small but mighty: Enhancing 3D point clouds semantic segmentation with U-Next framework

Abstract

We investigate the problem of 3D point clouds semantic segmentation. Recently, a large amount of research work has focused on local feature aggregation. However, the foundational framework of semantic segmentation of 3D point clouds has been neglected, where the majority of current methods default to the U-Net framework. In this study, we present U-Next, a small but mighty framework designed specifically for point cloud semantic segmentation. The key innovation of this framework is to capture multi-scale hierarchical features. Specifically, we construct the U-Next by stacking multiple U-Net L1 sub-networks in a dense arrangement to diminish the semantic gap. Concurrently, it integrates feature maps across various scales to proficiently restore intricate fine-grained details. Additionally, a multi-level deep supervision mechanism is introduced for smoothing gradient propagation and facilitating network optimization. We conduct extensive experiments on benchmarks, including the indoor S3DIS dataset, the LiDAR-based outdoor Toronto3D dataset, and the urban-scale photogrammetry-based SensatUrban dataset, demonstrate the superiority of U-Next. The U-Next framework consistently exhibits significant performance enhancements across various benchmarks and baselines, demonstrating its considerable potential as a versatile point-based framework for future endeavors. The code has been released at https://github.com/zeng-ziyin/U-Next.