SAPFormer: Shape-aware propagation Transformer for point clouds

Abstract

Transformer-based networks have achieved impressive performance on three-dimensional point cloud data. However, most existing methods focus on aggregating local features in the neighborhoods of a point cloud, ignoring the global feature information. Therefore, it is difficult to capture the long-range dependencies of a point cloud. In this paper, we propose the Shape-Aware Propagation Transformer (SAPFormer), which flexibly captures the semantic information of point clouds in geometric space and effectively extracts the contextual geometric space information. Specifically, we first design local group self-attention (LGA) to capture the local interaction information in each region. To capture the separated local region feature relationships, we propose local group propagation (LGP) to pass the information between different regions via query points. This allows features to propagate among neighbors for more fine-grained feature information. To further enlarge the receptive field, we propose the global shape feature module (GSFM) to learn global context information through key shape points (KSPs). Finally, to solve the positional information cues between global contexts, we introduce spatial-shape relative position encoding (SS-RPE), which obtains positional relationships between points. Extensive experiments demonstrate the effectiveness and superiority of our method on the S3DIS, SensatUrban, ScanNet V2, ShapeNetPart, and ModelNet40 datasets. The code is available at https://github.com/viivan/SAPFormer-main.