Point Class-Adaptive Transformer (PCaT): A Novel Approach for Efficient Point Cloud Classification and Segmentation

Abstract

Recent 3D point cloud classification has predominantly focused on local spatial attention, neglecting distant contextual relationships due to the inherent sparsity of LiDAR-generated data over longer distances. Existing 3D object detection methods prioritize local features, hindering the extraction of semantic information. Despite attempts with transformers, methods often reduce computations through local spatial attention, neglecting content class and scarcely establishing connections among distant global points. Our proposed point class-adaptive transformer (PCaT) addresses these limitations by establishing long-range feature dependencies while significantly reducing computations. PCaT includes three key modules: the class-adaptive transformer (CaT), which utilizes local self-attention and global self-attention based on class similarity to facilitate an efficient trade-off between capturing extended-global dependencies and managing computational challenges; nested binary clustering (NbC), which dynamically partitions queries into multiple clusters based on content features in each Transformer block; and the AfA, which aggregates high-dimensional features using max-pooling alongside a residual MLP component and low-dimensional features using average pooling and a CaT block. Additionally, PCaT incorporates point cloud segmentation via local–global feature aggregation (PcSeg) to facilitate effective point cloud segmentation. Extensive experimentation on the ModelNet40, ScanObjectNN, and S3DIS datasets demonstrates the superior performance and reasonable stability of PCaT compared with existing methods. PCaT achieves 94.2% overall accuracy (OA) and mIoU scores of 89.2% and 86.2% for the ScanObjectNN and S3DIS datasets, respectively.