The effective representation and feature extraction of 3-D scenes from sparse and unstructured point clouds pose a significant challenge in 3-D object detection. In this article, we propose TransMRE, a network that enables fully sparse multiple observation plane feature fusion using LiDAR point clouds as single-modal input. TransMRE achieves this by sparsely factorizing a 3-D voxel scene into three separate observation planes: XY, XZ, and YZ planes. In addition, we propose Observation Plane Sparse Fusion and Interaction to explore the internal relationship between different observation planes. The Transformer mechanism is employed to realize feature attention within a single observation plane and feature attention across multiple observation planes. This recursive application of attention is done during multiple observation plane projection feature aggregation to effectively model the entire 3-D scene. This approach addresses the limitation of insufficient feature representation ability under a single bird’s-eye view (BEV) constructed by extremely sparse point clouds. Furthermore, TransMRE maintains the full sparsity property of the entire network, eliminating the need to convert sparse feature maps into dense feature maps. As a result, it can be effectively applied to LiDAR point cloud data with large scanning ranges, such as Argoverse 2, while ensuring low computational complexity. Extensive experiments were conducted to evaluate the effectiveness of TransMRE, achieving 64.9 mAP and 70.4 NDS on the nuScenes detection benchmark, and 32.3 mAP on the Argoverse 2 detection benchmark. These results demonstrate that our method outperforms state-of-the-art methods.
{"title":"TransMRE: Multiple Observation Planes Representation Encoding With Fully Sparse Voxel Transformers for 3-D Object Detection","authors":"Ziming Zhu;Yu Zhu;Kezhi Zhang;Hangyu Li;Xiaofeng Ling","doi":"10.1109/TIM.2024.3480206","DOIUrl":"https://doi.org/10.1109/TIM.2024.3480206","url":null,"abstract":"The effective representation and feature extraction of 3-D scenes from sparse and unstructured point clouds pose a significant challenge in 3-D object detection. In this article, we propose TransMRE, a network that enables fully sparse multiple observation plane feature fusion using LiDAR point clouds as single-modal input. TransMRE achieves this by sparsely factorizing a 3-D voxel scene into three separate observation planes: XY, XZ, and YZ planes. In addition, we propose Observation Plane Sparse Fusion and Interaction to explore the internal relationship between different observation planes. The Transformer mechanism is employed to realize feature attention within a single observation plane and feature attention across multiple observation planes. This recursive application of attention is done during multiple observation plane projection feature aggregation to effectively model the entire 3-D scene. This approach addresses the limitation of insufficient feature representation ability under a single bird’s-eye view (BEV) constructed by extremely sparse point clouds. Furthermore, TransMRE maintains the full sparsity property of the entire network, eliminating the need to convert sparse feature maps into dense feature maps. As a result, it can be effectively applied to LiDAR point cloud data with large scanning ranges, such as Argoverse 2, while ensuring low computational complexity. Extensive experiments were conducted to evaluate the effectiveness of TransMRE, achieving 64.9 mAP and 70.4 NDS on the nuScenes detection benchmark, and 32.3 mAP on the Argoverse 2 detection benchmark. These results demonstrate that our method outperforms state-of-the-art methods.","PeriodicalId":13341,"journal":{"name":"IEEE Transactions on Instrumentation and Measurement","volume":"73 ","pages":"1-13"},"PeriodicalIF":5.6,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636362","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-04DOI: 10.1109/TIM.2024.3488141
Tangwen Yin;Hongtian Chen;Dan Huang;Hesheng Wang
Causality is an active relationship that transforms possibility into actuality, underscoring the limitation of relying on averages to address rare events. This study proposes a counterfactual covariate causal discovery mechanism on nonlinear extremal quantiles (CCCD-NEQs) to impute potential outcomes, measure unobservable causalities, and unveil hidden causal relationships in safety-critical systems. We created a multilevel statistical model called mixed-effect and causal-covariate statistical model with dynamic quantiles (MCSM-DQs), which incorporates mixed effects, causal covariates, and dynamic quantiles. Leveraging the exponential family distribution over MCSM-DQ ensures simplified parameter estimation and enhanced computation efficiency, enabling the bootstrapping prediction of counterfactual outcomes at dynamic quantiles to reveal causal relationships and mitigate confounding effects. We applied the CCCD-NEQ approach to identify the potential causal effects among aircraft configuration, decision-making capabilities, and flight safety. Results revealed previously unknown causal relationships concerning rare safety incidents that cannot be detected using conventional instrumental analytics. Our new counterfactual causal discovery mechanism provides opportunities to uncover hidden causality on nonlinear extremal quantiles, highlighting the forward design and optimization of systems for adaptability, robustness, intelligence, and safety.
{"title":"Counterfactual Covariate Causal Discovery on Nonlinear Extremal Quantiles","authors":"Tangwen Yin;Hongtian Chen;Dan Huang;Hesheng Wang","doi":"10.1109/TIM.2024.3488141","DOIUrl":"https://doi.org/10.1109/TIM.2024.3488141","url":null,"abstract":"Causality is an active relationship that transforms possibility into actuality, underscoring the limitation of relying on averages to address rare events. This study proposes a counterfactual covariate causal discovery mechanism on nonlinear extremal quantiles (CCCD-NEQs) to impute potential outcomes, measure unobservable causalities, and unveil hidden causal relationships in safety-critical systems. We created a multilevel statistical model called mixed-effect and causal-covariate statistical model with dynamic quantiles (MCSM-DQs), which incorporates mixed effects, causal covariates, and dynamic quantiles. Leveraging the exponential family distribution over MCSM-DQ ensures simplified parameter estimation and enhanced computation efficiency, enabling the bootstrapping prediction of counterfactual outcomes at dynamic quantiles to reveal causal relationships and mitigate confounding effects. We applied the CCCD-NEQ approach to identify the potential causal effects among aircraft configuration, decision-making capabilities, and flight safety. Results revealed previously unknown causal relationships concerning rare safety incidents that cannot be detected using conventional instrumental analytics. Our new counterfactual causal discovery mechanism provides opportunities to uncover hidden causality on nonlinear extremal quantiles, highlighting the forward design and optimization of systems for adaptability, robustness, intelligence, and safety.","PeriodicalId":13341,"journal":{"name":"IEEE Transactions on Instrumentation and Measurement","volume":"73 ","pages":"1-12"},"PeriodicalIF":5.6,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-04DOI: 10.1109/TIM.2024.3485462
Ze Chen;Yuan Zhang;Zifei Cao;Yongmeng Liu
As the “heart” of the aviation industry, high-performance aero-engines have always been a stumbling block restricting rapid development. Curvic couplings are widely used in the assembly of multistage aero-engine rotors. The coaxiality of the assembly significantly influences the performance and life of the aero-engine, so it is necessary to predict and optimize the assembly coaxiality. Aiming at three key problems, we propose an assembly coaxiality optimization and prediction approach. In this approach, we measure 3-D point clouds by a line-structured light array scanning measurement system and come up with a weighted iterative closest point (ICP) algorithm to perform a virtual assembly of the point cloud model to regulate the assembly precision. Ultimately, rotors with curvic couplings are used to experimentally validate the coaxiality prediction and optimization approach. According to the experimental findings, the two-/ three-stage rotors assemblies’ maximum coaxiality prediction errors under eight distinct assembly phases are 4.8 and $7.7~mu $