Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3523232
Byoungkwon Yoon;Seokhyun Hong;Dongjun Lee
We propose a novel cross-source point cloud registration (CSPR) method for USV-AAV cooperation in lentic environments. In the wild outdoors, which is the typical working domain of the USV-AAV team, CSPR faces significant challenges due to platform-domain problems (complex unstructured surroundings and viewing angle difference) in addition to sensor-domain problems (varying density, noise pattern, and scale). These characteristics make large discrepancies in local geometry, causing existing CSPR methods that rely on point-to-point correspondence based on local geometry around key points (e.g. surface normal, shape function, angle) to struggle. To address this challenge, we propose the novel concept of a directional correspondence-based iterative cross-source point cloud registration algorithm. Instead of using point-to-point correspondence under large discrepancies in local geometry, we build correspondence about directions to enable robust registration in the wild outdoors. Also, since the proposed directional correspondence uses bearing angle and normalized coordinate, we can separate scale estimation with transformation, effectively resolving the problem of different scales between two point clouds. Our algorithm outperforms the state-of-the-art methods, achieving an average error of $1.60^circ$ for rotation and 1.83% for translation. Additionally, we demonstrated a USV-AAV team operation with enhanced visual information achieved with the proposed method.
{"title":"Directional Correspondence Based Cross-Source Point Cloud Registration for USV-AAV Cooperation in Lentic Environments","authors":"Byoungkwon Yoon;Seokhyun Hong;Dongjun Lee","doi":"10.1109/LRA.2024.3523232","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523232","url":null,"abstract":"We propose a novel cross-source point cloud registration (CSPR) method for USV-AAV cooperation in lentic environments. In the wild outdoors, which is the typical working domain of the USV-AAV team, CSPR faces significant challenges due to platform-domain problems (complex unstructured surroundings and viewing angle difference) in addition to sensor-domain problems (varying density, noise pattern, and scale). These characteristics make large discrepancies in local geometry, causing existing CSPR methods that rely on point-to-point correspondence based on local geometry around key points (e.g. surface normal, shape function, angle) to struggle. To address this challenge, we propose the novel concept of a directional correspondence-based iterative cross-source point cloud registration algorithm. Instead of using point-to-point correspondence under large discrepancies in local geometry, we build correspondence about directions to enable robust registration in the wild outdoors. Also, since the proposed directional correspondence uses bearing angle and normalized coordinate, we can separate scale estimation with transformation, effectively resolving the problem of different scales between two point clouds. Our algorithm outperforms the state-of-the-art methods, achieving an average error of \u0000<inline-formula><tex-math>$1.60^circ$</tex-math></inline-formula>\u0000 for rotation and 1.83% for translation. Additionally, we demonstrated a USV-AAV team operation with enhanced visual information achieved with the proposed method.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1601-1608"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10816390","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3522841
Daichi Yashima;Ryosuke Korekata;Komei Sugiura
Growing labor shortages are increasing the demand for domestic service robots (DSRs) to assist in various settings. In this study, we develop a DSR that transports everyday objects to specified pieces of furniture based on open-vocabulary instructions. Our approach focuses on retrieving images of target objects and receptacles from pre-collected images of indoor environments. For example, given an instruction “Please get the right red towel hanging on the metal towel rack and put it in the white washing machine on the left,” the DSR is expected to carry the red towel to the washing machine based on the retrieved images. This is challenging because the correct images should be retrieved from thousands of collected images, which may include many images of similar towels and appliances. To address this, we propose RelaX-Former, which learns diverse and robust representations from among positive, unlabeled positive, and negative samples. We evaluated RelaX-Former on a dataset containing real-world indoor images and human annotated instructions including complex referring expressions. The experimental results demonstrate that RelaX-Former outperformed existing baseline models across standard image retrieval metrics. Moreover, we performed physical experiments using a DSR to evaluate the performance of our approach in a zero-shot transfer setting. The experiments involved the DSR to carry objects to specific receptacles based on open-vocabulary instructions, achieving an overall success rate of 75%.
{"title":"Open-Vocabulary Mobile Manipulation Based on Double Relaxed Contrastive Learning With Dense Labeling","authors":"Daichi Yashima;Ryosuke Korekata;Komei Sugiura","doi":"10.1109/LRA.2024.3522841","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522841","url":null,"abstract":"Growing labor shortages are increasing the demand for domestic service robots (DSRs) to assist in various settings. In this study, we develop a DSR that transports everyday objects to specified pieces of furniture based on open-vocabulary instructions. Our approach focuses on retrieving images of target objects and receptacles from pre-collected images of indoor environments. For example, given an instruction “Please get the right red towel hanging on the metal towel rack and put it in the white washing machine on the left,” the DSR is expected to carry the red towel to the washing machine based on the retrieved images. This is challenging because the correct images should be retrieved from thousands of collected images, which may include many images of similar towels and appliances. To address this, we propose RelaX-Former, which learns diverse and robust representations from among positive, unlabeled positive, and negative samples. We evaluated RelaX-Former on a dataset containing real-world indoor images and human annotated instructions including complex referring expressions. The experimental results demonstrate that RelaX-Former outperformed existing baseline models across standard image retrieval metrics. Moreover, we performed physical experiments using a DSR to evaluate the performance of our approach in a zero-shot transfer setting. The experiments involved the DSR to carry objects to specific receptacles based on open-vocabulary instructions, achieving an overall success rate of 75%.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1728-1735"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3523222
Oliver Grainge;Michael Milford;Indu Bodala;Sarvapali D. Ramchurn;Shoaib Ehsan
Visual Place Recognition (VPR) is fundamental for the global re-localization of robots and devices, enabling them to recognize previously visited locations based on visual inputs. This capability is crucial for maintaining accurate mapping and localization over large areas. Given that VPR methods need to operate in real-time on embedded systems, it is critical to optimize these systems for minimal resource consumption. While the most efficient VPR approaches employ standard convolutional backbones with fixed descriptor dimensions, these often lead to redundancy in the embedding space as well as in the network architecture. Our work introduces a novel structured pruning method, to not only streamline common VPR architectures but also to strategically remove redundancies within the feature embedding space. This dual focus significantly enhances the efficiency of the system, reducing both map and model memory requirements and decreasing feature extraction and retrieval latencies. Our approach has reduced memory usage and latency by 21% and 16%, respectively, across models, while minimally impacting recall@1 accuracy by less than 1%. This significant improvement enhances real-time applications on edge devices with negligible accuracy loss.
{"title":"Structured Pruning for Efficient Visual Place Recognition","authors":"Oliver Grainge;Michael Milford;Indu Bodala;Sarvapali D. Ramchurn;Shoaib Ehsan","doi":"10.1109/LRA.2024.3523222","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523222","url":null,"abstract":"Visual Place Recognition (VPR) is fundamental for the global re-localization of robots and devices, enabling them to recognize previously visited locations based on visual inputs. This capability is crucial for maintaining accurate mapping and localization over large areas. Given that VPR methods need to operate in real-time on embedded systems, it is critical to optimize these systems for minimal resource consumption. While the most efficient VPR approaches employ standard convolutional backbones with fixed descriptor dimensions, these often lead to redundancy in the embedding space as well as in the network architecture. Our work introduces a novel structured pruning method, to not only streamline common VPR architectures but also to strategically remove redundancies within the feature embedding space. This dual focus significantly enhances the efficiency of the system, reducing both map and model memory requirements and decreasing feature extraction and retrieval latencies. Our approach has reduced memory usage and latency by 21% and 16%, respectively, across models, while minimally impacting recall@1 accuracy by less than 1%. This significant improvement enhances real-time applications on edge devices with negligible accuracy loss.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"2024-2031"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142993064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Simulating soft robots offers a cost-effective approach to exploring their design and control strategies. While current models, such as finite element analysis, are effective in capturing soft robotic dynamics, the field still requires a broadly applicable and efficient numerical simulation method. In this letter, we introduce a discrete differential geometry-based framework for the model-based inverse design of a novel snap-actuated jumping robot. Our findings reveal that the snapping beam actuator exhibits both symmetric and asymmetric dynamic modes, enabling tunable robot trajectories (e.g., horizontal or vertical jumps). Leveraging this bistable beam as a robotic actuator, we propose a physics-data hybrid inverse design strategy to endow the snap-jump robot with a diverse range of jumping capabilities. By utilizing a physical engine to examine the effects of design parameters on jump dynamics, we then use extensive simulation data to establish a data-driven inverse design solution. This approach allows rapid exploration of parameter spaces to achieve targeted jump trajectories, providing a robust foundation for the robot's fabrication. Our methodology offers a powerful framework for advancing the design and control of soft robots through integrated simulation and data-driven techniques.
{"title":"Inverse Design of Snap-Actuated Jumping Robots Powered by Mechanics-Aided Machine Learning","authors":"Dezhong Tong;Zhuonan Hao;Mingchao Liu;Weicheng Huang","doi":"10.1109/LRA.2024.3523218","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523218","url":null,"abstract":"Simulating soft robots offers a cost-effective approach to exploring their design and control strategies. While current models, such as finite element analysis, are effective in capturing soft robotic dynamics, the field still requires a broadly applicable and efficient numerical simulation method. In this letter, we introduce a discrete differential geometry-based framework for the model-based inverse design of a novel snap-actuated jumping robot. Our findings reveal that the snapping beam actuator exhibits both symmetric and asymmetric dynamic modes, enabling tunable robot trajectories (e.g., horizontal or vertical jumps). Leveraging this bistable beam as a robotic actuator, we propose a physics-data hybrid inverse design strategy to endow the snap-jump robot with a diverse range of jumping capabilities. By utilizing a physical engine to examine the effects of design parameters on jump dynamics, we then use extensive simulation data to establish a data-driven inverse design solution. This approach allows rapid exploration of parameter spaces to achieve targeted jump trajectories, providing a robust foundation for the robot's fabrication. Our methodology offers a powerful framework for advancing the design and control of soft robots through integrated simulation and data-driven techniques.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1720-1727"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Unlike a standard camera that relies on exposure to obtain output frame by frame, an event camera only outputs an event when the change of brightness intensity in a pixel exceeds a threshold, and the outputs of different pixels are independent to each other. Benefited from its bio-inspired design, event camera has the advantages of low latency and high dynamic range. The researches on multi-sensor fusion with event camera are few so far. In this paper, we propose FAST-LIEO, a framework for fast and real-time LiDAR-inertial-event odometry. The framework tightly fuses LiDAR and event camera measurements without any feature extraction or matching. Besides, our system supports both LIEO and LIEVO (extended with RGB camera fusion). We design a novel EIO subsystem for LiDAR-event fusion. The EIO subsystem maintains a semi-dense event map and estimates the state by aligning the event representation to map. The semi-dense event map is built from LiDAR points by utilizing the edge information and temporal information provided by event representations. Besides testing our method on public benchmark dataset, we also collected real-world data by utilizing our sensor suite and conducted experiments on our self-captured dataset. The experiment results show the high robustness and accuracy of our method in challenging conditions with high real-time ability. To the best of our knowledge, our FAST-LIEO is the first system that can tightly fuse LiDAR, IMU, event camera and standard camera measurements in simultaneously localization and mapping.
{"title":"FAST-LIEO: Fast and Real-Time LiDAR-Inertial-Event-Visual Odometry","authors":"Zirui Wang;Yangtao Ge;Kewei Dong;I-Ming Chen;Jing Wu","doi":"10.1109/LRA.2024.3522843","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522843","url":null,"abstract":"Unlike a standard camera that relies on exposure to obtain output frame by frame, an event camera only outputs an event when the change of brightness intensity in a pixel exceeds a threshold, and the outputs of different pixels are independent to each other. Benefited from its bio-inspired design, event camera has the advantages of low latency and high dynamic range. The researches on multi-sensor fusion with event camera are few so far. In this paper, we propose FAST-LIEO, a framework for fast and real-time LiDAR-inertial-event odometry. The framework tightly fuses LiDAR and event camera measurements without any feature extraction or matching. Besides, our system supports both LIEO and LIEVO (extended with RGB camera fusion). We design a novel EIO subsystem for LiDAR-event fusion. The EIO subsystem maintains a semi-dense event map and estimates the state by aligning the event representation to map. The semi-dense event map is built from LiDAR points by utilizing the edge information and temporal information provided by event representations. Besides testing our method on public benchmark dataset, we also collected real-world data by utilizing our sensor suite and conducted experiments on our self-captured dataset. The experiment results show the high robustness and accuracy of our method in challenging conditions with high real-time ability. To the best of our knowledge, our FAST-LIEO is the first system that can tightly fuse LiDAR, IMU, event camera and standard camera measurements in simultaneously localization and mapping.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1680-1687"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975981","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3522769
Qinrui Liu;Biao Luo;Dongbo Zhang;Renjie Chen
Embodied AI has made prominent advances in interactive visual navigation tasks based on deep reinforcement learning. In the pursuit of higher success rates in navigation, previous work has typically focused on training embodied agents to push away interactable objects on the ground. However, such interactive visual navigation largely ignores the cost of interacting with the environment and interactions are sometimes counterproductive (e.g., push the obstacle but block the existing path). Considering these scenarios, we develop a efficient interactive visual navigation method. We propose Local Accessibility Prediction (LAP) Module to enable the agent to learn thinking about how the upcoming action will affect the environment and the navigation task before making a decision. Besides, we introduce the interaction penalty term to represent the cost of interacting with the environment. And different interaction penalties are imposed depending on the size of the obstacle pushed away. We introduce the average number of interactions as a new evaluation metric. Also, a two-stage training pipeline is employed to reach better learning performance. Our experiments in AI2-THOR environment show that our method outperforms the baseline in all evaluation metrics, achieving significant improvements in navigation performance.
{"title":"Thinking Before Decision: Efficient Interactive Visual Navigation Based on Local Accessibility Prediction","authors":"Qinrui Liu;Biao Luo;Dongbo Zhang;Renjie Chen","doi":"10.1109/LRA.2024.3522769","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522769","url":null,"abstract":"Embodied AI has made prominent advances in interactive visual navigation tasks based on deep reinforcement learning. In the pursuit of higher success rates in navigation, previous work has typically focused on training embodied agents to push away interactable objects on the ground. However, such interactive visual navigation largely ignores the cost of interacting with the environment and interactions are sometimes counterproductive (e.g., push the obstacle but block the existing path). Considering these scenarios, we develop a efficient interactive visual navigation method. We propose Local Accessibility Prediction (LAP) Module to enable the agent to learn thinking about how the upcoming action will affect the environment and the navigation task before making a decision. Besides, we introduce the interaction penalty term to represent the cost of interacting with the environment. And different interaction penalties are imposed depending on the size of the obstacle pushed away. We introduce the average number of interactions as a new evaluation metric. Also, a two-stage training pipeline is employed to reach better learning performance. Our experiments in AI2-THOR environment show that our method outperforms the baseline in all evaluation metrics, achieving significant improvements in navigation performance.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1688-1695"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Self-driving industries usually employ professional artists to build exquisite 3D cars. However, it is expensive to craft large-scale digital assets. Since there are already numerous datasets available that contain a vast number of images of cars, we focus on reconstructing high-quality 3D car models from these datasets. However, these datasets only contain one side of cars in the forward-moving scene. We try to use the existing generative models to provide more supervision information, but they struggle to generalize well in cars since they are trained on synthetic datasets not car-specific. In addition, The reconstructed 3D car texture misaligns due to a large error in camera pose estimation when dealing with in-the-wild images. These restrictions make it challenging for previous methods to reconstruct complete 3D cars. To address these problems, we propose a novel method, named DreamCar, which can reconstruct high-quality 3D cars given a few images even a single image. To generalize the generative model, we collect a car dataset, named Car360, with over 5,600 vehicles. With this dataset, we make the generative model more robust to cars. We use this generative prior specific to the car to guide its reconstruction via Score Distillation Sampling. To further complement the supervision information, we utilize the geometric and appearance symmetry of cars. Finally, we propose a pose optimization method that rectifies poses to tackle texture misalignment. Extensive experiments demonstrate that our method significantly outperforms existing methods in reconstructing high-quality 3D cars.
{"title":"DreamCar: Leveraging Car-Specific Prior for In-the-Wild 3D Car Reconstruction","authors":"Xiaobiao Du;Haiyang Sun;Ming Lu;Tianqing Zhu;Xin Yu","doi":"10.1109/LRA.2024.3523231","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523231","url":null,"abstract":"Self-driving industries usually employ professional artists to build exquisite 3D cars. However, it is expensive to craft large-scale digital assets. Since there are already numerous datasets available that contain a vast number of images of cars, we focus on reconstructing high-quality 3D car models from these datasets. However, these datasets only contain one side of cars in the forward-moving scene. We try to use the existing generative models to provide more supervision information, but they struggle to generalize well in cars since they are trained on synthetic datasets not car-specific. In addition, The reconstructed 3D car texture misaligns due to a large error in camera pose estimation when dealing with in-the-wild images. These restrictions make it challenging for previous methods to reconstruct complete 3D cars. To address these problems, we propose a novel method, named DreamCar, which can reconstruct high-quality 3D cars given a few images even a single image. To generalize the generative model, we collect a car dataset, named Car360, with over 5,600 vehicles. With this dataset, we make the generative model more robust to cars. We use this generative prior specific to the car to guide its reconstruction via Score Distillation Sampling. To further complement the supervision information, we utilize the geometric and appearance symmetry of cars. Finally, we propose a pose optimization method that rectifies poses to tackle texture misalignment. Extensive experiments demonstrate that our method significantly outperforms existing methods in reconstructing high-quality 3D cars.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1840-1847"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3522760
Kevin Haninger;Luka Peternel
For successful goal-directed human-robot interaction, the robot should adapt to the intentions and actions of the collaborating human. This can be supported by musculoskeletal or data-driven human models, where the former are limited to lower-level functioning such as ergonomics, and the latter have limited generalizability or data efficiency. What is missing, is the inclusion of human motor control models that can provide generalizable human behavior estimates and integrate into robot planning methods. We use well-studied models from human motor control based on the speed-accuracy and cost-benefit trade-offs to plan collaborative robot motions. In these models, the human trajectory minimizes an objective function, a formulation we adapt to numerical trajectory optimization. This can then be extended with constraints and new variables to realize collaborative motion planning and goal estimation. We deploy this model, as well as a multi-component movement strategy, in physical collaboration with uncertain goal-reaching and synchronized motion tasks, showing the ability of the approach to produce human-like trajectories over a range of conditions.
{"title":"Planning Human-Robot Co-Manipulation With Human Motor Control Objectives and Multi-Component Reaching Strategies","authors":"Kevin Haninger;Luka Peternel","doi":"10.1109/LRA.2024.3522760","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522760","url":null,"abstract":"For successful goal-directed human-robot interaction, the robot should adapt to the intentions and actions of the collaborating human. This can be supported by musculoskeletal or data-driven human models, where the former are limited to lower-level functioning such as ergonomics, and the latter have limited generalizability or data efficiency. What is missing, is the inclusion of human motor control models that can provide generalizable human behavior estimates and integrate into robot planning methods. We use well-studied models from human motor control based on the speed-accuracy and cost-benefit trade-offs to plan collaborative robot motions. In these models, the human trajectory minimizes an objective function, a formulation we adapt to numerical trajectory optimization. This can then be extended with constraints and new variables to realize collaborative motion planning and goal estimation. We deploy this model, as well as a multi-component movement strategy, in physical collaboration with uncertain goal-reaching and synchronized motion tasks, showing the ability of the approach to produce human-like trajectories over a range of conditions.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1433-1440"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142918352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3523228
Weixin Ma;Huan Yin;Patricia J. Y. Wong;Danwei Wang;Yuxiang Sun;Zhongqing Su
This study presents a system, TripletLoc, for fast and robust global registration of a single LiDAR scan to a large-scale reference map. In contrast to conventional methods using place recognition and point cloud registration, TripletLoc directly generates correspondences on lightweight semantics, which is close to how humans perceive the world. Specifically, TripletLoc first respectively extracts instances from the single query scan and the large-scale reference map to construct two semantic graphs. Then, a novel semantic triplet-based histogram descriptor is designed to achieve instance-level matching between the query scan and the reference map. Graph-theoretic outlier pruning is leveraged to obtain inlier correspondences from raw instance-to-instance correspondences for robust 6-DoF pose estimation. In addition, a novel Road Surface Normal (RSN) map is proposed to provide a prior rotation constraint to further enhance pose estimation. We evaluate TripletLoc extensively on a large-scale public dataset, HeliPR, which covers diverse and complex scenarios in urban environments. Experimental results demonstrate that TripletLoc could achieve fast and robust global localization under diverse and challenging environments, with high memory efficiency.
{"title":"TripletLoc: One-Shot Global Localization Using Semantic Triplet in Urban Environments","authors":"Weixin Ma;Huan Yin;Patricia J. Y. Wong;Danwei Wang;Yuxiang Sun;Zhongqing Su","doi":"10.1109/LRA.2024.3523228","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523228","url":null,"abstract":"This study presents a system, TripletLoc, for fast and robust global registration of a single LiDAR scan to a large-scale reference map. In contrast to conventional methods using place recognition and point cloud registration, TripletLoc directly generates correspondences on lightweight semantics, which is close to how humans perceive the world. Specifically, TripletLoc first respectively extracts instances from the single query scan and the large-scale reference map to construct two semantic graphs. Then, a novel semantic triplet-based histogram descriptor is designed to achieve instance-level matching between the query scan and the reference map. Graph-theoretic outlier pruning is leveraged to obtain inlier correspondences from raw instance-to-instance correspondences for robust 6-DoF pose estimation. In addition, a novel Road Surface Normal (RSN) map is proposed to provide a prior rotation constraint to further enhance pose estimation. We evaluate TripletLoc extensively on a large-scale public dataset, HeliPR, which covers diverse and complex scenarios in urban environments. Experimental results demonstrate that TripletLoc could achieve fast and robust global localization under diverse and challenging environments, with high memory efficiency.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1569-1576"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142937836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Large-scale 3D reconstruction is critical in the field of robotics, and the potential of 3D Gaussian Splatting (3DGS) for achieving accurate object-level reconstruction has been demonstrated. However, ensuring geometric accuracy in outdoor and unbounded scenes remains a significant challenge. This study introduces LI-GS, a reconstruction system that incorporates LiDAR and Gaussian Splatting to enhance geometric accuracy in large-scale scenes. 2D Gaussain surfels are employed as the map representation to enhance surface alignment. Additionally, a novel modeling method is proposed to convert LiDAR point clouds to plane-constrained multimodal Gaussian Mixture Models (GMMs). The GMMs are utilized during both initialization and optimization stages to ensure sufficient and continuous supervision over the entire scene while mitigating the risk of over-fitting. Furthermore, GMMs are employed in mesh extraction to eliminate artifacts and improve the overall geometric quality. Experiments demonstrate that our method outperforms state-of-the-art methods in large-scale 3D reconstruction, achieving higher accuracy compared to both LiDAR-based methods and Gaussian-based methods with improvements of 52.6% and 68.7%, respectively.
{"title":"LI-GS: Gaussian Splatting With LiDAR Incorporated for Accurate Large-Scale Reconstruction","authors":"Changjian Jiang;Ruilan Gao;Kele Shao;Yue Wang;Rong Xiong;Yu Zhang","doi":"10.1109/LRA.2024.3522846","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522846","url":null,"abstract":"Large-scale 3D reconstruction is critical in the field of robotics, and the potential of 3D Gaussian Splatting (3DGS) for achieving accurate object-level reconstruction has been demonstrated. However, ensuring geometric accuracy in outdoor and unbounded scenes remains a significant challenge. This study introduces LI-GS, a reconstruction system that incorporates LiDAR and Gaussian Splatting to enhance geometric accuracy in large-scale scenes. 2D Gaussain surfels are employed as the map representation to enhance surface alignment. Additionally, a novel modeling method is proposed to convert LiDAR point clouds to plane-constrained multimodal Gaussian Mixture Models (GMMs). The GMMs are utilized during both initialization and optimization stages to ensure sufficient and continuous supervision over the entire scene while mitigating the risk of over-fitting. Furthermore, GMMs are employed in mesh extraction to eliminate artifacts and improve the overall geometric quality. Experiments demonstrate that our method outperforms state-of-the-art methods in large-scale 3D reconstruction, achieving higher accuracy compared to both LiDAR-based methods and Gaussian-based methods with improvements of 52.6% and 68.7%, respectively.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1864-1871"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975982","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：