Autonomous robot navigation in off–road environments requires a comprehensive understanding of the terrain geometry and traversability. The degraded perceptual conditions and sparse geometric information at longer ranges make the problem challenging especially when driving at high speeds. Furthermore, the sensing–to–mapping latency and the look–ahead map range can limit the maximum speed of the vehicle. Building on top of the recent work RoadRunner, in this work, we address the challenge of long-range ( $pm 100 ,text{m}$ ) traversability estimation. Our RoadRunner (M&M) is an end-to-end learning-based framework that directly predicts the traversability and elevation maps at multiple ranges ( $pm 50 ,text{m}$ , $pm 100 ,text{m}$ ) and resolutions ( $0.2 ,text{m}$ , $0.8 ,text{m}$ ) taking as input multiple images and a LiDAR voxel map. Our method is trained in a self–supervised manner by leveraging the dense supervision signal generated by fusing predictions from an existing traversability estimation stack (X-Racer) in hindsight and satellite Digital Elevation Maps. RoadRunner M&M achieves a significant improvement of up to 50% for elevation mapping and 30% for traversability estimation over RoadRunner, and is able to predict in 30% more regions compared to X-Racer while achieving real–time performance. Experiments on various out–of–distribution datasets also demonstrate that our data-driven approach starts to generalize to novel unstructured environments. We integrate our proposed framework in closed–loop with the path planner to demonstrate autonomous high–speed off–road robotic navigation in challenging real–world environments.
{"title":"RoadRunner M&M - Learning Multi-Range Multi-Resolution Traversability Maps for Autonomous Off-Road Navigation","authors":"Manthan Patel;Jonas Frey;Deegan Atha;Patrick Spieler;Marco Hutter;Shehryar Khattak","doi":"10.1109/LRA.2024.3490404","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490404","url":null,"abstract":"Autonomous robot navigation in off–road environments requires a comprehensive understanding of the terrain geometry and traversability. The degraded perceptual conditions and sparse geometric information at longer ranges make the problem challenging especially when driving at high speeds. Furthermore, the sensing–to–mapping latency and the look–ahead map range can limit the maximum speed of the vehicle. Building on top of the recent work RoadRunner, in this work, we address the challenge of long-range (\u0000<inline-formula><tex-math>$pm 100 ,text{m}$</tex-math></inline-formula>\u0000) traversability estimation. Our RoadRunner (M&M) is an end-to-end learning-based framework that directly predicts the traversability and elevation maps at multiple ranges (\u0000<inline-formula><tex-math>$pm 50 ,text{m}$</tex-math></inline-formula>\u0000, \u0000<inline-formula><tex-math>$pm 100 ,text{m}$</tex-math></inline-formula>\u0000) and resolutions (\u0000<inline-formula><tex-math>$0.2 ,text{m}$</tex-math></inline-formula>\u0000, \u0000<inline-formula><tex-math>$0.8 ,text{m}$</tex-math></inline-formula>\u0000) taking as input multiple images and a LiDAR voxel map. Our method is trained in a self–supervised manner by leveraging the dense supervision signal generated by fusing predictions from an existing traversability estimation stack (X-Racer) in hindsight and satellite Digital Elevation Maps. RoadRunner M&M achieves a significant improvement of up to 50% for elevation mapping and 30% for traversability estimation over RoadRunner, and is able to predict in 30% more regions compared to X-Racer while achieving real–time performance. Experiments on various out–of–distribution datasets also demonstrate that our data-driven approach starts to generalize to novel unstructured environments. We integrate our proposed framework in closed–loop with the path planner to demonstrate autonomous high–speed off–road robotic navigation in challenging real–world environments.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11425-11432"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490393
Yanming Shao;Chenxi Xiao
Humans naturally perform bimanual skills to handle large and heavy objects. To enhance robots' object manipulation capabilities, generating effective bimanual grasp poses is essential. Nevertheless, bimanual grasp synthesis for dexterous hand manipulators remains underexplored. To bridge this gap, we propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects. The BimanGrasp algorithm generates grasp poses by optimizing an energy function that considers grasp stability and feasibility. Furthermore, the synthesized grasps are verified using the Isaac Gym physics simulation engine. These verified grasp poses form the BimanGrasp-Dataset, the first large-scale synthesized bimanual dexterous hand grasp pose dataset to our knowledge. The dataset comprises over 150k verified grasps on 900 objects, facilitating the synthesis of bimanual grasps through a data-driven approach. Last, we propose BimanGrasp-DDPM, a diffusion model trained on the BimanGrasp-Dataset. This model achieved a grasp synthesis success rate of 69.87% and significant acceleration in computational speed compared to BimanGrasp algorithm.
{"title":"Bimanual Grasp Synthesis for Dexterous Robot Hands","authors":"Yanming Shao;Chenxi Xiao","doi":"10.1109/LRA.2024.3490393","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490393","url":null,"abstract":"Humans naturally perform bimanual skills to handle large and heavy objects. To enhance robots' object manipulation capabilities, generating effective bimanual grasp poses is essential. Nevertheless, bimanual grasp synthesis for dexterous hand manipulators remains underexplored. To bridge this gap, we propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects. The BimanGrasp algorithm generates grasp poses by optimizing an energy function that considers grasp stability and feasibility. Furthermore, the synthesized grasps are verified using the Isaac Gym physics simulation engine. These verified grasp poses form the BimanGrasp-Dataset, the first large-scale synthesized bimanual dexterous hand grasp pose dataset to our knowledge. The dataset comprises over 150k verified grasps on 900 objects, facilitating the synthesis of bimanual grasps through a data-driven approach. Last, we propose BimanGrasp-DDPM, a diffusion model trained on the BimanGrasp-Dataset. This model achieved a grasp synthesis success rate of 69.87% and significant acceleration in computational speed compared to BimanGrasp algorithm.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11377-11384"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636537","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490407
Yuzhu Li;Wei Dong
Current formations commonly rely on invariant hierarchical structures, such as predetermined leaders or enumerated formation shapes. These structures could be unidirectional and sluggish, constraining their flexibility and agility when encountering cluttered environments. To surmount these constraints, this work proposes a dynamic hierarchical reorganization approach with affine formation. Central to our approach is the fluid leadership and authority redistribution, for which we develop a minimum time-driven leadership evaluation algorithm and a power transition control algorithm. These algorithms facilitate autonomous leader selection and ensure smooth power transitions, enabling the swarm to adapt hierarchically in alignment with the external environment. Extensive simulations and real-world experiments validate the effectiveness of the proposed method. The formation of five aerial robots successfully performs dynamic hierarchical reorganizations, enabling the execution of complex tasks such as swerving maneuvers and navigating through hoops at velocities of up to 1.05m/s. Comparative experimental results further demonstrate the significant advantages of hierarchical reorganization in enhancing formation flexibility and agility, particularly during complex maneuvers such as U-turns. Notably, in the aforementioned real-world experiments, the proposed method reduces the flight path length by at least 33.8% compared to formations without hierarchical reorganization.
目前的编队通常依赖于不变的层次结构，如预先确定的领导或列举的编队形状。这些结构可能是单向和迟缓的，在遇到杂乱环境时限制了它们的灵活性和敏捷性。为了克服这些限制，本研究提出了一种仿射编队的动态分层重组方法。我们的方法的核心是流畅的领导力和权力再分配，为此我们开发了一种由最短时间驱动的领导力评估算法和一种权力转换控制算法。这些算法有利于自主选择领导者，确保权力平稳过渡，使蜂群能够根据外部环境进行分层调整。大量的模拟和实际实验验证了所提方法的有效性。由五个空中机器人组成的编队成功地进行了动态分层重组，从而能够执行复杂的任务，如以高达 1.05 米/秒的速度进行转弯机动和环形导航。对比实验结果进一步证明了分层重组在提高编队灵活性和敏捷性方面的显著优势，尤其是在 U 形转弯等复杂机动过程中。值得注意的是，在上述真实世界的实验中，与没有进行分层重组的编队相比，所提出的方法至少减少了 33.8% 的飞行路径长度。
{"title":"Flexible Affine Formation Control Based on Dynamic Hierarchical Reorganization","authors":"Yuzhu Li;Wei Dong","doi":"10.1109/LRA.2024.3490407","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490407","url":null,"abstract":"Current formations commonly rely on invariant hierarchical structures, such as predetermined leaders or enumerated formation shapes. These structures could be unidirectional and sluggish, constraining their flexibility and agility when encountering cluttered environments. To surmount these constraints, this work proposes a dynamic hierarchical reorganization approach with affine formation. Central to our approach is the fluid leadership and authority redistribution, for which we develop a minimum time-driven leadership evaluation algorithm and a power transition control algorithm. These algorithms facilitate autonomous leader selection and ensure smooth power transitions, enabling the swarm to adapt hierarchically in alignment with the external environment. Extensive simulations and real-world experiments validate the effectiveness of the proposed method. The formation of five aerial robots successfully performs dynamic hierarchical reorganizations, enabling the execution of complex tasks such as swerving maneuvers and navigating through hoops at velocities of up to 1.05m/s. Comparative experimental results further demonstrate the significant advantages of hierarchical reorganization in enhancing formation flexibility and agility, particularly during complex maneuvers such as U-turns. Notably, in the aforementioned real-world experiments, the proposed method reduces the flight path length by at least 33.8% compared to formations without hierarchical reorganization.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11290-11297"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142598645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490377
Dongkun Zhang;Jiaming Liang;Sha Lu;Ke Guo;Qi Wang;Rong Xiong;Zhenwei Miao;Yue Wang
Autonomous driving demands proficient trajectory planning to ensure safety and comfort. This letter introduces Policy-Embedded Planner (PEP), a novel framework that enhances closed-loop performance of imitation learning (IL) based planners by embedding a neural policy for sequential ego pose generation, leveraging predicted trajectories of traffic agents. PEP addresses the challenges of distribution shift and causal confusion by decomposing multi-step planning into single-step policy rollouts, applying a coordinate transformation technique to simplify training. PEP allows for the parallel generation of multi-modal candidate trajectories and incorporates both neural and rule-based scoring functions for trajectory selection. To mitigate the negative effects of prediction error on closed-loop performance, we propose an information-mixing mechanism that alternates the utilization of traffic agents' predicted and ground-truth information during training. Experimental validations on nuPlan benchmark highlight PEP's superiority over IL- and rule-based state-of-the-art methods.
自动驾驶需要熟练的轨迹规划，以确保安全性和舒适性。这篇文章介绍了政策嵌入式规划器（PEP），这是一种新颖的框架，通过嵌入神经政策，利用交通参与者的预测轨迹，按顺序生成自我姿态，从而提高基于模仿学习（IL）的规划器的闭环性能。PEP 将多步规划分解为单步策略滚动，应用坐标变换技术简化训练，从而解决了分布偏移和因果混淆的难题。PEP 允许并行生成多模式候选轨迹，并结合神经和基于规则的评分函数进行轨迹选择。为了减轻预测误差对闭环性能的负面影响，我们提出了一种信息混合机制，在训练过程中交替使用交通代理的预测信息和地面实况信息。在 nuPlan 基准上的实验验证凸显了 PEP 优于基于 IL 和规则的最先进方法。
{"title":"PEP: Policy-Embedded Trajectory Planning for Autonomous Driving","authors":"Dongkun Zhang;Jiaming Liang;Sha Lu;Ke Guo;Qi Wang;Rong Xiong;Zhenwei Miao;Yue Wang","doi":"10.1109/LRA.2024.3490377","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490377","url":null,"abstract":"Autonomous driving demands proficient trajectory planning to ensure safety and comfort. This letter introduces Policy-Embedded Planner (PEP), a novel framework that enhances closed-loop performance of imitation learning (IL) based planners by embedding a neural policy for sequential ego pose generation, leveraging predicted trajectories of traffic agents. PEP addresses the challenges of distribution shift and causal confusion by decomposing multi-step planning into single-step policy rollouts, applying a coordinate transformation technique to simplify training. PEP allows for the parallel generation of multi-modal candidate trajectories and incorporates both neural and rule-based scoring functions for trajectory selection. To mitigate the negative effects of prediction error on closed-loop performance, we propose an information-mixing mechanism that alternates the utilization of traffic agents' predicted and ground-truth information during training. Experimental validations on nuPlan benchmark highlight PEP's superiority over IL- and rule-based state-of-the-art methods.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11361-11368"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
High-Definition (HD) map construction is essential for autonomous driving to accurately understand the surrounding environment. Most existing methods rely on single-frame inputs to predict local map, which often fail to effectively capture the temporal correlations between frames. This limitation results in discontinuities and instability in the generated map.To tackle this limitation, we propose a Ti ghtly C oupled temporal fusion Map Net work (TICMapNet). TICMapNet breaks down the fusion process into three sub-problems: PV feature alignment, BEV feature adjustment, and Query feature fusion. By doing so, we effectively integrate temporal information at different stages through three plug-and-play modules, using the proposed tightly coupled strategy. Unlike traditional methods, our approach does not rely on camera extrinsic parameters, offering a new perspective for addressing the visual fusion challenge in the field of object detection. Experimental results show that TICMapNet significantly improves upon its single-frame baseline model, achieving at least a 7.0% increase in mAP using just two consecutive frames on the nuScenes dataset, while also showing generalizability across other tasks.
{"title":"TICMapNet: A Tightly Coupled Temporal Fusion Pipeline for Vectorized HD Map Learning","authors":"Wenzhao Qiu;Shanmin Pang;Hao Zhang;Jianwu Fang;Jianru Xue","doi":"10.1109/LRA.2024.3490384","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490384","url":null,"abstract":"High-Definition (HD) map construction is essential for autonomous driving to accurately understand the surrounding environment. Most existing methods rely on single-frame inputs to predict local map, which often fail to effectively capture the temporal correlations between frames. This limitation results in discontinuities and instability in the generated map.To tackle this limitation, we propose a \u0000<italic>Ti</i>\u0000ghtly \u0000<italic>C</i>\u0000oupled temporal fusion \u0000<italic>Map</i>\u0000 \u0000<italic>Net</i>\u0000work (TICMapNet). TICMapNet breaks down the fusion process into three sub-problems: PV feature alignment, BEV feature adjustment, and Query feature fusion. By doing so, we effectively integrate temporal information at different stages through three plug-and-play modules, using the proposed tightly coupled strategy. Unlike traditional methods, our approach does not rely on camera extrinsic parameters, offering a new perspective for addressing the visual fusion challenge in the field of object detection. Experimental results show that TICMapNet significantly improves upon its single-frame baseline model, achieving at least a 7.0% increase in mAP using just two consecutive frames on the nuScenes dataset, while also showing generalizability across other tasks.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11289-11296"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142645394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490402
Dapeng Feng;Yuhua Qi;Shipeng Zhong;Zhiqiang Chen;Qiming Chen;Hongbo Chen;Jin Wu;Jun Ma
The burgeoning demand for collaborative robotic systems to execute complex tasks collectively has intensified the research community's focus on advancing simultaneous localization and mapping (SLAM) in a cooperative context. Despite this interest, the scalability and diversity of existing datasets for collaborative trajectories remain limited, especially in scenarios with constrained perspectives where the generalization capabilities of Collaborative SLAM (C-SLAM) are critical for the feasibility of multi-agent missions. Addressing this gap, we introduce S3E, an expansive multimodal dataset. Captured by a fleet of unmanned ground vehicles traversing four distinct collaborative trajectory paradigms, S3E encompasses 13 outdoor and 5 indoor sequences. These sequences feature meticulously synchronized and spatially calibrated data streams, including 360-degree LiDAR point cloud, high-resolution stereo imagery, high-frequency inertial measurement units (IMU), and Ultra-wideband (UWB) relative observations. Our dataset not only surpasses previous efforts in scale, scene diversity, and data intricacy but also provides a thorough analysis and benchmarks for both collaborative and individual SLAM methodologies.
{"title":"S3E: A Multi-Robot Multimodal Dataset for Collaborative SLAM","authors":"Dapeng Feng;Yuhua Qi;Shipeng Zhong;Zhiqiang Chen;Qiming Chen;Hongbo Chen;Jin Wu;Jun Ma","doi":"10.1109/LRA.2024.3490402","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490402","url":null,"abstract":"The burgeoning demand for collaborative robotic systems to execute complex tasks collectively has intensified the research community's focus on advancing simultaneous localization and mapping (SLAM) in a cooperative context. Despite this interest, the scalability and diversity of existing datasets for collaborative trajectories remain limited, especially in scenarios with constrained perspectives where the generalization capabilities of Collaborative SLAM (C-SLAM) are critical for the feasibility of multi-agent missions. Addressing this gap, we introduce S3E, an expansive multimodal dataset. Captured by a fleet of unmanned ground vehicles traversing four distinct collaborative trajectory paradigms, S3E encompasses 13 outdoor and 5 indoor sequences. These sequences feature meticulously synchronized and spatially calibrated data streams, including 360-degree LiDAR point cloud, high-resolution stereo imagery, high-frequency inertial measurement units (IMU), and Ultra-wideband (UWB) relative observations. Our dataset not only surpasses previous efforts in scale, scene diversity, and data intricacy but also provides a thorough analysis and benchmarks for both collaborative and individual SLAM methodologies.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11401-11408"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490392
Jianming Ma;Zhanxiang Cao;Yue Gao
Learning-based controllers show promising performances in robotic control tasks. However, they still present potential safety risks due to the difficulty in ensuring satisfaction of complex action constraints. We propose a novel action-constrained reinforcement learning method, which transforms the constrained action space into its dual space and uses Dirichlet distribution policy to guarantee strict constraint satisfaction as well as randomized exploration. We validate the proposed method in benchmark environments and in a real quadruped locomotion task. Our method outperforms other baselines with higher reward and faster inference speed. Results of the real robot experiments demonstrate the effectiveness and potential application of our method.
{"title":"Constrained Dirichlet Distribution Policy: Guarantee Zero Constraint Violation Reinforcement Learning for Continuous Robotic Control","authors":"Jianming Ma;Zhanxiang Cao;Yue Gao","doi":"10.1109/LRA.2024.3490392","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490392","url":null,"abstract":"Learning-based controllers show promising performances in robotic control tasks. However, they still present potential safety risks due to the difficulty in ensuring satisfaction of complex action constraints. We propose a novel action-constrained reinforcement learning method, which transforms the constrained action space into its dual space and uses Dirichlet distribution policy to guarantee strict constraint satisfaction as well as randomized exploration. We validate the proposed method in benchmark environments and in a real quadruped locomotion task. Our method outperforms other baselines with higher reward and faster inference speed. Results of the real robot experiments demonstrate the effectiveness and potential application of our method.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11690-11697"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142679287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490258
Sameh I. Beaber;Zhen Liu;Ye Sun
Soft robots, formulated by soft and compliant materials, have grown significantly in recent years toward safe and adaptable operations and interactions with dynamic environments. Modeling the complex, nonlinear behaviors and controlling the deformable structures of soft robots present challenges. This study aims to establish a physics-guided deep learning (PGDL) computational framework that integrates physical models into deep learning framework as surrogate models for soft robots. Once trained, these models can replace computationally expensive numerical simulations to shorten the computation time and enable real-time control. This PGDL framework is among the first to integrate first principle physics of soft robots into deep learning toward highly accurate yet computationally affordable models for soft robot modeling and control. The proposed framework has been implemented and validated using three different pneumatic soft fingers with different behaviors and geometries, along with two training and testing approaches, to demonstrate its effectiveness and generalizability. The results showed that the mean square error (MSE) of predicted deformed curvature and the maximum and minimum deformation at various loading conditions were as low as $10^{-4}$ mm $^{2}$ . The proposed PGDL framework is constructed from first principle physics and intrinsically can be applicable to various conditions by carefully considering the governing equations, auxiliary equations, and the corresponding boundary and initial conditions.
{"title":"Physics-Guided Deep Learning Enabled Surrogate Modeling for Pneumatic Soft Robots","authors":"Sameh I. Beaber;Zhen Liu;Ye Sun","doi":"10.1109/LRA.2024.3490258","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490258","url":null,"abstract":"Soft robots, formulated by soft and compliant materials, have grown significantly in recent years toward safe and adaptable operations and interactions with dynamic environments. Modeling the complex, nonlinear behaviors and controlling the deformable structures of soft robots present challenges. This study aims to establish a physics-guided deep learning (PGDL) computational framework that integrates physical models into deep learning framework as surrogate models for soft robots. Once trained, these models can replace computationally expensive numerical simulations to shorten the computation time and enable real-time control. This PGDL framework is among the first to integrate first principle physics of soft robots into deep learning toward highly accurate yet computationally affordable models for soft robot modeling and control. The proposed framework has been implemented and validated using three different pneumatic soft fingers with different behaviors and geometries, along with two training and testing approaches, to demonstrate its effectiveness and generalizability. The results showed that the mean square error (MSE) of predicted deformed curvature and the maximum and minimum deformation at various loading conditions were as low as \u0000<inline-formula><tex-math>$10^{-4}$</tex-math></inline-formula>\u0000 mm\u0000<inline-formula><tex-math>$^{2}$</tex-math></inline-formula>\u0000. The proposed PGDL framework is constructed from first principle physics and intrinsically can be applicable to various conditions by carefully considering the governing equations, auxiliary equations, and the corresponding boundary and initial conditions.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11441-11448"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490398
Tianyu Qiu;David Fridovich-Keil
In mobile robotics and autonomous driving, it is natural to model agent interactions as the Nash equilibrium of a noncooperative, dynamic game. These methods inherently rely on observations from sensors such as lidars and cameras to identify agents participating in the game and, therefore, have difficulty when some agents are occluded. To address this limitation, this paper presents an occlusion-aware game-theoretic inference method to estimate the locations of potentially occluded agents, and simultaneously infer the intentions of both visible and occluded agents, which best accounts for the observations of visible agents. Additionally, we propose a receding horizon planning strategy based on an occlusion-aware contingency game designed to navigate in scenarios with potentially occluded agents. Monte Carlo simulations validate our approach, demonstrating that it accurately estimates the game model and trajectories for both visible and occluded agents using noisy observations of visible agents. Our planning pipeline significantly enhances navigation safety when compared to occlusion-ignorant baseline as well.
{"title":"Inferring Occluded Agent Behavior in Dynamic Games From Noise Corrupted Observations","authors":"Tianyu Qiu;David Fridovich-Keil","doi":"10.1109/LRA.2024.3490398","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490398","url":null,"abstract":"In mobile robotics and autonomous driving, it is natural to model agent interactions as the Nash equilibrium of a noncooperative, dynamic game. These methods inherently rely on observations from sensors such as lidars and cameras to identify agents participating in the game and, therefore, have difficulty when some agents are occluded. To address this limitation, this paper presents an occlusion-aware game-theoretic inference method to estimate the locations of potentially occluded agents, and simultaneously infer the intentions of both visible and occluded agents, which best accounts for the observations of visible agents. Additionally, we propose a receding horizon planning strategy based on an occlusion-aware contingency game designed to navigate in scenarios with potentially occluded agents. Monte Carlo simulations validate our approach, demonstrating that it accurately estimates the game model and trajectories for both visible and occluded agents using noisy observations of visible agents. Our planning pipeline significantly enhances navigation safety when compared to occlusion-ignorant baseline as well.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11489-11496"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490395
Mingle Zhao;Jiahao Wang;Tianxiao Gao;Chengzhong Xu;Hui Kong
Robust initialization is crucial for online systems. In the letter, a high-frequency and resilient initialization framework is designed for LiDAR-inertial systems, leveraging both inertial sensors and Doppler LiDAR. The innovative FMCW Doppler LiDAR opens up a novel avenue for robotic sensing by capturing not only point range but also Doppler velocity via the intrinsic Doppler effect. By fusing point-wise Doppler velocity with inertial measurements under non-inertial kinematics, the proposed framework, Free-Init, eliminates reliance on motion undistortion of LiDAR scans, excitation motions, and map correspondences during the initialization phase. Free-Init is also plug-and-play compatible with typical LiDAR-inertial systems and is versatile to handle a wide range of initial motions when the system starts, including stationary, dynamic, and even violent motions. The embedded Doppler-inertial velocimeter ensures fast convergence and high-frequency performance, delivering outputs exceeding 10 kHz. Comprehensive experiments on diverse platforms and across myriad motion scenes validate the framework's effectiveness. The results demonstrate the superior performance of Free-Init, highlighting the necessity of fast, resilient, and dynamic initialization for online systems.
{"title":"Free-Init: Scan-Free, Motion-Free, and Correspondence-Free Initialization for Doppler LiDAR-Inertial Systems","authors":"Mingle Zhao;Jiahao Wang;Tianxiao Gao;Chengzhong Xu;Hui Kong","doi":"10.1109/LRA.2024.3490395","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490395","url":null,"abstract":"Robust initialization is crucial for online systems. In the letter, a high-frequency and resilient initialization framework is designed for LiDAR-inertial systems, leveraging both inertial sensors and Doppler LiDAR. The innovative FMCW Doppler LiDAR opens up a novel avenue for robotic sensing by capturing not only point range but also Doppler velocity via the intrinsic Doppler effect. By fusing point-wise Doppler velocity with inertial measurements under non-inertial kinematics, the proposed framework, Free-Init, eliminates reliance on motion undistortion of LiDAR scans, excitation motions, and map correspondences during the initialization phase. Free-Init is also plug-and-play compatible with typical LiDAR-inertial systems and is versatile to handle a wide range of initial motions when the system starts, including stationary, dynamic, and even violent motions. The embedded Doppler-inertial velocimeter ensures fast convergence and high-frequency performance, delivering outputs exceeding 10 kHz. Comprehensive experiments on diverse platforms and across myriad motion scenes validate the framework's effectiveness. The results demonstrate the superior performance of Free-Init, highlighting the necessity of fast, resilient, and dynamic initialization for online systems.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11329-11336"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：