首页 > 最新文献

IEEE Robotics and Automation Letters最新文献

英文 中文
TRG-Planner: Traversal Risk Graph-Based Path Planning in Unstructured Environments for Safe and Efficient Navigation TRG-Planner:基于遍历风险图的非结构化环境中安全高效导航路径规划
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524912
Dongkyu Lee;I Made Aswin Nahrendra;Minho Oh;Byeongho Yu;Hyun Myung
Unstructured environments such as mountains, caves, construction sites, or disaster areas are challenging for autonomous navigation because of terrain irregularities. In particular, it is crucial to plan a path to avoid risky terrain and reach the goal quickly and safely. In this paper, we propose a method for safe and distance-efficient path planning, leveraging Traversal Risk Graph (TRG), a novel graph representation that takes into account geometric traversability of the terrain. TRG nodes represent stability and reachability of the terrain, while edges represent relative traversal risk-weighted path candidates. Additionally, TRG is constructed in a wavefront propagation manner and managed hierarchically, enabling real-time planning even in large-scale environments. Lastly, we formulate a graph optimization problem on TRG that leads the robot to navigate by prioritizing both safe and short paths. Our approach demonstrated superior safety, distance efficiency, and fast processing time compared to the conventional methods. It was also validated in several real-world experiments using a quadrupedal robot. Notably, TRG-planner contributed as the global path planner of an autonomous navigation framework for the DreamSTEP team, which won the Quadruped Robot Challenge at ICRA 2023.
山区、洞穴、建筑工地、灾区等非结构化环境由于地形的不规则性,对自主导航构成了挑战。特别是,规划一条路径以避开危险地形并快速安全地到达目标是至关重要的。在本文中,我们提出了一种安全且距离有效的路径规划方法,利用遍历风险图(TRG),一种考虑地形几何可遍历性的新型图表示。TRG节点表示地形的稳定性和可达性,而边缘表示相对遍历风险加权路径候选。此外,TRG以波前传播方式构建并分层管理,即使在大规模环境中也能实现实时规划。最后,我们在TRG上提出了一个图形优化问题,该问题引导机器人通过优先考虑安全和短路径进行导航。与传统方法相比,我们的方法具有更高的安全性、距离效率和更快的处理时间。它也在几个使用四足机器人的真实实验中得到了验证。值得一提的是,TRG-planner作为自主导航框架的全局路径规划器为DreamSTEP团队做出了贡献,该团队在ICRA 2023上赢得了四足机器人挑战赛。
{"title":"TRG-Planner: Traversal Risk Graph-Based Path Planning in Unstructured Environments for Safe and Efficient Navigation","authors":"Dongkyu Lee;I Made Aswin Nahrendra;Minho Oh;Byeongho Yu;Hyun Myung","doi":"10.1109/LRA.2024.3524912","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524912","url":null,"abstract":"Unstructured environments such as mountains, caves, construction sites, or disaster areas are challenging for autonomous navigation because of terrain irregularities. In particular, it is crucial to plan a path to avoid risky terrain and reach the goal quickly and safely. In this paper, we propose a method for safe and distance-efficient path planning, leveraging Traversal Risk Graph (TRG), a novel graph representation that takes into account geometric traversability of the terrain. TRG nodes represent stability and reachability of the terrain, while edges represent relative traversal risk-weighted path candidates. Additionally, TRG is constructed in a wavefront propagation manner and managed hierarchically, enabling real-time planning even in large-scale environments. Lastly, we formulate a graph optimization problem on TRG that leads the robot to navigate by prioritizing both safe and short paths. Our approach demonstrated superior safety, distance efficiency, and fast processing time compared to the conventional methods. It was also validated in several real-world experiments using a quadrupedal robot. Notably, TRG-planner contributed as the global path planner of an autonomous navigation framework for the DreamSTEP team, which won the Quadruped Robot Challenge at ICRA 2023.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1736-1743"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Sensorimotor Learning With Stability Guarantees via Autonomous Neural Dynamic Policies 基于自主神经动态策略的稳定保证感觉运动学习
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524878
Dionis Totsila;Konstantinos Chatzilygeroudis;Valerio Modugno;Denis Hadjivelichkov;Dimitrios Kanoulas
State-of-the-art sensorimotor learning algorithms, either in the context of reinforcement learning or imitation learning, offer policies that can often produce unstable behaviors, damaging the robot and/or the environment. Moreover, it is very difficult to interpret the optimized controller and analyze its behavior and/or performance. Traditional robot learning, on the contrary, relies on dynamical system-based policies that can be analyzed for stability/safety. Such policies, however, are neither flexible nor generic and usually work only with proprioceptive sensor states. In this work, we bridge the gap between generic neural network policies and dynamical system-based policies, and we introduce Autonomous Neural Dynamic Policies (ANDPs) that: (a) are based on autonomous dynamical systems, (b) always produce asymptotically stable behaviors, and (c) are more flexible than traditional stable dynamical system-based policies. ANDPs are fully differentiable, flexible generic-policies that accept any observation input, while ensuring asymptotic stability. Through several experiments, we explore the flexibility and capacity of ANDPs in several imitation learning tasks including experiments with image observations. The results show that ANDPs combine the benefits of both neural network-based and dynamical system-based methods.
最先进的感觉运动学习算法,无论是在强化学习还是模仿学习的背景下,提供的策略往往会产生不稳定的行为,破坏机器人和/或环境。此外,很难解释优化后的控制器并分析其行为和/或性能。相反,传统的机器人学习依赖于基于动态系统的策略,可以对其稳定性/安全性进行分析。然而,这种策略既不灵活也不通用,通常只适用于本体感觉传感器状态。在这项工作中,我们弥合了一般神经网络策略和基于动态系统的策略之间的差距,我们引入了自主神经动态策略(ANDPs),它:(a)基于自主动态系统,(b)总是产生渐近稳定的行为,(c)比传统的基于稳定动态系统的策略更灵活。andp是完全可微的,灵活的通用策略,可以接受任何观测输入,同时确保渐近稳定性。通过几个实验,我们探索了andp在几个模仿学习任务中的灵活性和能力,包括图像观察实验。结果表明,ANDPs结合了基于神经网络和基于动力系统方法的优点。
{"title":"Sensorimotor Learning With Stability Guarantees via Autonomous Neural Dynamic Policies","authors":"Dionis Totsila;Konstantinos Chatzilygeroudis;Valerio Modugno;Denis Hadjivelichkov;Dimitrios Kanoulas","doi":"10.1109/LRA.2024.3524878","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524878","url":null,"abstract":"State-of-the-art sensorimotor learning algorithms, either in the context of reinforcement learning or imitation learning, offer policies that can often produce unstable behaviors, damaging the robot and/or the environment. Moreover, it is very difficult to interpret the optimized controller and analyze its behavior and/or performance. Traditional robot learning, on the contrary, relies on dynamical system-based policies that can be analyzed for stability/safety. Such policies, however, are neither flexible nor generic and usually work only with proprioceptive sensor states. In this work, we bridge the gap between generic neural network policies and dynamical system-based policies, and we introduce Autonomous Neural Dynamic Policies (ANDPs) that: (a) are based on autonomous dynamical systems, (b) always produce asymptotically stable behaviors, and (c) are more flexible than traditional stable dynamical system-based policies. ANDPs are fully differentiable, flexible generic-policies that accept any observation input, while ensuring asymptotic stability. Through several experiments, we explore the flexibility and capacity of ANDPs in several imitation learning tasks including experiments with image observations. The results show that ANDPs combine the benefits of both neural network-based and dynamical system-based methods.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1760-1767"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
MetaSonic: Advancing Robot Localization With Directional Embedded Acoustic Signals metultrasonic:利用定向嵌入声信号推进机器人定位
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524903
Junling Wang;Zhenlin An;Yi Guo
Indoor positioning in environments where GPS cannot be used is a fundamental technology for robot navigation and human-robot interaction. However, existing vision-based localization systems cannot work in low-visibility environments, and existing wireless or acoustic localization systems require specific transceivers, making them expensive and power-intensive — particularly challenging for microrobots. This letter proposes a new metasurface-assisted ultrasound positioning system. The key idea is to use a low-cost passive acoustic metasurface to transfer any speaker into a directional sound source, with the acoustic spectrum varying based on direction. This allows any microrobot with a simple, low-cost microphone to capture such modified sound to identify the direction of the sound source. We develop a lightweight convolutional neural network-based localization algorithm that can be efficiently deployed on low-power microcontrollers. We evaluate our system in a large complex office. It can achieve a direction estimation accuracy of 7.26$^circ$, improving by 42.2% compared to systems without the metasurface and matching the performance of a 4-microphone array, with a localization accuracy of 0.35 m.
在无法使用GPS的环境下进行室内定位是机器人导航和人机交互的基础技术。然而,现有的基于视觉的定位系统不能在低能见度环境中工作,现有的无线或声学定位系统需要特定的收发器,这使得它们昂贵且功耗高——对微型机器人来说尤其具有挑战性。本文提出了一种新的超表面辅助超声定位系统。关键思想是使用低成本的被动声学超表面将任何扬声器转换为定向声源,声谱根据方向变化。这使得任何微型机器人都可以用一个简单、低成本的麦克风捕捉到这种经过修改的声音,从而识别声源的方向。我们开发了一种轻量级的基于卷积神经网络的定位算法,可以有效地部署在低功耗微控制器上。我们在一个大而复杂的办公室里评估我们的系统。该系统的方向估计精度为7.26$^circ$,与没有超表面的系统相比提高了42.2%,定位精度为0.35 m。
{"title":"MetaSonic: Advancing Robot Localization With Directional Embedded Acoustic Signals","authors":"Junling Wang;Zhenlin An;Yi Guo","doi":"10.1109/LRA.2024.3524903","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524903","url":null,"abstract":"Indoor positioning in environments where GPS cannot be used is a fundamental technology for robot navigation and human-robot interaction. However, existing vision-based localization systems cannot work in low-visibility environments, and existing wireless or acoustic localization systems require specific transceivers, making them expensive and power-intensive — particularly challenging for microrobots. This letter proposes a new metasurface-assisted ultrasound positioning system. The key idea is to use a low-cost passive acoustic metasurface to transfer any speaker into a directional sound source, with the acoustic spectrum varying based on direction. This allows any microrobot with a simple, low-cost microphone to capture such modified sound to identify the direction of the sound source. We develop a lightweight convolutional neural network-based localization algorithm that can be efficiently deployed on low-power microcontrollers. We evaluate our system in a large complex office. It can achieve a direction estimation accuracy of 7.26<inline-formula><tex-math>$^circ$</tex-math></inline-formula>, improving by 42.2% compared to systems without the metasurface and matching the performance of a 4-microphone array, with a localization accuracy of 0.35 m.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1704-1711"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Hybrid Tendon-Actuated and Soft Magnetic Robotic Platform for Pancreatic Applications 用于胰腺的混合肌腱驱动和软磁机器人平台
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524889
Benjamin Calmé;Adam Metcalf;Michael Brockdorff;Haneul Jang;Yoonsue Choi;Peter Lloyd;Seok Chang Ryu;Pietro Valdastri
Magnetic Soft Continuum Robots (MSCR) are used in a wide variety of surgical interventions, including neurological, pancreatic, and cardiovascular procedures. To function effectively, these MSCRs require complex programmable magnetisation. However, they often suffer from limited manoeuvrability and imprecise positioning of the devices that carry them. Tendon-Driven Continuum Robots (TDCR) have the potential to address these issues. These navigation systems not only enable higher accuracy and precision but also offer the potential for remote control, thereby reducing clinicians' exposure to ionising radiation. Currently, MSCRs are deployed from manual flexible endoscopes without motion compensation, leading to uncertainty and trial-and-error insertion. In this study, the deployment of high aspect ratio MSCRs (60 mm long by 1.3 mm diameter) from a tendon-driven robot (25 cm long with a 2.8 mm diameter) is performed. By precisely positioning the deployment point, this paper evaluates the benefits of different magnetisation profiles. The comparison is carried out for a specific clinical scenario, assessing procedure time, the distance between the external permanent magnet (used for steering) and the MSCR, and the interaction force with the tissue. Clinical relevance is demonstrated through pancreatic and bile duct cannulation in a silicon phantom.
磁性软连续体机器人(MSCR)广泛应用于各种外科手术,包括神经、胰腺和心血管手术。为了有效地工作,这些mscr需要复杂的可编程磁化。然而,他们经常遭受有限的机动性和不精确定位的设备携带他们。肌腱驱动连续体机器人(TDCR)有可能解决这些问题。这些导航系统不仅具有更高的准确度和精度,而且还提供了远程控制的可能性,从而减少了临床医生对电离辐射的暴露。目前,mscr从手动柔性内窥镜部署,没有运动补偿,导致不确定性和反复试验插入。在这项研究中,从肌腱驱动机器人(长25厘米,直径2.8毫米)上展开高纵横比mscr(长60毫米,直径1.3毫米)。通过精确定位部署点,本文评估了不同磁化剖面的效益。比较是针对一个特定的临床场景进行的,评估手术时间、外部永磁体(用于转向)与MSCR之间的距离以及与组织的相互作用力。临床相关性是通过在硅假体中的胰腺和胆管插管来证明的。
{"title":"Hybrid Tendon-Actuated and Soft Magnetic Robotic Platform for Pancreatic Applications","authors":"Benjamin Calmé;Adam Metcalf;Michael Brockdorff;Haneul Jang;Yoonsue Choi;Peter Lloyd;Seok Chang Ryu;Pietro Valdastri","doi":"10.1109/LRA.2024.3524889","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524889","url":null,"abstract":"Magnetic Soft Continuum Robots (MSCR) are used in a wide variety of surgical interventions, including neurological, pancreatic, and cardiovascular procedures. To function effectively, these MSCRs require complex programmable magnetisation. However, they often suffer from limited manoeuvrability and imprecise positioning of the devices that carry them. Tendon-Driven Continuum Robots (TDCR) have the potential to address these issues. These navigation systems not only enable higher accuracy and precision but also offer the potential for remote control, thereby reducing clinicians' exposure to ionising radiation. Currently, MSCRs are deployed from manual flexible endoscopes without motion compensation, leading to uncertainty and trial-and-error insertion. In this study, the deployment of high aspect ratio MSCRs (60 mm long by 1.3 mm diameter) from a tendon-driven robot (25 cm long with a 2.8 mm diameter) is performed. By precisely positioning the deployment point, this paper evaluates the benefits of different magnetisation profiles. The comparison is carried out for a specific clinical scenario, assessing procedure time, the distance between the external permanent magnet (used for steering) and the MSCR, and the interaction force with the tissue. Clinical relevance is demonstrated through pancreatic and bile duct cannulation in a silicon phantom.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1784-1791"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
T-ESKF: Transformed Error-State Kalman Filter for Consistent Visual-Inertial Navigation T-ESKF:一致视觉惯性导航的变换误差状态卡尔曼滤波
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524905
Chungeng Tian;Ning Hao;Fenghua He
This paper presents a novel approach to address the inconsistency problem caused by observability mismatch in visual-inertial navigation systems (VINS). The key idea involves applying a linear time-varying transformation to the error-state within the Error-State Kalman Filter (ESKF). This transformation ensures that the unobservable subspace of the transformed error-state system becomes independent of the state, thereby preserving the correct observability of the transformed system against variations in linearization points. We introduce the Transformed ESKF (T-ESKF), a consistent VINS estimator that performs state estimation using the transformed error-state system. Furthermore, we develop an efficient propagation technique to accelerate the covariance propagation based on the transformation relationship between the transition and accumulated matrices of T-ESKF and ESKF. We validate the proposed method through extensive simulations and experiments, demonstrating better (or competitive at least) performance compared to state-of-the-art methods.
提出了一种解决视觉惯性导航系统中观测值失配导致的不一致问题的新方法。关键思想涉及到在错误状态卡尔曼滤波器(ESKF)中对错误状态应用线性时变变换。这种变换保证了变换后的误差状态系统的不可观测子空间独立于状态,从而保持了变换后的系统对线性化点变化的正确可观测性。我们引入了变换ESKF (T-ESKF),一种使用变换后的误差状态系统进行状态估计的一致性VINS估计器。此外,基于T-ESKF和ESKF的转移矩阵和累积矩阵之间的转换关系,我们开发了一种有效的传播技术来加速协方差的传播。我们通过广泛的模拟和实验验证了所提出的方法,与最先进的方法相比,证明了更好的(或至少有竞争力的)性能。
{"title":"T-ESKF: Transformed Error-State Kalman Filter for Consistent Visual-Inertial Navigation","authors":"Chungeng Tian;Ning Hao;Fenghua He","doi":"10.1109/LRA.2024.3524905","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524905","url":null,"abstract":"This paper presents a novel approach to address the inconsistency problem caused by observability mismatch in visual-inertial navigation systems (VINS). The key idea involves applying a linear time-varying transformation to the error-state within the Error-State Kalman Filter (ESKF). This transformation ensures that the unobservable subspace of the transformed error-state system becomes independent of the state, thereby preserving the correct observability of the transformed system against variations in linearization points. We introduce the Transformed ESKF (T-ESKF), a consistent VINS estimator that performs state estimation using the transformed error-state system. Furthermore, we develop an efficient propagation technique to accelerate the covariance propagation based on the transformation relationship between the transition and accumulated matrices of T-ESKF and ESKF. We validate the proposed method through extensive simulations and experiments, demonstrating better (or competitive at least) performance compared to state-of-the-art methods.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1808-1815"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Modern Take on Visual Relationship Reasoning for Grasp Planning 视觉关系推理在把握规划中的现代应用
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524910
Paolo Rabino;Tatiana Tommasi
Interacting with real-world cluttered scenes poses several challenges to robotic agents that need to understand complex spatial dependencies among the observed objects to determine optimal pick sequences or efficient object retrieval strategies. Existing solutions typically manage simplified scenarios and focus on predicting pairwise object relationships following an initial object detection phase, but often overlook the global context or struggle with handling redundant and missing object relations. In this work, we present a modern take on visual relational reasoning for grasp planning. We introduce D3GD, a novel testbed that includes bin picking scenes with up to 35 objects from 97 distinct categories. Additionally, we propose D3G, a new end-to-end transformer-based dependency graph generation model that simultaneously detects objects and produces an adjacency matrix representing their spatial relationships. Recognizing the limitations of standard metrics, we employ the Average Precision of Relationships for the first time to evaluate model performance, conducting an extensive experimental benchmark. The obtained results establish our approach as the new state-of-the-art for this task, laying the foundation for future research in robotic manipulation.
与现实世界中杂乱无章的场景进行交互给机器人代理带来了诸多挑战,他们需要理解所观察到的物体之间复杂的空间依赖关系,以确定最佳的拾取序列或高效的物体检索策略。现有的解决方案通常管理简化的场景,并侧重于在初始物体检测阶段之后预测成对物体之间的关系,但往往忽略了全局背景,或在处理冗余和缺失的物体关系时举步维艰。在这项工作中,我们提出了一种用于抓取规划的现代视觉关系推理方法。我们介绍了 D3GD,这是一个新颖的测试平台,包含了来自 97 个不同类别的多达 35 个物体的垃圾箱拣选场景。此外,我们还提出了 D3G,这是一种全新的端到端基于变换器的依赖图生成模型,可同时检测物体并生成代表其空间关系的邻接矩阵。认识到标准指标的局限性,我们首次采用了 "关系平均精度"(Average Precision of Relationships)来评估模型性能,并进行了广泛的实验基准测试。实验结果表明,我们的方法在这项任务中处于最新水平,为未来的机器人操纵研究奠定了基础。
{"title":"A Modern Take on Visual Relationship Reasoning for Grasp Planning","authors":"Paolo Rabino;Tatiana Tommasi","doi":"10.1109/LRA.2024.3524910","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524910","url":null,"abstract":"Interacting with real-world cluttered scenes poses several challenges to robotic agents that need to understand complex spatial dependencies among the observed objects to determine optimal pick sequences or efficient object retrieval strategies. Existing solutions typically manage simplified scenarios and focus on predicting pairwise object relationships following an initial object detection phase, but often overlook the global context or struggle with handling redundant and missing object relations. In this work, we present a modern take on visual relational reasoning for grasp planning. We introduce D3GD, a novel testbed that includes bin picking scenes with up to 35 objects from 97 distinct categories. Additionally, we propose D3G, a new end-to-end transformer-based dependency graph generation model that simultaneously detects objects and produces an adjacency matrix representing their spatial relationships. Recognizing the limitations of standard metrics, we employ the Average Precision of Relationships for the first time to evaluate model performance, conducting an extensive experimental benchmark. The obtained results establish our approach as the new state-of-the-art for this task, laying the foundation for future research in robotic manipulation.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1712-1719"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10819650","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Variation-Robust Few-Shot 3D Affordance Segmentation for Robotic Manipulation 面向机器人操作的变鲁棒少镜头三维视觉分割
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524904
Dingchang Hu;Tianyu Sun;Pengwei Xie;Siang Chen;Huazhong Yang;Guijin Wang
Traditional affordance segmentation on 3D point cloud objects requires massive amounts of annotated training data and can only make predictions within predefined classes and affordance tasks. To overcome these limitations, we propose a variation-robust few-shot 3D affordance segmentation network (VRNet) for robotic manipulation, which requires only several affordance annotations for novel object classes and manipulation tasks. In particular, we design an orientation-tolerant feature extractor to address pose variation between support and query point cloud objects, and present a multi-scale label propagation algorithm for variation in completeness. Extensive experiments on affordance datasets show that VRNet provides the best segmentation performance compared with previous works. Moreover, experiments in real robotic scenarios demonstrate the generalization ability of our method.
传统的3D点云对象的可视性分割需要大量带注释的训练数据,并且只能在预定义的类和可视性任务中进行预测。为了克服这些限制,我们提出了一种用于机器人操作的变化鲁棒的少镜头3D可视性分割网络(VRNet),它只需要对新对象类和操作任务进行几个可视性注释。特别地,我们设计了一个方向容忍特征提取器来处理支持和查询点云对象之间的姿态变化,并提出了一种多尺度标签传播算法来处理完整性的变化。在功能数据集上的大量实验表明,VRNet提供了较好的分割性能。此外,在真实机器人场景中的实验证明了我们的方法的泛化能力。
{"title":"Variation-Robust Few-Shot 3D Affordance Segmentation for Robotic Manipulation","authors":"Dingchang Hu;Tianyu Sun;Pengwei Xie;Siang Chen;Huazhong Yang;Guijin Wang","doi":"10.1109/LRA.2024.3524904","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524904","url":null,"abstract":"Traditional affordance segmentation on 3D point cloud objects requires massive amounts of annotated training data and can only make predictions within predefined classes and affordance tasks. To overcome these limitations, we propose a variation-robust few-shot 3D affordance segmentation network (VRNet) for robotic manipulation, which requires only several affordance annotations for novel object classes and manipulation tasks. In particular, we design an orientation-tolerant feature extractor to address pose variation between support and query point cloud objects, and present a multi-scale label propagation algorithm for variation in completeness. Extensive experiments on affordance datasets show that VRNet provides the best segmentation performance compared with previous works. Moreover, experiments in real robotic scenarios demonstrate the generalization ability of our method.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1672-1679"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An Image-Guided Robotic System for Transcranial Magnetic Stimulation: System Development and Experimental Evaluation 经颅磁刺激的图像引导机器人系统:系统开发和实验评估
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524900
Yihao Liu;Jiaming Zhang;Letian Ai;Jing Tian;Shahriar Sefati;Huan Liu;Alejandro Martin-Gomez;Amir Kheradmand;Mehran Armand
Transcranial magnetic stimulation is a noninvasive medical procedure that can modulate brain activity, and it is widely used in neuroscience, neurology research, and clinical practice. Compared to manual operators, robots may improve the outcome due to their superior accuracy and repeatability. However, there has not been a widely accepted standard protocol for performing robotic TMS using fine-segmented brain images, resulting in arbitrary planned angles with respect to the true boundaries of the modulated cortex. Given that the recent study in TMS simulation suggests a noticeable difference in outcomes when using different anatomical details, cortical shape should play a more significant role in deciding the optimal TMS coil pose. In this work, we introduce an image-guided robotic system for TMS that focuses on (1) establishing standardized planning methods to define a reference (true zero) for the coil poses and (2) solving the issue that the manual coil placement requires expert hand-eye coordination which often leading to low repeatability of the experiments. To validate the design of our robotic system, a phantom study and a preliminary human subject study were performed. Our results show that the robotic method can half the positional error and improve the rotational accuracy by up to two orders of magnitude. The accuracy is proven to be repeatable because the standard deviation of multiple trials is lowered by an order of magnitude. The improved actuation accuracy successfully translates to the TMS application, with a higher and more stable induced voltage in magnetic field sensors and a higher electromyography (EMG) reading in the preliminary human subject study.
经颅磁刺激是一种可以调节大脑活动的无创医疗手段,广泛应用于神经科学、神经学研究和临床实践。与人工操作员相比,机器人由于其优越的准确性和可重复性,可能会改善结果。然而,目前还没有一个被广泛接受的标准方案来执行机器人TMS使用精细分割的大脑图像,导致相对于调制皮层的真实边界的任意规划角度。鉴于最近的TMS模拟研究表明,使用不同解剖细节的结果有显著差异,皮质形状应该在决定最佳TMS线圈姿势方面发挥更重要的作用。在这项工作中,我们介绍了一种用于TMS的图像引导机器人系统,该系统的重点是:(1)建立标准化的规划方法来定义线圈姿态的参考(真零);(2)解决手动线圈放置需要专家手眼协调的问题,这通常导致实验的可重复性低。为了验证我们的机器人系统的设计,进行了模拟研究和初步的人体受试者研究。结果表明,该方法可以将定位误差减半,并将旋转精度提高两个数量级。由于多次试验的标准偏差降低了一个数量级,因此证明了准确性是可重复的。提高的驱动精度成功地转化为TMS应用,在初步的人体受试者研究中,磁场传感器具有更高和更稳定的感应电压和更高的肌电图(EMG)读数。
{"title":"An Image-Guided Robotic System for Transcranial Magnetic Stimulation: System Development and Experimental Evaluation","authors":"Yihao Liu;Jiaming Zhang;Letian Ai;Jing Tian;Shahriar Sefati;Huan Liu;Alejandro Martin-Gomez;Amir Kheradmand;Mehran Armand","doi":"10.1109/LRA.2024.3524900","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524900","url":null,"abstract":"Transcranial magnetic stimulation is a noninvasive medical procedure that can modulate brain activity, and it is widely used in neuroscience, neurology research, and clinical practice. Compared to manual operators, robots may improve the outcome due to their superior accuracy and repeatability. However, there has not been a widely accepted standard protocol for performing robotic TMS using fine-segmented brain images, resulting in arbitrary planned angles with respect to the true boundaries of the modulated cortex. Given that the recent study in TMS simulation suggests a noticeable difference in outcomes when using different anatomical details, cortical shape should play a more significant role in deciding the optimal TMS coil pose. In this work, we introduce an image-guided robotic system for TMS that focuses on (1) establishing standardized planning methods to define a reference (true zero) for the coil poses and (2) solving the issue that the manual coil placement requires expert hand-eye coordination which often leading to low repeatability of the experiments. To validate the design of our robotic system, a phantom study and a preliminary human subject study were performed. Our results show that the robotic method can half the positional error and improve the rotational accuracy by up to two orders of magnitude. The accuracy is proven to be repeatable because the standard deviation of multiple trials is lowered by an order of magnitude. The improved actuation accuracy successfully translates to the TMS application, with a higher and more stable induced voltage in magnetic field sensors and a higher electromyography (EMG) reading in the preliminary human subject study.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1936-1943"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142992976","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Duality of the Existing Geometric Variable Strain Models for the Dynamic Modeling of Continuum Robots 连续体机器人动力学建模中现有几何变应变模型的对偶性
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524898
A. Ouyoucef;Q. Peyron;V. Lebastard;F. Renda;G. Zheng;F. Boyer
The Cosserat rod theory has become a gold standard for modeling the statics and dynamics of serial and parallel continuum robots. Recently, a weak form of these Cosserat rod models called the geometric variable strain model has been derived where the robot deformations are projected on finite-dimensional basis functions. This model has very interesting features for continuum robotics, such as a Lagrangian form close to classical rigid robots and the ability to tune its performances in terms of computation time and accuracy. Two approaches have been proposed to obtain and compute it. The first is based on the Newton-Euler recursive algorithm and the second, on the projection of the strong form equations using Jacobian matrices. Although these approaches yield identical model forms, their disparate implementations and numerical schemes render each uniquely suited to specific applications. Notably, underlying these disparities lies a profound duality between these models, prompting our quest for a comprehensive overview of this duality along with an analysis of their algorithmic differences. Finally, we discuss perspectives for these two approaches, in particular their hybridization, based on the current knowledge of rigid robotics.
Cosserat杆理论已经成为一个黄金标准建模的静力学和动力学的串联和并联连续体机器人。最近,这些coserat棒模型的一种弱形式被称为几何变应变模型,其中机器人的变形被投影在有限维基函数上。对于连续体机器人来说,这个模型具有非常有趣的特征,例如接近经典刚性机器人的拉格朗日形式,以及在计算时间和精度方面调整其性能的能力。提出了两种方法来获取和计算它。第一个是基于牛顿-欧拉递归算法,第二个是基于强形式方程的投影,使用雅可比矩阵。尽管这些方法产生相同的模型形式,但它们不同的实现和数值方案使它们各自独特地适合于特定的应用程序。值得注意的是,这些差异的基础是这些模型之间深刻的二元性,促使我们寻求对这种二元性的全面概述以及对其算法差异的分析。最后,我们讨论了这两种方法的观点,特别是它们的杂交,基于当前刚性机器人技术的知识。
{"title":"Duality of the Existing Geometric Variable Strain Models for the Dynamic Modeling of Continuum Robots","authors":"A. Ouyoucef;Q. Peyron;V. Lebastard;F. Renda;G. Zheng;F. Boyer","doi":"10.1109/LRA.2024.3524898","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524898","url":null,"abstract":"The Cosserat rod theory has become a gold standard for modeling the statics and dynamics of serial and parallel continuum robots. Recently, a weak form of these Cosserat rod models called the geometric variable strain model has been derived where the robot deformations are projected on finite-dimensional basis functions. This model has very interesting features for continuum robotics, such as a Lagrangian form close to classical rigid robots and the ability to tune its performances in terms of computation time and accuracy. Two approaches have been proposed to obtain and compute it. The first is based on the Newton-Euler recursive algorithm and the second, on the projection of the strong form equations using Jacobian matrices. Although these approaches yield identical model forms, their disparate implementations and numerical schemes render each uniquely suited to specific applications. Notably, underlying these disparities lies a profound duality between these models, prompting our quest for a comprehensive overview of this duality along with an analysis of their algorithmic differences. Finally, we discuss perspectives for these two approaches, in particular their hybridization, based on the current knowledge of rigid robotics.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1848-1855"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975735","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reduced-Dimensional Whole-Body Control Based on Model Simplification for Bipedal Robots With Parallel Mechanisms 基于模型简化的两足并联机器人全身降维控制
IF 4.6 2区 计算机科学 Q2 ROBOTICS Pub Date : 2025-01-01 DOI: 10.1109/LRA.2024.3524902
Yunpeng Liang;Fulong Yin;Zhen Li;Zhilin Xiong;Zhihui Peng;Yanzheng Zhao;Weixin Yan
The presence of parallel mechanisms in bipedal robots increases the complexity of modeling and control, making it crucial to manage the trade-off between model accuracy and real-time control. In this letter, we propose a reduced-dimensional whole-body controller for series-parallel bipedal robots, utilizing a floating-base multi-rigid body model with kinematic loops. Notably, we neglect the joint acceleration and closed-loop acceleration constraints of the parallel mechanisms, reducing the dimensionality of variables and constraints in the whole-body optimization problem while ensuring compliance with actuated joint torque limits. Quantitative experiments indicate that, compared to the complete series-parallel model, the impact of inertial forces resulting from the parallel joint acceleration is negligible. Additionally, physical locomotion and disturbance tests demonstrate that our proposed controller can enhance computational efficiency by over 20%, with comparable locomotion performance and disturbance rejection ability.
双足机器人中并联机构的存在增加了建模和控制的复杂性,使得在模型精度和实时控制之间进行权衡变得至关重要。在这封信中,我们提出了一种降维的串联-并联双足机器人全身控制器,利用带有运动回路的浮基多刚体模型。值得注意的是,我们忽略了并联机构的关节加速度和闭环加速度约束,降低了整体优化问题中变量和约束的维数,同时保证了驱动关节的扭矩限制。定量实验表明,与完全串并联模型相比,并联关节加速度产生的惯性力的影响可以忽略不计。此外,物理运动和干扰测试表明,我们提出的控制器可以提高20%以上的计算效率,具有相当的运动性能和抗干扰能力。
{"title":"Reduced-Dimensional Whole-Body Control Based on Model Simplification for Bipedal Robots With Parallel Mechanisms","authors":"Yunpeng Liang;Fulong Yin;Zhen Li;Zhilin Xiong;Zhihui Peng;Yanzheng Zhao;Weixin Yan","doi":"10.1109/LRA.2024.3524902","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524902","url":null,"abstract":"The presence of parallel mechanisms in bipedal robots increases the complexity of modeling and control, making it crucial to manage the trade-off between model accuracy and real-time control. In this letter, we propose a reduced-dimensional whole-body controller for series-parallel bipedal robots, utilizing a floating-base multi-rigid body model with kinematic loops. Notably, we neglect the joint acceleration and closed-loop acceleration constraints of the parallel mechanisms, reducing the dimensionality of variables and constraints in the whole-body optimization problem while ensuring compliance with actuated joint torque limits. Quantitative experiments indicate that, compared to the complete series-parallel model, the impact of inertial forces resulting from the parallel joint acceleration is negligible. Additionally, physical locomotion and disturbance tests demonstrate that our proposed controller can enhance computational efficiency by over 20%, with comparable locomotion performance and disturbance rejection ability.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1696-1703"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975937","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
IEEE Robotics and Automation Letters
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1