Vision-Based Online Key Point Estimation of Deformable Robots

IF 6.8 Q1 AUTOMATION & CONTROL SYSTEMS Advanced intelligent systems (Weinheim an der Bergstrasse, Germany) Pub Date : 2024-07-03 DOI:10.1002/aisy.202400105
Hehui Zheng, Sebastian Pinzello, Barnabas Gavin Cangan, Thomas J. K. Buchner, Robert K. Katzschmann
{"title":"Vision-Based Online Key Point Estimation of Deformable Robots","authors":"Hehui Zheng,&nbsp;Sebastian Pinzello,&nbsp;Barnabas Gavin Cangan,&nbsp;Thomas J. K. Buchner,&nbsp;Robert K. Katzschmann","doi":"10.1002/aisy.202400105","DOIUrl":null,"url":null,"abstract":"<p>The precise control of soft and continuum robots requires knowledge of their shape, which has, in contrast to classical rigid robots, infinite degrees of freedom. To partially reconstruct the shape, proprioceptive techniques use built-in sensors, resulting in inaccurate results and increased fabrication complexity. Exteroceptive methods so far rely on expensive tracking systems with reflective markers placed on all components, which are infeasible for deformable robots interacting with the environment due to marker occlusion and damage. Here, a regression approach is presented for three-dimensional key point estimation using a convolutional neural network. The proposed approach uses data-driven supervised learning and is capable of online markerless estimation during inference. Two images of a robotic system are captured simultaneously at 25 Hz from different perspectives and fed to the network, which returns for each pair the parameterized key point or piecewise constant curvature shape representations. The proposed approach outperforms markerless state-of-the-art methods by a maximum of 4.5% in estimation accuracy while being more robust and requiring no prior knowledge of the shape. Online evaluations on two types of soft robotic arms and a soft robotic fish demonstrate the method's accuracy and versatility on highly deformable systems.</p>","PeriodicalId":93858,"journal":{"name":"Advanced intelligent systems (Weinheim an der Bergstrasse, Germany)","volume":null,"pages":null},"PeriodicalIF":6.8000,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/aisy.202400105","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Advanced intelligent systems (Weinheim an der Bergstrasse, Germany)","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400105","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

The precise control of soft and continuum robots requires knowledge of their shape, which has, in contrast to classical rigid robots, infinite degrees of freedom. To partially reconstruct the shape, proprioceptive techniques use built-in sensors, resulting in inaccurate results and increased fabrication complexity. Exteroceptive methods so far rely on expensive tracking systems with reflective markers placed on all components, which are infeasible for deformable robots interacting with the environment due to marker occlusion and damage. Here, a regression approach is presented for three-dimensional key point estimation using a convolutional neural network. The proposed approach uses data-driven supervised learning and is capable of online markerless estimation during inference. Two images of a robotic system are captured simultaneously at 25 Hz from different perspectives and fed to the network, which returns for each pair the parameterized key point or piecewise constant curvature shape representations. The proposed approach outperforms markerless state-of-the-art methods by a maximum of 4.5% in estimation accuracy while being more robust and requiring no prior knowledge of the shape. Online evaluations on two types of soft robotic arms and a soft robotic fish demonstrate the method's accuracy and versatility on highly deformable systems.

Abstract Image

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于视觉的可变形机器人在线关键点估计
要精确控制软体和连续机器人,就必须了解它们的形状,与传统的刚性机器人相比,它们具有无限的自由度。为部分重建形状,本体感觉技术使用内置传感器,但结果不准确,且增加了制造复杂性。迄今为止,外感知方法依赖于昂贵的跟踪系统,该系统在所有部件上都放置了反射标记,但由于标记遮挡和损坏,对于与环境交互的可变形机器人来说,这种方法是不可行的。本文介绍了一种利用卷积神经网络进行三维关键点估计的回归方法。该方法采用数据驱动的监督学习,能够在推理过程中进行无标记在线估计。以 25 Hz 的频率从不同角度同时捕捉机器人系统的两幅图像,并将其输入网络,网络会返回每对图像的参数化关键点或片断恒定曲率形状表示。所提出的方法在估计准确度方面比最先进的无标记方法高出最多 4.5%,同时更加稳健,而且不需要预先了解形状。在两种软机械臂和一种软机械鱼上进行的在线评估证明了该方法在高变形系统上的准确性和通用性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
1.30
自引率
0.00%
发文量
0
审稿时长
4 weeks
期刊最新文献
Masthead Reconstructing Soft Robotic Touch via In-Finger Vision A Cable-Actuated Soft Manipulator for Dexterous Grasping Based on Deep Reinforcement Learning Liquid Metal Chameleon Tongues: Modulating Surface Tension and Phase Transition to Enable Bioinspired Soft Actuators Reprogrammable, Recyclable Origami Robots Controlled by Magnetic Fields
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1