Pub Date : 2024-09-20DOI: 10.1109/TMRB.2024.3464671
Wei Meng;Zunmei Tian;Chang Zhu;Qingsong Ai;Quan Liu
In recent years, with the increasing problem of an aging population, there has been a significant increase in the number of stroke patients presenting with motor dysfunction of the lower limbs. In this study, a knee exoskeleton rehabilitation robot driven by a quasi-direct driver actuator is designed. The torque generation model is constructed based on the TCN-LSTM hybrid neural network, and the knee joint torque is generated by sEMG and angle signal. A joint attention mechanism is introduced to enhance the accuracy of torque generation model. The impedance control parameters are adaptively adjusted in accordance with the joint torque. The experimental results demonstrate that the TCN-LSTM hybrid neural network is capable of effectively estimating torque, the mean MAE and CC of the proposed model are 1.141Nm and 93.7%, respectively. The optimized impedance control can optimize the initial value of the impedance parameter, which reduced the torque error by 5.54% and 50.64% at uphill tasks and walking task, respectively, and adaptively adjust the impedance parameter to ensure the coordination of the gait rehabilitation and the friendly human-robot interaction.
近年来，随着人口老龄化问题的日益突出，出现下肢运动功能障碍的中风患者人数大幅增加。本研究设计了一种由准直接驱动器驱动的膝关节外骨骼康复机器人。扭矩生成模型基于 TCN-LSTM 混合神经网络构建，膝关节扭矩由 sEMG 和角度信号生成。为提高扭矩生成模型的准确性，引入了关节关注机制。阻抗控制参数根据关节扭矩进行自适应调节。实验结果表明，TCN-LSTM 混合神经网络能够有效估计扭矩，其平均 MAE 和 CC 分别为 1.141Nm 和 93.7%。优化的阻抗控制可以优化阻抗参数的初始值，在上坡任务和行走任务中分别减少了 5.54% 和 50.64% 的扭矩误差，并能自适应地调整阻抗参数，确保步态康复和友好的人机交互的协调性。
{"title":"Optimized Impedance Control of a Lightweight Gait Rehabilitation Exoskeleton Based on Accurate Knee Joint Torque Estimation","authors":"Wei Meng;Zunmei Tian;Chang Zhu;Qingsong Ai;Quan Liu","doi":"10.1109/TMRB.2024.3464671","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464671","url":null,"abstract":"In recent years, with the increasing problem of an aging population, there has been a significant increase in the number of stroke patients presenting with motor dysfunction of the lower limbs. In this study, a knee exoskeleton rehabilitation robot driven by a quasi-direct driver actuator is designed. The torque generation model is constructed based on the TCN-LSTM hybrid neural network, and the knee joint torque is generated by sEMG and angle signal. A joint attention mechanism is introduced to enhance the accuracy of torque generation model. The impedance control parameters are adaptively adjusted in accordance with the joint torque. The experimental results demonstrate that the TCN-LSTM hybrid neural network is capable of effectively estimating torque, the mean MAE and CC of the proposed model are 1.141Nm and 93.7%, respectively. The optimized impedance control can optimize the initial value of the impedance parameter, which reduced the torque error by 5.54% and 50.64% at uphill tasks and walking task, respectively, and adaptively adjust the impedance parameter to ensure the coordination of the gait rehabilitation and the friendly human-robot interaction.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1648-1657"},"PeriodicalIF":3.4,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600331","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-20DOI: 10.1109/TMRB.2024.3465024
Marco Puliti;David M. Marsh;Michael Goldfarb
Stair ascent is a challenging task for people with transfemoral amputation. It can be made substantially easier with a swing-assist prosthesis, actively supplementing the prosthesis nominally passive behavior to help the user place their foot on the next stair tread. A significant control challenge is providing power without competing with the user’s agency during the swing-phase movement. This paper presents a new control approach for stair ascent swing-phase assistance in swing-assist prostheses. The approach is designed to supplement swing-phase movement with power without introducing an additional exogenous control input, leaving the user as the sole source of prosthesis movement. Namely, this is achieved by adding power to modify the homogeneous dynamics of the prosthesis’s passive behavior. This control approach is developed in the paper, implemented on an experimental prosthesis, and assessed in stair ascent trials with three unilateral transfemoral amputees, comparing it with their daily-use device. Experimental results demonstrate, for step-over stair ascent aggregated across participants, the proposed approach: 1) increased peak knee angle by a factor of 2.5; 2) improved symmetry from 41% to 84% (where 100% is perfectly symmetric); and 3) required 2 times less hip effort to achieve a given knee motion, all relative to daily-use prostheses.
{"title":"Homogeneous Dynamic Control for Stair Ascent in a Swing-Assist Knee Prosthesis","authors":"Marco Puliti;David M. Marsh;Michael Goldfarb","doi":"10.1109/TMRB.2024.3465024","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3465024","url":null,"abstract":"Stair ascent is a challenging task for people with transfemoral amputation. It can be made substantially easier with a swing-assist prosthesis, actively supplementing the prosthesis nominally passive behavior to help the user place their foot on the next stair tread. A significant control challenge is providing power without competing with the user’s agency during the swing-phase movement. This paper presents a new control approach for stair ascent swing-phase assistance in swing-assist prostheses. The approach is designed to supplement swing-phase movement with power without introducing an additional exogenous control input, leaving the user as the sole source of prosthesis movement. Namely, this is achieved by adding power to modify the homogeneous dynamics of the prosthesis’s passive behavior. This control approach is developed in the paper, implemented on an experimental prosthesis, and assessed in stair ascent trials with three unilateral transfemoral amputees, comparing it with their daily-use device. Experimental results demonstrate, for step-over stair ascent aggregated across participants, the proposed approach: 1) increased peak knee angle by a factor of 2.5; 2) improved symmetry from 41% to 84% (where 100% is perfectly symmetric); and 3) required 2 times less hip effort to achieve a given knee motion, all relative to daily-use prostheses.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1637-1647"},"PeriodicalIF":3.4,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142672066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-20DOI: 10.1109/TMRB.2024.3464698
Deepak Raina;Mythra V. Balakuntala;Byung Wook Kim;Juan Wachs;Richard Voyles
Ultrasound is widely employed for clinical intervention and diagnosis, due to its advantages of offering non-invasive, radiation-free, and real-time imaging. However, the accessibility of this dexterous procedure is limited due to the substantial training and expertise required of operators. The robotic ultrasound (RUS) offers a viable solution to address this limitation; nonetheless, achieving human-level proficiency remains challenging. Learning from demonstrations (LfD) methods have been explored in RUS, which learns the policy prior from a dataset of offline demonstrations to encode the mental model of the expert sonographer. However, active engagement of experts, i.e., Coaching, during the training of RUS has not been explored thus far. Coaching is known for enhancing efficiency and performance in human training. This paper proposes a coaching framework for RUS to amplify its performance. The framework combines DRL (self-supervised practice) with sparse expert’s feedback through coaching. The DRL employs an off-policy Soft Actor-Critic (SAC) network, with a reward based on image quality rating. The coaching by experts is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert. The validation study on phantoms showed that coaching increases the learning rate by 25% and the number of high-quality image acquisition by 74.5%.
超声波具有无创、无辐射和实时成像的优点，被广泛用于临床干预和诊断。然而，由于操作人员需要大量的培训和专业知识，这种灵巧程序的可及性受到了限制。机器人超声（RUS）为解决这一局限性提供了可行的解决方案；然而，要达到人类水平的熟练程度仍具有挑战性。从演示中学习（LfD）方法已在 RUS 中进行了探索，该方法从离线演示数据集中学习先验策略，以编码超声波专家的心智模型。但是，在 RUS 的训练过程中，专家的积极参与（即辅导）迄今为止还没有被探索过。众所周知，教练可以提高人类培训的效率和绩效。本文为 RUS 提出了一个教练框架，以提高其性能。该框架将 DRL（自我监督练习）与稀疏专家反馈（通过辅导）相结合。DRL 采用非政策软演员-批评家（SAC）网络，奖励基于图像质量评级。专家的指导被建模为部分可观测马尔可夫决策过程（POMDP），并根据专家的纠正更新策略参数。在模型上进行的验证研究表明，辅导使学习率提高了 25%，高质量图像获取数量提高了 74.5%。
{"title":"Coaching a Robotic Sonographer: Learning Robotic Ultrasound With Sparse Expert’s Feedback","authors":"Deepak Raina;Mythra V. Balakuntala;Byung Wook Kim;Juan Wachs;Richard Voyles","doi":"10.1109/TMRB.2024.3464698","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464698","url":null,"abstract":"Ultrasound is widely employed for clinical intervention and diagnosis, due to its advantages of offering non-invasive, radiation-free, and real-time imaging. However, the accessibility of this dexterous procedure is limited due to the substantial training and expertise required of operators. The robotic ultrasound (RUS) offers a viable solution to address this limitation; nonetheless, achieving human-level proficiency remains challenging. Learning from demonstrations (LfD) methods have been explored in RUS, which learns the policy prior from a dataset of offline demonstrations to encode the mental model of the expert sonographer. However, active engagement of experts, i.e., Coaching, during the training of RUS has not been explored thus far. Coaching is known for enhancing efficiency and performance in human training. This paper proposes a coaching framework for RUS to amplify its performance. The framework combines DRL (self-supervised practice) with sparse expert’s feedback through coaching. The DRL employs an off-policy Soft Actor-Critic (SAC) network, with a reward based on image quality rating. The coaching by experts is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert. The validation study on phantoms showed that coaching increases the learning rate by 25% and the number of high-quality image acquisition by 74.5%.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1391-1396"},"PeriodicalIF":3.4,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-20DOI: 10.1109/TMRB.2024.3464690
Daniel Colley;Collin D. Bowersock;Zachary F. Lerner
This study introduces a novel lightweight elbow joint exoskeleton designed to enhance the safety and efficiency of industrial workers engaged in manual handling tasks. Our design leveraged a Bowden cable transmission system and a practical control strategy utilizing instrumented gloves to deliver reactive bi-directional support for dynamic box lifting and pressing activities. The primary focus of this work was to (1) to present an engineering validation analysis and (2) assess the exoskeleton’s impact on reducing muscle activity, increasing endurance, and maintaining overall user comfort during upper-extremity lifting or carrying tasks. We observed significant and consistent reductions in muscle activity and an increase in endurance (e.g., 2.4x more repetitions) during box lifting tasks, without compromising user comfort. These findings provide promising evidence of the exoskeleton’s effectiveness and represent a crucial first step working towards demonstrating efficacy in real-world workplace environments.
{"title":"A Lightweight Powered Elbow Exoskeleton for Manual Handling Tasks","authors":"Daniel Colley;Collin D. Bowersock;Zachary F. Lerner","doi":"10.1109/TMRB.2024.3464690","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464690","url":null,"abstract":"This study introduces a novel lightweight elbow joint exoskeleton designed to enhance the safety and efficiency of industrial workers engaged in manual handling tasks. Our design leveraged a Bowden cable transmission system and a practical control strategy utilizing instrumented gloves to deliver reactive bi-directional support for dynamic box lifting and pressing activities. The primary focus of this work was to (1) to present an engineering validation analysis and (2) assess the exoskeleton’s impact on reducing muscle activity, increasing endurance, and maintaining overall user comfort during upper-extremity lifting or carrying tasks. We observed significant and consistent reductions in muscle activity and an increase in endurance (e.g., 2.4x more repetitions) during box lifting tasks, without compromising user comfort. These findings provide promising evidence of the exoskeleton’s effectiveness and represent a crucial first step working towards demonstrating efficacy in real-world workplace environments.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1627-1636"},"PeriodicalIF":3.4,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-20DOI: 10.1109/TMRB.2024.3464683
Laura Connolly;Aravind S. Kumar;Kapi Ketan Mehta;Lidia Al-Zogbi;Peter Kazanzides;Parvin Mousavi;Gabor Fichtinger;Axel Krieger;Junichi Tokuda;Russell H. Taylor;Simon Leonard;Anton Deguet
Image-guided robotic interventions involve the use of medical imaging in tandem with robotics. SlicerROS2 is a software module that combines 3D Slicer and robot operating system (ROS) in pursuit of a standard integration approach for medical robotics research. The first release of SlicerROS2 demonstrated the feasibility of using the C++ API from 3D Slicer and ROS to load and visualize robots in real time. Since this initial release, we’ve rewritten and redesigned the module to offer greater modularity, access to low-level features, access to 3D Slicer’s Python API, and better data transfer protocols. In this paper, we introduce this new design as well as four applications that leverage the core functionalities of SlicerROS2 in realistic image-guided robotics scenarios.
图像引导的机器人干预涉及将医学成像与机器人技术相结合。SlicerROS2 是将 3D Slicer 和机器人操作系统 (ROS) 结合在一起的软件模块，旨在为医疗机器人研究提供标准的集成方法。SlicerROS2 的第一个版本展示了使用 3D Slicer 和 ROS 的 C++ API 实时加载和可视化机器人的可行性。自首次发布以来，我们对该模块进行了重写和重新设计，以提供更强的模块性、访问底层功能、访问 3D Slicer 的 Python API 以及更好的数据传输协议。在本文中，我们将介绍这一新设计以及在现实图像引导机器人场景中利用 SlicerROS2 核心功能的四个应用。
{"title":"SlicerROS2: A Research and Development Module for Image-Guided Robotic Interventions","authors":"Laura Connolly;Aravind S. Kumar;Kapi Ketan Mehta;Lidia Al-Zogbi;Peter Kazanzides;Parvin Mousavi;Gabor Fichtinger;Axel Krieger;Junichi Tokuda;Russell H. Taylor;Simon Leonard;Anton Deguet","doi":"10.1109/TMRB.2024.3464683","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464683","url":null,"abstract":"Image-guided robotic interventions involve the use of medical imaging in tandem with robotics. SlicerROS2 is a software module that combines 3D Slicer and robot operating system (ROS) in pursuit of a standard integration approach for medical robotics research. The first release of SlicerROS2 demonstrated the feasibility of using the C++ API from 3D Slicer and ROS to load and visualize robots in real time. Since this initial release, we’ve rewritten and redesigned the module to offer greater modularity, access to low-level features, access to 3D Slicer’s Python API, and better data transfer protocols. In this paper, we introduce this new design as well as four applications that leverage the core functionalities of SlicerROS2 in realistic image-guided robotics scenarios.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1334-1344"},"PeriodicalIF":3.4,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Retinal microsurgery is crucial for treating various ocular diseases, but challenging due to the structure size, physiological tremor and limited depth perception. This study aims to develop an innovative real-time needle tracking system that utilizes only a small amount of Optical Coherence Tomography (OCT) A-scans. We introduce a spiral scanning pattern, that is dynamically updated to efficiently capture the needle tip and the retina area with 2000 A-scans. An imaging pipeline is proposed that initiates with an initial Region of Interest (ROI) identification, followed by image segmentation, 3D reconstruction, and needle pose estimation. The ROI is dynamically adjusted to keep the needle tip centrally within the spiral scan, facilitating tracking at clinically relevant speeds. Preliminary testing on phantom eye models demonstrated that our system can maintain an average tracking error of 0.04 mm in spatial coordinates and an error of 0.06 mm in estimating the distance between the needle tip and the retina. These results suggest the system’s potential to enhance surgical outcomes by providing surgeons with improved depth perception and precise, real-time feedback. By efficiently utilizing spirally sampled OCT data, this system sets the groundwork for future integrations of real-time 4D imaging and physiological motion detection capabilities.
视网膜显微手术对治疗各种眼部疾病至关重要，但由于其结构尺寸、生理震颤和有限的深度知觉而具有挑战性。本研究旨在开发一种创新的实时针跟踪系统，该系统只需利用少量的光学相干断层扫描（OCT）A 扫描图像。我们引入了一种螺旋扫描模式，通过动态更新，以 2000 次 A 扫描有效捕捉针尖和视网膜区域。我们提出了一个成像流水线，首先进行感兴趣区（ROI）识别，然后进行图像分割、三维重建和针尖姿态估计。对感兴趣区进行动态调整，使针尖位于螺旋扫描的中心位置，便于以临床相关的速度进行跟踪。对假眼模型的初步测试表明，我们的系统能将空间坐标的平均跟踪误差保持在 0.04 毫米，将针尖与视网膜之间距离的估计误差保持在 0.06 毫米。这些结果表明，该系统可以为外科医生提供更好的深度感知和精确的实时反馈，从而提高手术效果。通过有效利用螺旋采样 OCT 数据，该系统为未来整合实时 4D 成像和生理运动检测功能奠定了基础。
{"title":"Fast OCT-Based Needle Tracking for Retinal Microsurgery Using Dynamic Spiral Scanning","authors":"Pengwei Xu;Mouloud Ourak;Gianni Borghesan;Emmanuel Vander Poorten","doi":"10.1109/TMRB.2024.3464693","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464693","url":null,"abstract":"Retinal microsurgery is crucial for treating various ocular diseases, but challenging due to the structure size, physiological tremor and limited depth perception. This study aims to develop an innovative real-time needle tracking system that utilizes only a small amount of Optical Coherence Tomography (OCT) A-scans. We introduce a spiral scanning pattern, that is dynamically updated to efficiently capture the needle tip and the retina area with 2000 A-scans. An imaging pipeline is proposed that initiates with an initial Region of Interest (ROI) identification, followed by image segmentation, 3D reconstruction, and needle pose estimation. The ROI is dynamically adjusted to keep the needle tip centrally within the spiral scan, facilitating tracking at clinically relevant speeds. Preliminary testing on phantom eye models demonstrated that our system can maintain an average tracking error of 0.04 mm in spatial coordinates and an error of 0.06 mm in estimating the distance between the needle tip and the retina. These results suggest the system’s potential to enhance surgical outcomes by providing surgeons with improved depth perception and precise, real-time feedback. By efficiently utilizing spirally sampled OCT data, this system sets the groundwork for future integrations of real-time 4D imaging and physiological motion detection capabilities.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1502-1511"},"PeriodicalIF":3.4,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600318","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lower limb prosthetics, essential for restoring mobility in individuals with limb loss, have witnessed significant advancements in recent years. This systematic review reports the recent research advancements in the field of semi-active and active lower limb prostheses. The review focuses on the mechatronic features of the devices, the sensing and control strategies, and the performance verification with end-users. A total of 53 prosthetic prototypes were identified and analyzed, including 16 knee-ankle prostheses, 18 knee prostheses, and 19 ankle prostheses. The review highlights some of the open challenges in the field of prosthetic research.
{"title":"Advancements and Challenges in the Development of Robotic Lower Limb Prostheses: A Systematic Review","authors":"Ilaria Fagioli;Alessandro Mazzarini;Chiara Livolsi;Emanuele Gruppioni;Nicola Vitiello;Simona Crea;Emilio Trigili","doi":"10.1109/TMRB.2024.3464126","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464126","url":null,"abstract":"Lower limb prosthetics, essential for restoring mobility in individuals with limb loss, have witnessed significant advancements in recent years. This systematic review reports the recent research advancements in the field of semi-active and active lower limb prostheses. The review focuses on the mechatronic features of the devices, the sensing and control strategies, and the performance verification with end-users. A total of 53 prosthetic prototypes were identified and analyzed, including 16 knee-ankle prostheses, 18 knee prostheses, and 19 ankle prostheses. The review highlights some of the open challenges in the field of prosthetic research.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1409-1422"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10684266","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464114
Jixiu Li;Truman Cheng;Wai Shing Chan;Zixiao Chen;Yehui Li;Calvin Sze Hang Ng;Philip Wai Yan Chiu;Zheng Li
Remote Actuation Mechanisms (RAMs) play a vital role in minimally invasive surgery (MIS) by providing motion capabilities within limited spaces. This paper first focused on analyzing commonly employed RAMs to understand their strengths and limitations. Then, drawing inspiration from bionics and the biological structure of scorpions, we proposed a novel approach by integrating three RAMs-a magnet pair, a torque coil, and a soft bellow-to create a 5-degree-of-freedom (5-DOF) miniature remote actuation robot. In the design phase, we established the robot’s parameters using the magnetic dipole model and related constraints. A functional prototype of the robot, along with an external controller and user interface, was fabricated and assembled. Experimental investigations demonstrated motion performance across the 5 DOF, validating the robot’s feasibility. To assess the practicality of the system, the interaction interface was evaluated under controlled laboratory conditions and through a cadaver test. In conclusion, our innovative approach combines multiple RAMs into a 5-DOF remote actuation robot. Comprehensive tests validated its motion capabilities and highlighted its potential to advance MIS procedures.
{"title":"A Scorpion-Inspired 5-DOF Miniature Remote Actuation Robotic Endoscope for Minimally Invasive Surgery","authors":"Jixiu Li;Truman Cheng;Wai Shing Chan;Zixiao Chen;Yehui Li;Calvin Sze Hang Ng;Philip Wai Yan Chiu;Zheng Li","doi":"10.1109/TMRB.2024.3464114","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464114","url":null,"abstract":"Remote Actuation Mechanisms (RAMs) play a vital role in minimally invasive surgery (MIS) by providing motion capabilities within limited spaces. This paper first focused on analyzing commonly employed RAMs to understand their strengths and limitations. Then, drawing inspiration from bionics and the biological structure of scorpions, we proposed a novel approach by integrating three RAMs-a magnet pair, a torque coil, and a soft bellow-to create a 5-degree-of-freedom (5-DOF) miniature remote actuation robot. In the design phase, we established the robot’s parameters using the magnetic dipole model and related constraints. A functional prototype of the robot, along with an external controller and user interface, was fabricated and assembled. Experimental investigations demonstrated motion performance across the 5 DOF, validating the robot’s feasibility. To assess the practicality of the system, the interaction interface was evaluated under controlled laboratory conditions and through a cadaver test. In conclusion, our innovative approach combines multiple RAMs into a 5-DOF remote actuation robot. Comprehensive tests validated its motion capabilities and highlighted its potential to advance MIS procedures.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1748-1759"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464110
Abed Soleymani;Mahdi Tavakoli;Farzad Aghazadeh;Yafei Ou;Hossein Rouhani;Bin Zheng;Xingyu Li
Bimanual tasks, where the brain must simultaneously control and plan the movements of both hands, such as needle passing and tissue cutting, commonly exist in surgeries, e.g., robot-assisted minimally invasive surgery. In this study, we present a novel approach for quantifying the quality of hands coordination and correspondence in bimanual tasks by utilizing information theory concepts to build a mathematical framework for measuring the collaboration strength between the two hands. The introduced method makes no assumption about the dynamics and couplings within the robotic platform, executive task, or human motor control. We implemented the proposed approach on MEELS and JIGSAWS datasets, corresponding to conventional minimally invasive surgery (MIS) and robot-assisted MIS, respectively. We analyzed the advantages of hands collaboration features in the skills assessment and style recognition of robotic surgery tasks. Furthermore, we demonstrated that incorporating intuitive domain knowledge of bimanual tasks potentially paves the way for other complex applications, including, but not limited to, autonomous surgery with a high level of model explainability and interpretability. Finally, we presented preliminary results to argue that incorporating hands collaboration features in deep learning-based classifiers reduces uncertainty, improves accuracy, and enhances the out-of-distribution robustness of the final model.
{"title":"Hands Collaboration Evaluation for Surgical Skills Assessment: An Information Theoretical Approach","authors":"Abed Soleymani;Mahdi Tavakoli;Farzad Aghazadeh;Yafei Ou;Hossein Rouhani;Bin Zheng;Xingyu Li","doi":"10.1109/TMRB.2024.3464110","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464110","url":null,"abstract":"Bimanual tasks, where the brain must simultaneously control and plan the movements of both hands, such as needle passing and tissue cutting, commonly exist in surgeries, e.g., robot-assisted minimally invasive surgery. In this study, we present a novel approach for quantifying the quality of hands coordination and correspondence in bimanual tasks by utilizing information theory concepts to build a mathematical framework for measuring the collaboration strength between the two hands. The introduced method makes no assumption about the dynamics and couplings within the robotic platform, executive task, or human motor control. We implemented the proposed approach on MEELS and JIGSAWS datasets, corresponding to conventional minimally invasive surgery (MIS) and robot-assisted MIS, respectively. We analyzed the advantages of hands collaboration features in the skills assessment and style recognition of robotic surgery tasks. Furthermore, we demonstrated that incorporating intuitive domain knowledge of bimanual tasks potentially paves the way for other complex applications, including, but not limited to, autonomous surgery with a high level of model explainability and interpretability. Finally, we presented preliminary results to argue that incorporating hands collaboration features in deep learning-based classifiers reduces uncertainty, improves accuracy, and enhances the out-of-distribution robustness of the final model.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1490-1501"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464089
Ke Fan;Ziyang Chen;Qiaoling Liu;Giancarlo Ferrigno;Elena De Momi
3D pose reconstruction of surgical instruments from images stands as a critical component in environment perception within robotic minimally invasive surgery (RMIS). The current deep learning methods rely on complex networks to enhance accuracy, making real-time implementation difficult. Moreover, diverging from a singular rigid body, surgical instruments exhibit an articulation structure, making the annotation of 3D poses more challenging. In this paper, we present a novel approach to formulate the 3D pose reconstruction of articulated surgical instruments as a Markov Decision Process (MDP). A Reinforcement Learning (RL) agent employs 2D image labels to control a virtual articulated skeleton to reproduce the 3D pose of the real surgical instrument. Firstly, a convolutional neural network is used to estimate the 2D pixel positions of joint nodes of the surgical instrument skeleton. Subsequently, the agent controls the 3D virtual articulated skeleton to align its joint nodes’ projections on the image plane with those in the real image. Validation of our proposed method is conducted using a semi-synthetic dataset with precise 3D pose labels and two real datasets, demonstrating the accuracy and efficacy of our approach. The results indicate the potential of our method in achieving real-time 3D pose reconstruction for articulated surgical instruments in the context of RMIS, addressing the challenges posed by low-texture surfaces and articulated structures.
{"title":"A Reinforcement Learning Approach for Real-Time Articulated Surgical Instrument 3-D Pose Reconstruction","authors":"Ke Fan;Ziyang Chen;Qiaoling Liu;Giancarlo Ferrigno;Elena De Momi","doi":"10.1109/TMRB.2024.3464089","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464089","url":null,"abstract":"3D pose reconstruction of surgical instruments from images stands as a critical component in environment perception within robotic minimally invasive surgery (RMIS). The current deep learning methods rely on complex networks to enhance accuracy, making real-time implementation difficult. Moreover, diverging from a singular rigid body, surgical instruments exhibit an articulation structure, making the annotation of 3D poses more challenging. In this paper, we present a novel approach to formulate the 3D pose reconstruction of articulated surgical instruments as a Markov Decision Process (MDP). A Reinforcement Learning (RL) agent employs 2D image labels to control a virtual articulated skeleton to reproduce the 3D pose of the real surgical instrument. Firstly, a convolutional neural network is used to estimate the 2D pixel positions of joint nodes of the surgical instrument skeleton. Subsequently, the agent controls the 3D virtual articulated skeleton to align its joint nodes’ projections on the image plane with those in the real image. Validation of our proposed method is conducted using a semi-synthetic dataset with precise 3D pose labels and two real datasets, demonstrating the accuracy and efficacy of our approach. The results indicate the potential of our method in achieving real-time 3D pose reconstruction for articulated surgical instruments in the context of RMIS, addressing the challenges posed by low-texture surfaces and articulated structures.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1458-1467"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600437","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：