Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464089
Ke Fan;Ziyang Chen;Qiaoling Liu;Giancarlo Ferrigno;Elena De Momi
3D pose reconstruction of surgical instruments from images stands as a critical component in environment perception within robotic minimally invasive surgery (RMIS). The current deep learning methods rely on complex networks to enhance accuracy, making real-time implementation difficult. Moreover, diverging from a singular rigid body, surgical instruments exhibit an articulation structure, making the annotation of 3D poses more challenging. In this paper, we present a novel approach to formulate the 3D pose reconstruction of articulated surgical instruments as a Markov Decision Process (MDP). A Reinforcement Learning (RL) agent employs 2D image labels to control a virtual articulated skeleton to reproduce the 3D pose of the real surgical instrument. Firstly, a convolutional neural network is used to estimate the 2D pixel positions of joint nodes of the surgical instrument skeleton. Subsequently, the agent controls the 3D virtual articulated skeleton to align its joint nodes’ projections on the image plane with those in the real image. Validation of our proposed method is conducted using a semi-synthetic dataset with precise 3D pose labels and two real datasets, demonstrating the accuracy and efficacy of our approach. The results indicate the potential of our method in achieving real-time 3D pose reconstruction for articulated surgical instruments in the context of RMIS, addressing the challenges posed by low-texture surfaces and articulated structures.
{"title":"A Reinforcement Learning Approach for Real-Time Articulated Surgical Instrument 3-D Pose Reconstruction","authors":"Ke Fan;Ziyang Chen;Qiaoling Liu;Giancarlo Ferrigno;Elena De Momi","doi":"10.1109/TMRB.2024.3464089","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464089","url":null,"abstract":"3D pose reconstruction of surgical instruments from images stands as a critical component in environment perception within robotic minimally invasive surgery (RMIS). The current deep learning methods rely on complex networks to enhance accuracy, making real-time implementation difficult. Moreover, diverging from a singular rigid body, surgical instruments exhibit an articulation structure, making the annotation of 3D poses more challenging. In this paper, we present a novel approach to formulate the 3D pose reconstruction of articulated surgical instruments as a Markov Decision Process (MDP). A Reinforcement Learning (RL) agent employs 2D image labels to control a virtual articulated skeleton to reproduce the 3D pose of the real surgical instrument. Firstly, a convolutional neural network is used to estimate the 2D pixel positions of joint nodes of the surgical instrument skeleton. Subsequently, the agent controls the 3D virtual articulated skeleton to align its joint nodes’ projections on the image plane with those in the real image. Validation of our proposed method is conducted using a semi-synthetic dataset with precise 3D pose labels and two real datasets, demonstrating the accuracy and efficacy of our approach. The results indicate the potential of our method in achieving real-time 3D pose reconstruction for articulated surgical instruments in the context of RMIS, addressing the challenges posed by low-texture surfaces and articulated structures.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1458-1467"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600437","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464096
Kaifeng Wang;Aofei Tian;Yupeng Hao;Chengzhi Hu;Chaoyang Shi
This article proposes a fiber Bragg grating (FBG) based angle sensor with an extensive measurement range and high precision for human knee joint measurement. The proposed sensor mainly comprises an angle-linear displacement conversion cam, a crank-slider mechanism-inspired conversion flexure, an optical fiber embedded with an FBG element, and a sensor package. The cam transforms the wide-range knee angle input into vertical linear displacement output. The conversion flexure further converts such vertical displacement into a reduced horizontal displacement/stretching applied to the optical fiber with a motion scale ratio of 6:1. The flexure design features a symmetrical structure to improve stability and depress hysteresis. The fiber is suspended on the flexure’s output beams with a two-point pasting configuration. Both theory analysis and finite element method (FEM)-based simulations revealed the linear relationship between the input angle and the fiber strain. Static and dynamic experiments have verified the performance of the proposed sensor, demonstrating a sensitivity of 62.03 pm/° with a small linearity error of 1.36% within [0, 140°]. The root mean square errors (RMSE) were 0.72° and 0.84° for angle velocities of 80°/s and 350°/s, respectively. Wearable experiments during sitting and walking have been performed to validate the effectiveness of the proposed sensor.
{"title":"Development of a High-Precision and Large-Range FBG-Based Sensor Inspired by a Crank-Slider Mechanism for Wearable Measurement of Human Knee Joint Angles","authors":"Kaifeng Wang;Aofei Tian;Yupeng Hao;Chengzhi Hu;Chaoyang Shi","doi":"10.1109/TMRB.2024.3464096","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464096","url":null,"abstract":"This article proposes a fiber Bragg grating (FBG) based angle sensor with an extensive measurement range and high precision for human knee joint measurement. The proposed sensor mainly comprises an angle-linear displacement conversion cam, a crank-slider mechanism-inspired conversion flexure, an optical fiber embedded with an FBG element, and a sensor package. The cam transforms the wide-range knee angle input into vertical linear displacement output. The conversion flexure further converts such vertical displacement into a reduced horizontal displacement/stretching applied to the optical fiber with a motion scale ratio of 6:1. The flexure design features a symmetrical structure to improve stability and depress hysteresis. The fiber is suspended on the flexure’s output beams with a two-point pasting configuration. Both theory analysis and finite element method (FEM)-based simulations revealed the linear relationship between the input angle and the fiber strain. Static and dynamic experiments have verified the performance of the proposed sensor, demonstrating a sensitivity of 62.03 pm/° with a small linearity error of 1.36% within [0, 140°]. The root mean square errors (RMSE) were 0.72° and 0.84° for angle velocities of 80°/s and 350°/s, respectively. Wearable experiments during sitting and walking have been performed to validate the effectiveness of the proposed sensor.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1688-1698"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464095
R. C. Vijayan;N. M. Sheth;J. Wei;K. Venkataraman;D. Ghanem;B. Shafiq;J. H. Siewerdsen;W. Zbijewski;G. Li;K. Cleary;A. Uneri
Robot-assisted orthopaedic joint reduction offers enhanced precision and control across multiple axes of motion, enabling precise realignment according to predefined plans. However, the high levels of forces encountered may induce unintended anatomical motion and flex mechanical components. To address this, this work presents an approach that uses 2D fluoroscopic imaging to verify and readjust the 3D reduction path by tracking deviations from the planned trajectory. The proposed method involves a 3D-2D registration algorithm using a pair of fluoroscopic images, along with prior models of each body in the radiographic scene. This objective is formulated to couple and constrain multiple object poses (fibula, tibia, talus, and robot end effector), and incorporate novel methods for automatic view and hyperparameter selection to improve robustness. The algorithms were refined through cadaver studies and evaluated in a preclinical trial, employing a robotic system to manipulate a dislocated fibula. Studies with cadaveric specimens highlighted the joint-specific formulation’s high registration accuracy ( $Delta _{x} {=} 0.3~pm ~1$ .5 mm), further improved with the use of automatic view and hyperparameter selection ( $Delta _{x} {=} 0.2~pm ~0$ .8 mm). Preclinical studies demonstrated a high deviation between the intended and the actual path of the robotic system, which was accurately captured ( $Delta _{x}$ 1 mm) using the proposed techniques. The solution offers to close the loop on image-based guidance of robot-assisted joint reduction by tracking the robot and bones to dynamically correct the course. The approach uses standard clinical images and is expected to lower radiation exposure by providing 3D information and allowing the staff to stay clear of the x-ray beam.
{"title":"Robot-Assisted Reduction of the Ankle Joint via Multi-Body 3D–2D Image Registration","authors":"R. C. Vijayan;N. M. Sheth;J. Wei;K. Venkataraman;D. Ghanem;B. Shafiq;J. H. Siewerdsen;W. Zbijewski;G. Li;K. Cleary;A. Uneri","doi":"10.1109/TMRB.2024.3464095","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464095","url":null,"abstract":"Robot-assisted orthopaedic joint reduction offers enhanced precision and control across multiple axes of motion, enabling precise realignment according to predefined plans. However, the high levels of forces encountered may induce unintended anatomical motion and flex mechanical components. To address this, this work presents an approach that uses 2D fluoroscopic imaging to verify and readjust the 3D reduction path by tracking deviations from the planned trajectory. The proposed method involves a 3D-2D registration algorithm using a pair of fluoroscopic images, along with prior models of each body in the radiographic scene. This objective is formulated to couple and constrain multiple object poses (fibula, tibia, talus, and robot end effector), and incorporate novel methods for automatic view and hyperparameter selection to improve robustness. The algorithms were refined through cadaver studies and evaluated in a preclinical trial, employing a robotic system to manipulate a dislocated fibula. Studies with cadaveric specimens highlighted the joint-specific formulation’s high registration accuracy (\u0000<inline-formula> <tex-math>$Delta _{x} {=} 0.3~pm ~1$ </tex-math></inline-formula>\u0000.5 mm), further improved with the use of automatic view and hyperparameter selection (\u0000<inline-formula> <tex-math>$Delta _{x} {=} 0.2~pm ~0$ </tex-math></inline-formula>\u0000.8 mm). Preclinical studies demonstrated a high deviation between the intended and the actual path of the robotic system, which was accurately captured (\u0000<inline-formula> <tex-math>$Delta _{x}$ </tex-math></inline-formula>\u0000 1 mm) using the proposed techniques. The solution offers to close the loop on image-based guidance of robot-assisted joint reduction by tracking the robot and bones to dynamically correct the course. The approach uses standard clinical images and is expected to lower radiation exposure by providing 3D information and allowing the staff to stay clear of the x-ray beam.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1591-1602"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464107
Quan Xiong;Dannuo Li;Xuanyi Zhou;Wenci Xin;Chao Wang;Jonathan William Ambrose;Raye Chen-Hua Yeow
Humanoid robotic hands have significant potential in easing human burden and augmenting human labor. This paper introduces the SMUFR hand, a compliant and dexterous robotic humanoid hand powered by tendon-driven mechanisms, and features flexible beam-based bending joints serving as rotary joints with bidirectional bending compliance that ensure safety during human-robot interaction. Despite its light weight of only 363 g without remote transmission and actuation components, the SMUFR hand can grasp and support loads of up to 4.2 kg in various orientations, manipulate objects of different sizes and shapes, and even operate underwater. Of particular note is the SMUFR hand’s lightweight and compact one-to-more actuation system, comprising six rotary pneumatic clutches (RPC) for six active Degrees of Freedom (DoFs), all powered by a single motor. Each RPC, weighing 75 g, can exert up to 23 N force on the tendon. This innovative transmission system distributes the power of a single motor across five fingers and holds potential for configuring additional RPCs. We also integrated all the components on a compact wearable vest for potential mobile humanoid robotic applications. Additionally, a mathematical model was developed to predict tendon force and joint bending using the constant curvature deformation hypothesis. Experimental validation demonstrates the durability of both the RPC and the beam-based fingers of the SMUFR hand, which are capable of enduring up to 22,000 and 30,000 cycles, respectively.
{"title":"Single-Motor Ultraflexible Robotic (SMUFR) Humanoid Hand","authors":"Quan Xiong;Dannuo Li;Xuanyi Zhou;Wenci Xin;Chao Wang;Jonathan William Ambrose;Raye Chen-Hua Yeow","doi":"10.1109/TMRB.2024.3464107","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464107","url":null,"abstract":"Humanoid robotic hands have significant potential in easing human burden and augmenting human labor. This paper introduces the SMUFR hand, a compliant and dexterous robotic humanoid hand powered by tendon-driven mechanisms, and features flexible beam-based bending joints serving as rotary joints with bidirectional bending compliance that ensure safety during human-robot interaction. Despite its light weight of only 363 g without remote transmission and actuation components, the SMUFR hand can grasp and support loads of up to 4.2 kg in various orientations, manipulate objects of different sizes and shapes, and even operate underwater. Of particular note is the SMUFR hand’s lightweight and compact one-to-more actuation system, comprising six rotary pneumatic clutches (RPC) for six active Degrees of Freedom (DoFs), all powered by a single motor. Each RPC, weighing 75 g, can exert up to 23 N force on the tendon. This innovative transmission system distributes the power of a single motor across five fingers and holds potential for configuring additional RPCs. We also integrated all the components on a compact wearable vest for potential mobile humanoid robotic applications. Additionally, a mathematical model was developed to predict tendon force and joint bending using the constant curvature deformation hypothesis. Experimental validation demonstrates the durability of both the RPC and the beam-based fingers of the SMUFR hand, which are capable of enduring up to 22,000 and 30,000 cycles, respectively.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1666-1677"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600322","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464109
Ziwen Wang;Yingying Han;Baoliang Zhao;Haiqin Xie;Liang Yao;Bing Li;Max Q.-H. Meng;Ying Hu
Ultrasound (US) examination is widely used to diagnose carotid artery plaque, which requires the sonographer to guide the probe to scan along a specific path for complete coverage of the carotid artery region. Meanwhile, stable probe-neck interaction is important for high-quality image acquisition. In this study, a robotic system for autonomous carotid US scanning is proposed. To realize the autonomous visual servo movement of the probe, an object tracking method based on improved Siamese network is proposed. Meanwhile, a local quality assessment algorithm is proposed to ensure that carotid ultrasound images are clear and desirable for diagnosis. To address the issue of poor probe-neck contact and loss of carotid object during ultrasound scanning, an automatic recovery control method is proposed to ensure the continuity of the scanning process without the need to stop and restart the scanning. Experimental results show that the robotic system can successfully navigate the probe to move along a path that meets the clinical standard. In addition, the robot can autonomously rediscover the object and return to the normal scanning state if a stuck condition occurs.
超声波（US）检查被广泛用于诊断颈动脉斑块，这需要超声波技师引导探头沿着特定路径扫描，以完全覆盖颈动脉区域。同时，探头与颈部的稳定互动对于获取高质量图像非常重要。本研究提出了一种用于自主颈动脉 US 扫描的机器人系统。为实现探头的自主视觉伺服运动，提出了一种基于改进型连体网络的目标跟踪方法。同时，还提出了一种局部质量评估算法，以确保颈动脉超声图像的清晰度和诊断效果。针对超声扫描过程中探头与颈部接触不良和颈动脉物体丢失的问题，提出了一种自动恢复控制方法，以确保扫描过程的连续性，而无需停止和重新启动扫描。实验结果表明，机器人系统可以成功地引导探头沿着符合临床标准的路径移动。此外，如果出现卡住的情况，机器人还能自主地重新发现物体，并返回正常的扫描状态。
{"title":"Autonomous Robotic System for Carotid Artery Ultrasound Scanning With Visual Servo Navigation","authors":"Ziwen Wang;Yingying Han;Baoliang Zhao;Haiqin Xie;Liang Yao;Bing Li;Max Q.-H. Meng;Ying Hu","doi":"10.1109/TMRB.2024.3464109","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464109","url":null,"abstract":"Ultrasound (US) examination is widely used to diagnose carotid artery plaque, which requires the sonographer to guide the probe to scan along a specific path for complete coverage of the carotid artery region. Meanwhile, stable probe-neck interaction is important for high-quality image acquisition. In this study, a robotic system for autonomous carotid US scanning is proposed. To realize the autonomous visual servo movement of the probe, an object tracking method based on improved Siamese network is proposed. Meanwhile, a local quality assessment algorithm is proposed to ensure that carotid ultrasound images are clear and desirable for diagnosis. To address the issue of poor probe-neck contact and loss of carotid object during ultrasound scanning, an automatic recovery control method is proposed to ensure the continuity of the scanning process without the need to stop and restart the scanning. Experimental results show that the robotic system can successfully navigate the probe to move along a path that meets the clinical standard. In addition, the robot can autonomously rediscover the object and return to the normal scanning state if a stuck condition occurs.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1436-1447"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464115
Mohammad Mahdi Dalaee;Mohammad Zareinejad;Abdolreza Ohadi;Parsa Kabir
The hand is essential to a human’s daily activities. To rehabilitate patients with hand function disorders, the first step is the extension and flexion of fingers and then regaining the ability to pinch objects. A device suitable for pinch grasping rehabilitation has to be lightweight, small relative to a hand, work in a safe air pressure range, and produce enough range of motion and force. In this paper, a glove with the mentioned characteristics is designed, fabricated, and evaluated. The actuators used in the glove have undergone force and range of motion tests that show promising outputs, such as 3N of force at the tip of the finger, that is enough to pick a 240g object and sufficient range of motion for each joint to perform the box and block test. A model has been developed to be used in designing the device in accordance with the patient’s needs. This model can also be used to identify the stiffness of each knuckle to conduct better rehabilitation methods. The model was verified by being applied to a dummy finger. Furthermore, an actuation pack was developed and evaluated to enable device portability in everyday use conditions.
{"title":"Rehabilitation Glove With Soft Inflatable Actuators for Precision Grasping: Design, Fabrication, Modeling and Preliminary Evaluation","authors":"Mohammad Mahdi Dalaee;Mohammad Zareinejad;Abdolreza Ohadi;Parsa Kabir","doi":"10.1109/TMRB.2024.3464115","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464115","url":null,"abstract":"The hand is essential to a human’s daily activities. To rehabilitate patients with hand function disorders, the first step is the extension and flexion of fingers and then regaining the ability to pinch objects. A device suitable for pinch grasping rehabilitation has to be lightweight, small relative to a hand, work in a safe air pressure range, and produce enough range of motion and force. In this paper, a glove with the mentioned characteristics is designed, fabricated, and evaluated. The actuators used in the glove have undergone force and range of motion tests that show promising outputs, such as 3N of force at the tip of the finger, that is enough to pick a 240g object and sufficient range of motion for each joint to perform the box and block test. A model has been developed to be used in designing the device in accordance with the patient’s needs. This model can also be used to identify the stiffness of each knuckle to conduct better rehabilitation methods. The model was verified by being applied to a dummy finger. Furthermore, an actuation pack was developed and evaluated to enable device portability in everyday use conditions.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1760-1770"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600313","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464123
Colette Abah;Jared P. Lawson;Rohan Chitale;Nabil Simaan
The size limitations and tortuosity of the neurovasculature currently exceed the capabilities of existing robotic systems. Furthermore, safety considerations require a fail-safe design whereby some passive compliance is used for an added layer of safety and for sensing the lateral load on the steerable portion of the catheter. To address these needs, we propose a novel multi-articulated robotic catheter technology that aims to increase technical precision, reduce procedural time and radiation exposure, and enable the semi-automation of catheters during neuroendovascular procedures. This catheter uses joint-level sensing and fluoroscopic imaging to actively bend in two separate planes. Its design also uses series-elastic actuation for increased safety and active compliance (self-steering). We present the design, kinematic modeling, and calibration of this system. A multi-mode real-time control architecture of the system was implemented and experimentally validated. We demonstrate the use of the robotic catheter for branch selection, insertion in an unknown channel under active compliance, and autonomous deployment within a 2D vasculature model. Furthermore, we developed algorithms for intra-operative catheter tracking and pose filtering. Methods presented in this paper make significant strides towards the future goal of enabling semi-autonomous navigation for neuroendovascular procedures.
{"title":"Self-Steering Catheters for Neuroendovascular Interventions","authors":"Colette Abah;Jared P. Lawson;Rohan Chitale;Nabil Simaan","doi":"10.1109/TMRB.2024.3464123","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464123","url":null,"abstract":"The size limitations and tortuosity of the neurovasculature currently exceed the capabilities of existing robotic systems. Furthermore, safety considerations require a fail-safe design whereby some passive compliance is used for an added layer of safety and for sensing the lateral load on the steerable portion of the catheter. To address these needs, we propose a novel multi-articulated robotic catheter technology that aims to increase technical precision, reduce procedural time and radiation exposure, and enable the semi-automation of catheters during neuroendovascular procedures. This catheter uses joint-level sensing and fluoroscopic imaging to actively bend in two separate planes. Its design also uses series-elastic actuation for increased safety and active compliance (self-steering). We present the design, kinematic modeling, and calibration of this system. A multi-mode real-time control architecture of the system was implemented and experimentally validated. We demonstrate the use of the robotic catheter for branch selection, insertion in an unknown channel under active compliance, and autonomous deployment within a 2D vasculature model. Furthermore, we developed algorithms for intra-operative catheter tracking and pose filtering. Methods presented in this paper make significant strides towards the future goal of enabling semi-autonomous navigation for neuroendovascular procedures.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1726-1737"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10684241","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464119
Robert L. McGrath;Fabrizio Sergi
We developed, implemented, and assessed the performance of two forms of plug-in type repetitive controllers (RC) for enhancing the transparency of a lower extremity exoskeleton that operates to support walking function. One controller is a first order RC (SING) consisting of a single period matched to the self-selected cadence of the participant. The second is a novel ‘parallel’ RC (PARA) which consists of a library of integrated RCs with varying periods, intended to accommodate a wider range of gait cycle times. We assessed the effects of both RCs under free cadence walking (FREE) and when walking with a metronome prescribing a consistent cadence matching the participants’ self-selected value. Both conditions were evaluated both at fixed speed and under user-driven treadmill control (UDT), where the treadmill speed was regulated by the user’s anterior/posterior position on the treadmill. The implementation of RC to the knee joint of the ALEX II exoskeleton lead to a significant reduction in torque error of 10-15% at the knee joint during swing and smaller, non-significant effects at the hip joint. While the PARA RC reduced knee torque error more than the SING RC during the FREE cadence condition, a 15% reduction vs. 10% reduction, the difference between the two controllers was not statistically significant. During the UDT sections of walking conditions, participants increased GS under both the SING and PARA RC types. After controlling for the increase in torque error associated with speed, both the PARA and the SING controller reduced TE at the knee joint during swing relative to baseline by 13% and 14%, respectively, with no significant effects to the hip joint. Our work presents a novel formulation of RC and demonstrates the feasibility of applying RC to a robotic exoskeleton joint to assist walking. Future work should be geared toward improving the gait cycle prediction algorithm and developing robust methods for accounting for impact dynamics.
我们开发、实施并评估了两种形式的插入式重复控制器（RC）的性能，以提高下肢外骨骼的透明度，从而支持行走功能。其中一种控制器是一阶 RC（SING），由与参与者自选步频相匹配的单周期组成。第二个控制器是一个新颖的 "并行 "RC（PARA），由不同周期的集成 RC 库组成，旨在适应更广泛的步态周期时间。我们评估了这两种 RC 在自由步频行走（FREE）和使用节拍器行走时的效果，节拍器规定了与参与者自选值一致的步频。两种情况都在固定速度和用户驱动跑步机控制（UTT）下进行了评估，其中跑步机速度由用户在跑步机上的前后位置调节。在 ALEX II 外骨骼的膝关节上安装 RC 后，摆动过程中膝关节的扭矩误差显著减少了 10-15%，而髋关节的影响较小且不显著。在自由步速条件下，PARA RC 比 SING RC 更能减少膝关节扭矩误差，分别减少了 15% 和 10%，但两种控制器之间的差异在统计学上并不显著。在行走条件的 UDT 部分，参与者在 SING 和 PARA RC 类型下都增加了 GS。在控制了与速度相关的扭矩误差增加后，PARA 和 SING 控制器在摆动过程中分别将膝关节的 TE 相对于基线降低了 13% 和 14%，而对髋关节没有明显影响。我们的研究提出了一种新颖的 RC 配方，并证明了将 RC 应用于机器人外骨骼关节以辅助行走的可行性。未来的工作应着眼于改进步态周期预测算法和开发考虑冲击动力学的稳健方法。
{"title":"Repetitive Control of Knee Interaction Torque via a Lower Extremity Exoskeleton for Improved Transparency During Walking","authors":"Robert L. McGrath;Fabrizio Sergi","doi":"10.1109/TMRB.2024.3464119","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464119","url":null,"abstract":"We developed, implemented, and assessed the performance of two forms of plug-in type repetitive controllers (RC) for enhancing the transparency of a lower extremity exoskeleton that operates to support walking function. One controller is a first order RC (SING) consisting of a single period matched to the self-selected cadence of the participant. The second is a novel ‘parallel’ RC (PARA) which consists of a library of integrated RCs with varying periods, intended to accommodate a wider range of gait cycle times. We assessed the effects of both RCs under free cadence walking (FREE) and when walking with a metronome prescribing a consistent cadence matching the participants’ self-selected value. Both conditions were evaluated both at fixed speed and under user-driven treadmill control (UDT), where the treadmill speed was regulated by the user’s anterior/posterior position on the treadmill. The implementation of RC to the knee joint of the ALEX II exoskeleton lead to a significant reduction in torque error of 10-15% at the knee joint during swing and smaller, non-significant effects at the hip joint. While the PARA RC reduced knee torque error more than the SING RC during the FREE cadence condition, a 15% reduction vs. 10% reduction, the difference between the two controllers was not statistically significant. During the UDT sections of walking conditions, participants increased GS under both the SING and PARA RC types. After controlling for the increase in torque error associated with speed, both the PARA and the SING controller reduced TE at the knee joint during swing relative to baseline by 13% and 14%, respectively, with no significant effects to the hip joint. Our work presents a novel formulation of RC and demonstrates the feasibility of applying RC to a robotic exoskeleton joint to assist walking. Future work should be geared toward improving the gait cycle prediction algorithm and developing robust methods for accounting for impact dynamics.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1581-1590"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464093
Sergio A. Pertuz Mendez;Davi De Alencar Mendes;Marta Gherardini;Daniel M. Muñoz;Helon Vicente Hultmann Ayala;Christian Cipriani
Recently myokinetic interfaces have been proposed to exploit magnet tracking for controlling bionic prostheses. This interface derives information about muscle contractions from permanent magnets implanted into the amputee’s forearm muscles. Machine learning models have been mapped on Field Programmable Gate Arrays (FPGAs) to track a single magnet, achieving good precision and computational efficiency, but consuming a large area and hardware resources. To track several magnets, here we propose a novel solution based on dynamic partial reconfiguration, switching three prediction models: a linear regressor, a radial basis function neural network, and a multi-layer perceptron neural network. A system with five magnets and 128 magnetic sensor inputs was used and experimental data were collected to train a system with five hardware predictors. To reduce the complexity of the models, we applied principal component analysis, ranking by correlation the number of inputs of each model. This run-time reconfigurable solution allows the circuits to be reconfigured in order to select the most reliable predictor model for each magnet while the rest of the circuit continues to operate extracting the most significant information from the captured signals. Thus, the proposed solution remarkably reduces the hardware occupation and improves the computational efficiency compared to previous solutions.
{"title":"Dynamic Reconfiguration for Multi-Magnet Tracking in Myokinetic Prosthetic Interfaces","authors":"Sergio A. Pertuz Mendez;Davi De Alencar Mendes;Marta Gherardini;Daniel M. Muñoz;Helon Vicente Hultmann Ayala;Christian Cipriani","doi":"10.1109/TMRB.2024.3464093","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464093","url":null,"abstract":"Recently myokinetic interfaces have been proposed to exploit magnet tracking for controlling bionic prostheses. This interface derives information about muscle contractions from permanent magnets implanted into the amputee’s forearm muscles. Machine learning models have been mapped on Field Programmable Gate Arrays (FPGAs) to track a single magnet, achieving good precision and computational efficiency, but consuming a large area and hardware resources. To track several magnets, here we propose a novel solution based on dynamic partial reconfiguration, switching three prediction models: a linear regressor, a radial basis function neural network, and a multi-layer perceptron neural network. A system with five magnets and 128 magnetic sensor inputs was used and experimental data were collected to train a system with five hardware predictors. To reduce the complexity of the models, we applied principal component analysis, ranking by correlation the number of inputs of each model. This run-time reconfigurable solution allows the circuits to be reconfigured in order to select the most reliable predictor model for each magnet while the rest of the circuit continues to operate extracting the most significant information from the captured signals. Thus, the proposed solution remarkably reduces the hardware occupation and improves the computational efficiency compared to previous solutions.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1678-1687"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/TMRB.2024.3464106
Jiajing Zhang;Wenqing Zhang;Haodong Liu;Yingying Liu;Ningning Chen;Jianjia Zhang;Changhong Wang
Robotic-assisted implantation of screw-rod systems serves as an advanced therapy for spinal diseases. A precise curvature fit between rods and spines is critical to postoperative spinal stability. Currently, rod curvature is determined intraoperatively to accommodate screw positions, which is hardly conducive to optimal rod bending and is vulnerable to surgeons’ expertise. To address this challenge, we propose an automated and efficient method for measuring the centroid angle to guide preoperative rod design from CT images. The centroid angle is defined by lines connecting centroids of the upper and lower vertebrae pairs, providing a reliable measurement for spinal deformities. The proposed pipeline includes (1) 3D spine segmentation with multiscale multitask deep learning; (2) vertebrae recognition using graphical morphology; (3) automatic and reproducible centroid angle measurement. Our method is evaluated on both healthy and pathological spines. Compared to manual measurements by professional surgeons, our method achieves an accuracy of 94.50% and 91.93% on adjacent and non-adjacent vertebrae, respectively. A Slicer-based plugin for robotic-assisted screw-rod systems implantation is built, providing a new clinical tool to personalize screw-rod systems consistent with the natural spinal curvature, thereby enhancing biomechanical properties.
{"title":"Automatic Centroid Angle Measurement From CT Image for Preoperative Rod Design of Robotic-Assisted Screw-Rod System Implantation","authors":"Jiajing Zhang;Wenqing Zhang;Haodong Liu;Yingying Liu;Ningning Chen;Jianjia Zhang;Changhong Wang","doi":"10.1109/TMRB.2024.3464106","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3464106","url":null,"abstract":"Robotic-assisted implantation of screw-rod systems serves as an advanced therapy for spinal diseases. A precise curvature fit between rods and spines is critical to postoperative spinal stability. Currently, rod curvature is determined intraoperatively to accommodate screw positions, which is hardly conducive to optimal rod bending and is vulnerable to surgeons’ expertise. To address this challenge, we propose an automated and efficient method for measuring the centroid angle to guide preoperative rod design from CT images. The centroid angle is defined by lines connecting centroids of the upper and lower vertebrae pairs, providing a reliable measurement for spinal deformities. The proposed pipeline includes (1) 3D spine segmentation with multiscale multitask deep learning; (2) vertebrae recognition using graphical morphology; (3) automatic and reproducible centroid angle measurement. Our method is evaluated on both healthy and pathological spines. Compared to manual measurements by professional surgeons, our method achieves an accuracy of 94.50% and 91.93% on adjacent and non-adjacent vertebrae, respectively. A Slicer-based plugin for robotic-assisted screw-rod systems implantation is built, providing a new clinical tool to personalize screw-rod systems consistent with the natural spinal curvature, thereby enhancing biomechanical properties.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 4","pages":"1478-1489"},"PeriodicalIF":3.4,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600348","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：