Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661506
Andrew Lewis, Chen Gong, Yaxuan Zhou, Pengcheng Chen, Michael P. Porter, B. Hannaford, E. Seibel
Telecystoscopy can lower the barrier to access of critical urologic diagnostics for patients around the world. A challenge to robotic control of flexible cystoscopes and intuitive teleoperation is estimation of the pose of the scope tip. We demonstrate real-time localization using video recordings from prior cystoscopies and 3D reconstructions of the patient’s bladder to estimate cystoscope angulation We map prior video data into a low dimensional space as a dictionary so that new images can be matched to a nearest neighbor. The cystoscope position is estimated by the current image’s relationship to a matched dictionary frame. Frequent cystoscopies are necessary in post-treatment surveillance of bladder cancer patients, and video from previous procedures may be available during telecystoscopy. A cystoscope with servo-controlled angulation was inserted into a 2D + height bladder shape with a panorama of a urothelium on to the inside. Scans of the surface were performed with: varying speeds; different fields of view; and bladder tumors inserted into the panorama physically and digitally. Videos were used to create 3D reconstructions, dictionary sets, and test data sets for analyzing our algorithm’s computational efficiency and accuracy compared with a SIFT-only localization. Our algorithm found a nearest neighbor image in 96-100% of frames in under 60ms per image compared to SIFT’s ability to find an image match in 56-84% of frames in more than 6000ms per image. Our algorithm, with a first stage rate of nearly 20 Hz, is a promising tool for real-time estimation of tip location in robotic cystoscopy when prior cystoscopy images are available.
{"title":"Real Time Localization of Cystoscope Angulation in 2D Bladder Phantom for Telecystoscopy","authors":"Andrew Lewis, Chen Gong, Yaxuan Zhou, Pengcheng Chen, Michael P. Porter, B. Hannaford, E. Seibel","doi":"10.1109/ismr48346.2021.9661506","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661506","url":null,"abstract":"Telecystoscopy can lower the barrier to access of critical urologic diagnostics for patients around the world. A challenge to robotic control of flexible cystoscopes and intuitive teleoperation is estimation of the pose of the scope tip. We demonstrate real-time localization using video recordings from prior cystoscopies and 3D reconstructions of the patient’s bladder to estimate cystoscope angulation We map prior video data into a low dimensional space as a dictionary so that new images can be matched to a nearest neighbor. The cystoscope position is estimated by the current image’s relationship to a matched dictionary frame. Frequent cystoscopies are necessary in post-treatment surveillance of bladder cancer patients, and video from previous procedures may be available during telecystoscopy. A cystoscope with servo-controlled angulation was inserted into a 2D + height bladder shape with a panorama of a urothelium on to the inside. Scans of the surface were performed with: varying speeds; different fields of view; and bladder tumors inserted into the panorama physically and digitally. Videos were used to create 3D reconstructions, dictionary sets, and test data sets for analyzing our algorithm’s computational efficiency and accuracy compared with a SIFT-only localization. Our algorithm found a nearest neighbor image in 96-100% of frames in under 60ms per image compared to SIFT’s ability to find an image match in 56-84% of frames in more than 6000ms per image. Our algorithm, with a first stage rate of nearly 20 Hz, is a promising tool for real-time estimation of tip location in robotic cystoscopy when prior cystoscopy images are available.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132908075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661574
C. Abah, R. Chitale, N. Simaan
Endovascular intervention for ischemic stroke requires dexterous manipulation of catheters and guidewires. Robot-assisted steerable microcatheters can facilitate navigation along tortuous vasculature and bifurcation selection, thereby reducing the surgical skill demands. To achieve effective catheter deployment, the kinematic parameters of the catheter should ideally be selected to minimize the passive deflection of a given catheter. This paper presents a first-step towards automated selection of catheter parameters for traversal of a target anatomical vessel. The image-segmentation and vessel skeletonisation are presented along with a kinematic model of an antagonistic pair two-segment continuum robot. The nonlinear kinematic model is captured using a sparse representation achieved via a Krönecker product solution to a least-squares formulation of a matrix equation. The path planning is cast as a nonlinear parameter optimization that includes the lengths of the segments and their relative angles. The initial results suggest the possible utility of this approach for the development of a library of catheters that may be designed a-priori for anatomical regions while taking into account across-patient anatomical variabilities. For clinical-deployment, the same methods can be used to select the best catheter candidate from a library of catheters.
{"title":"Image-Guided Optimization of Robotic Catheters for Patient-Specific Endovascular Intervention","authors":"C. Abah, R. Chitale, N. Simaan","doi":"10.1109/ismr48346.2021.9661574","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661574","url":null,"abstract":"Endovascular intervention for ischemic stroke requires dexterous manipulation of catheters and guidewires. Robot-assisted steerable microcatheters can facilitate navigation along tortuous vasculature and bifurcation selection, thereby reducing the surgical skill demands. To achieve effective catheter deployment, the kinematic parameters of the catheter should ideally be selected to minimize the passive deflection of a given catheter. This paper presents a first-step towards automated selection of catheter parameters for traversal of a target anatomical vessel. The image-segmentation and vessel skeletonisation are presented along with a kinematic model of an antagonistic pair two-segment continuum robot. The nonlinear kinematic model is captured using a sparse representation achieved via a Krönecker product solution to a least-squares formulation of a matrix equation. The path planning is cast as a nonlinear parameter optimization that includes the lengths of the segments and their relative angles. The initial results suggest the possible utility of this approach for the development of a library of catheters that may be designed a-priori for anatomical regions while taking into account across-patient anatomical variabilities. For clinical-deployment, the same methods can be used to select the best catheter candidate from a library of catheters.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114619075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661578
Deepak Raina, Hardeep Singh, S. Saha, Chetan Arora, A. Agarwal, S. Chandrashekhara, K. Rangarajan, Suvayan Nandi
During the COVID-19 pandemic, the lives of healthcare professionals are at significant threat because of the enormous workload and cross-infection risk. Ultrasound (US) imaging plays a vital role in the diagnosis and follow-up of COVID-19 patients; however, it requires a close-physical contact by the sonographer. In this context, this paper presents a Telerobotic Ultrasound (TR-US) system for complete remote control of the US probe, thereby preventing direct physical contact between patients and sonographers. The system consists of a 6-DOF robot arm at the remote site and a haptic device at the doctor’s site. The control architecture precisely transmits the intended position and orientation of the US probe to the remote location for transversal and sagittal plane scanning. This architecture, when integrated with an admittance controller-based force modulation and feedback transmission, enables the radiologists to obtain high-quality images for diagnosis. The advantages and effectiveness of the system are demonstrated by conducting in-vivo feasibility study at AIIMS, Delhi, for imaging abdomen organs (liver, spleen, kidneys, bladders). The system provides image quality equivalent to a manually-guided probe, can identify various pathology and reports high acceptability among volunteers and doctors from a questionnaire survey.
{"title":"Comprehensive Telerobotic Ultrasound System for Abdominal Development and in-vivo Feasibility Study","authors":"Deepak Raina, Hardeep Singh, S. Saha, Chetan Arora, A. Agarwal, S. Chandrashekhara, K. Rangarajan, Suvayan Nandi","doi":"10.1109/ismr48346.2021.9661578","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661578","url":null,"abstract":"During the COVID-19 pandemic, the lives of healthcare professionals are at significant threat because of the enormous workload and cross-infection risk. Ultrasound (US) imaging plays a vital role in the diagnosis and follow-up of COVID-19 patients; however, it requires a close-physical contact by the sonographer. In this context, this paper presents a Telerobotic Ultrasound (TR-US) system for complete remote control of the US probe, thereby preventing direct physical contact between patients and sonographers. The system consists of a 6-DOF robot arm at the remote site and a haptic device at the doctor’s site. The control architecture precisely transmits the intended position and orientation of the US probe to the remote location for transversal and sagittal plane scanning. This architecture, when integrated with an admittance controller-based force modulation and feedback transmission, enables the radiologists to obtain high-quality images for diagnosis. The advantages and effectiveness of the system are demonstrated by conducting in-vivo feasibility study at AIIMS, Delhi, for imaging abdomen organs (liver, spleen, kidneys, bladders). The system provides image quality equivalent to a manually-guided probe, can identify various pathology and reports high acceptability among volunteers and doctors from a questionnaire survey.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127338766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Surgical robots have some important advantages in minimally invasive surgeries. However, conventional surgical robots mostly choose different operation modes or methods from what surgeons are used to, and the robotic surgery procedure itself also generate extra injuries on patients. To solve such problems and enhance clinical applications of surgical robots, we put forward the concept of surgeon and patient-friendly surgical robot as well as criteria for evaluating the usefulness of a surgical robot according to such concept. Here, we present how a surgeon and patient-friendly orthopedic surgical robot was developed by imitating surgeons’ manually conducting a fracture reduction surgery and keeping surgeons’ way of thinking and planning the surgery. Experiment results show that this robot can complete complicated fracture reduction surgeries by combining the strengths of the robot and the surgeons but with no extra injuries on patients.
{"title":"Development of a Surgeon and Patient-Friendly Orthopedic Surgical Robot","authors":"Shijie Zhu, Zhe Zhao, Yu Chen, Jiuzheng Deng, Jianjin Zhu, Yongwei Pan, G. Zheng","doi":"10.1109/ismr48346.2021.9661481","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661481","url":null,"abstract":"Surgical robots have some important advantages in minimally invasive surgeries. However, conventional surgical robots mostly choose different operation modes or methods from what surgeons are used to, and the robotic surgery procedure itself also generate extra injuries on patients. To solve such problems and enhance clinical applications of surgical robots, we put forward the concept of surgeon and patient-friendly surgical robot as well as criteria for evaluating the usefulness of a surgical robot according to such concept. Here, we present how a surgeon and patient-friendly orthopedic surgical robot was developed by imitating surgeons’ manually conducting a fracture reduction surgery and keeping surgeons’ way of thinking and planning the surgery. Experiment results show that this robot can complete complicated fracture reduction surgeries by combining the strengths of the robot and the surgeons but with no extra injuries on patients.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132745135","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661548
P. Sherpa, David Quintero
A challenge for lower-limb powered prostheses is developing a seamless control strategy to perform multiple locomotion tasks, such as changes in walking speed. Generally, powered prostheses implement different independent controllers that correspond to a specific task that each contain their own patient-specific control parameters to tune. This paper presents an online parameterize method of providing desired joint kinematic trajectories for a powered knee-ankle prosthesis controller to perform continuously smooth kinematic transitions unified across the gait cycle for level-ground activity. An active Catmull-Rom spline model generates the online desired knee and ankle joint trajectories as a virtual constraint controller that is a function of a phase variable and human desired speed. An offline optimization routine was implemented to produce optimal control point locations for the Catmull-Rom spline model to give transit across different kinematic walking speeds in a continuous manner. Results demonstrate speed adaptation for different walking speeds (i.e., slow, normal, and fast) as well as running to show versatility towards an adaptive unified virtual constraint control.
{"title":"A Unified Control Framework with Continuous Speed Adaptation used for Powered Prostheses Control","authors":"P. Sherpa, David Quintero","doi":"10.1109/ismr48346.2021.9661548","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661548","url":null,"abstract":"A challenge for lower-limb powered prostheses is developing a seamless control strategy to perform multiple locomotion tasks, such as changes in walking speed. Generally, powered prostheses implement different independent controllers that correspond to a specific task that each contain their own patient-specific control parameters to tune. This paper presents an online parameterize method of providing desired joint kinematic trajectories for a powered knee-ankle prosthesis controller to perform continuously smooth kinematic transitions unified across the gait cycle for level-ground activity. An active Catmull-Rom spline model generates the online desired knee and ankle joint trajectories as a virtual constraint controller that is a function of a phase variable and human desired speed. An offline optimization routine was implemented to produce optimal control point locations for the Catmull-Rom spline model to give transit across different kinematic walking speeds in a continuous manner. Results demonstrate speed adaptation for different walking speeds (i.e., slow, normal, and fast) as well as running to show versatility towards an adaptive unified virtual constraint control.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132303846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661497
Lorin Fasel, N. Gerig, P. Cattin, G. Rauter
When performing minimally invasive surgeries, surgeons are currently restricted by the rigidity and limited maneuverability of their tools. The tools could be extended by joints to provide additional degrees of freedom. However, manually controlling the movement of distal joints is challenging since the effective interaction forces at the tip are difficult to feel. Therefore, manipulation of distal joints to increase the maneuverability can lead to additional risks for harming the patient. To overcome limited maneuverability while providing inherent safety, we propose a novel concept for minimally invasive tool actuation based on the principle of series elastic actuation. In previous work, we showed successful torque control of an articulated robotic endoscope. In this paper, we extended torque control by high-level position control. We evaluated the position control experimentally for the case of a telemanipulated joint as well as for automatic target following. Automatic target following was achieved with visual servoing, i.e., an image stream from a miniature camera was processed to compute the joint position setpoint. The results showed that accurate and stable position control is feasible with an actuation based on series elastic actuation. Compared to traditional robotic endoscope actuation, which is designed to be as stiff as possible, our approach reduced impact forces and allowed to set the torque limit in the joint as desired. Therefore, torques exerted by the endoscope joint to adjacent structures can be kept within desired limits.
{"title":"The SEA-Scope: Torque-limited endoscopic joint control for telemanipulation or visual servoing through tendon force control with series elastic actuation","authors":"Lorin Fasel, N. Gerig, P. Cattin, G. Rauter","doi":"10.1109/ismr48346.2021.9661497","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661497","url":null,"abstract":"When performing minimally invasive surgeries, surgeons are currently restricted by the rigidity and limited maneuverability of their tools. The tools could be extended by joints to provide additional degrees of freedom. However, manually controlling the movement of distal joints is challenging since the effective interaction forces at the tip are difficult to feel. Therefore, manipulation of distal joints to increase the maneuverability can lead to additional risks for harming the patient. To overcome limited maneuverability while providing inherent safety, we propose a novel concept for minimally invasive tool actuation based on the principle of series elastic actuation. In previous work, we showed successful torque control of an articulated robotic endoscope. In this paper, we extended torque control by high-level position control. We evaluated the position control experimentally for the case of a telemanipulated joint as well as for automatic target following. Automatic target following was achieved with visual servoing, i.e., an image stream from a miniature camera was processed to compute the joint position setpoint. The results showed that accurate and stable position control is feasible with an actuation based on series elastic actuation. Compared to traditional robotic endoscope actuation, which is designed to be as stiff as possible, our approach reduced impact forces and allowed to set the torque limit in the joint as desired. Therefore, torques exerted by the endoscope joint to adjacent structures can be kept within desired limits.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127737376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661519
Kevin Huang, Digesh Chitrakar, Wenfan Jiang, Yun-Hsuan Su
This paper presents an approach to enhanced endoscopic tool segmentation combining separate pathways utilizing input images in two different coordinate representations. The proposed method examines U-Net convolutional neural networks with input endoscopic images represented via (1) the original rectangular coordinate format alongside (2) a morphological polar coordinate transformation. To maximize information and the breadth of the endoscope frustrum, imaging sensors are oftentimes larger than the image circle. This results in unused border regions. Ideally, the region of interest is proximal to the image center. The above two observations formed the basis for the morphological polar transformation pathway as an augmentation to typical rectangular input image representations. Results indicate that neither of the two investigated coordinate representations consistently yielded better segmentation performance as compared to the other. Improved segmentation can be achieved with a hybrid approach that carefully selects which of the two pathways to be used for individual input images. Towards that end, two binary classifiers were trained to identify, given an input endoscopic image, which of the two coordinate representation segmentation pathways (rectangular or polar), would result in better segmentation performance. Results are promising and suggest marked improvements using a hybrid pathway selection approach compared to either alone. The experiment used to evaluate the proposed hybrid method utilized a dataset consisting of 8360 endoscopic images from real surgery and evaluated segmentation performance with Dice coefficient and Intersection over Union. The results suggest that on-the-fly polar transformation for tool segmentation is useful when paired with the proposed hybrid tool-segmentation approach.
本文提出了一种增强内窥镜工具分割的方法，该方法结合了使用两种不同坐标表示的输入图像的单独路径。该方法通过(1)原始直角坐标格式和(2)形态极坐标变换来检测U-Net卷积神经网络。为了最大限度地获得信息和内窥镜挫折镜的宽度，成像传感器通常比图像圆大。这导致未使用的边界区域。理想情况下，感兴趣的区域靠近图像中心。以上两个观察结果构成了形态极性转换路径的基础，作为对典型矩形输入图像表示的增强。结果表明，两种调查的坐标表示都没有一致地产生更好的分割性能。改进的分割可以通过一种混合方法来实现，这种方法可以仔细选择用于单个输入图像的两条路径中的哪一条。为此，我们训练了两个二值分类器，在给定输入的内窥镜图像中，识别两种坐标表示分割路径(矩形或极坐标)中哪一种会产生更好的分割性能。结果是有希望的，并且表明使用混合途径选择方法与单独使用任何一种方法相比有显著的改进。实验利用8360张真实手术内窥镜图像数据集，利用Dice系数和Intersection over Union对分割效果进行评价。结果表明，当与所提出的混合工具分割方法配对时，动态极性变换工具分割是有用的。
{"title":"Enhanced U-Net Tool Segmentation using Hybrid Coordinate Representations of Endoscopic Images","authors":"Kevin Huang, Digesh Chitrakar, Wenfan Jiang, Yun-Hsuan Su","doi":"10.1109/ismr48346.2021.9661519","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661519","url":null,"abstract":"This paper presents an approach to enhanced endoscopic tool segmentation combining separate pathways utilizing input images in two different coordinate representations. The proposed method examines U-Net convolutional neural networks with input endoscopic images represented via (1) the original rectangular coordinate format alongside (2) a morphological polar coordinate transformation. To maximize information and the breadth of the endoscope frustrum, imaging sensors are oftentimes larger than the image circle. This results in unused border regions. Ideally, the region of interest is proximal to the image center. The above two observations formed the basis for the morphological polar transformation pathway as an augmentation to typical rectangular input image representations. Results indicate that neither of the two investigated coordinate representations consistently yielded better segmentation performance as compared to the other. Improved segmentation can be achieved with a hybrid approach that carefully selects which of the two pathways to be used for individual input images. Towards that end, two binary classifiers were trained to identify, given an input endoscopic image, which of the two coordinate representation segmentation pathways (rectangular or polar), would result in better segmentation performance. Results are promising and suggest marked improvements using a hybrid pathway selection approach compared to either alone. The experiment used to evaluate the proposed hybrid method utilized a dataset consisting of 8360 endoscopic images from real surgery and evaluated segmentation performance with Dice coefficient and Intersection over Union. The results suggest that on-the-fly polar transformation for tool segmentation is useful when paired with the proposed hybrid tool-segmentation approach.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129045319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661495
Ruixuan Li, K. Niu, E. V. Poorten
Ultrasound (US) has been increasingly used as medical imaging technology across various clinical diagnostic and therapeutic scenarios thanks to its availability and non-radiative nature. While 3D US probes are becoming available, most systems are still using 2D probes. For 3D US reconstruction based on 2D probes, US image calibration forms an essential step. Through calibration, one can find the transformation matrix between a coordinate frame attached to an optical marker or the robot’s end effector towards the coordinate frame of the US probe. Current US calibration methods usually require hereto lengthy free hand gestures as well as some manual interventions, which hampers the use and integration with advanced robotic systems. This paper introduces a reliable automatic calibration framework that is also fast. Demonstrated on a KUKA lightweight robot and 2D US probe, the full calibration procedure was completed in 224.8 seconds with a 1.29 mm mean 3D localization error. Within this procedure, camera-to-robot calibration was accomplished within only 47 seconds and reached a 0.17 mm mean error. Validation of the US image calibration was done through 3D printed model, leading to a mean deviation of 1.05 mm from the respective CAD models.
{"title":"A Framework for Fast Automatic Robot Ultrasound Calibration","authors":"Ruixuan Li, K. Niu, E. V. Poorten","doi":"10.1109/ismr48346.2021.9661495","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661495","url":null,"abstract":"Ultrasound (US) has been increasingly used as medical imaging technology across various clinical diagnostic and therapeutic scenarios thanks to its availability and non-radiative nature. While 3D US probes are becoming available, most systems are still using 2D probes. For 3D US reconstruction based on 2D probes, US image calibration forms an essential step. Through calibration, one can find the transformation matrix between a coordinate frame attached to an optical marker or the robot’s end effector towards the coordinate frame of the US probe. Current US calibration methods usually require hereto lengthy free hand gestures as well as some manual interventions, which hampers the use and integration with advanced robotic systems. This paper introduces a reliable automatic calibration framework that is also fast. Demonstrated on a KUKA lightweight robot and 2D US probe, the full calibration procedure was completed in 224.8 seconds with a 1.29 mm mean 3D localization error. Within this procedure, camera-to-robot calibration was accomplished within only 47 seconds and reached a 0.17 mm mean error. Validation of the US image calibration was done through 3D printed model, leading to a mean deviation of 1.05 mm from the respective CAD models.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127850967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661549
A. Sayadi, Hamid Reza Nourani, M. Jolaei, J. Dargahi, Amir Hooshiar
Monitoring and control of the contact force at the tip of soft flexural robots is of high application need, e.g., the tip force on radiofrequency ablation (RFA) catheters. In this study, a real-time tip force estimation method based on image-based shape-sensing and learning-from-simulation is provided. To this end, a generalized image-based shape-sensing technique for flexural robots was developed using the Bezier spline interpolation method. Afterward, the deflection of a commercial catheter subjected to a series of tip forces was simulated using nonlinear finite element modeling. Next, two independent data-driven models, i.e., artificial neural network (ANN) and support vector regression (SVR), were trained with a dataset with the Bezier spline control points as the inputs and tip forces as the output. For validation, the trained models were used for real-time tip force estimation while the catheter was pressed against porcine atrial tissue. The test was performed using a universal testing machine that recorded the ground-truth contact force. The comparison showed that the ANN model had a mean-absolute-error of 0.0217±0.0191 N, while the SVR model exhibited a mean absolute error of 0.0178 ± 0.0121 N and a correlation coefficient of 0.991. Moreover, the proposed method showed a minimum computational refresh rate of 646 Hz (ANN) and 917 Hz (SVR) during the validation experiment. The performance of the proposed method was in compliance with the clinical requirements of RFA therapy.
{"title":"Force Estimation on Steerable Catheters through Learning-from-Simulation with ex-vivo Validation*","authors":"A. Sayadi, Hamid Reza Nourani, M. Jolaei, J. Dargahi, Amir Hooshiar","doi":"10.1109/ismr48346.2021.9661549","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661549","url":null,"abstract":"Monitoring and control of the contact force at the tip of soft flexural robots is of high application need, e.g., the tip force on radiofrequency ablation (RFA) catheters. In this study, a real-time tip force estimation method based on image-based shape-sensing and learning-from-simulation is provided. To this end, a generalized image-based shape-sensing technique for flexural robots was developed using the Bezier spline interpolation method. Afterward, the deflection of a commercial catheter subjected to a series of tip forces was simulated using nonlinear finite element modeling. Next, two independent data-driven models, i.e., artificial neural network (ANN) and support vector regression (SVR), were trained with a dataset with the Bezier spline control points as the inputs and tip forces as the output. For validation, the trained models were used for real-time tip force estimation while the catheter was pressed against porcine atrial tissue. The test was performed using a universal testing machine that recorded the ground-truth contact force. The comparison showed that the ANN model had a mean-absolute-error of 0.0217±0.0191 N, while the SVR model exhibited a mean absolute error of 0.0178 ± 0.0121 N and a correlation coefficient of 0.991. Moreover, the proposed method showed a minimum computational refresh rate of 646 Hz (ANN) and 917 Hz (SVR) during the validation experiment. The performance of the proposed method was in compliance with the clinical requirements of RFA therapy.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123423975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-11-17DOI: 10.1109/ismr48346.2021.9661485
Kaitlin G. Rabe, Nicholas P. Fey
The advancement of robotic lower-limb assistive devices has heightened the need for accurate and continuous sensing of user intent. Surface electromyography (EMG) has been extensively used to sense muscles, and estimate locomotion modes and limb motion. Recently, sonomyography has also been investigated as a novel sensing modality. However, the fusion of multiple sensing modalities has not been explored for the continuous prediction of multiple degrees-of-freedom of the lower limb, and during multiple ambulation tasks. In the present study, nine able-bodied subjects completed level, incline, decline, stair ascent, and stair descent tasks. Motion capture data was collected during each task, as well as data from a portable ultrasound transducer (aligned in a transverse orientation) on the anterior thigh and surface EMG sensors on eight lower-limb muscles. Subject-dependent, task-independent Gaussian process regression models were implemented for continuous prediction of knee and ankle angle and angular velocity during these ambulation tasks using three feature sets: (1) surface EMG, (2) sonomyography, and (3) sensor fusion of EMG with sonomyography. Surprisingly, there were no significant differences between sonomyography and sensor fusion-based prediction of knee or ankle angle and angular velocity during all tasks. However, sonomyography and sensor fusion resulted in reduced root mean square error of knee angle prediction during all ambulation tasks and knee angular velocity prediction during most ambulation tasks compared to surface EMG. Sensor fusion improved ankle angle prediction for all walking tasks except stair ascent in comparison to surface EMG. Ankle angular velocity prediction resulted in the lowest performance, overall.Clinical Relevance—This work compares the combination of surface electromyography and sonomyography, and each modality in isolation, for the continuous prediction of kinematics of the knee and ankle during widely-varying ambulatory tasks.
{"title":"Continuous Prediction of Leg Kinematics During Ambulation using Peripheral Sensing of Muscle Activity and Morphology","authors":"Kaitlin G. Rabe, Nicholas P. Fey","doi":"10.1109/ismr48346.2021.9661485","DOIUrl":"https://doi.org/10.1109/ismr48346.2021.9661485","url":null,"abstract":"The advancement of robotic lower-limb assistive devices has heightened the need for accurate and continuous sensing of user intent. Surface electromyography (EMG) has been extensively used to sense muscles, and estimate locomotion modes and limb motion. Recently, sonomyography has also been investigated as a novel sensing modality. However, the fusion of multiple sensing modalities has not been explored for the continuous prediction of multiple degrees-of-freedom of the lower limb, and during multiple ambulation tasks. In the present study, nine able-bodied subjects completed level, incline, decline, stair ascent, and stair descent tasks. Motion capture data was collected during each task, as well as data from a portable ultrasound transducer (aligned in a transverse orientation) on the anterior thigh and surface EMG sensors on eight lower-limb muscles. Subject-dependent, task-independent Gaussian process regression models were implemented for continuous prediction of knee and ankle angle and angular velocity during these ambulation tasks using three feature sets: (1) surface EMG, (2) sonomyography, and (3) sensor fusion of EMG with sonomyography. Surprisingly, there were no significant differences between sonomyography and sensor fusion-based prediction of knee or ankle angle and angular velocity during all tasks. However, sonomyography and sensor fusion resulted in reduced root mean square error of knee angle prediction during all ambulation tasks and knee angular velocity prediction during most ambulation tasks compared to surface EMG. Sensor fusion improved ankle angle prediction for all walking tasks except stair ascent in comparison to surface EMG. Ankle angular velocity prediction resulted in the lowest performance, overall.Clinical Relevance—This work compares the combination of surface electromyography and sonomyography, and each modality in isolation, for the continuous prediction of kinematics of the knee and ankle during widely-varying ambulatory tasks.","PeriodicalId":405817,"journal":{"name":"2021 International Symposium on Medical Robotics (ISMR)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2021-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125255912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：