Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems最新文献

Retinal vein cannulation (RVC) is a minimally invasive microsurgical procedure for treating retinal vein occlusion (RVO), a leading cause of vision impairment. However, the small size and fragility of retinal veins, coupled with the need for high-precision, tremor-free needle manipulation, create significant technical challenges. These limitations highlight the need for robotic assistance to improve accuracy and stability. This study presents an automated robotic system with a top-down microscope and B-scan optical coherence tomography (OCT) imaging for precise depth sensing. Deep learning-based models enable real-time needle navigation, contact detection, and vein puncture recognition, using a chicken embryo model as a surrogate for human retinal veins. The system autonomously detects needle position and puncture events with 85% accuracy. The experiments demonstrate notable reductions in navigation and puncture times compared to manual methods. Our results demonstrate the potential of integrating advanced imaging and deep learning to automate microsurgical tasks, providing a pathway for safer and more reliable RVC procedures with enhanced precision and reproducibility.

In diagnosing and treating prostate cancer the flexible bevel tip needle insertion surgical technique is commonly used. Bevel tip needles experience asymmetric loading on the needle's tip, inducing natural bending of the needle and enabling control mechanisms for precise placement of the needle during surgery. Several methods leverage the needles natural bending to provide autonomous control of needle insertion for accurate needle placement in an effort to reduce excess tissue damage and improve patient outcomes from needle insertion intraventions. Moreover, control methods using lateral deflection of the needle intra-operatively to steer the needle during insertion have been studied and have shown promising results. Thus, to enable these autonomous control methods, real-time, intra-operative shape-sensing feedback is pivotal for optimal performance of the needle insertion control. This work presents an extension of our proven Lie-group theoretic shape-sensing model to handle lateral deflection of the needle during needle insertion and validate this extension with robotic needle insertions in phantom tissue using stereo vision as a ground truth. Furthermore, the system configuration for real-time shape-sensing is implemented using ROS 2, demonstrating average feedback frequency of 15 ± 8 Hz. Average needle shape errors realized from this extension under 1 mm, validating the shape-sensing models' extension.

Satellite images provide an effective way to observe the earth surface on a large scale. 3D landscape models can provide critical structural information, such as forestry and crop growth. However, there has been very limited research to estimate the depth and the 3D models of the earth based on satellite images. LiDAR measurements on satellites are usually quite sparse. RGB images have higher resolution than LiDAR, but there has been little research on 3D surface measurements based on satellite RGB images. In comparison with in-situ sensing, satellite RGB images are usually low resolution. In this research, we explore the method that can enhance the satellite image resolution to generate super-resolution images and then conduct depth estimation and 3D reconstruction based on higher-resolution satellite images. Leveraging the strong generation capability of diffusion models, we developed a simultaneous diffusion model learning framework that can train diffusion models for both super-resolution images and depth estimation. With the super-resolution images and the corresponding depth maps, 3D surface reconstruction models with detailed landscape information can be generated. We evaluated the proposed methodology on multiple satellite datasets for both super-resolution and depth estimation tasks, which have demonstrated the effectiveness of our methodology.

Obstacle detection and tracking represent a critical component in robot autonomous navigation. In this paper, we propose ODTFormer, a Transformer-based model that addresses both obstacle detection and tracking problems. For the detection task, our approach leverages deformable attention to construct a 3D cost volume, which is decoded progressively in the form of voxel occupancy grids. We further track the obstacles by matching the voxels between consecutive frames. The entire model can be optimized in an end-to-end manner. Through extensive experiments on DrivingStereo and KITTI benchmarks, our model achieves state-of-the-art performance in the obstacle detection task. We also report comparable accuracy to state-of-the-art obstacle tracking models while requiring only a fraction of their computation cost, typically ten-fold to twenty-fold less. Our code is available on https://github.com/neu-vi/ODTFormer.

In soft tissue surgeries, such as tumor resections, achieving precision is of utmost importance. Surgeons conventionally achieve this precision through intraoperative adjustments to the cutting plan, responding to deformations from tool-tissue interactions. This study examines the integration of physics-based tissue cutting simulations into autonomous robotic surgery to preoperatively predict and compensate for such deformations, aiming to improve surgical precision and reduce the necessity for dynamic adjustments during autonomous surgeries. This study adapts a real-to-sim-to-real workflow. Initially, the Autonomous System for Tumor Resection (ASTR) was employed to evaluate its accuracy in performing preoperatively intended incisions along the irregular contours of porcine tongue pseudotumors. Following this, a finite element analysis-based simulation, utilizing the Simulation Open Framework Architecture (SOFA), was developed and tuned to accurately mimic these tissue and incision interactions. Insights gained from this simulation were applied to refine the robot's path planning, ensuring a closer alignment of actual incisions with the initially intended surgical plan. The efficacy of this approach was validated by comparing surface incision precision on ex vivo porcine tongues, with the average absolute error reducing from 1.73mm to 1.46mm after applying simulation-driven path adjustments (p < 0.001). Additionally, our method not only demonstrated improvements in maintaining the intended cutting shapes and locations, with shape matching scores using Hu moments enhancing from 0.10 to 0.06 and centroid shifts decreasing from 2.09mm to 1.33mm, but it also potentially reduced the likelihood of adverse oncologic outcomes by preventing clinically suggested excessively close margins of 2.2mm. This feasibility study suggests that merging physics-based cutting simulations with autonomous robotic surgery could potentially lead to more accurate incisions.

We propose MetaEMG, a meta-learning approach for fast adaptation in intent inferral on a robotic hand orthosis for stroke. One key challenge in machine learning for assistive and rehabilitative robotics with disabled-bodied subjects is the difficulty of collecting labeled training data. Muscle tone and spasticity often vary significantly among stroke subjects, and hand function can even change across different use sessions of the device for the same subject. We investigate the use of meta-learning to mitigate the burden of data collection needed to adapt high-capacity neural networks to a new session or subject. Our experiments on real clinical data collected from five stroke subjects show that MetaEMG can improve the intent inferral accuracy with a small session- or subject-specific dataset and very few fine-tuning epochs. To the best of our knowledge, we are the first to formulate intent inferral on stroke subjects as a meta-learning problem and demonstrate fast adaptation to a new session or subject for controlling a robotic hand orthosis with EMG signals.

Thermal ablation therapy is a major minimally invasive treatment. One of the challenges is that the targeted region and therapeutic progression are often invisible to clinicians, requiring feedback provided in numerical information or imaging. Several emerging imaging modalities offer visualization of the ablation-induced necrosis formation; however, relying solely on necrosis monitoring can result in tissue overheating and endangering patients. Some of the necrosis monitoring modalities are known for their capabilities in temperature sensing, but the principles on which they are based have several limitations, such as sensitivity to the tissue motion and their environment. In this study, we propose a necrosis progression-based temperature estimation technique as an added safety feature for avoiding overheating. This model-based method does not require additional sensing hardware. It is designed to work as an independent estimator or a complimentary estimation component with other thermometers for improved robustness. For this objective, the Neural State Space model is used to approximate the ablation therapy, whose theoretical models involve nonlinear partial differential equations. Then, the Extended Kalman Filter is designed based on the model. The simulation study shows the estimation module robustly estimates the tissue temperature under several types of noise. The maximum estimation error observed before terminating ablation was around 1 °C, and the desired safety feature was successfully demonstrated. The estimator is expected to be used in a variety of necrosis monitoring modalities to guarantee more precise and safer treatment. More ambitiously, the architecture with the Neural State Space model and Extended Kalman Filter is generalizable to other medical/biological procedures involving nonlinear and patient/environment-specific physics and even to procedures having no reliable theoretical models.

Existing controllers for robotic powered prostheses regulate the prosthesis speed, timing, and energy generation using predefined position or torque trajectories. This approach enables climbing stairs step-over-step. However, it does not provide amputees with direct volitional control of the robotic prosthesis, a functionality necessary to restore full mobility to the user. Here we show that proportional electromyographic (EMG) control of the prosthesis knee torque enables volitional control of a powered knee prosthesis during stair climbing. The proposed EMG controller continuously regulates knee torque based on activation of the residual hamstrings, measured using a single EMG electrode located within the socket. The EMG signal is mapped to a desired knee flexion/extension torque based on the prosthesis knee position, the residual limb position, and the interaction with the ground. As a result, the proposed EMG controller enabled an above-knee amputee to climb stairs at different speeds, while carrying additional loads, and even backwards. By enabling direct, volitional control of powered robotic knee prostheses, the proposed EMG controller has the potential to improve amputee mobility in the real world.

Contact force Jacobian relates the changes in the contact force to the changes in the actuation of a robotic catheter in contact with a surface. In this paper, we present an analytical method for calculating the contact force Jacobian for the Cosserat rod model of an MRI-actuated robotic catheter. First, the Cosserat rod model of the MRI-actuated robotic catheter under tip contact position constraint is introduced. For the analytical derivation of contact force Jacobian, the initial value problem parameter derivatives are defined and calculated analytically. Finally, simulation results show that the presented analytical method calculates the contact force Jacobian in significantly shorter computation time with comparable accuracy, compared to direct numerical computation.

Emerging partial-assistance exoskeletons can enhance able-bodied performance and aid people with pathological gait or age-related immobility. However, every person walks differently, which makes it difficult to directly compute assistance torques from joint kinematics. Gait-state estimation-based controllers use phase (normalized stride time) and task variables (e.g., stride length and ground inclination) to parameterize the joint torques. Using kinematic models that depend on the gait-state, prior work has used an Extended Kalman filter (EKF) to estimate the gait-state online. However, this EKF suffered from kinematic errors since it used a subject-independent measurement model, and it is still unknown how personalization of this measurement model would reduce gait-state tracking error. This paper quantifies how much gait-state tracking improvement a personalized measurement model can have over a subject-independent measurement model when using an EKF-based gait-state estimator. Since the EKF performance depends on the measurement model covariance matrix, we tested on multiple different tuning parameters. Across reasonable values of tuning parameters that resulted in good performance, personalization improved estimation error on average by 8.5 ± 13.8% for phase (mean ± standard deviation), 27.2 ± 8.1% for stride length, and 10.5 ± 13.5% for ground inclination. These findings support the hypothesis that personalization of the measurement model significantly improves gait-state estimation performance in EKF based gait-state tracking ($P\ll 0.05$), which could ultimately enable reliable responses to faster human gait changes.