Pub Date : 2024-11-01DOI: 10.1109/LRA.2024.3490406
Zhenhang Chen;Zhiqiang Miao;Min Liu;Chengzhong Wu;Yaonan Wang
Mainstream visual-inertial SLAM systems use point features for motion estimation and localization. However, point features do not perform well in scenes such as weak texture and motion blur. Therefore, the introduction of line features has received a lot of attention. In this letter, we propose a point-line based real-time monocular visual inertial odometry. Aiming at the problem that most of the current works do not fully utilize the line feature properties, we derive the point-line based hybrid Multi-State Constraint Kalman Filter (hybrid MSCKF) in detail. To further improve the line feature initialization accuracy, we propose a two-step line triangulation method. Since filter-based methods are susceptible to visual outliers, we also propose a redundant line feature removal strategy suitable for the filtering framework. According to the experimental results in EuRoC data set and real environment, the proposed algorithm outperforms other state-of-the-art algorithms in accuracy and real-time performance.
主流的视觉惯性 SLAM 系统使用点特征进行运动估计和定位。然而，点特征在弱纹理和运动模糊等场景中表现不佳。因此，线特征的引入受到了广泛关注。在这封信中，我们提出了一种基于点-线的实时单目视觉惯性里程计。针对目前大多数研究没有充分利用线特征特性的问题，我们详细推导了基于点-线的混合多态约束卡尔曼滤波器（hybrid MSCKF）。为了进一步提高线特征初始化精度，我们提出了一种两步线三角测量法。由于基于滤波的方法容易受到视觉异常值的影响，我们还提出了适合滤波框架的冗余线条特征去除策略。根据在 EuRoC 数据集和真实环境中的实验结果，所提出的算法在准确性和实时性上都优于其他最先进的算法。
{"title":"A Fast and Accurate Visual Inertial Odometry Using Hybrid Point-Line Features","authors":"Zhenhang Chen;Zhiqiang Miao;Min Liu;Chengzhong Wu;Yaonan Wang","doi":"10.1109/LRA.2024.3490406","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490406","url":null,"abstract":"Mainstream visual-inertial SLAM systems use point features for motion estimation and localization. However, point features do not perform well in scenes such as weak texture and motion blur. Therefore, the introduction of line features has received a lot of attention. In this letter, we propose a point-line based real-time monocular visual inertial odometry. Aiming at the problem that most of the current works do not fully utilize the line feature properties, we derive the point-line based hybrid Multi-State Constraint Kalman Filter (hybrid MSCKF) in detail. To further improve the line feature initialization accuracy, we propose a two-step line triangulation method. Since filter-based methods are susceptible to visual outliers, we also propose a redundant line feature removal strategy suitable for the filtering framework. According to the experimental results in EuRoC data set and real environment, the proposed algorithm outperforms other state-of-the-art algorithms in accuracy and real-time performance.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11345-11352"},"PeriodicalIF":4.6,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-31DOI: 10.1109/LRA.2024.3490397
Na Li;Fanghao Yuan;Yan Li;Wenwen Lv
E-visits have emerged as a pivotal tool for enhancing both patients' medical experience and the efficient utilization of medical resources. By modeling stylized queueing models, this letter explores the optimal online medical resource allocation decision on providing e-visits in a public hospital, and examines the impact of the external Internet hospital on the public hospital's decision. The public hospital aims to maximize its utility by determining the optimal online medical resource allocation, while patients make their visit decisions based on utility. Results indicate that increased sensitivity of the revisit rate to resource allocation, or higher unit gain associated with e-visit patients (achieved from providing convenient access to medical services and minimizing cross-infection risks), prompts the public hospital to allocate more resources to e-visits. Notably, external Internet hospital price and capacity do not alter the basic influence patterns of the decision, but the resource allocation proportion increases with the external e-visit service price. Moreover, as the external e-visit service capacity expands, the optimal resource allocation proportion decreases when the sensitivity of the revisit rate is low (which indicates the quality is influenced slightly by resource allocation), and exhibits an initial decrease followed by an increase when the sensitivity is high.
{"title":"Public Hospital's Decision Analysis on Providing E-Visit Services","authors":"Na Li;Fanghao Yuan;Yan Li;Wenwen Lv","doi":"10.1109/LRA.2024.3490397","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490397","url":null,"abstract":"E-visits have emerged as a pivotal tool for enhancing both patients' medical experience and the efficient utilization of medical resources. By modeling stylized queueing models, this letter explores the optimal online medical resource allocation decision on providing e-visits in a public hospital, and examines the impact of the external Internet hospital on the public hospital's decision. The public hospital aims to maximize its utility by determining the optimal online medical resource allocation, while patients make their visit decisions based on utility. Results indicate that increased sensitivity of the revisit rate to resource allocation, or higher unit gain associated with e-visit patients (achieved from providing convenient access to medical services and minimizing cross-infection risks), prompts the public hospital to allocate more resources to e-visits. Notably, external Internet hospital price and capacity do not alter the basic influence patterns of the decision, but the resource allocation proportion increases with the external e-visit service price. Moreover, as the external e-visit service capacity expands, the optimal resource allocation proportion decreases when the sensitivity of the revisit rate is low (which indicates the quality is influenced slightly by resource allocation), and exhibits an initial decrease followed by an increase when the sensitivity is high.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11577-11584"},"PeriodicalIF":4.6,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142672202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-31DOI: 10.1109/LRA.2024.3490260
Youssef Michel;Youssef Abdelhalem;Gordon Cheng
In this work, we present a novel bilateral teleoperation architecture with variable impedance control for orientational contact tasks. We exploit Unit Quaternions and tools from Lie Theory to model and manipulate robot orientations, as well as Learning-from-Demonstration to learn a stiffness adaptation policy from the demonstrated task dynamics. The learnt policy then shapes the rotational stiffness of the remote robot during contact with the environment. We also present a passivity analysis where we use energy tanks to guarantee the passivity of the closed loop system, and hence the stable interaction. Our approach is validated on real robot hardware in a cutting task along a curve, and in a user study.
{"title":"Passivity-Based Teleoperation With Variable Rotational Impedance Control","authors":"Youssef Michel;Youssef Abdelhalem;Gordon Cheng","doi":"10.1109/LRA.2024.3490260","DOIUrl":"https://doi.org/10.1109/LRA.2024.3490260","url":null,"abstract":"In this work, we present a novel bilateral teleoperation architecture with variable impedance control for orientational contact tasks. We exploit Unit Quaternions and tools from Lie Theory to model and manipulate robot orientations, as well as Learning-from-Demonstration to learn a stiffness adaptation policy from the demonstrated task dynamics. The learnt policy then shapes the rotational stiffness of the remote robot during contact with the environment. We also present a passivity analysis where we use energy tanks to guarantee the passivity of the closed loop system, and hence the stable interaction. Our approach is validated on real robot hardware in a cutting task along a curve, and in a user study.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11658-11665"},"PeriodicalIF":4.6,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10740678","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142679313","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-30DOI: 10.1109/LRA.2024.3488402
Jonas Kiemel;Ludovic Righetti;Torsten Kröger;Tamim Asfour
In this paper, we present an approach for learning collision-free robot trajectories in the presence of moving obstacles. As a first step, we train a backup policy to generate evasive movements from arbitrary initial robot states using model-free reinforcement learning. When learning policies for other tasks, the backup policy can be used to estimate the potential risk of a collision and to offer an alternative action if the estimated risk is considered too high. No matter which action is selected, our action space ensures that the kinematic limits of the robot joints are not violated. We analyze and evaluate two different methods for estimating the risk of a collision. A physics simulation performed in the background is computationally expensive but provides the best results in deterministic environments. If a data-based risk estimator is used instead, the computational effort is significantly reduced, but an additional source of error is introduced. For evaluation, we successfully learn a reaching task and a basketball task while keeping the risk of collisions low. The results demonstrate the effectiveness of our approach for deterministic and stochastic environments, including a human-robot scenario and a ball environment, where no state can be considered permanently safe. By conducting experiments with a real robot, we show that our approach can generate safe trajectories in real time.
{"title":"Safe Reinforcement Learning of Robot Trajectories in the Presence of Moving Obstacles","authors":"Jonas Kiemel;Ludovic Righetti;Torsten Kröger;Tamim Asfour","doi":"10.1109/LRA.2024.3488402","DOIUrl":"https://doi.org/10.1109/LRA.2024.3488402","url":null,"abstract":"In this paper, we present an approach for learning collision-free robot trajectories in the presence of moving obstacles. As a first step, we train a backup policy to generate evasive movements from arbitrary initial robot states using model-free reinforcement learning. When learning policies for other tasks, the backup policy can be used to estimate the potential risk of a collision and to offer an alternative action if the estimated risk is considered too high. No matter which action is selected, our action space ensures that the kinematic limits of the robot joints are not violated. We analyze and evaluate two different methods for estimating the risk of a collision. A physics simulation performed in the background is computationally expensive but provides the best results in deterministic environments. If a data-based risk estimator is used instead, the computational effort is significantly reduced, but an additional source of error is introduced. For evaluation, we successfully learn a reaching task and a basketball task while keeping the risk of collisions low. The results demonstrate the effectiveness of our approach for deterministic and stochastic environments, including a human-robot scenario and a ball environment, where no state can be considered permanently safe. By conducting experiments with a real robot, we show that our approach can generate safe trajectories in real time.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11353-11360"},"PeriodicalIF":4.6,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-30DOI: 10.1109/LRA.2024.3488400
L. Nowakowski;R. V. Patel
Accurately estimating tool-tissue interaction forces during robotics-assisted minimally invasive surgery is an important aspect of enabling haptics-based teleoperation. By collecting data regarding the state of a robot in a variety of configurations, neural networks can be trained to predict this interaction force. This paper extends existing work in this domain based on collecting one of the largest known ground truth force datasets for stationary as well as moving phantoms that replicate tissue motions found in clinical procedures. Existing methods, and a new transformer-based architecture, are evaluated to demonstrate the domain gap between stationary and moving phantom tissue data and the impact that data scaling has on each architecture's ability to generalize the force estimation task. It was found that temporal networks were more sensitive to the moving domain than single-sample Feed Forward Networks (FFNs) that were trained on stationary tissue data. However, the transformer approach results in the lowest Root Mean Square Error (RMSE) when evaluating networks trained on examples of both stationary and moving phantom tissue samples. The results demonstrate the domain gap between stationary and moving surgical environments and the effectiveness of scaling datasets for increased accuracy of interaction force prediction.
{"title":"Learning Based Estimation of Tool-Tissue Interaction Forces for Stationary and Moving Environments","authors":"L. Nowakowski;R. V. Patel","doi":"10.1109/LRA.2024.3488400","DOIUrl":"https://doi.org/10.1109/LRA.2024.3488400","url":null,"abstract":"Accurately estimating tool-tissue interaction forces during robotics-assisted minimally invasive surgery is an important aspect of enabling haptics-based teleoperation. By collecting data regarding the state of a robot in a variety of configurations, neural networks can be trained to predict this interaction force. This paper extends existing work in this domain based on collecting one of the largest known ground truth force datasets for stationary as well as moving phantoms that replicate tissue motions found in clinical procedures. Existing methods, and a new transformer-based architecture, are evaluated to demonstrate the domain gap between stationary and moving phantom tissue data and the impact that data scaling has on each architecture's ability to generalize the force estimation task. It was found that temporal networks were more sensitive to the moving domain than single-sample Feed Forward Networks (FFNs) that were trained on stationary tissue data. However, the transformer approach results in the lowest Root Mean Square Error (RMSE) when evaluating networks trained on examples of both stationary and moving phantom tissue samples. The results demonstrate the domain gap between stationary and moving surgical environments and the effectiveness of scaling datasets for increased accuracy of interaction force prediction.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11266-11273"},"PeriodicalIF":4.6,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142598640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-29DOI: 10.1109/LRA.2024.3487490
Grzegorz Bartyzel
Transferability, along with sample efficiency, is a critical factor for a reinforcement learning (RL) agent's successful application in real-world contact-rich manipulation tasks, such as product assembly. For instance, in the case of the industrial insertion task on high-mix, low-volume (HMLV) production lines, transferability could eliminate the need for machine retooling, thus reducing production line downtimes. In our work, we introduce a method called Multimodal Variational DeepMDP (MVDeepMDP) that demonstrates the ability to generalize to various environmental variations not encountered during training. The key feature of our approach involves learning a multimodal latent dynamic representation. We demonstrate the effectiveness of our method in the context of an electronic parts insertion task, which is challenging for RL agents due to the diverse physical properties of the non-standardized components, as well as simple 3D printed blocks insertion. Furthermore, we evaluate the transferability of MVDeepMDP and analyze the impact of the balancing mechanism of the generalized Product-of-Experts (gPoE), which is used to combine observable modalities. Finally, we explore the influence of separately processing state modalities of different physical quantities, such as pose and 6D force/torque (F/T) data.
{"title":"Multimodal Variational DeepMDP: An Efficient Approach for Industrial Assembly in High-Mix, Low-Volume Production","authors":"Grzegorz Bartyzel","doi":"10.1109/LRA.2024.3487490","DOIUrl":"https://doi.org/10.1109/LRA.2024.3487490","url":null,"abstract":"Transferability, along with sample efficiency, is a critical factor for a reinforcement learning (RL) agent's successful application in real-world contact-rich manipulation tasks, such as product assembly. For instance, in the case of the industrial insertion task on high-mix, low-volume (HMLV) production lines, transferability could eliminate the need for machine retooling, thus reducing production line downtimes. In our work, we introduce a method called Multimodal Variational DeepMDP (MVDeepMDP) that demonstrates the ability to generalize to various environmental variations not encountered during training. The key feature of our approach involves learning a multimodal latent dynamic representation. We demonstrate the effectiveness of our method in the context of an electronic parts insertion task, which is challenging for RL agents due to the diverse physical properties of the non-standardized components, as well as simple 3D printed blocks insertion. Furthermore, we evaluate the transferability of MVDeepMDP and analyze the impact of the balancing mechanism of the \u0000<italic>generalized Product-of-Experts</i>\u0000 (gPoE), which is used to combine observable modalities. Finally, we explore the influence of separately processing state modalities of different physical quantities, such as pose and 6D force/torque (F/T) data.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11297-11304"},"PeriodicalIF":4.6,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142645555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-29DOI: 10.1109/LRA.2024.3487491
Pu Feng;Rongye Shi;Size Wang;Junkang Liang;Xin Yu;Simin Li;Wenjun Wu
Reinforcement learning (RL) has shown great promise in addressing multi-agent collision avoidance challenges. However, existing RL-based methods often suffer from low training efficiency and poor action safety. To tackle these issues, we introduce a physics-informed reinforcement learning framework equipped with two modules: a Potential Field (PF) module and a Multi-Agent Multi-Level Safety (MAMLS) module. The PF module uses the Artificial Potential Field method to compute a regularization loss, adaptively integrating it into the critic's loss to enhance training efficiency. The MAMLS module formulates action safety as a constrained optimization problem, deriving safe actions by solving this optimization. Furthermore, to better address the characteristics of multi-agent collision avoidance tasks, multi-agent multi-level constraints are introduced. The results of simulations and real-world experiments showed that our physics-informed framework offers a significant improvement in terms of both the efficiency of training and safety-related metrics over advanced baseline methods.
{"title":"Safe and Efficient Multi-Agent Collision Avoidance With Physics-Informed Reinforcement Learning","authors":"Pu Feng;Rongye Shi;Size Wang;Junkang Liang;Xin Yu;Simin Li;Wenjun Wu","doi":"10.1109/LRA.2024.3487491","DOIUrl":"https://doi.org/10.1109/LRA.2024.3487491","url":null,"abstract":"Reinforcement learning (RL) has shown great promise in addressing multi-agent collision avoidance challenges. However, existing RL-based methods often suffer from low training efficiency and poor action safety. To tackle these issues, we introduce a physics-informed reinforcement learning framework equipped with two modules: a Potential Field (PF) module and a Multi-Agent Multi-Level Safety (MAMLS) module. The PF module uses the Artificial Potential Field method to compute a regularization loss, adaptively integrating it into the critic's loss to enhance training efficiency. The MAMLS module formulates action safety as a constrained optimization problem, deriving safe actions by solving this optimization. Furthermore, to better address the characteristics of multi-agent collision avoidance tasks, multi-agent multi-level constraints are introduced. The results of simulations and real-world experiments showed that our physics-informed framework offers a significant improvement in terms of both the efficiency of training and safety-related metrics over advanced baseline methods.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11138-11145"},"PeriodicalIF":4.6,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142598643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-29DOI: 10.1109/LRA.2024.3487503
Bach-Thuan Bui;Huy-Hoang Bui;Dinh-Tuan Tran;Joo-Ho Lee
State-of-the-art visual localization methods mostly rely on complex procedures to match local descriptors and 3D point clouds. However, these procedures can incur significant costs in terms of inference, storage, and updates over time. In this study, we propose a direct learning-based approach that utilizes a simple network named D2S to represent complex local descriptors and their scene coordinates. Our method is characterized by its simplicity and cost-effectiveness. It solely leverages a single RGB image for localization during the testing phase and only requires a lightweight model to encode a complex sparse scene. The proposed D2S employs a combination of a simple loss function and graph attention to selectively focus on robust descriptors while disregarding areas such as clouds, trees, and several dynamic objects. This selective attention enables D2S to effectively perform a binary-semantic classification for sparse descriptors. Additionally, we propose a simple outdoor dataset to evaluate the capabilities of visual localization methods in scene-specific generalization and self-updating from unlabeled observations. Our approach outperforms the previous regression-based methods in both indoor and outdoor environments. It demonstrates the ability to generalize beyond training data, including scenarios involving transitions from day to night and adapting to domain shifts.
{"title":"D2S: Representing Sparse Descriptors and 3D Coordinates for Camera Relocalization","authors":"Bach-Thuan Bui;Huy-Hoang Bui;Dinh-Tuan Tran;Joo-Ho Lee","doi":"10.1109/LRA.2024.3487503","DOIUrl":"https://doi.org/10.1109/LRA.2024.3487503","url":null,"abstract":"State-of-the-art visual localization methods mostly rely on complex procedures to match local descriptors and 3D point clouds. However, these procedures can incur significant costs in terms of inference, storage, and updates over time. In this study, we propose a direct learning-based approach that utilizes a simple network named D2S to represent complex local descriptors and their scene coordinates. Our method is characterized by its simplicity and cost-effectiveness. It solely leverages a single RGB image for localization during the testing phase and only requires a lightweight model to encode a complex sparse scene. The proposed D2S employs a combination of a simple loss function and graph attention to selectively focus on robust descriptors while disregarding areas such as clouds, trees, and several dynamic objects. This selective attention enables D2S to effectively perform a binary-semantic classification for sparse descriptors. Additionally, we propose a simple outdoor dataset to evaluate the capabilities of visual localization methods in scene-specific generalization and self-updating from unlabeled observations. Our approach outperforms the previous regression-based methods in both indoor and outdoor environments. It demonstrates the ability to generalize beyond training data, including scenarios involving transitions from day to night and adapting to domain shifts.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11449-11456"},"PeriodicalIF":4.6,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-29DOI: 10.1109/LRA.2024.3487506
Jan Ole von Hartz;Tim Welschehold;Abhinav Valada;Joschka Boedecker
Task Parametrized Gaussian Mixture Models (TP-GMM) are a sample-efficient method for learning object-centric robot manipulation tasks. However, there are several open challenges to applying TP-GMMs in the wild. In this work, we tackle three crucial challenges synergistically. First, end-effector velocities are non-Euclidean and thus hard to model using standard GMMs. We thus propose to factorize the robot's end-effector velocity into its direction and magnitude, and model them using Riemannian GMMs. Second, we leverage the factorized velocities to segment and sequence skills from complex demonstration trajectories. Through the segmentation, we further align skill trajectories and hence leverage time as a powerful inductive bias. Third, we present a method to automatically detect relevant task parameters per skill from visual observations. Our approach enables learning complex manipulation tasks from just five demonstrations while using only RGB-D observations. Extensive experimental evaluations on RLBench demonstrate that our approach achieves state-of-the-art performance with 20-fold improved sample efficiency. Our policies generalize across different environments, object instances, and object positions, while the learned skills are reusable.
{"title":"The Art of Imitation: Learning Long-Horizon Manipulation Tasks From Few Demonstrations","authors":"Jan Ole von Hartz;Tim Welschehold;Abhinav Valada;Joschka Boedecker","doi":"10.1109/LRA.2024.3487506","DOIUrl":"https://doi.org/10.1109/LRA.2024.3487506","url":null,"abstract":"Task Parametrized Gaussian Mixture Models (TP-GMM) are a sample-efficient method for learning object-centric robot manipulation tasks. However, there are several open challenges to applying TP-GMMs in the wild. In this work, we tackle three crucial challenges synergistically. First, end-effector velocities are non-Euclidean and thus hard to model using standard GMMs. We thus propose to factorize the robot's end-effector velocity into its direction and magnitude, and model them using Riemannian GMMs. Second, we leverage the factorized velocities to segment and sequence skills from complex demonstration trajectories. Through the segmentation, we further align skill trajectories and hence leverage time as a powerful inductive bias. Third, we present a method to automatically detect relevant task parameters \u0000<italic>per</i>\u0000 skill from visual observations. Our approach enables learning complex manipulation tasks from just five demonstrations while using only RGB-D observations. Extensive experimental evaluations on RLBench demonstrate that our approach achieves state-of-the-art performance with 20-fold improved sample efficiency. Our policies generalize across different environments, object instances, and object positions, while the learned skills are reusable.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11369-11376"},"PeriodicalIF":4.6,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robot-assisted minimally invasive surgery (RAMIS) provides substantial benefits over traditional open and laparoscopic methods. However, a significant limitation of robot-assisted minimally invasive surgery (RAMIS) is the surgeon's inability to palpate tissues, a crucial technique for examining tissue properties and detecting abnormalities, restricting the widespread adoption of RAMIS. To overcome this obstacle, we introduce MiniTac, a novel vision-based tactile sensor with an ultra-compact cross-sectional diameter of 8mm, designed for seamless integration into mainstream RAMIS devices, particularly the Da Vinci surgical systems. MiniTac features a novel mechanoresponsive photonic elastomer membrane that changes color distribution under varying contact pressures. This color change is captured by an embedded miniature camera, allowing MiniTac to detect tumors both on the tissue surface and in deeper layers typically obscured from endoscopic view. MiniTac's efficacy has been rigorously tested on both phantoms and ex-vivo tissues. By leveraging advanced mechanoresponsive photonic materials, MiniTac represents a significant advancement in integrating tactile sensing into RAMIS, potentially expanding its applicability to a wider array of clinical scenarios that currently rely on traditional surgical approaches.
{"title":"MiniTac: An Ultra-Compact $text{8 mm}$ Vision-Based Tactile Sensor for Enhanced Palpation in Robot-Assisted Minimally Invasive Surgery","authors":"Wanlin Li;Zihang Zhao;Leiyao Cui;Weiyi Zhang;Hangxin Liu;Li-An Li;Yixin Zhu","doi":"10.1109/LRA.2024.3487516","DOIUrl":"https://doi.org/10.1109/LRA.2024.3487516","url":null,"abstract":"Robot-assisted minimally invasive surgery (RAMIS) provides substantial benefits over traditional open and laparoscopic methods. However, a significant limitation of robot-assisted minimally invasive surgery (RAMIS) is the surgeon's inability to palpate tissues, a crucial technique for examining tissue properties and detecting abnormalities, restricting the widespread adoption of RAMIS. To overcome this obstacle, we introduce MiniTac, a novel vision-based tactile sensor with an ultra-compact cross-sectional diameter of 8mm, designed for seamless integration into mainstream RAMIS devices, particularly the Da Vinci surgical systems. MiniTac features a novel mechanoresponsive photonic elastomer membrane that changes color distribution under varying contact pressures. This color change is captured by an embedded miniature camera, allowing MiniTac to detect tumors both on the tissue surface and in deeper layers typically obscured from endoscopic view. MiniTac's efficacy has been rigorously tested on both phantoms and ex-vivo tissues. By leveraging advanced mechanoresponsive photonic materials, MiniTac represents a significant advancement in integrating tactile sensing into RAMIS, potentially expanding its applicability to a wider array of clinical scenarios that currently rely on traditional surgical approaches.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11170-11177"},"PeriodicalIF":4.6,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142598672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：