Pub Date : 2025-01-06DOI: 10.1109/LRA.2025.3526568
Huming Zhu;Yiyu Xue;Ximiao Dong;Xinyue Cheng
Distant object detection is a difficult problem in LiDAR-based 3D object detection. In recent years, the 3D detection of distant objects has achieved great success with the proposed fusion method of the virtual points generated by depth completion and LiDAR points. However, the inaccuracy of depth completion brings a lot of noise which significantly reduces the detection accuracy. To reduce noise and improve the detection accuracy of distant objects, we propose a solution called VirInteraction, which is a semantic-guided Virtual-LiDAR fusion method to enhance the interaction of virtual points and LiDAR points. Specifically, VirInteraction mainly includes three new designs: 1) Foreground-based adaptive Voxel Denoising (FgVD), 2) Semantic neighboring Sampling (Se-Sampling), and 3) Multi-scale Density-aware Cross Attention (MDC-Attention). FgVD uses Kernel Density Estimation (KDE) to adaptively denoise the foreground and background voxels. Se-Sampling completes the shape cues of distant objects using bidirectional sampling based on self-attention mechanism. Meanwhile, we built on these two designs and VirConvNet to develop a more robust VirInterNet as our virtual-point-based backbone. Finally, MDC-Attention elegantly aggregates the features of the images and points at the feature level according to the density distribution. Extensive experiments on KITTI and nuScenes datasets demonstrate the effectiveness of VirInteraction.
{"title":"VirInteraction: Enhancing Virtual-LiDAR Points Interaction by Using Image Semantics and Density Estimation for 3D Object Detection","authors":"Huming Zhu;Yiyu Xue;Ximiao Dong;Xinyue Cheng","doi":"10.1109/LRA.2025.3526568","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526568","url":null,"abstract":"Distant object detection is a difficult problem in LiDAR-based 3D object detection. In recent years, the 3D detection of distant objects has achieved great success with the proposed fusion method of the virtual points generated by depth completion and LiDAR points. However, the inaccuracy of depth completion brings a lot of noise which significantly reduces the detection accuracy. To reduce noise and improve the detection accuracy of distant objects, we propose a solution called VirInteraction, which is a semantic-guided Virtual-LiDAR fusion method to enhance the interaction of virtual points and LiDAR points. Specifically, VirInteraction mainly includes three new designs: 1) Foreground-based adaptive Voxel Denoising (FgVD), 2) Semantic neighboring Sampling (Se-Sampling), and 3) Multi-scale Density-aware Cross Attention (MDC-Attention). FgVD uses Kernel Density Estimation (KDE) to adaptively denoise the foreground and background voxels. Se-Sampling completes the shape cues of distant objects using bidirectional sampling based on self-attention mechanism. Meanwhile, we built on these two designs and VirConvNet to develop a more robust VirInterNet as our virtual-point-based backbone. Finally, MDC-Attention elegantly aggregates the features of the images and points at the feature level according to the density distribution. Extensive experiments on KITTI and nuScenes datasets demonstrate the effectiveness of VirInteraction.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1872-1879"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Unmanned Aerial Vehicles (UAVs) are increasingly utilized across commercial, military, and public safety domains, where target tracking is crucial for imagery content analysis and various applications. This letter explores the challenges in UAV target tracking, particularly addressing deficiencies in RGBD tracking datasets and algorithms for aerial targets. We introduce a new dataset, the Reconstructed and Simulated Aerial RGBD Tracking Dataset (RSTrack), created by reconstructing an existing aerial tracking dataset through depth estimation and enhancing it with simulation environments, containing 209 challenging sequences with over 247 K frames. Experiments conducted on RSTrack demonstrate the effectiveness of depth information in improving aerial target tracking and the benefits of simulation data in enhancing the performance of RGBD trackers in UAV applications. Furthermore, we develop Shallow Feature Fusion-based RGBD Tracking (SFT). Our results indicate that SFT achieves an optimal balance between robustness and speed for UAV applications. These findings are expected to offer a valuable dataset for RGBD tracking and advance methodologies in the field of aerial target tracking.
{"title":"Reconstructed and Simulated Dataset for Aerial RGBD Tracking","authors":"Juntao Liang;Jiaqi Zhou;Wei Li;Yong Wang;Tianjiang Hu;Qi Wu","doi":"10.1109/LRA.2025.3526565","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526565","url":null,"abstract":"Unmanned Aerial Vehicles (UAVs) are increasingly utilized across commercial, military, and public safety domains, where target tracking is crucial for imagery content analysis and various applications. This letter explores the challenges in UAV target tracking, particularly addressing deficiencies in RGBD tracking datasets and algorithms for aerial targets. We introduce a new dataset, the Reconstructed and Simulated Aerial RGBD Tracking Dataset (RSTrack), created by reconstructing an existing aerial tracking dataset through depth estimation and enhancing it with simulation environments, containing 209 challenging sequences with over 247 K frames. Experiments conducted on RSTrack demonstrate the effectiveness of depth information in improving aerial target tracking and the benefits of simulation data in enhancing the performance of RGBD trackers in UAV applications. Furthermore, we develop Shallow Feature Fusion-based RGBD Tracking (SFT). Our results indicate that SFT achieves an optimal balance between robustness and speed for UAV applications. These findings are expected to offer a valuable dataset for RGBD tracking and advance methodologies in the field of aerial target tracking.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"2008-2015"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142992985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-06DOI: 10.1109/LRA.2025.3526444
Andrea Fortuna;Marta Lorenzini;Younggeol Cho;Robin Arbaud;Stefano Filippo Castiglia;Mariano Serrao;Alberto Ranavalo;Elena De Momi;Arash Ajoudani
Walking aids for individuals with musculoskeletal frailty or motor disabilities must ensure adequate physical support and assistance to their users. To this end, sensor-enabled human state monitoring and estimation are crucial. This letter proposes an innovative approach to assessing users' stability while walking with WANDER, a novel gait assistive device, by exploiting the correlation between the eXtrapolated Center of Mass ($XCoM$) and the Base of Support ($BoS$) edges. First, the soundness of this metric in monitoring gait stability is proven. Experiments on 25 healthy individuals show that the median value of Pearson's correlation coefficient (p-value $< $ 0.05) remained high during the forward walk for all subjects. Next, a correlation-based variable admittance (CVA) controller is implemented, whose parameters are tuned to physically support users when a gait perturbation is detected (i.e. low values of Pearson's correlation coefficient). To validate this approach, 13 healthy subjects were asked to compare our controller with a force threshold-based (FVA) one. The CVA controller's performance in discriminating stable and perturbed gait conditions showed a high sensitivity value, comparable to FVA, and improved performance in terms of specificity. The number of false and missed detections of gait perturbation was considerably reduced, independently of walking speed, exhibiting a higher level of safety and smoothness compared to the FVA controller. Overall, the outcome of this study gives promising evidence of the proposed metric capability in identifying user stability and triggering WANDER's assistance.
{"title":"Assisting Gait Stability in Walking Aid Users Exploiting Biomechanical Variables Correlation","authors":"Andrea Fortuna;Marta Lorenzini;Younggeol Cho;Robin Arbaud;Stefano Filippo Castiglia;Mariano Serrao;Alberto Ranavalo;Elena De Momi;Arash Ajoudani","doi":"10.1109/LRA.2025.3526444","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526444","url":null,"abstract":"Walking aids for individuals with musculoskeletal frailty or motor disabilities must ensure adequate physical support and assistance to their users. To this end, sensor-enabled human state monitoring and estimation are crucial. This letter proposes an innovative approach to assessing users' stability while walking with WANDER, a novel gait assistive device, by exploiting the correlation between the eXtrapolated Center of Mass (<inline-formula><tex-math>$XCoM$</tex-math></inline-formula>) and the Base of Support (<inline-formula><tex-math>$BoS$</tex-math></inline-formula>) edges. First, the soundness of this metric in monitoring gait stability is proven. Experiments on 25 healthy individuals show that the median value of Pearson's correlation coefficient (p-value <inline-formula><tex-math>$< $</tex-math></inline-formula> 0.05) remained high during the forward walk for all subjects. Next, a correlation-based variable admittance (CVA) controller is implemented, whose parameters are tuned to physically support users when a gait perturbation is detected (i.e. low values of Pearson's correlation coefficient). To validate this approach, 13 healthy subjects were asked to compare our controller with a force threshold-based (FVA) one. The CVA controller's performance in discriminating stable and perturbed gait conditions showed a high sensitivity value, comparable to FVA, and improved performance in terms of specificity. The number of false and missed detections of gait perturbation was considerably reduced, independently of walking speed, exhibiting a higher level of safety and smoothness compared to the FVA controller. Overall, the outcome of this study gives promising evidence of the proposed metric capability in identifying user stability and triggering WANDER's assistance.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"2040-2047"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10829677","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142993066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Exoskeleton robots are an effective method for enhancing human walking ability. This letter introduces a quasi-passive variable stiffness ankle exoskeleton, which absorbs negative work produced during ankle dorsiflexion in the stance phase of the gait cycle and releases energy to assist plantar flexion during push-off. Compared to powered exoskeletons, this design does not require high-power actuators but instead relies on a clutch and elastic component to mimic the interaction between muscles and tendons for assistance. Compared to passive exoskeletons, the designed clutch can adapt to different users. Compared to fixed-stiffness exoskeletons, the novel variable stiffness energy storage mechanism passively adjusts stiffness to mimic the biomechanical properties of the ankle joint. The proposed exoskeleton identifies gait phases based on a control strategy using foot force sensors. This strategy controls the exoskeleton's energy recovery and release by determining the gait cycle phase and changing the clutch state. Finally, a level-ground walking experiment was conducted with six healthy participants. Results showed that wearing the exoskeleton reduced the root mean square (RMS) change rate of soleus EMG activity by 7.25% and decreased the net metabolic rate during walking by 3.6%.
{"title":"Design and Experimental Verification of a Quasi-Passive Variable Stiffness Ankle Exoskeleton for Human Walking Assistance","authors":"Jinlu Wang;Qingcong Wu;Yiqi Zou;Yanghui Zhu;Hongtao Wang;Hongtao Wu","doi":"10.1109/LRA.2025.3526450","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526450","url":null,"abstract":"Exoskeleton robots are an effective method for enhancing human walking ability. This letter introduces a quasi-passive variable stiffness ankle exoskeleton, which absorbs negative work produced during ankle dorsiflexion in the stance phase of the gait cycle and releases energy to assist plantar flexion during push-off. Compared to powered exoskeletons, this design does not require high-power actuators but instead relies on a clutch and elastic component to mimic the interaction between muscles and tendons for assistance. Compared to passive exoskeletons, the designed clutch can adapt to different users. Compared to fixed-stiffness exoskeletons, the novel variable stiffness energy storage mechanism passively adjusts stiffness to mimic the biomechanical properties of the ankle joint. The proposed exoskeleton identifies gait phases based on a control strategy using foot force sensors. This strategy controls the exoskeleton's energy recovery and release by determining the gait cycle phase and changing the clutch state. Finally, a level-ground walking experiment was conducted with six healthy participants. Results showed that wearing the exoskeleton reduced the root mean square (RMS) change rate of soleus EMG activity by 7.25% and decreased the net metabolic rate during walking by 3.6%.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1856-1863"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-06DOI: 10.1109/LRA.2025.3526562
Jiayuan He;Shunqi Qu;Chuang Lin;Ning Jiang
Compared to the forearm, the wrist is suitable for the combination of myoelectric control with the popular wearable devices, enabling human machine interaction in an intuitive and effortless way. The change of limb condition is a common disturbance degrading the performance of wrist myoelectric control in practical applications. Though multi-position training is a simple and effective strategy of mitigating the influence, with the addition of training data, the performance of the traditional method could be plateaued before reaching the perfect. This study proposed a multi-scale one-dimensional convolutional neural network (MSCNN) with the end-to-end learning to improve the data generalization from different limb conditions. The results showed that the proposed method outperformed the traditional method by from 7.2% with single limb condition training to 9.5% with seven limb condition training, where the classification accuracy of the proposed method, i.e., 97.1%, was close to the perfect. The difference was from the performance on the data from the training conditions, which was maintained by MSCNN, but dropped by the traditional method with the addition of the training data. This work improved the robustness of group strategy against limb condition effect. The results could facilitate the development of the wrist-based wearable devices, as well as the applications of the myoelectric control-based human machine interface into more areas.
{"title":"Improving Multi-Position Training Performance on Reducing Limb Condition Effect in Wrist Myoelectric Control","authors":"Jiayuan He;Shunqi Qu;Chuang Lin;Ning Jiang","doi":"10.1109/LRA.2025.3526562","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526562","url":null,"abstract":"Compared to the forearm, the wrist is suitable for the combination of myoelectric control with the popular wearable devices, enabling human machine interaction in an intuitive and effortless way. The change of limb condition is a common disturbance degrading the performance of wrist myoelectric control in practical applications. Though multi-position training is a simple and effective strategy of mitigating the influence, with the addition of training data, the performance of the traditional method could be plateaued before reaching the perfect. This study proposed a multi-scale one-dimensional convolutional neural network (MSCNN) with the end-to-end learning to improve the data generalization from different limb conditions. The results showed that the proposed method outperformed the traditional method by from 7.2% with single limb condition training to 9.5% with seven limb condition training, where the classification accuracy of the proposed method, i.e., 97.1%, was close to the perfect. The difference was from the performance on the data from the training conditions, which was maintained by MSCNN, but dropped by the traditional method with the addition of the training data. This work improved the robustness of group strategy against limb condition effect. The results could facilitate the development of the wrist-based wearable devices, as well as the applications of the myoelectric control-based human machine interface into more areas.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1800-1807"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
End-to-end robotlearning, particularly for long-horizon tasks, often results in unpredictable outcomes and poor generalization. To address these challenges, we propose a novel Therblig-Based Backbone Framework (TBBF) as a fundamental structure to enhance interpretability, data efficiency, and generalization in robotic systems. TBBF utilizes expert demonstrations to enable therblig-level task decomposition, facilitate efficient action-object mapping, and generate adaptive trajectories for new scenarios. The approach consists of two stages: offline training and online testing. During the offline training stage, we developed the Meta-RGate SynerFusion (MGSF) network for accurate therblig segmentation across various tasks. In the online testing stage, after a one-shot demonstration of a new task is collected, our MGSF network extracts high-level knowledge, which is then encoded into the image using Action Registration (ActionREG). Additionally, Large Language Model (LLM)-Alignment Policy for Visual Correction (LAP-VC) is employed to ensure precise action registration, facilitating trajectory transfer in novel robot scenarios. Experimental results validate these methods, achieving 94.37% recall in therblig segmentation and success rates of 94.4% and 80% in real-world online robot testing for simple and complex scenarios, respectively.
{"title":"A Backbone for Long-Horizon Robot Task Understanding","authors":"Xiaoshuai Chen;Wei Chen;Dongmyoung Lee;Yukun Ge;Nicolas Rojas;Petar Kormushev","doi":"10.1109/LRA.2025.3526441","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526441","url":null,"abstract":"End-to-end robotlearning, particularly for long-horizon tasks, often results in unpredictable outcomes and poor generalization. To address these challenges, we propose a novel <italic>Therblig-Based Backbone Framework (TBBF)</i> as a fundamental structure to enhance interpretability, data efficiency, and generalization in robotic systems. TBBF utilizes expert demonstrations to enable therblig-level task decomposition, facilitate efficient action-object mapping, and generate adaptive trajectories for new scenarios. The approach consists of two stages: offline training and online testing. During the offline training stage, we developed the <italic>Meta-RGate SynerFusion (MGSF)</i> network for accurate therblig segmentation across various tasks. In the online testing stage, after a one-shot demonstration of a new task is collected, our <italic>MGSF</i> network extracts high-level knowledge, which is then encoded into the image using <italic>Action Registration (ActionREG)</i>. Additionally, <italic>Large Language Model (LLM)-Alignment Policy for Visual Correction (LAP-VC)</i> is employed to ensure precise action registration, facilitating trajectory transfer in novel robot scenarios. Experimental results validate these methods, achieving 94.37% recall in therblig segmentation and success rates of 94.4% and 80% in real-world online robot testing for simple and complex scenarios, respectively.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"2048-2055"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142992963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-06DOI: 10.1109/LRA.2025.3526570
Yi Yang;Linfeng Fan;Tao Weng;Yi Zhao;Bingxing Chen;Wenqiang Li
Physical intelligence for aerial robots greatly enhances grasping and perching performance, but remains in emerging stages. This letter proposes a novel bistable soft gripper for aerial robots with high response speed (0.11 s), large holding force (23.47 N), and active/passive adaptive grasping and perching. The soft gripper is constructed by four bistable fingers, tension nets, and a bidirectional actuation system. The soft finger evolves from a simple bistable rotational joint. Tension nets inspired by spider webs are proposed to improve the energy barrier and grasping performance. Experiments are conducted to measure the gripper's potential energy variation and grasping performance. One peak and two local minima in the energy curve indicate the gripper's bistability. Experimental results show that tension nets can enhance the gripper's energy barrier, response speed, and maximum holding force by 915.07%, 38.55%, and 62.08%, respectively. The gripper's adjustability of the energy barrier is validated, enabling it to switch active/passive modes as needed. The experiments demonstrated static/dynamic grasping and perching for various daily objects with different shapes, sizes, and stiffness for the gripper and aerial robot. Finally, the robot can transport objects outdoors, and can be aerially manipulated by external force, demonstrating its great potential in aerial application.
{"title":"Bistable Soft Gripper With Tension Net Applied to UAV","authors":"Yi Yang;Linfeng Fan;Tao Weng;Yi Zhao;Bingxing Chen;Wenqiang Li","doi":"10.1109/LRA.2025.3526570","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526570","url":null,"abstract":"Physical intelligence for aerial robots greatly enhances grasping and perching performance, but remains in emerging stages. This letter proposes a novel bistable soft gripper for aerial robots with high response speed (0.11 s), large holding force (23.47 N), and active/passive adaptive grasping and perching. The soft gripper is constructed by four bistable fingers, tension nets, and a bidirectional actuation system. The soft finger evolves from a simple bistable rotational joint. Tension nets inspired by spider webs are proposed to improve the energy barrier and grasping performance. Experiments are conducted to measure the gripper's potential energy variation and grasping performance. One peak and two local minima in the energy curve indicate the gripper's bistability. Experimental results show that tension nets can enhance the gripper's energy barrier, response speed, and maximum holding force by 915.07%, 38.55%, and 62.08%, respectively. The gripper's adjustability of the energy barrier is validated, enabling it to switch active/passive modes as needed. The experiments demonstrated static/dynamic grasping and perching for various daily objects with different shapes, sizes, and stiffness for the gripper and aerial robot. Finally, the robot can transport objects outdoors, and can be aerially manipulated by external force, demonstrating its great potential in aerial application.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1920-1927"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142992975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-06DOI: 10.1109/LRA.2025.3526436
Peiyan Li;Hongtao Wu;Yan Huang;Chilam Cheang;Liang Wang;Tao Kong
The robotics community has consistently aimed to achieve generalizable robot manipulation with flexible natural language instructions. One primary challenge is that obtaining robot trajectories fully annotated with both actions and texts is time-consuming and labor-intensive. However, partially-annotated data, such as human activity videos without action labels and robot trajectories without text labels, are much easier to collect. Can we leverage these data to enhance the generalization capabilities of robots? In this letter, we propose GR-MG, a novel method which supports conditioning on a text instruction and a goal image. During training, GR-MG samples goal images from trajectories and conditions on both the text and the goal image or solely on the image when text is not available. During inference, where only the text is provided, GR-MG generates the goal image via a diffusion-based image-editing model and conditions on both the text and the generated image. This approach enables GR-MG to leverage large amounts of partially-annotated data while still using languages to flexibly specify tasks. To generate accurate goal images, we propose a novel progress-guided goal image generation model which injects task progress information into the generation process. In simulation experiments, GR-MG improves the average number of tasks completed in a row of 5 from 3.35 to 4.04. In real-robot experiments, GR-MG is able to perform 58 different tasks and improves the success rate from 68.7% to 78.1% and 44.4% to 60.6% in simple and generalization settings, respectively. It also outperforms comparing baseline methods in few-shot learning of novel skills.
{"title":"GR-MG: Leveraging Partially-Annotated Data via Multi-Modal Goal-Conditioned Policy","authors":"Peiyan Li;Hongtao Wu;Yan Huang;Chilam Cheang;Liang Wang;Tao Kong","doi":"10.1109/LRA.2025.3526436","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526436","url":null,"abstract":"The robotics community has consistently aimed to achieve generalizable robot manipulation with flexible natural language instructions. One primary challenge is that obtaining robot trajectories fully annotated with both actions and texts is time-consuming and labor-intensive. However, partially-annotated data, such as human activity videos without action labels and robot trajectories without text labels, are much easier to collect. Can we leverage these data to enhance the generalization capabilities of robots? In this letter, we propose GR-MG, a novel method which supports conditioning on a text instruction and a goal image. During training, GR-MG samples goal images from trajectories and conditions on both the text and the goal image or solely on the image when text is not available. During inference, where only the text is provided, GR-MG generates the goal image via a diffusion-based image-editing model and conditions on both the text and the generated image. This approach enables GR-MG to leverage large amounts of partially-annotated data while still using languages to flexibly specify tasks. To generate accurate goal images, we propose a novel progress-guided goal image generation model which injects task progress information into the generation process. In simulation experiments, GR-MG improves the average number of tasks completed in a row of 5 from 3.35 to 4.04. In real-robot experiments, GR-MG is able to perform 58 different tasks and improves the success rate from 68.7% to 78.1% and 44.4% to 60.6% in simple and generalization settings, respectively. It also outperforms comparing baseline methods in few-shot learning of novel skills.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1912-1919"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142993079","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tethered robots find widespread application in underwater and disaster recovery missions. This study focuses on the coverage path planning (CPP) problem for a tethered robot, considering cable constraints and the presence of forbidden areas in the environment. We propose adapting the spanning tree- based coverage algorithm to address CPP. Theoretical complexity analysis reveals NP-completeness in cases involving forbidden areas. We show how to solve CPP by searching for a tree in a configuration graph, and how to reduce the size of this graph to compute approximate solutions faster. We introduce Integer Linear Programming (ILP) models corresponding to these approximations and experimentally compare them on various instances.
{"title":"Spanning-Tree Based Coverage for a Tethered Robot","authors":"Xiao Peng;François Schwarzentruber;Olivier Simonin;Christine Solnon","doi":"10.1109/LRA.2025.3526564","DOIUrl":"https://doi.org/10.1109/LRA.2025.3526564","url":null,"abstract":"Tethered robots find widespread application in underwater and disaster recovery missions. This study focuses on the coverage path planning (CPP) problem for a tethered robot, considering cable constraints and the presence of forbidden areas in the environment. We propose adapting the spanning tree- based coverage algorithm to address CPP. Theoretical complexity analysis reveals NP-completeness in cases involving forbidden areas. We show how to solve CPP by searching for a tree in a configuration graph, and how to reduce the size of this graph to compute approximate solutions faster. We introduce Integer Linear Programming (ILP) models corresponding to these approximations and experimentally compare them on various instances.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1888-1895"},"PeriodicalIF":4.6,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wheels require extra space for steering. Omnidirectional wheels are ideal for confined spaces as they can move in all directions: forward/backward and left/right. Conventional omnidirectional wheels with passive rollers achieve this movement by combining multiple wheels. However, if even one wheel loses contact with the ground, the vehicle becomes inoperable. To overcome this limitation, omnidirectional wheels with actively driven rollers have been proposed. These designs, however, require additional components, which increase weight. This is because multi-step intermediate transmission mechanisms are needed to convert spindle rotation into roller rotation. Eliminating the intermediate transmission mechanism reduces the number of components and provides more space to enhance wheel strength. This study proposed a mechanism without intermediate transmission, clarified its design framework, and experimentally demonstrated its feasibility as an active omnidirectional wheel. The proposed design framework defines conditions to maximize both power transmission efficiency and strength. Experimental results showed that the transmission efficiency of the proposed mechanism is comparable to that of conventional mechanisms.
{"title":"Poloidal Drive: Direct-Drive Transmission Mechanism for Active Omni-Wheels With Spoke Interference Avoidance","authors":"Shunsuke Sano;Kenjiro Tadakuma;Ryotaro Kayawake;Masahiro Watanabe;Kazuki Abe;Yuto Kemmotsu;Satoshi Tadokoro","doi":"10.1109/LRA.2024.3524885","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524885","url":null,"abstract":"Wheels require extra space for steering. Omnidirectional wheels are ideal for confined spaces as they can move in all directions: forward/backward and left/right. Conventional omnidirectional wheels with passive rollers achieve this movement by combining multiple wheels. However, if even one wheel loses contact with the ground, the vehicle becomes inoperable. To overcome this limitation, omnidirectional wheels with actively driven rollers have been proposed. These designs, however, require additional components, which increase weight. This is because multi-step intermediate transmission mechanisms are needed to convert spindle rotation into roller rotation. Eliminating the intermediate transmission mechanism reduces the number of components and provides more space to enhance wheel strength. This study proposed a mechanism without intermediate transmission, clarified its design framework, and experimentally demonstrated its feasibility as an active omnidirectional wheel. The proposed design framework defines conditions to maximize both power transmission efficiency and strength. Experimental results showed that the transmission efficiency of the proposed mechanism is comparable to that of conventional mechanisms.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1824-1831"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10819649","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975734","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：