Pub Date : 2025-01-01DOI: 10.1109/LRA.2024.3524876
Ke Shi;Zainan Jiang;Borui Liu;Guocai Yang;Minghe Jin
Transformable-wheeled robots exhibit efficient locomotion and obstacle negotiation through mode transformation, which underpins the development of the multimodal robot MTABot—a previously validated platform. However, existing literature primarily focuses on structural design, leaving autonomous mode transitions across varying terrains as a significant challenge. This paper presents a unified terrain-adaptive morphing and trajectory tracking approach for MTABot, utilizing the Nonlinear Model Predictive Control (NMPC) framework. This method eliminates the need for environmental recognition or prior training. Specifically, a segmented kinematic model for the transformable wheel has been developed, ensuring the feasibility of motion in both rolling and climbing modes. Additionally, a virtual ground attachment constraint is proposed to guide adaptive morphing for overcoming single or small obstacles. An online weight adjustment method for NMPC is introduced to synchronize wheel motion and overcome continuous large obstacles. Comprehensive experiments in multi-terrain composite scenarios and various obstacle-crossing tests validated the effectiveness of the proposed approach.
{"title":"Synergistic Terrain-Adaptive Morphing and Trajectory Tracking in a Transformable-Wheeled Robot","authors":"Ke Shi;Zainan Jiang;Borui Liu;Guocai Yang;Minghe Jin","doi":"10.1109/LRA.2024.3524876","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524876","url":null,"abstract":"Transformable-wheeled robots exhibit efficient locomotion and obstacle negotiation through mode transformation, which underpins the development of the multimodal robot MTABot—a previously validated platform. However, existing literature primarily focuses on structural design, leaving autonomous mode transitions across varying terrains as a significant challenge. This paper presents a unified terrain-adaptive morphing and trajectory tracking approach for MTABot, utilizing the Nonlinear Model Predictive Control (NMPC) framework. This method eliminates the need for environmental recognition or prior training. Specifically, a segmented kinematic model for the transformable wheel has been developed, ensuring the feasibility of motion in both rolling and climbing modes. Additionally, a virtual ground attachment constraint is proposed to guide adaptive morphing for overcoming single or small obstacles. An online weight adjustment method for NMPC is introduced to synchronize wheel motion and overcome continuous large obstacles. Comprehensive experiments in multi-terrain composite scenarios and various obstacle-crossing tests validated the effectiveness of the proposed approach.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1656-1663"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-01DOI: 10.1109/LRA.2024.3524897
Nitesh Kumar;Jaekyung Jackie Lee;Sivakumar Rathinam;Swaroop Darbha;P. B. Sujit;Rajiv Raman
This paper introduces a novel formulation for finding the recharging schedule for a fleet of $n$ heterogeneous robots that minimizes utilization of recharging resources. This study provides a foundational framework applicable to Multi-Robot Mission Planning, particularly in scenarios demanding Long-Duration Autonomy (LDA) or other contexts that necessitate periodic recharging of multiple robots. A novel Integer Linear Programming (ILP) model is proposed to calculate the optimal initial conditions (partial charge) for individual robots, leading to minimal utilization of charging stations. This formulation was further generalized to maximize the servicing time for robots when charging stations are limited. The efficacy of the proposed formulation is evaluated through a comparative analysis, measuring its performance against the thrift price scheduling algorithm documented in the existing literature. The findings not only corroborate the effectiveness of the proposed approach but also underscore its potential as a valuable tool in optimizing resource allocation for a range of robotic and engineering applications.
{"title":"The Persistent Robot Charging Problem for Long-Duration Autonomy","authors":"Nitesh Kumar;Jaekyung Jackie Lee;Sivakumar Rathinam;Swaroop Darbha;P. B. Sujit;Rajiv Raman","doi":"10.1109/LRA.2024.3524897","DOIUrl":"https://doi.org/10.1109/LRA.2024.3524897","url":null,"abstract":"This paper introduces a novel formulation for finding the recharging schedule for a fleet of <inline-formula><tex-math>$n$</tex-math></inline-formula> heterogeneous robots that minimizes utilization of recharging resources. This study provides a foundational framework applicable to Multi-Robot Mission Planning, particularly in scenarios demanding Long-Duration Autonomy (LDA) or other contexts that necessitate periodic recharging of multiple robots. A novel Integer Linear Programming (ILP) model is proposed to calculate the optimal initial conditions (partial charge) for individual robots, leading to minimal utilization of charging stations. This formulation was further generalized to maximize the servicing time for robots when charging stations are limited. The efficacy of the proposed formulation is evaluated through a comparative analysis, measuring its performance against the thrift price scheduling algorithm documented in the existing literature. The findings not only corroborate the effectiveness of the proposed approach but also underscore its potential as a valuable tool in optimizing resource allocation for a range of robotic and engineering applications.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 3","pages":"2191-2198"},"PeriodicalIF":4.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143106703","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-30DOI: 10.1109/LRA.2024.3522690
{"title":"2024 Index IEEE Robotics and Automation Letters Vol. 9","authors":"","doi":"10.1109/LRA.2024.3522690","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522690","url":null,"abstract":"","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11889-12172"},"PeriodicalIF":4.6,"publicationDate":"2024-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10817820","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142905957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3523203
Jiayi Jin;Siyuan Feng;Shuguang Li
Soft grippers are increasingly favored due to their passive compliance, lack of need for precise force control, and high adaptability to various object shapes. Unlike previous soft grippers that are mostly universal, we propose a framework for the computational design and rapid fabrication of customized soft grippers using a specific class of vacuum-driven pneumatic actuators. The algorithm can automatically generate a 3D-printable model of the optimized gripper design, and then the gripper can be rapidly fabricated at a low cost. Grasping experiments demonstrate that this framework can customize grippers for various daily objects with different geometries. The results also show the extensional abilities of customizing a gripper for multiple or heavy objects. This framework enables the rapid design and fabrication of grippers optimized for specific tasks while maintaining versatility for handling various objects.
{"title":"Computational Design of Customized Vacuum-Driven Soft Grippers","authors":"Jiayi Jin;Siyuan Feng;Shuguang Li","doi":"10.1109/LRA.2024.3523203","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523203","url":null,"abstract":"Soft grippers are increasingly favored due to their passive compliance, lack of need for precise force control, and high adaptability to various object shapes. Unlike previous soft grippers that are mostly universal, we propose a framework for the computational design and rapid fabrication of customized soft grippers using a specific class of vacuum-driven pneumatic actuators. The algorithm can automatically generate a 3D-printable model of the optimized gripper design, and then the gripper can be rapidly fabricated at a low cost. Grasping experiments demonstrate that this framework can customize grippers for various daily objects with different geometries. The results also show the extensional abilities of customizing a gripper for multiple or heavy objects. This framework enables the rapid design and fabrication of grippers optimized for specific tasks while maintaining versatility for handling various objects.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1641-1648"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3522838
Martín Bayón-Gutiérrez;Natalia Prieto-Fernández;María Teresa García-Ordás;José Alberto Benítez-Andrades;Héctor Alaiz-Moretón;Giorgio Grisetti
Robotic mobile platforms are key building blocks for numerous applications and cooperation between robots and humans is a key aspect to enhance productivity and reduce labor cost. To operate safely, robots typically rely on a custom map of the environment that depends on the sensor configuration of the platform. In contrast, blueprints stand as an abstract representation of the environment. The use of both CAD and SLAM maps allows robots to collaborate using the blueprint as a common language, while also easing the control for non-robotics experts. In this work we present an adaptive system to project a 2D pose in the blueprint to the SLAM map and vice-versa. Previous work in the literature aims at morphing a SLAM map to a previously available map. In contrast, CAD2SLAM does not alter the internal map representation used by the SLAM and localization algorithms running on the robot, preserving its original properties. We believe that our system is beneficial for the control and supervision of multiple heterogeneous robotic platforms that are monitored and controlled through the CAD map. Finally, we present a set of experiments that support our claims as well as open-source implementation.
{"title":"CAD2SLAM: Adaptive Projection Between CAD Blueprints and SLAM Maps","authors":"Martín Bayón-Gutiérrez;Natalia Prieto-Fernández;María Teresa García-Ordás;José Alberto Benítez-Andrades;Héctor Alaiz-Moretón;Giorgio Grisetti","doi":"10.1109/LRA.2024.3522838","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522838","url":null,"abstract":"Robotic mobile platforms are key building blocks for numerous applications and cooperation between robots and humans is a key aspect to enhance productivity and reduce labor cost. To operate safely, robots typically rely on a custom map of the environment that depends on the sensor configuration of the platform. In contrast, blueprints stand as an abstract representation of the environment. The use of both CAD and SLAM maps allows robots to collaborate using the blueprint as a common language, while also easing the control for non-robotics experts. In this work we present an adaptive system to project a 2D pose in the blueprint to the SLAM map and vice-versa. Previous work in the literature aims at morphing a SLAM map to a previously available map. In contrast, \u0000<italic>CAD2SLAM</i>\u0000 does not alter the internal map representation used by the SLAM and localization algorithms running on the robot, preserving its original properties. We believe that our system is beneficial for the control and supervision of multiple heterogeneous robotic platforms that are monitored and controlled through the CAD map. Finally, we present a set of experiments that support our claims as well as open-source implementation.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1529-1536"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10816387","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142938311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3522780
Haonan Shi;Luzheng Bi;Zhenge Yang;Haorui Ge;Weijie Fei;Ling Wang
Brain-machine interfaces (BMIs) can enable humans to bypass the peripheral nervous system and directly control devices through the central nervous system. In this way, operators' hands are freed up, allowing them to interact with other devices, thus enabling multitasking operations. In this letter, to improve the performance of air-ground collaborative systems, we propose an adaptive model prediction control framework of brain-controlled air-ground collaboration systems, which consists of a BMI with a probabilistic output model, an interface model based on fuzzy logic, and an adaptive model-predictive-control shared controller based on game theory. We establish a human-in-the-loop experimental platform to validate the proposed method by trajectory tracking and obstacle avoidance scenarios. The experimental results show the effectiveness of the proposed method in improving performance and decreasing operators' workload. This work can contribute to the research and development of air-ground collaboration and provide new insights into the study of human-machine integration.
{"title":"Adaptive Model Prediction Control Framework With Game Theory for Brain-Controlled Air-Ground Collaborative Autonomous System","authors":"Haonan Shi;Luzheng Bi;Zhenge Yang;Haorui Ge;Weijie Fei;Ling Wang","doi":"10.1109/LRA.2024.3522780","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522780","url":null,"abstract":"Brain-machine interfaces (BMIs) can enable humans to bypass the peripheral nervous system and directly control devices through the central nervous system. In this way, operators' hands are freed up, allowing them to interact with other devices, thus enabling multitasking operations. In this letter, to improve the performance of air-ground collaborative systems, we propose an adaptive model prediction control framework of brain-controlled air-ground collaboration systems, which consists of a BMI with a probabilistic output model, an interface model based on fuzzy logic, and an adaptive model-predictive-control shared controller based on game theory. We establish a human-in-the-loop experimental platform to validate the proposed method by trajectory tracking and obstacle avoidance scenarios. The experimental results show the effectiveness of the proposed method in improving performance and decreasing operators' workload. This work can contribute to the research and development of air-ground collaboration and provide new insights into the study of human-machine integration.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1577-1584"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142937837","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3523229
Junyi Shen;Tetsuro Miyazaki;Kenji Kawashima
The nonlinearities of soft robots bring control challenges like hysteresis but also provide them with computational capacities. This letter introduces a fuzzy pneumatic physical reservoir computing (FPRC) model for feedforward hysteresis compensation in motion tracking control of soft actuators. Our method utilizes a pneumatic bending actuator as a physical reservoir with nonlinear computing capacities to control another pneumatic bending actuator. The FPRC model employs a Takagi-Sugeno (T-S) fuzzy logic to process outputs from the physical reservoir. The proposed FPRC model shows equivalent training performance to an Echo State Network (ESN) model, whereas it exhibits better test accuracies with significantly reduced execution time. Experiments validate the FPRC model's effectiveness in controlling the bending motion of a pneumatic soft actuator with open-loop and closed-loop control system setups. The proposed FPRC model's robustness against environmental disturbances has also been experimentally verified. To the authors' knowledge, this is the first implementation of a physical system in the feedforward hysteresis compensation model for controlling soft actuators. This study is expected to advance physical reservoir computing in nonlinear control applications and extend the feedforward hysteresis compensation methods for controlling soft actuators.
{"title":"Control Pneumatic Soft Bending Actuator With Feedforward Hysteresis Compensation by Pneumatic Physical Reservoir Computing","authors":"Junyi Shen;Tetsuro Miyazaki;Kenji Kawashima","doi":"10.1109/LRA.2024.3523229","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523229","url":null,"abstract":"The nonlinearities of soft robots bring control challenges like hysteresis but also provide them with computational capacities. This letter introduces a fuzzy pneumatic physical reservoir computing (FPRC) model for feedforward hysteresis compensation in motion tracking control of soft actuators. Our method utilizes a pneumatic bending actuator as a physical reservoir with nonlinear computing capacities to control another pneumatic bending actuator. The FPRC model employs a Takagi-Sugeno (T-S) fuzzy logic to process outputs from the physical reservoir. The proposed FPRC model shows equivalent training performance to an Echo State Network (ESN) model, whereas it exhibits better test accuracies with significantly reduced execution time. Experiments validate the FPRC model's effectiveness in controlling the bending motion of a pneumatic soft actuator with open-loop and closed-loop control system setups. The proposed FPRC model's robustness against environmental disturbances has also been experimentally verified. To the authors' knowledge, this is the first implementation of a physical system in the feedforward hysteresis compensation model for controlling soft actuators. This study is expected to advance physical reservoir computing in nonlinear control applications and extend the feedforward hysteresis compensation methods for controlling soft actuators.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1664-1671"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10816480","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The stability of visual odometry (VO) systems is undermined by degraded image quality, especially in environments with significant illumination changes. This study employs a deep reinforcement learning (DRL) framework to train agents for exposure control, aiming to enhance imaging performance in challenging conditions. A lightweight image simulator is developed to facilitate the training process, enabling the diversification of image exposure and sequence trajectory. This setup enables completely offline training, eliminating the need for direct interaction with camera hardware and the real environments. Different levels of reward functions are crafted to enhance the VO systems, equipping the DRL agents with varying intelligence. Extensive experiments have shown that our exposure control agents achieve superior efficiency—with an average inference duration of 1.58 ms per frame on a CPU—and respond more quickly than traditional feedback control schemes. By choosing an appropriate reward function, agents acquire an intelligent understanding of motion trends and can anticipate future changes in illumination. This predictive capability allows VO systems to deliver more stable and precise odometry results.
{"title":"Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning","authors":"Shuyang Zhang;Jinhao He;Yilong Zhu;Jin Wu;Jie Yuan","doi":"10.1109/LRA.2024.3523224","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523224","url":null,"abstract":"The stability of visual odometry (VO) systems is undermined by degraded image quality, especially in environments with significant illumination changes. This study employs a deep reinforcement learning (DRL) framework to train agents for exposure control, aiming to enhance imaging performance in challenging conditions. A lightweight image simulator is developed to facilitate the training process, enabling the diversification of image exposure and sequence trajectory. This setup enables completely offline training, eliminating the need for direct interaction with camera hardware and the real environments. Different levels of reward functions are crafted to enhance the VO systems, equipping the DRL agents with varying intelligence. Extensive experiments have shown that our exposure control agents achieve superior efficiency—with an average inference duration of 1.58 ms per frame on a CPU—and respond more quickly than traditional feedback control schemes. By choosing an appropriate reward function, agents acquire an intelligent understanding of motion trends and can anticipate future changes in illumination. This predictive capability allows VO systems to deliver more stable and precise odometry results.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1609-1616"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3523232
Byoungkwon Yoon;Seokhyun Hong;Dongjun Lee
We propose a novel cross-source point cloud registration (CSPR) method for USV-AAV cooperation in lentic environments. In the wild outdoors, which is the typical working domain of the USV-AAV team, CSPR faces significant challenges due to platform-domain problems (complex unstructured surroundings and viewing angle difference) in addition to sensor-domain problems (varying density, noise pattern, and scale). These characteristics make large discrepancies in local geometry, causing existing CSPR methods that rely on point-to-point correspondence based on local geometry around key points (e.g. surface normal, shape function, angle) to struggle. To address this challenge, we propose the novel concept of a directional correspondence-based iterative cross-source point cloud registration algorithm. Instead of using point-to-point correspondence under large discrepancies in local geometry, we build correspondence about directions to enable robust registration in the wild outdoors. Also, since the proposed directional correspondence uses bearing angle and normalized coordinate, we can separate scale estimation with transformation, effectively resolving the problem of different scales between two point clouds. Our algorithm outperforms the state-of-the-art methods, achieving an average error of $1.60^circ$ for rotation and 1.83% for translation. Additionally, we demonstrated a USV-AAV team operation with enhanced visual information achieved with the proposed method.
{"title":"Directional Correspondence Based Cross-Source Point Cloud Registration for USV-AAV Cooperation in Lentic Environments","authors":"Byoungkwon Yoon;Seokhyun Hong;Dongjun Lee","doi":"10.1109/LRA.2024.3523232","DOIUrl":"https://doi.org/10.1109/LRA.2024.3523232","url":null,"abstract":"We propose a novel cross-source point cloud registration (CSPR) method for USV-AAV cooperation in lentic environments. In the wild outdoors, which is the typical working domain of the USV-AAV team, CSPR faces significant challenges due to platform-domain problems (complex unstructured surroundings and viewing angle difference) in addition to sensor-domain problems (varying density, noise pattern, and scale). These characteristics make large discrepancies in local geometry, causing existing CSPR methods that rely on point-to-point correspondence based on local geometry around key points (e.g. surface normal, shape function, angle) to struggle. To address this challenge, we propose the novel concept of a directional correspondence-based iterative cross-source point cloud registration algorithm. Instead of using point-to-point correspondence under large discrepancies in local geometry, we build correspondence about directions to enable robust registration in the wild outdoors. Also, since the proposed directional correspondence uses bearing angle and normalized coordinate, we can separate scale estimation with transformation, effectively resolving the problem of different scales between two point clouds. Our algorithm outperforms the state-of-the-art methods, achieving an average error of \u0000<inline-formula><tex-math>$1.60^circ$</tex-math></inline-formula>\u0000 for rotation and 1.83% for translation. Additionally, we demonstrated a USV-AAV team operation with enhanced visual information achieved with the proposed method.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1601-1608"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10816390","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-26DOI: 10.1109/LRA.2024.3522841
Daichi Yashima;Ryosuke Korekata;Komei Sugiura
Growing labor shortages are increasing the demand for domestic service robots (DSRs) to assist in various settings. In this study, we develop a DSR that transports everyday objects to specified pieces of furniture based on open-vocabulary instructions. Our approach focuses on retrieving images of target objects and receptacles from pre-collected images of indoor environments. For example, given an instruction “Please get the right red towel hanging on the metal towel rack and put it in the white washing machine on the left,” the DSR is expected to carry the red towel to the washing machine based on the retrieved images. This is challenging because the correct images should be retrieved from thousands of collected images, which may include many images of similar towels and appliances. To address this, we propose RelaX-Former, which learns diverse and robust representations from among positive, unlabeled positive, and negative samples. We evaluated RelaX-Former on a dataset containing real-world indoor images and human annotated instructions including complex referring expressions. The experimental results demonstrate that RelaX-Former outperformed existing baseline models across standard image retrieval metrics. Moreover, we performed physical experiments using a DSR to evaluate the performance of our approach in a zero-shot transfer setting. The experiments involved the DSR to carry objects to specific receptacles based on open-vocabulary instructions, achieving an overall success rate of 75%.
{"title":"Open-Vocabulary Mobile Manipulation Based on Double Relaxed Contrastive Learning With Dense Labeling","authors":"Daichi Yashima;Ryosuke Korekata;Komei Sugiura","doi":"10.1109/LRA.2024.3522841","DOIUrl":"https://doi.org/10.1109/LRA.2024.3522841","url":null,"abstract":"Growing labor shortages are increasing the demand for domestic service robots (DSRs) to assist in various settings. In this study, we develop a DSR that transports everyday objects to specified pieces of furniture based on open-vocabulary instructions. Our approach focuses on retrieving images of target objects and receptacles from pre-collected images of indoor environments. For example, given an instruction “Please get the right red towel hanging on the metal towel rack and put it in the white washing machine on the left,” the DSR is expected to carry the red towel to the washing machine based on the retrieved images. This is challenging because the correct images should be retrieved from thousands of collected images, which may include many images of similar towels and appliances. To address this, we propose RelaX-Former, which learns diverse and robust representations from among positive, unlabeled positive, and negative samples. We evaluated RelaX-Former on a dataset containing real-world indoor images and human annotated instructions including complex referring expressions. The experimental results demonstrate that RelaX-Former outperformed existing baseline models across standard image retrieval metrics. Moreover, we performed physical experiments using a DSR to evaluate the performance of our approach in a zero-shot transfer setting. The experiments involved the DSR to carry objects to specific receptacles based on open-vocabulary instructions, achieving an overall success rate of 75%.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 2","pages":"1728-1735"},"PeriodicalIF":4.6,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142976068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：