Pub Date : 2024-11-08DOI: 10.1109/LRA.2024.3494653
Sike Zeng;Xi Chen;Li Chai
Multi-agent path finding (MAPF) involves finding collision-free paths for multiple agents while minimizing the total path costs. Explicit estimation conflict-based search (EECBS) represents a state-of-the-art variant of the widely used conflict-based search (CBS) method, offering bounded-suboptimal solutions. However, both CBS and its variants rely on pairwise conflict resolution methods. A conflict boom means many conflicts occur at one location, which frequently exists in scenarios that a large number of agents operate in small space, and usually leads to heavy computational burden. The location that conflict boom occurs is regarded as conflict boom vertex. This letter proposes a novel method, the Virtual Obstacles Regulation, to expedite algorithmic solving processes (such as EECBS) for MAPF. The proposed method identifies conflicts boom vertices and strategically regulates them as global or local virtual obstacles to circumvent concentrated conflicts. Then, the pairwise conflict resolution processes on conflicts boom vertices are significantly simplified, hence accelerating overall algorithm runtime–often dominated by conflict resolution. Numerical studies validate the efficacy of this approach.
{"title":"Virtual Obstacles Regulation for Multi-Agent Path Finding","authors":"Sike Zeng;Xi Chen;Li Chai","doi":"10.1109/LRA.2024.3494653","DOIUrl":"https://doi.org/10.1109/LRA.2024.3494653","url":null,"abstract":"Multi-agent path finding (MAPF) involves finding collision-free paths for multiple agents while minimizing the total path costs. Explicit estimation conflict-based search (EECBS) represents a state-of-the-art variant of the widely used conflict-based search (CBS) method, offering bounded-suboptimal solutions. However, both CBS and its variants rely on pairwise conflict resolution methods. A conflict boom means many conflicts occur at one location, which frequently exists in scenarios that a large number of agents operate in small space, and usually leads to heavy computational burden. The location that conflict boom occurs is regarded as conflict boom vertex. This letter proposes a novel method, the Virtual Obstacles Regulation, to expedite algorithmic solving processes (such as EECBS) for MAPF. The proposed method identifies conflicts boom vertices and strategically regulates them as global or local virtual obstacles to circumvent concentrated conflicts. Then, the pairwise conflict resolution processes on conflicts boom vertices are significantly simplified, hence accelerating overall algorithm runtime–often dominated by conflict resolution. Numerical studies validate the efficacy of this approach.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11417-11424"},"PeriodicalIF":4.6,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-08DOI: 10.1109/LRA.2024.3494652
Wanzhang Li;Fukun Yin;Wen Liu;Yiying Yang;Xin Chen;Biao Jiang;Gang Yu;Jiayuan Fan
Modeling large-scale scenes from multi-view images is challenging due to the trade-off dilemma between visual quality and computational cost. Existing NeRF-based methods have made advancements in neural implicit representation through volumetric ray-marching, but still struggle to deal with cubically growing sampling space in large-scale scenes. Fortunately, the rendering approach based on 3D Gaussian splatting (3DGS) has shown promising results, inspiring further exploration in the splatting setting. However, 3DGS has the limitation of inadequate Gaussian points for modeling distant backgrounds, leading to “splotchy” artifacts. To address this problem, we introduce a novel hybrid neural representation called Unbounded 3D Gaussian. For foreground area, we employs an explicit 3D Gaussian representation to efficiently model the geometry and appearance through splatting weighted Gaussians. For far-away background, we additionally introduce an implicit module comprising Multi-layer Perceptions (MLPs) to directly predict far-away background colors from positional encodings of view positions and ray directions. Furthermore, we design a seamless blending mechanism between the color predictions of the explicit splatting and implicit branches to reconstruct holistic scenes. Extensive experiments demonstrate that our proposed Unbounded-GS inherits the advantages of both faster convergence and high-fidelity rendering quality.
{"title":"Unbounded-GS: Extending 3D Gaussian Splatting With Hybrid Representation for Unbounded Large-Scale Scene Reconstruction","authors":"Wanzhang Li;Fukun Yin;Wen Liu;Yiying Yang;Xin Chen;Biao Jiang;Gang Yu;Jiayuan Fan","doi":"10.1109/LRA.2024.3494652","DOIUrl":"https://doi.org/10.1109/LRA.2024.3494652","url":null,"abstract":"Modeling large-scale scenes from multi-view images is challenging due to the trade-off dilemma between visual quality and computational cost. Existing NeRF-based methods have made advancements in neural implicit representation through volumetric ray-marching, but still struggle to deal with cubically growing sampling space in large-scale scenes. Fortunately, the rendering approach based on 3D Gaussian splatting (3DGS) has shown promising results, inspiring further exploration in the splatting setting. However, 3DGS has the limitation of inadequate Gaussian points for modeling distant backgrounds, leading to “splotchy” artifacts. To address this problem, we introduce a novel hybrid neural representation called Unbounded 3D Gaussian. For foreground area, we employs an explicit 3D Gaussian representation to efficiently model the geometry and appearance through splatting weighted Gaussians. For far-away background, we additionally introduce an implicit module comprising Multi-layer Perceptions (MLPs) to directly predict far-away background colors from positional encodings of view positions and ray directions. Furthermore, we design a seamless blending mechanism between the color predictions of the explicit splatting and implicit branches to reconstruct holistic scenes. Extensive experiments demonstrate that our proposed Unbounded-GS inherits the advantages of both faster convergence and high-fidelity rendering quality.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11529-11536"},"PeriodicalIF":4.6,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142672009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-08DOI: 10.1109/LRA.2024.3495374
Junhyoung Kwon;Junchi Sa;Serim Lee;Gunhee Jang
Magnetic helical robots (MHRs) can be actuated by a rotating magnetic field (RMF) and can tunnel through a clogged blood vessel using their rotational motion. We propose a method minimizing the voltage required for each coil to generate the RMF for a magnetic navigation system (MNS). The proposed method maximizes the RMF under the rated voltage of the MNS. In addition, to suppress the increase in impedance during high-speed rotational motion of the MHR, the proposed method utilizes the resonance to which the minimax optimization method is applied. The voltage needed to generate the RMF was analytically derived, with the goal of a fast optimization process in mind, so that the MHR could be controlled in real-time. The proposed method was experimentally verified by measuring the magnetic flux density. In addition, we demonstrated the enhanced navigating and tunneling performance of the MHR from in vitro experiments. Finally, we validated the navigating and tunneling motion of the MHR in an in vivo experiment within a superficial femoral artery of a pig.
{"title":"A Voltage Minimization Control Method for Magnetic Navigation Systems to Enhance the Rotating Magnetic Field","authors":"Junhyoung Kwon;Junchi Sa;Serim Lee;Gunhee Jang","doi":"10.1109/LRA.2024.3495374","DOIUrl":"https://doi.org/10.1109/LRA.2024.3495374","url":null,"abstract":"Magnetic helical robots (MHRs) can be actuated by a rotating magnetic field (RMF) and can tunnel through a clogged blood vessel using their rotational motion. We propose a method minimizing the voltage required for each coil to generate the RMF for a magnetic navigation system (MNS). The proposed method maximizes the RMF under the rated voltage of the MNS. In addition, to suppress the increase in impedance during high-speed rotational motion of the MHR, the proposed method utilizes the resonance to which the minimax optimization method is applied. The voltage needed to generate the RMF was analytically derived, with the goal of a fast optimization process in mind, so that the MHR could be controlled in real-time. The proposed method was experimentally verified by measuring the magnetic flux density. In addition, we demonstrated the enhanced navigating and tunneling performance of the MHR from \u0000<italic>in vitro</i>\u0000 experiments. Finally, we validated the navigating and tunneling motion of the MHR in an \u0000<italic>in vivo</i>\u0000 experiment within a superficial femoral artery of a pig.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11561-11568"},"PeriodicalIF":4.6,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142672096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-07DOI: 10.1109/LRA.2024.3494655
Lev Smolentsev;Alexandre Krupa;François Chaumette
In this letter, we propose a shape visual servoing approach for manipulating a suspended cable attached between two quadrotor drones. A leader-follower control strategy is presented, where a human operator controls the rigid motion of the cable by teleoperating one drone (the leader), while the second drone (the follower) performs a shape visual servoing task to autonomously apply a desired deformation to the cable. The proposed shape visual servoing approach uses an RGB-D camera embedded on the follower drone and has the advantage to rely on a simple geometrical model of the cable that only requires the knowledge of its length. In the same time, our control strategy maintains the best visibility of the cable in the camera field of view. A robust image processing pipeline allows detecting and tracking in real-time the cable shape from the data provided by the onboard RGB-D camera. Experimental results demonstrate the effectiveness of the proposed visual control approach to shape a flexible cable into a desired shape. In addition, we demonstrate experimentally that such system can be used to perform an aerial transport task by grasping with the cable an object fitted with a hook, then moving and releasing it at another location.
{"title":"Shape Visual Servoing of a Cable Suspended Between Two Drones","authors":"Lev Smolentsev;Alexandre Krupa;François Chaumette","doi":"10.1109/LRA.2024.3494655","DOIUrl":"https://doi.org/10.1109/LRA.2024.3494655","url":null,"abstract":"In this letter, we propose a shape visual servoing approach for manipulating a suspended cable attached between two quadrotor drones. A leader-follower control strategy is presented, where a human operator controls the rigid motion of the cable by teleoperating one drone (the leader), while the second drone (the follower) performs a shape visual servoing task to autonomously apply a desired deformation to the cable. The proposed shape visual servoing approach uses an RGB-D camera embedded on the follower drone and has the advantage to rely on a simple geometrical model of the cable that only requires the knowledge of its length. In the same time, our control strategy maintains the best visibility of the cable in the camera field of view. A robust image processing pipeline allows detecting and tracking in real-time the cable shape from the data provided by the onboard RGB-D camera. Experimental results demonstrate the effectiveness of the proposed visual control approach to shape a flexible cable into a desired shape. In addition, we demonstrate experimentally that such system can be used to perform an aerial transport task by grasping with the cable an object fitted with a hook, then moving and releasing it at another location.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11473-11480"},"PeriodicalIF":4.6,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636274","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-07DOI: 10.1109/LRA.2024.3494662
Song Lin;Jiansheng Dai;Yifeng Song;Hongguang Wang;Bingbing Yuan;Yingbin Feng
To improve the environmental adaptability of the mechanism, this letter proposes a metamorphic mechanism configuration synthesis method based on environmental constraint characteristics. In particular, a novel 6R line-symmetric double-centered metamorphic mechanism is synthesized and the multi-bifurcation characteristics and environmental adaptability are analyzed and verified. Firstly, a configuration synthesis method is proposed based on the bifocal geometric constraints of the elliptical oblique section of the pipe. The variation of the instantaneous mobility with the number of links is derived and m = n =3 is the minimum number of links that the mechanism is movable. Then, for the 6R mechanism, the joint velocity solution spaces of all the motion branches are revealed through the closed-loop and higher-order kinematics. Further, the correlation between the joint angle, configuration, and environment layers is demonstrated. The results show that the 6R mechanism has four motion branches ( MB $_{1}$ , MB $_{2}$ , MB $_{3}$ , and MB $_{4}$ ) with different topological structures corresponding to different pipe environments and four 4R serial motion branches ( SMB $_{1}$ , SMB $_{2}$ , SMB $_{3}$ , and SMB $_{4}$ ). Eventually, a wheeled metamorphic robot is designed, and the controllable motion branch transformation of the 6R mechanism and the pipe environment adaptability of the robot are proved by prototype experiments.
{"title":"Multi-Bifurcation and Environmental Adaptability of a Novel Line-Symmetric Double-Centered Metamorphic Mechanism","authors":"Song Lin;Jiansheng Dai;Yifeng Song;Hongguang Wang;Bingbing Yuan;Yingbin Feng","doi":"10.1109/LRA.2024.3494662","DOIUrl":"https://doi.org/10.1109/LRA.2024.3494662","url":null,"abstract":"To improve the environmental adaptability of the mechanism, this letter proposes a metamorphic mechanism configuration synthesis method based on environmental constraint characteristics. In particular, a novel 6R line-symmetric double-centered metamorphic mechanism is synthesized and the multi-bifurcation characteristics and environmental adaptability are analyzed and verified. Firstly, a configuration synthesis method is proposed based on the bifocal geometric constraints of the elliptical oblique section of the pipe. The variation of the instantaneous mobility with the number of links is derived and \u0000<italic>m</i>\u0000=\u0000<italic>n</i>\u0000=3 is the minimum number of links that the mechanism is movable. Then, for the 6R mechanism, the joint velocity solution spaces of all the motion branches are revealed through the closed-loop and higher-order kinematics. Further, the correlation between the joint angle, configuration, and environment layers is demonstrated. The results show that the 6R mechanism has four motion branches (\u0000<italic>MB</i>\u0000<inline-formula><tex-math>$_{1}$</tex-math></inline-formula>\u0000, \u0000<italic>MB</i>\u0000<inline-formula><tex-math>$_{2}$</tex-math></inline-formula>\u0000, \u0000<italic>MB</i>\u0000<inline-formula><tex-math>$_{3}$</tex-math></inline-formula>\u0000, and \u0000<italic>MB</i>\u0000<inline-formula><tex-math>$_{4}$</tex-math></inline-formula>\u0000) with different topological structures corresponding to different pipe environments and four 4R serial motion branches (\u0000<italic>SMB</i>\u0000<inline-formula><tex-math>$_{1}$</tex-math></inline-formula>\u0000, \u0000<italic>SMB</i>\u0000<inline-formula><tex-math>$_{2}$</tex-math></inline-formula>\u0000, \u0000<italic>SMB</i>\u0000<inline-formula><tex-math>$_{3}$</tex-math></inline-formula>\u0000, and \u0000<italic>SMB</i>\u0000<inline-formula><tex-math>$_{4}$</tex-math></inline-formula>\u0000). Eventually, a wheeled metamorphic robot is designed, and the controllable motion branch transformation of the 6R mechanism and the pipe environment adaptability of the robot are proved by prototype experiments.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11746-11753"},"PeriodicalIF":4.6,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The fusion of event cameras and conventional frame cameras is a novel research field, and a stereo structure consisting of an event camera and a frame camera can incorporate the advantages of both. This letter develops a dynamic calibration framework for the event-frame stereo camera system. In this framework, the first step is to complete the initial detection on a circle-grid calibration pattern, and a sliding-window time matching method is proposed to match the event-frame pairs. Then, a refining method is devised for two cameras to get the accurate information of the pattern. Particularly, for the event camera, a patch-size motion compensation method with high computational efficiency is designed to achieve time synchronization for two cameras and fit circles in an image of warped events. Finally, the pose between two cameras is globally optimized by constructing a pose-landmark graph with two types of edges. The proposed calibration framework has the advantages of high real-time performance and easy deployment, and its effectiveness is verified by experiments based on self-recorded datasets.
{"title":"A Dynamic Calibration Framework for the Event-Frame Stereo Camera System","authors":"Rui Hu;Jürgen Kogler;Margrit Gelautz;Min Lin;Yuanqing Xia","doi":"10.1109/LRA.2024.3491426","DOIUrl":"https://doi.org/10.1109/LRA.2024.3491426","url":null,"abstract":"The fusion of event cameras and conventional frame cameras is a novel research field, and a stereo structure consisting of an event camera and a frame camera can incorporate the advantages of both. This letter develops a dynamic calibration framework for the event-frame stereo camera system. In this framework, the first step is to complete the initial detection on a circle-grid calibration pattern, and a sliding-window time matching method is proposed to match the event-frame pairs. Then, a refining method is devised for two cameras to get the accurate information of the pattern. Particularly, for the event camera, a patch-size motion compensation method with high computational efficiency is designed to achieve time synchronization for two cameras and fit circles in an image of warped events. Finally, the pose between two cameras is globally optimized by constructing a pose-landmark graph with two types of edges. The proposed calibration framework has the advantages of high real-time performance and easy deployment, and its effectiveness is verified by experiments based on self-recorded datasets.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11465-11472"},"PeriodicalIF":4.6,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636428","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-04DOI: 10.1109/LRA.2024.3491412
Ahmed Zermane;Léo Moussafir;Youcan Yan;Abderrahmane Kheddar
We present a planning and control method that computes a minimal sequence of pokes to slide a given object from an initial pose to a desired final one (or as close to it as possible) on a planar surface. Both planning and control are based on impact models to generate pokes. Our framework takes into account the object's dynamics with a rich contact model and parameters to plan the poking sequence. The planning is conducted in the joint-space and generates trajectories tracked using an impact-aware QP control, which corrects for post-pokes errors using discrete visual feedback. We implemented our method on a Panda robot arm and assessed its versatility and robustness. The experimental results show that the proposed poking approach can bring the object to the desired position and orientation with minimal errors (0.05 m for translation and 0.2 rd for rotation), highlighting its potential application in diverse industrial scenarios such as logistics.
{"title":"Minimal Impact Pokes to Place Objects on Planar Surfaces","authors":"Ahmed Zermane;Léo Moussafir;Youcan Yan;Abderrahmane Kheddar","doi":"10.1109/LRA.2024.3491412","DOIUrl":"https://doi.org/10.1109/LRA.2024.3491412","url":null,"abstract":"We present a planning and control method that computes a minimal sequence of pokes to slide a given object from an initial pose to a desired final one (or as close to it as possible) on a planar surface. Both planning and control are based on impact models to generate pokes. Our framework takes into account the object's dynamics with a rich contact model and parameters to plan the poking sequence. The planning is conducted in the joint-space and generates trajectories tracked using an impact-aware QP control, which corrects for post-pokes errors using discrete visual feedback. We implemented our method on a Panda robot arm and assessed its versatility and robustness. The experimental results show that the proposed poking approach can bring the object to the desired position and orientation with minimal errors (0.05 m for translation and 0.2 rd for rotation), highlighting its potential application in diverse industrial scenarios such as logistics.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11393-11400"},"PeriodicalIF":4.6,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We introduce a new multi-modal collaborative SLAM (C-SLAM) dataset for multiple service robots in various indoor service environments, called C -SLAM dataset in S ervice E nvironments ( CSE ). We use the NVIDIA Isaac Sim to generate data in various indoor service environments with the challenges that may occur in real-world service environments. By using the simulator, we can provide precisely time-synchronized sensor data, such as stereo RGB/depth, IMU, and ground truth (GT) poses. We configure three common indoor service environments ( Hospital , Office , and Warehouse ), each featuring dynamic objects performing motions suited to the environment. In addition, we drive the robots to mimic the actions of real service robots. Through these factors, we generate a realistic C-SLAM dataset for multiple service robots. We demonstrate our CSE dataset by evaluating diverse state-of-the-art single-robot SLAM and multi-robot SLAM methods. Additionally, we provide a detailed tutorial on generating C-SLAM data using the simulator.
我们介绍了一种新的多模式协作 SLAM（C-SLAM）数据集，用于在各种室内服务环境中使用多个服务机器人，称为服务环境中的 C-SLAM 数据集（CSE）。我们使用英伟达 Isaac Sim 在各种室内服务环境中生成数据，以应对真实世界服务环境中可能出现的挑战。通过使用模拟器，我们可以提供精确的时间同步传感器数据，如立体 RGB/深度、IMU 和地面实况（GT）姿势。我们配置了三种常见的室内服务环境（医院、办公室和仓库），每种环境中的动态物体都会做出与环境相适应的动作。此外，我们还模仿真实服务机器人的动作来驱动机器人。通过这些因素，我们为多个服务机器人生成了一个逼真的 C-SLAM 数据集。我们通过评估各种最先进的单机器人 SLAM 和多机器人 SLAM 方法来展示我们的 CSE 数据集。此外，我们还提供了使用模拟器生成 C-SLAM 数据的详细教程。
{"title":"A Benchmark Dataset for Collaborative SLAM in Service Environments","authors":"Harin Park;Inha Lee;Minje Kim;Hyungyu Park;Kyungdon Joo","doi":"10.1109/LRA.2024.3491415","DOIUrl":"https://doi.org/10.1109/LRA.2024.3491415","url":null,"abstract":"We introduce a new multi-modal collaborative SLAM (C-SLAM) dataset for multiple service robots in various indoor service environments, called \u0000<monospace>C</monospace>\u0000-SLAM dataset in \u0000<monospace>S</monospace>\u0000ervice \u0000<monospace>E</monospace>\u0000nvironments (\u0000<monospace>CSE</monospace>\u0000). We use the NVIDIA Isaac Sim to generate data in various indoor service environments with the challenges that may occur in real-world service environments. By using the simulator, we can provide precisely time-synchronized sensor data, such as stereo RGB/depth, IMU, and ground truth (GT) poses. We configure three common indoor service environments (\u0000<italic>Hospital</i>\u0000, \u0000<italic>Office</i>\u0000, and \u0000<italic>Warehouse</i>\u0000), each featuring dynamic objects performing motions suited to the environment. In addition, we drive the robots to mimic the actions of real service robots. Through these factors, we generate a realistic C-SLAM dataset for multiple service robots. We demonstrate our \u0000<monospace>CSE</monospace>\u0000 dataset by evaluating diverse state-of-the-art single-robot SLAM and multi-robot SLAM methods. Additionally, we provide a detailed tutorial on generating C-SLAM data using the simulator.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11337-11344"},"PeriodicalIF":4.6,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600210","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-04DOI: 10.1109/LRA.2024.3491428
Alessio Caporali;Kevin Galassi;Gianluca Palli
The perception of Deformable Linear Objects (DLOs) is a challenging task due to their complex and ambiguous appearance, lack of discernible features, typically small sizes, and deformability. Despite these challenges, achieving a robust and effective segmentation of DLOs is crucial to introduce robots into environments where they are currently underrepresented, such as domestic and complex industrial settings. In this context, the integration of language-based inputs can simplify the perception task while also enabling the possibility of introducing robots as human companions. Therefore, this letter proposes a novel architecture for the perception of DLOs, wherein the input image is augmented with a text-based prompt guiding the segmentation of the target DLO. After encoding the image and text separately, a Perceiver-inspired structure is exploited to compress the concatenated data into transformer layers and generate the output mask from a latent vector representation. The method is experimentally evaluated on real-world images of DLOs like electrical cables and ropes, validating its efficacy and efficiency in real practical scenarios.
{"title":"DLO Perceiver: Grounding Large Language Model for Deformable Linear Objects Perception","authors":"Alessio Caporali;Kevin Galassi;Gianluca Palli","doi":"10.1109/LRA.2024.3491428","DOIUrl":"https://doi.org/10.1109/LRA.2024.3491428","url":null,"abstract":"The perception of Deformable Linear Objects (DLOs) is a challenging task due to their complex and ambiguous appearance, lack of discernible features, typically small sizes, and deformability. Despite these challenges, achieving a robust and effective segmentation of DLOs is crucial to introduce robots into environments where they are currently underrepresented, such as domestic and complex industrial settings. In this context, the integration of language-based inputs can simplify the perception task while also enabling the possibility of introducing robots as human companions. Therefore, this letter proposes a novel architecture for the perception of DLOs, wherein the input image is augmented with a text-based prompt guiding the segmentation of the target DLO. After encoding the image and text separately, a Perceiver-inspired structure is exploited to compress the concatenated data into transformer layers and generate the output mask from a latent vector representation. The method is experimentally evaluated on real-world images of DLOs like electrical cables and ropes, validating its efficacy and efficiency in real practical scenarios.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11385-11392"},"PeriodicalIF":4.6,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10742556","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-04DOI: 10.1109/LRA.2024.3491417
Lucas Nogueira Nobrega;Ewerton de Oliveira;Martin Saska;Tiago Nascimento
The human-robot interaction (HRI) is a growing area of research. In HRI, complex command (action) classification is still an open problem that usually prevents the real applicability of such a technique. The literature presents some works that use neural networks to detect these actions. However, occlusion is still a major issue in HRI, especially when using uncrewed aerial vehicles (UAVs), since, during the robot's movement, the human operator is often out of the robot's field of view. Furthermore, in multi-robot scenarios, distributed training is also an open problem. In this sense, this work proposes an action recognition and control approach based on Long Short-Term Memory (LSTM) Deep Neural Networks with two layers in association with three densely connected layers and Federated Learning (FL) embedded in multiple drones. The FL enabled our approach to be trained in a distributed fashion, i.e., access to data without the need for cloud or other repositories, which facilitates the multi-robot system's learning. Furthermore, our multi-robot approach results also prevented occlusion situations, with experiments with real robots achieving an accuracy greater than 96%.
{"title":"Proximal Control of UAVs With Federated Learning for Human-Robot Collaborative Domains","authors":"Lucas Nogueira Nobrega;Ewerton de Oliveira;Martin Saska;Tiago Nascimento","doi":"10.1109/LRA.2024.3491417","DOIUrl":"https://doi.org/10.1109/LRA.2024.3491417","url":null,"abstract":"The human-robot interaction (HRI) is a growing area of research. In HRI, complex command (action) classification is still an open problem that usually prevents the real applicability of such a technique. The literature presents some works that use neural networks to detect these actions. However, occlusion is still a major issue in HRI, especially when using uncrewed aerial vehicles (UAVs), since, during the robot's movement, the human operator is often out of the robot's field of view. Furthermore, in multi-robot scenarios, distributed training is also an open problem. In this sense, this work proposes an action recognition and control approach based on Long Short-Term Memory (LSTM) Deep Neural Networks with two layers in association with three densely connected layers and Federated Learning (FL) embedded in multiple drones. The FL enabled our approach to be trained in a distributed fashion, i.e., access to data without the need for cloud or other repositories, which facilitates the multi-robot system's learning. Furthermore, our multi-robot approach results also prevented occlusion situations, with experiments with real robots achieving an accuracy greater than 96%.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11305-11312"},"PeriodicalIF":4.6,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142645395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：