Pub Date : 2023-11-03DOI: 10.1177/02783649231201437
Hongkai Dai, Alexandre Amice, Peter Werner, Annan Zhang, Russ Tedrake
Understanding the geometry of collision-free configuration space (C-free) in the presence of Cartesian-space obstacles is an essential ingredient for collision-free motion planning. While it is possible to check for collisions at a point using standard algorithms, to date no practical method exists for computing C-free regions with rigorous certificates due to the complexity of mapping Cartesian-space obstacles through the kinematics. In this work, we present the first to our knowledge rigorous method for approximately decomposing a rational parametrization of C-free into certified polyhedral regions. Our method, called C-Iris (C-space Iterative Regional Inflation by Semidefinite programming), generates large, convex polytopes in a rational parameterization of the configuration space which are rigorously certified to be collision-free. Such regions have been shown to be useful for both optimization-based and randomized motion planning. Based on convex optimization, our method works in arbitrary dimensions, only makes assumptions about the convexity of the obstacles in the 3D Cartesian space, and is fast enough to scale to realistic problems in manipulation. We demonstrate our algorithm’s ability to fill a non-trivial amount of collision-free C-space in several 2-DOF examples where the C-space can be visualized, as well as the scalability of our algorithm on a 7-DOF KUKA iiwa, a 6-DOF UR3e, and 12-DOF bimanual manipulators. An implementation of our algorithm is open-sourced in Drake . We furthermore provide examples of our algorithm in interactive Python notebooks .
{"title":"Certified polyhedral decompositions of collision-free configuration space","authors":"Hongkai Dai, Alexandre Amice, Peter Werner, Annan Zhang, Russ Tedrake","doi":"10.1177/02783649231201437","DOIUrl":"https://doi.org/10.1177/02783649231201437","url":null,"abstract":"Understanding the geometry of collision-free configuration space (C-free) in the presence of Cartesian-space obstacles is an essential ingredient for collision-free motion planning. While it is possible to check for collisions at a point using standard algorithms, to date no practical method exists for computing C-free regions with rigorous certificates due to the complexity of mapping Cartesian-space obstacles through the kinematics. In this work, we present the first to our knowledge rigorous method for approximately decomposing a rational parametrization of C-free into certified polyhedral regions. Our method, called C-Iris (C-space Iterative Regional Inflation by Semidefinite programming), generates large, convex polytopes in a rational parameterization of the configuration space which are rigorously certified to be collision-free. Such regions have been shown to be useful for both optimization-based and randomized motion planning. Based on convex optimization, our method works in arbitrary dimensions, only makes assumptions about the convexity of the obstacles in the 3D Cartesian space, and is fast enough to scale to realistic problems in manipulation. We demonstrate our algorithm’s ability to fill a non-trivial amount of collision-free C-space in several 2-DOF examples where the C-space can be visualized, as well as the scalability of our algorithm on a 7-DOF KUKA iiwa, a 6-DOF UR3e, and 12-DOF bimanual manipulators. An implementation of our algorithm is open-sourced in Drake . We furthermore provide examples of our algorithm in interactive Python notebooks .","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"5 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135873889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-02DOI: 10.1177/02783649231207974
Woo-Ri Ko, Minsu Jang, Jaeyeon Lee, Jaehong Kim
Social robots facilitate improved human–robot interactions through nonverbal behaviors such as handshakes or hugs. However, the traditional methods, which rely on precoded motions, are predictable and can detract from the perception of robots as interactive agents. To address this issue, we have introduced a Seq2Seq-based neural network model that learns social behaviors from human–human interactions in an end-to-end manner. To mitigate the risk of invalid pose sequences during long-term behavior generation, we incorporated a generative adversarial network (GAN). This proposed method was tested using the humanoid robot, Pepper, in a simulated environment. Given the challenges in assessing the success of social behavior generation, we devised novel metrics to quantify the discrepancy between the generated and ground-truth behaviors. Our analysis reveals the impact of different networks on behavior generation performance and compares the efficacy of learning multiple behaviors versus a single behavior. We anticipate that our method will find application in various sectors, including home service, guide, delivery, educational, and virtual robots, thereby enhancing user interaction and enjoyment.
{"title":"Nonverbal social behavior generation for social robots using end-to-end learning","authors":"Woo-Ri Ko, Minsu Jang, Jaeyeon Lee, Jaehong Kim","doi":"10.1177/02783649231207974","DOIUrl":"https://doi.org/10.1177/02783649231207974","url":null,"abstract":"Social robots facilitate improved human–robot interactions through nonverbal behaviors such as handshakes or hugs. However, the traditional methods, which rely on precoded motions, are predictable and can detract from the perception of robots as interactive agents. To address this issue, we have introduced a Seq2Seq-based neural network model that learns social behaviors from human–human interactions in an end-to-end manner. To mitigate the risk of invalid pose sequences during long-term behavior generation, we incorporated a generative adversarial network (GAN). This proposed method was tested using the humanoid robot, Pepper, in a simulated environment. Given the challenges in assessing the success of social behavior generation, we devised novel metrics to quantify the discrepancy between the generated and ground-truth behaviors. Our analysis reveals the impact of different networks on behavior generation performance and compares the efficacy of learning multiple behaviors versus a single behavior. We anticipate that our method will find application in various sectors, including home service, guide, delivery, educational, and virtual robots, thereby enhancing user interaction and enjoyment.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135875571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-02DOI: 10.1177/02783649231207654
Hyungtae Lim, Beomsoo Kim, Daebeom Kim, Eungchang Mason Lee, Hyun Myung
Global registration is a fundamental task that estimates the relative pose between two viewpoints of 3D point clouds. However, there are two issues that degrade the performance of global registration in LiDAR SLAM: one is the sparsity issue and the other is degeneracy. The sparsity issue is caused by the sparse characteristics of the 3D point cloud measurements in a mechanically spinning LiDAR sensor. The degeneracy issue sometimes occurs because the outlier-rejection methods reject too many correspondences, leaving less than three inliers. These two issues have become more severe as the pose discrepancy between the two viewpoints of 3D point clouds becomes greater. To tackle these problems, we propose a robust global registration framework, called Quatro++. Extending our previous work that solely focused on the global registration itself, we address the robust global registration in terms of the loop closing in LiDAR SLAM. To this end, ground segmentation is exploited to achieve robust global registration. Through the experiments, we demonstrate that our proposed method shows a higher succeiasdsss rate than the state-of-the-art global registration methods, overcoming the sparsity and degeneracy issues. In addition, we show that ground segmentation asdasd asignificantly helps to idfdfsncrease the success rate for the ground vehicles. Finally, we apply our proposed method to the loop clossdasdlksajing modulasdse in LiDAR SLAM and confirm that the quality of the loop constraints is improved, showing more precise mapping results. Therefore, the experimental evidence corroborated the suitabilitiasdasdy of our method as an initial alignment in the loop closing. Our code is available at https://quatro-plusplus.github.io .
{"title":"Quatro++: Robust global registration exploiting ground segmentation for loop closing in LiDAR SLAM","authors":"Hyungtae Lim, Beomsoo Kim, Daebeom Kim, Eungchang Mason Lee, Hyun Myung","doi":"10.1177/02783649231207654","DOIUrl":"https://doi.org/10.1177/02783649231207654","url":null,"abstract":"Global registration is a fundamental task that estimates the relative pose between two viewpoints of 3D point clouds. However, there are two issues that degrade the performance of global registration in LiDAR SLAM: one is the sparsity issue and the other is degeneracy. The sparsity issue is caused by the sparse characteristics of the 3D point cloud measurements in a mechanically spinning LiDAR sensor. The degeneracy issue sometimes occurs because the outlier-rejection methods reject too many correspondences, leaving less than three inliers. These two issues have become more severe as the pose discrepancy between the two viewpoints of 3D point clouds becomes greater. To tackle these problems, we propose a robust global registration framework, called Quatro++. Extending our previous work that solely focused on the global registration itself, we address the robust global registration in terms of the loop closing in LiDAR SLAM. To this end, ground segmentation is exploited to achieve robust global registration. Through the experiments, we demonstrate that our proposed method shows a higher succeiasdsss rate than the state-of-the-art global registration methods, overcoming the sparsity and degeneracy issues. In addition, we show that ground segmentation asdasd asignificantly helps to idfdfsncrease the success rate for the ground vehicles. Finally, we apply our proposed method to the loop clossdasdlksajing modulasdse in LiDAR SLAM and confirm that the quality of the loop constraints is improved, showing more precise mapping results. Therefore, the experimental evidence corroborated the suitabilitiasdasdy of our method as an initial alignment in the loop closing. Our code is available at https://quatro-plusplus.github.io .","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"22 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135973131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We propose a structured prediction approach for robot imitation learning from demonstrations. Among various tools for robot imitation learning, supervised learning has been observed to have a prominent role. Structured prediction is a form of supervised learning that enables learning models to operate on output spaces with complex structures. Through the lens of structured prediction, we show how robots can learn to imitate trajectories belonging to not only Euclidean spaces but also Riemannian manifolds. Exploiting ideas from information theory, we propose a class of loss functions based on the f-divergence to measure the information loss between the demonstrated and reproduced probabilistic trajectories. Different types of f-divergence will result in different policies, which we call imitation modes. Furthermore, our approach enables the incorporation of spatial and temporal trajectory modulation, which is necessary for robots to be adaptive to the change in working conditions. We benchmark our algorithm against state-of-the-art methods in terms of trajectory reproduction and adaptation. The quantitative evaluation shows that our approach outperforms other algorithms regarding both accuracy and efficiency. We also report real-world experimental results on learning manifold trajectories in a polishing task with a KUKA LWR robot arm, illustrating the effectiveness of our algorithmic framework.
{"title":"A structured prediction approach for robot imitation learning","authors":"Anqing Duan, Iason Batzianoulis, Raffaello Camoriano, Lorenzo Rosasco, Daniele Pucci, Aude Billard","doi":"10.1177/02783649231204656","DOIUrl":"https://doi.org/10.1177/02783649231204656","url":null,"abstract":"We propose a structured prediction approach for robot imitation learning from demonstrations. Among various tools for robot imitation learning, supervised learning has been observed to have a prominent role. Structured prediction is a form of supervised learning that enables learning models to operate on output spaces with complex structures. Through the lens of structured prediction, we show how robots can learn to imitate trajectories belonging to not only Euclidean spaces but also Riemannian manifolds. Exploiting ideas from information theory, we propose a class of loss functions based on the f-divergence to measure the information loss between the demonstrated and reproduced probabilistic trajectories. Different types of f-divergence will result in different policies, which we call imitation modes. Furthermore, our approach enables the incorporation of spatial and temporal trajectory modulation, which is necessary for robots to be adaptive to the change in working conditions. We benchmark our algorithm against state-of-the-art methods in terms of trajectory reproduction and adaptation. The quantitative evaluation shows that our approach outperforms other algorithms regarding both accuracy and efficiency. We also report real-world experimental results on learning manifold trajectories in a polishing task with a KUKA LWR robot arm, illustrating the effectiveness of our algorithmic framework.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"97 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135372905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-31DOI: 10.1177/02783649231203342
Graeme Best, Rohit Garg, John Keller, Geoffrey A. Hollinger, Sebastian Scherer
We present a coordinated autonomy pipeline for multi-sensor exploration of confined environments. We simultaneously address four broad challenges that are typically overlooked in prior work: (a) make effective use of both range and vision sensing modalities, (b) perform this exploration across a wide range of environments, (c) be resilient to adverse events, and (d) execute this onboard teams of physical robots. Our solution centers around a behavior tree architecture, which adaptively switches between various behaviors involving coordinated exploration and responding to adverse events. Our exploration strategy exploits the benefits of both visual and range sensors with a generalized frontier-based exploration algorithm and an OpenVDB-based map processing pipeline. Our local planner utilizes a dynamically feasible trajectory library and a GPU-based Euclidean distance transform map to allow fast and safe navigation through both tight doorways and expansive spaces. The autonomy pipeline is evaluated with an extensive set of field experiments, with teams of up to three robots that fly up to 3 m/s and distances exceeding 1 km in confined spaces. We provide a summary of various field experiments and detail resilient behaviors that arose: maneuvering narrow doorways, adapting to unexpected environment changes, and emergency landing. Experiments are also detailed from the DARPA Subterranean Challenge, where our proposed autonomy pipeline contributed to us winning the “Most Sectors Explored” award. We provide an extended discussion of lessons learned, release software as open source, and present a video that illustrates our extensive field trials.
{"title":"Multi-robot, multi-sensor exploration of multifarious environments with full mission aerial autonomy","authors":"Graeme Best, Rohit Garg, John Keller, Geoffrey A. Hollinger, Sebastian Scherer","doi":"10.1177/02783649231203342","DOIUrl":"https://doi.org/10.1177/02783649231203342","url":null,"abstract":"We present a coordinated autonomy pipeline for multi-sensor exploration of confined environments. We simultaneously address four broad challenges that are typically overlooked in prior work: (a) make effective use of both range and vision sensing modalities, (b) perform this exploration across a wide range of environments, (c) be resilient to adverse events, and (d) execute this onboard teams of physical robots. Our solution centers around a behavior tree architecture, which adaptively switches between various behaviors involving coordinated exploration and responding to adverse events. Our exploration strategy exploits the benefits of both visual and range sensors with a generalized frontier-based exploration algorithm and an OpenVDB-based map processing pipeline. Our local planner utilizes a dynamically feasible trajectory library and a GPU-based Euclidean distance transform map to allow fast and safe navigation through both tight doorways and expansive spaces. The autonomy pipeline is evaluated with an extensive set of field experiments, with teams of up to three robots that fly up to 3 m/s and distances exceeding 1 km in confined spaces. We provide a summary of various field experiments and detail resilient behaviors that arose: maneuvering narrow doorways, adapting to unexpected environment changes, and emergency landing. Experiments are also detailed from the DARPA Subterranean Challenge, where our proposed autonomy pipeline contributed to us winning the “Most Sectors Explored” award. We provide an extended discussion of lessons learned, release software as open source, and present a video that illustrates our extensive field trials.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"12 21","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135813224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-30DOI: 10.1177/02783649231210325
Naiyao Wang, Bo Zhang, Haixu Chi, Hua Wang, Seán McLoone, Hongbo Liu
Reliable obstacle avoidance, which is essential for safe autonomous robot interaction with the real world, raises various challenges such as difficulties with obstacle perception and latent factor cognition impacting multi-modal obstacle avoidance. In this paper, we propose a Depth visUal Ego-motion Learning (DUEL) model, consisting of a cognitive generation network, a policy decision network and a potential partition network, to learn autonomous obstacle avoidance from expert policies. The DUEL model takes advantage of binocular vision to perceive scene depth. This serves as the input to the cognitive generation network which generates obstacle avoidance policies by maximizing its causal entropy. The policy decision network then optimizes the generation of the policies referring to expert policies. The generated obstacle avoidance policies are simultaneously transferred to the potential partition network to capture the latent factors contained within expert policies and perform multi-modal obstacle avoidance. These three core networks iteratively optimize the multi-modal policies relying on causal entropy and mutual information theorems, which are proven theoretically. Experimental comparisons with state-of-the-art models on 7 metrics demonstrate the effectiveness of the DUEL model. It achieves the best performance with an average ADE (Average Displacement Error) of 0.29 and average FDE (Final Displacement Error) of 0.55 across five different scenarios. Results show that the DUEL model can maintain an average obstacle avoidance success rate of 97% for both simulated and real world scenarios with multiple obstacles, demonstrating its success at capturing latent factors from expert policies. Our source codes are available at https://github.com/ACoTAI/DUEL .
{"title":"DUEL: Depth visUal Ego-motion Learning for autonomous robot obstacle avoidance","authors":"Naiyao Wang, Bo Zhang, Haixu Chi, Hua Wang, Seán McLoone, Hongbo Liu","doi":"10.1177/02783649231210325","DOIUrl":"https://doi.org/10.1177/02783649231210325","url":null,"abstract":"Reliable obstacle avoidance, which is essential for safe autonomous robot interaction with the real world, raises various challenges such as difficulties with obstacle perception and latent factor cognition impacting multi-modal obstacle avoidance. In this paper, we propose a Depth visUal Ego-motion Learning (DUEL) model, consisting of a cognitive generation network, a policy decision network and a potential partition network, to learn autonomous obstacle avoidance from expert policies. The DUEL model takes advantage of binocular vision to perceive scene depth. This serves as the input to the cognitive generation network which generates obstacle avoidance policies by maximizing its causal entropy. The policy decision network then optimizes the generation of the policies referring to expert policies. The generated obstacle avoidance policies are simultaneously transferred to the potential partition network to capture the latent factors contained within expert policies and perform multi-modal obstacle avoidance. These three core networks iteratively optimize the multi-modal policies relying on causal entropy and mutual information theorems, which are proven theoretically. Experimental comparisons with state-of-the-art models on 7 metrics demonstrate the effectiveness of the DUEL model. It achieves the best performance with an average ADE (Average Displacement Error) of 0.29 and average FDE (Final Displacement Error) of 0.55 across five different scenarios. Results show that the DUEL model can maintain an average obstacle avoidance success rate of 97% for both simulated and real world scenarios with multiple obstacles, demonstrating its success at capturing latent factors from expert policies. Our source codes are available at https://github.com/ACoTAI/DUEL .","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"47 2","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136019443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-30DOI: 10.1177/02783649231210326
Stefan Jorgensen, Marco Pavone
Consider deploying a team of robots in order to visit sites in a risky environment (i.e., where a robot might be lost during a traversal), subject to team-based operational constraints such as limits on team composition, traffic throughputs, and launch constraints. We formalize this problem using a graph to represent the environment, enforcing probabilistic survival constraints for each robot, and using a matroid (which generalizes linear independence to sets) to capture the team-based operational constraints. The resulting “Matroid Team Surviving Orienteers” (MTSO) problem has broad applications for robotics such as informative path planning, resource delivery, and search and rescue. We demonstrate that the objective for the MTSO problem has submodular structure, which leads us to develop two polynomial time algorithms which are guaranteed to find a solution with value within a constant factor of the optimum. The second of our algorithms is an extension of the accelerated continuous greedy algorithm, and can be applied to much broader classes of constraints while maintaining bounds on suboptimality. In addition to in-depth analysis, we demonstrate the efficiency of our approaches by applying them to a scenario where a team of robots must gather information while avoiding dangers in the Coral Triangle and characterize scaling and parameter selection using a synthetic dataset.
考虑部署一个机器人团队，以便访问危险环境中的站点(例如，机器人可能在遍历过程中丢失)，并受到基于团队的操作约束，例如团队组成、流量吞吐量和发射约束的限制。我们使用一个图来表示环境，对每个机器人实施概率生存约束，并使用一个矩阵(将线性独立性推广到集合)来捕获基于团队的操作约束，从而形式化了这个问题。由此产生的“Matroid Team survival Orienteers”(MTSO)问题在机器人领域有着广泛的应用，比如信息路径规划、资源传递、搜索和救援。我们证明了MTSO问题的目标具有子模结构，这使得我们开发了两种多项式时间算法，保证找到值在最优的常数因子内的解。我们的第二种算法是加速连续贪婪算法的扩展，可以应用于更广泛的约束类别，同时保持次优性的界限。除了深入分析之外，我们还通过将我们的方法应用于一个场景来证明我们的方法的效率，在这个场景中，一组机器人必须收集信息，同时避免珊瑚三角中的危险，并使用合成数据集描述缩放和参数选择。
{"title":"The matroid team surviving orienteers problem and its variants: Constrained routing of heterogeneous teams with risky traversal","authors":"Stefan Jorgensen, Marco Pavone","doi":"10.1177/02783649231210326","DOIUrl":"https://doi.org/10.1177/02783649231210326","url":null,"abstract":"Consider deploying a team of robots in order to visit sites in a risky environment (i.e., where a robot might be lost during a traversal), subject to team-based operational constraints such as limits on team composition, traffic throughputs, and launch constraints. We formalize this problem using a graph to represent the environment, enforcing probabilistic survival constraints for each robot, and using a matroid (which generalizes linear independence to sets) to capture the team-based operational constraints. The resulting “Matroid Team Surviving Orienteers” (MTSO) problem has broad applications for robotics such as informative path planning, resource delivery, and search and rescue. We demonstrate that the objective for the MTSO problem has submodular structure, which leads us to develop two polynomial time algorithms which are guaranteed to find a solution with value within a constant factor of the optimum. The second of our algorithms is an extension of the accelerated continuous greedy algorithm, and can be applied to much broader classes of constraints while maintaining bounds on suboptimality. In addition to in-depth analysis, we demonstrate the efficiency of our approaches by applying them to a scenario where a team of robots must gather information while avoiding dangers in the Coral Triangle and characterize scaling and parameter selection using a synthetic dataset.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"330 ","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136019442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-30DOI: 10.1177/02783649231210965
Linda Lastrico, Valerio Belcamino, Alessandro Carfì, Alessia Vignolo, Alessandra Sciutti, Fulvio Mastrogiovanni, Francesco Rea
We propose a dataset to study the influence of object-specific characteristics on human pick-and-place movements and compare the quality of the motion kinematics extracted by various sensors. This dataset is also suitable for promoting a broader discussion on general learning problems in the hand-object interaction domain, such as intention recognition or motion generation with applications in the Robotics field. The dataset consists of the recordings of 15 subjects performing 80 repetitions of a pick-and-place action under various experimental conditions, for a total of 1200 pick-and-places. The data has been collected thanks to a multimodal setup composed of multiple cameras, observing the actions from different perspectives, a motion capture system, and a wrist-worn inertial measurement unit. All the objects manipulated in the experiments are identical in shape, size, and appearance but differ in weight and liquid filling, which influences the carefulness required for their handling.
{"title":"The effects of selected object features on a pick-and-place task: A human multimodal dataset","authors":"Linda Lastrico, Valerio Belcamino, Alessandro Carfì, Alessia Vignolo, Alessandra Sciutti, Fulvio Mastrogiovanni, Francesco Rea","doi":"10.1177/02783649231210965","DOIUrl":"https://doi.org/10.1177/02783649231210965","url":null,"abstract":"We propose a dataset to study the influence of object-specific characteristics on human pick-and-place movements and compare the quality of the motion kinematics extracted by various sensors. This dataset is also suitable for promoting a broader discussion on general learning problems in the hand-object interaction domain, such as intention recognition or motion generation with applications in the Robotics field. The dataset consists of the recordings of 15 subjects performing 80 repetitions of a pick-and-place action under various experimental conditions, for a total of 1200 pick-and-places. The data has been collected thanks to a multimodal setup composed of multiple cameras, observing the actions from different perspectives, a motion capture system, and a wrist-worn inertial measurement unit. All the objects manipulated in the experiments are identical in shape, size, and appearance but differ in weight and liquid filling, which influences the carefulness required for their handling.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"135 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136068402","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-25DOI: 10.1177/02783649231210011
Mercedes Marzoa Tanco, Guillermo Trinidad Barnech, Federico Andrade, Javier Baliosian, Martin LLofriu, JM Di Martino, Gonzalo Tejera
The agricultural industry is being transformed, thanks to recent innovations in computer vision and deep learning. However, the lack of specific datasets collected in natural agricultural environments is, arguably, the main bottleneck for novel discoveries and benchmarking. The present work provides a novel dataset, Magro, and a framework to expand data collection. We present the first version of the Magro Dataset V1.0, consisting of nine ROS bags (and the corresponding raw data) containing data collected in apple and pear crops. Data were gathered, repeating a fixed trajectory on different days under different illumination and weather conditions. To support the evaluation of loop closure algorithms, the trajectories are designed to have loop closures, revisiting some places from different viewpoints. We use a Clearpath’s Jackal robot equipped with stereo cameras pointing to the front and left side, a 3D LIDAR, three inertial measurement units (IMU), and wheel encoders. Additionally, we provide calibrated RTK GPS data that can be used as ground truth. Our dataset is openly available, and it will be updated to have more data and variability. Finally, we tested two existing state-of-the-art algorithms for vision and point cloud-based localization and mapping on our novel dataset to validate the dataset’s usability.
{"title":"MAgro dataset: A dataset for simultaneous localization and mapping in agricultural environments","authors":"Mercedes Marzoa Tanco, Guillermo Trinidad Barnech, Federico Andrade, Javier Baliosian, Martin LLofriu, JM Di Martino, Gonzalo Tejera","doi":"10.1177/02783649231210011","DOIUrl":"https://doi.org/10.1177/02783649231210011","url":null,"abstract":"The agricultural industry is being transformed, thanks to recent innovations in computer vision and deep learning. However, the lack of specific datasets collected in natural agricultural environments is, arguably, the main bottleneck for novel discoveries and benchmarking. The present work provides a novel dataset, Magro, and a framework to expand data collection. We present the first version of the Magro Dataset V1.0, consisting of nine ROS bags (and the corresponding raw data) containing data collected in apple and pear crops. Data were gathered, repeating a fixed trajectory on different days under different illumination and weather conditions. To support the evaluation of loop closure algorithms, the trajectories are designed to have loop closures, revisiting some places from different viewpoints. We use a Clearpath’s Jackal robot equipped with stereo cameras pointing to the front and left side, a 3D LIDAR, three inertial measurement units (IMU), and wheel encoders. Additionally, we provide calibrated RTK GPS data that can be used as ground truth. Our dataset is openly available, and it will be updated to have more data and variability. Finally, we tested two existing state-of-the-art algorithms for vision and point cloud-based localization and mapping on our novel dataset to validate the dataset’s usability.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134973524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robots usually establish contacts at rigid surfaces with near-zero relative velocities. Otherwise, impact-induced energy propagates in the robot’s linkage and may cause irreversible damage to the hardware. Moreover, abrupt changes in task-space contact velocity and peak impact forces also result in abrupt changes in robot joint velocities and torques; which can compromise controllers’ stability, especially for those based on smooth models. In reality, several tasks would require establishing contact with moderately high velocity. We propose to enhance task-space multi-objective controllers formulated as a quadratic program to be resilient to frictional impacts in three dimensions. We devise new constraints and reformulate the usual ones to be robust to the abrupt joint state changes mentioned earlier. The impact event becomes a controlled process once the optimal control search space is aware of: (1) the hardware-affordable impact bounds and (2) analytically computed feasible set (polyhedra) that constrain post-impact critical states. Prior to and nearby the targeted contact spot, we assume, at each control cycle, that the impact will occur at the next iteration. This somewhat one-step preview makes our controller robust to impact time and location. To assess our approach, we experimented its resilience to moderate impacts with the Panda manipulator and achieved swift grabbing tasks with the HRP-4 humanoid robot.
{"title":"Impact-aware task-space quadratic-programming control","authors":"Yuquan Wang, Niels Dehio, Arnaud Tanguy, Abderrahmane Kheddar","doi":"10.1177/02783649231198558","DOIUrl":"https://doi.org/10.1177/02783649231198558","url":null,"abstract":"Robots usually establish contacts at rigid surfaces with near-zero relative velocities. Otherwise, impact-induced energy propagates in the robot’s linkage and may cause irreversible damage to the hardware. Moreover, abrupt changes in task-space contact velocity and peak impact forces also result in abrupt changes in robot joint velocities and torques; which can compromise controllers’ stability, especially for those based on smooth models. In reality, several tasks would require establishing contact with moderately high velocity. We propose to enhance task-space multi-objective controllers formulated as a quadratic program to be resilient to frictional impacts in three dimensions. We devise new constraints and reformulate the usual ones to be robust to the abrupt joint state changes mentioned earlier. The impact event becomes a controlled process once the optimal control search space is aware of: (1) the hardware-affordable impact bounds and (2) analytically computed feasible set (polyhedra) that constrain post-impact critical states. Prior to and nearby the targeted contact spot, we assume, at each control cycle, that the impact will occur at the next iteration. This somewhat one-step preview makes our controller robust to impact time and location. To assess our approach, we experimented its resilience to moderate impacts with the Panda manipulator and achieved swift grabbing tasks with the HRP-4 humanoid robot.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136079275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：