Pub Date : 2023-09-23DOI: 10.1177/02783649231201196
Matteo Saveriano, Fares J. Abu-Dakka, Aljaz Kramberger, Luka Peternel
Biological systems, including human beings, have the innate ability to perform complex tasks in a versatile and agile manner. Researchers in sensorimotor control have aimed to comprehend and formally define this innate characteristic. The idea, supported by several experimental findings, that biological systems are able to combine and adapt basic units of motion into complex tasks finally leads to the formulation of the motor primitives’ theory. In this respect, Dynamic Movement Primitives (DMPs) represent an elegant mathematical formulation of the motor primitives as stable dynamical systems and are well suited to generate motor commands for artificial systems like robots. In the last decades, DMPs have inspired researchers in different robotic fields including imitation and reinforcement learning, optimal control, physical interaction, and human–robot co-working, resulting in a considerable amount of published papers. The goal of this tutorial survey is two-fold. On one side, we present the existing DMP formulations in rigorous mathematical terms and discuss the advantages and limitations of each approach as well as practical implementation details. In the tutorial vein, we also search for existing implementations of presented approaches and release several others. On the other side, we provide a systematic and comprehensive review of existing literature and categorize state-of-the-art work on DMP. The paper concludes with a discussion on the limitations of DMPs and an outline of possible research directions.
生物系统，包括人类，具有以灵活多样的方式执行复杂任务的天生能力。感觉运动控制的研究人员一直致力于理解和正式定义这种先天特征。生物系统能够将基本的运动单位组合并适应于复杂的任务，这一观点得到了几个实验结果的支持，最终导致了运动原语理论的形成。在这方面，动态运动原语(Dynamic Movement Primitives, dmp)代表了一种优雅的数学公式，将运动原语作为稳定的动力系统，非常适合为机器人等人工系统生成运动命令。在过去的几十年里，dmp启发了不同机器人领域的研究人员，包括模仿和强化学习、最优控制、物理交互和人机协同工作，并发表了大量论文。本教程调查的目的有两个。一方面，我们用严格的数学术语介绍了现有的DMP公式，并讨论了每种方法的优点和局限性以及实际实现细节。在本教程中，我们还搜索了所提供方法的现有实现，并发布了其他一些实现。另一方面，我们对现有文献进行了系统和全面的回顾，并对DMP的最新工作进行了分类。本文最后讨论了DMPs的局限性，并概述了可能的研究方向。
{"title":"Dynamic movement primitives in robotics: A tutorial survey","authors":"Matteo Saveriano, Fares J. Abu-Dakka, Aljaz Kramberger, Luka Peternel","doi":"10.1177/02783649231201196","DOIUrl":"https://doi.org/10.1177/02783649231201196","url":null,"abstract":"Biological systems, including human beings, have the innate ability to perform complex tasks in a versatile and agile manner. Researchers in sensorimotor control have aimed to comprehend and formally define this innate characteristic. The idea, supported by several experimental findings, that biological systems are able to combine and adapt basic units of motion into complex tasks finally leads to the formulation of the motor primitives’ theory. In this respect, Dynamic Movement Primitives (DMPs) represent an elegant mathematical formulation of the motor primitives as stable dynamical systems and are well suited to generate motor commands for artificial systems like robots. In the last decades, DMPs have inspired researchers in different robotic fields including imitation and reinforcement learning, optimal control, physical interaction, and human–robot co-working, resulting in a considerable amount of published papers. The goal of this tutorial survey is two-fold. On one side, we present the existing DMP formulations in rigorous mathematical terms and discuss the advantages and limitations of each approach as well as practical implementation details. In the tutorial vein, we also search for existing implementations of presented approaches and release several others. On the other side, we provide a systematic and comprehensive review of existing literature and categorize state-of-the-art work on DMP. The paper concludes with a discussion on the limitations of DMPs and an outline of possible research directions.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135957920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-19DOI: 10.1177/02783649231198898
Basak Sakcak, Kalle G Timperi, Vadim Weinstein, Steven M LaValle
This paper addresses the lower limits of encoding and processing the information acquired through interactions between an internal system (robot algorithms or software) and an external system (robot body and its environment) in terms of action and observation histories. Both are modeled as transition systems. We want to know the weakest internal system that is sufficient for achieving passive (filtering) and active (planning) tasks. We introduce the notion of an information transition system (ITS) for the internal system which is a transition system over a space of information states that reflect a robot’s or other observer’s perspective based on limited sensing, memory, computation, and actuation. An ITS is viewed as a filter and a policy or plan is viewed as a function that labels the states of this ITS. Regardless of whether internal systems are obtained by learning algorithms, planning algorithms, or human insight, we want to know the limits of feasibility for given robot hardware and tasks. We establish, in a general setting, that minimal information transition systems (ITSs) exist up to reasonable equivalence assumptions, and are unique under some general conditions. We then apply the theory to generate new insights into several problems, including optimal sensor fusion/filtering, solving basic planning tasks, and finding minimal representations for modeling a system given input-output relations.
{"title":"A mathematical characterization of minimally sufficient robot brains","authors":"Basak Sakcak, Kalle G Timperi, Vadim Weinstein, Steven M LaValle","doi":"10.1177/02783649231198898","DOIUrl":"https://doi.org/10.1177/02783649231198898","url":null,"abstract":"This paper addresses the lower limits of encoding and processing the information acquired through interactions between an internal system (robot algorithms or software) and an external system (robot body and its environment) in terms of action and observation histories. Both are modeled as transition systems. We want to know the weakest internal system that is sufficient for achieving passive (filtering) and active (planning) tasks. We introduce the notion of an information transition system (ITS) for the internal system which is a transition system over a space of information states that reflect a robot’s or other observer’s perspective based on limited sensing, memory, computation, and actuation. An ITS is viewed as a filter and a policy or plan is viewed as a function that labels the states of this ITS. Regardless of whether internal systems are obtained by learning algorithms, planning algorithms, or human insight, we want to know the limits of feasibility for given robot hardware and tasks. We establish, in a general setting, that minimal information transition systems (ITSs) exist up to reasonable equivalence assumptions, and are unique under some general conditions. We then apply the theory to generate new insights into several problems, including optimal sensor fusion/filtering, solving basic planning tasks, and finding minimal representations for modeling a system given input-output relations.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135061052","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-11DOI: 10.1177/02783649231196925
Maria Bauza, Antonia Bronars, Alberto Rodriguez
In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored perception model in simulation that estimates a probability distribution over possible object poses given a tactile observation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor, we match it against the pre-computed set using an object-specific embedding learned using contrastive learning. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB (red, green, blue) tactile observations to binary contact shapes. This mapping, which can be reused across object and sensor instances, is the only step trained with real sensor data. This results in a perception model that localizes objects from the first real tactile observation. Importantly, it produces pose distributions and can incorporate additional pose constraints coming from other perception systems, multiple contacts, or priors. We provide quantitative results for 20 objects. Tac2Pose provides high accuracy pose estimations from distinctive tactile observations while regressing meaningful pose distributions to account for those contact shapes that could result from different object poses. We extend and test Tac2Pose in multi-contact scenarios where two tactile sensors are simultaneously in contact with the object, as during a grasp with a parallel jaw gripper. We further show that when the output pose distribution is filtered with a prior on the object pose, Tac2Pose is often able to improve significantly on the prior. This suggests synergistic use of Tac2Pose with additional sensing modalities (e.g., vision) even in cases where the tactile observation from a grasp is not sufficiently discriminative. Given a coarse estimate of an object’s pose, even ambiguous contacts can be used to determine an object’s pose precisely. We also test Tac2Pose on object models reconstructed from a 3D scanner, to evaluate the robustness to uncertainty in the object model. We show that even in the presence of model uncertainty, Tac2Pose is able to achieve fine accuracy comparable to when the object model is the manufacturer’s CAD (computer-aided design) model. Finally, we demonstrate the advantages of Tac2Pose compared with three baseline methods for tactile pose estimation: directly regressing the object pose with a neural network, matching an observed contact to a set of possible contacts using a standard classification neural network, and direct pixel comparison of an observed contact with a set of possible contacts. Website: mcube.mit.edu/research/tac2pose.html
{"title":"Tac2Pose: Tactile object pose estimation from the first touch","authors":"Maria Bauza, Antonia Bronars, Alberto Rodriguez","doi":"10.1177/02783649231196925","DOIUrl":"https://doi.org/10.1177/02783649231196925","url":null,"abstract":"In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored perception model in simulation that estimates a probability distribution over possible object poses given a tactile observation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor, we match it against the pre-computed set using an object-specific embedding learned using contrastive learning. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB (red, green, blue) tactile observations to binary contact shapes. This mapping, which can be reused across object and sensor instances, is the only step trained with real sensor data. This results in a perception model that localizes objects from the first real tactile observation. Importantly, it produces pose distributions and can incorporate additional pose constraints coming from other perception systems, multiple contacts, or priors. We provide quantitative results for 20 objects. Tac2Pose provides high accuracy pose estimations from distinctive tactile observations while regressing meaningful pose distributions to account for those contact shapes that could result from different object poses. We extend and test Tac2Pose in multi-contact scenarios where two tactile sensors are simultaneously in contact with the object, as during a grasp with a parallel jaw gripper. We further show that when the output pose distribution is filtered with a prior on the object pose, Tac2Pose is often able to improve significantly on the prior. This suggests synergistic use of Tac2Pose with additional sensing modalities (e.g., vision) even in cases where the tactile observation from a grasp is not sufficiently discriminative. Given a coarse estimate of an object’s pose, even ambiguous contacts can be used to determine an object’s pose precisely. We also test Tac2Pose on object models reconstructed from a 3D scanner, to evaluate the robustness to uncertainty in the object model. We show that even in the presence of model uncertainty, Tac2Pose is able to achieve fine accuracy comparable to when the object model is the manufacturer’s CAD (computer-aided design) model. Finally, we demonstrate the advantages of Tac2Pose compared with three baseline methods for tactile pose estimation: directly regressing the object pose with a neural network, matching an observed contact to a set of possible contacts using a standard classification neural network, and direct pixel comparison of an observed contact with a set of possible contacts. Website: mcube.mit.edu/research/tac2pose.html","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135981270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-07DOI: 10.1177/02783649231201201
Cheng Chi, Benjamin Burchfiel, Eric Cousineau, Siyuan Feng, Shuran Song
This paper tackles the task of goal-conditioned dynamic manipulation of deformable objects. This task is highly challenging due to its complex dynamics (introduced by object deformation and high-speed action) and strict task requirements (defined by a precise goal specification). To address these challenges, we present Iterative Residual Policy (IRP), a general learning framework applicable to repeatable tasks with complex dynamics. IRP learns an implicit policy via delta dynamics—instead of modeling the entire dynamical system and inferring actions from that model, IRP learns delta dynamics that predict the effects of delta action on the previously observed trajectory. When combined with adaptive action sampling, the system can quickly optimize its actions online to reach a specified goal. We demonstrate the effectiveness of IRP on two tasks: whipping a rope to hit a target point and swinging a cloth to reach a target pose. Despite being trained only in simulation on a fixed robot setup, IRP is able to efficiently generalize to noisy real-world dynamics, new objects with unseen physical properties, and even different robot hardware embodiments, demonstrating its excellent generalization capability relative to alternative approaches.
{"title":"Iterative residual policy: For goal-conditioned dynamic manipulation of deformable objects","authors":"Cheng Chi, Benjamin Burchfiel, Eric Cousineau, Siyuan Feng, Shuran Song","doi":"10.1177/02783649231201201","DOIUrl":"https://doi.org/10.1177/02783649231201201","url":null,"abstract":"This paper tackles the task of goal-conditioned dynamic manipulation of deformable objects. This task is highly challenging due to its complex dynamics (introduced by object deformation and high-speed action) and strict task requirements (defined by a precise goal specification). To address these challenges, we present Iterative Residual Policy (IRP), a general learning framework applicable to repeatable tasks with complex dynamics. IRP learns an implicit policy via delta dynamics—instead of modeling the entire dynamical system and inferring actions from that model, IRP learns delta dynamics that predict the effects of delta action on the previously observed trajectory. When combined with adaptive action sampling, the system can quickly optimize its actions online to reach a specified goal. We demonstrate the effectiveness of IRP on two tasks: whipping a rope to hit a target point and swinging a cloth to reach a target pose. Despite being trained only in simulation on a fixed robot setup, IRP is able to efficiently generalize to noisy real-world dynamics, new objects with unseen physical properties, and even different robot hardware embodiments, demonstrating its excellent generalization capability relative to alternative approaches.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135048632","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-05DOI: 10.1177/02783649231197723
Alberto Jaenal, Francisco-Angel Moreno, J. Gonzalez-Jimenez
Representing the scene appearance by a global image descriptor (BoW, NetVLAD, etc.) is a widely adopted choice to address Visual Place Recognition (VPR). The main reasons are that appearance descriptors can be effectively provided with radiometric and perspective invariances as well as they can deal with large environments because of their compactness. However, addressing metric localization with such descriptors (a problem called Appearance-based Localization or AbL) achieves much poorer accuracy than those techniques exploiting the observation of 3D landmarks, which represent the standard for visual localization. In this paper, we propose ALLOM (Appearance-based Localization with Local Observation Models) which addresses AbL by leveraging the topological location of a robot within a map to achieve accurate metric estimations. This topology-assisted metric localization is implemented with a sequential Monte Carlo Bayesian filter that applies a specific observation model for each different place of the environment, thus taking advantage of the local correlation between the pose and the appearance descriptor within each region. ALLOM also benefits from the topological structure of the map to detect eventual robot loss-of-tracking and to effectively cope with its relocalization by applying VPR. Our proposal demonstrates superior metric localization capability compared to different state-of-the-art AbL methods under a wide range of situations.
{"title":"Sequential Monte Carlo localization in topometric appearance maps","authors":"Alberto Jaenal, Francisco-Angel Moreno, J. Gonzalez-Jimenez","doi":"10.1177/02783649231197723","DOIUrl":"https://doi.org/10.1177/02783649231197723","url":null,"abstract":"Representing the scene appearance by a global image descriptor (BoW, NetVLAD, etc.) is a widely adopted choice to address Visual Place Recognition (VPR). The main reasons are that appearance descriptors can be effectively provided with radiometric and perspective invariances as well as they can deal with large environments because of their compactness. However, addressing metric localization with such descriptors (a problem called Appearance-based Localization or AbL) achieves much poorer accuracy than those techniques exploiting the observation of 3D landmarks, which represent the standard for visual localization. In this paper, we propose ALLOM (Appearance-based Localization with Local Observation Models) which addresses AbL by leveraging the topological location of a robot within a map to achieve accurate metric estimations. This topology-assisted metric localization is implemented with a sequential Monte Carlo Bayesian filter that applies a specific observation model for each different place of the environment, thus taking advantage of the local correlation between the pose and the appearance descriptor within each region. ALLOM also benefits from the topological structure of the map to detect eventual robot loss-of-tracking and to effectively cope with its relocalization by applying VPR. Our proposal demonstrates superior metric localization capability compared to different state-of-the-art AbL methods under a wide range of situations.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":" ","pages":""},"PeriodicalIF":9.2,"publicationDate":"2023-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47074445","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-04DOI: 10.1177/02783649231198900
Bohan Yang, Congying Sui, Fangxun Zhong, Yun-Hui Liu
Deformable object manipulation (DOM) with point clouds has great potential as nonrigid 3D shapes can be measured without detecting and tracking image features. However, robotic shape control of deformable objects with point clouds is challenging due to: the unknown point correspondences and the noisy partial observability of raw point clouds; the modeling difficulties of the relationship between point clouds and robot motions. To tackle these challenges, this paper introduces a novel modal-graph framework for the model-free shape servoing of deformable objects with raw point clouds. Unlike the existing works studying the object’s geometry structure, we propose a modal graph to describe the low-frequency deformation structure of the DOM system, which is robust to the measurement irregularities. The modal graph enables us to directly extract low-dimensional deformation features from raw point clouds without extra processing of registrations, refinements, and occlusion removal. It also preserves the spatial structure of the DOM system to inverse the feature changes into robot motions. Moreover, as the framework is built with unknown physical and geometric object models, we design an adaptive robust controller to deform the object toward the desired shape while tackling the modeling uncertainties, noises, and disturbances online. The system is proved to be input-to-state stable (ISS) using Lyapunov-based methods. Extensive experiments are conducted to validate our method using linear, planar, tubular, and volumetric objects under different settings.
{"title":"Modal-graph 3D shape servoing of deformable objects with raw point clouds","authors":"Bohan Yang, Congying Sui, Fangxun Zhong, Yun-Hui Liu","doi":"10.1177/02783649231198900","DOIUrl":"https://doi.org/10.1177/02783649231198900","url":null,"abstract":"Deformable object manipulation (DOM) with point clouds has great potential as nonrigid 3D shapes can be measured without detecting and tracking image features. However, robotic shape control of deformable objects with point clouds is challenging due to: the unknown point correspondences and the noisy partial observability of raw point clouds; the modeling difficulties of the relationship between point clouds and robot motions. To tackle these challenges, this paper introduces a novel modal-graph framework for the model-free shape servoing of deformable objects with raw point clouds. Unlike the existing works studying the object’s geometry structure, we propose a modal graph to describe the low-frequency deformation structure of the DOM system, which is robust to the measurement irregularities. The modal graph enables us to directly extract low-dimensional deformation features from raw point clouds without extra processing of registrations, refinements, and occlusion removal. It also preserves the spatial structure of the DOM system to inverse the feature changes into robot motions. Moreover, as the framework is built with unknown physical and geometric object models, we design an adaptive robust controller to deform the object toward the desired shape while tackling the modeling uncertainties, noises, and disturbances online. The system is proved to be input-to-state stable (ISS) using Lyapunov-based methods. Extensive experiments are conducted to validate our method using linear, planar, tubular, and volumetric objects under different settings.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135403222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-01DOI: 10.1177/02783649231199537
Cedric Le Gentil, Teresa Vidal-Calleja
Traditionally, the pose and velocity prediction of a system at time t2 given inertial measurements from a 6-DoF IMU depends on the knowledge of the system’s state at time t1. It involves a series of integration and double integration that can be computationally expensive if performed regularly, in particular in the context of inertial-aided optimisation-based state estimation. The concept of preintegration consists of creating pseudo-measurements that are independent of the system’s initial conditions (pose and velocity at t1) in order to predict the system’s state at t2. These pseudo-measurements, so-called preintegrated measurements, were originally computed numerically using the integration rectangle rule. This article presents a novel method to perform continuous preintegration using Gaussian processes (GPs) to model the system’s dynamics focusing on high accuracy. It represents the preintegrated measurement’s derivatives in a continuous latent state that is learnt/optimised according to asynchronous IMU gyroscope and accelerometer measurements. The GP models allow for analytical integration and double integration of the latent state to generate accurate preintegrated measurements called unified Gaussian preintegrated measurements (UGPMs). We show through extensive quantitative experiments that the proposed UGPMs outperform the standard preintegration method by an order of magnitude. Additionally, we demonstrate that the UGPMs can be integrated into off-the-shelf multi-modal estimation frameworks with ease based on lidar-inertial, RGBD-inertial, and visual-inertial real-world experiments.
{"title":"Continuous latent state preintegration for inertial-aided systems","authors":"Cedric Le Gentil, Teresa Vidal-Calleja","doi":"10.1177/02783649231199537","DOIUrl":"https://doi.org/10.1177/02783649231199537","url":null,"abstract":"Traditionally, the pose and velocity prediction of a system at time t2 given inertial measurements from a 6-DoF IMU depends on the knowledge of the system’s state at time t1. It involves a series of integration and double integration that can be computationally expensive if performed regularly, in particular in the context of inertial-aided optimisation-based state estimation. The concept of preintegration consists of creating pseudo-measurements that are independent of the system’s initial conditions (pose and velocity at t1) in order to predict the system’s state at t2. These pseudo-measurements, so-called preintegrated measurements, were originally computed numerically using the integration rectangle rule. This article presents a novel method to perform continuous preintegration using Gaussian processes (GPs) to model the system’s dynamics focusing on high accuracy. It represents the preintegrated measurement’s derivatives in a continuous latent state that is learnt/optimised according to asynchronous IMU gyroscope and accelerometer measurements. The GP models allow for analytical integration and double integration of the latent state to generate accurate preintegrated measurements called unified Gaussian preintegrated measurements (UGPMs). We show through extensive quantitative experiments that the proposed UGPMs outperform the standard preintegration method by an order of magnitude. Additionally, we demonstrate that the UGPMs can be integrated into off-the-shelf multi-modal estimation frameworks with ease based on lidar-inertial, RGBD-inertial, and visual-inertial real-world experiments.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":" ","pages":""},"PeriodicalIF":9.2,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42681604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-19DOI: 10.1177/02783649231193710
Haoshu Fang, Minghao Gou, Chenxi Wang, Cewu Lu
Robust object grasping in cluttered scenes is vital to all robotic prehensile manipulation. In this paper, we present the GraspNet-1Billion benchmark that contains rich real-world captured cluttered scenarios and abundant annotations. This benchmark aims at solving two critical problems for the cluttered scenes parallel-finger grasping: the insufficient real-world training data and the lacking of evaluation benchmark. We first contribute a large-scale grasp pose detection dataset. Two different depth cameras based on structured-light and time-of-flight technologies are adopted. Our dataset contains 97,280 RGB-D images with over one billion grasp poses. In total, 190 cluttered scenes are collected, among which 100 are training set and 90 are for testing. Meanwhile, we build an evaluation system that is general and user-friendly. It directly reports a predicted grasp pose’s quality by analytic computation, which is able to evaluate any kind of grasp representation without exhaustively labeling the ground-truth. We further divide the test set into three difficulties to better evaluate algorithms’ generalization ability. Our dataset, accessing API and evaluation code, are publicly available at www.graspnet.net.
{"title":"Robust grasping across diverse sensor qualities: The GraspNet-1Billion dataset","authors":"Haoshu Fang, Minghao Gou, Chenxi Wang, Cewu Lu","doi":"10.1177/02783649231193710","DOIUrl":"https://doi.org/10.1177/02783649231193710","url":null,"abstract":"Robust object grasping in cluttered scenes is vital to all robotic prehensile manipulation. In this paper, we present the GraspNet-1Billion benchmark that contains rich real-world captured cluttered scenarios and abundant annotations. This benchmark aims at solving two critical problems for the cluttered scenes parallel-finger grasping: the insufficient real-world training data and the lacking of evaluation benchmark. We first contribute a large-scale grasp pose detection dataset. Two different depth cameras based on structured-light and time-of-flight technologies are adopted. Our dataset contains 97,280 RGB-D images with over one billion grasp poses. In total, 190 cluttered scenes are collected, among which 100 are training set and 90 are for testing. Meanwhile, we build an evaluation system that is general and user-friendly. It directly reports a predicted grasp pose’s quality by analytic computation, which is able to evaluate any kind of grasp representation without exhaustively labeling the ground-truth. We further divide the test set into three difficulties to better evaluate algorithms’ generalization ability. Our dataset, accessing API and evaluation code, are publicly available at www.graspnet.net.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":" ","pages":""},"PeriodicalIF":9.2,"publicationDate":"2023-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46583695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-11DOI: 10.1177/02783649231188387
Baxi Chong, Tianyu Wang, Lin Bo, Shengkai Li, Pranav Muthukrishnan, Juntao He, Daniel Irvine, H. Choset, Grigoriy Blekherman, D. Goldman
Contact planning is crucial to the locomotion performance of robots: to properly self-propel forward, it is not only important to determine the sequence of internal shape changes (e.g., body bending and limb shoulder joint oscillation) but also the sequence by which contact is made and broken between the mechanism and its environment. Prior work observed that properly coupling contact patterns and shape changes allows for computationally tractable gait design and efficient gait performance. The state of the art, however, made assumptions, albeit motivated by biological observation, as to how contact and shape changes can be coupled. In this paper, we extend the geometric mechanics (GM) framework to design contact patterns. Specifically, we introduce the concept of “contact space” to the GM framework. By establishing the connection between velocities in shape and position spaces, we can estimate the benefits of each contact pattern change and therefore optimize the sequence of contact patterns. In doing so, we can also analyze how a contact pattern sequence will respond to perturbations. We apply our framework to sidewinding robots and enable (1) effective locomotion direction control and (2) robust locomotion performance as the spatial resolution decreases. We also apply our framework to a hexapod robot with two back-bending joints and show that we can simplify existing hexapod gaits by properly reducing the number of contact state switches (during a gait cycle) without significant loss of locomotion speed. We test our designed gaits with robophysical experiments, and we obtain good agreement between theory and experiments.
{"title":"Optimizing contact patterns for robot locomotion via geometric mechanics","authors":"Baxi Chong, Tianyu Wang, Lin Bo, Shengkai Li, Pranav Muthukrishnan, Juntao He, Daniel Irvine, H. Choset, Grigoriy Blekherman, D. Goldman","doi":"10.1177/02783649231188387","DOIUrl":"https://doi.org/10.1177/02783649231188387","url":null,"abstract":"Contact planning is crucial to the locomotion performance of robots: to properly self-propel forward, it is not only important to determine the sequence of internal shape changes (e.g., body bending and limb shoulder joint oscillation) but also the sequence by which contact is made and broken between the mechanism and its environment. Prior work observed that properly coupling contact patterns and shape changes allows for computationally tractable gait design and efficient gait performance. The state of the art, however, made assumptions, albeit motivated by biological observation, as to how contact and shape changes can be coupled. In this paper, we extend the geometric mechanics (GM) framework to design contact patterns. Specifically, we introduce the concept of “contact space” to the GM framework. By establishing the connection between velocities in shape and position spaces, we can estimate the benefits of each contact pattern change and therefore optimize the sequence of contact patterns. In doing so, we can also analyze how a contact pattern sequence will respond to perturbations. We apply our framework to sidewinding robots and enable (1) effective locomotion direction control and (2) robust locomotion performance as the spatial resolution decreases. We also apply our framework to a hexapod robot with two back-bending joints and show that we can simplify existing hexapod gaits by properly reducing the number of contact state switches (during a gait cycle) without significant loss of locomotion speed. We test our designed gaits with robophysical experiments, and we obtain good agreement between theory and experiments.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"1 1","pages":""},"PeriodicalIF":9.2,"publicationDate":"2023-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41922088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-08-08DOI: 10.1177/02783649231188984
Marcus Hoerger, Hanna Kurniawati, Dirk Kroese, Nan Ye
Solving continuous Partially Observable Markov Decision Processes (POMDPs) is challenging, particularly for high-dimensional continuous action spaces. To alleviate this difficulty, we propose a new sampling-based online POMDP solver, called A daptive D iscretization using V oronoi T rees (ADVT). It uses Monte Carlo Tree Search in combination with an adaptive discretization of the action space as well as optimistic optimization to efficiently sample high-dimensional continuous action spaces and compute the best action to perform. Specifically, we adaptively discretize the action space for each sampled belief using a hierarchical partition called Voronoi tree, which is a Binary Space Partitioning that implicitly maintains the partition of a cell as the Voronoi diagram of two points sampled from the cell. ADVT uses the estimated diameters of the cells to form an upper-confidence bound on the action value function within the cell, guiding the Monte Carlo Tree Search expansion and further discretization of the action space. This enables ADVT to better exploit local information with respect to the action value function, allowing faster identification of the most promising regions in the action space, compared to existing solvers. Voronoi trees keep the cost of partitioning and estimating the diameter of each cell low, even in high-dimensional spaces where many sampled points are required to cover the space well. ADVT additionally handles continuous observation spaces, by adopting an observation progressive widening strategy, along with a weighted particle representation of beliefs. Experimental results indicate that ADVT scales substantially better to high-dimensional continuous action spaces, compared to state-of-the-art methods.
{"title":"Adaptive Discretization using Voronoi trees for continuous pOMDPs","authors":"Marcus Hoerger, Hanna Kurniawati, Dirk Kroese, Nan Ye","doi":"10.1177/02783649231188984","DOIUrl":"https://doi.org/10.1177/02783649231188984","url":null,"abstract":"Solving continuous Partially Observable Markov Decision Processes (POMDPs) is challenging, particularly for high-dimensional continuous action spaces. To alleviate this difficulty, we propose a new sampling-based online POMDP solver, called A daptive D iscretization using V oronoi T rees (ADVT). It uses Monte Carlo Tree Search in combination with an adaptive discretization of the action space as well as optimistic optimization to efficiently sample high-dimensional continuous action spaces and compute the best action to perform. Specifically, we adaptively discretize the action space for each sampled belief using a hierarchical partition called Voronoi tree, which is a Binary Space Partitioning that implicitly maintains the partition of a cell as the Voronoi diagram of two points sampled from the cell. ADVT uses the estimated diameters of the cells to form an upper-confidence bound on the action value function within the cell, guiding the Monte Carlo Tree Search expansion and further discretization of the action space. This enables ADVT to better exploit local information with respect to the action value function, allowing faster identification of the most promising regions in the action space, compared to existing solvers. Voronoi trees keep the cost of partitioning and estimating the diameter of each cell low, even in high-dimensional spaces where many sampled points are required to cover the space well. ADVT additionally handles continuous observation spaces, by adopting an observation progressive widening strategy, along with a weighted particle representation of beliefs. Experimental results indicate that ADVT scales substantially better to high-dimensional continuous action spaces, compared to state-of-the-art methods.","PeriodicalId":54942,"journal":{"name":"International Journal of Robotics Research","volume":"126 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135794789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：