Pub Date : 2024-02-12DOI: 10.1177/02783649241229725
Nathan Hughes, Yun Chang, Siyi Hu, Rajat Talak, Rumaia Abdulhai, Jared Strader, Luca Carlone
3D spatial perception is the problem of building and maintaining an actionable and persistent representation of the environment in real-time using sensor data and prior knowledge. Despite the fast-paced progress in robot perception, most existing methods either build purely geometric maps (as in traditional SLAM) or “flat” metric-semantic maps that do not scale to large environments or large dictionaries of semantic labels. The first part of this paper is concerned with representations: we show that scalable representations for spatial perception need to be hierarchical in nature. Hierarchical representations are efficient to store, and lead to layered graphs with small treewidth, which enable provably efficient inference. We then introduce an example of hierarchical representation for indoor environments, namely a 3D scene graph, and discuss its structure and properties. The second part of the paper focuses on algorithms to incrementally construct a 3D scene graph as the robot explores the environment. Our algorithms combine 3D geometry (e.g., to cluster the free space into a graph of places), topology (to cluster the places into rooms), and geometric deep learning (e.g., to classify the type of rooms the robot is moving across). The third part of the paper focuses on algorithms to maintain and correct 3D scene graphs during long-term operation. We propose hierarchical descriptors for loop closure detection and describe how to correct a scene graph in response to loop closures, by solving a 3D scene graph optimization problem. We conclude the paper by combining the proposed perception algorithms into Hydra, a real-time spatial perception system that builds a 3D scene graph from visual-inertial data in real-time. We showcase Hydra’s performance in photo-realistic simulations and real data collected by a Clearpath Jackal robots and a Unitree A1 robot. We release an open-source implementation of Hydra at https://github.com/MIT-SPARK/Hydra .
{"title":"Foundations of spatial perception for robotics: Hierarchical representations and real-time systems","authors":"Nathan Hughes, Yun Chang, Siyi Hu, Rajat Talak, Rumaia Abdulhai, Jared Strader, Luca Carlone","doi":"10.1177/02783649241229725","DOIUrl":"https://doi.org/10.1177/02783649241229725","url":null,"abstract":"3D spatial perception is the problem of building and maintaining an actionable and persistent representation of the environment in real-time using sensor data and prior knowledge. Despite the fast-paced progress in robot perception, most existing methods either build purely geometric maps (as in traditional SLAM) or “flat” metric-semantic maps that do not scale to large environments or large dictionaries of semantic labels. The first part of this paper is concerned with representations: we show that scalable representations for spatial perception need to be hierarchical in nature. Hierarchical representations are efficient to store, and lead to layered graphs with small treewidth, which enable provably efficient inference. We then introduce an example of hierarchical representation for indoor environments, namely a 3D scene graph, and discuss its structure and properties. The second part of the paper focuses on algorithms to incrementally construct a 3D scene graph as the robot explores the environment. Our algorithms combine 3D geometry (e.g., to cluster the free space into a graph of places), topology (to cluster the places into rooms), and geometric deep learning (e.g., to classify the type of rooms the robot is moving across). The third part of the paper focuses on algorithms to maintain and correct 3D scene graphs during long-term operation. We propose hierarchical descriptors for loop closure detection and describe how to correct a scene graph in response to loop closures, by solving a 3D scene graph optimization problem. We conclude the paper by combining the proposed perception algorithms into Hydra, a real-time spatial perception system that builds a 3D scene graph from visual-inertial data in real-time. We showcase Hydra’s performance in photo-realistic simulations and real data collected by a Clearpath Jackal robots and a Unitree A1 robot. We release an open-source implementation of Hydra at https://github.com/MIT-SPARK/Hydra .","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945628","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-09DOI: 10.1177/02783649241231298
P. Capsi-Morales, Cristina Piazza, G. Grioli, A. Bicchi, M. G. Catalano
Wrist mobility contributes significantly to the execution of upper limb motor tasks. Despite this, current prosthetic wrists are far less advanced than other artificial joints. Typically, prosthetic wrists offer limited degrees of freedom, if any, which forces users to execute compensatory movements during task performance. This addition increases weight and complexity, two unwelcome factors in upper limb prostheses. This article presents the design of a 3-degree-of-freedom friction-lockable prosthetic wrist actuated by a single motor. The design features adaptable behavior when unlocked, promoting a gentle interaction with the environment, and enables users to adjust the hand configuration during pre-grasping phases. The proposed system was tested, combined with a hand prosthesis, and compared to a commercial rotational wrist during the execution of functional movements. Experiments involved nine able-bodied subjects and one prosthesis user. Participants also performed the experiments with their biological wrist (the intact wrist for the prosthesis user) as a control. Results showed that the lockable wrist was used actively 20% more often than the commercial solution without compromising users’ execution time. Interaction tests reveal that compensatory movements are reduced when using the proposed design, resulting in closer resemblance to the control wrist’s performance. The average satisfaction and usability scores were significantly higher for the proposed wrist, indicating its potential acceptance. Finally, the system was validated in a set of activities of daily living performed by the prosthesis user. The study contributes to the development of more intuitive and adaptable prostheses that can improve the quality of life of amputees.
{"title":"A three degrees of freedom switchable impedance myoelectric prosthetic wrist","authors":"P. Capsi-Morales, Cristina Piazza, G. Grioli, A. Bicchi, M. G. Catalano","doi":"10.1177/02783649241231298","DOIUrl":"https://doi.org/10.1177/02783649241231298","url":null,"abstract":"Wrist mobility contributes significantly to the execution of upper limb motor tasks. Despite this, current prosthetic wrists are far less advanced than other artificial joints. Typically, prosthetic wrists offer limited degrees of freedom, if any, which forces users to execute compensatory movements during task performance. This addition increases weight and complexity, two unwelcome factors in upper limb prostheses. This article presents the design of a 3-degree-of-freedom friction-lockable prosthetic wrist actuated by a single motor. The design features adaptable behavior when unlocked, promoting a gentle interaction with the environment, and enables users to adjust the hand configuration during pre-grasping phases. The proposed system was tested, combined with a hand prosthesis, and compared to a commercial rotational wrist during the execution of functional movements. Experiments involved nine able-bodied subjects and one prosthesis user. Participants also performed the experiments with their biological wrist (the intact wrist for the prosthesis user) as a control. Results showed that the lockable wrist was used actively 20% more often than the commercial solution without compromising users’ execution time. Interaction tests reveal that compensatory movements are reduced when using the proposed design, resulting in closer resemblance to the control wrist’s performance. The average satisfaction and usability scores were significantly higher for the proposed wrist, indicating its potential acceptance. Finally, the system was validated in a set of activities of daily living performed by the prosthesis user. The study contributes to the development of more intuitive and adaptable prostheses that can improve the quality of life of amputees.","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"43 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139850005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-09DOI: 10.1177/02783649241231298
P. Capsi-Morales, Cristina Piazza, G. Grioli, A. Bicchi, M. G. Catalano
Wrist mobility contributes significantly to the execution of upper limb motor tasks. Despite this, current prosthetic wrists are far less advanced than other artificial joints. Typically, prosthetic wrists offer limited degrees of freedom, if any, which forces users to execute compensatory movements during task performance. This addition increases weight and complexity, two unwelcome factors in upper limb prostheses. This article presents the design of a 3-degree-of-freedom friction-lockable prosthetic wrist actuated by a single motor. The design features adaptable behavior when unlocked, promoting a gentle interaction with the environment, and enables users to adjust the hand configuration during pre-grasping phases. The proposed system was tested, combined with a hand prosthesis, and compared to a commercial rotational wrist during the execution of functional movements. Experiments involved nine able-bodied subjects and one prosthesis user. Participants also performed the experiments with their biological wrist (the intact wrist for the prosthesis user) as a control. Results showed that the lockable wrist was used actively 20% more often than the commercial solution without compromising users’ execution time. Interaction tests reveal that compensatory movements are reduced when using the proposed design, resulting in closer resemblance to the control wrist’s performance. The average satisfaction and usability scores were significantly higher for the proposed wrist, indicating its potential acceptance. Finally, the system was validated in a set of activities of daily living performed by the prosthesis user. The study contributes to the development of more intuitive and adaptable prostheses that can improve the quality of life of amputees.
{"title":"A three degrees of freedom switchable impedance myoelectric prosthetic wrist","authors":"P. Capsi-Morales, Cristina Piazza, G. Grioli, A. Bicchi, M. G. Catalano","doi":"10.1177/02783649241231298","DOIUrl":"https://doi.org/10.1177/02783649241231298","url":null,"abstract":"Wrist mobility contributes significantly to the execution of upper limb motor tasks. Despite this, current prosthetic wrists are far less advanced than other artificial joints. Typically, prosthetic wrists offer limited degrees of freedom, if any, which forces users to execute compensatory movements during task performance. This addition increases weight and complexity, two unwelcome factors in upper limb prostheses. This article presents the design of a 3-degree-of-freedom friction-lockable prosthetic wrist actuated by a single motor. The design features adaptable behavior when unlocked, promoting a gentle interaction with the environment, and enables users to adjust the hand configuration during pre-grasping phases. The proposed system was tested, combined with a hand prosthesis, and compared to a commercial rotational wrist during the execution of functional movements. Experiments involved nine able-bodied subjects and one prosthesis user. Participants also performed the experiments with their biological wrist (the intact wrist for the prosthesis user) as a control. Results showed that the lockable wrist was used actively 20% more often than the commercial solution without compromising users’ execution time. Interaction tests reveal that compensatory movements are reduced when using the proposed design, resulting in closer resemblance to the control wrist’s performance. The average satisfaction and usability scores were significantly higher for the proposed wrist, indicating its potential acceptance. Finally, the system was validated in a set of activities of daily living performed by the prosthesis user. The study contributes to the development of more intuitive and adaptable prostheses that can improve the quality of life of amputees.","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":" 17","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139790223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-05DOI: 10.1177/02783649241230640
Wenda Zhao, Abhishek Goudar, Xinyuan Qiao, Angela P. Schoellig
Ultra-wideband (UWB) time-difference-of-arrival (TDOA)-based localization has emerged as a promising, low-cost, and scalable indoor localization solution, which is especially suited for multi-robot applications. However, there is a lack of public datasets to study and benchmark UWB TDOA positioning technology in cluttered indoor environments. We fill in this gap by presenting a comprehensive dataset using Decawave’s DWM1000 UWB modules. To characterize the UWB TDOA measurement performance under various line-of-sight (LOS) and non-line-of-sight (NLOS) conditions, we collected signal-to-noise ratio (SNR), power difference values, and raw UWB TDOA measurements during the identification experiments. We also conducted a cumulative total of around 150 min of real-world flight experiments on a customized quadrotor platform to benchmark the UWB TDOA localization performance for mobile robots. The quadrotor was commanded to fly with an average speed of 0.45 m/s in both obstacle-free and cluttered environments using four different UWB anchor constellations. Raw sensor data including UWB TDOA, inertial measurement unit (IMU), optical flow, time-of-flight (ToF) laser altitude, and millimeter-accurate ground truth robot poses were collected during the flights. The dataset and development kit are available at https://utiasdsl.github.io/util-uwb-dataset/ .
{"title":"UTIL: An ultra-wideband time-difference-of-arrival indoor localization dataset","authors":"Wenda Zhao, Abhishek Goudar, Xinyuan Qiao, Angela P. Schoellig","doi":"10.1177/02783649241230640","DOIUrl":"https://doi.org/10.1177/02783649241230640","url":null,"abstract":"Ultra-wideband (UWB) time-difference-of-arrival (TDOA)-based localization has emerged as a promising, low-cost, and scalable indoor localization solution, which is especially suited for multi-robot applications. However, there is a lack of public datasets to study and benchmark UWB TDOA positioning technology in cluttered indoor environments. We fill in this gap by presenting a comprehensive dataset using Decawave’s DWM1000 UWB modules. To characterize the UWB TDOA measurement performance under various line-of-sight (LOS) and non-line-of-sight (NLOS) conditions, we collected signal-to-noise ratio (SNR), power difference values, and raw UWB TDOA measurements during the identification experiments. We also conducted a cumulative total of around 150 min of real-world flight experiments on a customized quadrotor platform to benchmark the UWB TDOA localization performance for mobile robots. The quadrotor was commanded to fly with an average speed of 0.45 m/s in both obstacle-free and cluttered environments using four different UWB anchor constellations. Raw sensor data including UWB TDOA, inertial measurement unit (IMU), optical flow, time-of-flight (ToF) laser altitude, and millimeter-accurate ground truth robot poses were collected during the flights. The dataset and development kit are available at https://utiasdsl.github.io/util-uwb-dataset/ .","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"168 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vision-based estimation of the motion of a moving target is usually formulated as a bearing-only estimation problem where the visual measurement is modeled as a bearing vector. Although the bearing-only approach has been studied for decades, a fundamental limitation of this approach is that it requires extra lateral motion of the observer to enhance the target’s observability. Unfortunately, the extra lateral motion conflicts with the desired motion of the observer in many tasks. It is well-known that, once a target has been detected in an image, a bounding box that surrounds the target can be obtained. Surprisingly, this common visual measurement especially its size information has not been well explored up to now. In this paper, we propose a new bearing-angle approach to estimate the motion of a target by modeling its image bounding box as bearing-angle measurements. Both theoretical analysis and experimental results show that this approach can significantly enhance the observability without relying on additional lateral motion of the observer. The benefit of the bearing-angle approach comes with no additional cost because a bounding box is a standard output of object detection algorithms. The approach simply exploits the information that has not been fully exploited in the past. No additional sensing devices or special detection algorithms are required.
{"title":"A bearing-angle approach for unknown target motion analysis based on visual measurements","authors":"Zian Ning, Yin Zhang, Jianan Li, Zhang Chen, Shiyu Zhao","doi":"10.1177/02783649241229172","DOIUrl":"https://doi.org/10.1177/02783649241229172","url":null,"abstract":"Vision-based estimation of the motion of a moving target is usually formulated as a bearing-only estimation problem where the visual measurement is modeled as a bearing vector. Although the bearing-only approach has been studied for decades, a fundamental limitation of this approach is that it requires extra lateral motion of the observer to enhance the target’s observability. Unfortunately, the extra lateral motion conflicts with the desired motion of the observer in many tasks. It is well-known that, once a target has been detected in an image, a bounding box that surrounds the target can be obtained. Surprisingly, this common visual measurement especially its size information has not been well explored up to now. In this paper, we propose a new bearing-angle approach to estimate the motion of a target by modeling its image bounding box as bearing-angle measurements. Both theoretical analysis and experimental results show that this approach can significantly enhance the observability without relying on additional lateral motion of the observer. The benefit of the bearing-angle approach comes with no additional cost because a bounding box is a standard output of object detection algorithms. The approach simply exploits the information that has not been fully exploited in the past. No additional sensing devices or special detection algorithms are required.","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"41 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945632","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-30DOI: 10.1177/02783649231225811
John Lloyd, Nathan F. Lepora
Tactile servoing is an important technique because it enables robots to manipulate objects with precision and accuracy while adapting to changes in their environments in real-time. One approach for tactile servo control with high-resolution soft tactile sensors is to estimate the contact pose relative to an object surface using a convolutional neural network (CNN) for use as a feedback signal. In this paper, we investigate how the surface pose estimation model can be extended to include shear, and utilise these combined pose-and-shear models to develop a tactile robotic system that can be programmed for diverse non-prehensile manipulation tasks, such as object tracking, surface-following, single-arm object pushing and dual-arm object pushing. In doing this, two technical challenges had to be overcome. Firstly, the use of tactile data that includes shear-induced slippage can lead to error-prone estimates unsuitable for accurate control, and so we modified the CNN into a Gaussian-density neural network and used a discriminative Bayesian filter to improve the predictions with a state dynamics model that utilises the robot kinematics. Secondly, to achieve smooth robot motion in 3D space while interacting with objects, we used SE(3) velocity-based servo control, which required re-deriving the Bayesian filter update equations using Lie group theory, as many standard assumptions do not hold for state variables defined on non-Euclidean manifolds. In future, we believe that pose-and-shear-based tactile servoing will enable many object manipulation tasks and the fully-dexterous utilisation of multi-fingered tactile robot hands.
{"title":"Pose-and-shear-based tactile servoing","authors":"John Lloyd, Nathan F. Lepora","doi":"10.1177/02783649231225811","DOIUrl":"https://doi.org/10.1177/02783649231225811","url":null,"abstract":"Tactile servoing is an important technique because it enables robots to manipulate objects with precision and accuracy while adapting to changes in their environments in real-time. One approach for tactile servo control with high-resolution soft tactile sensors is to estimate the contact pose relative to an object surface using a convolutional neural network (CNN) for use as a feedback signal. In this paper, we investigate how the surface pose estimation model can be extended to include shear, and utilise these combined pose-and-shear models to develop a tactile robotic system that can be programmed for diverse non-prehensile manipulation tasks, such as object tracking, surface-following, single-arm object pushing and dual-arm object pushing. In doing this, two technical challenges had to be overcome. Firstly, the use of tactile data that includes shear-induced slippage can lead to error-prone estimates unsuitable for accurate control, and so we modified the CNN into a Gaussian-density neural network and used a discriminative Bayesian filter to improve the predictions with a state dynamics model that utilises the robot kinematics. Secondly, to achieve smooth robot motion in 3D space while interacting with objects, we used SE(3) velocity-based servo control, which required re-deriving the Bayesian filter update equations using Lie group theory, as many standard assumptions do not hold for state variables defined on non-Euclidean manifolds. In future, we believe that pose-and-shear-based tactile servoing will enable many object manipulation tasks and the fully-dexterous utilisation of multi-fingered tactile robot hands.","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"111 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We present a lazy incremental search algorithm, Lifelong-GLS (L-GLS), along with its bounded suboptimal version, Bounded L-GLS (B-LGLS) that combine the search efficiency of incremental search algorithms with the evaluation efficiency of lazy search algorithms for fast replanning in problem domains where edge evaluations are more expensive than vertex expansions. The proposed algorithms generalize Lifelong Planning A* (LPA*) and its bounded suboptimal version, Truncated LPA* (TLPA*), within the Generalized Lazy Search (GLS) framework, so as to restrict expensive edge evaluations only to the current shortest subpath when the cost-to-come inconsistencies are propagated during repair. We also present dynamic versions of the L-GLS and B-LGLS algorithms, called Generalized D* (GD*) and Bounded Generalized D* (B-GD*), respectively, for efficient replanning with non-stationary queries, designed specifically for navigation of mobile robots. We prove that the proposed algorithms are complete and correct in finding a solution that is guaranteed not to exceed the optimal solution cost by a user-chosen factor. Our numerical and experimental results support the claim that the proposed integration of the incremental and lazy search frameworks can help find solutions faster compared to the regular incremental or regular lazy search algorithms when the underlying graph representation changes often.
{"title":"Lazy incremental search for efficient replanning with bounded suboptimality guarantees","authors":"Jaein Lim, Mahdi Ghanei, R. Connor Lawson, Siddhartha Srinivasa, Panagiotis Tsiotras","doi":"10.1177/02783649241227869","DOIUrl":"https://doi.org/10.1177/02783649241227869","url":null,"abstract":"We present a lazy incremental search algorithm, Lifelong-GLS (L-GLS), along with its bounded suboptimal version, Bounded L-GLS (B-LGLS) that combine the search efficiency of incremental search algorithms with the evaluation efficiency of lazy search algorithms for fast replanning in problem domains where edge evaluations are more expensive than vertex expansions. The proposed algorithms generalize Lifelong Planning A* (LPA*) and its bounded suboptimal version, Truncated LPA* (TLPA*), within the Generalized Lazy Search (GLS) framework, so as to restrict expensive edge evaluations only to the current shortest subpath when the cost-to-come inconsistencies are propagated during repair. We also present dynamic versions of the L-GLS and B-LGLS algorithms, called Generalized D* (GD*) and Bounded Generalized D* (B-GD*), respectively, for efficient replanning with non-stationary queries, designed specifically for navigation of mobile robots. We prove that the proposed algorithms are complete and correct in finding a solution that is guaranteed not to exceed the optimal solution cost by a user-chosen factor. Our numerical and experimental results support the claim that the proposed integration of the incremental and lazy search frameworks can help find solutions faster compared to the regular incremental or regular lazy search algorithms when the underlying graph representation changes often.","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"42 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, advancements in Light Detection and Ranging (LiDAR) technology have made 3D LiDAR sensors more compact, lightweight, and affordable. This progress has spurred interest in integrating LiDAR with sensors such as Inertial Measurement Units (IMUs) and cameras for Simultaneous Localization and Mapping (SLAM) research. Public datasets covering different scenarios, platforms, and viewpoints are crucial for multi-sensor fusion SLAM studies, yet most focus on handheld or vehicle-mounted devices with front or 360-degree views. Data from aerial vehicles with downward-looking views is scarce, existing relevant datasets usually feature low altitudes and are mostly limited to small campus environments. To fill this gap, we introduce the Multi-sensor Aerial Robots SLAM dataset (MARS-LVIG dataset), providing unique aerial downward-looking LiDAR-Visual-Inertial-GNSS data with viewpoints from altitudes between 80 m and 130 m. The dataset not only offers new aspects to test and evaluate existing SLAM algorithms, but also brings new challenges which can facilitate researches and developments of more advanced SLAM algorithms. The MARS-LVIG dataset contains 21 sequences, acquired across diversified large-area environments including an aero-model airfield, an island, a rural town, and a valley. Within these sequences, the UAV has speeds varying from 3 m/s to 12 m/s, a scanning area reaching up to 577,000 m2, and the max path length of 7.148 km in a single flight. This dataset encapsulates data collected by a lightweight, hardware-synchronized sensor package that includes a solid-state 3D LiDAR, a global-shutter RGB camera, IMUs, and a raw message receiver of the Global Navigation Satellite System (GNSS). For algorithm evaluation, this dataset releases ground truth of both localization and mapping, which are acquired by on-board Real-time Kinematic (RTK) and DJI L1 (post-processed by its supporting software DJI Terra), respectively. The dataset can be downloaded from: https://mars.hku.hk/dataset.html .
近年来，光探测与测距（LiDAR）技术的进步使 3D LiDAR 传感器变得更加紧凑、轻便和经济。这一进步激发了人们将激光雷达与惯性测量单元（IMU）和相机等传感器集成起来进行同步定位和绘图（SLAM）研究的兴趣。涵盖不同场景、平台和视角的公共数据集对多传感器融合 SLAM 研究至关重要，但大多数数据集都集中在具有前视或 360 度视角的手持或车载设备上。具有俯视视角的航空飞行器数据非常稀少，现有的相关数据集通常高度较低，且大多局限于小型校园环境。为了填补这一空白，我们引入了多传感器空中机器人SLAM数据集（MARS-LVIG数据集），该数据集提供了独特的空中下视LiDAR-Visual-Inertial-GNSS数据，视点高度在80米至130米之间。MARS-LVIG 数据集包含 21 个序列，在不同的大面积环境中采集，包括一个航空模型机场、一个岛屿、一个乡村小镇和一个山谷。在这些序列中，无人机的速度从 3 米/秒到 12 米/秒不等，扫描面积达 577,000 平方米，单次飞行的最大路径长度为 7.148 千米。该数据集囊括了由轻型硬件同步传感器包收集的数据，其中包括固态 3D 激光雷达、全局快门 RGB 摄像机、IMU 和全球导航卫星系统 (GNSS) 原始信息接收器。为进行算法评估，该数据集发布了定位和测绘的地面实况，分别由机载实时运动学（RTK）和大疆 L1（由其支持软件 DJI Terra 进行后处理）获取。该数据集可从以下网址下载： https://mars.hku.hk/dataset.html 。
{"title":"MARS-LVIG dataset: A multi-sensor aerial robots SLAM dataset for LiDAR-visual-inertial-GNSS fusion","authors":"Haotian Li, Yuying Zou, Nan Chen, Jiarong Lin, Xiyuan Liu, Wei Xu, Chunran Zheng, Rundong Li, Dongjiao He, Fanze Kong, Yixi Cai, Zheng Liu, Shunbo Zhou, Kaiwen Xue, Fu Zhang","doi":"10.1177/02783649241227968","DOIUrl":"https://doi.org/10.1177/02783649241227968","url":null,"abstract":"In recent years, advancements in Light Detection and Ranging (LiDAR) technology have made 3D LiDAR sensors more compact, lightweight, and affordable. This progress has spurred interest in integrating LiDAR with sensors such as Inertial Measurement Units (IMUs) and cameras for Simultaneous Localization and Mapping (SLAM) research. Public datasets covering different scenarios, platforms, and viewpoints are crucial for multi-sensor fusion SLAM studies, yet most focus on handheld or vehicle-mounted devices with front or 360-degree views. Data from aerial vehicles with downward-looking views is scarce, existing relevant datasets usually feature low altitudes and are mostly limited to small campus environments. To fill this gap, we introduce the Multi-sensor Aerial Robots SLAM dataset (MARS-LVIG dataset), providing unique aerial downward-looking LiDAR-Visual-Inertial-GNSS data with viewpoints from altitudes between 80 m and 130 m. The dataset not only offers new aspects to test and evaluate existing SLAM algorithms, but also brings new challenges which can facilitate researches and developments of more advanced SLAM algorithms. The MARS-LVIG dataset contains 21 sequences, acquired across diversified large-area environments including an aero-model airfield, an island, a rural town, and a valley. Within these sequences, the UAV has speeds varying from 3 m/s to 12 m/s, a scanning area reaching up to 577,000 m2, and the max path length of 7.148 km in a single flight. This dataset encapsulates data collected by a lightweight, hardware-synchronized sensor package that includes a solid-state 3D LiDAR, a global-shutter RGB camera, IMUs, and a raw message receiver of the Global Navigation Satellite System (GNSS). For algorithm evaluation, this dataset releases ground truth of both localization and mapping, which are acquired by on-board Real-time Kinematic (RTK) and DJI L1 (post-processed by its supporting software DJI Terra), respectively. The dataset can be downloaded from: https://mars.hku.hk/dataset.html .","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139593211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-25DOI: 10.1177/02783649231223546
Mohammad Hekmatnejad, Bardh Hoxha, Jyotirmoy V. Deshmukh, Yezhou Yang, Georgios Fainekos
Automated vehicles (AV) heavily depend on robust perception systems. Current methods for evaluating vision systems focus mainly on frame-by-frame performance. Such evaluation methods appear to be inadequate in assessing the performance of a perception subsystem when used within an AV. In this paper, we present a logic—referred to as Spatio-Temporal Perception Logic (STPL)—which utilizes both spatial and temporal modalities. STPL enables reasoning over perception data using spatial and temporal operators. One major advantage of STPL is that it facilitates basic sanity checks on the functional performance of the perception system, even without ground truth data in some cases. We identify a fragment of STPL which is efficiently monitorable offline in polynomial time. Finally, we present a range of specifications for AV perception systems to highlight the types of requirements that can be expressed and analyzed through offline monitoring with STPL.
{"title":"Formalizing and evaluating requirements of perception systems for automated vehicles using spatio-temporal perception logic","authors":"Mohammad Hekmatnejad, Bardh Hoxha, Jyotirmoy V. Deshmukh, Yezhou Yang, Georgios Fainekos","doi":"10.1177/02783649231223546","DOIUrl":"https://doi.org/10.1177/02783649231223546","url":null,"abstract":"Automated vehicles (AV) heavily depend on robust perception systems. Current methods for evaluating vision systems focus mainly on frame-by-frame performance. Such evaluation methods appear to be inadequate in assessing the performance of a perception subsystem when used within an AV. In this paper, we present a logic—referred to as Spatio-Temporal Perception Logic (STPL)—which utilizes both spatial and temporal modalities. STPL enables reasoning over perception data using spatial and temporal operators. One major advantage of STPL is that it facilitates basic sanity checks on the functional performance of the perception system, even without ground truth data in some cases. We identify a fragment of STPL which is efficiently monitorable offline in polynomial time. Finally, we present a range of specifications for AV perception systems to highlight the types of requirements that can be expressed and analyzed through offline monitoring with STPL.","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"31 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-24DOI: 10.1177/02783649241228607
Can Tezel, Gokhan Bayar
The modeling structures of rollers, mecanum wheels, and mecanum wheeled mobile robots presented in the literature use single contact force assumption. This assumption may give good results in a simulation environment; however, it is not strong enough to reflect reality. To make an improvement, a new aspect of mecanum wheel model is proposed in this study. The model takes the variable roller contact forces into account and investigates their effects on the performance of motion of a mecanum wheeled mobile robot. It uses all points on each roller’s curved shape so that the slippage phenomena is also taken into consideration which makes it possible to get less position estimation errors in real-time operations. The modeling structure introduced aims to reflect reality both in simulation and real applications. A simulation environment is developed for this study. To make verification, an experimental setup including a four-mecanum-wheeled mobile robot, its mechanical and electrical hardware and software infrastructures, and a ground-truth system is designed and constructed. A Robot Operating System (ROS) based control system is created and integrated into the experimental system. Different types of reference trajectories including straight-line, square-shaped, Z-shaped, and wave(S)-shaped are used to test the performance of the model proposed in both simulation and experimental studies. The tests are also conducted using the model that involves single contact force assumption to make comparisons. The details of the variable contact forces model proposed, simulation environment developed, experimental setup built, simulation and experimental studies, their results, and comparisons are given in this paper.
{"title":"Theoretical and experimental investigation of variable contact forces on the rollers of a mecanum wheeled mobile robot","authors":"Can Tezel, Gokhan Bayar","doi":"10.1177/02783649241228607","DOIUrl":"https://doi.org/10.1177/02783649241228607","url":null,"abstract":"The modeling structures of rollers, mecanum wheels, and mecanum wheeled mobile robots presented in the literature use single contact force assumption. This assumption may give good results in a simulation environment; however, it is not strong enough to reflect reality. To make an improvement, a new aspect of mecanum wheel model is proposed in this study. The model takes the variable roller contact forces into account and investigates their effects on the performance of motion of a mecanum wheeled mobile robot. It uses all points on each roller’s curved shape so that the slippage phenomena is also taken into consideration which makes it possible to get less position estimation errors in real-time operations. The modeling structure introduced aims to reflect reality both in simulation and real applications. A simulation environment is developed for this study. To make verification, an experimental setup including a four-mecanum-wheeled mobile robot, its mechanical and electrical hardware and software infrastructures, and a ground-truth system is designed and constructed. A Robot Operating System (ROS) based control system is created and integrated into the experimental system. Different types of reference trajectories including straight-line, square-shaped, Z-shaped, and wave(S)-shaped are used to test the performance of the model proposed in both simulation and experimental studies. The tests are also conducted using the model that involves single contact force assumption to make comparisons. The details of the variable contact forces model proposed, simulation environment developed, experimental setup built, simulation and experimental studies, their results, and comparisons are given in this paper.","PeriodicalId":501362,"journal":{"name":"The International Journal of Robotics Research","volume":"30 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：