Autonomous Robots最新文献

Recent advances in mapping techniques have enabled the creation of highly accurate dense 3D maps during robotic missions, such as point clouds, meshes, or NeRF-based representations. These developments present new opportunities for reusing these maps for localization. However, there remains a lack of a unified approach that can operate seamlessly across different map representations. This paper presents and evaluates a global visual localization system capable of localizing a single camera image across various 3D map representations built using both visual and lidar sensing. Our system generates a database by synthesizing novel views of the scene, creating RGB and depth image pairs. Leveraging the precise 3D geometric map, our method automatically defines rendering poses, reducing the number of database images while preserving retrieval performance. To bridge the domain gap between real query camera images and synthetic database images, our approach utilizes learning-based descriptors and feature detectors. We evaluate the system’s performance through extensive real-world experiments conducted in both indoor and outdoor settings, assessing the effectiveness of each map representation and demonstrating its advantages over traditional structure-from-motion (SfM) localization approaches. The results show that all three map representations can achieve consistent localization success rates of 55% and higher across various environments. NeRF synthesized images show superior performance, localizing query images at an average success rate of 72%. Furthermore, we demonstrate an advantage over SfM-based approaches that our synthesized database enables localization in the reverse travel direction which is unseen during the mapping process. Our system, operating in real-time on a mobile laptop equipped with a GPU, achieves a processing rate of 1 Hz.

From refrigerators to kitchen drawers, humans interact with articulated objects effortlessly every day while completing household chores. For automating these tasks, service robots must be capable of manipulating a variety of common articulated objects. Recent deep learning methods have been shown to predict valuable priors on the affordance of articulated objects from vision. In contrast, many other works estimate object articulations by observing the articulation motion, but this requires the robot to already be capable of manipulating the object. In this article, we propose a novel approach combining these methods by using a factor graph for online estimation of articulation, which fuses learned visual priors and proprioceptive sensing during interaction into an analytical model of articulation based on Screw Theory. With our method, a robotic system makes an initial prediction of articulation from vision before touching the object, and then quickly updates the estimate from kinematic and force sensing during manipulation. We evaluate our method extensively in both simulations and real-world robotic manipulation experiments. We demonstrate several closed-loop estimation and manipulation experiments in which the robot was capable of opening previously unseen drawers. In real hardware experiments, the robot achieved a 75% success rate for autonomous opening of unknown articulated objects.

Recent advancements in 3D Gaussian Splatting (3DGS) have demonstrated promising results in dense visual SLAM. However, existing Gaussian-based SLAM systems struggle to generate Gaussian maps in sparse views, constrained by memory limitations and real-time performance requirements. This typically leads to two key problems: First, the rendering quality and localization accuracy of Gaussian primitives are highly sensitive to unstable initial point clouds, with errors accumulating during the densification process. Second, when observing Gaussians from multiple views, the attributes of the Gaussian primitives can become overly influenced by the final training view, leading to a forgetting problem where earlier views are not adequately retained. To address these issues, we propose a robust RGB-D SLAM framework that incorporates enhanced spatial distribution and view-consistency optimization. Specifically, we introduce: (1) a texture-density-driven sampling and graph-based structure densification method that uses geometry information to improve Gaussian primitives accuracy, and (2) an optimization strategy based on Gaussian attributes fusion of view-consistency, which explores Gaussian attributes in different views, enabling Gaussian primitives to adapt to various scene perspectives. Our evaluation on Replica and TUM RGB-D datasets demonstrates superior performance, offering new insights for robust 3D reconstruction in resource-constrained systems.

In this article, we present a distributed algorithm for multi-robot persistent monitoring and target detection. In particular, we propose a novel solution that effectively integrates the Time-inverted Kuramoto model, three-dimensional Lissajous curves, and Model Predictive Control. We focus on the implementation of this algorithm on aerial robots, addressing the practical challenges involved in deploying our approach under real-world conditions. Our method ensures an effective and robust solution that maintains operational efficiency even in the presence of what we define as type I and type II failures. Type I failures refer to short-time disruptions, such as tracking errors and communication delays, while type II failures account for long-time disruptions, including malicious attacks, severe communication failures, and battery depletion. Our approach guarantees persistent monitoring and target detection despite these challenges. Furthermore, we validate our method with extensive field experiments involving up to eleven aerial robots, demonstrating the effectiveness, resilience, and scalability of our solution. Video—https://mrs.fel.cvut.cz/persistent-monitoring-auro2025Code—https://github.com/ctu-mrs/distributed-area-monitoring

This paper considers a robot moving in a 3D environment that is tasked with estimating a quasi-stationary environmental field (e.g., temperature, concentration of a chemical pollutant, or distribution of light radiation density) in the presence of localization uncertainties, as is typical in underwater or other GPS-denied environments. Gaussian process regression has been widely adopted to model environmental fields. However, a drawback of Gaussian process regression is its difficulty in accounting for data with uncertain input. This work proposes a novel multi-fidelity Gaussian process-based regression approach to address the challenge by splitting the data collected by the robot into different datasets corresponding to the amount of input (localization) uncertainty. Furthermore, a sampling-based trajectory planning algorithm is proposed for adaptive robot exploration that optimizes a field-reconstruction objective function while accommodating resource constraints. The proposed approach is experimentally evaluated using a miniature gliding robotic fish that measures light intensity in a large indoor tank. The adaptive exploration algorithm is tested using both a multi-fidelity Gaussian process model and a baseline single-fidelity model. Two objective functions, based on the information gain and an ergodic metric, respectively, are adopted in the evaluation. The experiments show that, for both objective functions, using multi-fidelity Gaussian process reduces the weighted mean squared error between the model prediction and the ground-truth field compared to using the baseline single-fidelity model that ignores localization uncertainty. Accompanying code available at Coleman (Adaptive exploration under localization uncertainty using multi-fidelity Gaussian processes, 2025, https://github.com/colem404/Adaptive-Exploration-Under-Localization-Uncertainty-Using-Multi-fidelity-Gaussian-Processes/tree/main).

Robots should be able to learn complex behaviors from human demonstrations. In practice, these human-provided datasets are inevitably imbalanced: i.e., the human demonstrates some subtasks more frequently than others. State-of-the-art methods default to treating each element of the human’s dataset as equally important. So if—for instance—the majority of the human’s data focuses on reaching a goal, and only a few state-action pairs move to avoid an obstacle, the learning algorithm will place greater emphasis on goal reaching. More generally, misalignment between the relative amounts of data and the importance of that data causes fundamental problems for imitation learning approaches. In this paper we analyze and develop learning methods that automatically account for mixed datasets. We formally prove that imbalanced data leads to imbalanced policies when each state-action pair is weighted equally; these policies emulate the most represented behaviors, and not the human’s complex, multi-task demonstrations. We next explore algorithms that rebalance offline datasets (i.e., reweight the importance of different state-action pairs) without human oversight. Reweighting the dataset can enhance the overall policy performance. However, there is no free lunch: each method for autonomously rebalancing brings its own pros and cons. We formulate these advantages and disadvantages, helping other researchers identify when each type of approach is most appropriate. We conclude by introducing a novel meta-gradient rebalancing algorithm that addresses the primary limitations behind existing approaches. Our experiments show that dataset rebalancing leads to better downstream learning, improving the performance of general imitation learning algorithms without requiring additional data collection. See our project website: https://collab.me.vt.edu/data_curation/.

We present a novel approach for enhancing robotic exploration by using generative occupancy mapping. We implement SceneSense, a diffusion model designed and trained for predicting 3D occupancy maps given partial observations. Our proposed approach probabilistically fuses these predictions into a running occupancy map in real-time, resulting in significant improvements in map quality and traversability. We deploy SceneSense on a quadruped robot and validate its performance with real-world experiments to demonstrate the effectiveness of the model. In these experiments we show that occupancy maps enhanced with SceneSense predictions better estimate the distribution of our fully observed ground truth data (24.44% FID improvement around the robot and 75.59% improvement at range). We additionally show that integrating SceneSense enhanced maps into our robotic exploration stack as a “drop-in” map improvement, utilizing an existing off-the-shelf planner, results in improvements in robustness and traversability time. Finally, we show results of full exploration evaluations with our proposed system in two dissimilar environments and find that locally enhanced maps provide more consistent exploration results than maps constructed only from direct sensor measurements.

In this paper, we address the problem of coordinating multiple robots to explore large-scale underground areas covered with low-bandwidth communication. Based on the evaluation of existing coordination methods, we found that well-performing methods rely on exchanging significant amounts of data, including maps. Such extensive data exchange becomes infeasible using only low-bandwidth communication, which is suitable for underground environments. Therefore, we propose a coordination method that satisfies low-bandwidth constraints by sharing only the robot’s positions. The proposed method employs a fully decentralized principle called Cross-rank that computes how to distribute robots uniformly at intersections and subsequently orders exploration waypoints based on the traveling salesman problem formulation. The proposed principle has been evaluated based on exploration time, traveled distance, and coverage in five large-scale simulated subterranean environments and a real-world deployment with three quadruped robots. The results suggest that the proposed approach provides a suitable tradeoff between the required communication bandwidth and the time needed for exploration.

We present a novel method that, given a grid map of a partially explored indoor environment, estimates the amount of the explored area in the map and whether it is worth continuing to explore the uncovered part of the environment. Our method is based on the idea that modern deep learning models can successfully solve this task by leveraging visual clues in the map. Thus, we train a deep convolutional neural network on images depicting grid maps from partially explored environments, with annotations derived from the knowledge of the entire map, which is not available when the network is used for inference. We show that our network can be used to define a stopping criterion to successfully terminate the exploration process when this is expected to no longer add relevant details about the environment to the map, saving more than 35% of the total exploration time compared to covering the whole environment area.

An important class of robotic applications involves multiple agents cooperating to provide state observations to plan joint actions. We study planning under uncertainty when more than one participant must proactively plan perception and/or communication acts, and decide whether the cost to obtain a state estimate is justified by the benefits accrued by the information thus obtained. The approach we introduce is suitable for settings where observations are of high quality and they—either alone or along with communication—recover the system’s joint state, but the costs incurred mean this happens only infrequently. We formulate the problem as a type of Markov decision process (mdp) to be solved over macro-actions, sidestepping the construction of the full joint belief space, a well-known source of intractability. We then give a suitable Bellman-like recurrence that immediately suggests a means of solution. In their most general form, policies for these problems simultaneously describe (1) low-level actions to be taken, (2) stages when system-wide state is recovered, and (3) commitments to future rescheduling acts. The formulation expresses multi-agency in a variety of distinct practical forms, including: one party assisting by providing observations of, or reference points for, another; several agents communicating sensor information to fuse data and recover joint state; multiple agents coordinating activities to arrive at states that make joint state simultaneously observable to all individuals. Though solved in centralized form over joint states, the mdp is structured to allow decentralized execution, under some assumptions of synchrony in activities. After providing small-scale simulation studies of the general formulation, we discuss a specific scenario motivated by underwater gliders. We report on a physical robot implementation mocked-up to respect these same constraints, showing that joint plans are found and executed effectively by individual robots after appropriate projection. On the basis of our experience with hardware, we examine enhancements to the model that address nonidealities we have identified in practice, including the assumptions regarding synchrony.