Frontiers in Robotics and AI最新文献

Introduction: Animal-involved scenarios pose significant challenges for autonomous driving systems due to their rarity, unpredictability, and safety-critical nature. Despite their importance, existing vision-language datasets for autonomous driving largely overlook these long-tail situations.

Methods: To address this gap, we introduce AniDriveQA, a novel visual question answering (VQA) dataset specifically designed to evaluate vision-language models (VLMs) in driving scenarios involving animals. The dataset is constructed through a scalable pipeline that collects diverse animal-related traffic scenes from internet videos, filters and annotates them using object detection and scene classification models, and generates multi-task VQA labels with a large vision-language model. AniDriveQA includes three key task types: scene description, animal description, and driving suggestion.

Results: For evaluation, a hybrid scheme was employed that combined classification accuracy for structured tasks with LLM-based scoring for open-ended responses. Extensive experiments on various open-source VLMs revealed large performance disparities across models and task types.

Discussion: The experimental results demonstrate that AniDriveQA effectively exposes the limitations of current VLMs in rare yet safety-critical autonomous driving scenarios. The dataset provides a valuable diagnostic benchmark for advancing reasoning, perception, and decision-making capabilities in future vision-language models.

Limited battery capacity poses a challenge for autonomous robots. We believe that instead of relying solely on electric motors and batteries, basically Conventional Autonomous Robots (CAR), one way to address this challenge may be to develop Biohybrid Autonomous Robots (BAR), based on current achievements of the field of biohybrid robotics. The BAR approach is based on the facts that fat store high amount of energy, that biological muscles generate decent force per unit of cross-sectional area and that biological muscles have capability for regeneration and adaptation compared to electric motors. To reach conclusions about the feasibility of BAR, this study uses data from the fields of muscle energetics, robotics, engineering, physiology, biomechanics and others to perform analysis and interdisciplinary calculations. Our calculations show that the BAR approach is up to 5.1 times more efficient in terms of the mass of energy substrate to useful energy transported than the Conventional Autonomous Robots (CAR) with mass-produced batteries in an ideal scenario. The study also presents the model for determining the point of the rational use of the BAR, taking into the account basal metabolism of living systems. The results of this study provide a preliminary basis for further research of the BAR, putting it into the context of the other possible solutions for energy autonomy problem: Generator-Powered Autonomous Robots (GPAR) and Fuell-Cell Autonomous Robots (FCAR).

This paper presents a robust vision-based motion planning framework for dual-arm manipulators that introduces a novel three-way force equilibrium with velocity-dependent stabilization. The framework combines an improved Artificial Potential Field (iAPF) for linear velocity control with a Proportional-Derivative (PD) controller for angular velocity, creating a hybrid twist command for precise manipulation. A priority-based state machine enables human-like asymmetric dual-arm manipulation. Lyapunov stability analysis proves the asymptotic convergence to desired configurations. The method introduces a computationally efficient continuous distance calculation between links based on line segment configurations, enabling real-time collision monitoring. Experimental validation integrates a real-time vision system using YOLOv8 OBB that achieves 20 frames per second with 0.99/0.97 detection accuracy for bolts/nuts. Comparative tests against traditional APF methods demonstrate that the proposed approach provides stabilized motion planning with smoother trajectories and optimized spatial separation, effectively preventing inter-arm collisions during industrial component sorting.

Musculoskeletal disorders, particularly low back pain, are some of the most common occupational health issues globally, causing significant personal suffering and economic burdens. Workers performing repetitive manual material handling tasks are especially at risk. FleXo, a lightweight (1.35 kg), flexible, ergonomic, and passive back-support exoskeleton is intended to reduce lower back strain during lifting tasks while allowing full freedom of movement for activities like walking, sitting, or side bending. FleXo's design results from an advanced multi-objective design optimization approach that balances functionality and user comfort. In this work, validated through user feedback in a series of relevant repetitive tasks, it is demonstrated that FleXo can reduce the perceived physical effort during lifting tasks, enhance user satisfaction, improve employee wellbeing, promote workplace safety, decrease injuries, and lower the costs (both to society and companies) associated with lower back pain and injury.

Industrial terminal assembly tasks are often repetitive and involve handling components with tight tolerances that are susceptible to damage. Learning an effective terminal assembly policy in real-world is challenging, as collisions between parts and the environment can lead to slippage or part breakage. In this paper, we propose a safe reinforcement learning approach to develop a visuo-tactile assembly policy that is robust to variations in grasp poses. Our method minimizes collisions between the terminal head and terminal base by decomposing the assembly task into three distinct phases. In the first grasp phase,a vision-guided model is trained to pick the terminal head from an initial bin. In the second align phase, a tactile-based grasp pose estimation model is employed to align the terminal head with the terminal base. In the final assembly phase, a visuo-tactile policy is learned to precisely insert the terminal head into the terminal base. To ensure safe training, the robot leverages human demonstrations and interventions. Experimental results on PLC terminal assembly demonstrate that the proposed method achieves 100% successful insertions across 100 different initial end-effector and grasp poses, while imitation learning and online-RL policy yield only 9% and 0%.

The integration of Vision-Language Models (VLMs) into autonomous systems is of growing importance for improving Human-Robot Interaction (HRI), enabling robots to operate within complex and unstructured environments and collaborate with non-expert users. For mobile robots to be effectively deployed in dynamic settings such as domestic or industrial areas, the ability to interpret and execute natural language commands is crucial. However, while VLMs offer powerful zero-shot, open-vocabulary recognition capabilities, their high computational cost presents a significant challenge for real-time performance on resource-constrained edge devices. This study provides a systematic analysis of the trade-offs involved in optimizing a real-time robotic perception pipeline on the NVIDIA Jetson AGX Orin 64GB platform. We investigate the relationship between accuracy and latency by evaluating combinations of two open-vocabulary detection models and two prompt-based segmentation models. Each pipeline is optimized using various precision levels (FP32, FP16, and Best) via NVIDIA TensorRT. We present a quantitative comparison of the mean Intersection over Union (mIoU) and latency for each configuration, offering practical insights and benchmarks for researchers and developers deploying these advanced models on embedded systems.

Transcatheter Aortic Valve Implantation (TAVI) is a minimally invasive procedure in which a transcatheter heart valve (THV) is implanted within the patient's diseased native aortic valve. The procedure is increasingly chosen even for intermediate-risk and younger patients, as it combines complication rates comparable to open-heart surgery with the advantage of being far less invasive. Despite its benefits, challenges remain in achieving accurate and repeatable valve positioning, with inaccuracies potentially leading to complications such as THV migration, coronary obstruction, and conduction disturbances (CD). The latter often requires a permanent pacemaker implantation as a costly and life-changing mitigation. Robotic assistance may offer solutions, enhancing precision, standardization, and reducing radiation exposure for clinicians. This article introduces a novel solution for robot-assisted TAVI, addressing the growing need for skilled clinicians and improving procedural outcomes. We present an in-vivo animal demonstration of robotic-assisted TAVI, showing feasibility of tele-operative instrument control and THV deployment. This, done at safer distances from radiation sources by a single operator. Furthermore, THV positioning and deployment under supervised autonomy is demonstrated on phantom, and shown to be feasible using both camera- and fluoroscopy-based imaging feedback and AI. Finally, an initial operator study probes performance and potential added value of various technology augmentations with respect to a manual expert operator, indicating equivalent to superior accuracy and repeatability using robotic assistance. It is concluded that robot-assisted TAVI is technically feasible in-vivo, and presents a strong case for a clinically meaningful application of level-3 autonomy. These findings support the potential of surgical robotic technology to enhance TAVI accuracy and repeatability, ultimately improving patient outcomes and expanding procedural accessibility.

Mobile robots require knowledge of the environment, especially of humans located in its vicinity. While the most common approaches for detecting humans involve computer vision, an often overlooked hardware feature of robots for people detection are their 2D range finders. These were originally intended for obstacle avoidance and mapping/SLAM tasks. In most robots, they are conveniently located at a height approximately between the ankle and the knee, so they can be used for detecting people too, and with a larger field of view and depth resolution compared to cameras. In this paper, we present a new dataset for people detection using knee-high 2D range finders called FROG. This dataset has greater laser resolution, scanning frequency, and more complete annotation data compared to existing datasets such as DROW (Beyer et al., 2018). Particularly, the FROG dataset contains annotations for 100% of its laser scans (unlike DROW which only annotates 5%), 17x more annotated scans, 100x more people annotations, and over twice the distance traveled by the robot. We propose a benchmark based on the FROG dataset, and analyze a collection of state-of-the-art people detectors based on 2D range finder data. We also propose and evaluate a new end-to-end deep learning approach for people detection. Our solution works with the raw sensor data directly (not needing hand-crafted input data features), thus avoiding CPU preprocessing and releasing the developer of understanding specific domain heuristics. Experimental results show how the proposed people detector attains results comparable to the state of the art, while an optimized implementation for ROS can operate at more than 500 Hz.

Due to their utility in replacing workers in tasks unsuitable for humans, unmanned underwater vehicles (UUVs) have become increasingly common tools in the fish farming industry. However, earlier studies and anecdotal evidence from farmers imply that farmed fish tend to move away from and avoid intrusive objects such as vehicles that are deployed and operated inside net pens. Such responses could imply a discomfort associated with the intrusive objects, which, in turn, can lead to stress and impaired welfare in the fish. To prevent this, vehicles and their control systems should be designed to automatically adjust operations when they perceive that they are repelling the fish. A necessary first step in this direction is to develop on-vehicle observation systems for assessing object/vehicle-fish distances in real-time settings that can provide inputs to the control algorithms. Due to their small size and low weight, modern cameras are ideal for this purpose. Moreover, the ongoing rapid developments within deep learning methods are enabling the use of increasingly sophisticated methods for analyzing footage from cameras. To explore this potential, we developed three new pipelines for the automated assessment of fish-camera distances in video and images. These methods were complemented using a recently published method, yielding four pipelines in total, namely, SegmentDepth, BBoxDepth, and SuperGlue that were based on stereo-vision and DepthAnything that was monocular. The overall performance was investigated using field data by comparing the fish-object distances obtained from the methods with those measured using a sonar. The four methods were then benchmarked by comparing the number of objects detected and the quality and overall accuracy of the stereo matches (only stereo-based methods). SegmentDepth, DepthAnything, and SuperGlue performed well in comparison with the sonar data, yielding mean absolute errors (MAE) of 0.205 m (95% CI: 0.050-0.360), 0.412 m (95% CI: 0.148-0.676), and 0.187 m (95% CI: 0.073-0.300), respectively, and were integrated into the Robot Operating System (ROS2) framework to enable real-time application in fish behavior identification and the control of robotic vehicles such as UUVs.