Frontiers in Robotics and AI最新文献

Industrial terminal assembly tasks are often repetitive and involve handling components with tight tolerances that are susceptible to damage. Learning an effective terminal assembly policy in real-world is challenging, as collisions between parts and the environment can lead to slippage or part breakage. In this paper, we propose a safe reinforcement learning approach to develop a visuo-tactile assembly policy that is robust to variations in grasp poses. Our method minimizes collisions between the terminal head and terminal base by decomposing the assembly task into three distinct phases. In the first grasp phase,a vision-guided model is trained to pick the terminal head from an initial bin. In the second align phase, a tactile-based grasp pose estimation model is employed to align the terminal head with the terminal base. In the final assembly phase, a visuo-tactile policy is learned to precisely insert the terminal head into the terminal base. To ensure safe training, the robot leverages human demonstrations and interventions. Experimental results on PLC terminal assembly demonstrate that the proposed method achieves 100% successful insertions across 100 different initial end-effector and grasp poses, while imitation learning and online-RL policy yield only 9% and 0%.

The integration of Vision-Language Models (VLMs) into autonomous systems is of growing importance for improving Human-Robot Interaction (HRI), enabling robots to operate within complex and unstructured environments and collaborate with non-expert users. For mobile robots to be effectively deployed in dynamic settings such as domestic or industrial areas, the ability to interpret and execute natural language commands is crucial. However, while VLMs offer powerful zero-shot, open-vocabulary recognition capabilities, their high computational cost presents a significant challenge for real-time performance on resource-constrained edge devices. This study provides a systematic analysis of the trade-offs involved in optimizing a real-time robotic perception pipeline on the NVIDIA Jetson AGX Orin 64GB platform. We investigate the relationship between accuracy and latency by evaluating combinations of two open-vocabulary detection models and two prompt-based segmentation models. Each pipeline is optimized using various precision levels (FP32, FP16, and Best) via NVIDIA TensorRT. We present a quantitative comparison of the mean Intersection over Union (mIoU) and latency for each configuration, offering practical insights and benchmarks for researchers and developers deploying these advanced models on embedded systems.

Transcatheter Aortic Valve Implantation (TAVI) is a minimally invasive procedure in which a transcatheter heart valve (THV) is implanted within the patient's diseased native aortic valve. The procedure is increasingly chosen even for intermediate-risk and younger patients, as it combines complication rates comparable to open-heart surgery with the advantage of being far less invasive. Despite its benefits, challenges remain in achieving accurate and repeatable valve positioning, with inaccuracies potentially leading to complications such as THV migration, coronary obstruction, and conduction disturbances (CD). The latter often requires a permanent pacemaker implantation as a costly and life-changing mitigation. Robotic assistance may offer solutions, enhancing precision, standardization, and reducing radiation exposure for clinicians. This article introduces a novel solution for robot-assisted TAVI, addressing the growing need for skilled clinicians and improving procedural outcomes. We present an in-vivo animal demonstration of robotic-assisted TAVI, showing feasibility of tele-operative instrument control and THV deployment. This, done at safer distances from radiation sources by a single operator. Furthermore, THV positioning and deployment under supervised autonomy is demonstrated on phantom, and shown to be feasible using both camera- and fluoroscopy-based imaging feedback and AI. Finally, an initial operator study probes performance and potential added value of various technology augmentations with respect to a manual expert operator, indicating equivalent to superior accuracy and repeatability using robotic assistance. It is concluded that robot-assisted TAVI is technically feasible in-vivo, and presents a strong case for a clinically meaningful application of level-3 autonomy. These findings support the potential of surgical robotic technology to enhance TAVI accuracy and repeatability, ultimately improving patient outcomes and expanding procedural accessibility.

Mobile robots require knowledge of the environment, especially of humans located in its vicinity. While the most common approaches for detecting humans involve computer vision, an often overlooked hardware feature of robots for people detection are their 2D range finders. These were originally intended for obstacle avoidance and mapping/SLAM tasks. In most robots, they are conveniently located at a height approximately between the ankle and the knee, so they can be used for detecting people too, and with a larger field of view and depth resolution compared to cameras. In this paper, we present a new dataset for people detection using knee-high 2D range finders called FROG. This dataset has greater laser resolution, scanning frequency, and more complete annotation data compared to existing datasets such as DROW (Beyer et al., 2018). Particularly, the FROG dataset contains annotations for 100% of its laser scans (unlike DROW which only annotates 5%), 17x more annotated scans, 100x more people annotations, and over twice the distance traveled by the robot. We propose a benchmark based on the FROG dataset, and analyze a collection of state-of-the-art people detectors based on 2D range finder data. We also propose and evaluate a new end-to-end deep learning approach for people detection. Our solution works with the raw sensor data directly (not needing hand-crafted input data features), thus avoiding CPU preprocessing and releasing the developer of understanding specific domain heuristics. Experimental results show how the proposed people detector attains results comparable to the state of the art, while an optimized implementation for ROS can operate at more than 500 Hz.

Due to their utility in replacing workers in tasks unsuitable for humans, unmanned underwater vehicles (UUVs) have become increasingly common tools in the fish farming industry. However, earlier studies and anecdotal evidence from farmers imply that farmed fish tend to move away from and avoid intrusive objects such as vehicles that are deployed and operated inside net pens. Such responses could imply a discomfort associated with the intrusive objects, which, in turn, can lead to stress and impaired welfare in the fish. To prevent this, vehicles and their control systems should be designed to automatically adjust operations when they perceive that they are repelling the fish. A necessary first step in this direction is to develop on-vehicle observation systems for assessing object/vehicle-fish distances in real-time settings that can provide inputs to the control algorithms. Due to their small size and low weight, modern cameras are ideal for this purpose. Moreover, the ongoing rapid developments within deep learning methods are enabling the use of increasingly sophisticated methods for analyzing footage from cameras. To explore this potential, we developed three new pipelines for the automated assessment of fish-camera distances in video and images. These methods were complemented using a recently published method, yielding four pipelines in total, namely, SegmentDepth, BBoxDepth, and SuperGlue that were based on stereo-vision and DepthAnything that was monocular. The overall performance was investigated using field data by comparing the fish-object distances obtained from the methods with those measured using a sonar. The four methods were then benchmarked by comparing the number of objects detected and the quality and overall accuracy of the stereo matches (only stereo-based methods). SegmentDepth, DepthAnything, and SuperGlue performed well in comparison with the sonar data, yielding mean absolute errors (MAE) of 0.205 m (95% CI: 0.050-0.360), 0.412 m (95% CI: 0.148-0.676), and 0.187 m (95% CI: 0.073-0.300), respectively, and were integrated into the Robot Operating System (ROS2) framework to enable real-time application in fish behavior identification and the control of robotic vehicles such as UUVs.

As aquaculture expands to meet global food demand, it remains dependent on manual, costly, infrequent, and high-risk operations due to reliance on high-end Remotely Operated Vehicles (ROVs). Scalable and autonomous systems are needed to enable safer and more efficient practices. This paper proposes a cost-effective autonomous inspection framework for the monitoring of mooring systems, a critical component ensuring structural integrity and regulatory compliance for both the aquaculture and floating offshore wind (FOW) sectors. The core contribution of this paper is a modular and scalable vision-based inspection pipeline built on the open-source Robot Operating System 2 (ROS 2) and implemented on a low-cost Blueye X3 underwater drone. The system integrates real-time image enhancement, YOLOv5-based object detection, and 4-DOF visual servoing for autonomous tracking of mooring lines. Additionally, the pipeline supports 3D reconstruction of the observed structure using tools such as ORB-SLAM3 and Meshroom, enabling future capabilities in change detection and defect identification. Validation results from simulation, dock and sea trials showed that the underwater drone can effective inspect of mooring system critical components with real-time processing on edge hardware. A cost estimation for the proposed approach showed a substantial reduction as compared with traditional ROV-based inspections. By increasing the Level of Autonomy (LoA) of off-the-shelf drones, this work provides (1) safer operations by replacing crew-dependent and costly operations that require a ROV and a mothership, (2) scalable monitoring and (3) regulatory-ready documentation. This offers a practical, cross-industry solution for sustainable offshore infrastructure management.

Introduction: This paper presents a participatory design approach for developing assistive robots, addressing the critical gap between designing robotic applications and real-world user needs. Traditional design methodologies often fail to capture authentic requirements due to users' limited familiarity with robotic technologies and the disconnection between design activities and actual deployment contexts.

Methods: We propose a methodology centred on iterative in-situ co-design, where stakeholders collaborate with researchers using functional low-fidelity prototypes within the actual environment of use. Our approach comprises three phases: observation and inspiration, in-situ co-design through prototyping, which is the core of the methodology, and longitudinal evaluation. We implemented this methodology over 10 months at an intermediate healthcare centre. The process involved healthcare staff in defining functionality, designing interactions, and refining system behaviour through hands-on experience with teleoperated prototypes.

Results: The resulting autonomous patrolling robot operated continuously across a two-month deployment. The evaluation through questionnaires on usability, usage and understanding of the robotic system, along with open-ended questions revealed diverse user adoption patterns, with five distinct personas emerging: enthusiastic high-adopter, disillusioned high-adopter, unconvinced mid-adopter, satisfied mid-adopter and non-adopter, which are discussed in detail.

Discussion: During the final evaluation deployment, user feedback still identified both new needs and practical improvements, as co-design iterations have the potential to continue indefinitely. Moreover, despite some performance issues, the robot's presence seemed to generate a placebo effect on both staff and patients, while it appears that staff's behaviours were also influenced by the regular observation of the researchers. The obtained results prove valuable insights into long-term human-robot interaction dynamics, highlighting the importance of context-based requirements gathering.

Social robots have the potential to support the health activities of older adults. However, they need to be designed for their specific needs; be accepted by and useful to them; and be integrated into their healthcare ecosystem and care network. We explored the research literature to determine the evidence base to guide design considerations necessary for socially assistive robots (SARs) for older adults in the context of healthcare. We identified various elements of the user-centered design of SARs to meet the needs of older adults within the constraints of a home environment. We emphasized the potential benefits of SARs in empowering older adults and supporting their autonomy for health applications. We identified research gaps and provided a road map for future development and deployment to enhance SAR functionality within digital health systems.

With ageing populations and decreasing numbers of care personnel, care technologies such as socially assistive robots offer innovative solutions for healthcare workers and older adults, supporting ageing in place. Among others, SARs are used for both daytime structure support and social companionship, particularly benefiting people with dementia by providing structure in earlier stages of the disease and comfort in later stages. This research introduces the concept of Huggable Integrated SARs (HI-SAR): a novel subtype of SARs combining a soft, comforting, huggable form with integrated socially assistive functionalities, such as verbal prompts for daytime structure, interactive companionship, and activity monitoring via sensor data, enabling the possibility of more context-aware interaction. While HI-SARs have shown promise in Asian care contexts, real-world application and potential in diverse long-term care contexts remain limited and underexplored. This research investigates the potential of HI-SARs in Dutch healthcare settings (eldercare, disability care, and rehabilitation) through three studies conducted between September 2023 and December 2024. Study I examined HI-SAR functions and integration in Dutch care practice via focus groups with professionals, innovation managers, and older adults (N = 36). Study II explored user preferences through sessions with clients with intellectual disabilities and professionals (N = 32). Study III involved two case studies in care settings with clients and caregivers (N = 4). Results indicate that HI-SARs were generally well-received by professionals and older adults, who appreciated their support for daily routines and social engagement, particularly for clients with cognitive disabilities such as dementia. However, concerns were raised about hygiene, the functioning of activity monitoring, and limited interactivity. Based on these findings, we recommend four design and implementation strategies to improve the effectiveness of HI-SARs: (1) integrating personalisation options such as customizable voices to increase user acceptance; (2) optimising activity monitoring by simplifying data output and using sensor input more proactively to trigger interactions; (3) considering persons with cognitive impairments as a first target user group; and (4) encouraging individual use to enhance hygiene and tailor experiences to client needs. Overall, this research demonstrates the potential of HI-SARs in diverse long-term care settings, although further research is needed to explore their applicability, usability, and long-term impact.

Introduction: The challenges encountered in the design of multi-robot teams (MRT) highlight the need for different levels of human involvement, creating human-in-the-loop multi-robot teams. By integrating human cognitive abilities with the functionalities of the robots in the MRT, we can enhance overall system performance. Designing such a human-in-the-loop MRT requires several decisions based on the specific context of application. Before implementing these systems in real-world scenarios, it is essential to model and simulate the various components of the MRT to evaluate their impact on performance and the different roles a human operator might play.

Methods: We developed a simulation framework for a human-in-the-loop MRT using the Java Agent DEvelopment framework (JADE) and investigated the effects of different numbers of robots in the MRT, MRT architectures, and levels of human involvement (human collaboration and human intervention) on performance metrics.

Results: Results show that task execution outcomes and request completion times (RCT) improve with an increasing number of robots in the MRT. Human collaboration reduced the RCT, while human intervention increased the RCT, regardless of the number of robots in the MRT. The effect of system architecture was only significant when the number of robots in the MRT was low.

Discussion: This study demonstrates that both the number of robots in a multi-robot team (MRT) and the inclusion of a human in the loop significantly influence system performance. The findings also highlight the value of simulation as a cost- and time-efficiency strategy to evaluate MRT configurations prior to real-world implementation.