Journal of Field Robotics最新文献

This paper investigates the differentiable dynamic modeling of mobile manipulators to facilitate efficient motion planning and physical design of actuators, where the actuator design is parameterized by physically meaningful motor geometry parameters. The proposed differentiable modeling comprises two major components. First, the dynamic model of the mobile manipulator is derived, which differs from the state-of-the-art in two aspects: (1) the model parameters, including magnetic flux, link mass, inertia, and center-of-mass, are represented as analytical functions of actuator design parameters; (2) the dynamic coupling between the base and the manipulator is captured. Second, the state and control constraints, such as maximum angular velocity and torque capacity, are established as analytical functions of actuator design parameters. This paper further showcases two typical use cases of the proposed differentiable modeling work: integrated locomotion and manipulation planning; simultaneous actuator design and motion planning. Numerical experiments demonstrate the effectiveness of differentiable modeling. That is, for motion planning, it can effectively reduce computation time as well as result in shorter task completion time and lower energy consumption, compared with an established sequential motion planning approach. Furthermore, actuator design and motion planning can be jointly optimized toward higher performance.

The task of collecting and transporting luggage trolleys in airports, characterized by its complexity within dynamic public environments, presents both an ongoing challenge and a promising opportunity for automated service robots. Previous research has primarily developed on universal platforms with robot arms or focused on handling a single trolley, creating a gap in providing cost-effective and efficient solutions for practical scenarios. In this paper, we propose a low-cost mobile manipulation robot incorporated with an autonomy framework for the collection and transportation of multiple trolleys that can significantly enhance operational efficiency. The method involves a novel design of the mechanical system and a vision-based control strategy. We design a lightweight manipulator and the docking mechanism, optimized for the sequential stacking and transportation of trolleys. On the basis of the Control Lyapunov Function and Control Barrier Function, we propose a vision-based controller with online Quadratic Programming, which improves the docking accuracy. The practical application of our system is demonstrated in real-world scenarios, where it successfully executes the multiple-trolley collection task.

A machine learning-based approach was utilized to develop a device for groundnut bud necrosis virus (GBNV) disease severity detection and estimation in tomato plants (Solanum lycopersicum L.). The study involved inoculating tomato plants with GBNV, monitoring changes in morphological and spectral characteristics, evaluating machine learning algorithms (decision tree [DT] classifier) for analysis and classification of disease severity, and developing and validating a device for disease detection and severity estimation. Spectral data analysis revealed distinct patterns in reflectance, with notable peaks observed in the 680 and 760 nm bands, while reflectance remained low and constant beyond 900 nm. Machine learning techniques, specifically a DT model, were employed to classify disease severity based on spectral data with high accuracy (95.01% training accuracy and 93.65% testing accuracy). The model identified the near-infrared band as highly correlated (correlation coefficient of 0.82) with disease severity. Furthermore, a compact handheld device integrating a spectral sensor, organic light-emitting diode display, and Raspberry Pi 3B was developed for real-time disease severity estimation. The device demonstrated robust performance, accurately predicting disease severity at different growth stages, even in the absence of visible symptoms. Additionally, disease severity percentages obtained via reverse transcription polymerase chain reaction were used to validate the accuracy of the device's estimations. Its responsive nature, with estimated response times ranging from milliseconds to seconds, facilitates timely interventions in agricultural settings. Overall, this interdisciplinary approach, combining spectral analysis, machine learning, and device development, presents a promising solution for efficient disease monitoring and management in agriculture, contributing to enhanced crop health and food security.

Intelligent electric shovels are being developed for intelligent mining in open-pit mines. Complex environment detection and target recognition based on image recognition technology are prerequisites for achieving intelligent electric shovel operation. However, there is a large amount of sand–dust in open-pit mines, which can lead to low visibility and color shift in the environment during data collection, resulting in low-quality images. The images collected for environmental perception in sand–dust environment can seriously affect the target detection and scene segmentation capabilities of intelligent electric shovels. Therefore, developing an effective image processing algorithm to solve these problems and improve the perception ability of intelligent electric shovels has become crucial. At present, methods based on deep learning have achieved good results in image dehazing, and have a certain correlation in image sand–dust removal. However, deep learning heavily relies on data sets, but existing data sets are concentrated in haze environments, with significant gaps in the data set of sand–dust images, especially in open-pit mining scenes. Another bottleneck is the limited performance associated with traditional methods when removing sand–dust from images, such as image distortion and blurring. To address the aforementioned issues, a method for generating sand–dust image data based on atmospheric physical models and CIELAB color space features is proposed. The impact mechanism of sand–dust on images was analyzed through atmospheric physical models, and the formation of sand–dust images was divided into two parts: blurring and color deviation. We studied the blurring and color deviation effect generation theories based on atmospheric physical models and CIELAB color space, and designed a two-stage sand–dust image generation method. We also constructed an open-pit mine sand–dust data set in a real mining environment. Last but not least, this article takes generative adversarial network (GAN) as the research foundation and focuses on the formation mechanism of sand–dust image effects. The CIELAB color features are fused with the discriminator of GAN as basic priors and additional constraints to improve the discrimination effect. By combining the three feature components of CIELAB color space and comparing the algorithm performance, a feature fusion scheme is determined. The results show that the proposed method can generate clear and realistic images well, which helps to improve the performance of target detection and scene segmentation tasks in heavy sand–dust open-pit mines.

Accurate and robust localization is essential for autonomous mobile robots. Map matching based on Light Detection and Ranging (LiDAR) sensors has been widely adopted to estimate the global location of robots. However, map-matching performance can be degraded when the environment changes or when sufficient features are unavailable. Indiscriminately incorporating inaccurate map-matching poses for localization can significantly decrease the reliability of pose estimation. This paper aims to develop a robust LiDAR-based localization method based on map matching. We focus on determining appropriate weights that are computed from the uncertainty of map-matching poses. The uncertainty of map-matching poses is estimated by the probability distribution over the poses. We exploit the normal distribution transform map to derive the probability distribution. A factor graph is employed to combine the map-matching pose, LiDAR-inertial odometry, and global navigation satellite system information. Experimental verification was successfully conducted outdoors on the university campus in three different scenarios, each involving changing or dynamic environments. We compared the performance of the proposed method with three LiDAR-based localization methods. The experimental results show that robust localization performances can be achieved even when map-matching poses are inaccurate in various outdoor environments. The experimental video can be found at https://youtu.be/L6p8gwxn4ak.

Selective harvesting by autonomous robots will be a critical enabling technology for future farming. Increases in inflation and shortages of skilled labor are driving factors that can help encourage user acceptability of robotic harvesting. For example, robotic strawberry harvesting requires real-time high-precision fruit localization, three-dimensional (3D) mapping, and path planning for 3D cluster manipulation. Whilst industry and academia have developed multiple strawberry harvesting robots, none have yet achieved human–cost parity. Achieving this goal requires increased picking speed (perception, control, and movement), accuracy, and the development of low-cost robotic system designs. We propose the edge-server over 5G for Selective Harvesting (E5SH) system, which is an integration of high bandwidth and low latency Fifth-Generation (5G) mobile network into a crop harvesting robotic platform, which we view as an enabler for future robotic harvesting systems. We also consider processing scale and speed in conjunction with system environmental and energy costs. A system architecture is presented and evaluated with support from quantitative results from a series of experiments that compare the performance of the system in response to different architecture choices, including image segmentation models, network infrastructure (5G vs. Wireless Fidelity), and messaging protocols, such as Message Queuing Telemetry Transport and Transport Control Protocol Robot Operating System. Our results demonstrate that the E5SH system delivers step-change peak processing performance speedup of above 18-fold than a standalone embedded computing Nvidia Jetson Xavier NX system.

Despite impressive results achieved by many on-land visual mapping algorithms in the recent decades, transferring these methods from land to the deep sea remains a challenge due to harsh environmental conditions. Images captured by autonomous underwater vehicles, equipped with high-resolution cameras and artificial illumination systems, often suffer from heterogeneous illumination and quality degradation caused by attenuation and scattering, on top of refraction of light rays. These challenges often result in the failure of on-land Simultaneous Localization and Mapping (SLAM) approaches when applied underwater or cause Structure-from-Motion (SfM) approaches to exhibit drifting or omit challenging images. Consequently, this leads to gaps, jumps, or weakly reconstructed areas. In this work, we present a navigation-aided hierarchical reconstruction approach to facilitate the automated robotic three-dimensional reconstruction of hectares of seafloor. Our hierarchical approach combines the advantages of SLAM and global SfM that are much more efficient than incremental SfM, while ensuring the completeness and consistency of the global map. This is achieved through identifying and revisiting problematic or weakly reconstructed areas, avoiding to omit images and making better use of limited dive time. The proposed system has been extensively tested and evaluated during several research cruises, demonstrating its robustness and practicality in real-world conditions.

Operating a two-wheel paddy transplanter traditionally poses physical strain and cognitive workload challenges for farm workers, especially during headland turns. This study introduces a virtual reality (VR)/augmented reality (AR)based remote-control system for a two-wheel paddy transplanter to resolve these issues. The system replaces manual controls with VR interfaces, integrating gear motors and an electronic control unit. Front and rear-view cameras provide real-time field perception on light-emitting diode screens, displaying path trajectories via an autopilot controller and real-time kinematic global navigation satellite systems module. Human operators manipulate the machine using a hand-held remote controller while observing live camera feeds and path navigation trajectories. The study found that forward speed necessitated optimization within manageable limits of 1.75–2.00 km h⁻¹ for walk-behind types and 2.00–2.25 km h⁻¹ for remote-controlled systems. While higher speeds enhanced field capacity by 11.67%–12.95%, they also resulted in 0.74%–1.17% lower field efficiency. Additionally, Operators' physiological workload analysis revealed significant differences between walk-behind and remotely controlled operators. Significant differences in energy expenditure rate (EER) were observed between walk-behind and remote-controlled paddy transplanters, with EER values ranging from 8.20 ± 0.80 to 27.67 ± 0.45 kJ min⁻¹ and 7.56 ± 0.55 to 9.72 ± 0.37 kJ min⁻¹, respectively (p < 0.05). Overall, the VR-based remote-control system shows promise in enhancing operational efficiency and reducing physical strain in paddy transplanting operations.

Heavy-duty construction tasks implemented by hydraulic manipulators are highly challenging due to unstructured hazardous environments. Considering many tasks have quasirepetitive features (such as cyclic material handling or excavation), a multitarget adaptive virtual fixture (MAVF) method by teleoperation-based learning from demonstration is proposed to improve task efficiency and safety, by generating an online variable assistance force on the master. First, the demonstration trajectory of picking scattered materials is learned to extract its distribution and the nominal trajectory is generated. Then, the MAVF is established and adjusted online by a defined nonlinear variable stiffness and position deviation from the nominal trajectory. An energy tank is introduced to regulate the stiffness so that passivity and stability can be ensured. Taking the operation mode without virtual fixture (VF) assistance and with traditional weighted adaptation VF as comparisons, two groups of tests with and without time delay were carried out to validate the proposed method.