Reinforcement learning-based trajectory planning for continuous digging of excavator working devices in trenching tasks

Abstract

This paper addresses the challenge of real-time, continuous trajectory planning for autonomous excavation. A hybrid method combining particle swarm optimization (PSO) and reinforcement learning (RL) is proposed. First, three types of excavation trajectories are defined for different geometric shapes of the digging area. Then, an excavation trajectory optimization method based on the PSO algorithm is established, resulting in optimal trajectories, the sensitive parameters, and the corresponding variation ranges. Second, an RL model is built, and the optimization results obtained offline are used as training samples. The RL-based method can be applied for continuous digging tasks, which is beneficial for improving the overall efficiency of the autonomous operation of the excavator. Finally, simulation experiments were conducted in four distinct conditions. The results demonstrate that the proposed method effectively accomplishes excavation tasks, with trajectory generation completed within 0.5 s. Comprehensive performance metrics remained below 0.14, and the excavation rate exceeded 92%, surpassing or matching the performance of the optimization-based method and PINN-based method. Moreover, the proposed method produced consistently balanced trajectory performance across all sub-tasks. These results underline the method's effectiveness in achieving real-time, multi-objective, and continuous trajectory planning for autonomous excavators.