Path curvature incorporated reinforcement learning method for accurate path tracking of agricultural vehicles

Abstract

Traditional path tracking control of agricultural vehicles greatly relay on precision modelling or parameter tuning, cause sensitive to the environment condition change such as different land slip rate and unflat field. To address those issues and to realize stable and accuracy path tracking, this research presents a deep reinforcement learning-based path tracking control algorithm that incorporates path curvature. A Deep Q-Network (DQN) based on a five-layer Back Propagation (BP) neural network was constructed, achieving a lightweight and highly portable algorithm. The network’s input state is optimized by integrating the average path curvature over a set distance ahead of the vehicle, thereby enhancing the vehicle’s path tracking precision. The convergence of the designed DQN-based path tracking control algorithm was validated in simulated and hardened road environments; in addition, its tracking performance was compared with the pure pursuit control (PPC) method under two different field ground conditions. On soft and flat ground, the average tracking errors of the vehicle on straight path segments at 6 m and 5 m intervals were 0.023 m and 0.026 m, respectively, and 0.024 m and 0.036 m on curved path segments. On hard and uneven ground, the average tracking errors at 6 m and 5 m intervals were 0.029 m and 0.034 m, respectively, and 0.037 m and 0.035 m on curved segments, all outperforming the PPC method. These findings confirm that the proposed path tracking control algorithm exhibits excellent adaptability and stability and achieves precise path tracking under different road conditions and path curvatures.