Obstacle avoidance method based on reinforcement learning dual-layer decision model for AGV with visual perception

Abstract

In this paper, the reinforcement learning dual-layer decision strategy-based obstacle avoidance method for AGV with visual perception is proposed. Initially, the complementary obstacle detection system is established by combining the radar and RGB-D camera. The global environment information is obtained through radar scanning, while the obstacles information within the near-field of view are detected by camera, thereby simplifying the data processing complexities associated with multi-sensor fusion. Subsequently, the perturbation factor is formulated based on the position information of obstacles detected by the RGB-D camera, directly participating in the action value estimation of the Critic Network and enhancing the collision avoidance capability of AGV. Finally, the dual-layer decision method incorporating Deep Deterministic Policy Gradient (DDPG) and Deep Q-Network (DQN) is developed to control the rotate angle, ensuring the credibility of the steering decisions. Experiment results demonstrate that the proposed obstacle avoidance method, utilizing the dual-layer decision model with visual perception, exhibits superior obstacle avoidance performance, faster convergence speed, and stronger stability.