Deep Reinforcement Learning-Based Mapless Navigation for Mobile Robot in Unknown Environment With Local Optima

Abstract

Local optima issues challenge mobile robots mapless navigation with the dilemma of avoiding collisions and approaching the target. Planning-based methods rely on environmental models and manual strategies to guide the robot. In contrast, learning-based methods can process original sensor data to navigate the robot in real-time but struggle with local optima. To address this, we designed reward rules that punish the robot for previously visited areas that may trap the robot, and reward it for exploring local areas in diverse ways and escaping from local optima areas. Then, we improved the Soft Actor-Critic (SAC) algorithm by making its temperature parameter adaptive to the current training status, and memorizing it in experiences for strategy updating, bringing additional exploratory behaviors and necessary stability into the training. Finally, with the assistance of auxiliary networks, the robot learns to handle various navigation tasks with local optima risks. Simulations demonstrate the advantages of our method in terms of both success rate and path efficiency compared to several existing methods. Experiments verified the proposed method in real-world scenarios.