Improved ACO algorithm fused with improved Q-Learning algorithm for Bessel curve global path planning of search and rescue robots

Abstract

Addressing issues with traditional ant colony and reinforcement learning algorithms, such as low search efficiency and the tendency to produce insufficiently smooth paths that easily fall into local optima, this paper designs an improved ant colony optimization algorithm fusion with improved Q-Learning (IAC-IQL) algorithm for Bessel curve global path planning of search and rescue (SAR) robots. First, the heuristic function model in the ant colony algorithm is improved, the elite ant search strategy and the adaptive pheromone volatility factor strategy are introduced, and the initial path is searched in realize the motion environment with the help of the improved ant colony algorithm, and the initialized pheromone matrix is constructed. Second, the improved ant colony algorithm and Q-Learning (QL) algorithm are fused by utilizing the similarity between the pheromone matrix in the improved ant colony algorithm and the Q-matrix in the QL algorithm. A heuristic learning evaluation model is designed to dynamically adjust the learning factor and provide guidance for the search path. Additionally, a dynamic adaptive greedy strategy is introduced to balance the exploration and exploitation of the robot in the environment. Finally, the paths are smoothed using third-order Bessel curves to eliminate the problem of excessive steering angles. Through three sets of comparative simulation experiments conducted in Pycharm platform, the effectiveness, superiority, and practicality of the IAC-IQL algorithm were verified. The experimental results demonstrated that the IAC-IQL algorithm integrates the strong search capability of ant colony algorithm and the self-learning characteristics of QL algorithm. SAR robots equipped with the improved IAC-IQL algorithm exhibit significantly enhanced iterative search efficiency in grid simulation environment and image sampling simulation environment. The global path optimization indicators demonstrate high efficiency, and the paths are smoother.