A Real-time Smooth Lifting Path Planning for Tower Crane Based on TD3 with Discrete-Continuous Hybrid Action Space

Abstract

Smoothness and rapidity are two important performance indexes for crane lifting path planning. Traditionally, cranes are modeled as multi-freedom robots and use robot path planning algorithms to plan path in continuous space. However, the paths planned by these methods are not smooth enough to operate. In addition, presently, most of proposed lifting path planning algorithms focus on static environments, requiring accurate environmental information to build maps, which is too computation expensive to meet the requirement of real-time path planning. In this paper, we propose a deep reinforcement learning-based lifting path planning algorithm for hybrid action spaces. The network structure is developed based on TD3. A new reward function is designed and hindsight experience replay is used to solve the reward sparsity problem in long distance path planning. The planning path is smooth and can achieve real-time path planning in unknown environments. The result of simulation experiments demonstrates the effectiveness of the proposed approach.