Real-time scheduling for production-logistics collaborative environment using multi-agent deep reinforcement learning

Abstract

With the extensive application of automated guided vehicle (AGV), production-logistics collaborative scheduling problem (PLCSP) becomes challenging for enterprises. Meanwhile, large-scale order and dynamic events bring more complexity and uncertainty. At present, deep reinforcement learning (DRL) has emerged as a promising scheduling approach. Therefore, this paper proposes a real-time scheduling method based on multi-agent DRL for PLCSP with dynamic job arrivals to minimize the total weighted tardiness. Specifically, a novel scheduling framework is designed in which a new logistics task release moment is given to reserve lots of AGV preparation time and avoid unnecessary premature decisions. Then, a training algorithm based on multi-agent proximal policy optimization is proposed to achieve job filtering, job selection and AGV selection. The action space and action space pruning strategy are designed for each agent to ensure the sufficient exploration and reduce the learning difficulty. Moreover, three state spaces with serial relationship and a reward function considering job classification are proposed. Experiments on 120 instances show that the proposed method has superiority and generality compared with scheduling rules and genetic programming, as well as three popular DRL-based methods, and the performance improvement mostly exceeds 10%. Furthermore, a real-world case is studied to show that the proposed method is applicable to solve the complex production-logistics collaborative scheduling problems.