A centralized reinforcement learning approach for proactive scheduling in manufacturing

Abstract

Due to rapid development of information and communications technology (ICT) and the impetus for more effective, efficient and adaptive manufacturing, the concept of ICT based advanced manufacturing has increasingly become a prominent research topic across academia and industry during recent years. One critical aspect of advanced manufacturing is how to incorporate real time information and then optimally schedule manufacturing processes with multiple objectives. Due to its complexity and the need for adaptation, the manufacturing scheduling problem presents challenges for utilizing advanced ICT and thus calls for new approaches. The paper proposes a centralized reinforcement learning approach for optimally scheduling of a manufacturing system of multi-stage processes and multiple machines for multiple types of products. The approach, which employs learning and control algorithms to enable real time cooperation of each processing unit inside the system, is able to adaptively respond to dynamic scheduling changes. More specifically, we first formally define the scheduling problem through the construction of an objective function and related heuristic constraints for the underlying manufacturing tasks. Next, to effectively deal with the problem we defined, we maintain a distributed weighted vector to capture the cooperative pattern of massive action space and apply the reinforcement-learning approach to achieve the optimal policies for a set of processing machines according to a real time production environment, including dynamic requests for various products. Numerical experiments demonstrate that compared to different heuristic methods and multi-agent algorithms, the proposed centralized reinforcement learning method can provide more reliable solutions for the scheduling problem.