Enhancing Monte-Carlo Tree Search with Multi-Agent Deep Q-Network in Open Shop Scheduling

Abstract

Production disruptions, e.g., machine breakdowns, cannot be predicted in any case. Such disruptions lead to a deviation of the planned and optimized production schedule, and the actual production process. Instead of manually re-routing products, an online scheduler can re-route products automatically and maintain the best possible production throughput. To establish such an online scheduler, a framework for combining Monte-Carlo Tree Search (MCTS) and a multi-agent Deep Q-Network (MADQN) to solve the Open Shop Scheduling Problem (OSSP) is developed. Similar to approaches of using some sort of single-agent to guide the MCTS during the exploration phase, this approach deploys a multi-agent. Although the combination of single agents and MCTS have shown promising results in relatively small environments, applications relying on this approach have a very limited number of use cases in a real production scenario due to the considerable number of machines and products [10]. However, for that particular use case, the multi-agents promise a scalable solution even for large environments [6]. To do so, the problem has to be formulated such that a multi-agent can solve it. In addition to that, a learning framework is presented, and the developed approach is compared to an MCTS and single-agent combination.