An improved Q-learning based rescheduling method for flexible job-shops with machine failures

Abstract

Scheduling of flexible job shop has been researched over several decades and continues to attract the interests of many scholars. But in the real manufacturing system, dynamic events such as machine failures are major issues. In this paper, an improved Q-learning algorithm with double-layer actions is proposed to solve the dynamic flexible job-shop scheduling problem (DFJSP) considering machine failures. The initial scheduling scheme is obtained by Genetic Algorithm (GA), and the rescheduling strategy is acquired by the Agent of the proposed Q-learning based on dispatching rules. The agent of Q-learning is able to select both operations and alternative machines optimally when machine failure occurs. To testify this approach, experiments are designed and performed based on Mk03 problem of FJSP. Results demonstrate that the optimal rescheduling strategy varies in different machine failure status. And compared with adopting a single dispatching rule all the time, the proposed Q-learning can reduce time of delay in a frequent dynamic environment, which shows that agent-based method is suitable for DFJSP.