Two-stage double deep Q-network algorithm considering external non-dominant set for multi-objective dynamic flexible job shop scheduling problems

With the continuous advancement of the manufacturing industry, many random disturbances gradually appear in the job shop production process, such as the insertion of new jobs or random machine failures. This type of disruption often creates production chaos and scheduling problems. This paper takes new job insertion and random machine failure as dynamic events. It establishes an optimization model for dynamic flexible job shop scheduling (DFJSP) problems to cope with this issue. A two-stage double deep Q-network, TS-DDQN algorithm is proposed based on an improved deep reinforcement learning algorithm to solve the complex multi-objective DFJSP optimization problem by adopting two-stage decision-making. In the TS-DDQN, an external non-dominated set is used to enhance the training speed of the network and the quality of the solution, which can store the non-dominated solutions. Moreover, the solutions on the Pareto front are used to train the network parameters. Extensive experimentation is conducted on benchmark datasets to evaluate the performance of the proposed algorithm against the existing scheduling methods. The outcomes underscore the superior efficacy of the proposed algorithm concerning solution quality, convergence speed, and adaptability within dynamic environments. This research contributes to advancing the methods in solving multi-objective DFJSP problems and highlights the potential of deep reinforcement learning to yield managerial advantages in manufacturing industries.