Distributed heterogeneous flexible job-shop scheduling problem considering automated guided vehicle transportation via improved deep Q network

Abstract

Distributed manufacturing has become a research hotspot in the context of economic globalization. The distributed heterogeneous flexible job-shop scheduling problem considering automated guided vehicle transportation (DHFJSP-AGV) extends the classic flexible job-shop scheduling problem (FJSP) but remains underexplored. DHFJSP-AGV involves four subproblems: assigning jobs to heterogeneous factories, scheduling jobs to machines, sequencing operations on machines and transporting jobs between machines using AGVs. Due to its complexity, this study proposes an improved deep Q network (DQN) real-time scheduling method aimed at minimizing makespan. A mixed integer linear programming model (MILP) of DHFJSP-AGV is developed and transformed into a Markov decision process (MDP). Eight general state features are extracted and normalized to represent the state space, while appropriate combination dispatching rules are selected as the action space. The state features of each scheduling point are input to the DQN, determining the factory, job, machine, and AGV for each process. Additionally, double DQN and an improved ε-greedy exploration are used to enhance the DQN. Numerical comparison experiments under different production configurations and real-world application in distributed flexible job-shop with dynamic map environment demonstrate the effectiveness and generalization capabilities of improved DQN.