Attention enhanced reinforcement learning for flexible job shop scheduling with transportation constraints

Abstract

In smart manufacturing systems, the flexible job-shop scheduling problem with transportation constraints (FJSPT) is a critical challenge that can significantly improve production efficiency. FJSPT extends the traditional flexible job-shop scheduling problem (FJSP) by integrating the scheduling of automated guided vehicles (AGVs) to transport intermediate products between machines. Recent advances in data-driven methods, particularly deep reinforcement learning (DRL), have addressed challenging combinatorial optimization problems. DRL effectively solves discrete optimization problems by generating high-quality solutions within reasonable time. This paper presents an end-to-end DRL approach for the simultaneous scheduling of machines and AGVs in FJSPT. To apply DRL to the FJSPT, this paper first formulates a Markov decision process (MDP) model. The action space combines operation selection, machine assignment, and AGV planning. To capture problem characteristics, the scheduling agent uses a graph attention network (GAT) and multi-layer perceptron (MLP) for feature extraction, combined with proximal policy optimization (PPO) for stable training. Experimental evaluations conducted on synthetic data and public instances demonstrate that the proposed method outperforms dispatching rules and state-of-the-art models in both scheduling performance and computational efficiency.