Co-Evolutionary NSGA-III with deep reinforcement learning for multi-objective distributed flexible job shop scheduling

Abstract

The multi-objective distributed flexible job shop scheduling problem (MO-DFJSP) is important to balance manufacturing efficiency and environmental impacts. This work aims to address the MO-DFJSP, simultaneously minimizing makespan, total tardiness, and carbon emission. Previous research has highlighted the effectiveness of integrating Reinforcement Learning (RL) methods with evolutionary algorithms (EAs). However, existing works often execute EAs and RL independently, with RL influencing only specific parameters. This constrains the algorithms’ overall optimization capabilities. To fully exploit the advantages of RL, this paper presents a co-evolutionary non-dominated sorting genetic algorithm-III (NSGA-III) integrated with deep reinforcement learning (CEGA-DRL). In CEGA-DRL, we incorporate an innovative gene operator into the NSGA framework, enabling the RL agent to directly derive excellent gene combinations from a chromosome and feed them back into NSGA-III. This accelerates the learning process of NSGA-III. In addition, we present a dual experience-pool elite backtracking strategy (DEEBS) to offer NSGA-III’s high-quality solution as experiences for the RL agent. This, in turn, improves the learning efficiency of the RL agent. The performance of CEGA-DRL is evaluated on the self-constructed MO-DFJSP benchmarks with various transit time, energy consumption, and workshop configurations. Experimental results demonstrate that, in comparison to the state-of-the-art intelligent optimization algorithms, CEGA-DRL achieves superior results across all the scheduling objectives and exhibits significant advantages in solution convergence and distribution.