Deep reinforcement learning-based preventive maintenance for repairable machines with deterioration in a flow line system

Abstract

In manufacturing systems, preventive maintenance plays a critical role in maintaining product yield, product quality, and machine reliability. Inappropriate maintenance strategies can lead to low yields, faulty products, machine failures, and disrupted operation of upstream and downstream machines. However, developing maintenance strategies in a stochastic factory environment can be challenging due to factors such as varying levels of deterioration, unpredictable maintenance times, and fluctuating machine workloads. Since previous studies formulated the maintenance decision using a Markov decision process, we propose a deep reinforcement learning method to derive the maintenance policy. We also consider the multi-objective method, hypervolume, to illustrate the trade-off between maintenance cost, production loss, and yield loss. The simulation study shows that our proposed method outperforms age-dependent and run-to-failure strategies in ten different scenarios. In addition to obtaining an optimal approximate policy, visualizing action trajectories provides managerial insights for optimizing and balancing different costs. Moreover, implementing preventive maintenance policies derived from our proposed method can enhance the robustness of supply chain operations. By reducing the risk of unexpected equipment failures, supply chains can achieve higher levels of operational reliability and continuity.