{"title":"基于深度强化学习的预防性维护,适用于流水线系统中出现劣化的可维修机器","authors":"Yu-Hsin Hung, Hong-Ying Shen, Chia-Yen Lee","doi":"10.1007/s10479-024-06207-x","DOIUrl":null,"url":null,"abstract":"<p>In manufacturing systems, preventive maintenance plays a critical role in maintaining product yield, product quality, and machine reliability. Inappropriate maintenance strategies can lead to low yields, faulty products, machine failures, and disrupted operation of upstream and downstream machines. However, developing maintenance strategies in a stochastic factory environment can be challenging due to factors such as varying levels of deterioration, unpredictable maintenance times, and fluctuating machine workloads. Since previous studies formulated the maintenance decision using a Markov decision process, we propose a deep reinforcement learning method to derive the maintenance policy. We also consider the multi-objective method, hypervolume, to illustrate the trade-off between maintenance cost, production loss, and yield loss. The simulation study shows that our proposed method outperforms age-dependent and run-to-failure strategies in ten different scenarios. In addition to obtaining an optimal approximate policy, visualizing action trajectories provides managerial insights for optimizing and balancing different costs. Moreover, implementing preventive maintenance policies derived from our proposed method can enhance the robustness of supply chain operations. By reducing the risk of unexpected equipment failures, supply chains can achieve higher levels of operational reliability and continuity.</p>","PeriodicalId":8215,"journal":{"name":"Annals of Operations Research","volume":"28 1","pages":""},"PeriodicalIF":4.4000,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Deep reinforcement learning-based preventive maintenance for repairable machines with deterioration in a flow line system\",\"authors\":\"Yu-Hsin Hung, Hong-Ying Shen, Chia-Yen Lee\",\"doi\":\"10.1007/s10479-024-06207-x\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>In manufacturing systems, preventive maintenance plays a critical role in maintaining product yield, product quality, and machine reliability. Inappropriate maintenance strategies can lead to low yields, faulty products, machine failures, and disrupted operation of upstream and downstream machines. However, developing maintenance strategies in a stochastic factory environment can be challenging due to factors such as varying levels of deterioration, unpredictable maintenance times, and fluctuating machine workloads. Since previous studies formulated the maintenance decision using a Markov decision process, we propose a deep reinforcement learning method to derive the maintenance policy. We also consider the multi-objective method, hypervolume, to illustrate the trade-off between maintenance cost, production loss, and yield loss. The simulation study shows that our proposed method outperforms age-dependent and run-to-failure strategies in ten different scenarios. In addition to obtaining an optimal approximate policy, visualizing action trajectories provides managerial insights for optimizing and balancing different costs. Moreover, implementing preventive maintenance policies derived from our proposed method can enhance the robustness of supply chain operations. By reducing the risk of unexpected equipment failures, supply chains can achieve higher levels of operational reliability and continuity.</p>\",\"PeriodicalId\":8215,\"journal\":{\"name\":\"Annals of Operations Research\",\"volume\":\"28 1\",\"pages\":\"\"},\"PeriodicalIF\":4.4000,\"publicationDate\":\"2024-08-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annals of Operations Research\",\"FirstCategoryId\":\"91\",\"ListUrlMain\":\"https://doi.org/10.1007/s10479-024-06207-x\",\"RegionNum\":3,\"RegionCategory\":\"管理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"OPERATIONS RESEARCH & MANAGEMENT SCIENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Operations Research","FirstCategoryId":"91","ListUrlMain":"https://doi.org/10.1007/s10479-024-06207-x","RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"OPERATIONS RESEARCH & MANAGEMENT SCIENCE","Score":null,"Total":0}
Deep reinforcement learning-based preventive maintenance for repairable machines with deterioration in a flow line system
In manufacturing systems, preventive maintenance plays a critical role in maintaining product yield, product quality, and machine reliability. Inappropriate maintenance strategies can lead to low yields, faulty products, machine failures, and disrupted operation of upstream and downstream machines. However, developing maintenance strategies in a stochastic factory environment can be challenging due to factors such as varying levels of deterioration, unpredictable maintenance times, and fluctuating machine workloads. Since previous studies formulated the maintenance decision using a Markov decision process, we propose a deep reinforcement learning method to derive the maintenance policy. We also consider the multi-objective method, hypervolume, to illustrate the trade-off between maintenance cost, production loss, and yield loss. The simulation study shows that our proposed method outperforms age-dependent and run-to-failure strategies in ten different scenarios. In addition to obtaining an optimal approximate policy, visualizing action trajectories provides managerial insights for optimizing and balancing different costs. Moreover, implementing preventive maintenance policies derived from our proposed method can enhance the robustness of supply chain operations. By reducing the risk of unexpected equipment failures, supply chains can achieve higher levels of operational reliability and continuity.
期刊介绍:
The Annals of Operations Research publishes peer-reviewed original articles dealing with key aspects of operations research, including theory, practice, and computation. The journal publishes full-length research articles, short notes, expositions and surveys, reports on computational studies, and case studies that present new and innovative practical applications.
In addition to regular issues, the journal publishes periodic special volumes that focus on defined fields of operations research, ranging from the highly theoretical to the algorithmic and the applied. These volumes have one or more Guest Editors who are responsible for collecting the papers and overseeing the refereeing process.