Dynamic Inspection and Maintenance Scheduling for Multi-State Systems Under Time-Varying Demand: Proximal Policy Optimization

IF 2 3区 工程技术 Q3 ENGINEERING, INDUSTRIAL IISE Transactions Pub Date : 2023-09-15 DOI:10.1080/24725854.2023.2259949
Yiming Chen, Yu Liu, Tangfan Xiahou
{"title":"Dynamic Inspection and Maintenance Scheduling for Multi-State Systems Under Time-Varying Demand: Proximal Policy Optimization","authors":"Yiming Chen, Yu Liu, Tangfan Xiahou","doi":"10.1080/24725854.2023.2259949","DOIUrl":null,"url":null,"abstract":"AbstractInspection and maintenance activities are effective ways to reveal and restore the health conditions of many industrial systems, respectively. Most extant works on inspection and maintenance optimization problems assumed that systems operate under a time-invariant demand. Such a simplified assumption is oftentimes violated by a changeable market environment, seasonal factors, and even unexpected emergencies. In this article, with the aim of minimizing the expected total cost associated with inspections, maintenance, and unsupplied demand, a dynamic inspection and maintenance scheduling model is put forth for multi-state systems (MSSs) under a time-varying demand. Non-periodic inspections are performed on the components of an MSS and imperfect maintenance actions are dynamically scheduled based on the inspection results. By introducing the concept of decision epochs, the resulting inspection and maintenance scheduling problem is formulated as a Markov decision process (MDP). The deep reinforcement learning (DRL) method with a proximal policy optimization (PPO) algorithm is customized to cope with the “curse of dimensionality” of the resulting sequential decision problem. As an extra input feature for the agent, the category of decision epochs is formulated to improve the effectiveness of the customized DRL method. A six-component MSS, along with a multi-state coal transportation system, is given to demonstrate the effectiveness of the proposed method.Keywords: multi-state systemdeep reinforcement learningdynamic inspection and maintenance schedulingproximal policy optimizationtime-varying demandDisclaimerAs a service to authors and researchers we are providing this version of an accepted manuscript (AM). Copyediting, typesetting, and review of the resulting proofs will be undertaken on this manuscript before final publication of the Version of Record (VoR). During production and pre-press, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal relate to these versions also. Additional informationNotes on contributorsYiming ChenYiming Chen received the Ph.D. degrees in mechanical engineering from the University of Electronic Science and Technology of China in 2022. He is currently a Lecturer with the College of Marine Equipment and Mechanical Engineering, Jimei University. His research interests include maintenance decisions, stochastic dynamic programming, and deep reinforcement learning.Yu LiuYu Liu is a professor of industrial engineering with the School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China. He received a PhD degree from the University of Electronic Science and Technology of China in 2010. He was a Visiting Predoctoral Fellow with the Department of Mechanical Engineering, Northwestern University, USA, from 2008 to 2010 and a Postdoctoral Research Fellow with the Department of Mechanical Engineering, University of Alberta, Canada, from 2012 to 2013. He has authored or coauthored more than 90 peer reviewed papers in international journals. His research interests include system reliability modeling and analysis, maintenance decisions, prognostics and health management, and design under uncertainty. He is an editorial board member of several international journals, such as Reliability Engineering & System Safety, Quality and Reliability Engineering International and an Associate Editor of IISE Transactions and IEEE Transactions on Reliability.Tangfan XiahouTangfan Xiahou received the M.Sc. and Ph.D. degrees in mechanical engineering from the University of Electronic Science and Technology of China in 2018 and 2022, respectively. He is currently a Lecturer with the School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China. His research interests include reliability modeling under uncertainty, Dempster–Shafer evidence theory, and prognostics and health management.","PeriodicalId":56039,"journal":{"name":"IISE Transactions","volume":"31 1","pages":"0"},"PeriodicalIF":2.0000,"publicationDate":"2023-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IISE Transactions","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/24725854.2023.2259949","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
引用次数: 0

Abstract

AbstractInspection and maintenance activities are effective ways to reveal and restore the health conditions of many industrial systems, respectively. Most extant works on inspection and maintenance optimization problems assumed that systems operate under a time-invariant demand. Such a simplified assumption is oftentimes violated by a changeable market environment, seasonal factors, and even unexpected emergencies. In this article, with the aim of minimizing the expected total cost associated with inspections, maintenance, and unsupplied demand, a dynamic inspection and maintenance scheduling model is put forth for multi-state systems (MSSs) under a time-varying demand. Non-periodic inspections are performed on the components of an MSS and imperfect maintenance actions are dynamically scheduled based on the inspection results. By introducing the concept of decision epochs, the resulting inspection and maintenance scheduling problem is formulated as a Markov decision process (MDP). The deep reinforcement learning (DRL) method with a proximal policy optimization (PPO) algorithm is customized to cope with the “curse of dimensionality” of the resulting sequential decision problem. As an extra input feature for the agent, the category of decision epochs is formulated to improve the effectiveness of the customized DRL method. A six-component MSS, along with a multi-state coal transportation system, is given to demonstrate the effectiveness of the proposed method.Keywords: multi-state systemdeep reinforcement learningdynamic inspection and maintenance schedulingproximal policy optimizationtime-varying demandDisclaimerAs a service to authors and researchers we are providing this version of an accepted manuscript (AM). Copyediting, typesetting, and review of the resulting proofs will be undertaken on this manuscript before final publication of the Version of Record (VoR). During production and pre-press, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal relate to these versions also. Additional informationNotes on contributorsYiming ChenYiming Chen received the Ph.D. degrees in mechanical engineering from the University of Electronic Science and Technology of China in 2022. He is currently a Lecturer with the College of Marine Equipment and Mechanical Engineering, Jimei University. His research interests include maintenance decisions, stochastic dynamic programming, and deep reinforcement learning.Yu LiuYu Liu is a professor of industrial engineering with the School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China. He received a PhD degree from the University of Electronic Science and Technology of China in 2010. He was a Visiting Predoctoral Fellow with the Department of Mechanical Engineering, Northwestern University, USA, from 2008 to 2010 and a Postdoctoral Research Fellow with the Department of Mechanical Engineering, University of Alberta, Canada, from 2012 to 2013. He has authored or coauthored more than 90 peer reviewed papers in international journals. His research interests include system reliability modeling and analysis, maintenance decisions, prognostics and health management, and design under uncertainty. He is an editorial board member of several international journals, such as Reliability Engineering & System Safety, Quality and Reliability Engineering International and an Associate Editor of IISE Transactions and IEEE Transactions on Reliability.Tangfan XiahouTangfan Xiahou received the M.Sc. and Ph.D. degrees in mechanical engineering from the University of Electronic Science and Technology of China in 2018 and 2022, respectively. He is currently a Lecturer with the School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China. His research interests include reliability modeling under uncertainty, Dempster–Shafer evidence theory, and prognostics and health management.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
时变需求下多状态系统动态检修调度:近端策略优化
摘要检测和维护活动分别是揭示和恢复许多工业系统健康状况的有效途径。现有的大多数关于检查和维护优化问题的工作都假设系统在定常需求下运行。这种简化的假设常常被多变的市场环境、季节性因素,甚至是意外的紧急情况所违背。针对时变需求下的多状态系统(mss),以最小化与检测、维护和未供给需求相关的预期总成本为目标,建立了动态检测和维护调度模型。对MSS的组件执行非定期检查,并根据检查结果动态安排不完善的维护操作。通过引入决策周期的概念,将检修调度问题表述为马尔可夫决策过程(MDP)。基于近端策略优化(PPO)算法的深度强化学习(DRL)方法是为解决序列决策问题的“维数诅咒”而定制的。为了提高自定义DRL方法的有效性,作为智能体的额外输入特征,制定了决策时代的类别。以一个六分量的MSS和一个多状态煤炭运输系统为例,验证了该方法的有效性。关键词:多状态系统深度强化学习动态检修调度近端策略优化时变需求免责声明作为对作者和研究人员的服务,我们提供此版本的已接受稿件(AM)。在最终出版版本记录(VoR)之前,将对该手稿进行编辑、排版和审查。在制作和印前,可能会发现可能影响内容的错误,所有适用于期刊的法律免责声明也与这些版本有关。陈奕明,陈奕明,博士,2022年毕业于中国电子科技大学机械工程专业。现任集美大学船舶装备与机械工程学院讲师。他的研究兴趣包括维护决策、随机动态规划和深度强化学习。刘宇,中国电子科技大学机电工程学院工业工程系教授。2010年获中国电子科技大学博士学位。2008 - 2010年任美国西北大学机械工程系访问前研究员,2012 - 2013年任加拿大阿尔伯塔大学机械工程系博士后研究员。他在国际期刊上撰写或合作撰写了90多篇同行评议论文。他的研究兴趣包括系统可靠性建模和分析、维护决策、预测和健康管理以及不确定性下的设计。他是几个国际期刊的编辑委员会成员,如可靠性工程与系统安全,质量和可靠性工程国际,以及IISE交易和IEEE可靠性交易的副主编。唐凡夏侯,分别于2018年和2022年获得中国电子科技大学机械工程专业硕士和博士学位。现任中国电子科技大学机电工程学院讲师。主要研究方向为不确定性下的可靠性建模、邓普斯特-谢弗证据理论、预测与健康管理。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
IISE Transactions
IISE Transactions Engineering-Industrial and Manufacturing Engineering
CiteScore
5.70
自引率
7.70%
发文量
93
期刊介绍: IISE Transactions is currently abstracted/indexed in the following services: CSA/ASCE Civil Engineering Abstracts; CSA-Computer & Information Systems Abstracts; CSA-Corrosion Abstracts; CSA-Electronics & Communications Abstracts; CSA-Engineered Materials Abstracts; CSA-Materials Research Database with METADEX; CSA-Mechanical & Transportation Engineering Abstracts; CSA-Solid State & Superconductivity Abstracts; INSPEC Information Services and Science Citation Index. Institute of Industrial and Systems Engineers and our publisher Taylor & Francis make every effort to ensure the accuracy of all the information (the "Content") contained in our publications. However, Institute of Industrial and Systems Engineers and our publisher Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Institute of Industrial and Systems Engineers and our publisher Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Institute of Industrial and Systems Engineers and our publisher Taylor & Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to, or arising out of the use of the Content. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions .
期刊最新文献
Robust Optimization Approaches in Inventory Management: Part A - The Survey Robust Optimization Approaches in Inventory Management: Part B - The Comparative Study FUSION3D: Multimodal Data Fusion for 3D Shape Reconstruction - A Soft Sensing Approach Adversarial Client Detection via Non-parametric Subspace Monitoring in the Internet of Federated Things Robust Radiotherapy Planning with Spatially Based Uncertainty Sets
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1