强化学习在污水处理过程控制优化中的应用:方法、挑战和前进道路

IF 11.4 1区 环境科学与生态学 Q1 ENVIRONMENTAL SCIENCES Critical Reviews in Environmental Science and Technology Pub Date : 2023-03-06 DOI:10.1080/10643389.2023.2183699
Henry C. Croll, Kaoru Ikuma, S. Ong, S. Sarkar
{"title":"强化学习在污水处理过程控制优化中的应用:方法、挑战和前进道路","authors":"Henry C. Croll, Kaoru Ikuma, S. Ong, S. Sarkar","doi":"10.1080/10643389.2023.2183699","DOIUrl":null,"url":null,"abstract":"Abstract Wastewater treatment process control optimization is a complex task in a highly nonlinear environment. Reinforcement learning (RL) is a machine learning technique that stands out for its ability to perform better than human operators for certain high-dimensional, complex decision-making problems, making it an ideal candidate for wastewater treatment process control optimization. However, while RL control optimization strategies have shown potential to provide operational cost savings and effluent quality improvements, RL has proven slow to be adopted among environmental engineers. This review provides an overview of existing RL applications for wastewater treatment control optimization found in literature and evaluates five key challenges that must be addressed prior to widespread adoption: practical RL implementation, managing data, integrating existing process models, building trust in empirical control strategies, and bridging gaps in professional training. Finally, this review discusses potential paths forward to addressing each key challenge, including leveraging soft sensing to improve online data collection, working with process engineers to integrate RL programming with existing industry software, utilizing supervised training to build expert knowledge into the RL agent, and focusing research efforts on known scenarios such as the Benchmark Simulation Model No. 1 to build a robust database of RL agent control optimization results. GRAPHICAL ABSTRACT","PeriodicalId":10823,"journal":{"name":"Critical Reviews in Environmental Science and Technology","volume":"53 1","pages":"1775 - 1794"},"PeriodicalIF":11.4000,"publicationDate":"2023-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":"{\"title\":\"Reinforcement learning applied to wastewater treatment process control optimization: Approaches, challenges, and path forward\",\"authors\":\"Henry C. Croll, Kaoru Ikuma, S. Ong, S. Sarkar\",\"doi\":\"10.1080/10643389.2023.2183699\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract Wastewater treatment process control optimization is a complex task in a highly nonlinear environment. Reinforcement learning (RL) is a machine learning technique that stands out for its ability to perform better than human operators for certain high-dimensional, complex decision-making problems, making it an ideal candidate for wastewater treatment process control optimization. However, while RL control optimization strategies have shown potential to provide operational cost savings and effluent quality improvements, RL has proven slow to be adopted among environmental engineers. This review provides an overview of existing RL applications for wastewater treatment control optimization found in literature and evaluates five key challenges that must be addressed prior to widespread adoption: practical RL implementation, managing data, integrating existing process models, building trust in empirical control strategies, and bridging gaps in professional training. Finally, this review discusses potential paths forward to addressing each key challenge, including leveraging soft sensing to improve online data collection, working with process engineers to integrate RL programming with existing industry software, utilizing supervised training to build expert knowledge into the RL agent, and focusing research efforts on known scenarios such as the Benchmark Simulation Model No. 1 to build a robust database of RL agent control optimization results. GRAPHICAL ABSTRACT\",\"PeriodicalId\":10823,\"journal\":{\"name\":\"Critical Reviews in Environmental Science and Technology\",\"volume\":\"53 1\",\"pages\":\"1775 - 1794\"},\"PeriodicalIF\":11.4000,\"publicationDate\":\"2023-03-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"4\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Critical Reviews in Environmental Science and Technology\",\"FirstCategoryId\":\"93\",\"ListUrlMain\":\"https://doi.org/10.1080/10643389.2023.2183699\",\"RegionNum\":1,\"RegionCategory\":\"环境科学与生态学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENVIRONMENTAL SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Critical Reviews in Environmental Science and Technology","FirstCategoryId":"93","ListUrlMain":"https://doi.org/10.1080/10643389.2023.2183699","RegionNum":1,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
引用次数: 4

摘要

摘要污水处理过程控制优化是一个高度非线性环境下的复杂任务。强化学习(RL)是一种机器学习技术,它能够在某些高维、复杂的决策问题上比人类操作员表现得更好,是废水处理过程控制优化的理想候选者。然而,尽管RL控制优化策略已显示出节省运营成本和改善出水质量的潜力,但事实证明,RL在环境工程师中的应用进展缓慢。这篇综述概述了文献中现有的RL在废水处理控制优化中的应用,并评估了在广泛采用之前必须解决的五个关键挑战:实际的RL实施、数据管理、集成现有的过程模型、在经验控制策略中建立信任,以及弥合专业培训中的差距。最后,这篇综述讨论了解决每一个关键挑战的潜在途径,包括利用软测量改进在线数据收集,与流程工程师合作将RL编程与现有行业软件集成,利用监督培训将专家知识构建到RL代理中,并将研究重点放在已知场景上,如Benchmark Simulation Model No.1,以建立RL agent控制优化结果的鲁棒数据库。图形摘要
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Reinforcement learning applied to wastewater treatment process control optimization: Approaches, challenges, and path forward
Abstract Wastewater treatment process control optimization is a complex task in a highly nonlinear environment. Reinforcement learning (RL) is a machine learning technique that stands out for its ability to perform better than human operators for certain high-dimensional, complex decision-making problems, making it an ideal candidate for wastewater treatment process control optimization. However, while RL control optimization strategies have shown potential to provide operational cost savings and effluent quality improvements, RL has proven slow to be adopted among environmental engineers. This review provides an overview of existing RL applications for wastewater treatment control optimization found in literature and evaluates five key challenges that must be addressed prior to widespread adoption: practical RL implementation, managing data, integrating existing process models, building trust in empirical control strategies, and bridging gaps in professional training. Finally, this review discusses potential paths forward to addressing each key challenge, including leveraging soft sensing to improve online data collection, working with process engineers to integrate RL programming with existing industry software, utilizing supervised training to build expert knowledge into the RL agent, and focusing research efforts on known scenarios such as the Benchmark Simulation Model No. 1 to build a robust database of RL agent control optimization results. GRAPHICAL ABSTRACT
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
27.30
自引率
1.60%
发文量
64
审稿时长
2 months
期刊介绍: Two of the most pressing global challenges of our era involve understanding and addressing the multitude of environmental problems we face. In order to tackle them effectively, it is essential to devise logical strategies and methods for their control. Critical Reviews in Environmental Science and Technology serves as a valuable international platform for the comprehensive assessment of current knowledge across a wide range of environmental science topics. Environmental science is a field that encompasses the intricate and fluid interactions between various scientific disciplines. These include earth and agricultural sciences, chemistry, biology, medicine, and engineering. Furthermore, new disciplines such as environmental toxicology and risk assessment have emerged in response to the increasing complexity of environmental challenges. The purpose of Critical Reviews in Environmental Science and Technology is to provide a space for critical analysis and evaluation of existing knowledge in environmental science. By doing so, it encourages the advancement of our understanding and the development of effective solutions. This journal plays a crucial role in fostering international cooperation and collaboration in addressing the pressing environmental issues of our time.
期刊最新文献
Bisphenol A removal with nitrification and denitrification: Occurrence, mechanism, and microbial mutualism Carbon dots derived from non-biomass waste for a sensing application Comparison of fish and mammalian adipogenesis in vitro: Implications for regulating endocrine disruptors Anaerobic electrochemical membrane bioreactors: A panoramic tool for wastewater treatment and resource recovery Reactive oxidative species generation in pyrite abiotic-oxidation process: Origins, influencing factors, applications for environmental remediation
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1