An integrated reinforcement learning framework for simultaneous generation, design, and control of chemical process flowsheets

Abstract

This study introduces a Reinforcement Learning (RL) approach for synthesis, design, and control of chemical process flowsheets (CPFs). The proposed RL framework makes use of an inlet stream and a set of unit operations (UOs) available in the RL environment to build, evaluate and test CPFs. Moreover, the framework harnesses the power of surrogate models, specifically Neural Networks (NNs), to expedite the learning process of the RL agent and avoid reliance on mechanistic dynamic models embedded within the RL environment. These surrogate models approximate key process variables and descriptive closed-loop performance metrics for complex dynamic UO models. The proposed framework is evaluated through case studies, including a system where more than one type of UO is considered for simultaneous synthesis, design and control. The results show that the RL agent effectively learns to maintain the dynamic operability of the UOs under disturbances, adhere to equipment design and operational constraints, and generate viable and economically attractive CPFs.