Multi agent physics informed reinforcement learning for waterflooding optimization

Abstract

Waterflooding optimization is a critical process for enhancing oil recovery in mature oil fields, where conventional approaches often rely on fixed injection rates over an extended period. However, this may not be the most efficient strategy due to reservoir heterogeneity and complexity. In this study, we propose a multi-agent physics informed reinforcement learning (MAPIRL) framework to optimize the waterflooding process. The MAPIRL approach utilizes a Markov decision process to formulate the optimization problem, where multiple RL agents are trained to interact with a reservoir simulation model and receive rewards for each action. The proposed approach uses an actor–critic RL architecture to train the agents to find the optimal strategy. The agents interact with the environment during several episodes until convergence is achieved. We evaluated the effectiveness of the MAPIRL approach based on the improvement in net present value (NPV), which reflects the economic benefits of the optimized waterflooding strategy. Then, we compared the MAPIRL approach with the multi-objective particle swarm optimization (MOPSO) algorithm. The comparison revealed that the MAPIRL approach outperformed the MOPSO algorithm in terms of net present value. In conclusion, the MAPIRL approach is a scientifically accurate method for optimizing waterflooding in mature oil fields, providing a more efficient and robust waterflooding strategy that reduces water consumption and associated costs while maximizing the economic benefits. The ability of the MAPIRL approach to optimize the waterflooding process with a high degree of complexity makes it a promising tool for the energy industry, and further research is needed to explore its potential for addressing other complex problems in this domain.