Water Age Control for Water Distribution Networks via Safe Reinforcement Learning

Reinforcement learning (RL) is a widely used control technique that finds an optimal policy using the feedback of its actions. The search for the optimal policy requires that the system explores a broad region of the state space. This search puts at risk the safe operation, since some of the explored regions might be near the physical system limits. Implementing learning methods in industrial applications is limited because of its uncertain behavior when finding an optimal policy. This work proposes an RL control algorithm with a filter that supervises the safety of the exploration based on a nominal model. The performance of this safety filter is increased by modeling the uncertainty with a Gaussian process (GP) regression. This method is applied to the optimization of the management of a water distribution network (WDN) with an elevated reservoir; the management objectives are to regulate the tank filling while maintaining an adequate water turnover. The proposed method is validated in a laboratory setup that emulates the hydraulic features of a WDN.