Multi-agent reinforcement learning framework for real-time scheduling of pump and valve in water distribution networks

Abstract

With energy and water resources shortages, the energy and water resources managements of water distribution networks (WDNs) have become increasingly important. However, achieving real-time scheduling of pump and valve in dynamic environments remains challenging. Thus, this study proposes a multi-agent reinforcement learning scheduling framework to address the uncertainty of water demand in WDNs. First, we constructed a WDN environment and modelled the scheduling problem as a Markov decision process. Second, a multi-agent deep deterministic policy gradient (MADDPG) method was used to determine the strategy of the fully cooperative multi-agent task. Moreover, the impacts of energy and water loss costs on the scheduling strategy were explored. Finally, the results were compared with those of a genetic algorithm (GA), particle swarm optimisation (PSO), and differential evolution (DE) to verify the performance and robustness of the proposed model. The results show that water loss dominates the scheduling process, and the scheduling solutions for minimising water loss and energy costs are mainly affected by the demand pattern of consumers rather than the energy tariff. The proposed MADDPG model outperforms the GA, PSO, and DE models, achieving a significantly faster solution, which is advantageous for practical applications.