A semi-centralized multi-agent RL framework for efficient irrigation scheduling

Abstract

Efficient water management in agriculture is essential for addressing the growing freshwater scarcity crisis. Multi-Agent Reinforcement Learning (MARL) has emerged as a promising method for solving daily irrigation scheduling problems in spatially variable fields, where management zones are employed to account for field variability. To enhance the application of MARL to address daily irrigation scheduling in large-scale fields with significant spatial variation, this study proposes a Semi-Centralized MARL (SCMARL) framework. The SCMARL framework adopts a hierarchical structure, decomposing the daily irrigation scheduling problem into two levels of decision-making. At the top level, a centralized coordinator agent determines irrigation timing, which is modeled as a discrete variable, based on field-wide soil moisture data, crop conditions, and weather forecasts. At the lower level, decentralized local agents use local soil moisture, crop, and weather information to determine the appropriate irrigation amounts for each management zone. To address the issue of non-stationarity in this framework, a state augmentation technique is employed, wherein the coordinator’s decision is incorporated into the decision-making process of the local agents. The SCMARL framework, which leverages the Proximal Policy Optimization algorithm for training the agents, is evaluated on a large-scale field in Lethbridge, Canada, and compared with an existing MARL irrigation scheduling approach. The results demonstrate improved performance, achieving a 4.0% reduction in water use and a 6.3% increase in irrigation water use efficiency.