D. Zanutto, A. Ficchì, M. Giuliani, A. Castelletti
Hydrological forecasts have significantly improved in skill over recent years, encouraging their systematic exploitation in multipurpose reservoir operations to improve reliability and resilience to extreme events. Despite the growing availability of multi-timescale forecasts, there is still a lack of transparent and integrated methods for selecting the most suitable forecast products, variables, and lead times for specific operational challenges. In this work, we propose a holistic approach based on Reinforcement Learning (RL) to design multipurpose dam operating policies informed by available multi-timescale forecast products. Our approach extends the traditional Evolutionary Multi-Objective Direct Policy Search method by parametrizing both the operating policy and the forecast information extraction process. We compare our RL approach with a state-of-the-art two-step procedure in which the forecast selection and processing are performed before the policy optimization. We demonstrate the value of the method for the multipurpose operation of Lake Como (Italy) by considering multi-timescale forecasts from short to seasonal lead times to manage flood- and drought-related operational objectives. Our approach identifies solutions achieving an 18% improvement in hypervolume indicator compared to policies not informed by forecasts and a 6% improvement over those designed using the two-step reference methodology. These improvements are accompanied by increased flexibility in policy design and trade-off analysis by directly extracting forecast information within the multi-objective optimization. This study demonstrates the feasibility and benefits of integrating policy design with forecast information extraction, particularly when multiple operational forecasts are available.
{"title":"Reinforcement Learning of Multi-Timescale Forecast Information for Designing Operating Policies of Multi-Purpose Reservoirs","authors":"D. Zanutto, A. Ficchì, M. Giuliani, A. Castelletti","doi":"10.1029/2023wr036724","DOIUrl":"https://doi.org/10.1029/2023wr036724","url":null,"abstract":"Hydrological forecasts have significantly improved in skill over recent years, encouraging their systematic exploitation in multipurpose reservoir operations to improve reliability and resilience to extreme events. Despite the growing availability of multi-timescale forecasts, there is still a lack of transparent and integrated methods for selecting the most suitable forecast products, variables, and lead times for specific operational challenges. In this work, we propose a holistic approach based on Reinforcement Learning (RL) to design multipurpose dam operating policies informed by available multi-timescale forecast products. Our approach extends the traditional Evolutionary Multi-Objective Direct Policy Search method by parametrizing both the operating policy and the forecast information extraction process. We compare our RL approach with a state-of-the-art two-step procedure in which the forecast selection and processing are performed before the policy optimization. We demonstrate the value of the method for the multipurpose operation of Lake Como (Italy) by considering multi-timescale forecasts from short to seasonal lead times to manage flood- and drought-related operational objectives. Our approach identifies solutions achieving an 18% improvement in hypervolume indicator compared to policies not informed by forecasts and a 6% improvement over those designed using the two-step reference methodology. These improvements are accompanied by increased flexibility in policy design and trade-off analysis by directly extracting forecast information within the multi-objective optimization. This study demonstrates the feasibility and benefits of integrating policy design with forecast information extraction, particularly when multiple operational forecasts are available.","PeriodicalId":23799,"journal":{"name":"Water Resources Research","volume":"11 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143393482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
N. Bulovic, F. Johnson, H. Lievens, T. E. Shaw, J. McPhee, S. Gascoin, M. Demuzere, N. McIntyre
Monitoring and estimating mountain snowpack mass over regional scales is still a challenge because of the inadequacy of observational networks in capturing spatiotemporal variability, and limitations in remotely sensed retrievals. Recent work using C-band synthetic aperture radar (SAR) backscatter data from the Sentinel-1 satellite mission has shown good promise for tracking mountain snow depth over specific northern hemisphere ranges, although the broader potential is still unknown. Here, we extend the new Sentinel-1 based modeling framework beyond the northern hemisphere by only utilizing globally available input data, and evaluate different model parametrization and model performance over the Chilean and Argentine Andes mountains, which contain the largest mountain snowpack in the southern hemisphere. The accuracy of Sentinel-1 snow depth estimates is evaluated against an extensive in situ network available for the region. Satellite-retrieved snow depth is found to have poorer performance across the Andes than observed for northern hemisphere mountain ranges because of greater sensitivity to evergreen forest cover and shallower snowpacks. The algorithm does offer some skill but performance is variable and site-dependent. Algorithm performance is best over regions with limited evergreen forest cover (