K. Fowler, Natasha Ballis, A. Horne, A. John, R. Nathan, M. Peel
“Bottom-up” methods are increasingly used to assess the vulnerability of water systems to climate change. Central to these methods is the climate “stress test”, where the system is subjected to various climatic changes to test for unacceptable outcomes. We present a framework for climate stress testing on a monthly timestep, suitable for systems whose dominant dynamic is seasonal or longer (eg. water resource systems with carry-over storage). The framework integrates multi-site stochastic climate generation with perturbation methods and in-built rainfall runoff modelling. The stochastic generation includes a low frequency component suitable for representing multi-annual fluctuations. Multiple perturbation options are provided, ranging from simple delta change through to altered seasonality and low frequency dynamics. The framework runs rapidly, supporting comprehensive multi-dimensional stress testing without recourse to supercomputing facilities. We demonstrate the framework on a large water resource system in southern Australia. The Matlab/Octave framework is freely available for download from https://doi.org/10.5281/zenodo.5617008.
{"title":"Integrated framework for rapid climate stress testing on a monthly timestep","authors":"K. Fowler, Natasha Ballis, A. Horne, A. John, R. Nathan, M. Peel","doi":"10.31223/x5vw4n","DOIUrl":"https://doi.org/10.31223/x5vw4n","url":null,"abstract":"“Bottom-up” methods are increasingly used to assess the vulnerability of water systems to climate change. Central to these methods is the climate “stress test”, where the system is subjected to various climatic changes to test for unacceptable outcomes. We present a framework for climate stress testing on a monthly timestep, suitable for systems whose dominant dynamic is seasonal or longer (eg. water resource systems with carry-over storage). The framework integrates multi-site stochastic climate generation with perturbation methods and in-built rainfall runoff modelling. The stochastic generation includes a low frequency component suitable for representing multi-annual fluctuations. Multiple perturbation options are provided, ranging from simple delta change through to altered seasonality and low frequency dynamics. The framework runs rapidly, supporting comprehensive multi-dimensional stress testing without recourse to supercomputing facilities. We demonstrate the framework on a large water resource system in southern Australia. The Matlab/Octave framework is freely available for download from https://doi.org/10.5281/zenodo.5617008.","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"22 1","pages":"105339"},"PeriodicalIF":0.0,"publicationDate":"2021-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86222450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sensors measuring environmental phenomena at high frequency commonly report anomalies related to fouling, sensor drift and calibration, and datalogging and transmission issues. Suitability of data for analyses and decision making often depends on manual review and adjustment of data. Machine learning techniques have potential to automate identification and correction of anomalies, streamlining the quality control process. We explored approaches for automating anomaly detection and correction of aquatic sensor data for implementation in a Python package (PyHydroQC). We applied both classical and deep learning time series regression models that estimate values, identify anomalies based on dynamic thresholds, and offer correction estimates. Techniques were developed and performance assessed using data reviewed, corrected, and labeled by technicians in an aquatic monitoring use case. Auto-Regressive Integrated Moving Average (ARIMA) consistently performed best, and aggregating results from multiple models improved detection. PyHydroQC includes custom functions and a workflow for anomaly detection and correction.
{"title":"Toward automating post processing of aquatic sensor data","authors":"A. Jones, T. Jones, J. Horsburgh","doi":"10.31223/x5z62x","DOIUrl":"https://doi.org/10.31223/x5z62x","url":null,"abstract":"Sensors measuring environmental phenomena at high frequency commonly report anomalies related to fouling, sensor drift and calibration, and datalogging and transmission issues. Suitability of data for analyses and decision making often depends on manual review and adjustment of data. Machine learning techniques have potential to automate identification and correction of anomalies, streamlining the quality control process. We explored approaches for automating anomaly detection and correction of aquatic sensor data for implementation in a Python package (PyHydroQC). We applied both classical and deep learning time series regression models that estimate values, identify anomalies based on dynamic thresholds, and offer correction estimates. Techniques were developed and performance assessed using data reviewed, corrected, and labeled by technicians in an aquatic monitoring use case. Auto-Regressive Integrated Moving Average (ARIMA) consistently performed best, and aggregating results from multiple models improved detection. PyHydroQC includes custom functions and a workflow for anomaly detection and correction.","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"18 1","pages":"105364"},"PeriodicalIF":0.0,"publicationDate":"2021-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79901738","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we present the interactive Reservoir Operations Notebooks and Software (iRONS) toolbox for reservoir modelling and optimisation. The toolbox is meant to serve the research and professional community in hydrology and water resource management and contribute to bridge the gaps between them. iRONS is composed of a package of Python core functions and a set of interactive Jupyter Notebooks. Core functions implement typical reservoir modelling tasks and the interactive Jupyter Notebooks illustrate, with practical examples, the key functionalities of iRONS. We describe our development philosophy, the key features of iRONS, and report some results of evaluating the effectiveness of interactive Jupyter Notebooks for training and knowledge transfer. The paper may be of interest also beyond the water resources management field, as an example of how Jupyter Notebooks and interactive visualisation help improving the documentation and sharing of open-source code and the communication of underpinning methodologies.
{"title":"An open-source package with interactive Jupyter Notebooks to enhance the accessibility of reservoir operations simulation and optimisation","authors":"Andrés Peñuela, C. Hutton, F. Pianosi","doi":"10.31223/x58p7f","DOIUrl":"https://doi.org/10.31223/x58p7f","url":null,"abstract":"In this paper we present the interactive Reservoir Operations Notebooks and Software (iRONS) toolbox for reservoir modelling and optimisation. The toolbox is meant to serve the research and professional community in hydrology and water resource management and contribute to bridge the gaps between them. iRONS is composed of a package of Python core functions and a set of interactive Jupyter Notebooks. Core functions implement typical reservoir modelling tasks and the interactive Jupyter Notebooks illustrate, with practical examples, the key functionalities of iRONS. We describe our development philosophy, the key features of iRONS, and report some results of evaluating the effectiveness of interactive Jupyter Notebooks for training and knowledge transfer. The paper may be of interest also beyond the water resources management field, as an example of how Jupyter Notebooks and interactive visualisation help improving the documentation and sharing of open-source code and the communication of underpinning methodologies.","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"9 1","pages":"105188"},"PeriodicalIF":0.0,"publicationDate":"2021-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81938115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Carlos Erazo Ramirez, Y. Sermet, F. Molkenthin, I. Demir
This paper presents HydroLang, an open-source and integrated community-driven computational web framework to support research and education in hydrology and water resources. HydroLang uses client-side web technologies and standards to perform different routines which aim towards the acquisition, management, transformation, analysis and visualization of hydrological datasets. HydroLang is comprised of four main high-cohesion low-coupling modules for: (1) retrieving, manipulating, and transforming raw hydrological data, (2) statistical operations, hydrological analysis, and creating models, (3) generating graphical and tabular data representations, and (4) mapping and geospatial data visualization. Two extensive case studies (i.e., evaluation of lumped models and development of a rainfall disaggregation model) have been presented to demonstrate the framework’s capabilities, portability, and interoperability. HydroLang’s unique modular architecture and open-source nature allow it to be easily tailored into any use case and web framework and promote iterative enhancements with community involvement to establish the comprehensive next-generation hydrological software toolkit.
{"title":"HydroLang: An open-source web-based programming framework for hydrological sciences","authors":"Carlos Erazo Ramirez, Y. Sermet, F. Molkenthin, I. Demir","doi":"10.31223/x5m31d","DOIUrl":"https://doi.org/10.31223/x5m31d","url":null,"abstract":"This paper presents HydroLang, an open-source and integrated community-driven computational web framework to support research and education in hydrology and water resources. HydroLang uses client-side web technologies and standards to perform different routines which aim towards the acquisition, management, transformation, analysis and visualization of hydrological datasets. HydroLang is comprised of four main high-cohesion low-coupling modules for: (1) retrieving, manipulating, and transforming raw hydrological data, (2) statistical operations, hydrological analysis, and creating models, (3) generating graphical and tabular data representations, and (4) mapping and geospatial data visualization. Two extensive case studies (i.e., evaluation of lumped models and development of a rainfall disaggregation model) have been presented to demonstrate the framework’s capabilities, portability, and interoperability. HydroLang’s unique modular architecture and open-source nature allow it to be easily tailored into any use case and web framework and promote iterative enhancements with community involvement to establish the comprehensive next-generation hydrological software toolkit.","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"1 1","pages":"105525"},"PeriodicalIF":0.0,"publicationDate":"2021-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83310466","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-06-30DOI: 10.1002/ESSOAR.10507438.1
L. J. Halloran
Time-lapse gravimetry is a powerful tool for monitoring temporal mass distribution variations, including seasonal and long-term groundwater storage changes (GWSC). This geophysical method for measu...
{"title":"Improving groundwater storage change estimates using time-lapse gravimetry with Gravi4GW","authors":"L. J. Halloran","doi":"10.1002/ESSOAR.10507438.1","DOIUrl":"https://doi.org/10.1002/ESSOAR.10507438.1","url":null,"abstract":"Time-lapse gravimetry is a powerful tool for monitoring temporal mass distribution variations, including seasonal and long-term groundwater storage changes (GWSC). This geophysical method for measu...","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"83 1","pages":"105340"},"PeriodicalIF":0.0,"publicationDate":"2021-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90040544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-06-07DOI: 10.1101/2021.06.06.21258419
M. Saez, M. Barceló
Our objective in this work was to present a hierarchical Bayesian spatiotemporal model that allowed us to make spatial predictions of air pollution levels in an effective way and with very few computational costs. We specified a hierarchical spatiotemporal model, using the Stochastic Partial Differential Equations of the integrated nested Laplace approximations approximation. This approach allowed us to spatially predict, in the territory of Catalonia (Spain), the levels of the four pollutants for which there is the most evidence of an adverse health effect. Our model allowed us to make fairly accurate spatial predictions of both long-term and short-term exposure to air pollutants, with a low computational cost. The only requirements of the method we propose are the minimum number of stations distributed throughout the territory where the predictions are to be made, and that the spatial and temporal dimensions are either independent or separable.
{"title":"Spatial prediction of air pollution levels using a hierarchical Bayesian spatiotemporal model in Catalonia, Spain","authors":"M. Saez, M. Barceló","doi":"10.1101/2021.06.06.21258419","DOIUrl":"https://doi.org/10.1101/2021.06.06.21258419","url":null,"abstract":"Our objective in this work was to present a hierarchical Bayesian spatiotemporal model that allowed us to make spatial predictions of air pollution levels in an effective way and with very few computational costs. We specified a hierarchical spatiotemporal model, using the Stochastic Partial Differential Equations of the integrated nested Laplace approximations approximation. This approach allowed us to spatially predict, in the territory of Catalonia (Spain), the levels of the four pollutants for which there is the most evidence of an adverse health effect. Our model allowed us to make fairly accurate spatial predictions of both long-term and short-term exposure to air pollutants, with a low computational cost. The only requirements of the method we propose are the minimum number of stations distributed throughout the territory where the predictions are to be made, and that the spatial and temporal dimensions are either independent or separable.","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"1 1","pages":"105369"},"PeriodicalIF":0.0,"publicationDate":"2021-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84918539","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-03-03DOI: 10.5194/EGUSPHERE-EGU21-6031
Camilo J. Bastidas Pacheco, Joseph C. Brewer, J. Horsburgh, J. Caraballo
Collecting and managing high temporal resolution (< 1 minute) residential water use data is challenging due to cost and technical requirements associated with the volume and velocity of data collected. It is well known that this type of data has potential to expand our knowledge of residential water use, inform future water use predictions, and improve water conservation strategies. However, most studies collecting this type of data have been focused on the practical application of the data (e.g., developing and applying end use disaggregation algorithms) with much less focus on how the data were collected, retrieved, quality controlled, and managed to enable data visualization and analysis. We developed an open-source, modular, generalized cyberinfrastructure system to automate the process from data collection to analysis. The system has three main architectural components: first, the sensors and dataloggers for water use monitoring; second, the data communication, parsing and archival tools; and third, the analyses, visualization and presentations of data produced for different audiences. For the first component, we present a low-cost datalogging device, designed for installation on top of existing, analog, magnetically driven, positive displacement, residential water meters that can collect data at a user configurable time resolution interval. The second component consists of a system developed using existing open-source software technologies that manages the data collected, including services and databasing. The final element includes software tools for retrieving the data that can be integrated with advanced data analytics tools. The system was used in a single family residential water use data collection case study to test the scalability and performance of its functionalities within our design constraints. Testing with a base system configuration, our results show that the system requires approximately six minutes to process a single day of data collected at a four second temporal resolution for 500 properties. Thus, the system proved to be effective beyond the typical number of participants observed in similar studies of residential water use and would scale well beyond this even with the modest system resources we used for testing. All elements of the cyberinfrastructure developed are freely available in open source repositories for re-use.
{"title":"An open source cyberinfrastructure for collecting, processing, storing and accessing high temporal resolution residential water use data","authors":"Camilo J. Bastidas Pacheco, Joseph C. Brewer, J. Horsburgh, J. Caraballo","doi":"10.5194/EGUSPHERE-EGU21-6031","DOIUrl":"https://doi.org/10.5194/EGUSPHERE-EGU21-6031","url":null,"abstract":"<p>Collecting and managing high temporal resolution (< 1 minute) residential water use data is challenging due to cost and technical requirements associated with the volume and velocity of data collected. It is well known that this type of data has potential to expand our knowledge of residential water use, inform future water use predictions, and improve water conservation strategies. However, most studies collecting this type of data have been focused on the practical application of the data (e.g., developing and applying end use disaggregation algorithms) with much less focus on how the data were collected, retrieved, quality controlled, and managed to enable data visualization and analysis. We developed an open-source, modular, generalized cyberinfrastructure system to automate the process from data collection to analysis. The system has three main architectural components: first, the sensors and dataloggers for water use monitoring; second, the data communication, parsing and archival tools; and third, the analyses, visualization and presentations of data produced for different audiences. For the first component, we present a low-cost datalogging device, designed for installation on top of existing, analog, magnetically driven, positive displacement, residential water meters that can collect data at a user configurable time resolution interval. The second component consists of a system developed using existing open-source software technologies that manages the data collected, including services and databasing. The final element includes software tools for retrieving the data that can be integrated with advanced data analytics tools. The system was used in a single family residential water use data collection case study to test the scalability and performance of its functionalities within our design constraints. Testing with a base system configuration, our results show that the system requires approximately six minutes to process a single day of data collected at a four second temporal resolution for 500 properties. Thus, the system proved to be effective beyond the typical number of participants observed in similar studies of residential water use and would scale well beyond this even with the modest system resources we used for testing. All elements of the cyberinfrastructure developed are freely available in open source repositories for re-use.</p>","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"28 1","pages":"105137"},"PeriodicalIF":0.0,"publicationDate":"2021-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86039955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-02-25DOI: 10.1016/j.envsoft.2021.105102
D. Vanzo, Samuel J. Peter, L. Vonwiller, Matthias Buergler, Manuel Weberndorfer, A. Siviglia, D. Conde, D. Vetsch
{"title":"basement v3: A modular freeware for river process modelling over multiple computational backends","authors":"D. Vanzo, Samuel J. Peter, L. Vonwiller, Matthias Buergler, Manuel Weberndorfer, A. Siviglia, D. Conde, D. Vetsch","doi":"10.1016/j.envsoft.2021.105102","DOIUrl":"https://doi.org/10.1016/j.envsoft.2021.105102","url":null,"abstract":"","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"27 1","pages":"105102"},"PeriodicalIF":0.0,"publicationDate":"2021-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88958996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-01-25DOI: 10.1002/ESSOAR.10506045.1
S. Razavi
Recent breakthroughs in artificial intelligence (AI), and particularly in deep learning (DL), have created tremendous excitement and opportunities in the earth and environmental sciences communitie...
最近人工智能(AI)，特别是深度学习(DL)的突破，在地球和环境科学界创造了巨大的兴奋和机会……
{"title":"Deep learning, explained: Fundamentals, explainability, and bridgeability to process-based modelling","authors":"S. Razavi","doi":"10.1002/ESSOAR.10506045.1","DOIUrl":"https://doi.org/10.1002/ESSOAR.10506045.1","url":null,"abstract":"Recent breakthroughs in artificial intelligence (AI), and particularly in deep learning (DL), have created tremendous excitement and opportunities in the earth and environmental sciences communitie...","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"1 1","pages":"105159"},"PeriodicalIF":0.0,"publicationDate":"2021-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73478388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-01-20DOI: 10.1016/j.envsoft.2021.105115
Marouane Il Idrissi, V. Chabridon, B. Iooss
{"title":"Developments and applications of Shapley effects to reliability-oriented sensitivity analysis with correlated inputs","authors":"Marouane Il Idrissi, V. Chabridon, B. Iooss","doi":"10.1016/j.envsoft.2021.105115","DOIUrl":"https://doi.org/10.1016/j.envsoft.2021.105115","url":null,"abstract":"","PeriodicalId":12033,"journal":{"name":"Environ. Model. Softw.","volume":"20 5 1","pages":"105115"},"PeriodicalIF":0.0,"publicationDate":"2021-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73236691","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：