Pub Date : 2024-03-13DOI: 10.1175/aies-d-23-0052.1
Jeong-Hwan Kim, Y. Ham, Daehyun Kim, Tim Li, Chen Ma
Forecasting the intensity of a tropical cyclone (TC) remains challenging, particularly when it undergoes rapid changes in intensity. This study aims to develop a Convolutional Neural Network (CNN) for 24-hour forecasts of the TC intensity changes and their rapid intensifications over the western Pacific. The CNN model, the DeepTC, is trained using a unique loss function - an amplitude focal loss, to better capture large intensity changes, such as those during rapid intensification (RI) events. We showed that the DeepTC outperforms operational forecasts, with a lower mean absolute error (8.9-10.2%) and a higher coefficient of determination (31.7-35%). In addition, the DeepTC exhibits a substantially better skill at capturing RI events than operational forecasts. To understand the superior performance of the DeepTC in RI forecasts, we conduct an occlusion sensitivity analysis to quantify the relative importance of each predictor. Results revealed that scalar quantities such as latitude, previous intensity change, initial intensity, and vertical wind shear play critical roles in successful RI prediction. Additionally, DeepTC utilizes the three-dimensional distribution of relative humidity to distinguish RI cases from non-RI cases, with higher dry-moist moisture gradients in the mid-to-low troposphere and steeper radial moisture gradients in the upper troposphere showed during RI events. These relationship between the identified key variables and intensity change was successfully simulated by the DeepTC, implying that the relationship is physically reasonable. Our study demonstrates that the DeepTC can be a powerful tool for improving RI understanding and enhancing the reliability of TC intensity forecasts.
预测热带气旋（TC）的强度仍然具有挑战性，尤其是当其强度发生快速变化时。本研究旨在开发一种卷积神经网络（CNN），用于 24 小时预报热带气旋强度变化及其在西太平洋上空的快速增强。该卷积神经网络模型（DeepTC）使用独特的损失函数--振幅焦点损失进行训练，以更好地捕捉大规模强度变化，如快速增强（RI）事件期间的强度变化。我们的研究表明，DeepTC 的性能优于业务预报，平均绝对误差更低（8.9%-10.2%），决定系数更高（31.7%-35%）。为了了解 DeepTC 在 RI 预测中的卓越表现，我们进行了闭塞敏感性分析，以量化每个预测因子的相对重要性。结果显示，纬度、先前强度变化、初始强度和垂直风切变等标量对成功预测 RI 起着至关重要的作用。此外，DeepTC 还利用相对湿度的三维分布来区分 RI 和非 RI，在 RI 事件中，对流层中低层的干湿度梯度更高，对流层高层的径向湿度梯度更陡。我们的研究表明，DeepTC 可以作为一个强大的工具，用于提高对 RI 的理解和 TC 强度预报的可靠性。
{"title":"Improvement in forecasting short-term tropical cyclone intensity change and their rapid intensification using deep learning","authors":"Jeong-Hwan Kim, Y. Ham, Daehyun Kim, Tim Li, Chen Ma","doi":"10.1175/aies-d-23-0052.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0052.1","url":null,"abstract":"\u0000Forecasting the intensity of a tropical cyclone (TC) remains challenging, particularly when it undergoes rapid changes in intensity. This study aims to develop a Convolutional Neural Network (CNN) for 24-hour forecasts of the TC intensity changes and their rapid intensifications over the western Pacific. The CNN model, the DeepTC, is trained using a unique loss function - an amplitude focal loss, to better capture large intensity changes, such as those during rapid intensification (RI) events. We showed that the DeepTC outperforms operational forecasts, with a lower mean absolute error (8.9-10.2%) and a higher coefficient of determination (31.7-35%). In addition, the DeepTC exhibits a substantially better skill at capturing RI events than operational forecasts.\u0000To understand the superior performance of the DeepTC in RI forecasts, we conduct an occlusion sensitivity analysis to quantify the relative importance of each predictor. Results revealed that scalar quantities such as latitude, previous intensity change, initial intensity, and vertical wind shear play critical roles in successful RI prediction. Additionally, DeepTC utilizes the three-dimensional distribution of relative humidity to distinguish RI cases from non-RI cases, with higher dry-moist moisture gradients in the mid-to-low troposphere and steeper radial moisture gradients in the upper troposphere showed during RI events.\u0000These relationship between the identified key variables and intensity change was successfully simulated by the DeepTC, implying that the relationship is physically reasonable. Our study demonstrates that the DeepTC can be a powerful tool for improving RI understanding and enhancing the reliability of TC intensity forecasts.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"2006 5","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140246451","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-07DOI: 10.1175/aies-d-23-0066.1
Neelesh Rampal, S. Hobeichi, Peter B. Gibson, Jorge Baño-Medina, G. Abramowitz, Tom Beucler, Jose González-Abad, William Chapman, Paula Harder, José Manuel Gutiérrez
Despite the sophistication of Global Climate Models (GCMs), their coarse spatial resolution limits their ability to resolve important aspects of climate variability and change at the local scale. Both dynamical and empirical methods are used for enhancing the resolution of climate projections through downscaling, each with distinct advantages and challenges. Dynamical downscaling is physics-based but comes with a large computational cost, posing a barrier for downscaling an ensemble of GCMs large enough for reliable uncertainty quantification of climate risks. In contrast, empirical downscaling, which encompasses statistical and machine learning techniques, provides a computationally efficient alternative to downscaling GCMs. Empirical downscaling algorithms can be developed to emulate the behaviour of dynamical models directly, or through frameworks such as perfect prognosis in which relationships are established between large-scale atmospheric conditions and local weather variables using observational data. However, the ability of empirical downscaling algorithms to apply their learnt relationships out-of-distribution into future climates remains uncertain, as is their ability to represent certain types of extreme events. This review covers the growing potential of machine learning methods to address these challenges, offering a thorough exploration of the current applications, and training strategies that can circumvent certain issues. Additionally, we propose an evaluation framework for machine learning algorithms specific to the problem of climate downscaling, as needed to improve transparency and foster trust in climate projections.
{"title":"Enhancing Regional Climate Downscaling Through Advances in Machine Learning","authors":"Neelesh Rampal, S. Hobeichi, Peter B. Gibson, Jorge Baño-Medina, G. Abramowitz, Tom Beucler, Jose González-Abad, William Chapman, Paula Harder, José Manuel Gutiérrez","doi":"10.1175/aies-d-23-0066.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0066.1","url":null,"abstract":"\u0000Despite the sophistication of Global Climate Models (GCMs), their coarse spatial resolution limits their ability to resolve important aspects of climate variability and change at the local scale. Both dynamical and empirical methods are used for enhancing the resolution of climate projections through downscaling, each with distinct advantages and challenges. Dynamical downscaling is physics-based but comes with a large computational cost, posing a barrier for downscaling an ensemble of GCMs large enough for reliable uncertainty quantification of climate risks. In contrast, empirical downscaling, which encompasses statistical and machine learning techniques, provides a computationally efficient alternative to downscaling GCMs. Empirical downscaling algorithms can be developed to emulate the behaviour of dynamical models directly, or through frameworks such as perfect prognosis in which relationships are established between large-scale atmospheric conditions and local weather variables using observational data. However, the ability of empirical downscaling algorithms to apply their learnt relationships out-of-distribution into future climates remains uncertain, as is their ability to represent certain types of extreme events. This review covers the growing potential of machine learning methods to address these challenges, offering a thorough exploration of the current applications, and training strategies that can circumvent certain issues. Additionally, we propose an evaluation framework for machine learning algorithms specific to the problem of climate downscaling, as needed to improve transparency and foster trust in climate projections.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"26 12","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140259700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-05DOI: 10.1175/aies-d-23-0104.1
Harold E. Brooks, Montgomery L. Flora, Michael E. Baldwin
Forecast evaluation metrics have been discovered and rediscovered in a variety of contexts, leading to confusion. We look at measures from the 2x2 contingency table and the history of their development and illustrate how different fields working on similar problems has led to different approaches and perspectives of the same mathematical concepts. For example, Probability of Detection is a quantity in meteorology that was also called Prefigurance in the field, while the same thing is named Recall in information science and machine learning, and Sensitivity and True Positive Rate in the medical literature. Many of the scores that combine three elements of the 2x2 table can be seen as either coming from a perspective of Venn diagrams or from the Pythagorean means, possibly weighted, of two ratios of performance measures. Although there are algebraic relationships between the two perspectives, the approaches taken by authors led them in different directions, making it unlikely that they would discover scores that naturally arose from the other approach. We close by discussing the importance of understanding the implicit or explicit values expressed by the choice of scores. In addition, we make some simple recommendations about the appropriate nomenclature to use when publishing interdisciplinary work.
{"title":"A rose by any other name: On basic scores from the 2x2 table and the plethora of names attached to them","authors":"Harold E. Brooks, Montgomery L. Flora, Michael E. Baldwin","doi":"10.1175/aies-d-23-0104.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0104.1","url":null,"abstract":"\u0000Forecast evaluation metrics have been discovered and rediscovered in a variety of contexts, leading to confusion. We look at measures from the 2x2 contingency table and the history of their development and illustrate how different fields working on similar problems has led to different approaches and perspectives of the same mathematical concepts. For example, Probability of Detection is a quantity in meteorology that was also called Prefigurance in the field, while the same thing is named Recall in information science and machine learning, and Sensitivity and True Positive Rate in the medical literature. Many of the scores that combine three elements of the 2x2 table can be seen as either coming from a perspective of Venn diagrams or from the Pythagorean means, possibly weighted, of two ratios of performance measures. Although there are algebraic relationships between the two perspectives, the approaches taken by authors led them in different directions, making it unlikely that they would discover scores that naturally arose from the other approach.\u0000We close by discussing the importance of understanding the implicit or explicit values expressed by the choice of scores. In addition, we make some simple recommendations about the appropriate nomenclature to use when publishing interdisciplinary work.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"3 5","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140263872","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-21DOI: 10.1175/aies-d-23-0055.1
Mahsa Payami, Yunsoo Choi, A. K. Salman, Seyedali Mousavinezhad, Jincheol Park, A. Pouyaei
In this study, we developed an emulator of the Community Multiscale Air Quality (CMAQ) model by employing a 1-dimensional Convolutional Neural Network (CNN) algorithm to predict hourly surface nitrogen dioxide (NO2) concentrations over the most densely populated urban regions in Texas. The inputs for the emulator were the same as those for the CMAQ model, which includes emission, meteorology, and land use land cover data. We trained the model over June, July, and August (JJA) of 2011 and 2014 and then tested it on JJA of 2017, achieving an Index of Agreement (IOA) of 0.95 and a correlation of 0.90. We also employed temporal 3-fold cross-validation to evaluate the model’s performance, ensuring the robustness and generalizability of the results. To gain deeper insights and understand the factors influencing the model’s surface NO2 predictions, we conducted a Shapley Additive Explanations analysis. The results revealed solar radiation reaching the surface, Planetary Boundary Layer height, and NOx (NO + NO2) emissions are key variables driving the model’s predictions. These findings highlight the emulator’s ability to capture the individual impact of each variable on the model’s NO2 predictions. Furthermore, our emulator outperformed the CMAQ model in terms of computational efficiency, being more than 900 times faster in predicting NO2 concentrations, enabling the rapid assessment of various pollution management scenarios. This work offers a valuable resource for air pollution mitigation efforts, not just in Texas, but with appropriate regional data training, its utility could be extended to other regions and pollutants as well.
{"title":"A 1D CNN-based emulator of CMAQ: Predicting NO2 concentration over the most populated urban regions in Texas","authors":"Mahsa Payami, Yunsoo Choi, A. K. Salman, Seyedali Mousavinezhad, Jincheol Park, A. Pouyaei","doi":"10.1175/aies-d-23-0055.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0055.1","url":null,"abstract":"\u0000In this study, we developed an emulator of the Community Multiscale Air Quality (CMAQ) model by employing a 1-dimensional Convolutional Neural Network (CNN) algorithm to predict hourly surface nitrogen dioxide (NO2) concentrations over the most densely populated urban regions in Texas. The inputs for the emulator were the same as those for the CMAQ model, which includes emission, meteorology, and land use land cover data. We trained the model over June, July, and August (JJA) of 2011 and 2014 and then tested it on JJA of 2017, achieving an Index of Agreement (IOA) of 0.95 and a correlation of 0.90. We also employed temporal 3-fold cross-validation to evaluate the model’s performance, ensuring the robustness and generalizability of the results. To gain deeper insights and understand the factors influencing the model’s surface NO2 predictions, we conducted a Shapley Additive Explanations analysis. The results revealed solar radiation reaching the surface, Planetary Boundary Layer height, and NOx (NO + NO2) emissions are key variables driving the model’s predictions. These findings highlight the emulator’s ability to capture the individual impact of each variable on the model’s NO2 predictions. Furthermore, our emulator outperformed the CMAQ model in terms of computational efficiency, being more than 900 times faster in predicting NO2 concentrations, enabling the rapid assessment of various pollution management scenarios. This work offers a valuable resource for air pollution mitigation efforts, not just in Texas, but with appropriate regional data training, its utility could be extended to other regions and pollutants as well.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"3 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140442593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-29DOI: 10.1175/aies-d-23-0069.1
Junsu Kim, Yeon-Hee Kim, Hyejeong Bok, Sungbin Jang, Eunju Cho, Seung-Bum Kim
We developed an advanced post-processing model for precipitation forecasting using a micro-genetic algorithm (MGA). The algorithm determines the optimal combination of three general circulation models: the Korean Integrated Model, the Unified Model, and the Integrated Forecast System model. To measure model accuracy, including the critical success index (CSI), probability of detection (POD), and frequency bias index, the MGA calculates optimal weights for individual models based on a fitness function that considers various indices. Our optimized multi-model yielded up to 13% and 10% improvement in CSI and POD performance compared to each individual model, respectively. Notably, when applied to an operational definition that considers precipitation thresholds from three models and averages the precipitation amount from the satisfactory models, our optimized multi-model outperformed the current operational model used by the Korea Meteorological Administration by up to 1.0% and 6.8% in terms of CSI and false alarm ratio performance, respectively. This study highlights the effectiveness of a weighted combination of global models to enhance the forecasting accuracy for regional precipitation. By utilizing the MGA for the fine-tuning of model weights, we achieved superior precipitation prediction compared to that of individual models and existing standard post-processing operations. This approach can significantly improve the accuracy of precipitation forecasts.
我们利用微遗传算法（MGA）开发了一种先进的降水预报后处理模型。该算法确定了三种大气环流模式的最佳组合：韩国综合模式、统一模式和综合预报系统模式。为了衡量模式的准确性，包括关键成功指数（CSI）、检测概率（POD）和频率偏差指数，MGA 根据考虑了各种指数的适应度函数计算各个模式的最佳权重。与单个模型相比，我们的优化多模型在 CSI 和 POD 性能方面分别提高了 13% 和 10%。值得注意的是，当应用于考虑了三个模型的降水阈值并对满意模型的降水量进行平均的业务定义时，我们的优化多模型在 CSI 和误报率性能方面分别比韩国气象局目前使用的业务模型高出 1.0% 和 6.8%。这项研究强调了全球模式加权组合在提高区域降水预报精度方面的有效性。通过利用 MGA 对模型权重进行微调，我们实现了比单个模型和现有标准后处理操作更优越的降水预测。这种方法可以大大提高降水预报的准确性。
{"title":"Development of an Optimal Post-Processing Model Using the Micro Genetic Algorithm to Improve Precipitation Forecasting in Korea","authors":"Junsu Kim, Yeon-Hee Kim, Hyejeong Bok, Sungbin Jang, Eunju Cho, Seung-Bum Kim","doi":"10.1175/aies-d-23-0069.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0069.1","url":null,"abstract":"\u0000We developed an advanced post-processing model for precipitation forecasting using a micro-genetic algorithm (MGA). The algorithm determines the optimal combination of three general circulation models: the Korean Integrated Model, the Unified Model, and the Integrated Forecast System model. To measure model accuracy, including the critical success index (CSI), probability of detection (POD), and frequency bias index, the MGA calculates optimal weights for individual models based on a fitness function that considers various indices. Our optimized multi-model yielded up to 13% and 10% improvement in CSI and POD performance compared to each individual model, respectively. Notably, when applied to an operational definition that considers precipitation thresholds from three models and averages the precipitation amount from the satisfactory models, our optimized multi-model outperformed the current operational model used by the Korea Meteorological Administration by up to 1.0% and 6.8% in terms of CSI and false alarm ratio performance, respectively. This study highlights the effectiveness of a weighted combination of global models to enhance the forecasting accuracy for regional precipitation. By utilizing the MGA for the fine-tuning of model weights, we achieved superior precipitation prediction compared to that of individual models and existing standard post-processing operations. This approach can significantly improve the accuracy of precipitation forecasts.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"76 26","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140486116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-01DOI: 10.1175/aies-d-23-0064.1
Alexander J. DesRosiers, Michael M. Bell
Airborne Doppler radar provides detailed and targeted observations of winds and precipitation in weather systems over remote or difficult-to-access regions that can help to improve scientific understanding and weather forecasts. Quality control (QC) is necessary to remove nonweather echoes from raw radar data for subsequent analysis. The complex decision-making ability of the machine learning random-forest technique is employed to create a generalized QC method for airborne radar data in convective weather systems. A manually QCed dataset was used to train the model containing data from the Electra Doppler Radar (ELDORA) in mature and developing tropical cyclones, a tornadic supercell, and a bow echo. Successful classification of ∼96% and ∼93% of weather and nonweather radar gates, respectively, in withheld testing data indicate the generalizability of the method. Dual-Doppler analysis from the genesis phase of Hurricane Ophelia (2005) using data not previously seen by the model produced a comparable wind field to that from manual QC. The framework demonstrates a proof of concept that can be applied to newer airborne Doppler radars. Airborne Doppler radar is an invaluable tool for making detailed measurements of wind and precipitation in weather systems over remote or difficult to access regions, such as hurricanes over the ocean. Using the collected radar data depends strongly on quality control (QC) procedures to classify weather and nonweather radar echoes and to then remove the latter before subsequent analysis or assimilation into numerical weather prediction models. Prior QC techniques require interactive editing and subjective classification by trained researchers and can demand considerable time for even small amounts of data. We present a new machine learning algorithm that is trained on past QC efforts from radar experts, resulting in an accurate, fast technique with far less user input required that can greatly reduce the time required for QC. The new technique is based on the random forest, which is a machine learning model composed of decision trees, to classify weather and nonweather radar echoes. Continued efforts to build on this technique could benefit future weather forecasts by quickly and accurately quality-controlling data from other airborne radars for research or operational meteorology.
{"title":"Airborne Radar Quality Control with Machine Learning","authors":"Alexander J. DesRosiers, Michael M. Bell","doi":"10.1175/aies-d-23-0064.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0064.1","url":null,"abstract":"\u0000Airborne Doppler radar provides detailed and targeted observations of winds and precipitation in weather systems over remote or difficult-to-access regions that can help to improve scientific understanding and weather forecasts. Quality control (QC) is necessary to remove nonweather echoes from raw radar data for subsequent analysis. The complex decision-making ability of the machine learning random-forest technique is employed to create a generalized QC method for airborne radar data in convective weather systems. A manually QCed dataset was used to train the model containing data from the Electra Doppler Radar (ELDORA) in mature and developing tropical cyclones, a tornadic supercell, and a bow echo. Successful classification of ∼96% and ∼93% of weather and nonweather radar gates, respectively, in withheld testing data indicate the generalizability of the method. Dual-Doppler analysis from the genesis phase of Hurricane Ophelia (2005) using data not previously seen by the model produced a comparable wind field to that from manual QC. The framework demonstrates a proof of concept that can be applied to newer airborne Doppler radars.\u0000\u0000\u0000Airborne Doppler radar is an invaluable tool for making detailed measurements of wind and precipitation in weather systems over remote or difficult to access regions, such as hurricanes over the ocean. Using the collected radar data depends strongly on quality control (QC) procedures to classify weather and nonweather radar echoes and to then remove the latter before subsequent analysis or assimilation into numerical weather prediction models. Prior QC techniques require interactive editing and subjective classification by trained researchers and can demand considerable time for even small amounts of data. We present a new machine learning algorithm that is trained on past QC efforts from radar experts, resulting in an accurate, fast technique with far less user input required that can greatly reduce the time required for QC. The new technique is based on the random forest, which is a machine learning model composed of decision trees, to classify weather and nonweather radar echoes. Continued efforts to build on this technique could benefit future weather forecasts by quickly and accurately quality-controlling data from other airborne radars for research or operational meteorology.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"7 2-3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140520216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-21DOI: 10.1175/aies-d-23-0045.1
Sam J. Silva, Christoph A. Keller
Explainable Artificial Intelligence (XAI) methods are becoming popular tools for scientific discovery in the Earth and atmospheric sciences. While these techniques have the potential to revolutionize the scientific process, there are known limitations in their applicability that are frequently ignored. These limitations include that XAI methods explain the behavior of the A.I. model, not the behavior of the training dataset, and that caution should be used when these methods are applied to datasets with correlated and dependent features. Here, we explore the potential cost associated with ignoring these limitations with a simple case-study from the atmospheric chemistry literature – learning the reaction rate of a bimolecular reaction. We demonstrate that dependent and highly correlated input features can lead to spurious process-level explanations. We posit that the current generation of XAI techniques should largely only be used for understanding system-level behavior and recommend caution when using XAI methods for process-level scientific discovery in the Earth and atmospheric sciences.
{"title":"Limitations of XAI methods for process-level understanding in the atmospheric sciences","authors":"Sam J. Silva, Christoph A. Keller","doi":"10.1175/aies-d-23-0045.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0045.1","url":null,"abstract":"\u0000Explainable Artificial Intelligence (XAI) methods are becoming popular tools for scientific discovery in the Earth and atmospheric sciences. While these techniques have the potential to revolutionize the scientific process, there are known limitations in their applicability that are frequently ignored. These limitations include that XAI methods explain the behavior of the A.I. model, not the behavior of the training dataset, and that caution should be used when these methods are applied to datasets with correlated and dependent features. Here, we explore the potential cost associated with ignoring these limitations with a simple case-study from the atmospheric chemistry literature – learning the reaction rate of a bimolecular reaction. We demonstrate that dependent and highly correlated input features can lead to spurious process-level explanations. We posit that the current generation of XAI techniques should largely only be used for understanding system-level behavior and recommend caution when using XAI methods for process-level scientific discovery in the Earth and atmospheric sciences.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138951095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-09DOI: 10.1175/aies-d-23-0018.1
Montgomery L. Flora, Corey K. Potvin, Amy McGovern, Shawn Handler
Abstract With increasing interest in explaining machine learning (ML) models, this paper synthesizes many topics related to ML explainability. We distinguish explainability from interpretability, local from global explainability, and feature importance versus feature relevance. We demonstrate and visualize different explanation methods, how to interpret them, and provide a complete Python package (scikit-explain) to allow future researchers and model developers to explore these explainability methods. The explainability methods include Shapley additive explanations (SHAP), Shapley additive global explanation (SAGE), and accumulated local effects (ALE). Our focus is primarily on Shapley-based techniques, which serve as a unifying framework for various existing methods to enhance model explainability. For example, SHAP unifies methods like local interpretable model-agnostic explanations (LIME) and tree interpreter for local explainability, while SAGE unifies the different variations of permutation importance for global explainability. We provide a short tutorial for explaining ML models using three disparate datasets: a convection-allowing model dataset for severe weather prediction, a nowcasting dataset for sub-freezing road surface prediction, and satellite-based data for lightning prediction. In addition, we showcase the adverse effects that correlated features can have on the explainability of a model. Finally, we demonstrate the notion of evaluating model impacts of feature groups instead of individual features. Evaluating the feature groups mitigates the impacts of feature correlations and can provide a more holistic understanding of the model. All code, models, and data used in this study are freely available to accelerate the adoption of machine learning explainability in the atmospheric and other environmental sciences.
{"title":"A Machine Learning Explainability Tutorial for Atmospheric Sciences","authors":"Montgomery L. Flora, Corey K. Potvin, Amy McGovern, Shawn Handler","doi":"10.1175/aies-d-23-0018.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0018.1","url":null,"abstract":"Abstract With increasing interest in explaining machine learning (ML) models, this paper synthesizes many topics related to ML explainability. We distinguish explainability from interpretability, local from global explainability, and feature importance versus feature relevance. We demonstrate and visualize different explanation methods, how to interpret them, and provide a complete Python package (scikit-explain) to allow future researchers and model developers to explore these explainability methods. The explainability methods include Shapley additive explanations (SHAP), Shapley additive global explanation (SAGE), and accumulated local effects (ALE). Our focus is primarily on Shapley-based techniques, which serve as a unifying framework for various existing methods to enhance model explainability. For example, SHAP unifies methods like local interpretable model-agnostic explanations (LIME) and tree interpreter for local explainability, while SAGE unifies the different variations of permutation importance for global explainability. We provide a short tutorial for explaining ML models using three disparate datasets: a convection-allowing model dataset for severe weather prediction, a nowcasting dataset for sub-freezing road surface prediction, and satellite-based data for lightning prediction. In addition, we showcase the adverse effects that correlated features can have on the explainability of a model. Finally, we demonstrate the notion of evaluating model impacts of feature groups instead of individual features. Evaluating the feature groups mitigates the impacts of feature correlations and can provide a more holistic understanding of the model. All code, models, and data used in this study are freely available to accelerate the adoption of machine learning explainability in the atmospheric and other environmental sciences.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":" 2","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135286280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-08DOI: 10.1175/aies-d-23-0048.1
Daniel Galea, Hsi-Yen Ma, Wen-Ying Wu, Daigo Kobayashi
Abstract The identification of atmospheric rivers (ARs) is crucial for weather and climate predictions as they are often associated with severe storm systems and extreme precipitation, which can cause large impacts on the society. This study presents a deep learning model, termed ARDetect, for image segmentation of ARs using ERA5 data from 1960 to 2020 with labels obtained from the TempestExtremes tracking algorithm. ARDetect is a CNN-based UNet model, with its structure having been optimized using automatic hyperparameter tuning. Inputs to ARDetect were selected to be the integrated water vapour transport (IVT) and total column water (TCW) fields, as well as the AR mask from TempestExtremes from the previous timestep to the one being considered. ARDetect achieved a mean intersection-over-union (mIoU) rate of 89.04% for ARs, indicating its high accuracy in identifying these weather patterns and a superior performance than most deep learning-based models for AR detection. In addition, ARDetect can be executed faster than the TempestExtremes method (seconds vs minutes) for the same period. This provides a significant benefit for online AR detection, especially for high-resolution global models. An ensemble of 10 models, each trained on the same dataset but having different starting weights, was used to further improve on the performance produced by ARDetect, thus demonstrating the importance of model diversity in improving performance. ARDetect provides an effective and fast deep learning-based model for researchers and weather forecasters to better detect and understand ARs, which have significant impacts on weather-related events such as floods and droughts.
{"title":"Deep Learning Image Segmentation for Atmospheric Rivers","authors":"Daniel Galea, Hsi-Yen Ma, Wen-Ying Wu, Daigo Kobayashi","doi":"10.1175/aies-d-23-0048.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0048.1","url":null,"abstract":"Abstract The identification of atmospheric rivers (ARs) is crucial for weather and climate predictions as they are often associated with severe storm systems and extreme precipitation, which can cause large impacts on the society. This study presents a deep learning model, termed ARDetect, for image segmentation of ARs using ERA5 data from 1960 to 2020 with labels obtained from the TempestExtremes tracking algorithm. ARDetect is a CNN-based UNet model, with its structure having been optimized using automatic hyperparameter tuning. Inputs to ARDetect were selected to be the integrated water vapour transport (IVT) and total column water (TCW) fields, as well as the AR mask from TempestExtremes from the previous timestep to the one being considered. ARDetect achieved a mean intersection-over-union (mIoU) rate of 89.04% for ARs, indicating its high accuracy in identifying these weather patterns and a superior performance than most deep learning-based models for AR detection. In addition, ARDetect can be executed faster than the TempestExtremes method (seconds vs minutes) for the same period. This provides a significant benefit for online AR detection, especially for high-resolution global models. An ensemble of 10 models, each trained on the same dataset but having different starting weights, was used to further improve on the performance produced by ARDetect, thus demonstrating the importance of model diversity in improving performance. ARDetect provides an effective and fast deep learning-based model for researchers and weather forecasters to better detect and understand ARs, which have significant impacts on weather-related events such as floods and droughts.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"1 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135391333","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-08DOI: 10.1175/aies-d-23-0009.1
Daniel Getter, Julie Bessac, Johann Rudi, Yan Feng
Abstract Machine learning models have been employed to perform either physics-free data-driven or hybrid dynamical downscaling of climate data. Most of these implementations operate over relatively small downscaling factors because of the challenge of recovering fine-scale information from coarse data. This limits their compatibility with many global climate model outputs, often available between ∼50–100 km resolution, to scales of interest such as cloud resolving or urban scales. This study systematically examines the capability of a type of super-resolving convolutional neural network (SR-CNNs) to downscale surface wind speed data over land surface from different coarse resolutions (25 km, 48 km, and 100 km resolution) to 3 km. For each downscaling factor, we consider three CNN configurations that generate super-resolved predictions of fine-scale wind speed, which take between 1 to 3 input fields: coarse wind speed, fine-scale topography, and diurnal cycle. In addition to fine-scale wind speeds, probability density function parameters are generated, through which sample wind speeds can be generated accounting for the intrinsic stochasticity of wind speed. For generalizability assessment, CNN models are tested on regions with different topography and climate that are unseen during training. The evaluation of super-resolved predictions focuses on subgrid-scale variability and the recovery of extremes. Models with coarse wind and fine topography as inputs exhibit the best performance compared with other model configurations, operating across the same downscaling factor. Our diurnal cycle encoding results in lower out-of-sample generalizability compared with other input configurations.
{"title":"Statistical treatment of convolutional neural network super-resolution of inland surface wind for subgrid-scale variability quantification","authors":"Daniel Getter, Julie Bessac, Johann Rudi, Yan Feng","doi":"10.1175/aies-d-23-0009.1","DOIUrl":"https://doi.org/10.1175/aies-d-23-0009.1","url":null,"abstract":"Abstract Machine learning models have been employed to perform either physics-free data-driven or hybrid dynamical downscaling of climate data. Most of these implementations operate over relatively small downscaling factors because of the challenge of recovering fine-scale information from coarse data. This limits their compatibility with many global climate model outputs, often available between ∼50–100 km resolution, to scales of interest such as cloud resolving or urban scales. This study systematically examines the capability of a type of super-resolving convolutional neural network (SR-CNNs) to downscale surface wind speed data over land surface from different coarse resolutions (25 km, 48 km, and 100 km resolution) to 3 km. For each downscaling factor, we consider three CNN configurations that generate super-resolved predictions of fine-scale wind speed, which take between 1 to 3 input fields: coarse wind speed, fine-scale topography, and diurnal cycle. In addition to fine-scale wind speeds, probability density function parameters are generated, through which sample wind speeds can be generated accounting for the intrinsic stochasticity of wind speed. For generalizability assessment, CNN models are tested on regions with different topography and climate that are unseen during training. The evaluation of super-resolved predictions focuses on subgrid-scale variability and the recovery of extremes. Models with coarse wind and fine topography as inputs exhibit the best performance compared with other model configurations, operating across the same downscaling factor. Our diurnal cycle encoding results in lower out-of-sample generalizability compared with other input configurations.","PeriodicalId":94369,"journal":{"name":"Artificial intelligence for the earth systems","volume":"5 2‐3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135341716","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：