Impact of NWP Model Configuration and Training Sample Characteristics on Random Forest-Based Day-1 Convective Outlook Guidance

Abstract

This study aims to quantify and better understand the impact of an upgrade to the configuration of an FV3 (Finite Volume cubed-sphere) LAM (Limited Area Model) convection-allowing ensemble on the skill of the RF models trained on cases before the upgrade and forecast on cases after the upgrade. Specifically, Random Forest (RF) models were used to produce probabilistic forecasts of severe weather, significant severe weather, and individual hazards of wind, hail, and tornado for the purpose of day-1 convective outlook guidance. The RF models are trained and forecast on different subsets of the available data set of forecast cases from the spring seasons of 2019 and 2021 (before the FV3 LAM upgrade) and 2022 (after the upgrade) and evaluated both quantitatively and qualitatively. It is found for most predictands that the RF models forecasting 2022 (2019/2021) cases are statistically significantly more skillful when trained on other cases from the 2022 (2019/2021) data set using a leave-one-out approach. However, within the 2019/2021 data set, training on cases from a different year than the year being forecast also leads to statistically significant degradations of skill, apparently at least in part due to the different sample climate between 2019 and 2021. For this particular NWP (Numerical Weather Prediction) model configuration change, the consistency in sample climate between training and forecast cases is at least as important as consistency in model configuration. Finally, increases in skill resulting from increasing the number of forecast cases used to train the RF levels off around 30 forecast cases.