WoFS and the Wisdom of the Crowd: The Impact of the Warn-on-Forecast System on Hourly Forecasts during the 2021 NOAA Hazardous Weather Testbed Spring Forecasting Experiment
Burkely T. Gallo, Adam J. Clark, I. Jirak, David A. Imy, Brett Roberts, Jacob Vancil, Kent Knopfmeier, P. Burke
{"title":"WoFS and the Wisdom of the Crowd: The Impact of the Warn-on-Forecast System on Hourly Forecasts during the 2021 NOAA Hazardous Weather Testbed Spring Forecasting Experiment","authors":"Burkely T. Gallo, Adam J. Clark, I. Jirak, David A. Imy, Brett Roberts, Jacob Vancil, Kent Knopfmeier, P. Burke","doi":"10.1175/waf-d-23-0033.1","DOIUrl":null,"url":null,"abstract":"\nDuring the 2021 Spring Forecasting Experiment (SFE), the usefulness of the experimental Warn-on-Forecast System (WoFS) ensemble guidance was tested with the issuance of short-term probabilistic hazard forecasts. One group of participants used the WoFS guidance, while another group did not. Individual forecasts issued by two NWS participants in each group were evaluated alongside a consensus forecast from the remaining participants. Participant forecasts of tornadoes, hail, and wind at lead times of ∼2–3 h and valid 2200–2300 UTC, 2300–0000 UTC, and 0000–0100 UTC were evaluated subjectively during the SFE by participants the day after issuance, and objectively after the SFE concluded. These forecasts exist between the watch and the warning time frame, where WoFS is anticipated to be particularly impactful.\nThe hourly probabilistic forecasts were skillful according to objective metrics like the Fractions Skill Score. While the tornado forecasts were more reliable than the other hazards, there was no clear indication of any one hazard scoring highest across all metrics. WoFS availability improved the hourly probabilistic forecasts as measured by the subjective ratings and several objective metrics, including increased POD and decreased FAR at high probability thresholds. Generally, expert forecasts performed better than consensus forecasts, though expert forecasts over-forecasted. Finally, this work explored the appropriate construction of practically perfect fields used during subjective verification, which participants frequently found to be too small and precise. Using a Gaussian smoother with σ=70 km is recommended to create hourly practically perfect fields in future experiments.","PeriodicalId":49369,"journal":{"name":"Weather and Forecasting","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2024-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Weather and Forecasting","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1175/waf-d-23-0033.1","RegionNum":3,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"METEOROLOGY & ATMOSPHERIC SCIENCES","Score":null,"Total":0}
引用次数: 0
Abstract
During the 2021 Spring Forecasting Experiment (SFE), the usefulness of the experimental Warn-on-Forecast System (WoFS) ensemble guidance was tested with the issuance of short-term probabilistic hazard forecasts. One group of participants used the WoFS guidance, while another group did not. Individual forecasts issued by two NWS participants in each group were evaluated alongside a consensus forecast from the remaining participants. Participant forecasts of tornadoes, hail, and wind at lead times of ∼2–3 h and valid 2200–2300 UTC, 2300–0000 UTC, and 0000–0100 UTC were evaluated subjectively during the SFE by participants the day after issuance, and objectively after the SFE concluded. These forecasts exist between the watch and the warning time frame, where WoFS is anticipated to be particularly impactful.
The hourly probabilistic forecasts were skillful according to objective metrics like the Fractions Skill Score. While the tornado forecasts were more reliable than the other hazards, there was no clear indication of any one hazard scoring highest across all metrics. WoFS availability improved the hourly probabilistic forecasts as measured by the subjective ratings and several objective metrics, including increased POD and decreased FAR at high probability thresholds. Generally, expert forecasts performed better than consensus forecasts, though expert forecasts over-forecasted. Finally, this work explored the appropriate construction of practically perfect fields used during subjective verification, which participants frequently found to be too small and precise. Using a Gaussian smoother with σ=70 km is recommended to create hourly practically perfect fields in future experiments.
期刊介绍:
Weather and Forecasting (WAF) (ISSN: 0882-8156; eISSN: 1520-0434) publishes research that is relevant to operational forecasting. This includes papers on significant weather events, forecasting techniques, forecast verification, model parameterizations, data assimilation, model ensembles, statistical postprocessing techniques, the transfer of research results to the forecasting community, and the societal use and value of forecasts. The scope of WAF includes research relevant to forecast lead times ranging from short-term “nowcasts” through seasonal time scales out to approximately two years.