End-to-end data-driven weather prediction

Abstract

Weather prediction is critical for a range of human activities including transportation, agriculture and industry, as well as the safety of the general public. Machine learning is transforming numerical weather prediction (NWP) by replacing the numerical solver with neural networks, improving the speed and accuracy of the forecasting component of the prediction pipeline ^1,2,3,4,5,6. However, current models rely on numerical systems at initialisation and to produce local forecasts, limiting their achievable gains. Here we show that a single machine learning model can replace the entire NWP pipeline. Aardvark Weather, an end-to-end data-driven weather prediction system, ingests observations and produces global gridded forecasts and local station forecasts. The global forecasts outperform an operational NWP baseline for multiple variables and lead times. The local station forecasts are skillful up to ten days lead time, competing with a post-processed global NWP baseline and a state-of-the-art end-to-end forecasting system with input from human forecasters. End-to-end tuning further improves the accuracy of local forecasts. Our results show that skillful forecasting is possible without relying on NWP at deployment time, which will enable the full speed and accuracy benefits of data-driven models to be realised. We believe Aardvark Weather will be the starting point for a new generation of end-to-end models that will reduce computational costs by orders of magnitude, and enable rapid, affordable creation of customised models for a range of end-users.