Evaluation of five global AI models for predicting weather in Eastern Asia and Western Pacific

Abstract

Recent development of artificial intelligence (AI) technology has resulted in the fruition of machine learning-based weather prediction (MLWP) systems. Five prominent global MLWP model, Pangu-Weather, FourCastNet v2 (FCN2), GraphCast, FuXi, and FengWu, emerged. This study conducts a homogeneous comparison of these models utilizing identical initial conditions from ERA5. The performance is evaluated in the Eastern Asia and Western Pacific from June to November 2023. The evaluation comprises Root Mean Square Error and Anomaly Correlation Coefficients within the designated region, typhoon track and intensity predictions, and a case study for Typhoon Haikui. Results indicate that FengWu emerges as the best-performing model, followed by FuXi and GraphCast, with FCN2 and Pangu-Weather ranking lower. A multi-model ensemble, constructed by averaging predictions from the five models, demonstrates superior performance, rivaling that of FengWu. For the 11 typhoons in 2023, FengWu demonstrates the most accurate track prediction; however, it also has the largest intensity errors.