Multimodal Fusion Learning for Predicting Tropical Cyclone Intensity Over Western North Pacific

Abstract

Tropical cyclones (TCs) are highly destructive weather phenomena that cause extensive human and economic losses in affected regions. Accurate prediction of tropical cyclone intensity (TCI) is crucial for disaster preparedness and mitigation. Traditional TCI forecasting methods fail to extract nonlinear features and suffer from high computation costs. In recent years, deep learning methods have been increasingly used to address this challenge. However, current approaches often underutilize meteorological variables and satellite cloud imagery, and fail to capture correlations between multimodal data. In this article, we propose TCIque, a sequence-to-sequence model specifically designed for TCI forecasting. TCIque is designed to integrate multimodal data and retrieve correlational features between them based on the Wide and Deep concept. The “Wide” component leverages domain knowledge to extract statistical features, while the “Deep” component captures nonlinear correlations and spatio-temporal dynamics based on self-attention mechanisms. This unique combination allows the model to fully utilize diverse data sources, such as meteorological variables, satellite imagery, and expert-driven features, ensuring robust feature fusion. Furthermore, a predictive encoder–decoder architecture associated with the self-attention mechanism is employed to address the challenge of long-term dependency decay. Experimental results demonstrate that the TCIque model outperforms existing methods, achieving more accurate performance in TCI prediction by 60.9%, 51.6%, 39.2%, and 1.8% compared to the best performance of baselines, which includes ConvLSTM, PredRNN, TC-Pred, SCSTque, SAF-Net, TCI-Net, Tint, and Pred_3d at 6h, 12h, 18h, and 24h forecast, respectively.