Reducing computational time of closed-loop weather monitoring: A Complex Event Processing and Machine Learning based approach

Abstract

Modern weather forecasting models are developed to maximize the accuracy of forecasts by running computationally intensive algorithms with vast volumes of data. Consequently, algorithms take a long time to execute, and it may adversely affect the timeliness of forecast. One solution to this problem is to run the complex weather forecasting models only on the potentially hazardous events, which are pre-identified by a lightweight data filtering algorithm. We propose a Complex Event Processing (CEP) and Machine Learning (ML) based weather monitoring framework using open source resources that can be extended and customized according to the users' requirements. The CEP engine continuously filters out the input weather data stream to identify potentially hazardous weather events, and then generates a rough boundary enclosing all the data points within the Areas of Interest (AOI). Filtered data points are then fed to the machine learner, where the rough boundary gets more refined by clustering it into a set of AOIs. Each cluster is then concurrently processed by complex weather algorithms of the WRF model. This reduces the computational time by ~75%, as resource heavy weather algorithms are executed using a small subset of data that corresponds to only the areas with potentially hazardous weather.