Regional Cloud Forecast Verification using Standard, Spatial and Object-Oriented Methods

Abstract

This study assesses the accuracy of the Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) forecasts for clouds within stable and unstable environments (thereafter refers as “stable” and “unstable” clouds). This evaluation is conducted by comparing these forecasts against satellite retrievals through a combination of traditional, spatial, and object-based methods. To facilitate this assessment, the Model Evaluation Tools (MET) community tool is employed. The findings underscore the significance of fine-tuning the MET parameters to achieve a more accurate representation of the features under scrutiny. The study's results reveal that when employing traditional point-wise statistics (e.g., frequency bias and equitable threat score), there is consistency in the results whether calculated from Method for Object-Based Diagnostic Evaluation (MODE)-based objects or derived from the complete fields. Furthermore, the object-based statistics offer valuable insights, indicating that COAMPS generally predicts cloud object locations accurately, though the spread of these predicted locations tends to increase with time. It tends to over-predict the object area for unstable clouds while under-predicting it for stable clouds over time. These results are in alignment with the traditional pointwise bias scores for the entire grid. Overall, the spatial metrics provided by the object-based verification methods emerge as crucial and practical tools for the validation of cloud forecasts.