Toward robust pattern similarity metric for distributed model evaluation

Abstract

SPAtial EFficiency (SPAEF) metric is one of the most thoroughly used metrics in hydrologic community. In this study, our aim is to improve SPAEF by replacing the histogram match component with other statistical indices, i.e. kurtosis and earth mover’s distance, or by adding a fourth or fifth component such as kurtosis and skewness. The existing spatial metrics i.e. SPAtial efficiency (SPAEF), structural similarity (SSIM) and spatial pattern efficiency metric (SPEM) were compared with newly proposed metrics to assess their converging performance. The mesoscale hydrologic model (mHM) of the Moselle River is used to simulate streamflow (Q) and actual evapotranspiration (AET). The two-source energy balance AET during the growing season is used as monthly reference maps to calculate the spatial performance of the model. The moderate resolution imaging spectroradiometer based leaf area index is utilized by the mHM via pedo-transfer functions and multi-scale parameter regionalization approach to scale the potential ET. In addition to the real monthly AET maps, we also tested these metrics using a synthetic true AET map simulated with a known parameter set for a randomly selected day. The results demonstrate that the newly developed four-component metric i.e. SPAtial Hybrid 4 (SPAH4) slightly outperforms conventional three-component metric i.e. SPAEF (3% better). However, SPAH4 significantly outperforms the other existing metrics i.e. 40% better than SSIM and 50% better than SPEM. We believe that other fields such as remote sensing, change detection, function space optimization and image processing can also benefit from SPAH4.