Locally tail-scale invariant scoring rules for evaluation of extreme value forecasts

Abstract

Statistical analysis of extremes can be used to predict the probability of future extreme events, such as large rainfalls or devastating windstorms. The quality of these forecasts can be measured through scoring rules. Locally scale invariant scoring rules give equal importance to the forecasts at different locations regardless of differences in the prediction uncertainty. This is a useful feature when computing average scores but can be an unnecessarily strict requirement when one is mostly concerned with extremes. We propose the concept of local weight-scale invariance, describing scoring rules fulfilling local scale invariance in a certain region of interest, and as a special case, local tail-scale invariance for large events. Moreover, a new version of the weighted continuous ranked probability score (wCRPS) called the scaled wCRPS (swCRPS) that possesses this property is developed and studied. The score is a suitable alternative for scoring extreme value models over areas with a varying scale of extreme events, and we derive explicit formulas of the score for the generalised extreme value distribution. The scoring rules are compared through simulations, and their usage is illustrated by modelling extreme water levels and annual maximum rainfall, and in an application to non-extreme forecasts for the prediction of air pollution.