A probabilistic verification score for contours: Methodology and application to Arctic ice-edge forecasts

We introduce a verification score for probabilistic forecasts of contours – the Spatial

Probability Score (SPS). Defined as the spatial integral of local (Half) Brier

Scores, the SPS can be considered the spatial analogue of the Continuous Ranked

Probability Score (CRPS). Applying the SPS to idealized ensemble forecasts of the

Arctic sea-ice edge in a global coupled climate model, we demonstrate that the metric

responds in a meaningful way to ensemble size, spread, and bias. When applied

to individual forecasts or ensemble means (or quantiles), the SPS is reduced to the

‘volume’ of mismatch, which in the case of the ice edge corresponds to the Integrated

Ice Edge Error (IIEE). By comparing initialized forecasts with climatological

and persistence forecasts, we confirm earlier findings on the potential predictability

of the Arctic sea-ice edge from a probabilistic viewpoint. We conclude that the SPS

is a promising probabilistic verification metric, for contour forecasts in general and

for ice-edge forecasts in particular