Metric

CRPS

Continuous rank probability score for distributional predictions.

Also known as:

integrated squared loss (ISL)
integrated Brier loss (IBL)
energy loss

For a predictive distribution \(d\) and a ground truth value \(y\), the CRPS is defined as \(L(y, d):= \mathbb{E}_{Y \sim d}|Y-y| - \frac{1}{2} \mathbb{E}_{X,Y \sim d}|X-Y|\).

evaluate computes the average test sample loss.
evaluate_by_index produces the loss sample by test data point.
multivariate controls averaging over variables.

Schnellstart

python

from sktime.performance_metrics.forecasting.probabilistic import CRPS

estimator = CRPS(multioutput='uniform_average', multivariate=False)

Parameter(2)

multioutput‘uniform_average’ (default), 1D array-like, or ‘raw_values’: Whether and how to aggregate metric for multivariate (multioutput) data.
If 'uniform_average' (default), errors of all outputs are averaged with uniform weight.
If 1D array-like, errors are averaged across variables, with values used as averaging weights (same order).
If 'raw_values', does not average across variables (outputs), per-variable errors are returned.
multivariatebool, optional, default=False: if True, behaves as multivariate CRPS: the score is computed for entire row, results one score per row
if False, is univariate CRPS: the score is computed per variable marginal, results in many scores per row