Metric

TheilU2

Theil’s U2 statistic comparing forecast to naive (random walk) forecast.

Theil’s U2 compares the root mean squared error of the forecast to the root mean squared error of a naive (random walk) forecast. Output is non-negative floating point, lower is better, with 0.0 indicating a perfect forecast and 1.0 indicating performance equal to the naive forecast.

For a univariate, non-hierarchical sample of true values \(y_1, \dots, y_n\), predicted values \(\widehat{y}_1, \dots, \widehat{y}_n\), and in-sample training values \(y_1^{\text{train}}, \dots, y_m^{\text{train}}\), evaluate or call returns

\[U_2 = \sqrt{ \frac{ \frac{1}{n}\sum_{i=1}^{n}(y_i - \widehat{y}_i)^2 }{ \frac{1}{n}\sum_{i=1}^{n}(y_i - y_{i-s})^2 } }\]

where \(s\) is the seasonal periodicity (sp, default 1), and \(y_{i-s}\) is the naive seasonal forecast: the actual value \(s\) periods before time \(i\). For the first \(s\) test-period values, the naive forecast uses the last \(s\) values from y_train.

To avoid division by zero, the denominator is clamped to eps.

evaluate_by_index returns jackknife pseudo-values since U2 is sqrt(mean/mean), not a simple per-index mean.

multioutput and multilevel control averaging across variables and hierarchy indices, see below.

Schnellstart

python

from sktime.performance_metrics.forecasting import TheilU2

estimator = TheilU2(multioutput='uniform_average', multilevel='uniform_average', sp=1, by_index=False, eps=None)

Parameter(5)

spint, default=1: Seasonal periodicity of the data. sp=1 corresponds to the standard random-walk naive forecast.
epsfloat, default=None: Numerical epsilon used in denominator to avoid division by zero. Values smaller than eps are replaced by eps. If None, defaults to np.finfo(np.float64).eps
multioutput‘uniform_average’ (default), 1D array-like, or ‘raw_values’: Whether and how to aggregate metric for multivariate (multioutput) data.
If 'uniform_average' (default), errors of all outputs are averaged with uniform weight.
If 1D array-like, errors are averaged across variables, with values used as averaging weights (same order).
If 'raw_values', does not average across variables (outputs), per-variable errors are returned.
multilevel{‘raw_values’, ‘uniform_average’, ‘uniform_average_time’}: How to aggregate the metric for hierarchical data (with levels).
If 'uniform_average' (default), errors are mean-averaged across levels.
If 'uniform_average_time', metric is applied to all data, ignoring level index.
If 'raw_values', does not average errors across levels, hierarchy is retained.
by_indexbool, default=False: Controls averaging over time points in direct call to metric object.
If False (default), direct call to the metric object averages over time points, equivalent to a call of the evaluate method.
If True, direct call to the metric object evaluates the metric at each time point, equivalent to a call of the evaluate_by_index method.

Beispiele

>>> import numpy as np
>>> from sktime.performance_metrics.forecasting import TheilU2
>>> y_train = np. array ([5, 0.5, 4, 6, 3, 5, 2 ])
>>> y_true = np. array ([3, - 0.5, 2, 7, 2 ])
>>> y_pred = np. array ([2.5, 0.0, 2, 8, 1.25 ])
>>> theilu2 = TheilU2 ()
>>> theilu2 (y_true, y_pred, y_train = y_train) np.float64(0.17226798597767884)

Referenzen

Chen, Z. and Yang, Y. (2004). “Assessing Forecast Accuracy Measures”, Preprint 2004-10, Iowa State University.

Theil, H. (1966). “Applied Economic Forecasting”, North-Holland, Amsterdam.