KLDivergenceDoubleExponential

class KLDivergenceDoubleExponential(multioutput='uniform_average', multilevel='uniform_average', by_index=False, eps=None, window=None)[source]

KL-divergence based forecast error assuming double-exponential errors (KL-DE2).

KL-DE2 uses the Kullback-Leibler divergence between the actual and predicted distributions, assuming double-exponential (Laplace) distributed forecast errors scaled by a rolling mean absolute deviation (MAD) of the true values. Output is non-negative floating point, lower is better, with 0.0 indicating a perfect forecast.

For a univariate, non-hierarchical sample of true values \(y_1, \dots, y_n\) and predicted values \(\widehat{y}_1, \dots, \widehat{y}_n\), evaluate or call returns

\[\text{KL-DE2} = \frac{1}{n}\sum_{i=1}^{n} \left[ \exp\!\left(-\frac{|e_i|}{\hat\sigma_i}\right) + \frac{|e_i|}{\hat\sigma_i} - 1 \right]\]

where \(e_i = y_i - \widehat{y}_i\) and

\[\hat\sigma_i = \frac{1}{i-1}\sum_{j=1}^{i-1}|y_j - \bar{y}_{i-1}| \quad (i \ge 2)\]

is the rolling mean absolute deviation (MAD) of the first \(i-1\) true values (same rolling MAD as in msMAPE).

\(\hat\sigma_1\) is undefined (no prior observations); it is clamped to eps.

Since KL-DE2 is a simple mean of per-index terms, evaluate_by_index returns the per-index KL-DE2 terms directly.

multioutput and multilevel control averaging across variables and hierarchy indices, see below.

Parameters:

multioutput‘uniform_average’ (default), 1D array-like, or ‘raw_values’

Whether and how to aggregate metric for multivariate (multioutput) data.

If 'uniform_average' (default), errors of all outputs are averaged with uniform weight.
If 1D array-like, errors are averaged across variables, with values used as averaging weights (same order).
If 'raw_values', does not average across variables (outputs), per-variable errors are returned.

multilevel{‘raw_values’, ‘uniform_average’, ‘uniform_average_time’}

How to aggregate the metric for hierarchical data (with levels).

If 'uniform_average' (default), errors are mean-averaged across levels.
If 'uniform_average_time', metric is applied to all data, ignoring level index.
If 'raw_values', does not average errors across levels, hierarchy is retained.

by_indexbool, default=False

Controls averaging over time points in direct call to metric object.

If False (default), direct call to the metric object averages over time points, equivalent to a call of the evaluate method.
If True, direct call to the metric object evaluates the metric at each time point, equivalent to a call of the evaluate_by_index method.

epsfloat, default=None

Numerical epsilon used in denominator to avoid division by zero. Values smaller than eps are replaced by eps. If None, defaults to np.finfo(np.float64).eps

windowint or None, default=None

Number of prior observations used to estimate the rolling MAD.

If None (default), an expanding window is used: all prior observations \(y_1, \dots, y_{i-1}\) contribute to \(\hat\sigma_i\).
If int, a fixed-length rolling window of the given size is used.

References

Chen, Z. and Yang, Y. (2004). “Assessing Forecast Accuracy Measures”, Preprint 2004-10, Iowa State University.

Examples

>>> import numpy as np
>>> from sktime.performance_metrics.forecasting import (
...     KLDivergenceDoubleExponential,
... )
>>> y_true = np.array([3.0, 5.0, 2.0, 7.0, 4.0, 6.0])
>>> y_pred = np.array([3.0, 5.0, 3.0, 6.0, 5.0, 5.5])
>>> klde2 = KLDivergenceDoubleExponential()
>>> klde2(y_true, y_pred)
np.float64(0.14407771573805175)

Methods

__call__(y_true, y_pred, **kwargs)

Calculate metric value using underlying metric function.

__call__(y_true, y_pred, **kwargs)[source]

Calculate metric value using underlying metric function.

Parameters:

y_truetime series in sktime compatible data container format.

Ground truth (correct) target values.

Individual data formats in sktime are so-called mtype specifications, each mtype implements an abstract scitype.

Series scitype = individual time series, vanilla forecasting. pd.DataFrame, pd.Series, or np.ndarray (1D or 2D)
Panel scitype = collection of time series, global/panel forecasting. pd.DataFrame with 2-level row MultiIndex (instance, time), 3D np.ndarray (instance, variable, time), list of Series typed pd.DataFrame
Hierarchical scitype = hierarchical collection, for hierarchical forecasting. pd.DataFrame with 3 or more level row MultiIndex (hierarchy_1, ..., hierarchy_n, time)

For further details on data format, see glossary on mtype. For usage, see forecasting tutorial examples/01_forecasting.ipynb

y_predtime series in sktime compatible data container format

Predicted values to evaluate against ground truth. Must be of same format as y_true, same indices and columns if indexed.

y_pred_benchmarkoptional, time series in sktime compatible data container format

Benchmark predictions to compare y_pred to, used for relative metrics. Required only if metric requires benchmark predictions, as indicated by tag requires-y-pred-benchmark. Otherwise, can be passed to ensure interface consistency, but is ignored. Must be of same format as y_true, same indices and columns if indexed.

y_trainoptional, time series in sktime compatible data container format

Training data used to normalize the error metric. Required only if metric requires training data, as indicated by tag requires-y-train. Otherwise, can be passed to ensure interface consistency, but is ignored. Must be of same format as y_true, same columns if indexed, but not necessarily same indices.

sample_weightoptional, 1D array-like, or callable, default=None

Sample weights for each time point.

If None, the time indices are considered equally weighted.
If an array, must be 1D. If y_true and y_pred``are a single time series, ``sample_weight must be of the same length as y_true. If the time series are panel or hierarchical, the length of all individual time series must be the same, and equal to the length of sample_weight, for all instances of time series passed.
If a callable, it must follow SampleWeightGenerator interface, or have one of the following signatures: y_true: pd.DataFrame -> 1D array-like, or y_true: pd.DataFrame x y_pred: pd.DataFrame -> 1D array-like.

Returns:

lossfloat, np.ndarray, or pd.DataFrame

Calculated metric, averaged or by variable. Weighted by sample_weight if provided.

float if multioutput="uniform_average" or array-like, and ``multilevel="uniform_average" or “uniform_average_time”``. Value is metric averaged over variables and levels (see class docstring)
np.ndarray of shape (y_true.columns,) if multioutput=”raw_values”` and multilevel="uniform_average" or "uniform_average_time". i-th entry is the, metric calculated for i-th variable
pd.DataFrame if multilevel="raw_values". of shape (n_levels, ), if multioutput="uniform_average"; of shape (n_levels, y_true.columns) if multioutput="raw_values". metric is applied per level, row averaging (yes/no) as in multioutput.