MeanLinexError

class MeanLinexError(a=1.0, b=1.0, multioutput='uniform_average', multilevel='uniform_average', by_index=False)[source]

Mean Linear Exponential (LinEx) error.

Output is non-negative floating point. Smaller values are better, the minimal possible value is 0.0.

The LinEx error is an asymmetric loss function, with parameter a controlling the penalty for over- vs under-predictions.

For a univariate, non-hierarchical sample of true values \(y_1, \dots, y_n\) and predicted values \(\widehat{y}_1, \dots, \widehat{y}_n\) (in \(mathbb{R}\)), at time indices \(t_1, \dots, t_n\), evaluate or call returns the mean LinEx loss:

\[\frac{b}{n}\sum_{i=1}^n \left( \exp(a \cdot e_i) - a \cdot e_i - 1 \right)\]

where \(e_i = y_i - \widehat{y}_i\), and \(a \neq 0, b > 0\) are parameters of the metric, a and b in the constructor.

a controls the asymmetry of the penalty:

If a > 0, the penalty for over-predictions is approximately linear, while the penalty for under-predictions is approximately exponential.
If a < 0, the penalty for under-predictions is approximately linear, while the penalty for over-predictions is approximately exponential.

b is a scale parameter that controls the overall magnitude of the penalty.

multioutput and multilevel decide how results are averaged when there are multiple variables (multioutput) or hierarchical levels in the data. See below.

evaluate_by_index returns, at a time index \(t_i\) , the LinEx loss at that time index, \(b \cdot (\exp(a \cdot e_i) - a \cdot e_i -1)\) , where \(e_i = y_i - \widehat{y}_i\) , for all time indices \(t_1, \dots, t_n\) in the input.

Parameters:

aint or float, default = 1

Controls whether over- or under- predictions receive an approximately linear or exponential penalty. If a > 0 then negative errors (over-predictions) are penalized approximately linearly and positive errors (under-predictions) are penalized approximately exponentially. If a < 0 the reverse is true.

bint or float, default = 1

Multiplicative penalty to apply to calculated errors controlled by scale parameter.

multioutput‘uniform_average’ (default), 1D array-like, or ‘raw_values’

Whether and how to aggregate metric for multivariate (multioutput) data.

If 'uniform_average' (default), errors of all outputs are averaged with uniform weight.
If 1D array-like, errors are averaged across variables, with values used as averaging weights (same order).
If 'raw_values', does not average across variables (outputs), per-variable errors are returned.

multilevel{‘raw_values’, ‘uniform_average’, ‘uniform_average_time’}

How to aggregate the metric for hierarchical data (with levels).

If 'uniform_average' (default), errors are mean-averaged across levels.
If 'uniform_average_time', metric is applied to all data, ignoring level index.
If 'raw_values', does not average errors across levels, hierarchy is retained.

by_indexbool, default=False

Controls averaging over time points in direct call to metric object.

If False (default), direct call to the metric object averages over time points, equivalent to a call of the evaluate method.
If True, direct call to the metric object evaluates the metric at each time point, equivalent to a call of the evaluate_by_index method.

References

[1]

Hyndman, R. J and Koehler, A. B. (2006). “Another look at measures of forecast accuracy”, International Journal of Forecasting, Volume 22, Issue 4.

[2]

Diebold, Francis X. (2007). “Elements of Forecasting (4th ed.)”, Thomson, South-Western: Ohio, US.

Examples

>>> import numpy as np
>>> from sktime.performance_metrics.forecasting import MeanLinexError
>>> linex_error = MeanLinexError()
>>> y_true = np.array([3, -0.5, 2, 7, 2])
>>> y_pred = np.array([2.5, 0.0, 2, 8, 1.25])
>>> linex_error(y_true, y_pred)
np.float64(0.19802627763937575)
>>> linex_error = MeanLinexError(b=2)
>>> linex_error(y_true, y_pred)
np.float64(0.3960525552787515)
>>> linex_error = MeanLinexError(a=-1)
>>> linex_error(y_true, y_pred)
np.float64(0.2391800623225643)
>>> y_true = np.array([[0.5, 1], [-1, 1], [7, -6]])
>>> y_pred = np.array([[0, 2], [-1, 2], [8, -5]])
>>> linex_error = MeanLinexError()
>>> linex_error(y_true, y_pred)
np.float64(0.2700398392309829)
>>> linex_error = MeanLinexError(a=-1)
>>> linex_error(y_true, y_pred)
np.float64(0.49660966225813563
>>> linex_error = MeanLinexError(multioutput='raw_values')
>>> linex_error(y_true, y_pred)
array([0.17220024, 0.36787944])
>>> linex_error = MeanLinexError(multioutput=[0.3, 0.7])
>>> linex_error(y_true, y_pred)
np.float64(0.30917568000716666)

Methods

__call__(y_true, y_pred, **kwargs)

Calculate metric value using underlying metric function.

__call__(y_true, y_pred, **kwargs)[source]

Calculate metric value using underlying metric function.

Parameters:

y_truetime series in sktime compatible data container format.

Ground truth (correct) target values.

Individual data formats in sktime are so-called mtype specifications, each mtype implements an abstract scitype.

Series scitype = individual time series, vanilla forecasting. pd.DataFrame, pd.Series, or np.ndarray (1D or 2D)
Panel scitype = collection of time series, global/panel forecasting. pd.DataFrame with 2-level row MultiIndex (instance, time), 3D np.ndarray (instance, variable, time), list of Series typed pd.DataFrame
Hierarchical scitype = hierarchical collection, for hierarchical forecasting. pd.DataFrame with 3 or more level row MultiIndex (hierarchy_1, ..., hierarchy_n, time)

For further details on data format, see glossary on mtype. For usage, see forecasting tutorial examples/01_forecasting.ipynb

y_predtime series in sktime compatible data container format

Predicted values to evaluate against ground truth. Must be of same format as y_true, same indices and columns if indexed.

y_pred_benchmarkoptional, time series in sktime compatible data container format

Benchmark predictions to compare y_pred to, used for relative metrics. Required only if metric requires benchmark predictions, as indicated by tag requires-y-pred-benchmark. Otherwise, can be passed to ensure interface consistency, but is ignored. Must be of same format as y_true, same indices and columns if indexed.

y_trainoptional, time series in sktime compatible data container format

Training data used to normalize the error metric. Required only if metric requires training data, as indicated by tag requires-y-train. Otherwise, can be passed to ensure interface consistency, but is ignored. Must be of same format as y_true, same columns if indexed, but not necessarily same indices.

sample_weightoptional, 1D array-like, or callable, default=None

Sample weights for each time point.

If None, the time indices are considered equally weighted.
If an array, must be 1D. If y_true and y_pred``are a single time series, ``sample_weight must be of the same length as y_true. If the time series are panel or hierarchical, the length of all individual time series must be the same, and equal to the length of sample_weight, for all instances of time series passed.
If a callable, it must follow SampleWeightGenerator interface, or have one of the following signatures: y_true: pd.DataFrame -> 1D array-like, or y_true: pd.DataFrame x y_pred: pd.DataFrame -> 1D array-like.

Returns:

lossfloat, np.ndarray, or pd.DataFrame

Calculated metric, averaged or by variable. Weighted by sample_weight if provided.

float if multioutput="uniform_average" or array-like, and ``multilevel="uniform_average" or “uniform_average_time”``. Value is metric averaged over variables and levels (see class docstring)
np.ndarray of shape (y_true.columns,) if multioutput=”raw_values”` and multilevel="uniform_average" or "uniform_average_time". i-th entry is the, metric calculated for i-th variable
pd.DataFrame if multilevel="raw_values". of shape (n_levels, ), if multioutput="uniform_average"; of shape (n_levels, y_true.columns) if multioutput="raw_values". metric is applied per level, row averaging (yes/no) as in multioutput.