Back to models
Metric

MeanSquaredError

API referenceSource
Multivariate

Mean squared error (MSE) or root mean squared error (RMSE).

MSE and RMSE output is non-negative floating point. MSE has units of the input data squared, while RMSE is of the same unit as the input data. Lower is better, and the lowest possible value is 0.0.

For a univariate, non-hierarchical sample of true values \(y_1, \dots, y_n\) and predicted values \(\widehat{y}_1, \dots, \widehat{y}_n\) (in \(mathbb{R}\)), at time indices \(t_1, \dots, t_n\), evaluate or call returns:

  • if square_root is False, the Mean Squared Error, \(\frac{1}{n}\sum_{i=1}^n \left(y_i - \widehat{y}_i\right)^2\)

  • if square_root is True, the Root Mean Squared Error, \(\sqrt{\frac{1}{n}\sum_{i=1}^n \left(y_i - \widehat{y}_i\right)^2}\)

MSE and RMSE are both non-negative floating point, lower values are better. The lowest possible value is 0.0.

multioutput and multilevel control averaging across variables and hierarchy indices, see below. If square_root is True, averages are taken over square roots of squared errors.

evaluate_by_index returns, at a time index \(t_i\):

  • if square_root is False, the squared error at that time index, \(\left(y_i - \widehat{y}_i\right)^2\), for all time indices \(t_1, \dots, t_n\) in the input.

  • if square_root is True, the jackknife pseudo-value of the RMSE at that time index, \(n * \bar{\varepsilon} - (n-1) * \varepsilon_i\), where \(\bar{\varepsilon}\) is the RMSE over all time indices, and \(\varepsilon_i\) is the RMSE with the i-th time index removed, i.e., using values \(y_1, \dots, y_{i-1}, y_{i+1}, \dots, y_n\), and \(\widehat{y}_1, \dots, \widehat{y}_{i-1}, \widehat{y}_{i+1}, \dots, \widehat{y}_n\).

MSE is measured in squared units of the input data, and RMSE is on the same scale as the data. Because MSE and RMSE square the forecast error rather than taking the absolute value, they penalize large errors more than MAE.

Quickstart

python
from sktime.performance_metrics.forecasting import MeanSquaredError

estimator = MeanSquaredError(multioutput='uniform_average', multilevel='uniform_average', square_root=False, by_index=False)

Parameters(4)

square_rootbool, default = False
Whether to take the square root of the metric
multioutput‘uniform_average’ (default), 1D array-like, or ‘raw_values’

Whether and how to aggregate metric for multivariate (multioutput) data.

  • If 'uniform_average' (default), errors of all outputs are averaged with uniform weight.

  • If 1D array-like, errors are averaged across variables, with values used as averaging weights (same order).

  • If 'raw_values', does not average across variables (outputs), per-variable errors are returned.

multilevel{‘raw_values’, ‘uniform_average’, ‘uniform_average_time’}

How to aggregate the metric for hierarchical data (with levels).

  • If 'uniform_average' (default), errors are mean-averaged across levels.

  • If 'uniform_average_time', metric is applied to all data, ignoring level index.

  • If 'raw_values', does not average errors across levels, hierarchy is retained.

by_indexbool, default=False

Controls averaging over time points in direct call to metric object.

  • If False (default), direct call to the metric object averages over time points, equivalent to a call of the evaluate method.

  • If True, direct call to the metric object evaluates the metric at each time point, equivalent to a call of the evaluate_by_index method.

Examples

>>> import numpy as np
>>> from sktime.performance_metrics.forecasting import MeanSquaredError
>>> y_true = np. array ([3, - 0.5, 2, 7, 2 ])
>>> y_pred = np. array ([2.5, 0.0, 2, 8, 1.25 ])
>>> mse = MeanSquaredError ()
>>> mse (y_true, y_pred) np.float64(0.4125)
>>> y_true = np. array ([[0.5, 1 ], [- 1, 1 ], [7, - 6 ]])
>>> y_pred = np. array ([[0, 2 ], [- 1, 2 ], [8, - 5 ]])
>>> mse (y_true, y_pred) np.float64(0.7083333333333334)
>>> rmse = MeanSquaredError (square_root = True)
>>> rmse (y_true, y_pred) np.float64(0.8227486121839513)
>>> rmse = MeanSquaredError (multioutput = 'raw_values')
>>> rmse (y_true, y_pred) array([0.41666667, 1. ])
>>> rmse = MeanSquaredError (multioutput = 'raw_values', square_root = True)
>>> rmse (y_true, y_pred) array([0.64549722, 1. ])
>>> rmse = MeanSquaredError (multioutput = [0.3, 0.7 ])
>>> rmse (y_true, y_pred) np.float64(0.825)
>>> rmse = MeanSquaredError (multioutput = [0.3, 0.7 ], square_root = True)
>>> rmse (y_true, y_pred) np.float64(0.8936491673103708)

References

  1. Hyndman, R. J and Koehler, A. B. (2006). “Another look at measures of forecast accuracy”, International Journal of Forecasting, Volume 22, Issue 4.