Performance metrics#
The sktime.performance_metrics
module contains metrics for evaluating and tuning time series models.
All parameter estimators in sktime
can be listed using the
sktime.registry.all_estimators
utility,
using estimator_types="metric"
, optionally filtered by tags.
Valid tags can be listed using sktime.registry.all_tags
.
A full table with tag based search is also available on the Estimator Search Page (select “metric” in the “Estimator type” dropdown).
Metrics for assessing model performance.
Forecasting#
Point forecasts - classes#
|
Mean absolute scaled error (MASE). |
|
Median absolute scaled error (MdASE). |
|
Mean squared scaled error (MSSE) or root mean squared scaled error (RMSSE). |
|
Median squared scaled error (MdSSE) or root median squared scaled error (RMdSSE). |
|
Mean absolute error (MAE). |
|
Mean squared error (MSE) or root mean squared error (RMSE). |
|
Median absolute error (MdAE). |
|
Median squared error (MdSE) or root median squared error (RMdSE). |
|
Geometric mean absolute error (GMAE). |
|
Geometric mean squared error (GMSE) or Root geometric mean squared error (RGMSE). |
|
Mean absolute percentage error (MAPE) or symmetric version. |
|
Median absolute percentage error (MdAPE) or symmetric version. |
|
Mean squared percentage error (MSPE) or square root version. |
|
Median squared percentage error (MdSPE) or square root version. |
|
Mean relative absolute error (MRAE). |
|
Median relative absolute error (MdRAE). |
Geometric mean relative absolute error (GMRAE). |
|
Geometric mean relative squared error (GMRSE). |
|
|
Calculate mean of asymmetric loss function. |
|
Calculate mean linex error. |
|
Calculate relative loss of forecast versus benchmark forecast. |
Point forecasts - functions#
|
Create a metric class from a metric function. |
|
Mean absolute scaled error (MASE). |
|
Median absolute scaled error (MdASE). |
|
Mean squared scaled error (MSSE) or root mean squared scaled error (RMSSE). |
|
Median squared scaled error (MdSSE) or root median squared scaled error (RMdSSE). |
|
Mean absolute error (MAE). |
|
Mean squared error (MSE) or root mean squared error (RMSE). |
|
Median absolute error (MdAE). |
|
Median squared error (MdSE) or root median squared error (RMdSE). |
|
Geometric mean absolute error (GMAE). |
|
Geometric mean squared error (GMSE) or Root geometric mean squared error (RGMSE). |
|
Mean absolute percentage error (MAPE) or symmetric version. |
|
Median absolute percentage error (MdAPE) or symmetric version. |
|
Mean squared percentage error (MSPE) or square root version. |
|
Median squared percentage error (MdSPE) or square root version. |
|
Mean relative absolute error (MRAE). |
|
Median relative absolute error (MdRAE). |
Geometric mean relative absolute error (GMRAE). |
|
|
Geometric mean relative squared error (GMRSE). |
|
Calculate mean of asymmetric loss function. |
|
Calculate mean linex error. |
|
Relative loss of forecast versus benchmark forecast for a given metric. |
Quantile and interval forecasts#
|
Pinball loss aka quantile loss for quantile/interval predictions. |
|
Empirical coverage percentage for interval predictions. |
|
Percentage of interval constraint violations for interval predictions. |
Distribution forecasts#
|
Area under the calibration curve for distributional predictions. |
|
Continuous rank probability score for distributional predictions. |
|
Logarithmic loss for distributional predictions. |
|
Squared loss for distributional predictions. |
Time series segmentation#
|
Error counting the difference in the number of change points. |
|
Compute the Hausdorff distance between two sets of change points. |
|
Prediction ratio is the ratio of number of predicted to true change points. |