MedianSquaredScaledError#
- class MedianSquaredScaledError(multioutput='uniform_average', multilevel='uniform_average', sp=1, square_root=False)[source]#
Median squared scaled error (MdSSE) or root median squared scaled error (RMdSSE).
If square_root is False then calculates MdSSE, otherwise calculates RMdSSE if square_root is True. Both MdSSE and RMdSSE output is non-negative floating point. The best value is 0.0.
This is a squared varient of the MdASE loss metric. Like MASE and other scaled performance metrics this scale-free metric can be used to compare forecast methods on a single series or between series.
This metric is also suited for intermittent-demand series because it will not give infinite or undefined values unless the training data is a flat timeseries. In this case the function returns a large value instead of inf.
Works with multioutput (multivariate) timeseries data with homogeneous seasonal periodicity.
- Parameters
- spint, default = 1
Seasonal periodicity of data.
- square_rootbool, default = False
Whether to take the square root of the metric
- multioutput{‘raw_values’, ‘uniform_average’} or array-like of shape (n_outputs,), default=’uniform_average’
Defines how to aggregate metric for multivariate (multioutput) data. If array-like, values used as weights to average the errors. If ‘raw_values’, returns a full set of errors in case of multioutput input. If ‘uniform_average’, errors of all outputs are averaged with uniform weight.
- Attributes
greater_is_better
Whether greater is better for the metric.
References
M5 Competition Guidelines. https://mofc.unic.ac.cy/wp-content/uploads/2020/03/M5-Competitors-Guide-Final-10-March-2020.docx
Hyndman, R. J and Koehler, A. B. (2006). “Another look at measures of forecast accuracy”, International Journal of Forecasting, Volume 22, Issue 4.
Examples
>>> import numpy as np >>> from sktime.performance_metrics.forecasting import MedianSquaredScaledError >>> y_train = np.array([5, 0.5, 4, 6, 3, 5, 2]) >>> y_true = np.array([3, -0.5, 2, 7, 2]) >>> y_pred = np.array([2.5, 0.0, 2, 8, 1.25]) >>> rmdsse = MedianSquaredScaledError(square_root=True) >>> rmdsse(y_true, y_pred, y_train=y_train) 0.16666666666666666 >>> y_train = np.array([[0.5, 1], [-1, 1], [7, -6]]) >>> y_true = np.array([[0.5, 1], [-1, 1], [7, -6]]) >>> y_pred = np.array([[0, 2], [-1, 2], [8, -5]]) >>> rmdsse(y_true, y_pred, y_train=y_train) 0.1472819539849714 >>> rmdsse = MedianSquaredScaledError(multioutput='raw_values', square_root=True) >>> rmdsse(y_true, y_pred, y_train=y_train) array([0.08687445, 0.20203051]) >>> rmdsse = MedianSquaredScaledError(multioutput=[0.3, 0.7], square_root=True) >>> rmdsse(y_true, y_pred, y_train=y_train) 0.16914781383660782
Methods
__call__
(y_true, y_pred, **kwargs)Calculate metric value using underlying metric function.
clone
()Obtain a clone of the object with same hyper-parameters.
clone_tags
(estimator[, tag_names])clone/mirror tags from another estimator as dynamic override.
create_test_instance
([parameter_set])Construct Estimator instance if possible.
create_test_instances_and_names
([parameter_set])Create list of all test instances and a list of names for them.
evaluate
(y_true, y_pred, **kwargs)Evaluate the desired metric on given inputs.
evaluate_by_index
(y_true, y_pred, **kwargs)Return the metric evaluated at each time point.
func
(y_pred[, sp, horizon_weight, …])Median squared scaled error (MdSSE) or root median squared scaled error (RMdSSE).
get_class_tag
(tag_name[, tag_value_default])Get tag value from estimator class (only class tags).
Get class tags from estimator class and all its parent classes.
get_params
([deep])Get parameters for this estimator.
get_tag
(tag_name[, tag_value_default, …])Get tag value from estimator class and dynamic tag overrides.
get_tags
()Get tags from estimator class and dynamic tag overrides.
get_test_params
([parameter_set])Return testing parameter settings for the estimator.
Check if the object is composite.
reset
()Reset the object to a clean post-init state.
set_params
(**params)Set the parameters of this estimator.
set_tags
(**tag_dict)Set dynamic tags to given values.
- func(y_pred, sp=1, horizon_weight=None, multioutput='uniform_average', square_root=False, **kwargs)[source]#
Median squared scaled error (MdSSE) or root median squared scaled error (RMdSSE).
If square_root is False then calculates MdSSE, otherwise calculates RMdSSE if square_root is True. Both MdSSE and RMdSSE output is non-negative floating point. The best value is 0.0.
This is a squared varient of the MdASE loss metric. Like MASE and other scaled performance metrics this scale-free metric can be used to compare forecast methods on a single series or between series.
This metric is also suited for intermittent-demand series because it will not give infinite or undefined values unless the training data is a flat timeseries. In this case the function returns a large value instead of inf.
Works with multioutput (multivariate) timeseries data with homogeneous seasonal periodicity.
- Parameters
- y_truepd.Series, pd.DataFrame or np.array of shape (fh,) or (fh, n_outputs) where fh is the forecasting horizon
Ground truth (correct) target values.
- y_predpd.Series, pd.DataFrame or np.array of shape (fh,) or (fh, n_outputs) where fh is the forecasting horizon
Forecasted values.
- y_trainpd.Series, pd.DataFrame or np.array of shape (n_timepoints,) or (n_timepoints, n_outputs), default = None
Observed training values.
- spint
Seasonal periodicity of training data.
- horizon_weightarray-like of shape (fh,), default=None
Forecast horizon weights.
- multioutput{‘raw_values’, ‘uniform_average’} or array-like of shape (n_outputs,), default=’uniform_average’
Defines how to aggregate metric for multivariate (multioutput) data. If array-like, values used as weights to average the errors. If ‘raw_values’, returns a full set of errors in case of multioutput input. If ‘uniform_average’, errors of all outputs are averaged with uniform weight.
- Returns
- lossfloat
RMdSSE loss. If multioutput is ‘raw_values’, then RMdSSE is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average RMdSSE of all output errors is returned.
References
M5 Competition Guidelines. https://mofc.unic.ac.cy/wp-content/uploads/2020/03/M5-Competitors-Guide-Final-10-March-2020.docx
Hyndman, R. J and Koehler, A. B. (2006). “Another look at measures of forecast accuracy”, International Journal of Forecasting, Volume 22, Issue 4.
Examples
>>> from sktime.performance_metrics.forecasting import median_squared_scaled_error >>> y_train = np.array([5, 0.5, 4, 6, 3, 5, 2]) >>> y_true = np.array([3, -0.5, 2, 7, 2]) >>> y_pred = np.array([2.5, 0.0, 2, 8, 1.25]) >>> median_squared_scaled_error(y_true, y_pred, y_train=y_train, square_root=True) 0.16666666666666666 >>> y_train = np.array([[0.5, 1], [-1, 1], [7, -6]]) >>> y_true = np.array([[0.5, 1], [-1, 1], [7, -6]]) >>> y_pred = np.array([[0, 2], [-1, 2], [8, -5]]) >>> median_squared_scaled_error(y_true, y_pred, y_train=y_train, square_root=True) 0.1472819539849714 >>> median_squared_scaled_error(y_true, y_pred, y_train=y_train, multioutput='raw_values', square_root=True) array([0.08687445, 0.20203051]) >>> median_squared_scaled_error(y_true, y_pred, y_train=y_train, multioutput=[0.3, 0.7], square_root=True) 0.16914781383660782
- __call__(y_true, y_pred, **kwargs)[source]#
Calculate metric value using underlying metric function.
- Parameters
- y_truetime series in sktime compatible data container format
Ground truth (correct) target values y can be in one of the following formats: Series scitype: pd.Series, pd.DataFrame, or np.ndarray (1D or 2D) Panel scitype: pd.DataFrame with 2-level row MultiIndex,
3D np.ndarray, list of Series pd.DataFrame, or nested pd.DataFrame
Hierarchical scitype: pd.DataFrame with 3 or more level row MultiIndex
- y_predtime series in sktime compatible data container format
Forecasted values to evaluate must be of same format as y_true, same indices and columns if indexed
- Returns
- lossfloat or np.ndarray
Calculated metric, averaged or by variable. float if self.multioutput=”uniform_average” or array-like
value is metric averaged over variables (see class docstring)
- np.ndarray of shape (y_true.columns,) if self.multioutput=”raw_values”
i-th entry is metric calculated for i-th variable
- clone()[source]#
Obtain a clone of the object with same hyper-parameters.
A clone is a different object without shared references, in post-init state. This function is equivalent to returning sklearn.clone of self. Equal in value to type(self)(**self.get_params(deep=False)).
- clone_tags(estimator, tag_names=None)[source]#
clone/mirror tags from another estimator as dynamic override.
- Parameters
- estimatorestimator inheriting from :class:BaseEstimator
- tag_namesstr or list of str, default = None
Names of tags to clone. If None then all tags in estimator are used as tag_names.
- Returns
- Self
Reference to self.
Notes
Changes object state by setting tag values in tag_set from estimator as dynamic tags in self.
- classmethod create_test_instance(parameter_set='default')[source]#
Construct Estimator instance if possible.
- Parameters
- parameter_setstr, default=”default”
Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.
- Returns
- instanceinstance of the class with default parameters
Notes
get_test_params can return dict or list of dict. This function takes first or single dict that get_test_params returns, and constructs the object with that.
- classmethod create_test_instances_and_names(parameter_set='default')[source]#
Create list of all test instances and a list of names for them.
- Parameters
- parameter_setstr, default=”default”
Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.
- Returns
- objslist of instances of cls
i-th instance is cls(**cls.get_test_params()[i])
- nameslist of str, same length as objs
i-th element is name of i-th instance of obj in tests convention is {cls.__name__}-{i} if more than one instance otherwise {cls.__name__}
- parameter_setstr, default=”default”
Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.
- evaluate(y_true, y_pred, **kwargs)[source]#
Evaluate the desired metric on given inputs.
- Parameters
- y_truetime series in sktime compatible data container format
Ground truth (correct) target values y can be in one of the following formats: Series scitype: pd.Series, pd.DataFrame, or np.ndarray (1D or 2D) Panel scitype: pd.DataFrame with 2-level row MultiIndex,
3D np.ndarray, list of Series pd.DataFrame, or nested pd.DataFrame
Hierarchical scitype: pd.DataFrame with 3 or more level row MultiIndex
- y_predtime series in sktime compatible data container format
Forecasted values to evaluate must be of same format as y_true, same indices and columns if indexed
- Returns
- lossfloat or np.ndarray
Calculated metric, averaged or by variable. float if self.multioutput=”uniform_average” or array-like
value is metric averaged over variables (see class docstring)
- np.ndarray of shape (y_true.columns,) if self.multioutput=”raw_values”
i-th entry is metric calculated for i-th variable
- evaluate_by_index(y_true, y_pred, **kwargs)[source]#
Return the metric evaluated at each time point.
- Parameters
- y_truetime series in sktime compatible data container format
Ground truth (correct) target values y can be in one of the following formats: Series scitype: pd.Series, pd.DataFrame, or np.ndarray (1D or 2D) Panel scitype: pd.DataFrame with 2-level row MultiIndex,
3D np.ndarray, list of Series pd.DataFrame, or nested pd.DataFrame
Hierarchical scitype: pd.DataFrame with 3 or more level row MultiIndex
- y_predtime series in sktime compatible data container format
Forecasted values to evaluate must be of same format as y_true, same indices and columns if indexed
- Returns
- losspd.Series or pd.DataFrame
Calculated metric, by time point (default=jackknife pseudo-values). pd.Series if self.multioutput=”uniform_average” or array-like
index is equal to index of y_true entry at index i is metric at time i, averaged over variables
- pd.DataFrame if self.multioutput=”raw_values”
index and columns equal to those of y_true i,j-th entry is metric at time i, at variable j
- classmethod get_class_tag(tag_name, tag_value_default=None)[source]#
Get tag value from estimator class (only class tags).
- Parameters
- tag_namestr
Name of tag value.
- tag_value_defaultany type
Default/fallback value if tag is not found.
- Returns
- tag_value
Value of the tag_name tag in self. If not found, returns tag_value_default.
- classmethod get_class_tags()[source]#
Get class tags from estimator class and all its parent classes.
- Returns
- collected_tagsdict
Dictionary of tag name : tag value pairs. Collected from _tags class attribute via nested inheritance. NOT overridden by dynamic tags set by set_tags or mirror_tags.
- get_params(deep=True)[source]#
Get parameters for this estimator.
- Parameters
- deepbool, default=True
If True, will return the parameters for this estimator and contained subobjects that are estimators.
- Returns
- paramsdict
Parameter names mapped to their values.
- get_tag(tag_name, tag_value_default=None, raise_error=True)[source]#
Get tag value from estimator class and dynamic tag overrides.
- Parameters
- tag_namestr
Name of tag to be retrieved
- tag_value_defaultany type, optional; default=None
Default/fallback value if tag is not found
- raise_errorbool
whether a ValueError is raised when the tag is not found
- Returns
- tag_value
Value of the tag_name tag in self. If not found, returns an error if raise_error is True, otherwise it returns tag_value_default.
- Raises
- ValueError if raise_error is True i.e. if tag_name is not in self.get_tags(
- ).keys()
- get_tags()[source]#
Get tags from estimator class and dynamic tag overrides.
- Returns
- collected_tagsdict
Dictionary of tag name : tag value pairs. Collected from _tags class attribute via nested inheritance and then any overrides and new tags from _tags_dynamic object attribute.
- classmethod get_test_params(parameter_set='default')[source]#
Return testing parameter settings for the estimator.
- Parameters
- parameter_setstr, default=”default”
Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.
- Returns
- paramsdict or list of dict, default = {}
Parameters to create testing instances of the class Each dict are parameters to construct an “interesting” test instance, i.e., MyClass(**params) or MyClass(**params[i]) creates a valid test instance. create_test_instance uses the first (or only) dictionary in params
- is_composite()[source]#
Check if the object is composite.
A composite object is an object which contains objects, as parameters. Called on an instance, since this may differ by instance.
- Returns
- composite: bool, whether self contains a parameter which is BaseObject
- reset()[source]#
Reset the object to a clean post-init state.
Equivalent to sklearn.clone but overwrites self. After self.reset() call, self is equal in value to type(self)(**self.get_params(deep=False))
Detail behaviour: removes any object attributes, except:
hyper-parameters = arguments of __init__ object attributes containing double-underscores, i.e., the string “__”
runs __init__ with current values of hyper-parameters (result of get_params)
Not affected by the reset are: object attributes containing double-underscores class and object methods, class attributes
- set_params(**params)[source]#
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as
Pipeline
). The latter have parameters of the form<component>__<parameter>
so that it’s possible to update each component of a nested object.- Parameters
- **paramsdict
Estimator parameters.
- Returns
- selfestimator instance
Estimator instance.