ConformalIntervals

class ConformalIntervals(forecaster, method='empirical', initial_window=None, sample_frac=None, verbose=False, n_jobs=None)[source]

Empirical and conformal prediction intervals.

Implements empirical and conformal prediction intervals, on absolute residuals. Empirical prediction intervals are based on sliding window empirical quantiles. Conformal prediction intervals are implemented as described in [1].

All intervals wrap an arbitrary forecaster, i.e., add probabilistic prediction capability to a given point prediction forecaster (first argument).

method=”conformal_bonferroni” is the method described in [1],: where an arbitrary forecaster is used instead of the RNN.
method=”conformal” is the method in [1], but without Bonferroni correction.: i.e., separate forecasts are made which results in H=1 (at all horizons).
method=”empirical” uses quantiles of relative signed residuals on training set,: i.e., y_t+h^(i) - y-hat_t+h^(i), ranging over i, in the notation of [1], at quantiles 0.5-0.5*coverage (lower) and 0.5+0.5*coverage (upper), as offsets to the point prediction at forecast horizon h
method=”empirical_residual” uses empirical quantiles of absolute residuals: on the training set, i.e., quantiles of epsilon-h (in notation [1]), at quantile point (1-coverage)/2 quantiles, as offsets to point prediction

Parameters:

forecasterestimator

Estimator to which probabilistic forecasts are being added

methodstr, optional, default=”empirical”

“empirical”: predictive interval bounds are empirical quantiles from training “empirical_residual”: upper/lower are plusminus (1-coverage)/2 quantiles

of the absolute residuals at horizon, i.e., of epsilon-h

“conformal_bonferroni”: Bonferroni, as in Stankeviciute et al: Caveat: this does not give frequentist but conformal predictive intervals
“conformal”: as in Stankeviciute et al, but with H=1,: i.e., no Bonferroni correction under number of indices in the horizon

initial_windowfloat, int or None, optional (default=max(10, 0.1*len(y)))

Defines the size of the initial training window If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include for the initial window for the train split. If int, represents the relative number of train samples in the initial window. If None, the value is set to the larger of 0.1*len(y) and 10

sample_fracfloat, optional, default=None

value in range (0,1) corresponding to fraction of y index to calculate residuals matrix values for (for speeding up calculation)

verbosebool, optional, default=False

whether to print warnings if windows with too few data points occur

n_jobsint or None, optional, default=1

The number of jobs to run in parallel for fit. -1 means using all processors.

Attributes:

cutoff: Cut-off = “present time” state of forecaster.
fh: Forecasting horizon that was passed.
is_fitted: Whether fit has been called.
state: State of the estimator.

References

[1] (1,2,3,4,5)

Kamile Stankeviciute, Ahmed M Alaa and Mihaela van der Schaar. Conformal Time Series Forecasting. NeurIPS 2021.

Examples

>>> from sktime.datasets import load_airline
>>> from sktime.forecasting.conformal import ConformalIntervals
>>> from sktime.forecasting.naive import NaiveForecaster
>>> y = load_airline()
>>> forecaster = NaiveForecaster(strategy="drift")
>>> conformal_forecaster = ConformalIntervals(forecaster)
>>> conformal_forecaster.fit(y, fh=[1, 2, 3])
ConformalIntervals(...)
>>> pred_int = conformal_forecaster.predict_interval()

recommended use of ConformalIntervals together with ForecastingGridSearch is by 1. first running grid search, 2. then ConformalIntervals on the tuned params otherwise, nested sliding windows will cause high compute requirement

>>> from sktime.datasets import load_airline
>>> from sktime.forecasting.conformal import ConformalIntervals
>>> from sktime.forecasting.naive import NaiveForecaster
>>> from sktime.forecasting.model_selection import ForecastingGridSearchCV
>>> from sktime.split import ExpandingWindowSplitter
>>> from sktime.param_est.plugin import PluginParamsForecaster
>>> # part 1 = grid search
>>> cv = ExpandingWindowSplitter(fh=[1, 2, 3])
>>> forecaster = NaiveForecaster()
>>> param_grid = {"strategy" : ["last", "mean", "drift"]}
>>> gscv = ForecastingGridSearchCV(
...     forecaster=forecaster,
...     param_grid=param_grid,
...     cv=cv,
... )
>>> # part 2 = plug in results of grid search into conformal intervals estimator
>>> conformal_with_fallback = ConformalIntervals(NaiveForecaster())
>>> gscv_with_conformal = PluginParamsForecaster(
...     gscv,
...     conformal_with_fallback,
...     params={"forecaster": "best_forecaster"},
... )
>>> y = load_airline()
>>> gscv_with_conformal.fit(y, fh=[1, 2, 3])
PluginParamsForecaster(...)
>>> y_pred_quantiles = gscv_with_conformal.predict_quantiles()

Methods

`check_is_fitted`([method_name])	Check if the estimator has been fitted.
`clone`()	Obtain a clone of the object with same hyper-parameters and config.
`clone_tags`(estimator[, tag_names])	Clone tags from another object as dynamic override.
`create_test_instance`([parameter_set])	Construct an instance of the class, using first test parameter set.
`create_test_instances_and_names`([parameter_set])	Create list of all test instances and a list of names for them.
`fit`(y[, X, fh])	Fit forecaster to training data.
`fit_predict`(y[, X, fh, X_pred])	Fit and forecast time series at future horizon.
`get_class_tag`(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
`get_class_tags`()	Get class tags from class, with tag level inheritance from parent classes.
`get_config`()	Get config flags for self.
`get_fitted_params`([deep])	Get fitted parameters.
`get_param_defaults`()	Get object's parameter defaults.
`get_param_names`([sort])	Get object's parameter names.
`get_params`([deep])	Get a dict of parameters values for this object.
`get_pretrained_params`([deep])	Get pretrained parameters of this estimator.
`get_tag`(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
`get_tags`()	Get tags from instance, with tag level inheritance and overrides.
`get_test_params`([parameter_set])	Return testing parameter settings for the estimator.
`is_composite`()	Check if the object is composed of other BaseObjects.
`load_from_path`(serial)	Load object from file location.
`load_from_serial`(serial)	Load object from serialized memory container.
`predict`([fh, X])	Forecast time series at future horizon.
`predict_interval`([fh, X, coverage])	Compute/return prediction interval forecasts.
`predict_proba`([fh, X, marginal])	Compute/return fully probabilistic forecasts.
`predict_quantiles`([fh, X, alpha])	Compute/return quantile forecasts.
`predict_residuals`([y, X])	Return residuals of time series forecasts.
`predict_var`([fh, X, cov])	Compute/return variance forecasts.
`pretrain`(y[, X, fh])	Pre-train forecaster on panel (global) data.
`reset`()	Reset the object to a clean post-init state.
`save`([path, serialization_format])	Save serialized self to bytes-like object or to (.zip) file.
`score`(y[, X, fh])	Scores forecast against ground truth, using MAPE (non-symmetric).
`set_config`(**config_dict)	Set config flags to given values.
`set_params`(**params)	Set the parameters of this object.
`set_random_state`([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
`set_tags`(**tag_dict)	Set instance level tag overrides to given values.
`update`(y[, X, update_params])	Update cutoff value and, optionally, fitted parameters.
`update_predict`(y[, cv, X, update_params, ...])	Make predictions and update model iteratively over the test set.
`update_predict_single`([y, fh, X, update_params])	Update model with new data and make forecasts.