ConformalIntervals
Empirical and conformal prediction intervals.
Implements empirical and conformal prediction intervals, on absolute residuals. Empirical prediction intervals are based on sliding window empirical quantiles. Conformal prediction intervals are implemented as described in [1].
All intervals wrap an arbitrary forecaster, i.e., add probabilistic prediction capability to a given point prediction forecaster (first argument).
- method=”conformal_bonferroni” is the method described in [1],
where an arbitrary forecaster is used instead of the RNN.
- method=”conformal” is the method in [1], but without Bonferroni correction.
i.e., separate forecasts are made which results in H=1 (at all horizons).
- method=”empirical” uses quantiles of relative signed residuals on training set,
i.e., y_t+h^(i) - y-hat_t+h^(i), ranging over i, in the notation of [1], at quantiles 0.5-0.5*coverage (lower) and 0.5+0.5*coverage (upper), as offsets to the point prediction at forecast horizon h
- method=”empirical_residual” uses empirical quantiles of absolute residuals
on the training set, i.e., quantiles of epsilon-h (in notation [1]), at quantile point (1-coverage)/2 quantiles, as offsets to point prediction
Quickstart
from sktime.forecasting.conformal import ConformalIntervals
estimator = ConformalIntervals(forecaster, method='empirical', initial_window=None, sample_frac=None, verbose=False, n_jobs=None)Parameters(6)
- forecasterestimator
- Estimator to which probabilistic forecasts are being added
- methodstr, optional, default=”empirical”
“empirical”: predictive interval bounds are empirical quantiles from training “empirical_residual”: upper/lower are plusminus (1-coverage)/2 quantiles
of the absolute residuals at horizon, i.e., of epsilon-h
- “conformal_bonferroni”: Bonferroni, as in Stankeviciute et al
Caveat: this does not give frequentist but conformal predictive intervals
- initial_windowfloat, int or None, optional (default=max(10, 0.1*len(y)))
- Defines the size of the initial training window If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include for the initial window for the train split. If int, represents the relative number of train samples in the initial window. If None, the value is set to the larger of 0.1*len(y) and 10
- sample_fracfloat, optional, default=None
- value in range (0,1) corresponding to fraction of y index to calculate residuals matrix values for (for speeding up calculation)
- verbosebool, optional, default=False
- whether to print warnings if windows with too few data points occur
- n_jobsint or None, optional, default=1
- The number of jobs to run in parallel for fit. -1 means using all processors.
Examples
>>> from sktime.datasets import load_airline
>>> from sktime.forecasting.conformal import ConformalIntervals
>>> from sktime.forecasting.naive import NaiveForecaster
>>> y = load_airline ()
>>> forecaster = NaiveForecaster (strategy = "drift")
>>> conformal_forecaster = ConformalIntervals (forecaster)
>>> conformal_forecaster. fit (y, fh = [1, 2, 3 ]) ConformalIntervals(
... )
>>> pred_int = conformal_forecaster. predict_interval () recommended use of ConformalIntervals together with ForecastingGridSearch is by 1. first running grid search, 2. then ConformalIntervals on the tuned params otherwise, nested sliding windows will cause high compute requirement
>>> from sktime.datasets import load_airline
>>> from sktime.forecasting.conformal import ConformalIntervals
>>> from sktime.forecasting.naive import NaiveForecaster
>>> from sktime.forecasting.model_selection import ForecastingGridSearchCV
>>> from sktime.split import ExpandingWindowSplitter
>>> from sktime.param_est.plugin import PluginParamsForecaster
>>> # part 1 = grid search
>>> cv = ExpandingWindowSplitter (fh = [1, 2, 3 ])
>>> forecaster = NaiveForecaster ()
>>> param_grid = { "strategy": ["last", "mean", "drift" ]}
>>> gscv = ForecastingGridSearchCV (
... forecaster = forecaster,
... param_grid = param_grid,
... cv = cv,
... )
>>> # part 2 = plug in results of grid search into conformal intervals estimator
>>> conformal_with_fallback = ConformalIntervals (NaiveForecaster ())
>>> gscv_with_conformal = PluginParamsForecaster (
... gscv,
... conformal_with_fallback,
... params = { "forecaster": "best_forecaster" },
... )
>>> y = load_airline ()
>>> gscv_with_conformal. fit (y, fh = [1, 2, 3 ]) PluginParamsForecaster(
... )
>>> y_pred_quantiles = gscv_with_conformal. predict_quantiles ()