HFTransformersForecaster

class HFTransformersForecaster(model_path: str = None, fit_strategy='minimal', validation_split=0.2, config=None, training_args=None, compute_metrics=None, deterministic=False, callbacks=None, peft_config=None)[source]

Forecaster that uses a huggingface model for forecasting.

This forecaster fetches the model from the huggingface model hub. Note, this forecaster is in an experimental state. It is currently only working for Informer, Autoformer, and TimeSeriesTransformer.

Parameters:

model_pathstr or PreTrainedModel

Path to the huggingface model to use for forecasting. Currently, Informer, Autoformer, and TimeSeriesTransformer are supported. This can be one of the following: - A string specifying the Hugging Face model name or path

(e.g., “huggingface/autoformer-tourism-monthly”).

An instance of a PreTrainedModel, allowing manual initialization and configuration.

fit_strategystr, default=”minimal”

Strategy to use for fitting (fine-tuning) the model. This can be one of the following:

“minimal”: Fine-tunes only a small subset of the model parameters, allowing for quick adaptation with limited computational resources.
“full”: Fine-tunes all model parameters, which may result in better performance but requires more computational power and time.
“peft”: Applies Parameter-Efficient Fine-Tuning (PEFT) techniques to adapt the model with fewer trainable parameters, saving computational resources.

Note: If the ‘peft’ package is not available, a ModuleNotFoundError will be raised, indicating that the ‘peft’ package is required. Please install it using pip install peft to use this fit strategy.

validation_splitfloat, default=0.2

Fraction of the data to use for validation

configdict, default={}

Configuration to use for the model. See the transformers documentation for details.

training_argsdict, default={}

Training arguments to use for the model. See transformers.TrainingArguments for details. Note that the output_dir argument is required.

compute_metricslist, default=None

List of metrics to compute during training. See transformers.Trainer for details.

deterministicbool, default=False

Whether the predictions should be deterministic or not.

callbackslist, default=[]

List of callbacks to use during training. See transformers.Trainer

peft_configpeft.PeftConfig, default=None

Configuration for Parameter-Efficient Fine-Tuning. When fit_strategy is set to “peft”, this will be used to set up PEFT parameters for the model. See the peft documentation for details.

Attributes:

cutoff: Cut-off = “present time” state of forecaster.
fh: Forecasting horizon that was passed.
is_fitted: Whether fit has been called.
state: State of the estimator.

Examples

Using a Pretrained Model from Hugging Face

>>> from sktime.forecasting.hf_transformers import HFTransformersForecaster
>>> from sktime.datasets import load_airline
>>> y = load_airline()
>>> forecaster = HFTransformersForecaster(
...    model_path="huggingface/autoformer-tourism-monthly",
...    training_args ={
...        "num_train_epochs": 20,
...        "output_dir": "test_output",
...        "per_device_train_batch_size": 32,
...    },
...    config={
...         "lags_sequence": [1, 2, 3],
...         "context_length": 2,
...         "prediction_length": 4,
...         "use_cpu": True,
...         "label_length": 2,
...    },
... )
>>> forecaster.fit(y)
>>> fh = [1, 2, 3]
>>> y_pred = forecaster.predict(fh)

Using PEFT for Fine-Tuning

>>> from sktime.forecasting.hf_transformers import HFTransformersForecaster
>>> from sktime.datasets import load_airline
>>> from peft import LoraConfig
>>> y = load_airline()
>>> forecaster = HFTransformersForecaster(
...    model_path="huggingface/autoformer-tourism-monthly",
...    fit_strategy="peft",
...    training_args={
...        "num_train_epochs": 20,
...        "output_dir": "test_output",
...        "per_device_train_batch_size": 32,
...    },
...    config={
...         "lags_sequence": [1, 2, 3],
...         "context_length": 2,
...         "prediction_length": 4,
...         "use_cpu": True,
...         "label_length": 2,
...    },
...    peft_config=LoraConfig(
...        r=8,
...        lora_alpha=32,
...        target_modules=["q_proj", "v_proj"],
...        lora_dropout=0.01,
...    )
... )
>>> forecaster.fit(y)
>>> fh = [1, 2, 3]
>>> y_pred = forecaster.predict(fh)

Using an Initialized Model

>>> from sktime.datasets import load_airline
>>> from transformers import AutoformerConfig, AutoformerForPrediction
>>> from sktime.forecasting.hf_transformers import HFTransformersForecaster
>>> y = load_airline()

>>> # Define model configuration
>>> config = AutoformerConfig(
...     num_dynamic_real_features=0,
...     num_static_real_features=0,
...     num_static_categorical_features=0,
...     num_time_features=0,
...     context_length=32,
...     prediction_length=8,
...     lags_sequence=[1, 2, 3],
... )

>>> # Initialize the model
>>> model = AutoformerForPrediction(config)

>>> # Initialize the forecaster with the model
>>> forecaster = HFTransformersForecaster(
...     model_path=model,
...     fit_strategy="minimal",
...     training_args={
...         "num_train_epochs": 10,
...         "output_dir": "output",
...         "per_device_train_batch_size": 4
...     },
... )

>>> forecaster.fit(y)
>>> fh = [1, 2, 3]
>>> y_pred = forecaster.predict(fh)

Methods

`check_is_fitted`([method_name])	Check if the estimator has been fitted.
`clone`()	Obtain a clone of the object with same hyper-parameters and config.
`clone_tags`(estimator[, tag_names])	Clone tags from another object as dynamic override.
`create_test_instance`([parameter_set])	Construct an instance of the class, using first test parameter set.
`create_test_instances_and_names`([parameter_set])	Create list of all test instances and a list of names for them.
`fit`(y[, X, fh])	Fit forecaster to training data.
`fit_predict`(y[, X, fh, X_pred])	Fit and forecast time series at future horizon.
`get_class_tag`(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
`get_class_tags`()	Get class tags from class, with tag level inheritance from parent classes.
`get_config`()	Get config flags for self.
`get_fitted_params`([deep])	Get fitted parameters.
`get_param_defaults`()	Get object's parameter defaults.
`get_param_names`([sort])	Get object's parameter names.
`get_params`([deep])	Get a dict of parameters values for this object.
`get_pretrained_params`([deep])	Get pretrained parameters of this estimator.
`get_tag`(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
`get_tags`()	Get tags from instance, with tag level inheritance and overrides.
`get_test_params`([parameter_set])	Return testing parameter settings for the estimator.
`is_composite`()	Check if the object is composed of other BaseObjects.
`load_from_path`(serial)	Load object from file location.
`load_from_serial`(serial)	Load object from serialized memory container.
`predict`([fh, X])	Forecast time series at future horizon.
`predict_interval`([fh, X, coverage])	Compute/return prediction interval forecasts.
`predict_proba`([fh, X, marginal])	Compute/return fully probabilistic forecasts.
`predict_quantiles`([fh, X, alpha])	Compute/return quantile forecasts.
`predict_residuals`([y, X])	Return residuals of time series forecasts.
`predict_var`([fh, X, cov])	Compute/return variance forecasts.
`pretrain`(y[, X, fh])	Pre-train forecaster on panel (global) data.
`reset`()	Reset the object to a clean post-init state.
`save`([path, serialization_format])	Save serialized self to bytes-like object or to (.zip) file.
`score`(y[, X, fh])	Scores forecast against ground truth, using MAPE (non-symmetric).
`set_config`(**config_dict)	Set config flags to given values.
`set_params`(**params)	Set the parameters of this object.
`set_random_state`([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
`set_tags`(**tag_dict)	Set instance level tag overrides to given values.
`update`(y[, X, update_params])	Update cutoff value and, optionally, fitted parameters.
`update_predict`(y[, cv, X, update_params, ...])	Make predictions and update model iteratively over the test set.
`update_predict_single`([y, fh, X, update_params])	Update model with new data and make forecasts.