HFTransformersForecaster

Forecaster that uses a huggingface model for forecasting.

Quickstart

from sktime.forecasting.hf_transformers import HFTransformersForecaster

estimator = HFTransformersForecaster(model_path: str=None, fit_strategy='minimal', validation_split=0.2, config=None, training_args=None, compute_metrics=None, deterministic=False, callbacks=None, peft_config=None)

Parameters(9)

model_pathstr or PreTrainedModel

Path to the huggingface model to use for forecasting. Currently, Informer, Autoformer, and TimeSeriesTransformer are supported. This can be one of the following: - A string specifying the Hugging Face model name or path

(e.g., “huggingface/autoformer-tourism-monthly”).

• An instance of a PreTrainedModel, allowing manual initialization and configuration.

fit_strategystr, default=”minimal”

Strategy to use for fitting (fine-tuning) the model. This can be one of the following:

• “minimal”: Fine-tunes only a small subset of the model parameters, allowing for quick adaptation with limited computational resources.

• “full”: Fine-tunes all model parameters, which may result in better performance but requires more computational power and time.

• “peft”: Applies Parameter-Efficient Fine-Tuning (PEFT) techniques to adapt the model with fewer trainable parameters, saving computational resources.

  Note: If the ‘peft’ package is not available, a ModuleNotFoundError will be raised, indicating that the ‘peft’ package is required. Please install it using pip install peft to use this fit strategy.

validation_splitfloat, default=0.2

Fraction of the data to use for validation

configdict, default={}

Configuration to use for the model. See the transformers documentation for details.

training_argsdict, default={}

Training arguments to use for the model. See transformers.TrainingArguments for details. Note that the output_dir argument is required.

compute_metricslist, default=None

List of metrics to compute during training. See transformers.Trainer for details.

deterministicbool, default=False

Whether the predictions should be deterministic or not.

callbackslist, default=[]

List of callbacks to use during training. See transformers.Trainer

peft_configpeft.PeftConfig, default=None

Configuration for Parameter-Efficient Fine-Tuning. When fit_strategy is set to “peft”, this will be used to set up PEFT parameters for the model. See the peft documentation for details.

Examples

Using a Pretrained Model from Hugging Face
>>> from sktime.forecasting.hf_transformers import HFTransformersForecaster
>>> from sktime.datasets import load_airline
>>> y = load_airline ()
>>> forecaster = HFTransformersForecaster (
... model_path = "huggingface/autoformer-tourism-monthly",
... training_args = {
... "num_train_epochs": 20,
... "output_dir": "test_output",
... "per_device_train_batch_size": 32,
... },
... config = {
... "lags_sequence": [1, 2, 3 ],
... "context_length": 2,
... "prediction_length": 4,
... "use_cpu": True,
... "label_length": 2,
... },
... )
>>> forecaster. fit (y)
>>> fh = [1, 2, 3 ]
>>> y_pred = forecaster. predict (fh) Using PEFT for Fine-Tuning
>>> from sktime.forecasting.hf_transformers import HFTransformersForecaster
>>> from sktime.datasets import load_airline
>>> from peft import LoraConfig
>>> y = load_airline ()
>>> forecaster = HFTransformersForecaster (
... model_path = "huggingface/autoformer-tourism-monthly",
... fit_strategy = "peft",
... training_args = {
... "num_train_epochs": 20,
... "output_dir": "test_output",
... "per_device_train_batch_size": 32,
... },
... config = {
... "lags_sequence": [1, 2, 3 ],
... "context_length": 2,
... "prediction_length": 4,
... "use_cpu": True,
... "label_length": 2,
... },
... peft_config = LoraConfig (
... r = 8,
... lora_alpha = 32,
... target_modules = ["q_proj", "v_proj" ],
... lora_dropout = 0.01,
... )
... )
>>> forecaster. fit (y)
>>> fh = [1, 2, 3 ]
>>> y_pred = forecaster. predict (fh) Using an Initialized Model
>>> from sktime.datasets import load_airline
>>> from transformers import AutoformerConfig, AutoformerForPrediction
>>> from sktime.forecasting.hf_transformers import HFTransformersForecaster
>>> y = load_airline ()
>>> # Define model configuration
>>> config = AutoformerConfig (
... num_dynamic_real_features = 0,
... num_static_real_features = 0,
... num_static_categorical_features = 0,
... num_time_features = 0,
... context_length = 32,
... prediction_length = 8,
... lags_sequence = [1, 2, 3 ],
... )
>>> # Initialize the model
>>> model = AutoformerForPrediction (config)
>>> # Initialize the forecaster with the model
>>> forecaster = HFTransformersForecaster (
... model_path = model,
... fit_strategy = "minimal",
... training_args = {
... "num_train_epochs": 10,
... "output_dir": "output",
... "per_device_train_batch_size": 4
... },
... )
>>> forecaster. fit (y)
>>> fh = [1, 2, 3 ]
>>> y_pred = forecaster. predict (fh)