Back to models
Forecaster

PatchTSTForecaster

API referenceSource
Categorical in XMultivariate

Interface for the PatchTST forecaster.

Quickstart

python
from sktime.forecasting.patch_tst import PatchTSTForecaster

estimator = PatchTSTForecaster(model_path=None, fit_strategy='full', validation_split=0.2, config=None, training_args=None, compute_metrics=None, callbacks=None)

Parameters(7)

model_pathstr or PatchTSTModel, optional

Path to the Huggingface model to use for global forecasting. If model_path is passed, the remaining model config parameters will be ignored except for specific training or dataset parameters. This has 3 options:

  • model id to an online pretrained PatchTST Model hosted on HuggingFace

  • A path or url to a saved configuration JSON file

  • A path to a directory containing a configuration file saved

using the ~PretrainedConfig.save_pretrained method or the ~PreTrainedModel.save_pretrained method

fit_strategystr, values = [“full”,”minimal”,”zero-shot”], default = “full”

String to set the fit_strategy of the model.

  • This strategy is used to create and train a new model from scratch

(pre-pretraining) or to update all of the weights in a pre-trained model (also known as full fine-tuning). If fit_strategy is set to full, requires either the model_path parameter or the config` parameter to be passed in, but not both. If only config is passed, it will initialize an new model with untrained weights with the specified config arguments. If only model_path is passed, it will fine-tune ALL of the pre-trained weights of the model.

  • If fit_strategy is set to “minimal” requires both the model_path

and config parameter. We will use the model_path and the specified config to compare the weight shapes of the passed pre-trained model to those in the config. If there are weight size mismatches, the model will reinitialize new weights to match the weight shapes inside the config. The y argument will then be fit to fine-tune the model. In the case where there are no newly initialized weights (i.e the config weight shapes match the pretrained model weight shapes), it will behave the same as the “full” strategy where only the model_path is passed in.

  • If fit_strategy is set to “zero-shot”, requires only the model_path

parameter. It will load the model via the fit function with the argument model_path and ignore any passed y.

validation_splitfloat, optional, default = 0.2
Fraction of the data to use for validation.
configdict, optional, default = {}
A config dict specifying parameters to initialize an full PatchTST model. Missing parameters in the config will be automatically replaced by their default values. See the PatchTSTConfig config on huggingface for more details. Note: if prediction_length is passed as in larger than the passed fh in the fit function, the prediction_length will be used to train the model. If prediction_length is passed as in smaller than the passed fh in the fit function, the passed fh will be used to train the model.
training_argsdict, optional, default = None
Training arguments to use for the model. If this is passed, the remaining applicable training arguments will be ignored
compute_metricslist or function, default = None
List of metrics or function to use during training
callbacks: list or function, default = None
List of callbacks or callback function to use during training

Examples

>>> #Example with a new model initialized from config only
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> from sktime.datasets import load_airline
>>> y = load_airline ()
>>> forecaster = PatchTSTForecaster (
... config = {
... "patch_length": 1,
... "context_length": 2,
... "patch_stride": 1,
... "d_model": 64,
... "num_attention_heads": 2,
... "ffn_dim": 32,
... "head_dropout": 0.3,
... },
... training_args = {
... "output_dir": "test_output",
... "overwrite_output_dir": True,
... "learning_rate": 1e-4,
... "num_train_epochs": 1,
... "per_device_train_batch_size": 16,
... }
... ) #initialize an full model
>>> forecaster. fit (y, fh = [1, 2, 3 ])
>>> y_pred = forecaster. predict ()
>>> #Example full fine-tuning with a pre-trained model
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> import pandas as pd
>>> dataset_path = pd. read_csv (
... "https://raw.githubusercontent.com/zhouhaoyi/ETDataset/main/ETT-small/ETTh1.csv"
... ). drop (columns = ["date" ]
... )
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler ()
>>> scaler. set_output (transform = "pandas")
>>> scaler = scaler. fit (dataset_path. values)
>>> df = scaler. transform (dataset_path)
>>> df. columns = dataset_path. columns
>>> forecaster = PatchTSTForecaster (
... model_path = "namctin/patchtst_etth1_forecast",
... fit_strategy = "full",
... training_args = {
... "output_dir": "test_output",
... "overwrite_output_dir": True,
... "learning_rate": 1e-4,
... "num_train_epochs": 1,
... "per_device_train_batch_size": 16,
... }
... )
>>> forecaster. fit (y = df, fh = list (range (1, 4)))
>>> y_pred = forecaster. predict ()
>>> #Example of minimal fine-tuning with a pre-trained model and an altered config
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> import pandas as pd
>>> dataset_path = pd. read_csv (
... "https://raw.githubusercontent.com/zhouhaoyi/ETDataset/main/ETT-small/ETTh1.csv"
... ). drop (columns = ["date" ]
... )
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler ()
>>> scaler. set_output (transform = "pandas")
>>> scaler = scaler. fit (dataset_path. values)
>>> df = scaler. transform (dataset_path)
>>> df. columns = dataset_path. columns
>>> forecaster = PatchTSTForecaster (
... model_path = "namctin/patchtst_etth1_forecast",
... config = {
... "patch_length": 8,
... "context_length": 512,
... "patch_stride": 8,
... "d_model": 128,
... "num_attention_heads": 2,
... "ffn_dim": 512,
... "head_dropout": 0.3,
... "prediction_length": 64
... },
... fit_strategy = "minimal",
... training_args = {
... "output_dir": "test_output",
... "overwrite_output_dir": True,
... "learning_rate": 1e-4,
... "num_train_epochs": 1,
... "per_device_train_batch_size": 16,
... }
... )
>>> forecaster. fit (y = df, fh = list (range (1, 63)))
>>> y_pred = forecaster. predict ()
>>> #Example with a pre-trained model to do zero-shot forecasting
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> import pandas as pd
>>> dataset_path = pd. read_csv (
... "https://raw.githubusercontent.com/zhouhaoyi/ETDataset/main/ETT-small/ETTh1.csv"
... ). drop (columns = ["date" ]
... )
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler ()
>>> scaler. set_output (transform = "pandas")
>>> scaler = scaler. fit (dataset_path. values)
>>> df = scaler. transform (dataset_path)
>>> df. columns = dataset_path. columns
>>> forecaster = PatchTSTForecaster (
... model_path = "namctin/patchtst_etth1_forecast",
... fit_strategy = "zero-shot",
... training_args = {
... "output_dir": "test_output",
... "overwrite_output_dir": True,
... "learning_rate": 1e-4,
... "num_train_epochs": 1,
... "per_device_train_batch_size": 16,
... }
... )
>>> forecaster. fit (y = df, fh = [1, 2, 3, 4, 5 ])
>>> y_pred = forecaster. predict ()

References

  1. [1] A Time Series is Worth 64 Words: Long-term Forecasting with Transformers Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, Jayant Kalagnanam Paper: https://arxiv.org/abs/2211.14730
  2. [2] HuggingFace PatchTST Page: https://huggingface.co/docs/transformers/en/model_doc/patchtst