PatchTSTForecaster

PatchTSTForecaster

class PatchTSTForecaster(model_path=None, fit_strategy='full', validation_split=0.2, config=None, training_args=None, compute_metrics=None, callbacks=None)[source]

Interface for the PatchTST forecaster.

This forecaster interfaces the Huggingface library’s PatchTST model for time series forecasting. The model was originally designed by Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong and Jayant Kalagnanam. It utilizes a transformer model architecture and splits the time series data into patches that are then processed by the model. Visit [1] for more information on the model architecture and its authors. For tips on how to construct your own PatchTST config, see [2].

The PatchTST forecaster can be used in three ways:

1) Full training via a new model initialized from a config or a loaded model with pre-trained weights 2) Minimal fine-tuning with a pre-trained model and an altered config 3) Zero-shot forecasting with a pre-trained model

For more details, please visit the fit_strategy parameter

Parameters:

model_pathstr or PatchTSTModel, optional

Path to the Huggingface model to use for global forecasting. If model_path is passed, the remaining model config parameters will be ignored except for specific training or dataset parameters. This has 3 options:

• model id to an online pretrained PatchTST Model hosted on HuggingFace

• A path or url to a saved configuration JSON file

• A path to a directory containing a configuration file saved

using the ~PretrainedConfig.save_pretrained method or the ~PreTrainedModel.save_pretrained method

fit_strategystr, values = [“full”,”minimal”,”zero-shot”], default = “full”

String to set the fit_strategy of the model.

• This strategy is used to create and train a new model from scratch

(pre-pretraining) or to update all of the weights in a pre-trained model (also known as full fine-tuning). If fit_strategy is set to full, requires either the model_path parameter or the config` parameter to be passed in, but not both. If only config is passed, it will initialize an new model with untrained weights with the specified config arguments. If only model_path is passed, it will fine-tune ALL of the pre-trained weights of the model.

• If fit_strategy is set to “minimal” requires both the model_path

and config parameter. We will use the model_path and the specified config to compare the weight shapes of the passed pre-trained model to those in the config. If there are weight size mismatches, the model will reinitialize new weights to match the weight shapes inside the config. The y argument will then be fit to fine-tune the model. In the case where there are no newly initialized weights (i.e the config weight shapes match the pretrained model weight shapes), it will behave the same as the “full” strategy where only the model_path is passed in.

• If fit_strategy is set to “zero-shot”, requires only the model_path

parameter. It will load the model via the fit function with the argument model_path and ignore any passed y.

validation_splitfloat, optional, default = 0.2

Fraction of the data to use for validation.

configdict, optional, default = {}

A config dict specifying parameters to initialize an full PatchTST model. Missing parameters in the config will be automatically replaced by their default values. See the PatchTSTConfig config on huggingface for more details. Note: if prediction_length is passed as in larger than the passed fh in the fit function, the prediction_length will be used to train the model. If prediction_length is passed as in smaller than the passed fh in the fit function, the passed fh will be used to train the model.

training_argsdict, optional, default = None

Training arguments to use for the model. If this is passed, the remaining applicable training arguments will be ignored

compute_metricslist or function, default = None

List of metrics or function to use during training

callbacks: list or function, default = None

List of callbacks or callback function to use during training

Attributes:

cutoff

Cut-off = “present time” state of forecaster.

fh

Forecasting horizon that was passed.

is_fitted

Whether fit has been called.

state

State of the estimator.

References

[1] A Time Series is Worth 64 Words: Long-term Forecasting with Transformers

Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, Jayant Kalagnanam Paper: https://arxiv.org/abs/2211.14730

[2] HuggingFace PatchTST Page:

https://huggingface.co/docs/transformers/en/model_doc/patchtst

Examples

>>> #Example with a new model initialized from config only
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> from sktime.datasets import load_airline
>>> y = load_airline()
>>> forecaster = PatchTSTForecaster(
... config = {
...     "patch_length": 1,
...      "context_length": 2,
...      "patch_stride": 1,
...      "d_model": 64,
...      "num_attention_heads": 2,
...      "ffn_dim": 32,
...      "head_dropout": 0.3,
...    },
...    training_args = {
...         "output_dir":"test_output",
...         "overwrite_output_dir":True,
...         "learning_rate":1e-4,
...         "num_train_epochs":1,
...         "per_device_train_batch_size":16,
...    }
... ) #initialize an full model
>>> forecaster.fit(y, fh=[1, 2, 3])
>>> y_pred = forecaster.predict()

>>> #Example full fine-tuning with a pre-trained model
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> import pandas as pd
>>> dataset_path = pd.read_csv(
...     "https://raw.githubusercontent.com/zhouhaoyi/ETDataset/main/ETT-small/ETTh1.csv"
...     ).drop(columns = ["date"]
... )
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler()
>>> scaler.set_output(transform="pandas")
>>> scaler = scaler.fit(dataset_path.values)
>>> df = scaler.transform(dataset_path)
>>> df.columns = dataset_path.columns
>>> forecaster = PatchTSTForecaster(
...     model_path="namctin/patchtst_etth1_forecast",
...     fit_strategy = "full",
...     training_args = {
...         "output_dir":"test_output",
...         "overwrite_output_dir":True,
...         "learning_rate":1e-4,
...         "num_train_epochs":1,
...         "per_device_train_batch_size":16,
...     }
... )
>>> forecaster.fit(y = df, fh = list(range(1,4)))
>>> y_pred = forecaster.predict()

>>> #Example of minimal fine-tuning with a pre-trained model and an altered config
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> import pandas as pd
>>> dataset_path = pd.read_csv(
...     "https://raw.githubusercontent.com/zhouhaoyi/ETDataset/main/ETT-small/ETTh1.csv"
...     ).drop(columns = ["date"]
... )
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler()
>>> scaler.set_output(transform="pandas")
>>> scaler = scaler.fit(dataset_path.values)
>>> df = scaler.transform(dataset_path)
>>> df.columns = dataset_path.columns
>>> forecaster = PatchTSTForecaster(
...     model_path="namctin/patchtst_etth1_forecast",
...     config = {
...         "patch_length": 8,
...         "context_length": 512,
...         "patch_stride": 8,
...         "d_model": 128,
...         "num_attention_heads": 2,
...         "ffn_dim": 512,
...         "head_dropout": 0.3,
...         "prediction_length": 64
...     },
...     fit_strategy = "minimal",
...     training_args = {
...         "output_dir":"test_output",
...         "overwrite_output_dir":True,
...         "learning_rate":1e-4,
...         "num_train_epochs":1,
...         "per_device_train_batch_size":16,
...     }
... )
>>> forecaster.fit(y = df, fh = list(range(1,63)))
>>> y_pred = forecaster.predict()

>>> #Example with a pre-trained model to do zero-shot forecasting
>>> from sktime.forecasting.patch_tst import PatchTSTForecaster
>>> import pandas as pd
>>> dataset_path = pd.read_csv(
...     "https://raw.githubusercontent.com/zhouhaoyi/ETDataset/main/ETT-small/ETTh1.csv"
...     ).drop(columns = ["date"]
... )
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler()
>>> scaler.set_output(transform="pandas")
>>> scaler = scaler.fit(dataset_path.values)
>>> df = scaler.transform(dataset_path)
>>> df.columns = dataset_path.columns
>>> forecaster = PatchTSTForecaster(
...     model_path="namctin/patchtst_etth1_forecast",
...     fit_strategy = "zero-shot",
...     training_args = {
...         "output_dir":"test_output",
...         "overwrite_output_dir":True,
...         "learning_rate":1e-4,
...         "num_train_epochs":1,
...         "per_device_train_batch_size":16,
...     }
... )
>>> forecaster.fit(y = df, fh = [1,2,3,4,5])
>>> y_pred = forecaster.predict()

Methods

check_is_fitted([method_name])	Check if the estimator has been fitted.
clone()	Obtain a clone of the object with same hyper-parameters and config.
clone_tags(estimator[, tag_names])	Clone tags from another object as dynamic override.
create_test_instance([parameter_set])	Construct an instance of the class, using first test parameter set.
create_test_instances_and_names([parameter_set])	Create list of all test instances and a list of names for them.
fit(y[, X, fh])	Fit forecaster to training data.
fit_predict(y[, X, fh, X_pred])	Fit and forecast time series at future horizon.
get_class_tag(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
get_class_tags()	Get class tags from class, with tag level inheritance from parent classes.
get_config()	Get config flags for self.
get_fitted_params([deep])	Get fitted parameters.
get_param_defaults()	Get object's parameter defaults.
get_param_names([sort])	Get object's parameter names.
get_params([deep])	Get a dict of parameters values for this object.
get_pretrained_params([deep])	Get pretrained parameters of this estimator.
get_tag(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
get_tags()	Get tags from instance, with tag level inheritance and overrides.
get_test_params([parameter_set])	Return testing parameter settings for the estimator.
is_composite()	Check if the object is composed of other BaseObjects.
load_from_path(serial)	Load object from file location.
load_from_serial(serial)	Load object from serialized memory container.
predict([fh, X])	Forecast time series at future horizon.
predict_interval([fh, X, coverage])	Compute/return prediction interval forecasts.
predict_proba([fh, X, marginal])	Compute/return fully probabilistic forecasts.
predict_quantiles([fh, X, alpha])	Compute/return quantile forecasts.
predict_residuals([y, X])	Return residuals of time series forecasts.
predict_var([fh, X, cov])	Compute/return variance forecasts.
pretrain(y[, X, fh])	Pre-train forecaster on panel (global) data.
reset()	Reset the object to a clean post-init state.
save([path, serialization_format])	Save serialized self to bytes-like object or to (.zip) file.
score(y[, X, fh])	Scores forecast against ground truth, using MAPE (non-symmetric).
set_config(**config_dict)	Set config flags to given values.
set_params(**params)	Set the parameters of this object.
set_random_state([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
set_tags(**tag_dict)	Set instance level tag overrides to given values.
update(y[, X, update_params])	Update cutoff value and, optionally, fitted parameters.
update_predict(y[, cv, X, update_params, ...])	Make predictions and update model iteratively over the test set.
update_predict_single([y, fh, X, update_params])	Update model with new data and make forecasts.