Skip to content
sktime

TinyTimeMixerForecaster

TinyTimeMixerForecaster

class TinyTimeMixerForecaster(model_path='ibm/TTM', revision='main', validation_split=0.2, config=None, training_args=None, compute_metrics=None, callbacks=None, broadcasting=False, use_source_package=False, fit_strategy='minimal', device='cpu', freq=None, verbose=False, padding_mask='observed')[source]

TinyTimeMixer Forecaster for Zero-Shot Forecasting of Multivariate Time Series.

Wrapping implementation in [1] of method proposed in [2]. See [3] for tutorial by creators.

TinyTimeMixer (TTM) are compact pre-trained models for Time-Series Forecasting, open-sourced by IBM Research. With less than 1 Million parameters, TTM introduces the notion of the first-ever “tiny” pre-trained models for Time-Series Forecasting.

Fit Strategies: Full, Minimal, and Zero-shot

This model supports three fit strategies: zero-shot for direct predictions without training, minimal fine-tuning for lightweight adaptation to new data, and full fine-tuning for comprehensive model training. The selected strategy is determined by the model’s fit_strategy parameter

Initialization Process:

  1. Model Path: The model_path parameter points to a local folder or Hugging Face repo that contains both configuration files and pretrained weights. Public TinyTimeMixer checkpoints are available for the R1 [4], R2 [5], research R2 [6], and R3 [7] model families.

  2. Default Configuration: The model loads its default configuration from the configuration files.

  3. Custom Configuration: Users can provide a custom configuration via the config parameter during model initialization.

  4. Configuration Override: If custom configuration is provided, it overrides the default configuration.

  5. Forecasting Horizon: If revision=None, the forecasting horizon (fh) specified during fit is used to select a compatible checkpoint revision. For prediction, the requested horizon must fit within the loaded model’s config.prediction_length.

  6. Model Architecture: The final configuration is used to construct the model architecture.

  7. Pretrained Weights: pretrained weights are loaded from the model_path, these weights are then aligned and loaded into the model architecture.

  8. Weight Alignment: However sometimes, pretrained weights do not align with the model architecture, because the config was changed which created a model architecture of different size than the default one. This causes some of the weights in model architecture to be reinitialized randomly instead of using the pre-trained weights.

Training Strategies:

  • Zero-shot Forecasting: When all the pre-trained weights are correctly aligned with the model architecture, fine-tuing part is bypassed and the model preforms zero-short forecasting.

  • Minimal Fine-tuning: When not all the pre-trained weights are correctly aligned with the model architecture, rather some weights are re-initialized, these re-initialized weights are fine-tuned on the provided data.

  • Full Fine-tuning: The model is fully fine-tuned on new data, updating all parameters. This approach offers maximum adaptation to the dataset but requires more computational resources.

Exogenous Variables Support

TTM supports exogenous variables (external factors) that can improve forecasting accuracy. The model accepts exogenous variables that are known for both the historical period and the future forecasting horizon.

When using exogenous variables: - The X parameter should contain exogenous data covering both

past and future periods

  • Exogenous variables must have the same index structure as the target series

  • For prediction, exogenous data must extend into the forecasting horizon

Parameters:
model_pathstr, default=”ibm/TTM”

Path to the Hugging Face model to use for forecasting. This can be either:

  • The name of a Hugging Face repository, for example "ibm-research/ttm-r3" [7]. Related checkpoints are listed in the TinyTimeMixer [8], Granite time series [9], and IBM Research time series [10] collections.

  • A local path to a folder containing model files in a format supported by transformers. In this case, ensure that the directory contains all necessary files (e.g., configuration, tokenizer, and model weights).

  • If this parameter is None, fit_strategy should be full to allow training a randomly initialized model from the provided or default config, else ValueError is raised.

revisionstr or None, default=”main”

Revision of the model to use:

  • None: Automatically select a compatible revision based on the training context length and forecasting horizon.

  • “main”: Load the main branch of the selected checkpoint.

  • A checkpoint branch name such as “52-16-ft-r2.1” can be used to load a specific TinyTimeMixer variant.

This param becomes irrelevant when model_path is None.

validation_splitfloat, default=0.2

Fraction of the data to use for validation

configdict or None, default={}

Configuration to use for the model. See the transformers documentation for details. The provided configuration must be valid for the selected TinyTimeMixer model architecture. Configuration inherits from transformers.PretrainedConfig and can be used to control the model outputs.

context_length (int, optional, defaults to 64)

The context/history length for the input sequence.

patch_length (int, optional, defaults to 8)

The patch length for the input sequence.

num_input_channels (int):

Number of input variates. For Univariate, set it to 1.

patch_stride (int, optional, defaults to 8):

Amount of points to stride. If its value is same as patch_length, we get non-overlapping patches.

d_model (int, optional, defaults to 16):

Hidden feature size of the model.

prediction_length (int, optional, defaults to 16)

Number of time steps to forecast for a forecasting task. Also known as the Forecast Horizon.

num_parallel_samples (int, optional, defaults to 100):

The number of samples to generate in parallel for probabilistic forecast.

expansion_factor (int, optional, defaults to 2):

Expansion factor to use inside MLP. Recommended range is 2-5. Larger value indicates more complex model.

num_layers (int, optional, defaults to 3):

Number of layers to use. Recommended range is 3-15. Larger value indicates more complex model.

dropout (float, optional, defaults to 0.2):

The dropout probability the TinyTimeMixer backbone. Recommended range is 0.2-0.7

mode (str, optional, defaults to “common_channel”):

Mixer Mode. Determines how to process the channels. Allowed values: “common_channel”, “mix_channel”. In “common_channel” mode, we follow Channel-independent modelling with no explicit channel-mixing. Channel mixing happens in an implicit manner via shared weights across channels. (preferred first approach) In “mix_channel” mode, we follow explicit channel-mixing in addition to patch and feature mixer. (preferred approach when channel correlations are very important to model)

gated_attn (bool, optional, defaults to True):

Enable Gated Attention.

norm_mlp (str, optional, defaults to “LayerNorm”):

Normalization layer (BatchNorm or LayerNorm).

self_attn (bool, optional, defaults to False):

Enable Tiny self attention across patches. This can be enabled when the output of Vanilla TinyTimeMixer with gated attention is not satisfactory. Enabling this leads to explicit pair-wise attention and modelling across patches.

self_attn_heads (int, optional, defaults to 1):

Number of self-attention heads. Works only when self_attn is set to True.

use_positional_encoding (bool, optional, defaults to False):

Enable the use of positional embedding for the tiny self-attention layers. Works only when self_attn is set to True.

positional_encoding_type (str, optional, defaults to “sincos”):

Positional encodings. Options “random” and “sincos” are supported. Works only when use_positional_encoding is set to True

scaling (string or bool, optional, defaults to “std”):

Whether to scale the input targets via “mean” scaler, “std” scaler or no scaler if None. If True, the scaler is set to “mean”.

loss (string, optional, defaults to “mse”):

The loss function to finetune or pretrain the the model. Allowed values are “mse” or “mae” or “pinball” or “huber”. Use pinball loss for probabilistic forecasts of different quantiles. Distribution head (nll) is currently disabled and not allowed.

init_std (float, optional, defaults to 0.02):

The standard deviation of the truncated normal weight initialization distribution.

post_init (bool, optional, defaults to False):

Whether to use custom weight initialization from transformers library, or the default initialization in PyTorch. Setting it to False performs PyTorch weight initialization.

norm_eps (float, optional, defaults to 1e-05):

A value added to the denominator for numerical stability of normalization.

adaptive_patching_levels (int, optional, defaults to 0):

If adaptive_patching_levels is i, then we will have i levels with each level having n_layers. Level id starts with 0. num_patches at level i will be multipled by (2^i) and num_features at level i will be divided by (2^i). For Ex. if adaptive_patching_levels is 3 - then we will have 3 levels:

level 2: num_features//(2^2), num_patches*(2^2) level 1: num_features//(2^1), num_patches*(2^1) level 0: num_features//(2^0), num_patches*(2^0)

adaptive_patching_levels = 1 is same as one level PatchTSMixer. This module gets disabled when adaptive_patching_levels is 0 or neg value. Defaults to 0 (off mode).

resolution_prefix_tuning (bool, optional, defaults to False):

Enable if your dataloader has time resolution information as defined in get_freq_mapping function in modelling_tinytimemixer.

frequency_token_vocab_size (int, optional, defaults to 5):

Vocab size to use when resolution_prefix_tuning is enabled.

head_dropout (float, optional, defaults to 0.2):

The dropout probability the TinyTimeMixer head.

distribution_output (string, optional, defaults to “student_t”):

The distribution emission head for the model when loss is “nll”. Could be either “student_t”, “normal” or “negative_binomial”.

prediction_channel_indices (list, optional):

List of channel indices to forecast. If None, forecast all channels. Target data is expected to have all channels and we explicitly filter the channels in prediction and target before loss computation. Please provide the indices in sorted ascending order.

exogenous_channel_indices (list, optional):

List of channel indices whose values are known in the forecast period. Please provide the indices in sorted ascending order.

decoder_num_layers (int, optional, defaults to 8):

Number of layers to use in decoder

decoder_d_model(int, optional, defaults to 16):

Defines the hidden feature size of the decoder.

decoder_adaptive_patching_levels (int, optional, defaults to 0):

Adaptive Patching levels for decoder. Preferable to set it to 0 for decoder to keep it light weight.

decoder_raw_residual (bool, optional, defaults to False):

Flag to enable merging of raw embedding with encoder embedding for decoder input. Defaults to False.

decoder_mode (string, optional, defaults to “common_channel”):

Decoder channel mode. Use “common_channel” for channel-independent modelling and `”mix_channel” for channel-mixing modelling

use_decoder (bool, optional, defaults to True):

Enable to use decoder.

enable_forecast_channel_mixing (bool, optional, defaults to False):

Enable if we want to reconcile forecasts across all channels and also to enable exogenous infusion, if you have them.

fcm_gated_attn (bool, optional, defaults to True):

Enable gated attention in forecast channel mixing block.

fcm_context_length (int, optional, defaults to `1):

Surrounding context length to use. For Ex. If we want to consider 2 lag point before and after a data point, provide value 2 for fcm_context_length

fcm_use_mixer (bool, optional, defaults to True):

Enable Mixing in forecast channel mixing block.

fcm_mix_layers (int, optional, defaults to 2):

Number of mixer layers to use if fcm_use_mixer is enabled

fcm_prepend_past (bool, optional, defaults to True):

Prepend last context for forecast reconciliation

fcm_prepend_past_offset (int, optional, defaults to None): categorical_vocab_size_list (list, optional):

List of vocab size for all the tokenized categorical variables to use. Pass it in the same order as used in the foreward call param static_categorical_values.

prediction_filter_length (int,*optional*, defaults to None):

Actual length in the prediction output to use for loss calculations.

training_argsdict or None, default={}

Training arguments to use for the model. See transformers.TrainingArguments for details [11]. Note that the output_dir argument is required.

compute_metricslist, default=[]

List of metrics to compute during training. See transformers.Trainer for details.

callbackslist, default=None

List of callbacks to use during training. See transformers.Trainer

broadcastingbool, default=False

if True, multiindex data input will be broadcasted to single series. For each single series, one copy of this forecaster will try to fit and predict on it. The broadcasting is happening inside automatically, from the outerside api perspective, the input and output are the same, only one multiindex output from predict.

use_source_packagebool, default=False

If True, the model and configuration will be loaded directly from the source package tsfm_public.models.tinytimemixer. This is useful if you want to bypass the local version of the package or when working in an environment where the latest updates from the source package are needed. If False, the model and configuration will be loaded from the local version of package maintained in sktime because of model’s unavailability on pypi. To install the source package, follow the instructions here [1].

fit_strategystr, default=”minimal”

Strategy to use for fitting (fine-tuning) the model. This can be one of the following: - “zero-shot”: Uses pre-trained model as it is. If model path is None

with this strategy, ValueError is raised.

  • “minimal”: Fine-tunes only a small subset of the model parameters, allowing for quick adaptation with limited computational resources. If model path is None with this strategy, ValueError is raised.

  • “full”: Fine-tunes all model parameters, which may result in better performance but requires more computational power and time. Allows model path to be None.

devicestr, default=”cpu”

Device for model inference and fine-tuning, for example "cpu", "cuda", "cuda:0", or "auto". "auto" is passed to transformers device_map and selects an available accelerator.

freqstr or None, default=None

Frequency to pass to models that use resolution prefix tuning, such as TTM-R2 [5] and research R2 [6]. If None, the frequency is inferred from the forecasting horizon or time index where possible, and falls back to the out-of-vocabulary token.

verbosebool, default=False

If True, show training output from transformers.Trainer.

padding_maskstr, default=”observed”

Controls how synthetic padding is masked when the input history is shorter than the model’s context_length. Missing history is left-padded with zeros. This can be one of the following:

  • “observed”: Marks the padded zeros as observed, matching Granite-TSFM’s preprocessing (past_observed_mask built with ~np.isnan, so numeric zeros count as observed). Pretrained TinyTimeMixer checkpoints were trained under these semantics, so this is required to reproduce Granite-TSFM’s forecasts and is the only valid option when fit_strategy="zero-shot".

  • “unobserved”: Marks the padded zeros as unobserved, so they are not treated as part of the observed series. This does not match how public checkpoints were pretrained, and is only valid when fit_strategy is "minimal" or "full", so the model can be fine-tuned under these mask semantics; otherwise a ValueError is raised.

Attributes:
cutoff

Cut-off = “present time” state of forecaster.

fh

Forecasting horizon that was passed.

is_fitted

Whether fit has been called.

state

State of the estimator.

References

[2]

Ekambaram, V., Jati, A., Dayama, P., Mukherjee, S., Nguyen, N.H., Gifford, W.M., Reddy, C. and Kalagnanam, J., 2024. Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series. CoRR.

Examples

Zero-shot forecasting with a pretrained TinyTimeMixer R3 checkpoint [7]. Other supported public checkpoints include R1 [4], R2 [5], and research R2 [6] variants:

>>> from sktime.forecasting.ttm import TinyTimeMixerForecaster
>>> from sktime.datasets import load_airline
>>> y = load_airline()
>>> forecaster = TinyTimeMixerForecaster(
...     model_path="ibm-research/ttm-r3",
...     # Other supported public checkpoints include:
...     # model_path="ibm-granite/granite-timeseries-ttm-r1",
...     # model_path="ibm-granite/granite-timeseries-ttm-r2",
...     # model_path="ibm-granite/granite-timeseries-ttm-r2",
...     # revision="52-16-ft-r2.1",
...     # model_path="ibm-research/ttm-research-r2",
... )
>>> forecaster.fit(y)
TinyTimeMixerForecaster(...)
>>> y_pred = forecaster.predict(fh=[1, 2, 3])

Automatically select the best compatible model revision from the context length and prediction length:

>>> from sktime.datasets import load_airline
>>> from sktime.forecasting.ttm import TinyTimeMixerForecaster
>>> y = load_airline()
>>> forecaster = TinyTimeMixerForecaster(
...     model_path="ibm-research/ttm-r3",
...     revision=None,
... )
>>> forecaster.fit(y, fh=[1, 2, 3])
TinyTimeMixerForecaster(...)
>>> y_pred = forecaster.predict()

Forecasting with exogenous variables known during training and prediction:

>>> from sktime.forecasting.ttm import TinyTimeMixerForecaster
>>> from sktime.datasets import load_longley
>>> from sktime.split import temporal_train_test_split
>>> y, X = load_longley()
>>> y_train, _, X_train, X_future = temporal_train_test_split(y, X, test_size=2)
>>> forecaster = TinyTimeMixerForecaster(
...     model_path="ibm-research/ttm-r3",
... )
>>> forecaster.fit(y_train, X=X_train, fh=[1, 2])
TinyTimeMixerForecaster(...)
>>> y_pred = forecaster.predict(X=X_future)

Minimal fine-tuning updates only parameters that are not loaded from the checkpoint, for example when the supplied configuration changes the model shape:

>>> from sktime.datasets import load_airline
>>> from sktime.forecasting.ttm import TinyTimeMixerForecaster
>>> y = load_airline()
>>> forecaster = TinyTimeMixerForecaster(
...     model_path="ibm-research/ttm-r3",
...     fit_strategy="minimal",
...     config={
...         "context_length": 24,
...         "trend_patch_length": 6,
...         "trend_patch_stride": 6,
...         "prediction_length": 12,
...     },
...     training_args={
...         "max_steps": 10,
...         "output_dir": "test_output",
...         "per_device_train_batch_size": 4,
...         "report_to": "none",
...     },
... )
>>> forecaster.fit(y)
TinyTimeMixerForecaster(...)
>>> y_pred = forecaster.predict(fh=[1, 2, 3])

Initialize a random model when model_path is None and preform full fine-tuning:

>>> from sktime.datasets import load_airline
>>> from sktime.forecasting.ttm import TinyTimeMixerForecaster
>>> y = load_airline()
>>> forecaster = TinyTimeMixerForecaster(
...     model_path=None,
...     fit_strategy="full",
...     training_args={
...         "max_steps": 10,
...         "output_dir": "test_output",
...         "per_device_train_batch_size": 4,
...         "report_to": "none",
...     },
... )
>>> forecaster.fit(y)
TinyTimeMixerForecaster(...)
>>> y_pred = forecaster.predict(fh=[1, 2, 3])

Pretrain on panel data before fitting to the target forecasting series:

>>> from sktime.forecasting.ttm import TinyTimeMixerForecaster
>>> from sktime.datasets import load_airline, load_tecator
>>> y = load_airline()
>>> y_panel = load_tecator(
...     return_type="pd-multiindex",
...     return_X_y=False,
... )
>>> y_panel.drop(["class_val"], axis=1, inplace=True)
>>> forecaster = TinyTimeMixerForecaster(
...     model_path="ibm-research/ttm-r3",
...     fit_strategy="full",
...     training_args={
...         "max_steps": 10,
...         "output_dir": "test_output",
...         "per_device_train_batch_size": 4,
...         "report_to": "none",
...     },
... )
>>> forecaster.pretrain(y_panel)
TinyTimeMixerForecaster(...)
>>> forecaster.fit(y)
TinyTimeMixerForecaster(...)
>>> y_pred = forecaster.predict(fh=[1, 2, 3])

Methods

check_is_fitted([method_name])

Check if the estimator has been fitted.

clone()

Obtain a clone of the object with same hyper-parameters and config.

clone_tags(estimator[, tag_names])

Clone tags from another object as dynamic override.

create_test_instance([parameter_set])

Construct an instance of the class, using first test parameter set.

create_test_instances_and_names([parameter_set])

Create list of all test instances and a list of names for them.

fit(y[, X, fh])

Fit forecaster to training data.

fit_predict(y[, X, fh, X_pred])

Fit and forecast time series at future horizon.

get_class_tag(tag_name[, tag_value_default])

Get class tag value from class, with tag level inheritance from parents.

get_class_tags()

Get class tags from class, with tag level inheritance from parent classes.

get_config()

Get config flags for self.

get_fitted_params([deep])

Get fitted parameters.

get_param_defaults()

Get object's parameter defaults.

get_param_names([sort])

Get object's parameter names.

get_params([deep])

Get a dict of parameters values for this object.

get_pretrained_params([deep])

Get pretrained parameters of this estimator.

get_tag(tag_name[, tag_value_default, ...])

Get tag value from instance, with tag level inheritance and overrides.

get_tags()

Get tags from instance, with tag level inheritance and overrides.

get_test_params([parameter_set])

Return testing parameter settings for the estimator.

is_composite()

Check if the object is composed of other BaseObjects.

load_from_path(serial)

Load object from file location.

load_from_serial(serial)

Load object from serialized memory container.

predict([fh, X])

Forecast time series at future horizon.

predict_interval([fh, X, coverage])

Compute/return prediction interval forecasts.

predict_proba([fh, X, marginal])

Compute/return fully probabilistic forecasts.

predict_quantiles([fh, X, alpha])

Compute/return quantile forecasts.

predict_residuals([y, X])

Return residuals of time series forecasts.

predict_var([fh, X, cov])

Compute/return variance forecasts.

pretrain(y[, X, fh])

Pre-train forecaster on panel (global) data.

reset()

Reset the object to a clean post-init state.

save([path, serialization_format])

Save serialized self to bytes-like object or to (.zip) file.

score(y[, X, fh])

Scores forecast against ground truth, using MAPE (non-symmetric).

set_config(**config_dict)

Set config flags to given values.

set_params(**params)

Set the parameters of this object.

set_random_state([random_state, deep, ...])

Set random_state pseudo-random seed parameters for self.

set_tags(**tag_dict)

Set instance level tag overrides to given values.

update(y[, X, update_params])

Update cutoff value and, optionally, fitted parameters.

update_predict(y[, cv, X, update_params, ...])

Make predictions and update model iteratively over the test set.

update_predict_single([y, fh, X, update_params])

Update model with new data and make forecasts.