MomentFMForecaster
Interface for forecasting with the deep learning time series model momentfm.
MomentFM is a collection of open source foundation models for the general purpose of time series analysis. The Moment Foundation Model is a pre-trained model that is capable of accomplishing various time series tasks, such as:
Long Term Forecasting
Short Term Forecasting
This interface with MomentFM focuses on the forecasting task, in which the foundation model uses a user fine tuned ‘forecasting head’ to predict h steps ahead. This model does NOT have zero shot capabilities and requires fine-tuning to achieve performance on user inputted data.
For more information: see https://github.com/moment-timeseries-foundation-model/moment
For information regarding licensing and use of the momentfm model please visit: https://huggingface.co/AutonLab/MOMENT-1-large
- pretrained_model_name_or_pathstr
Path to the pretrained Momentfm model. Default is AutonLab/MOMENT-1-large
- freeze_encoderbool
Selection of whether or not to freeze the weights of the encoder Default = True
- freeze_embedderbool
Selection whether or not to freeze the patch embedding layer Default = True
- freeze_headbool
Selection whether or not to freeze the forecasting head. Recommendation is that the linear forecasting head must be trained Default = False
- dropoutfloat
Dropout value of the model. Values range between [0.0, 1.0] Default = 0.1
- head_dropoutfloat
Dropout value of the forecasting head. Values range between [0.0, 1.0] Default = 0.1
- seq_lenint
length of sequences or length of historical values that are passed to the model for training at each time point. the momentfm model requires sequence lengths to be 512 exactly, so if less, padding will be used. If the sequence length is > 512, it will be reduced to 512. default = 512
- batch_sizeint
size of batches to train the model on default = 32
- eval_batch_sizeint or “all”
size of batches to evaluate the model on. If the string “all” is specified, then we process the entire validation set as a single batch default = 32
- epochsint
Number of epochs to fit tune the model on default = 1
- max_lrfloat
Maximum learning rate that the learning rate scheduler will use default = 1e-4
- devicestr
torch device to use default = “auto” If set to auto, it will automatically use whatever device that accelerate detects.
- pct_startfloat
percentage of total iterations where the learning rate rises during one epoch default = 0.3
- max_normfloat
Float value used to clip gradients during training default = 5.0
- train_val_splitfloat
float value between 0 and 1 to determine portions of training and validation splits default = 0.2
- transformer_backbonestr
d_model of a pre-trained transformer model to use. See SUPPORTED_HUGGINGFACE_MODELS to specify valid models to use. Default is ‘google/flan-t5-large’.
- configdict, default = {}
If desired, user can pass in a config detailing all momentfm parameters that they wish to set in dictionary form, so that parameters do not need to be individually set. If a parameter inside a config is a duplicate of one already passed in individually, it will be overwritten.
- criterioncriterion, default = torch.nn.MSELoss
Criterion to use during training.
- return_model_to_cpubool, default = False
After fitting and training, will return the momentfm model to the cpu.
Quickstart
from sktime.forecasting.momentfm import MomentFMForecaster
estimator = MomentFMForecaster(pretrained_model_name_or_path='AutonLab/MOMENT-1-large', freeze_encoder=True, freeze_embedder=True, freeze_head=False, dropout=0.1, head_dropout=0.1, seq_len=512, batch_size=32, eval_batch_size=32, epochs=1, max_lr=0.0001, device='auto', pct_start=0.3, max_norm=5.0, train_val_split=0.2, transformer_backbone='google/flan-t5-large', criterion=None, config=None, return_model_to_cpu=False)Examples
>>> from sktime.forecasting.momentfm import MomentFMForecaster
>>> from sktime.datasets import load_airline
>>> y = load_airline ()
>>> forecaster = MomentFMForecaster (seq_len = 2)
>>> forecaster. fit (y, fh = [1, 2, 3 ])
>>> y_pred = forecaster. predict (y = y)