Classifier

MVTSTransformerClassifier

Multivariate Time Series Transformer for Classification, as described in [1].

Quickstart

python

from sktime.classification.deep_learning.mvts_transformer import MVTSTransformerClassifier

estimator = MVTSTransformerClassifier(d_model=256, n_heads=4, num_layers=4, dim_feedforward=128, dropout=0.1, pos_encoding='fixed', activation='relu', norm='BatchNorm', freeze=False, num_epochs=10, batch_size=8, criterion=None, criterion_kwargs=None, optimizer=None, optimizer_kwargs=None, lr=0.001, verbose=True, random_state=None)

Parameters(18)

d_modelint, optional (default=256): The number of expected features in the input (i.e., the dimension of the model).
n_headsint, optional (default=4): The number of heads in the multihead attention mechanism.
num_layersint, optional (default=4): The number of layers (or blocks) in the transformer encoder.
dim_feedforwardint, optional (default=128): The dimension of the feedforward network model.
dropoutfloat, optional (default=0.1): The dropout rate to apply.
pos_encodingstr, optional (default=”fixed”): The type of positional encoding to use. Options: [“fixed”, “learnable”].
activationstr, optional (default=”relu”): The activation function to use. Options: [“relu”, “gelu”].
normstr, optional (default=”BatchNorm”): The type of normalization to use. Options: [“BatchNorm”, “LayerNorm”].
freezebool, optional (default=False): If True, the transformer layers will be frozen and not trained.
num_epochsint, optional (default=10): The number of epochs to train the model.
batch_sizeint, optional (default=8): The size of each mini-batch during training.
criterioncallable, optional (default=None): The loss function to use. If None, CrossEntropyLoss will be used.
criterion_kwargsdict, optional (default=None): Additional keyword arguments to pass to the loss function.
optimizerstr, optional (default=None): The optimizer to use. If None, Adam optimizer will be used.
optimizer_kwargsdict, optional (default=None): Additional keyword arguments to pass to the optimizer.
lrfloat, optional (default=0.001): The learning rate for the optimizer.
verbosebool, optional (default=True): If True, prints progress messages during training.
random_stateint or None, optional (default=None): Seed for the random number generator.

Examples

>>> from sktime.datasets import load_unit_test
>>> from sktime.classification.deep_learning import MVTSTransformerClassifier
>>> 
>>> X_train, y_train = load_unit_test (split = "train")
>>> X_test, _ = load_unit_test (split = "test")
>>> 
>>> model = MVTSTransformerClassifier ()
>>> model. fit (X_train, y_train)
>>> preds = model. predict (X_test)

References

[1 ] (1, 2) George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. 2021. A Transformer-based Framework for Multivariate Time Series Representation Learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD ‘21). Association for Computing Machinery, New York, NY, USA, 2114-2124. https://doi.org/10.1145/3447548.3467401... [R646b85c59e3b-2] https://github.com/gzerveas/mvts_transformer