MVTSTransformerClassifier

class MVTSTransformerClassifier(d_model=256, n_heads=4, num_layers=4, dim_feedforward=128, dropout=0.1, pos_encoding='fixed', activation='relu', norm='BatchNorm', freeze=False, num_epochs=10, batch_size=8, criterion=None, criterion_kwargs=None, optimizer=None, optimizer_kwargs=None, lr=0.001, verbose=True, random_state=None)[source]

Multivariate Time Series Transformer for Classification, as described in [1].

This classifier has been wrapped around the official pytorch implementation of Transformer from [R646b85c59e3b-2], provided by the authors of the paper [1].

Parameters:

d_modelint, optional (default=256): The number of expected features in the input (i.e., the dimension of the model).
n_headsint, optional (default=4): The number of heads in the multihead attention mechanism.
num_layersint, optional (default=4): The number of layers (or blocks) in the transformer encoder.
dim_feedforwardint, optional (default=128): The dimension of the feedforward network model.
dropoutfloat, optional (default=0.1): The dropout rate to apply.
pos_encodingstr, optional (default=”fixed”): The type of positional encoding to use. Options: [“fixed”, “learnable”].
activationstr, optional (default=”relu”): The activation function to use. Options: [“relu”, “gelu”].
normstr, optional (default=”BatchNorm”): The type of normalization to use. Options: [“BatchNorm”, “LayerNorm”].
freezebool, optional (default=False): If True, the transformer layers will be frozen and not trained.
num_epochsint, optional (default=10): The number of epochs to train the model.
batch_sizeint, optional (default=8): The size of each mini-batch during training.
criterioncallable, optional (default=None): The loss function to use. If None, CrossEntropyLoss will be used.
criterion_kwargsdict, optional (default=None): Additional keyword arguments to pass to the loss function.
optimizerstr, optional (default=None): The optimizer to use. If None, Adam optimizer will be used.
optimizer_kwargsdict, optional (default=None): Additional keyword arguments to pass to the optimizer.
lrfloat, optional (default=0.001): The learning rate for the optimizer.
verbosebool, optional (default=True): If True, prints progress messages during training.
random_stateint or None, optional (default=None): Seed for the random number generator.

Attributes:

is_fitted: Whether fit has been called.

References

[1] (1,2)

George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty,

and Carsten Eickhoff. 2021. A Transformer-based Framework for Multivariate Time Series Representation Learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD ‘21). Association for Computing Machinery, New York, NY, USA, 2114-2124. https://doi.org/10.1145/3447548.3467401. .. [R646b85c59e3b-2] https://github.com/gzerveas/mvts_transformer

Examples

>>> from sktime.datasets import load_unit_test
>>> from sktime.classification.deep_learning import MVTSTransformerClassifier
>>>
>>> X_train, y_train = load_unit_test(split="train")
>>> X_test, _ = load_unit_test(split="test")
>>>
>>> model = MVTSTransformerClassifier()
>>> model.fit(X_train, y_train)
>>> preds = model.predict(X_test)

Methods

`check_is_fitted`([method_name])	Check if the estimator has been fitted.
`clone`()	Obtain a clone of the object with same hyper-parameters and config.
`clone_tags`(estimator[, tag_names])	Clone tags from another object as dynamic override.
`create_test_instance`([parameter_set])	Construct an instance of the class, using first test parameter set.
`create_test_instances_and_names`([parameter_set])	Create list of all test instances and a list of names for them.
`fit`(X, y)	Fit time series classifier to training data.
`fit_predict`(X, y[, cv, change_state])	Fit and predict labels for sequences in X.
`fit_predict_proba`(X, y[, cv, change_state])	Fit and predict labels probabilities for sequences in X.
`get_class_tag`(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
`get_class_tags`()	Get class tags from class, with tag level inheritance from parent classes.
`get_config`()	Get config flags for self.
`get_fitted_params`([deep])	Get fitted parameters.
`get_param_defaults`()	Get object's parameter defaults.
`get_param_names`([sort])	Get object's parameter names.
`get_params`([deep])	Get a dict of parameters values for this object.
`get_tag`(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
`get_tags`()	Get tags from instance, with tag level inheritance and overrides.
`get_test_params`([parameter_set])	Return testing parameter settings for the estimator.
`is_composite`()	Check if the object is composed of other BaseObjects.
`load_from_path`(serial)	Load object from file location.
`load_from_serial`(serial)	Load object from serialized memory container.
`predict`(X)	Predicts labels for sequences in X.
`predict_proba`(X)	Predicts labels probabilities for sequences in X.
`reset`()	Reset the object to a clean post-init state.
`save`([path, serialization_format])	Save serialized self to bytes-like object or to (.zip) file.
`score`(X, y)	Scores predicted labels against ground truth labels on X.
`set_config`(**config_dict)	Set config flags to given values.
`set_params`(**params)	Set the parameters of this object.
`set_random_state`([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
`set_tags`(**tag_dict)	Set instance level tag overrides to given values.