CNTCRegressorTorch

class CNTCRegressorTorch(context_filter_sizes: tuple[int, ...] = (16,), context_kernel_sizes: tuple[int, ...] = (3,), context_steps: int = 3, context_dropout: float | tuple[float, ...] = 0.8, conv_filter_sizes: tuple[int, ...] = (8,), conv_kernel_sizes: tuple[int, ...] = (3,), conv_dropout: float | tuple[float, ...] = 0.8, lstm_units: tuple[int, ...] = (8,), context_window: int = 3, lstm_dropout: float | tuple[float, ...] = 0.8, pool_size: int = 2, pool_type: str | None = 'max', pool_dropout: float = 0.6, attention_width: int | None = 10, attention_units: int = 32, attention_type: str = 'additive', attention_activation: str | Callable | None = None, attention_dropout: float = 0.5, dense_layers: tuple[int, ...] = (64, 64), dense_dropout: float | tuple[float, ...] = (0.5, 0.8), activation: str | Callable | None = None, activation_hidden: str | Callable | None = 'ReLU', init_weights: str | None = 'xavier_uniform', num_epochs: int = 150, batch_size: int = 16, optimizer: str | None | Callable = 'Adam', criterion: str | None | Callable = 'MSELoss', callbacks: None | str | tuple[str, ...] = 'ReduceLROnPlateau', optimizer_kwargs: dict | None = None, criterion_kwargs: dict | None = None, callback_kwargs: dict | None = None, metrics: None | str | Callable | tuple[str | Callable, ...] = None, lr: float = 0.001, verbose: bool = False, random_state: int = 0)[source]

Contextual Time-series Neural Regressor for TSC, implemented in PyTorch.

CNTC combines a Contextual Convolutional Neural Network (CCNN) and a Contextual Long Short-Term Memory network (CLSTM) as parallel feature extractors, concatenates their outputs per time step, refines them with self-attention and regresses with a multilayer perceptron, following [1].

The four stages are:

Feature extraction. The CCNN arm stacks contextual convolutional layers (recurrent convolutions, equation 1 of [1]) followed by standard convolutional layers. The CLSTM arm stacks contextual LSTM layers whose gates receive sliding-window means of the input as contextual features (equation 4 of [1]). Both arms see the same input.
Concatenation. The two arms are concatenated along the feature axis, per time step, giving c_k = concat(mu_k, h_k) (equation 5 of [1]).
Attention. Pooling downsamples the merged sequence, then sequential self-attention reweights it (equations 6 to 8 of [1]).
Multilayer perceptron. Fully connected layers with dropout, followed by the output layer, which has a single unit for regression.

Parameters:

context_filter_sizestuple of int, default = (16,)

Number of filters in each contextual convolutional layer of the CCNN arm. The length of the tuple sets the number of such layers; any length >= 1 is allowed. [1] uses a single layer with 8, 16, 32 or 64 filters.

context_kernel_sizestuple of int, default = (3,)

Length of the 1D convolution window of each contextual convolutional layer. Must have the same length as context_filter_sizes. Combined with context_steps, a kernel of size n gives an effective receptive field of (n - 1) * context_steps + 1.

context_stepsint, default = 3

Number of recurrent iterations K performed inside every contextual convolutional layer. context_steps=1 reduces the layer to an ordinary convolution.

context_dropoutfloat or tuple of float, default = 0.8

Dropout rate applied after each contextual convolutional layer. A float applies the same rate to all of them, a tuple sets them individually and must have the same length as context_filter_sizes.

conv_filter_sizestuple of int, default = (8,)

Number of filters in each standard convolutional layer, applied after the contextual convolutional layers. The length of the tuple sets the number of such layers; () disables them entirely. [1] uses a single layer with 8, 16 or 32 filters.

conv_kernel_sizestuple of int, default = (3,)

Length of the 1D convolution window of each standard convolutional layer. Must have the same length as conv_filter_sizes.

conv_dropoutfloat or tuple of float, default = 0.8

Dropout rate applied after each standard convolutional layer. A float applies the same rate to all of them, a tuple sets them individually and must have the same length as conv_filter_sizes.

lstm_unitstuple of int, default = (8,)

Number of cells in each contextual LSTM layer of the CLSTM arm. The length of the tuple sets the number of stacked layers; any length >= 1 is allowed. [1] uses a single layer with 8, 16, 32 or 64 cells.

context_windowint, default = 3

Size of the sliding window used to build the contextual features of the CLSTM arm. At each time step the context vector holds the context_window most recent window means, so it has dimension context_window * n_dims.

lstm_dropoutfloat or tuple of float, default = 0.8

Dropout rate applied after each contextual LSTM layer. A float applies the same rate to all of them, a tuple sets them individually and must have the same length as lstm_units. [1] uses 0.8.

pool_sizeint, default = 2

Size and stride of the pooling window applied to the merged sequence before attention. Values above 1 downsample the time axis. Clipped to the series length.

pool_typestr or None, default = “max”

Pooling to apply before attention. One of "max", "avg", "both" (max and average pooling concatenated along the feature axis), or None to disable pooling.

pool_dropoutfloat, default = 0.6

Dropout rate applied after pooling.

attention_widthint or None, default = 10

Width of the local attention window. None lets every time step attend to every other time step. [1] uses 8 or 10.

attention_unitsint, default = 32

Hidden dimension of the additive attention scorer. Unused when attention_type="multiplicative".

attention_typestr, default = “additive”

Attention scoring function, either "additive" or "multiplicative". [1] scores alignments with a feedforward network, which corresponds to "additive".

attention_activationstr, callable, torch.nn.Module or None, default = None

Non-linearity applied to the attention logits before normalisation. None normalises the raw alignment scores with a softmax, as in equation (7) of [1].

attention_dropoutfloat, default = 0.5

Dropout rate applied after attention. [1] uses 0.5.

dense_layerstuple of int, default = (64, 64)

Number of units in each dense layer of the multilayer perceptron. The length of the tuple sets the number of such layers; () connects the attention output straight to the output layer. [1] uses two layers of 64 units.

dense_dropoutfloat or tuple of float, default = (0.5, 0.8)

Dropout rate applied after each dense layer. A float applies the same rate to all of them, a tuple sets them individually and must have the same length as dense_layers. [1] uses 0.5 and 0.8.

activationstr, Callable, or None, default=None

Activation applied to the output layer.

Permitted values:

None: no activation is applied to the output layer and the network returns raw outputs. This is the usual choice for regression, where the target is unbounded.
str: name of a class in torch.nn. Case-sensitive names are recommended and must match PyTorch (e.g., "ReLU", "LeakyReLU"). Lowercase aliases for common activations are also accepted (e.g., "relu" is resolved to "ReLU"). The class is instantiated with default constructor arguments. Must be a valid torch.nn activation; see https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity
torch.nn.Module: an instance of a torch.nn.Module subclass, for example torch.nn.ReLU(). Arbitrary callables are not supported.

activation_hiddenstr, Callable, or None, default=”ReLU”

Activation applied to the hidden layers, that is the convolutional layers of the CCNN arm and the dense layers of the multilayer perceptron.

Permitted values:

None: no activation is applied to the hidden layers.
str: name of a class in torch.nn. Case-sensitive names are recommended and must match PyTorch (e.g., "ReLU", "LeakyReLU"). Lowercase aliases for common activations are also accepted (e.g., "relu" is resolved to "ReLU"). The class is instantiated with default constructor arguments. Must be a valid torch.nn activation; see https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity
torch.nn.Module: an instance of a torch.nn.Module subclass, for example torch.nn.ReLU(). Arbitrary callables are not supported.

init_weightsstr or None, default = “xavier_uniform”

The method used to initialize the weights of the convolutional and linear layers. Supported values are "kaiming_uniform", "kaiming_normal", "xavier_uniform", "xavier_normal", or None for the default PyTorch initialization. Biases are zeroed whenever a method is given.

num_epochsint, default = 150

The number of epochs to train the model.

batch_sizeint, default = 16

The size of each mini-batch during training. [1] uses 16, 32 or 64.

optimizercase insensitive str or None or an instance of optimizers

defined in torch.optim, default = “Adam” The optimizer to use for training the model. List of available optimizers: https://pytorch.org/docs/stable/optim.html#algorithms

criterioncase insensitive str or None or an instance of a loss function

defined in PyTorch, default = “MSELoss” The loss function to be used in training the neural network. List of available loss functions: https://pytorch.org/docs/stable/nn.html#loss-functions

callbacksNone or str or a tuple of str, default = “ReduceLROnPlateau”

Currently only learning rate schedulers are supported as callbacks. If more than one scheduler is passed, they are applied sequentially in the order they are passed. If None, then no learning rate scheduler is used. Note: Since PyTorch learning rate schedulers need to be initialized with the optimizer object, we only accept the class name (str) of the scheduler here and do not accept an instance of the scheduler. As that can lead to errors and unexpected behavior. List of available learning rate schedulers: https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate

optimizer_kwargsdict or None, default = None

Additional keyword arguments to pass to the optimizer.

criterion_kwargsdict or None, default = None

Additional keyword arguments to pass to the loss function.

callback_kwargsdict or None, default = None

The keyword arguments to be passed to the callbacks.

metricsNone or str or Callable or tuple of str and/or Callable, default = None

Metrics to compute during training. If None, no metrics are computed beyond the loss. Metrics are computed from torchmetrics library. If a string/Callable is passed, it must be one of the metrics defined in https://lightning.ai/docs/torchmetrics/stable/ Examples: “MeanSquaredError”, “MeanAbsoluteError”, “R2Score”

lrfloat, default = 0.001

The learning rate to use for the optimizer.

verbosebool, default = False

Whether to print progress information during training.

random_stateint, default = 0

Seed to ensure reproducibility.

Attributes:

is_fitted: Whether fit has been called.

References

[1] (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16)

Amadu Fullah Kamara, Enhong Chen, Qi Liu, Zhen Pan, Combin- ing Contextual Neural Networks for Time Series Classification, Neurocomputing (2019), doi: https://doi.org/10.1016/j.neucom.2019.10.113

Examples

>>> from sktime.regression.deep_learning.cntc import CNTCRegressorTorch
>>> from sktime.datasets import load_unit_test
>>> X_train, y_train = load_unit_test(split="train")
>>> reg = CNTCRegressorTorch(num_epochs=5, batch_size=4)
>>> reg.fit(X_train, y_train)
CNTCRegressorTorch(...)

Methods

`check_is_fitted`([method_name])	Check if the estimator has been fitted.
`clone`()	Obtain a clone of the object with same hyper-parameters and config.
`clone_tags`(estimator[, tag_names])	Clone tags from another object as dynamic override.
`create_test_instance`([parameter_set])	Construct an instance of the class, using first test parameter set.
`create_test_instances_and_names`([parameter_set])	Create list of all test instances and a list of names for them.
`fit`(X, y)	Fit time series regressor to training data.
`get_class_tag`(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
`get_class_tags`()	Get class tags from class, with tag level inheritance from parent classes.
`get_config`()	Get config flags for self.
`get_fitted_params`([deep])	Get fitted parameters.
`get_param_defaults`()	Get object's parameter defaults.
`get_param_names`([sort])	Get object's parameter names.
`get_params`([deep])	Get a dict of parameters values for this object.
`get_tag`(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
`get_tags`()	Get tags from instance, with tag level inheritance and overrides.
`get_test_params`([parameter_set])	Return testing parameter settings for the estimator.
`is_composite`()	Check if the object is composed of other BaseObjects.
`load_from_path`(serial)	Load object from file location.
`load_from_serial`(serial)	Load object from serialized memory container.
`predict`(X)	Predicts labels for sequences in X.
`reset`()	Reset the object to a clean post-init state.
`save`([path, serialization_format])	Save serialized self to bytes-like object or to (.zip) file.
`score`(X, y[, multioutput])	Scores predicted labels against ground truth labels on X.
`set_config`(**config_dict)	Set config flags to given values.
`set_params`(**params)	Set the parameters of this object.
`set_random_state`([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
`set_tags`(**tag_dict)	Set instance level tag overrides to given values.