Pipeline

Pipeline

class Pipeline(steps=None)[source]

Implementation of a Graphpipeline.

This class is a generalized graph pipeline. Generalized means that it can contain forecasters, classifiers, etc. The graph pipeline mean that the structure is not linear. I.e., the each element of the pipeline can be the input of multiple other steps and not only one successors.

fit(y, X, *args) - changes state by running fit on all sktime estimators and

transformers in the pipeline. Note that depending on the sktime estimators and transformers that are added to the pipeline, different keywords are required. E.g., if a forecaster is part of the pipeline, a forecast horizon (fh) should be provided.

predict(X, *args) - Results in calling predict on the estimators in the pipeline

and transform or the specified method on the other skobjects in the pipeline. Depending on the skobject added to the pipeline, you might need to pass additional parameters to predict.

predict_interval(X, fh), predict_quantiles(X, fh) - as predict(X, fh),

with predict_interval or predict_quantiles substituted for predict.

predict_var, predict_proba - are currently not supported get_params, set_params uses sklearn compatible nesting interface add_step(skobject, name, edges, method, **kwargs) - adds a skobject to the

pipeline and setting the name as identifier and the steps specified with edges as input steps (predecessors). Thereby the method that should be called can be overridden using the method kwarg. Further provided kwargs are directly provided to the skobject if it is called.

Parameters:

param stepsA list of dicts that specify the steps of the pipeline. Further

steps can be added by using add_step method. The dict requires the following keys:

• skobject: sktime object, the skobject that should be added to the

  pipeline

• name: str, the name of the step that is created

• edges: dict, a dict with string keys to string values. Identifying the

  predcessors. The keys of the edges dict specify to which argument of fit/predict/.. the output of the predecessors (the value of the dict specifies the predecessors name) should be passed.

• method: str, an optional argument allowing to determine the method that

  should be executed when the pipeline calls the provided skobject. If not specified, the pipeline selects the method based on the method that is called on the pipeline (e.g., predict, transform, ..)

• kwargs: additional kwargs are parameters that are provided to the

  skobject if fit/predict/.. is called.

Attributes:

attribute id_to_true_ida dict with integer keys and values,

mapping the python object id to skobject ids.

attribute id_to_obja dict with integer keys and weak references of

skobjects as values. The values are the weak references of the skobjects provided to the add_step method. We store the weak references to avoid that the id of the object is reassigned if the user deletes all it references to the object.

attribute model_dicta dict with integer keys and skobject values.

This is a mapping of the id of the skobjects provided to add_step to the cloned skobject.

attribute counterinteger, counts the number of steps in the pipeline.

attribute stepsa dict with string keys and step object values.

The key is the name that is specified if a skobject is added to the pipeline.

attribute kwargsa dict with str keys and object values. Stores all kwargs

that are specified and might be passed to skobjects in the pipeline.

References

[1]

@article{heidrich2021pywatts, title={pyWATTS: Python workflow automation tool for time series}, author={Heidrich, Benedikt and Bartschat, Andreas and Turowski, Marian and

Neumann, Oliver and Phipps, Kaleb and Meisenbacher, Stefan and Schmieder, Kai and Ludwig, Nicole and Mikut, Ralf and Hagenmeyer, Veit},

journal={arXiv preprint arXiv:2106.10157}, year={2021}

}

Examples

>>> from sktime.classification.distance_based import KNeighborsTimeSeriesClassifier
>>> from sktime.datasets import load_arrow_head, load_longley
>>> from sktime.split import temporal_train_test_split
>>> from sktime.forecasting.naive import NaiveForecaster
>>> from sktime.pipeline import Pipeline
>>> from sktime.transformations.compose import Id
>>> from sktime.transformations.boxcox import BoxCoxTransformer
>>> from sktime.transformations.exponent import ExponentTransformer

Example 1: Simple sequential pipeline of transformers using the generalized non-sequential pipeline implementation

>>> y, X = load_longley()
>>> general_pipeline = Pipeline()
>>> for step in [
...     {"skobject": ExponentTransformer(), "name": "exp", "edges": {"X": "X"}},
...     {"skobject": BoxCoxTransformer(), "name": "box", "edges": {"X": "exp"}},
...     ]:
...     general_pipeline = general_pipeline.add_step(**step)
>>> general_pipeline.fit(X=X)
>>> result_general = general_pipeline.transform(X)

Example 2: Classification sequential pipeline using the generalized

non-sequential pipeline implementation

>>> X, y = load_arrow_head(split="train", return_X_y=True)
>>> general_pipeline = Pipeline()
>>> for step in [
...     {"skobject": ExponentTransformer(), "name": "exp", "edges": {"X": "X"}},
...     {"skobject": KNeighborsTimeSeriesClassifier(),
...      "name": "knnclassifier",
...      "edges": {"X": "exp", "y": "y"}}]:
...     general_pipeline = general_pipeline.add_step(**step)
>>> general_pipeline.fit(X=X, y=y)
>>> result_general = general_pipeline.predict(X)

Example 3: Forecasting pipeline with exogenous features using the generalized non-sequential pipeline implementation

>>> y, X = load_longley()
>>> y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)
>>> general_pipeline = Pipeline()
>>> for step in [
...     {"skobject": ExponentTransformer(), "name": "exp", "edges": {"X": "X"}},
...     {"skobject": NaiveForecaster(),
...      "name": "SARIMAX",
...      "edges": {"X": "exp", "y": "y"}}]:
...     general_pipeline = general_pipeline.add_step(**step)
>>> general_pipeline.fit(y=y_train, X=X_train, fh=[1, 2, 3, 4])
>>> result_general = general_pipeline.predict(X=X_test)

Acknowledgements This graphical pipeline is inspired by pyWATTS that is developed by the Institute for Automation and Applied Informatics (IAI) at Karlsruhe Institute of Technology. The implementation is supported by IAI and the author benHeid is funded by HelmholtzAI. Furthermore, we also want to credit @ViktorKaz for his independent pipeline design that is similar to this one.

Methods

add_step(skobject, name, edges[, method])	Add a new skobject to the pipeline.
check_is_fitted([method_name])	Check if the estimator has been fitted.
clone()	Obtain a clone of the object with same hyper-parameters and config.
clone_tags(estimator[, tag_names])	Clone tags from another object as dynamic override.
create_test_instance([parameter_set])	Construct an instance of the class, using first test parameter set.
create_test_instances_and_names([parameter_set])	Create list of all test instances and a list of names for them.
fit([X, y])	Fit graph pipeline to training data.
fit_transform(X[, y])	Fit graph pipeline to training data and call transform afterward.
get_class_tag(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
get_class_tags()	Get class tags from class, with tag level inheritance from parent classes.
get_config()	Get config flags for self.
get_fitted_params([deep])	Get fitted parameters.
get_param_defaults()	Get object's parameter defaults.
get_param_names([sort])	Get object's parameter names.
get_params([deep])	Get the parameters of the pipeline.
get_tag(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
get_tags()	Get tags from instance, with tag level inheritance and overrides.
get_test_params([parameter_set])	Return testing parameter settings for the estimator.
is_composite()	Check if the object is composed of other BaseObjects.
load_from_path(serial)	Load object from file location.
load_from_serial(serial)	Load object from serialized memory container.
predict([X, y])	Perform a prediction.
predict_interval(X[, y])	Perform an interval prediction.
predict_quantiles(X[, y])	Perform a quantile prediction.
predict_residuals(X[, y])	Perform a residuals prediction.
reset()	Reset the object to a clean post-init state.
save([path, serialization_format])	Save serialized self to bytes-like object or to (.zip) file.
set_config(**config_dict)	Set config flags to given values.
set_params(**params)	Set the parameters of this estimator.
set_random_state([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
set_tags(**tag_dict)	Set instance level tag overrides to given values.
transform(X[, y])	Call transform on each element in the graph pipeline.