TSCGridSearchCV

class TSCGridSearchCV(estimator, param_grid, scoring=None, n_jobs='deprecated', refit=True, cv=None, verbose=0, pre_dispatch='deprecated', error_score=nan, return_train_score=False, tune_by_variable=False, greater_is_better='auto', backend=None, backend_params=None)[source]

Exhaustive search over specified parameter values for a classifier.

Optimizes hyper-parameters of estimator by exhaustive grid search, using sktime native backtesting via classification.model_evaluation.evaluate.

In fit, each parameter combination in param_grid is backtested on the data passed, using the cross-validation scheme cv and the metric scoring. All candidates are evaluated on identical folds.

The parameter combination with the best mean test score is set as best_params_, and a clone of estimator with those parameters is set as best_estimator_. If refit is not False, best_estimator_ is fitted to the entire data, and predict and predict-like methods of the tuner call the respective method of best_estimator_.

Parameters:

estimatorsktime classifier, BaseClassifier instance or interface compatible

The classifier to tune, must implement the sktime classifier interface.

param_griddict or list of dictionaries

Dictionary with parameters names (str) as keys and lists of parameter settings to try as values, or a list of such dictionaries, in which case the grids spanned by each dictionary in the list are explored. This enables searching over any sequence of parameter settings.

scoringNone, str, callable, sklearn scorer, or list or dict of these

Metric or metrics to evaluate the cross-validated model with.

a callable must have signature (y_true, y_pred) -> float, e.g., accuracy_score from sklearn.metrics. Its value is reported as is
a string must name a scikit-learn scorer, e.g., "accuracy". Values are reported with the sign convention of the scorer, so values of "neg_log_loss" are negative
a list or dict selects multiple metrics. The first is used to rank candidates, unless refit names another. Dict keys are used as the metric names in cv_results_
if None, defaults to accuracy_score

n_jobsint, optional, default=”deprecated”

Number of jobs to run in parallel over the parameter candidates.

Deprecated, and will be removed in sktime 1.3.0. If passed, the value is written to backend_params, and backend defaults to "loky", so behaviour is unchanged. To retain the behaviour after removal, pass backend="loky" and backend_params={"n_jobs": ...} instead.

refitbool, str, or callable, default=True

Refit best_estimator_ using the best found parameters on the whole dataset. If False, predict and predict-like methods raise, and the tuner can be used only to tune hyper-parameters, e.g., as a parameter estimator via get_fitted_params.

For multi-metric evaluation, this can be a str naming the metric to select the best parameters by.

Where there are considerations other than the best score in choosing the best parameters, refit can be a callable, which is applied to cv_results_ and returns the selected best_index_. In that case best_score_ is not available.

The refitted estimator is available at the best_estimator_ attribute, and permits calling predict directly on the tuner.

cvint, cross-validation generator, iterable of splits, or None, default=None

Determines the cross-validation splitting strategy. Possible inputs for cv are:

None, to use the default 5-fold cross-validation,
integer, to specify the number of folds in a (Stratified)KFold,
a cross-validation splitter with a split method,
an iterable yielding (train, test) splits as arrays of indices.

For integer and None inputs, StratifiedKFold is used if y is binary or multiclass, and KFold otherwise. Both are instantiated with shuffle=False, so the splits are the same across calls.

Splits are computed once, before the search, so that all parameter candidates are evaluated on the same folds.

verboseint, default=0

Controls the verbosity. If positive, the number of fits is printed.

pre_dispatchint or str, optional, default=”deprecated”

Number of jobs dispatched during parallel execution, a joblib parameter.

error_score‘raise’ or numeric, default=np.nan

Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error.

return_train_scorebool, default=False

Whether to include training scores in cv_results_.

Deprecated, and will be removed in sktime 1.3.0. Train scores are not computed by the native grid search, so the value passed is ignored and cv_results_ contains test scores only. Passing True raises a warning.

tune_by_variablebool, optional (default=False)

Whether to tune parameter by each time series variable separately, in case of multivariate data passed to the tuning estimator. Only applies if time series passed are strictly multivariate. If True, clones of the estimator will be fit to each variable separately, and are available in fields of the classifiers_ attribute. Has the same effect as applying ColumnEnsembleClassifier wrapper to self. If False, the same best parameter is selected for all variables.

greater_is_better“auto”, bool, optional, default=”auto”

Whether higher values of the reported metric are better, used to rank the parameter candidates.

“auto” determines the direction from the metric. Scikit-learn scorers are higher-is-better by their sign convention. For metric callables, the direction is inferred from the metric, e.g., accuracy_score is higher-is-better, and log_loss is lower-is-better
True or False set the direction explicitly, for all metrics

backendstr, optional, default=None

Parallelization backend for the search over parameter candidates.

None: executes loop sequentially, simple list comprehension
“loky”, “multiprocessing” and “threading”: uses joblib.Parallel loops
“joblib”: custom and 3rd party joblib backends, e.g., spark
“dask”: uses dask, requires dask package in environment
“ray”: uses ray, requires ray package in environment

Recommendation: use “dask” or “loky” for parallel grid search. “threading” is unlikely to see speed ups due to the GIL.

backend_paramsdict, optional

Additional parameters passed to the backend as config, directly passed to utils.parallel.parallelize. Valid keys depend on backend, see there for details.

Attributes:

cv_results_dict of str to numpy (masked) ndarray

A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame. Contains params, one param_<name> column per searched parameter, fit and score timings, and per metric the per-fold scores split<i>_test_<name>, their mean mean_test_<name>, standard deviation std_test_<name>, and rank rank_test_<name>, 1 being the best.

For a single metric, <name> is score, e.g., mean_test_score. For multiple metrics, <name> is the name of the respective metric.

best_estimator_estimator

Clone of estimator with the best found parameters set, i.e., the parameters which gave the best mean test score on the held out data. Fitted to the entire data if refit is not False, otherwise unfitted. See the refit parameter for more information on allowed values.

best_score_float

Mean cross-validated score of best_estimator_. Not available if refit is a callable.

best_params_dict

Parameter setting that gave the best results on the hold out data.

best_index_int

The index in the cv_results_ arrays which corresponds to the best candidate parameter setting.

The dict at cv_results_["params"][best_index_] gives the parameter setting for the best model, i.e., is identical with best_params_.

scorer_callable or dict of callable

Metric used on the held out data to choose the best parameters. For multi-metric evaluation, a dict of metric name to metric.

n_splits_int

The number of cross-validation splits (folds/iterations).

refit_time_float

Seconds used for refitting the best model on the whole dataset. This is present only if refit is not False.

multimetric_bool

Whether multiple metrics were passed in scoring.

classes_ndarray of shape (n_classes,)

The class labels seen in fit.

Examples

>>> from sklearn.metrics import accuracy_score
>>> from sktime.classification.dummy import DummyClassifier
>>> from sktime.classification.model_selection import TSCGridSearchCV
>>> from sktime.datasets import load_unit_test
>>>
>>> X, y = load_unit_test(split="train")
>>> tuned = TSCGridSearchCV(
...     DummyClassifier(),
...     param_grid={"strategy": ["most_frequent", "prior"]},
...     scoring=accuracy_score,
...     cv=2,
... )
>>> tuned = tuned.fit(X, y)
>>> y_pred = tuned.predict(X)
>>> best_params = tuned.best_params_

Methods

`check_is_fitted`([method_name])	Check if the estimator has been fitted.
`clone`()	Obtain a clone of the object with same hyper-parameters and config.
`clone_tags`(estimator[, tag_names])	Clone tags from another object as dynamic override.
`create_test_instance`([parameter_set])	Construct an instance of the class, using first test parameter set.
`create_test_instances_and_names`([parameter_set])	Create list of all test instances and a list of names for them.
`fit`(X, y)	Fit time series classifier to training data.
`fit_predict`(X, y[, cv, change_state])	Fit and predict labels for sequences in X.
`fit_predict_proba`(X, y[, cv, change_state])	Fit and predict labels probabilities for sequences in X.
`get_class_tag`(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
`get_class_tags`()	Get class tags from class, with tag level inheritance from parent classes.
`get_config`()	Get config flags for self.
`get_fitted_params`([deep])	Get fitted parameters.
`get_param_defaults`()	Get object's parameter defaults.
`get_param_names`([sort])	Get object's parameter names.
`get_params`([deep])	Get a dict of parameters values for this object.
`get_tag`(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
`get_tags`()	Get tags from instance, with tag level inheritance and overrides.
`get_test_params`([parameter_set])	Return testing parameter settings for the estimator.
`is_composite`()	Check if the object is composed of other BaseObjects.
`load_from_path`(serial)	Load object from file location.
`load_from_serial`(serial)	Load object from serialized memory container.
`predict`(X)	Predicts labels for sequences in X.
`predict_proba`(X)	Predicts labels probabilities for sequences in X.
`reset`()	Reset the object to a clean post-init state.
`save`([path, serialization_format])	Save serialized self to bytes-like object or to (.zip) file.
`score`(X, y)	Scores predicted labels against ground truth labels on X.
`set_config`(**config_dict)	Set config flags to given values.
`set_params`(**params)	Set the parameters of this object.
`set_random_state`([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
`set_tags`(**tag_dict)	Set instance level tag overrides to given values.