RegressionBenchmark

class RegressionBenchmark(id_format: str | None = None, backend=None, backend_params=None, return_data=False)[source]

Regression benchmark.

Run a series of regressors against a series of tasks defined via dataset loaders, cross validation splitting strategies and performance metrics, and return results as a df (as well as saving to file).

Parameters:

id_format: str, optional (default=None): A regex used to enforce task/estimator ID to match a certain format
backendstring, by default “None”.: Parallelization backend to use for runs. See ClassificationBenchmark for the list of valid backends.
backend_paramsdict, optional: additional parameters passed to the backend as config. Directly passed to utils.parallel.parallelize. Valid keys depend on the value of backend, see ClassificationBenchmark for details.
return_databool, optional (default=False): Whether to return the prediction and the ground truth data in the results.

Attributes:

failed_experiments: Failed task-estimator pairs from the most recent benchmark run.

Examples

>>> from sklearn.metrics import mean_squared_error
>>> from sklearn.model_selection import KFold
>>> from sktime.benchmarking.regression import RegressionBenchmark
>>> from sktime.regression.dummy import DummyRegressor
>>> from sktime.utils._testing.panel import make_regression_problem
>>> benchmark = RegressionBenchmark()
>>> benchmark.add_estimator(DummyRegressor())
>>> benchmark.add_task(
...     make_regression_problem,
...     KFold(n_splits=3),
...     [mean_squared_error],
... )
>>> results_df = benchmark.run(None)

Methods

`add`(*args)	Add estimators, task components, full task tuples, or catalogues.
`add_estimator`(estimator[, estimator_id])	Register an estimator to the benchmark.
`add_task`(dataset_loader, cv_splitter, scorers)	Register a classification task to the benchmark.
`register_stored_tasks`()	Register stored tasks from datasets, metrics, and CV splitters.
`run`([output_file, force_rerun])	Run the benchmarking for all tasks and estimators.