BaseBenchmark#

class BaseBenchmark(id_format: str | None = None)[source]#

Base class for benchmarks.

A benchmark consists of a set of tasks and a set of estimators.

Parameters:
id_format: str, optional (default=None)

A regex used to enforce task/estimator ID to match a certain format if None, no format is enforced on task/estimator ID

Methods

add_estimator(estimator[, estimator_id])

Register an estimator to the benchmark.

run(output_file)

Run the benchmark.

add_estimator(estimator: BaseEstimator, estimator_id: str | None = None)[source]#

Register an estimator to the benchmark.

Parameters:
estimatorDict, List or BaseEstimator object

Estimator to add to the benchmark. If Dict, keys are estimator_ids used to customise identifier ID and values are estimators. If List, each element is an estimator. estimator_ids are generated automatically using the estimator’s class name.

estimator_idstr, optional (default=None)

Identifier for estimator. If none given then uses estimator’s class name.

run(output_file: str) DataFrame[source]#

Run the benchmark.

Parameters:
output_filestr

Path to write results output file to.

Returns:
pandas DataFrame of results