BaseBenchmark#
- class BaseBenchmark(id_format: str | None = None)[source]#
Base class for benchmarks.
A benchmark consists of a set of tasks and a set of estimators.
- Parameters:
- id_format: str, optional (default=None)
A regex used to enforce task/estimator ID to match a certain format if None, no format is enforced on task/estimator ID
Methods
add_estimator
(estimator[, estimator_id])Register an estimator to the benchmark.
run
(output_file)Run the benchmark.
- add_estimator(estimator: BaseEstimator, estimator_id: str | None = None)[source]#
Register an estimator to the benchmark.
- Parameters:
- estimatorDict, List or BaseEstimator object
Estimator to add to the benchmark. If Dict, keys are estimator_ids used to customise identifier ID and values are estimators. If List, each element is an estimator. estimator_ids are generated automatically using the estimator’s class name.
- estimator_idstr, optional (default=None)
Identifier for estimator. If none given then uses estimator’s class name.