Benchmarking

The sktime.benchmarking module contains functionality to perform benchmarking.

Base

BaseBenchmark([id_format])

Base class for benchmarks.

ForecastingBenchmark([id_format, backend, ...])

Forecasting benchmark.

BaseMetric(name, **kwargs)
BaseResults()	Base results class.
BaseDataset(name)	Base dataset class.
HDDBaseResults(path)	HDD results.
HDDBaseDataset(path, name)	HDD dataset.

UEADataset(path, name[, suffix_train, ...])	Represent a dataset in UEA/UCR format on the hard-drive.
RAMDataset(dataset, name)	Represent a dataset in RAM.

Evaluator(results)

Analyze results of machine learning experiments.

run_clustering_experiment(trainX, clusterer, ...)	Run a clustering experiment and save the results to file.
load_and_run_clustering_experiment(...[, ...])	Run a clustering experiment.
run_classification_experiment(X_train, ...)	Run a classification experiment and save the results to file.
load_and_run_classification_experiment(...)	Load a dataset and run a classification experiment.

Orchestrator(tasks, datasets, strategies, ...)

Fit and predict one or more estimators on one or more datasets.

RAMResults()	In-memory results.
HDDResults(path)	HDD results.

BaseStrategy(estimator[, name])	Abstract base strategy class.
BaseSupervisedLearningStrategy(estimator[, name])	Abstract strategy class for time series supervised learning.
TSCStrategy(estimator[, name])	Strategy for time series classification.
TSRStrategy(estimator[, name])	Strategy for time series regression.

BaseTask(target[, features, metadata])	Abstract base task class.
TSCTask(target[, features, metadata])	Time series classification task.
TSRTask(target[, features, metadata])	Time series regression task.

PairwiseMetric(func[, name])	Compute metric pairwise.
AggregateMetric(func[, method, name])	Compute metric pairwise.