ParquetStorageHandler

ParquetStorageHandler

class ParquetStorageHandler(path)[source]

Storage handler for Parquet files, with ending .parquet.

Loads and saves results in Parquet format, as follows. Each result is stored as a row in a Parquet file with the following columns:

• model_id: The ID of the model.

• validation_id: The ID of the validation run.

• folds: A dictionary of fold results,

  where each key is a fold ID and the value is a dictionary of scores and dataframes.

The results are stored in a tabular format, where each row corresponds to a single model-validation pair.

Columns are the following:

• model_id: The ID of the model.

• validation_id: The ID of the validation run.

• folds.{fold_id}.scores.{score_name}: The score value for the given fold and score name.

• folds.{fold_id}.ground_truth: The ground truth dataframe for the given fold.

• folds.{fold_id}.predictions: The predictions dataframe for the given fold.

• folds.{fold_id}.train_data: The training data dataframe for the given fold.

Columns ground_truth, predictions, and train_data are included only if they were requested during benchmarking.

Parameters:

pathstr or pathlib.Path

Path to the Parquet results file. Must refer to a file with a .parquet extension, not a directory.

Methods

is_applicable(path)	Return whether this handler supports the given results file path.
load()	Load benchmark results from the file at self.path.
save(results)	Save benchmark results to the Parquet file at self.path.