Testing and continuous integration

This page gives a summary of:

• testing for contributors - code style and local testing

• testing for maintainers - continuous integration

If you are a contributor or developer, ensure that you have set up your developer environment, and installed a development version of sktime.

sktime use continuous integration (CI) services on GitHub to automatically check if new pull requests do not break anything and meet code quality standards such as a common coding style.

Local Testing

If you contribute to sktime, the below gives you a guide on how to test your code locally before you make a pull request.

We recommend:

• set up code quality checks in your local dev IDE

• learn how to use the check_estimator utility for estimators and sktime objects

• advanced contributions: ensure you can run the full pytest test suite locally, via your dev IDE, console, or docker

Prerequisite: local python environment

Local testing requires a development version of sktime, follow the link for detail instructions.

In your environment, ensure you have an editable development version of sktime with developer dependencies. To install, if not already installed:

pip install -e .[dev]

This installs an editable development version of sktime which will include the changes you make.

Note

For trouble shooting on different operating systems, please see our detailed installation instructions.

Code quality checks

We use pre-commit for code quality checks (a process we also refer to as “linting” checks).

We recommend that you also set this up locally as it will ensure that you never run into code quality errors when you make your first PR! These checks run automatically before you make a new commit. To setup, simply navigate to the sktime folder and install our pre-commit configuration:

pre-commit install

pre-commit should now automatically run anything you make a commit! Please let us know if you encounter any issues getting this setup.

For a detailed guide on code quality and linting for developers, see Coding standards.

Testing objects via check_estimator

For contributions that are localized to estimators or objects, the check_estimator utility can be used.

For this, follow the instructions in the estimator development guide

Full test suite runs via pytest

The full test suite can be run locally via pytest, which sktime uses for its testing framework.

To run all tests via the console via make (only unix based OS):

make test

or, from a console with pytest in the path, from the repository root:

pytest ./sktime

Further, developer IDEs such as pycharm or vs code will automatically recognize the tests via pytest, refer to the documentation of the IDEs for testing via the embedded graphical user interface.

Running docstring examples via doctest

sktime’s Python modules are equipped with docstrings that include examples demonstrating the usage of specific classes within each module. These examples can be executed using doctest to ensure the expected functionality of the modules after making modifications, thereby identifying any unforeseen bugs.

To run doctest on all the files with pytest, navigate to the root directory and execute the following command:

pytest --doctest-modules

To run doctest on all the files without pytest, navigate to the root directory and execute the following command:

(for UNIX based OS)

find . -name "*.py" -print0 | xargs -0 python -m doctest -v -o=ELLIPSIS

(for windows)

for /r %G in (*.py) do python -m doctest -v "%G" -o=ELLIPSIS

To run doctest on a specific module, navigate to the directory where the module is located and execute the following command:

python -m doctest -v -o=ELLIPSIS {filename}

Executing this command will display the test results for all the docstrings contained within the module.

Alternative: dockerized testing

We also provide an option to execute the test suite via docker containers. This requires a local docker installation. To install, follow the instructions here.

The docker images for the tests are in the folder build_tools/docker, with the image of name PYTHON_VERSION based on the following python versions:

Python version	PYTHON_VERSION
3.8 | py38
3.9 | py39
3.10 | py310
3.11 | py311
3.12 | py312

The dockerized tests can be also executed via make, via the command make dockertest PYTHON_VERSION=<python version>. The PYTHON_VERSION argument specifies the python version and is the same string as in the table above. For example, to execute the tests in the Python version 3.8, use make dockertest PYTHON_VERSION=py38.

Continuous integration

Infrastructure overview

This section gives an overview of the infrastructure and continuous integration services we use.

Platforms	Operation	Configuration
GitHub Action	Build/test/distribute on Linux, MacOS and Windows, run code quality checks	.github/workflows/
Read the Docs	Build/deploy documentation	.readthedocs.yml
Codecov	Test coverage	.codecov.yml, .coveragerc

Additional scripts used for building, unit testing and distribution can be found in build_tools/.

Test coverage

We use coverage, the pytest-cov plugin, and codecov for test coverage.