Running Tests

Tests can be roughly grouped into the following:

The Python tests are all the tests that are independent of Blender.
The Blender tests are the tests that use Blender.
The doctests are the tests run on the code snippets in our user guide. These are run within the Python and Blender test suites respectively.
The style tests are formatting tests on the code base.
The linters which nitpick your code so that reviewers won’t need to.

The test command-line interface: test.py

All tests can be run using the test.py cli. From project root directory, it can be invoked with the commands

./test.py

or

python3 test.py

By adding arguments to the above command, you can have more control over which tests to run and how to run them. To see all available tests run

python3 test.py --help

Passing more arguments

You can pass more arguments to the underlying test runners, e.g. pytest, sphinx and so on. Refer to the test.py help for more details. Some useful flags follow in the sequel, do refer to pytest -h, sphinx -h, black -h, flake8 -h for many more.

Display test names

Pytest only shows green and red dots by default, but it can also display the test names with

python3 test.py -p -- -v
python3 test.py -b -- -v
python3 test.py --blender-examples -- -v

Running tests and building documentation in parallel

This is particularly useful for Python tests (-p). Note that this doesn’t work for Blender tests (--blender) and Blender example tests (--blender -examples), because all of them manipulate global Blender state.

python3 test.py -p -- -v -n auto

and

# clean and build the docs
python3 test.py -d -- -j auto

# without cleaning, might result in buggy output
make SPHINXOPTS="-j auto" -C docs html

Don’t capture logs and output

This is particularly useful for the Blender examples, because otherwise pytest will swallow Blender’s rendering logs. It can also be used to see the result of print even when the tests succeed This is incompatible with -n auto.

python3 test.py -p -- -v -s
python3 test.py --blender-examples -- -v -s

Stopping at first error (-x)

python3 test.py -p -- -x

Checking which tests would be collected (–co)

Use the --co or --collect-only flag to get a list of tests that would be run, but don’t run them.

python3 test.py -p -- --co

Collect tests from a directory or file

Works for both Python and Blender tests.

./test.py -p -- testing/tests/ddg/datastructures

./test.py -b -- testing/tests/ddg/blender/mesh_test.py testing/tests/ddg/blender/bmesh_test.py

Note that this overwrites the test paths defined in the Python and Blender test configuration (see Python Tests and Blender Tests). That is, you could end up running tests requiring Blender when you try to run Python tests. To avoid this, either make sure all the tests in you subdirectory correspond to what you intend to test. Alternatively, use finer control mechanisms such as marks (-m) or substring matching (-k), see below.

Use marks to collect tests (-m)

You can add marks to your test functions to select which ones to test or exclude. To do this import pytest in your Python test file and add the following decorator above it : @pytest.mark.<your-mark>

You can then control the test collection using the flag -m followed by a MARKEXPR. For example

./test.py -p -- -m 'mark1 and not mark2'

would run all tests marked mark1 but omit the ones marked mark2. More detail on MARKEXPR can be found in pytest -h.

Use a substring to collect tests (-k)

The -k option followed by an expression selects functions and files names matching the expression. For example

./test.py -p -- -k 'quadric and not sphere'

would only run Python tests that have the substring quadric in their name but not the substring sphere.

Test Coverage

To create a html code coverage report for the Python API use

python3 test.py --coverage
firefox coverage/index.html