.. _Managing Side Effects:

Managing Side Effects and Randomness
------------------------------------
All Python tests and doctests are executed by one Python process.
All Blender tests and doctests are executed by one Blender process.
This means that if tests have side effects, then adding, removing, reordering and even rerunning tests can change the semantics for all tests.

The Problem
~~~~~~~~~~~
1. Getting random numbers from a random number generator changes its state.
   Suppose we use the same (deterministically seeded) random number generator for ``test_f``, ``test_g`` and ``test_h`` (in that order) and then omit ``test_f``.
   Then ``test_g`` would receive the random numbers that ``test_f`` previously received and and ``test_g`` would receive the random numbers that ``test_h`` previously received.

2. The vast majority of Blender specific functions are supposed to add, mutate and delete Blender data in some way or another.
   For example, if ``test_f`` and ``test_g`` are run in that order and ``test_f`` adds a Blender object, then ``test_g`` will fail if it assumes that ``bpy.data.objects`` is empty.

.. _Solution:

The Solution
~~~~~~~~~~~~
We use `pytest fixtures <https://docs.pytest.org/en/7.1.x/how-to/fixtures.html>`__ to provide a "clean slate" for each test.
They are passed to tests via the pytest config files.

For the random number generator fixture, see :ref:`Random tests`.
For the Blender fixture, see ``testing/utils/clear_scene_plugin.py``.



.. _Random tests:

Random tests
~~~~~~~~~~~~

It can be good for various reasons to use random instead of handpicked data for testing. To handle the randomness well, we use a mechanism based on `pytest fixtures <https://docs.pytest.org/en/7.1.x/how-to/fixtures.html>`_:

Basic usage
+++++++++++

Every random test should look roughly like this:

.. code-block:: python

    def test_something(rng):  # The argument "rng" is a magic keyword
        a = rng.standard_normal(3)
        assert a.size == 3

``rng`` stores a pre-seeded random generator instance returned by ``np.random.default_rng``. For basic usage, you can just treat the argument and variable ``rng`` like a magic incantation that gives you such a random generator. There are also two other fixtures ``seed`` and ``rng_seed`` that work similarly but give you seeds instead of random generators. ``rng_seed`` is the exact seed that is also used to seed ``rng`` and ``seed`` is a different seed you can use for other purposes.

The full mechanism
++++++++++++++++++

``rng`` is a pytest fixture, which is really a special kind of function. Writing a test function with an argument called ``rng`` "requests" the fixture, in pytest lingo. The return value of the fixture can then be used inside your test function as ``rng``. Fixtures can also request other fixtures. For details, see the pytest docs linked above.

The process works like this: Each test has a unique ID. There is also a global salt which can be set in the test config ``.ini`` files or passed to ``pytest`` using the ``--salt`` command line option. The fixtures use these two strings to generate **deterministic** seeds that are **local** and **unique** to each test and easily **refreshable** by changing the salt.

The code for all these fixtures can be found in ``testing/utils/rng_plugin.py``. It is a customized version of an existing pytest plugin. See the plugin's module description for details.