A community-maintained Python framework for creating mathematical animations.
Updated docs for the test refactor (Tests PR 2/3) (#1835)
Hugues Devimeux2ab04aee10d0
diff --git a/docs/source/contributing/testing.rst b/docs/source/contributing/testing.rstindex 4a4e56a09..897dc5e0f 100644--- a/docs/source/contributing/testing.rst+++ b/docs/source/contributing/testing.rst@@ -5,7 +5,7 @@ If you are adding new features to manim, you should add appropriate tests for thmanim from breaking at each change by checking that no otherfeature has been broken and/or been unintentionally modified.-How Manim Tests+How Manim tests---------------Manim uses pytest as its testing framework.@@ -20,7 +20,7 @@ Some useful pytest flags:- ``--skip_slow`` will skip the (arbitrarily) slow tests-- ``--show_diff`` will show a visual comparison in case an unit test is failing.+- ``--show_diff`` will show a visual comparison in case a unit test is failing.How it Works@@ -34,9 +34,8 @@ At the moment there are three types of tests:``Mobject``, that checks if it can be added to a Scene, etc.#. Graphical unit tests:-Because ``manim`` is a graphics library, we test frames. To do so, we create test scenes that render a specific feature.- When pytest runs, it compares the last frame of every render to the control data; If it matches, the tests+ When pytest runs, it compares the result of the test to the control data, either at 6 fps or just the last frame. If it matches, the testspass. If the test and control data differ, the tests fail. You canuse ``--show_diff`` flag with ``pytest`` to visually see the differences.@@ -167,69 +166,63 @@ For example, to test the ``Circle`` VMobject which resides in``manim/mobject/geometry.py``, add the CircleTest to``test/test_geometry.py``.-In ``test_geometry.py``, add:+The name of the module is indicated by the variable __module_test__, that **must** be declared in any graphical test file. The module name is used to store the graphical control data.++.. important::+ You will need to use the ``frames_comparison`` decorator to create a test. The test function **must** accept a+ parameter named ``scene`` that will be used like ``self`` in a standard ``construct`` method.++Here's an example in ``test_geometry.py``:.. code:: python- class CircleTest(Scene):- def construct(self):- circle = Circle()- self.play(Animation(circle))+ from manim import *+ from tests.test_graphical_units.testing.frames_comparison import frames_comparison-Scene names follow the syntax: ``<thing_to_test>Test``. In the example above,-we are testing whether Circle properly shows up with the generic-``Animation`` and not any specific animation.+ __module_test__ = "geometry"-.. note::- If the file already exists, edit it and add the test within the file. The- ``Scene`` will be tested thanks to the ``GraphicalUnitTester`` that lives- in ``tests/utils/GraphicalUnitTester.py``. Import it with ``from- ..utils.GraphicalUnitTester import GraphicalUnitTester``.+ @frames_comparison+ def test_circle(scene):+ circle = Circle()+ scene.play(Animation(circle))-To test all the scenes in the module, we do the following:+The decorator can be used with or without parentheses. **By default, the test only tests the last frame. To enable multi-frame testing, you have to set ``last_frame=False`` in the parameters.**.. code:: python- @pytest.mark.parametrize("scene_to_test", get_scenes_to_test(__name__), indirect=False)- def test_scene(scene_to_test, tmpdir, show_diff):- GraphicalUnitTester(scene_to_test[1], MODULE_NAME, tmpdir).test(show_diff=show_diff)+ @frames_comparison(last_frame=False)+ def test_circle(scene):+ circle = Circle()+ scene.play(Animation(circle))-The first line is a `pytest decorator-<https://docs.pytest.org/en/stable/parametrize.html>`_.-It is used to run a test function several times with different-parameters. Here, we pass in all the scenes as arguments.+You can also specify, when needed, which base scene you need (ThreeDScene, for example) :-.. warning::- If you run pytest now, you will get a ``FileNotFound`` error. This is because- you have not created control data for your test.+.. code:: python-Next, we'll want to create control data for ``CircleTest``. In-``tests/template_generate_graphical_units_data.py``, there exists the-function, ``set_test_scene``, for this purpose.+ @frames_comparison(last_frame=False, base_scene=ThreeDScene)+ def test_circle(scene):+ circle = Circle()+ scene.play(Animation(circle))-It will look something like this:+Feel free to check the documentation of ``@frames_comparison`` for more.-.. code:: python+Note that tests name must follow the syntax ``test_<thing_to_test>``, otherwise pytest will not recognize it as a test.++.. warning::+ If you run pytest now, you will get a ``FileNotFound`` error. This is because+ you have not created control data for your test.- class CircleTest(Scene):- def construct(self):- circle = Circle()- self.play(Animation(circle))+To create the control data for your test, you have to use the flag ``--set_test`` along with pytest.+For the example above, it would be+.. code-block:: bash- set_test_scene(CircleTest, "geometry")+ pytest test_geometry.py::test_circle --set_test -s-``set_test_scene`` takes two parameters: the scene to test, and the-module name. You can generate the test data by running the file (it suffices to type the name of the file in the terminal; you do not have to run-it like how you would normally run manim files). It will automatically generate the control data in the-right directory (in this case,-``tests/control_data/graphical_units_data/geometry/CircleTest.npz``).+(``-s`` is here to see manim logs, so you can see what's going on).-Please make sure to add the control data to git as-soon as it is produced with ``git add <your-control-data.npz>``. However, do not-include changes to the template script (template\_generate\_graphical\_units\_data.py) in your pull request so that others-may continue to use the unmodified file to generate their own tests.+Please make sure to add the control data to git as soon as it is produced with ``git add <your-control-data.npz>``.Videos tests