A community-maintained Python framework for creating mathematical animations.
A deep dive into Manim's internals
==================================
**Author:** `Benjamin Hackl <https://benjamin-hackl.at>`__
.. admonition:: Disclaimer
This guide was originally written for version ``v0.16.0`` and primarily
treats the Cairo renderer. Its high-level lifecycle discussion has been
updated for the :class:`.Manager` coordination layer, but some lower-level
implementation details may still differ from the latest version of Manim.
Introduction
------------
Manim can be a wonderful library, if it behaves the way you would like it to,
and/or the way you expect it to. Unfortunately, this is not always the case
(as you probably know if you have played with some manimations yourself already).
To understand where things *go wrong*, digging through the library's source code
is sometimes the only option -- but in order to do that, you need to know where
to start digging.
This article is intended as some sort of life line through the render process.
We aim to give an appropriate amount of detail describing what happens when
Manim reads your scene code and produces the corresponding animation. Throughout
this article, we will focus on the following toy example::
from manim import *
class ToyExample(Scene):
def construct(self):
orange_square = Square(color=ORANGE, fill_opacity=0.5)
blue_circle = Circle(color=BLUE, fill_opacity=0.5)
self.add(orange_square)
self.play(ReplacementTransform(orange_square, blue_circle, run_time=3))
small_dot = Dot()
small_dot.add_updater(lambda mob: mob.next_to(blue_circle, DOWN))
self.play(Create(small_dot))
self.play(blue_circle.animate.shift(RIGHT))
self.wait()
self.play(FadeOut(blue_circle, small_dot))
Before we go into details or even look at the rendered output of this scene,
let us first describe verbally what happens in this *manimation*. In the first
three lines of the ``construct`` method, a :class:`.Square` and a :class:`.Circle`
are initialized, then the square is added to the scene. The first frame of the
rendered output should thus show an orange square.
Then the actual animations happen: the square first transforms into a circle,
then a :class:`.Dot` is created (Where do you guess the dot is located when
it is first added to the scene? Answering this already requires detailed
knowledge about the render process.). The dot has an updater attached to it, and
as the circle moves right, the dot moves with it. In the end, all mobjects are
faded out.
Actually rendering the code yields the following video:
.. manim:: ToyExample
:hide_source:
class ToyExample(Scene):
def construct(self):
orange_square = Square(color=ORANGE, fill_opacity=0.5)
blue_circle = Circle(color=BLUE, fill_opacity=0.5)
self.add(orange_square)
self.play(ReplacementTransform(orange_square, blue_circle, run_time=3))
small_dot = Dot()
small_dot.add_updater(lambda mob: mob.next_to(blue_circle, DOWN))
self.play(Create(small_dot))
self.play(blue_circle.animate.shift(RIGHT))
self.wait()
self.play(FadeOut(blue_circle, small_dot))
For this example, the output (fortunately) coincides with our expectations.
Overview
--------
Because there is a lot of information in this article, here is a brief overview
discussing the contents of the following chapters on a very high level.
- `Preliminaries`_: In this chapter we unravel all the steps that take place
to prepare a scene for rendering; right until the point where the user-overridden
``construct`` method is ran. This includes a brief discussion on using Manim's CLI
versus other means of rendering (e.g., via Jupyter notebooks, or in your Python
script by calling the :meth:`.Scene.render` method yourself).
- `Mobject Initialization`_: For the second chapter we dive into creating and handling
Mobjects, the basic elements that should be displayed in our scene.
We discuss the :class:`.Mobject` base class, how there are essentially
three different types of Mobjects, and then discuss the most important of them,
vectorized Mobjects. In particular, we describe the internal point data structure
that governs how the mechanism responsible for drawing the vectorized Mobject
to the screen sets the corresponding Bézier curves. We conclude the chapter
with a tour into :meth:`.Scene.add`, the bookkeeping mechanism controlling which
mobjects should be rendered.
- `Animations and the Render Loop`_: And finally, in the last chapter we walk
through the instantiation of :class:`.Animation` objects (the blueprints that
hold information on how Mobjects should be modified when the render loop runs),
followed by a investigation of the infamous :meth:`.Scene.play` call. We will
see that there are three relevant parts in a :meth:`.Scene.play` call;
a part in which the passed animations and keyword arguments are processed
and prepared, followed by the actual "render loop" in which the library
steps through a time line and renders frame by frame. The final part
does some post-processing and seals the encoding job for a short video segment
("partial movie file") before cleaning up for the next call to
:meth:`.Scene.play`. The :class:`.Manager` is the coordination layer around
these calls, while the renderer still implements most of the per-animation
processing described here. Depending on the encoding configuration, the partial
movie file can continue encoding while the next animation is rendered. In the
end, after all of :meth:`.Scene.construct` has been run, the library waits for
any remaining encoding jobs and combines the partial movie files into one video.
And with that, let us get *in medias res*.
Preliminaries
-------------
Importing the library
^^^^^^^^^^^^^^^^^^^^^
You can ask Manim to render in several ways. For example, you can run
``manim -qm -p file_name.py ToyExample`` or render directly from a Python script
with a snippet like
::
with tempconfig({"quality": "medium_quality", "preview": True}):
scene = ToyExample()
scene.render()
Whether you are rendering this way or from a Jupyter notebook, you are still
telling your Python interpreter to import the library. The usual pattern used to
do this is
::
from manim import *
which (while being a debatable strategy in general) imports a lot of classes and
functions shipped with the library and makes them available in your global name space.
I explicitly avoided stating that it imports **all** classes and functions of the
library, because it does not do that: Manim makes use of the practice described
in `Section 6.4.1 of the Python tutorial <https://docs.python.org/3/tutorial/modules.html#importing-from-a-package>`__,
and all module members that should be exposed to the user upon running the ``*``-import
are explicitly declared in the ``__all__`` variable of the module.
Manim also uses this strategy internally: taking a peek at the file that is run when
the import is called, ``__init__.py`` (see
`here <https://github.com/ManimCommunity/manim/blob/main/manim/__init__.py>`__),
you will notice that most of the code in that module is concerned with importing
members from various different submodules, again using ``*``-imports.
.. hint::
If you would ever contribute a new submodule to Manim, the main
``__init__.py`` is where it would have to be listed in order to make its
members accessible to users after importing the library.
In that file, there is one particular import at the beginning of the file however,
namely::
from ._config import *
This initializes Manim's global configuration system, which is used in various places
throughout the library. After the library runs this line, the current configuration
options are set. The code in there takes care of reading the options in your ``.cfg``
files (all users have at least the global one that is shipped with the library)
as well as correctly handling command line arguments (if you used the CLI to render).
You can read more about the config system in the
:doc:`corresponding thematic guide </guides/configuration>`, and if you are interested in learning
more about the internals of the configuration system and how it is initialized,
follow the code flow starting in `the config module's init file
<https://github.com/ManimCommunity/manim/blob/main/manim/_config/__init__.py>`__.
Now that the library is imported, we can turn our attention to the next step:
reading your scene code (which is not particularly exciting, Python just creates
a new class ``ToyExample`` based on our code; Manim is virtually not involved
in that step, with the exception that ``ToyExample`` inherits from ``Scene``).
However, with the ``ToyExample`` class created and ready to go, there is a new
excellent question to answer: how is the code in our ``construct`` method
actually executed?
Scene instantiation and rendering
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The answer to this question depends on how exactly you are running the code.
To make things a bit clearer, let us first consider the case that you
have created a file ``toy_example.py`` which looks like this::
from manim import *
class ToyExample(Scene):
def construct(self):
orange_square = Square(color=ORANGE, fill_opacity=0.5)
blue_circle = Circle(color=BLUE, fill_opacity=0.5)
self.add(orange_square)
self.play(ReplacementTransform(orange_square, blue_circle, run_time=3))
small_dot = Dot()
small_dot.add_updater(lambda mob: mob.next_to(blue_circle, DOWN))
self.play(Create(small_dot))
self.play(blue_circle.animate.shift(RIGHT))
self.wait()
self.play(FadeOut(blue_circle, small_dot))
with tempconfig({"quality": "medium_quality", "preview": True}):
scene = ToyExample()
scene.render()
With such a file, the desired scene is rendered by simply running this Python
script via ``python toy_example.py``. Then, as described above, the library
is imported and Python has read and defined the ``ToyExample`` class (but,
read carefully: *no instance of this class has been created yet*).
At this point, the interpreter is about to enter the ``tempconfig`` context
manager. Even if you have not seen Manim's ``tempconfig`` before, its name
already suggests what it does: it creates a copy of the current state of the
configuration, applies the changes to the key-value pairs in the passed
dictionary, and upon leaving the context the original version of the
configuration is restored. TL;DR: it provides a fancy way of temporarily setting
configuration options.
Inside the context manager, two things happen: an actual ``ToyExample`` scene
object is instantiated, and its ``render`` method is called. The render lifecycle
is coordinated by a :class:`.Manager` attached to that scene. The CLI attaches a
manager explicitly, while direct calls to :meth:`.Scene.render` create one lazily
if necessary. To illustrate this, let us briefly look at the two most common ways
of rendering scenes:
**Command Line Interface.** When using the CLI and running the command
``manim -qm -p toy_example.py ToyExample`` in your terminal, the actual
entry point is Manim's ``__main__.py`` file (located
`here <https://github.com/ManimCommunity/manim/blob/main/manim/__main__.py>`__.
Manim uses `Click <https://click.palletsprojects.com/en/8.0.x/>`__ to implement
the command line interface, and the corresponding code is located in Manim's
``cli`` module (https://github.com/ManimCommunity/manim/tree/main/manim/cli).
The corresponding code instantiates the scene, attaches a :class:`.Manager`, and
then calls the scene's render method in the
`render command <https://github.com/ManimCommunity/manim/blob/main/manim/cli/render/commands.py>`__.
**Jupyter notebooks.** In Jupyter notebooks, the communication with the library
is handled by the ``%%manim`` magic command, which is implemented in the
``manim.utils.ipython_magic`` module. There is
:meth:`some documentation <.ManimMagic.manim>` available for the magic command.
The implementation instantiates the requested scene and calls its render method;
the scene attaches a manager lazily through that entry point.
Now that we know that either way, a :class:`.Scene` object is created, let us investigate
what Manim does when that happens. When instantiating our scene object
::
scene = ToyExample()
the ``Scene.__init__`` method is called, given that we did not implement our own initialization
method. Inspecting the corresponding code (see
`here <https://github.com/ManimCommunity/manim/blob/main/manim/scene/scene.py>`__)
reveals that ``Scene.__init__`` first sets several attributes of the scene objects that do not
depend on any configuration options set in ``config``. Then the scene inspects the value of
``config.renderer``, and based on its value, either instantiates a ``CairoRenderer`` or an
``OpenGLRenderer`` object and assigns it to its ``renderer`` attribute.
After selecting the renderer, the scene resolves the mutable configuration into
one immutable render-session specification. In abbreviated form, initialization
continues as follows:
::
self.session_spec = resolve_render_session(
config,
self.renderer.capabilities,
renderer_name=type(self.renderer).__name__,
)
self.output_plan = resolve_output_plan(
resolve_media_layout(...),
self.session_spec.output,
scene_name=type(self).__name__,
requested_output_name=...,
)
self.file_writer_settings = _SceneFileWriterSettings(
plan=self.output_plan,
video_encoder=self.session_spec.video_encoder,
max_inflight_encoders=config.max_inflight_encoders,
encoder_queue_size=config.encoder_queue_size,
max_files_cached=config.max_files_cached,
assets_dir=...,
)
self.renderer.init_scene(
self,
self.session_spec,
self.file_writer_settings,
)
The session specification separates primary artifact intent (an ``OutputSpec``)
from presentation requests such as opening the completed artifact or displaying
a live preview. For video output it also contains the resolved segment profile:
container, codec, pixel format, dimensions, exact frame rate, and codec options.
It records dry-run execution separately from artifact selection. A dry run
requests semantic scene evaluation without rasterizing
frames or using media and cache resources. In contrast, ``format = none`` only
suppresses the primary artifact; an OpenGL live preview with automatic output
still rasterizes and displays frames without writing a file. Both requests have
an effective output format of ``none``, so the session's ``dry_run`` field
preserves the intended execution behavior.
The current renderer loops do not yet enforce no-raster dry runs; the session
field preserves that request for Manager-owned execution. Resolution also
validates requests against the selected renderer's capabilities. For example,
Cairo rejects live preview, while OpenGL advertises support for it. A concrete
format records the live preview as well.
The scene then resolves existing directory templates once into an immutable
output plan containing exact scene-specific artifact, section, image-sequence,
and cache paths. Planning performs no file I/O and creates no directories. The
resolved format determines the artifact suffix; ``output_file`` supplies only a
name and cannot change the format.
The scene combines the output plan and segment profile with the encoder-pool,
cache-maintenance, and sound-asset inputs in immutable
``_SceneFileWriterSettings``. Both renderers instantiate a
:class:`.SceneFileWriter` from these settings. The writer does not retain a
renderer reference or read mutable global configuration. Directories are created
lazily when their owning operation first writes. The writer remains Manim's
interface to ``libav`` for media assembly. The Cairo renderer (see the
implementation `here
<https://github.com/ManimCommunity/manim/blob/main/manim/renderer/cairo_renderer.py>`__)
does not require further renderer-specific initialization. OpenGL creates a
window only when the resolved presentation specification requests a live preview.
The ``-p`` / ``--preview`` option does not create this window; it opens the
completed artifact after rendering.
After the renderer has been instantiated and initialized its file writer, the scene
populates further initial attributes (notable mention: the ``mobjects`` attribute
which keeps track of the mobjects that have been added to the scene). Its ``manager``
attribute is initially ``None`` unless the caller attaches a manager explicitly.
.. warning::
The scene captures the immutable session specification and output plan before
renderer initialization. The manager coordinates the scene lifecycle, while
the renderer still owns its camera, clock, play count, skip state, and file
writer. The manager exposes these through forwarding properties.
The rest of this article is concerned with the last line in our toy example script::
scene.render()
This is where the actual magic happens. :meth:`.Scene.render` obtains the scene's
manager, creating and attaching a :class:`.Manager` if necessary, and delegates to
:meth:`.Manager.render`. The manager describes the full *render cycle* of a scene
through four methods: :meth:`.Manager.setup`, :meth:`.Manager.construct`,
:meth:`.Manager.tear_down`, and :meth:`.Manager.post_construct`.
The first three call the corresponding customizable scene hooks:
- :meth:`.Scene.setup`, which is intended for preparing and, well, *setting up*
the scene for your animation (e.g., adding initial mobjects, assigning custom
attributes to your scene class, etc.),
- :meth:`.Scene.construct`, which is the *script* for your screen play and
contains programmatic descriptions of your animations, and
- :meth:`.Scene.tear_down`, which is intended for any operations you might
want to run on the scene after the last frame has already been rendered
(for example, this could run some code that generates a custom thumbnail
for the video based on the state of the objects in the scene -- this
hook is more relevant for situations where Manim is used within other
Python scripts).
After these hooks have run, :meth:`.Manager.post_construct` checks whether the
resolved output can be finalized. An explicit video format for a scene without
play calls raises an error instead of silently changing the requested artifact.
Otherwise, the manager asks the renderer to finish the scene. For Cairo this
calls :meth:`.CairoRenderer.scene_finished`, which tells the
:class:`.SceneFileWriter` to finish any time-based output. For video output, the
file writer waits for partial movie files that are still being encoded and
combines them into the final movie. Last-frame PNG output fast-forwards
animations and renders a single image. When automatic output selected a video
for a scene without play calls, the writer saves that image as a fallback PNG and
the manager logs a warning. The writer records the completed artifact as
``final_file_path``. After finalization, the manager uses the presentation
specification to open the artifact or reveal it in the file browser.
**Back in our toy example,** the call to :meth:`.Scene.render` creates a manager,
then :meth:`.Manager.render` triggers :meth:`.Scene.setup` (which only consists of
``pass``), followed by :meth:`.Scene.construct`. At this point, our *animation
script* is run, starting with the initialization of ``orange_square``.
Mobject Initialization
----------------------
Mobjects are, in a nutshell, the Python objects that represent all the
*things* we want to display in our scene. Before we follow our debugger
into the depths of mobject initialization code, it makes sense to
discuss Manim's different types of Mobjects and their basic data
structure.
What even is a Mobject?
^^^^^^^^^^^^^^^^^^^^^^^
:class:`.Mobject` stands for *mathematical object* or *Manim object*
(depends on who you ask 😄). The Python class :class:`.Mobject` is
the base class for all objects that should be displayed on screen.
Looking at the `initialization method
<https://github.com/ManimCommunity/manim/blob/5d72d9cfa2e3dd21c844b1da807576f5a7194fda/manim/mobject/mobject.py#L94>`__
of :class:`.Mobject`, you will find that not too much happens in there:
- some initial attribute values are assigned, like ``name`` (which makes the
render logs mention the name of the mobject instead of its type),
``submobjects`` (initially an empty list), ``color``, and some others.
- Then, two methods related to *points* are called: ``reset_points``
followed by ``generate_points``,
- and finally, ``init_colors`` is called.
Digging deeper, you will find that :meth:`.Mobject.reset_points` simply
sets the ``points`` attribute of the mobject to an empty NumPy vector,
while the other two methods, :meth:`.Mobject.generate_points` and
:meth:`.Mobject.init_colors` are just implemented as ``pass``.
This makes sense: :class:`.Mobject` is not supposed to be used as
an *actual* object that is displayed on screen; in fact the camera
(which we will discuss later in more detail; it is the class that is,
for the Cairo renderer, responsible for "taking a picture" of the
current scene) does not process "pure" :class:`Mobjects <.Mobject>`
in any way, they *cannot* even appear in the rendered output.
This is where different types of mobjects come into play. Roughly
speaking, the Cairo renderer setup knows three different types of
mobjects that can be rendered:
- :class:`.ImageMobject`, which represent images that you can display
in your scene,
- :class:`.PMobject`, which are very special mobjects used to represent
point clouds; we will not discuss them further in this guide,
- :class:`.VMobject`, which are *vectorized mobjects*, that is, mobjects
that consist of points that are connected via curves. These are pretty
much everywhere, and we will discuss them in detail in the next section.
... and what are VMobjects?
^^^^^^^^^^^^^^^^^^^^^^^^^^^
As just mentioned, :class:`VMobjects <.VMobject>` represent vectorized
mobjects. To render a :class:`.VMobject`, the camera looks at the
``points`` attribute of a :class:`.VMobject` and divides it into sets
of four points each. Each of these sets is then used to construct a
cubic Bézier curve with the first and last entry describing the
end points of the curve ("anchors"), and the second and third entry
describing the control points in between ("handles").
.. hint::
To learn more about Bézier curves, take a look at the excellent
online textbook `A Primer on Bézier curves <https://pomax.github.io/bezierinfo/>`__
by `Pomax <https://twitter.com/TheRealPomax>`__ -- there is a playground representing
cubic Bézier curves `in §1 <https://pomax.github.io/bezierinfo/#introduction>`__,
the red and yellow points are "anchors", and the green and blue
points are "handles".
In contrast to :class:`.Mobject`, :class:`.VMobject` can be displayed
on screen (even though, technically, it is still considered a base class).
To illustrate how points are processed, consider the following short example
of a :class:`.VMobject` with 8 points (and thus made out of 8/4 = 2 cubic
Bézier curves). The resulting :class:`.VMobject` is drawn in green.
The handles are drawn as red dots with a line to their closest anchor.
.. manim:: VMobjectDemo
:save_last_frame:
class VMobjectDemo(Scene):
def construct(self):
plane = NumberPlane()
my_vmobject = VMobject(color=GREEN).set_points(
[
[-2, -1, 0], # start of first curve
[-3, 1, 0],
[0, 3, 0],
[1, 3, 0], # end of first curve
[1, 3, 0], # start of second curve
[0, 1, 0],
[4, 3, 0],
[4, -2, 0], # end of second curve
]
)
handles = [
Dot(point, color=RED) for point in
[[-3, 1, 0], [0, 3, 0], [0, 1, 0], [4, 3, 0]]
]
handle_lines = [
Line(
my_vmobject.points[ind],
my_vmobject.points[ind+1],
color=RED,
stroke_width=2
) for ind in range(0, len(my_vmobject.points), 2)
]
self.add(plane, *handles, *handle_lines, my_vmobject)
.. warning::
Manually setting the points of your :class:`.VMobject` is usually
discouraged; there are specialized methods that can take care of
that for you -- but it might be relevant when implementing your own,
custom :class:`.VMobject`.
Squares and Circles: back to our Toy Example
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
With a basic understanding of different types of mobjects,
and an idea of how vectorized mobjects are built we can now
come back to our toy example and the execution of the
:meth:`.Scene.construct` method. In the first two lines
of our animation script, the ``orange_square`` and the
``blue_circle`` are initialized.
When creating the orange square by running
::
Square(color=ORANGE, fill_opacity=0.5)
the initialization method of :class:`.Square`,
``Square.__init__``, is called. `Looking at the
implementation <https://github.com/ManimCommunity/manim/blob/5d72d9cfa2e3dd21c844b1da807576f5a7194fda/manim/mobject/geometry/polygram.py#L607>`__,
we can see that the ``side_length`` attribute of the square is set,
and then
::
super().__init__(height=side_length, width=side_length, **kwargs)
is called. This ``super`` call is the Python way of calling the
initialization function of the parent class. As :class:`.Square`
inherits from :class:`.Rectangle`, the next method called
is ``Rectangle.__init__``. There, only the first three lines
are really relevant for us::
super().__init__(UR, UL, DL, DR, color=color, **kwargs)
self.stretch_to_fit_width(width)
self.stretch_to_fit_height(height)
First, the initialization function of the parent class of
:class:`.Rectangle` -- :class:`.Polygon` -- is called. The
four positional arguments passed are the four corners of
the polygon: ``UR`` is up right (and equal to ``UP + RIGHT``),
``UL`` is up left (and equal to ``UP + LEFT``), and so forth.
Before we follow our debugger deeper, let us observe what
happens with the constructed polygon: the remaining two lines
stretch the polygon to fit the specified width and height
such that a rectangle with the desired measurements is created.
The initialization function of :class:`.Polygon` is particularly
simple, it only calls the initialization function of its parent
class, :class:`.Polygram`. There, we have almost reached the end
of the chain: :class:`.Polygram` inherits from :class:`.VMobject`,
whose initialization function mainly sets the values of some
attributes (quite similar to ``Mobject.__init__``, but more specific
to the Bézier curves that make up the mobject).
After calling the initialization function of :class:`.VMobject`,
the constructor of :class:`.Polygram` also does something somewhat
odd: it sets the points (which, you might remember above, should
actually be set in a corresponding ``generate_points`` method
of :class:`.Polygram`).
.. warning::
In several instances, the implementation of mobjects does
not really stick to all aspects of Manim's interface. This
is unfortunate, and increasing consistency is something
that we actively work on. Help is welcome!
Without going too much into detail, :class:`.Polygram` sets its
``points`` attribute via :meth:`.VMobject.start_new_path`,
:meth:`.VMobject.add_points_as_corners`, which take care of
setting the quadruples of anchors and handles appropriately.
After the points are set, Python continues to process the
call stack until it reaches the method that was first called;
the initialization method of :class:`.Square`. After this,
the square is initialized and assigned to the ``orange_square``
variable.
The initialization of ``blue_circle`` is similar to the one of
``orange_square``, with the main difference being that the inheritance
chain of :class:`.Circle` is different. Let us briefly follow the trace
of the debugger:
The implementation of :meth:`.Circle.__init__` immediately calls
the initialization method of :class:`.Arc`, as a circle in Manim
is simply an arc with an angle of :math:`\tau = 2\pi`. When
initializing the arc, some basic attributes are set (like
``Arc.radius``, ``Arc.arc_center``, ``Arc.start_angle``, and
``Arc.angle``), and then the initialization method of its
parent class, :class:`.TipableVMobject`, is called (which is
a rather abstract base class for mobjects which a arrow tip can
be attached to). Note that in contrast to :class:`.Polygram`,
this class does **not** preemptively generate the points of the circle.
After that, things are less exciting: :class:`.TipableVMobject` again
sets some attributes relevant for adding arrow tips, and afterwards
passes to the initialization method of :class:`.VMobject`. From there,
:class:`.Mobject` is initialized and :meth:`.Mobject.generate_points`
is called, which actually runs the method implemented in
:meth:`.Arc.generate_points`.
After both our ``orange_square`` and the ``blue_circle`` are initialized,
the square is actually added to the scene. The :meth:`.Scene.add` method
is actually doing a few interesting things, so it is worth to dig a bit
deeper in the next section.
Adding Mobjects to the Scene
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The code in our ``construct`` method that is run next is
::
self.add(orange_square)
From a high-level point of view, :meth:`.Scene.add` adds the
``orange_square`` to the list of mobjects that should be rendered,
which is stored in the ``mobjects`` attribute of the scene. However,
it does so in a very careful way to avoid the situation that a mobject
is being added to the scene more than once. At a first glance, this
sounds like a simple task -- the problem is that ``Scene.mobjects``
is not a "flat" list of mobjects, but a list of mobjects which
might contain mobjects themselves, and so on.
Stepping through the code in :meth:`.Scene.add`, we see that first
it is checked whether we are currently using the OpenGL renderer
(which we are not) -- adding mobjects to the scene works slightly
different (and actually easier!) for the OpenGL renderer. Then, the
code branch for the Cairo renderer is entered and the list of so-called
foreground mobjects (which are rendered on top of all other mobjects)
is added to the list of passed mobjects. This is to ensure that the
foreground mobjects will stay above of the other mobjects, even after
adding the new ones. In our case, the list of foreground mobjects
is actually empty, and nothing changes.
Next, :meth:`.Scene.restructure_mobjects` is called with the list
of mobjects to be added as the ``to_remove`` argument, which might
sound odd at first. Practically, this ensures that mobjects are not
added twice, as mentioned above: if they were present in the scene
``Scene.mobjects`` list before (even if they were contained as a
child of some other mobject), they are first removed from the list.
The way :meth:`.Scene.restructure_mobjects` works is rather aggressive:
It always operates on a given list of mobjects; in the ``add`` method
two different lists occur: the default one, ``Scene.mobjects`` (no extra
keyword argument is passed), and ``Scene.moving_mobjects`` (which we will
discuss later in more detail). It iterates through all of the members of
the list, and checks whether any of the mobjects passed in ``to_remove``
are contained as children (in any nesting level). If so, **their parent
mobject is deconstructed** and their siblings are inserted directly
one level higher. Consider the following example::
>>> from manim import Scene, Square, Circle, Group
>>> test_scene = Scene()
>>> mob1 = Square()
>>> mob2 = Circle()
>>> mob_group = Group(mob1, mob2)
>>> test_scene.add(mob_group)
<manim.scene.scene.Scene object at ...>
>>> test_scene.mobjects
[Group]
>>> test_scene.restructure_mobjects(to_remove=[mob1])
<manim.scene.scene.Scene object at ...>
>>> test_scene.mobjects
[Circle]
Note that the group is disbanded and the circle moves into the
root layer of mobjects in ``test_scene.mobjects``.
After the mobject list is "restructured", the mobject to be added
are simply appended to ``Scene.mobjects``. In our toy example,
the ``Scene.mobjects`` list is actually empty, so the
``restructure_mobjects`` method does not actually do anything. The
``orange_square`` is simply added to ``Scene.mobjects``, and as
the aforementioned ``Scene.moving_mobjects`` list is, at this point,
also still empty, nothing happens and :meth:`.Scene.add` returns.
We will hear more about the ``moving_mobject`` list when we discuss
the render loop. Before we do that, let us look at the next line
of code in our toy example, which includes the initialization of
an animation class,
::
ReplacementTransform(orange_square, blue_circle, run_time=3)
Hence it is time to talk about :class:`.Animation`.
Animations and the Render Loop
------------------------------
Initializing animations
^^^^^^^^^^^^^^^^^^^^^^^
Before we follow the trace of the debugger, let us briefly discuss
the general structure of the (abstract) base class :class:`.Animation`.
An animation object holds all the information necessary for the renderer
to generate the corresponding frames. Animations (in the sense of
animation objects) in Manim are *always* tied to a specific mobject;
even in the case of :class:`.AnimationGroup` (which you should actually
think of as an animation on a group of mobjects rather than a group
of animations). Moreover, except for in a particular special case,
the run time of animations is also fixed and known beforehand.
The initialization of animations actually is not very exciting,
:meth:`.Animation.__init__` merely sets some attributes derived
from the passed keyword arguments and additionally ensures that
the ``Animation.starting_mobject`` and ``Animation.mobject``
attributes are populated. Once the animation is played, the
``starting_mobject`` attribute holds an unmodified copy of the
mobject the animation is attached to; during the initialization
it is set to a placeholder mobject. The ``mobject`` attribute
is set to the mobject the animation is attached to.
Animations have a few special methods which are called during the
render loop:
- :meth:`.Animation.begin`, which is called (as hinted by its name)
at the beginning of every animation, so before the first frame
is rendered. In it, all the required setup for the animation happens.
- :meth:`.Animation.finish` is the counterpart to the ``begin`` method
which is called at the end of the life cycle of the animation (after
the last frame has been rendered).
- :meth:`.Animation.interpolate` is the method that updates the mobject
attached to the animation to the corresponding animation completion
percentage. For example, if in the render loop,
``some_animation.interpolate(0.5)`` is called, the attached mobject
will be updated to the state where 50% of the animation are completed.
We will discuss details about these and some further animation methods
once we walk through the actual render loop. For now, we continue with
our toy example and the code that is run when initializing the
:class:`.ReplacementTransform` animation.
The initialization method of :class:`.ReplacementTransform` only
consists of a call to the constructor of its parent class,
:class:`.Transform`, with the additional keyword argument
``replace_mobject_with_target_in_scene`` set to ``True``.
:class:`.Transform` then sets attributes that control how the
points of the starting mobject are deformed into the points of
the target mobject, and then passes on to the initialization
method of :class:`.Animation`. Other basic properties of the
animation (like its ``run_time``, the ``rate_func``, etc.) are
processed there -- and then the animation object is fully
initialized and ready to be played.
The ``play`` call: preparing to enter Manim's render loop
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We are finally there, the render loop is in our reach. Let us
walk through the code that is run when :meth:`.Scene.play` is called.
.. hint::
Recall that this article is specifically about the Cairo renderer.
Up to here, things were more or less the same for the OpenGL renderer
as well; while some base mobjects might be different, the control flow
and lifecycle of mobjects is still more or less the same. There are more
substantial differences when it comes to the rendering loop.
As you will see when inspecting the method, :meth:`.Scene.play` first handles
an OpenGL interactive-thread guard and then delegates to :meth:`.Manager.play`.
The manager records the current renderer time and calls the renderer's ``play``
method, in our case :meth:`.CairoRenderer.play`. Once the renderer returns, the
manager uses the elapsed time to schedule an optional subcaption (see
:meth:`.Scene.play` and :meth:`.Scene.add_subcaption` for more information).
.. warning::
The manager currently coordinates the call but deliberately preserves the
existing renderer implementation. Most compilation, caching, frame iteration,
and output behavior described in the following paragraphs still belongs to the
renderer and scene during this transitional stage.
Inside :meth:`.CairoRenderer.play`, the renderer first checks whether
it may skip rendering of the current play call. This might happen, for example,
when ``-s`` is passed to the CLI (i.e., only the last frame should be rendered),
or when the ``-n`` flag is passed and the current play call is outside of the
specified render bounds. The "skipping status" is updated in form of the
call to :meth:`.CairoRenderer.update_skipping_status`.
Next, the renderer asks the scene to process the animations in the play
call so that renderer obtains all of the information it needs. To
be more concrete, :meth:`.Scene.compile_animation_data` is called,
which then takes care of several things:
- The method processes all animations and the keyword arguments passed
to the initial :meth:`.Scene.play` call. In particular, this means
that it makes sure all arguments passed to the play call are actually
animations (or ``.animate`` syntax calls, which are also assembled to
be actual :class:`.Animation`-objects at that point). It also propagates
any animation-related keyword arguments (like ``run_time``,
or ``rate_func``) passed to :class:`.Scene.play` to each individual
animation. The processed animations are then stored in the ``animations``
attribute of the scene (which the renderer later reads...).
- It adds all mobjects to which the animations that are played are
bound to the scene (provided the animation is not an mobject-introducing
animation -- for these, the addition to the scene happens later).
- In case the played animation is a :class:`.Wait` animation (this is the
case in a :meth:`.Scene.wait` call), the method checks whether a static
image should be rendered, or whether the render loop should be processed
as usual (see :meth:`.Scene.should_update_mobjects` for the exact conditions,
basically it checks whether there are any time-dependent updater functions
and so on).
- Finally, the method determines the total run time of the play call (which
at this point is computed as the maximum of the run times of the passed
animations). This is stored in the ``duration`` attribute of the scene.
After the animation data has been compiled by the scene, the renderer
continues to prepare for entering the render loop. It now checks the
skipping status which has been determined before. If the renderer can
skip this play call, it does so: it sets the current play call hash (which
we will get back to in a moment) to ``None`` and increases the time of the
renderer by the determined animation run time.
Otherwise, the renderer checks whether or not Manim's caching system should
be used. The idea of the caching system is simple: for every play call, a
hash value is computed, which is then stored and upon re-rendering the scene,
the hash is generated again and checked against the stored value. If it is the
same, the cached output is reused, otherwise it is fully rerendered again.
We will not go into details of the caching system here; if you would like
to learn more, the :func:`.get_hash_from_play_call` function in the
:mod:`.utils.hashing` module is essentially the entry point to the caching
mechanism.
In the event that the animation has to be rendered, the renderer gives its
:class:`.SceneFileWriter` the current animation index and asks it to start a
segment job. The writer creates a ``VideoSegmentEncoder`` from the resolved
profile and wraps it in a ``_PartialMovieEncodeJob``. The synchronous segment
encoder owns its container, video stream, sequential presentation timestamps,
and target cleanup. The job owns only the frame queue and worker thread. During
the render loop, concrete top-left-origin ``uint8`` RGBA arrays are added to the
queue and encoded by the worker. With the writing process in place, the renderer
then asks the scene to "begin" the animations.
By default, Manim finishes encoding each partial movie file before rendering the
next animation. If ``max_inflight_encoders`` is set to a value greater than 1,
encoding can instead overlap with rendering. The setting bounds the number of
partial movie encoders active at the same time, while ``encoder_queue_size``
bounds the number of pending frame buffers held by each encoder (the renderer
waits when the queue is full, so memory use does not grow with scene length).
First, it literally *begins* all of the animations by calling their
setup methods (:meth:`.Animation._setup_scene`, :meth:`.Animation.begin`).
In doing so, the mobjects that are newly introduced by an animation
(like via :class:`.Create` etc.) are added to the scene. Furthermore, the
animation suspends updater functions being called on its mobject, and
it sets its mobject to the state that corresponds to the first frame
of the animation.
After this has happened for all animations in the current ``play`` call,
the Cairo renderer determines which of the scene's mobjects can be
painted statically to the background, and which ones have to be
redrawn every frame. It does so by calling
:meth:`.Scene.get_moving_and_static_mobjects`, and the resulting
partition of mobjects is stored in the corresponding ``moving_mobjects``
and ``static_mobjects`` attributes.
.. NOTE::
The mechanism that determines static and moving mobjects is
specific for the Cairo renderer, the OpenGL renderer works differently.
Basically, moving mobjects are determined by checking whether they,
any of their children, or any of the mobjects "below" them (in the
sense of the order in which mobjects are processed in the scene)
either have an update function attached, or whether they appear
in one of the current animations. See the implementation of
:meth:`.Scene.get_moving_mobjects` for more details.
Up to this very point, we did not actually render any (partial)
image or movie files from the scene yet. This is, however, about to change.
Before we enter the render loop, let us briefly revisit our toy
example and discuss how the generic :meth:`.Scene.play` call
setup looks like there.
For the call that plays the :class:`.ReplacementTransform`, there
is no subcaption for the manager to schedule. The manager forwards the call,
and the renderer then asks the scene to compile the animation data: the passed
argument already is an animation (no additional preparations needed),
there is no need for processing any keyword arguments (as
we did not specify any additional ones to ``play``). The
mobject bound to the animation, ``orange_square``, is already
part of the scene (so again, no action taken). Finally, the run
time is extracted (3 seconds long) and stored in
``Scene.duration``. The renderer then checks whether it should
skip (it should not), then whether the animation is already
cached (it is not). The corresponding animation hash value is
determined and passed to the file writer. The writer resolves the segment target
from that key and starts its queued encoder, which waits for rendered frames.
The scene then ``begin``\ s the animation: for the
:class:`.ReplacementTransform` this means that the animation populates
all of its relevant animation attributes (i.e., compatible copies
of the starting and the target mobject so that it can safely interpolate
between the two).
The mechanism determining static and moving mobjects considers
all of the scenes mobjects (at this point only the
``orange_square``), and determines that the ``orange_square`` is
bound to an animation that is currently played. As a result,
the square is classified as a "moving mobject".
Time to render some frames.
The render loop (for real this time)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
As mentioned above, due to the mechanism that determines static and moving
mobjects in the scene, the renderer knows which mobjects it can paint
statically to the background of the scene. Practically, this means that
it partially renders a scene (to produce a background image), and then
when iterating through the time progression of the animation only the
"moving mobjects" are re-painted on top of the static background.
The renderer calls :meth:`.CairoRenderer.save_static_frame_data`, which
first checks whether there are currently any static mobjects, and if there
are, it updates the frame (only with the static mobjects; more about how
exactly this works in a moment) and then saves a NumPy array representing
the rendered frame in the ``static_image`` attribute. In our toy example,
there are no static mobjects, and so the ``static_image`` attribute is
simply set to ``None``.
Next, the renderer asks the scene whether the current animation is
a "frozen frame" animation, which would mean that the renderer actually
does not have to repaint the moving mobjects in every frame of the time
progression. It can then just take the latest static frame, and display it
throughout the animation.
.. NOTE::
An animation is considered a "frozen frame" animation if only a
static :class:`.Wait` animation is played. See the description
of :meth:`.Scene.compile_animation_data` above, or the
implementation of :meth:`.Scene.should_update_mobjects` for
more details.
If this is not the case (just as in our toy example), the renderer
then calls the :meth:`.Scene.play_internal` method, which is the
integral part of the render loop (in which the library steps through
the time progression of the animation and renders the corresponding
frames).
Within :meth:`.Scene.play_internal`, the following steps are performed:
- The scene determines the run time of the animations by calling
:meth:`.Scene.get_run_time`. This method basically takes the maximum
``run_time`` attribute of all of the animations passed to the
:meth:`.Scene.play` call.
- Then the *time progression* is constructed via the (internal)
:meth:`.Scene._get_animation_time_progression` method, which wraps
the actual :meth:`.Scene.get_time_progression` method. The time
progression is a ``tqdm`` `progress bar object <https://tqdm.github.io>`__
for an iterator over ``np.arange(0, run_time, 1 / config.frame_rate)``. In
other words, the time progression holds the time stamps (relative to the
current animations, so starting at 0 and ending at the total animation run time,
with the step size determined by the render frame rate) of the timeline where
a new animation frame should be rendered.
- Then the scene iterates over the time progression: for each time stamp ``t``,
:meth:`.Scene.update_to_time` is called, which ...
- ... first computes the time passed since the last update (which might be 0,
especially for the initial call) and references it as ``dt``,
- then (in the order in which the animations are passed to :meth:`.Scene.play`)
calls :meth:`.Animation.update_mobjects` to trigger all updater functions that
are attached to the respective animation except for the "main mobject" of
the animation (that is, for example, for :class:`.Transform` the unmodified
copies of start and target mobject -- see :meth:`.Animation.get_all_mobjects_to_update`
for more details),
- then the relative time progression with respect to the current animation
is computed (``alpha = t / animation.run_time``), which is then used to
update the state of the animation with a call to :meth:`.Animation.interpolate`.
- After all of the passed animations have been processed, the updater functions
of all mobjects in the scene, all meshes, and finally those attached to
the scene itself are run.
At this point, the internal (Python) state of all mobjects has been updated
to match the currently processed timestamp. If rendering should not be skipped,
then it is now time to *take a picture*!
.. NOTE::
The update of the internal state (iteration over the time progression) happens
*always* once :meth:`.Scene.play_internal` is entered. This ensures that even
if frames do not need to be rendered (because, e.g., the ``-n`` CLI flag has
been passed, something has been cached, or because we might be in a *Section*
with skipped rendering), updater functions still run correctly, and the state
of the first frame that *is* rendered is kept consistent.
To render an image, the scene calls the corresponding method of its renderer,
:meth:`.CairoRenderer.render` and passes just the list of *moving mobjects* (remember,
the *static mobjects* are assumed to have already been painted statically to
the background of the scene). All of the hard work then happens when the renderer
updates its current frame via a call to :meth:`.CairoRenderer.update_frame`:
First, the renderer prepares its :class:`.Camera` by checking whether the renderer
has a ``static_image`` different from ``None`` stored already. If so, it sets the
image as the *background image* of the camera via :meth:`.Camera.set_frame_to_background`,
and otherwise it just resets the camera via :meth:`.Camera.reset`. The camera is then
asked to capture the scene with a call to :meth:`.Camera.capture_mobjects`.
Things get a bit technical here, and at some point it is more efficient to
delve into the implementation -- but here is a summary of what happens once the
camera is asked to capture the scene:
- First, a flat list of mobjects is created (so submobjects get extracted from
their parents). This list is then processed in groups of the same type of
mobjects (e.g., a batch of vectorized mobjects, followed by a batch of image mobjects,
followed by more vectorized mobjects, etc. -- in many cases there will just be
one batch of vectorized mobjects).
- Depending on the type of the currently processed batch, the camera uses dedicated
*display functions* to convert the :class:`.Mobject` Python object to
a NumPy array stored in the camera's ``pixel_array`` attribute.
The most important example in that context is the display function for
vectorized mobjects, :meth:`.Camera.display_multiple_vectorized_mobjects`,
or the more particular (in case you did not add a background image to your
:class:`.VMobject`), :meth:`.Camera.display_multiple_non_background_colored_vmobjects`.
This method first gets the current Cairo context, and then, for every (vectorized)
mobject in the batch, calls :meth:`.Camera.display_vectorized`. There,
the actual background stroke, fill, and then stroke of the mobject is
drawn onto the context. See :meth:`.Camera.apply_stroke` and
:meth:`.Camera.set_cairo_context_color` for more details -- but it does not get
much deeper than that, in the latter method the actual Bézier curves
determined by the points of the mobject are drawn; this is where the low-level
interaction with Cairo happens.
After all batches have been processed, the camera has an image representation
of the Scene at the current time stamp in form of a NumPy array stored in its
``pixel_array`` attribute. The renderer passes a top-left-origin,
C-contiguous ``uint8`` RGBA array to its :class:`.SceneFileWriter`. OpenGL uses
the same array contract and performs GPU readback at this renderer boundary only
when file output needs a frame. This concludes one iteration of the render loop,
and once the time progression has been processed completely, a final bit
of cleanup is performed before the :meth:`.Scene.play_internal` call is completed.
A TL;DR for the render loop, in the context of our toy example, reads as follows:
- The scene finds that a 3 second long animation (the :class:`.ReplacementTransform`
changing the orange square to the blue circle) should be played. Given the requested
medium render quality, the frame rate is 30 frames per second, and so the time
progression with steps ``[0, 1/30, 2/30, ..., 89/30]`` is created.
- In the internal render loop, each of these time stamps is processed:
there are no updater functions, so effectively the scene updates the
state of the transformation animation to the desired time stamp (for example,
at time stamp ``t = 45/30``, the animation is completed to a rate of
``alpha = 0.5``).
- Then the scene asks the renderer to do its job. The renderer asks its camera
to capture the scene, the only mobject that needs to be processed at this point
is the main mobject attached to the transformation; the camera converts the
current state of the mobject to entries in a NumPy array. The renderer passes
this array to the file writer.
- At the end of the loop, 90 frames have been passed to the file writer.
Completing the render loop
^^^^^^^^^^^^^^^^^^^^^^^^^^
The last few steps in the :meth:`.Scene.play_internal` call are not too
exciting: for every animation, the corresponding :meth:`.Animation.finish`
and :meth:`.Animation.clean_up_from_scene` methods are called.
.. NOTE::
Note that as part of :meth:`.Animation.finish`, the :meth:`.Animation.interpolate`
method is called with an argument of 1.0 -- you might have noticed already that
the last frame of an animation can sometimes be a bit off or incomplete.
This is by current design! The last frame rendered in the render loop (and displayed
for a duration of ``1 / frame_rate`` seconds in the rendered video) corresponds to
the state of the animation ``1 / frame_rate`` seconds before it ends. To display
the final frame as well in the video, we would need to append another ``1 / frame_rate``
seconds to the video -- which would then mean that a 1 second rendered Manim video
would be slightly longer than 1 second. We decided against this at some point.
In the end, the time progression is closed (which completes the displayed progress bar)
in the terminal. With the closing of the time progression, the
:meth:`.Scene.play_internal` call is completed, and we return to the renderer,
which now orders the :class:`.SceneFileWriter` to close the partial movie stream.
This seals the corresponding encoding job so that it accepts no more frames. With
the default ``max_inflight_encoders = 1``, the file writer immediately waits for
the job and the partial movie file has been written when the call returns. With a
higher value, the job can remain in flight while the next animation is rendered;
the oldest jobs are joined as the configured limit is reached.
This pretty much concludes the walkthrough of a :class:`.Scene.play` call,
and actually there is not too much more to say for our toy example either: at
this point, the encoding job for a partial movie file representing the
:class:`.ReplacementTransform` has been sealed (and, with the default encoding
configuration, the file has been written). The initialization of the
:class:`.Dot` happens analogous to the initialization of ``blue_circle``, which
has been discussed above. The :meth:`.Mobject.add_updater` call literally just
attaches a function to the ``updaters`` attribute of the ``small_dot``. And the
remaining :meth:`.Scene.play` and :meth:`.Scene.wait` calls follow the exact same
procedure as discussed in the render loop section above; each such call produces
a corresponding partial movie file.
Once the :meth:`.Scene.construct` method has been fully processed, the manager
calls the scene's cleanup method :meth:`.Scene.tear_down`, followed by
:meth:`.Manager.post_construct`. The manager asks the renderer to finish the
scene, and the renderer in turn asks its scene file writer to wrap things up by
calling :meth:`.SceneFileWriter.finish`. The file writer first waits for all
remaining encoding jobs, then combines the completed partial movie files into the
final product and records its path. If rendering aborts during a play instead, the
manager asks the file writer to abort its encoding jobs; the incomplete current
partial movie file is removed so that it cannot be mistaken for a valid cached
result on a later render. Once output is finalized, the manager carries out any
post-render presentation request from the immutable session specification.
And there you go! This is a more or less detailed description of how Manim works
under the hood. While we did not discuss every single line of code in detail
in this walkthrough, it should still give you a fairly good idea of how the general
structural design of the library and at least the Cairo rendering flow in particular
looks like.