Skip to content

armgit / manim

Public Imported from GitHub
0

A community-maintained Python framework for creating mathematical animations.

Refactor: move video segment encoding to dedicated module and detach `SceneFileWriter` from renderer state (#4976)

Benjamin Hackl864704ac0ca7
diff --git a/docs/source/guides/configuration.rst b/docs/source/guides/configuration.rst
index ead4b3216..4b5f8279d 100644
--- a/docs/source/guides/configuration.rst
+++ b/docs/source/guides/configuration.rst
@@ -180,6 +180,25 @@ and serve the same purpose. Take, for example, the following config file.
Config files are parsed with the standard python library ``configparser``. In
particular, they will ignore any line that starts with a pound symbol ``#``.
+Video encoder profiles use dedicated sections because codec options are arbitrary
+key/value pairs:
+
+.. code-block:: ini
+
+ [video_encoder]
+ codec = libx264
+ pixel_format = yuv420p
+
+ [video_encoder.options]
+ crf = 18
+ preset = slow
+
+``codec`` and ``pixel_format`` default to ``auto``. Manim resolves them from the
+output format and whether alpha is required. The equivalent CLI options are
+``--video-codec``, ``--pixel-format``, and repeatable
+``--encoder-option KEY=VALUE``. If any ``--encoder-option`` is supplied, the CLI
+option map replaces the complete ``[video_encoder.options]`` map.
+
Now, executing the following command
.. code-block:: bash
diff --git a/docs/source/guides/deep_dive.rst b/docs/source/guides/deep_dive.rst
index d3891dc58..e315d5b0d 100644
--- a/docs/source/guides/deep_dive.rst
+++ b/docs/source/guides/deep_dive.rst
@@ -280,12 +280,26 @@ continues as follows:
scene_name=type(self).__name__,
requested_output_name=...,
)
- self.renderer.init_scene(self, self.session_spec)
+ self.file_writer_settings = _SceneFileWriterSettings(
+ plan=self.output_plan,
+ video_encoder=self.session_spec.video_encoder,
+ max_inflight_encoders=config.max_inflight_encoders,
+ encoder_queue_size=config.encoder_queue_size,
+ max_files_cached=config.max_files_cached,
+ assets_dir=...,
+ )
+ self.renderer.init_scene(
+ self,
+ self.session_spec,
+ self.file_writer_settings,
+ )
The session specification separates primary artifact intent (an ``OutputSpec``)
from presentation requests such as opening the completed artifact or displaying
-a live preview. It also records dry-run execution separately from artifact
-selection. A dry run requests semantic scene evaluation without rasterizing
+a live preview. For video output it also contains the resolved segment profile:
+container, codec, pixel format, dimensions, exact frame rate, and codec options.
+It records dry-run execution separately from artifact selection. A dry run
+requests semantic scene evaluation without rasterizing
frames or using media and cache resources. In contrast, ``format = none`` only
suppresses the primary artifact; an OpenGL live preview with automatic output
still rasterizes and displays frames without writing a file. Both requests have
@@ -304,12 +318,14 @@ and cache paths. Planning performs no file I/O and creates no directories. The
resolved format determines the artifact suffix; ``output_file`` supplies only a
name and cannot change the format.
-Inspecting the initialization methods of both renderers shows that they
-instantiate a :class:`.SceneFileWriter`. The writer must receive the already
-resolved ``OutputSpec`` and output plan; it does not reinterpret global
-configuration to decide what or where to write. Directories are created lazily
-when their owning operation first writes. The writer remains Manim's interface
-to ``libav`` for encoding media. The Cairo renderer (see the implementation `here
+The scene combines the output plan and segment profile with the encoder-pool,
+cache-maintenance, and sound-asset inputs in immutable
+``_SceneFileWriterSettings``. Both renderers instantiate a
+:class:`.SceneFileWriter` from these settings. The writer does not retain a
+renderer reference or read mutable global configuration. Directories are created
+lazily when their owning operation first writes. The writer remains Manim's
+interface to ``libav`` for media assembly. The Cairo renderer (see the
+implementation `here
<https://github.com/ManimCommunity/manim/blob/main/manim/renderer/cairo_renderer.py>`__)
does not require further renderer-specific initialization. OpenGL creates a
window only when the resolved presentation specification requests a live preview.
@@ -813,13 +829,15 @@ to learn more, the :func:`.get_hash_from_play_call` function in the
:mod:`.utils.hashing` module is essentially the entry point to the caching
mechanism.
-In the event that the animation has to be rendered, the renderer asks
-its :class:`.SceneFileWriter` to open a partial movie stream. The file writer
-uses ``libav`` to create a container and video stream, then wraps them in a
-``_PartialMovieEncodeJob``. Each encoding job owns its container, stream, frame
-queue, and worker thread. During the render loop, rendered raw frames are added
-to this queue and encoded by the worker. With the writing process in place, the
-renderer then asks the scene to "begin" the animations.
+In the event that the animation has to be rendered, the renderer gives its
+:class:`.SceneFileWriter` the current animation index and asks it to start a
+segment job. The writer creates a ``VideoSegmentEncoder`` from the resolved
+profile and wraps it in a ``_PartialMovieEncodeJob``. The synchronous segment
+encoder owns its container, video stream, sequential presentation timestamps,
+and target cleanup. The job owns only the frame queue and worker thread. During
+the render loop, concrete top-left-origin ``uint8`` RGBA arrays are added to the
+queue and encoded by the worker. With the writing process in place, the renderer
+then asks the scene to "begin" the animations.
By default, Manim finishes encoding each partial movie file before rendering the
next animation. If ``max_inflight_encoders`` is set to a value greater than 1,
@@ -872,9 +890,8 @@ time is extracted (3 seconds long) and stored in
``Scene.duration``. The renderer then checks whether it should
skip (it should not), then whether the animation is already
cached (it is not). The corresponding animation hash value is
-determined and passed to the file writer, which then also calls
-``libav`` to start the writing process which waits for rendered
-frames from the library.
+determined and passed to the file writer. The writer resolves the segment target
+from that key and starts its queued encoder, which waits for rendered frames.
The scene then ``begin``\ s the animation: for the
:class:`.ReplacementTransform` this means that the animation populates
@@ -1014,8 +1031,10 @@ camera is asked to capture the scene:
After all batches have been processed, the camera has an image representation
of the Scene at the current time stamp in form of a NumPy array stored in its
-``pixel_array`` attribute. The renderer then takes this array and passes it to
-its :class:`.SceneFileWriter`. This concludes one iteration of the render loop,
+``pixel_array`` attribute. The renderer passes a top-left-origin,
+C-contiguous ``uint8`` RGBA array to its :class:`.SceneFileWriter`. OpenGL uses
+the same array contract and performs GPU readback at this renderer boundary only
+when file output needs a frame. This concludes one iteration of the render loop,
and once the time progression has been processed completely, a final bit
of cleanup is performed before the :meth:`.Scene.play_internal` call is completed.
diff --git a/docs/source/tutorials/output_and_config.rst b/docs/source/tutorials/output_and_config.rst
index c04c91173..e5f20bde6 100644
--- a/docs/source/tutorials/output_and_config.rst
+++ b/docs/source/tutorials/output_and_config.rst
@@ -199,6 +199,21 @@ preview, but it counts as a separate execution request rather than modifying the
choice of output format. This allows Manim to function as if it were rendering a
normal scene, but without producing any artifact.
+Video output is assembled from silent cached segments. Manim selects their codec
+and pixel format automatically; ``--video-codec``, ``--pixel-format``, and
+repeatable ``--encoder-option KEY=VALUE`` provide explicit control when needed.
+These settings are part of segment cache identity, so changing one rerenders the
+affected segments. Audio is mixed into the final artifact separately.
+
+Cached segments for named scenes can be removed without starting a render:
+
+.. code-block:: bash
+
+ manim cache clear scene.py SceneName AnotherScene
+
+The command accepts path-affecting options such as ``--config-file``,
+``--media-dir``, ``--quality``, ``--resolution``, and ``--frame-rate``.
+
``-o`` / ``--output_file`` names the primary artifact for a single selected scene;
it does not select the format. Manim appends the resolved format suffix unless the
name already ends with it. For example, ``-o movie.mp4 --format=mp4`` produces
@@ -226,6 +241,7 @@ its own output video. The cuts between two sections can be set like this:
All the animations between two of these cuts get concatenated into a single output
video file.
+
Be aware that you need at least one animation in each section. For example this wouldn't create an output video:
.. code-block:: python
@@ -281,6 +297,10 @@ If you do this, the ``media`` folder will look like this:
├── ElaborateSceneWithSections_0003_fade-out.mp4
└── ElaborateSceneWithSections.json
+The ``partial_movie_file_list.txt`` file records the complete segment order used for
+main movie assembly and can help diagnose concatenation problems. Section assembly
+does not overwrite this diagnostic list.
+
As you can see each section receives their own output video in the ``sections`` directory.
Section names are normalized into safe filename components, while the original names
are retained in the JSON index. The JSON file contains some useful information for
diff --git a/manim/__main__.py b/manim/__main__.py
index 572053bf0..3b0247ab4 100644
--- a/manim/__main__.py
+++ b/manim/__main__.py
@@ -5,6 +5,7 @@ import cloup
from manim import __version__
from manim._config import cli_ctx_settings, console
+from manim.cli.cache.commands import cache
from manim.cli.cfg.group import cfg
from manim.cli.checkhealth.commands import checkhealth
from manim.cli.default_group import DefaultGroup
@@ -92,6 +93,7 @@ def main(ctx: click.Context) -> None:
main.add_command(checkhealth)
+main.add_command(cache)
main.add_command(cfg)
main.add_command(plugins)
main.add_command(init)
diff --git a/manim/_config/default.cfg b/manim/_config/default.cfg
index 757939ab9..c573c491a 100644
--- a/manim/_config/default.cfg
+++ b/manim/_config/default.cfg
@@ -126,10 +126,8 @@ frame_rate = 60
pixel_height = 1080
pixel_width = 1920
-# Use -1 to set max_files_cached to infinity.
+# Use -1 to keep an unlimited number of cached segment files.
max_files_cached = 100
-#Flush cache will delete all the cached partial-movie-files.
-flush_cache = False
disable_caching = False
# Disable the warning when there are too much submobjects to hash.
disable_caching_warning = False
@@ -196,9 +194,19 @@ log_height = -1
log_timestamps = True
repr_number = green
-[ffmpeg]
-# Uncomment the following line to manually set the loglevel for ffmpeg. See
-# ffmpeg manpage for accepted values
+[video_encoder]
+# Cached video-segment encoder. "auto" selects a codec and pixel format from
+# the requested output format and alpha requirement.
+codec = auto
+pixel_format = auto
+
+[video_encoder.options]
+# Arbitrary codec key/value options can be added here, for example:
+# crf = 18
+# preset = slow
+
+[media]
+# Logging level for media encoding, decoding, filtering, and muxing.
loglevel = ERROR
[jupyter]
diff --git a/manim/_config/output_plan.py b/manim/_config/output_plan.py
index 206eada0e..ce07bc8a9 100644
--- a/manim/_config/output_plan.py
+++ b/manim/_config/output_plan.py
@@ -50,8 +50,16 @@ class MediaLayoutSpec:
@dataclass(frozen=True, slots=True)
class OutputPlan:
- """Exact paths and dynamic child-name policy for one scene output."""
+ """Exact paths and dynamic child-name policy for one scene output.
+ The plan retains the immutable :class:`OutputSpec` from which it was
+ resolved. ``fallback_image`` is present only when automatic video output may
+ fall back to a last-frame PNG for a scene without play calls. For video output,
+ ``concat_manifest`` is the persistent diagnostic snapshot of the main scene's
+ segment order; assembly does not consume it.
+ """
+
+ output: OutputSpec
primary_artifact: Path | None
fallback_image: Path | None
image_sequence_dir: Path | None
@@ -121,6 +129,23 @@ def _required_dir(
return _absolute_lexical(path, working_directory)
+def resolve_segment_cache_directory(
+ config: _LayoutConfigSource,
+ *,
+ module_name: str,
+ scene_name: str,
+ working_directory: Path,
+) -> Path:
+ """Resolve the segment-cache directory for one scene."""
+ return _required_dir(
+ config,
+ "partial_movie_dir",
+ working_directory=working_directory,
+ module_name=module_name,
+ scene_name=scene_name,
+ )
+
+
def resolve_module_name(config: _LayoutConfigSource) -> str:
"""Resolve the source module name used by configured directory templates."""
if not config.input_file:
@@ -173,9 +198,8 @@ def resolve_media_layout(
module_name=module_name,
scene_name=scene_name,
)
- partial_movie_dir = _required_dir(
+ partial_movie_dir = resolve_segment_cache_directory(
config,
- "partial_movie_dir",
working_directory=working_directory,
module_name=module_name,
scene_name=scene_name,
@@ -241,6 +265,7 @@ def resolve_output_plan(
if output.format is OutputFormat.NONE:
return OutputPlan(
+ output=output,
primary_artifact=None,
fallback_image=None,
image_sequence_dir=None,
@@ -267,6 +292,7 @@ def resolve_output_plan(
if output.is_still:
assert versioned_png is not None
return OutputPlan(
+ output=output,
primary_artifact=versioned_png,
fallback_image=None,
image_sequence_dir=None,
@@ -284,6 +310,7 @@ def resolve_output_plan(
assert normalized_png is not None
sequence_dir = normalized_png.with_suffix("")
return OutputPlan(
+ output=output,
primary_artifact=sequence_dir,
fallback_image=None,
image_sequence_dir=sequence_dir,
@@ -320,6 +347,7 @@ def resolve_output_plan(
)
return OutputPlan(
+ output=output,
primary_artifact=primary_artifact,
fallback_image=versioned_png if output.fallback_to_still else None,
image_sequence_dir=None,
diff --git a/manim/_config/render_session.py b/manim/_config/render_session.py
index 480a1ad22..f615d7ef0 100644
--- a/manim/_config/render_session.py
+++ b/manim/_config/render_session.py
@@ -14,6 +14,7 @@ from typing import Protocol
from manim.renderer.protocol import RendererCapabilities
from .output import OutputFormat, OutputSpec
+from .video_encoder import VideoEncoderSpec, resolve_video_encoder
@dataclass(frozen=True, slots=True)
@@ -32,6 +33,7 @@ class RenderSessionSpec:
output: OutputSpec
presentation: PresentationSpec
dry_run: bool
+ video_encoder: VideoEncoderSpec | None
class _SessionConfigSource(Protocol):
@@ -43,6 +45,12 @@ class _SessionConfigSource(Protocol):
show_in_file_browser: bool
enable_gui: bool
dry_run: bool
+ pixel_width: int
+ pixel_height: int
+ frame_rate: float
+ video_codec: str
+ pixel_format: str
+ video_encoder_options: dict[str, str]
def resolve_render_session(
@@ -101,8 +109,18 @@ def resolve_render_session(
if presentation.show_in_file_browser and not output.enabled:
raise ValueError("--show_in_file_browser requires a media artifact.")
+ video_encoder = resolve_video_encoder(
+ output,
+ width=config.pixel_width,
+ height=config.pixel_height,
+ frame_rate=config.frame_rate,
+ codec=config.video_codec,
+ pixel_format=config.pixel_format,
+ options=config.video_encoder_options,
+ )
return RenderSessionSpec(
output=output,
presentation=presentation,
dry_run=dry_run,
+ video_encoder=video_encoder,
)
diff --git a/manim/_config/utils.py b/manim/_config/utils.py
index 2f9eeb8c8..9176e41b1 100644
--- a/manim/_config/utils.py
+++ b/manim/_config/utils.py
@@ -268,9 +268,7 @@ class ManimConfig(MutableMapping):
"dry_run",
"encoder_queue_size",
"enable_wireframe",
- "ffmpeg_loglevel",
"format",
- "flush_cache",
"frame_height",
"frame_rate",
"frame_width",
@@ -280,6 +278,7 @@ class ManimConfig(MutableMapping):
"images_dir",
"input_file",
"media_embed",
+ "media_loglevel",
"media_width",
"log_dir",
"log_to_file",
@@ -289,6 +288,7 @@ class ManimConfig(MutableMapping):
"notify_outdated_version",
"output_file",
"partial_movie_dir",
+ "pixel_format",
"pixel_height",
"pixel_width",
"plugins",
@@ -311,7 +311,9 @@ class ManimConfig(MutableMapping):
"use_projection_fill_shaders",
"use_projection_stroke_shaders",
"verbosity",
+ "video_codec",
"video_dir",
+ "video_encoder_options",
"sections_dir",
"fullscreen",
"window_position",
@@ -576,6 +578,19 @@ class ManimConfig(MutableMapping):
self._parser = parser
self.format = parser["CLI"].get("format", fallback="auto", raw=True)
+ self.video_codec = parser["video_encoder"].get(
+ "codec",
+ fallback="auto",
+ raw=True,
+ )
+ self.pixel_format = parser["video_encoder"].get(
+ "pixel_format",
+ fallback="auto",
+ raw=True,
+ )
+ self.video_encoder_options = dict(
+ parser.items("video_encoder.options", raw=True),
+ )
# boolean keys
for key in [
@@ -588,7 +603,6 @@ class ManimConfig(MutableMapping):
"log_to_file",
"disable_caching",
"disable_caching_warning",
- "flush_cache",
"enable_gui",
"fullscreen",
"use_projection_fill_shaders",
@@ -682,9 +696,9 @@ class ManimConfig(MutableMapping):
if progress_bar:
self.progress_bar = progress_bar
- ffmpeg_loglevel = parser["ffmpeg"].get("loglevel")
- if ffmpeg_loglevel:
- self.ffmpeg_loglevel = ffmpeg_loglevel
+ media_loglevel = parser["media"].get("loglevel")
+ if media_loglevel:
+ self.media_loglevel = media_loglevel
try:
media_embed = parser["jupyter"].getboolean("media_embed")
@@ -759,7 +773,6 @@ class ManimConfig(MutableMapping):
"write_all",
"disable_caching",
"format",
- "flush_cache",
"progress_bar",
"transparent",
"scene_names",
@@ -778,6 +791,9 @@ class ManimConfig(MutableMapping):
"seed",
"max_inflight_encoders",
"encoder_queue_size",
+ "video_codec",
+ "pixel_format",
+ "video_encoder_options",
]:
if hasattr(args, key):
attr = getattr(args, key)
@@ -1021,18 +1037,54 @@ class ManimConfig(MutableMapping):
)
@property
- def ffmpeg_loglevel(self) -> str:
- """Verbosity level of ffmpeg (no flag)."""
- return self._d["ffmpeg_loglevel"]
+ def video_codec(self) -> str:
+ """Video encoder used for cached segments."""
+ return self._d["video_codec"]
+
+ @video_codec.setter
+ def video_codec(self, value: str) -> None:
+ if not isinstance(value, str) or not value:
+ raise ValueError("video_codec must be a non-empty string")
+ self._d["video_codec"] = value
+
+ @property
+ def pixel_format(self) -> str:
+ """Pixel format used for cached video segments."""
+ return self._d["pixel_format"]
+
+ @pixel_format.setter
+ def pixel_format(self, value: str) -> None:
+ if not isinstance(value, str) or not value:
+ raise ValueError("pixel_format must be a non-empty string")
+ self._d["pixel_format"] = value
+
+ @property
+ def video_encoder_options(self) -> dict[str, str]:
+ """Codec options used for cached video segments."""
+ return dict(self._d["video_encoder_options"])
+
+ @video_encoder_options.setter
+ def video_encoder_options(self, value: Mapping[str, str]) -> None:
+ if not isinstance(value, Mapping) or any(
+ not isinstance(key, str) or not isinstance(option, str)
+ for key, option in value.items()
+ ):
+ raise TypeError("video_encoder_options must map strings to strings")
+ self._d["video_encoder_options"] = dict(value)
- @ffmpeg_loglevel.setter
- def ffmpeg_loglevel(self, val: str) -> None:
+ @property
+ def media_loglevel(self) -> str:
+ """Logging level for media operations."""
+ return self._d["media_loglevel"]
+
+ @media_loglevel.setter
+ def media_loglevel(self, value: str) -> None:
self._set_from_list(
- "ffmpeg_loglevel",
- val,
+ "media_loglevel",
+ value,
["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
)
- logging.getLogger("libav").setLevel(self.ffmpeg_loglevel)
+ logging.getLogger("libav").setLevel(self.media_loglevel)
@property
def media_embed(self) -> bool | None:
@@ -1181,7 +1233,12 @@ class ManimConfig(MutableMapping):
@max_files_cached.setter
def max_files_cached(self, value: int) -> None:
- self._set_pos_number("max_files_cached", value, True)
+ if isinstance(value, int) and value >= -1:
+ self._d["max_files_cached"] = value
+ else:
+ raise ValueError(
+ "max_files_cached must be a non-negative integer or -1 for unlimited",
+ )
@property
def max_inflight_encoders(self) -> int:
@@ -1223,15 +1280,6 @@ class ManimConfig(MutableMapping):
def window_monitor(self, value: int) -> None:
self._set_pos_number("window_monitor", value, True)
- @property
- def flush_cache(self) -> bool:
- """Whether to delete all the cached partial movie files."""
- return self._d["flush_cache"]
-
- @flush_cache.setter
- def flush_cache(self, value: bool) -> None:
- self._set_boolean("flush_cache", value)
-
@property
def disable_caching(self) -> bool:
"""Whether to use scene caching."""
diff --git a/manim/_config/video_encoder.py b/manim/_config/video_encoder.py
new file mode 100644
index 000000000..846fb43fb
--- /dev/null
+++ b/manim/_config/video_encoder.py
@@ -0,0 +1,235 @@
+"""Resolved settings for cached video-segment encoding."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import math
+from collections.abc import Mapping
+from dataclasses import dataclass
+from fractions import Fraction
+
+import av
+
+from .output import OutputSpec
+
+__all__ = [
+ "VideoEncoderSpec",
+ "resolve_video_encoder",
+ "to_av_frame_rate",
+ "video_encoder_fingerprint",
+]
+
+_CONFLICTING_OPTION_KEYS = frozenset(
+ {
+ "codec",
+ "codec_name",
+ "framerate",
+ "height",
+ "pix_fmt",
+ "pixel_format",
+ "rate",
+ "video_size",
+ "width",
+ },
+)
+_DEFAULT_OPTIONS = {
+ "libx264": {"crf": "23"},
+ "libvpx-vp9": {"crf": "23"},
+ "qtrle": {},
+}
+
+
+@dataclass(frozen=True, slots=True)
+class VideoEncoderSpec:
+ """Complete byte-affecting settings for one cached video segment."""
+
+ container_format: str
+ codec: str
+ pixel_format: str
+ width: int
+ height: int
+ frame_rate: Fraction
+ options: tuple[tuple[str, str], ...]
+
+
+def video_encoder_fingerprint(spec: VideoEncoderSpec | None) -> str:
+ """Return the stable cache-identity token for resolved encoder settings."""
+ if spec is None:
+ return "none"
+ payload = {
+ "schema": "manim-video-segment-v1",
+ "container_format": spec.container_format,
+ "codec": spec.codec,
+ "pixel_format": spec.pixel_format,
+ "width": spec.width,
+ "height": spec.height,
+ "frame_rate": {
+ "numerator": spec.frame_rate.numerator,
+ "denominator": spec.frame_rate.denominator,
+ },
+ "options": [
+ {"name": name, "value": value} for name, value in sorted(spec.options)
+ ],
+ }
+ serialized = json.dumps(
+ payload,
+ ensure_ascii=False,
+ separators=(",", ":"),
+ sort_keys=True,
+ )
+ return hashlib.sha256(serialized.encode("utf-8")).hexdigest()
+
+
+def to_av_frame_rate(frame_rate: int | float | Fraction) -> Fraction:
+ """Return a positive exact rate suitable for a PyAV video stream."""
+ if isinstance(frame_rate, bool):
+ raise ValueError("frame rate must be a positive finite number")
+ if isinstance(frame_rate, Fraction):
+ resolved = frame_rate
+ elif isinstance(frame_rate, int):
+ resolved = Fraction(frame_rate, 1)
+ elif isinstance(frame_rate, float) and math.isfinite(frame_rate):
+ if abs(frame_rate - round(frame_rate)) < 1e-4:
+ resolved = Fraction(round(frame_rate), 1)
+ else:
+ ntsc_rate = Fraction(round(frame_rate * 1001 / 1000) * 1000, 1001)
+ resolved = (
+ ntsc_rate
+ if abs(frame_rate - float(ntsc_rate)) < 0.02
+ else Fraction(str(frame_rate))
+ )
+ else:
+ raise ValueError("frame rate must be a positive finite number")
+
+ if resolved <= 0:
+ raise ValueError("frame rate must be positive")
+ return resolved
+
+
+def _default_profile(output: OutputSpec) -> tuple[str, str, str]:
+ extension = output.segment_extension
+ if extension is None:
+ raise ValueError("Video output requires a segment container.")
+
+ container_format = extension.removeprefix(".")
+ if extension == ".webm":
+ return (
+ container_format,
+ "libvpx-vp9",
+ "yuva420p" if output.transparent else "yuv420p",
+ )
+ if output.transparent:
+ return container_format, "qtrle", "argb"
+ return container_format, "libx264", "yuv420p"
+
+
+def _validate_geometry(width: int, height: int) -> None:
+ if (
+ isinstance(width, bool)
+ or isinstance(height, bool)
+ or not isinstance(width, int)
+ or not isinstance(height, int)
+ or width <= 0
+ or height <= 0
+ ):
+ raise ValueError("Video dimensions must be positive integers.")
+
+
+def _validate_profile(
+ *,
+ container_format: str,
+ codec_name: str,
+ pixel_format: str,
+ transparent: bool,
+) -> None:
+ try:
+ container = av.format.ContainerFormat(container_format)
+ except ValueError as error:
+ raise ValueError(f"Unknown output container: {container_format}") from error
+ if not container.is_output:
+ raise ValueError(f"Container does not support output: {container_format}")
+
+ try:
+ codec = av.codec.Codec(codec_name, "w")
+ except ValueError as error:
+ raise ValueError(f"Unknown video encoder: {codec_name}") from error
+ if codec.type != "video":
+ raise ValueError(f"Encoder is not a video encoder: {codec_name}")
+
+ try:
+ video_format = av.VideoFormat(pixel_format)
+ except ValueError as error:
+ raise ValueError(f"Unknown pixel format: {pixel_format}") from error
+
+ supported_formats = codec.video_formats
+ if supported_formats is not None and pixel_format not in {
+ supported.name for supported in supported_formats
+ }:
+ raise ValueError(
+ f"Pixel format {pixel_format} is not supported by encoder {codec_name}.",
+ )
+ if transparent and not any(
+ component.is_alpha for component in video_format.components
+ ):
+ raise ValueError(
+ f"Transparent output requires an alpha-bearing pixel format; "
+ f"got {pixel_format}.",
+ )
+
+
+def resolve_video_encoder(
+ output: OutputSpec,
+ *,
+ width: int,
+ height: int,
+ frame_rate: int | float | Fraction,
+ codec: str = "auto",
+ pixel_format: str = "auto",
+ options: Mapping[str, str] | None = None,
+) -> VideoEncoderSpec | None:
+ """Resolve and validate cached-segment encoder settings."""
+ if not output.is_video:
+ return None
+
+ _validate_geometry(width, height)
+ resolved_rate = to_av_frame_rate(frame_rate)
+ container_format, default_codec, default_pixel_format = _default_profile(output)
+ resolved_codec = default_codec if codec == "auto" else codec
+ resolved_pixel_format = (
+ default_pixel_format if pixel_format == "auto" else pixel_format
+ )
+
+ supplied_options = {} if options is None else dict(options)
+ if any(
+ not isinstance(key, str) or not isinstance(value, str)
+ for key, value in supplied_options.items()
+ ):
+ raise TypeError("Video encoder option keys and values must be strings.")
+ conflicting = sorted(_CONFLICTING_OPTION_KEYS.intersection(supplied_options))
+ if conflicting:
+ raise ValueError(
+ "Video encoder options conflict with explicit stream settings: "
+ + ", ".join(conflicting),
+ )
+
+ resolved_options = (
+ dict(_DEFAULT_OPTIONS[default_codec]) if resolved_codec == default_codec else {}
+ )
+ resolved_options.update(supplied_options)
+
+ _validate_profile(
+ container_format=container_format,
+ codec_name=resolved_codec,
+ pixel_format=resolved_pixel_format,
+ transparent=output.transparent,
+ )
+ return VideoEncoderSpec(
+ container_format=container_format,
+ codec=resolved_codec,
+ pixel_format=resolved_pixel_format,
+ width=width,
+ height=height,
+ frame_rate=resolved_rate,
+ options=tuple(sorted(resolved_options.items())),
+ )
diff --git a/manim/cli/cache/__init__.py b/manim/cli/cache/__init__.py
new file mode 100644
index 000000000..7e8f15d8b
--- /dev/null
+++ b/manim/cli/cache/__init__.py
@@ -0,0 +1 @@
+"""Commands for maintaining Manim's segment cache."""
diff --git a/manim/cli/cache/commands.py b/manim/cli/cache/commands.py
new file mode 100644
index 000000000..f345ea686
--- /dev/null
+++ b/manim/cli/cache/commands.py
@@ -0,0 +1,122 @@
+"""Commands for maintaining cached video segments."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import cloup
+
+from manim._config import config, console
+from manim._config.output_plan import (
+ resolve_module_name,
+ resolve_segment_cache_directory,
+)
+from manim._config.utils import _determine_quality
+from manim.cli.render.render_options import validate_resolution
+from manim.constants import EPILOG, QUALITIES
+from manim.utils.caching import clear_segment_cache
+
+__all__ = ["cache"]
+
+
+ context_settings=None,
+ no_args_is_help=True,
+ epilog=EPILOG,
+)
+def cache() -> None:
+ """Maintain cached video segments."""
+
+
+ context_settings=None,
+ no_args_is_help=True,
+ epilog=EPILOG,
+)
+ "file",
+ type=cloup.Path(path_type=Path, exists=True, dir_okay=False),
+ required=True,
+)
[email protected]("scene_names", required=True, nargs=-1)
+ "-c",
+ "--config-file",
+ type=cloup.Path(path_type=Path, exists=True, dir_okay=False),
+ default=None,
+ help="Use the specified configuration file.",
+)
+ "--media-dir",
+ type=cloup.Path(path_type=Path),
+ default=None,
+ help="Override the directory containing rendered media and caches.",
+)
+ "-q",
+ "--quality",
+ type=cloup.Choice(
+ list(reversed([q["flag"] for q in QUALITIES.values() if q["flag"]])),
+ case_sensitive=False,
+ ),
+ default=None,
+ help="Resolve the cache directory for this render quality.",
+)
+ "-r",
+ "--resolution",
+ callback=validate_resolution,
+ default=None,
+ help='Resolve the cache directory for resolution "W,H".',
+)
+ "--fps",
+ "--frame-rate",
+ "frame_rate",
+ type=float,
+ default=None,
+ help="Resolve the cache directory for this frame rate.",
+)
+def clear(
+ *,
+ file: Path,
+ scene_names: tuple[str, ...],
+ config_file: Path | None,
+ media_dir: Path | None,
+ quality: str | None,
+ resolution: tuple[int, int] | None,
+ frame_rate: float | None,
+) -> None:
+ """Delete cached segments for SCENE(S) from FILE."""
+ command_config = config.copy()
+ selected_config_file = file if file.suffix == ".cfg" else config_file
+ if selected_config_file is not None:
+ command_config.digest_file(selected_config_file)
+
+ if not command_config.input_file:
+ if file.suffix == ".cfg":
+ raise ValueError("A configuration file must define input_file.")
+ command_config.input_file = file.absolute()
+
+ if media_dir is not None:
+ command_config.media_dir = media_dir
+ if quality is not None:
+ command_config.quality = _determine_quality(quality)
+ if resolution is not None:
+ command_config.frame_size = resolution
+ if frame_rate is not None:
+ command_config.frame_rate = frame_rate
+
+ module_name = resolve_module_name(command_config)
+ working_directory = Path.cwd()
+ for scene_name in scene_names:
+ directory = resolve_segment_cache_directory(
+ command_config,
+ module_name=module_name,
+ scene_name=scene_name,
+ working_directory=working_directory,
+ )
+ removed = clear_segment_cache(directory)
+ console.print(
+ f"Removed {removed} cached segment(s) for {scene_name} from {directory}.",
+ )
diff --git a/manim/cli/render/global_options.py b/manim/cli/render/global_options.py
index fec179238..597deb957 100644
--- a/manim/cli/render/global_options.py
+++ b/manim/cli/render/global_options.py
@@ -67,12 +67,6 @@ global_options = option_group(
default=None,
help="Disable the use of the cache (still generates cache files).",
),
- option(
- "--flush_cache",
- is_flag=True,
- help="Remove cached partial movie files.",
- default=None,
- ),
option("--tex_template", help="Specify a custom TeX template file.", default=None),
option(
"-v",
@@ -81,7 +75,7 @@ global_options = option_group(
["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
case_sensitive=False,
),
- help="Verbosity of CLI output. Changes ffmpeg log level unless 5+.",
+ help="Verbosity of CLI output. Changes media log level unless 5+.",
default=None,
),
option(
diff --git a/manim/cli/render/render_options.py b/manim/cli/render/render_options.py
index faca82987..a48d5038b 100644
--- a/manim/cli/render/render_options.py
+++ b/manim/cli/render/render_options.py
@@ -5,6 +5,7 @@ import re
import sys
from typing import TYPE_CHECKING
+from click import BadParameter
from cloup import Choice, IntRange, option, option_group
from manim.constants import QUALITIES, RendererType
@@ -63,6 +64,35 @@ def validate_scene_range(
return start, end
+def validate_encoder_options(
+ ctx: Context,
+ param: Option,
+ value: tuple[str, ...],
+) -> dict[str, str] | None:
+ """Parse repeatable ``KEY=VALUE`` encoder options."""
+ if not value:
+ return None
+
+ options: dict[str, str] = {}
+ for entry in value:
+ key, separator, option_value = entry.partition("=")
+ key = key.strip()
+ if not separator or not key or not option_value:
+ raise BadParameter(
+ "encoder options must use KEY=VALUE with nonempty key and value",
+ ctx=ctx,
+ param=param,
+ )
+ if key in options:
+ raise BadParameter(
+ f"encoder option {key!r} was supplied more than once",
+ ctx=ctx,
+ param=param,
+ )
+ options[key] = option_value
+ return options
+
+
def validate_resolution(
ctx: Context, param: Option, value: str | None
) -> tuple[int, int] | None:
@@ -180,6 +210,27 @@ render_options = option_group(
default=None,
help="Render at this frame rate.",
),
+ option(
+ "--video-codec",
+ "video_codec",
+ default=None,
+ help="Video encoder used for cached segments (default: auto).",
+ ),
+ option(
+ "--pixel-format",
+ "pixel_format",
+ default=None,
+ help="Pixel format used for cached segments (default: auto).",
+ ),
+ option(
+ "--encoder-option",
+ "video_encoder_options",
+ multiple=True,
+ default=None,
+ callback=validate_encoder_options,
+ metavar="KEY=VALUE",
+ help="Set a codec option; repeat to set multiple options.",
+ ),
option(
"--max-inflight-encoders",
type=IntRange(min=1),
diff --git a/manim/renderer/cairo_renderer.py b/manim/renderer/cairo_renderer.py
index 768b27b82..755ecb068 100644
--- a/manim/renderer/cairo_renderer.py
+++ b/manim/renderer/cairo_renderer.py
@@ -6,6 +6,7 @@ from typing import TYPE_CHECKING, Any
from manim.utils.hashing import get_hash_from_play_call
from .. import config, logger
+from .._config.video_encoder import video_encoder_fingerprint
from ..camera.camera import Camera
from ..mobject.mobject import Mobject, _AnimationBuilder
from ..scene.scene_file_writer import SceneFileWriter
@@ -17,6 +18,7 @@ if TYPE_CHECKING:
from manim._config.render_session import RenderSessionSpec
from manim.animation.animation import Animation
from manim.scene.scene import Scene
+ from manim.scene.scene_file_writer import _SceneFileWriterSettings
from ..typing import PixelArray
@@ -57,13 +59,13 @@ class CairoRenderer:
self.time = 0.0
self.static_image: PixelArray | None = None
- def init_scene(self, scene: Scene, session_spec: RenderSessionSpec) -> None:
- self.file_writer: Any = self._file_writer_class(
- self,
- scene.__class__.__name__,
- output_spec=session_spec.output,
- output_plan=scene.output_plan,
- )
+ def init_scene(
+ self,
+ scene: Scene,
+ session_spec: RenderSessionSpec,
+ file_writer_settings: _SceneFileWriterSettings,
+ ) -> None:
+ self.file_writer: Any = self._file_writer_class(file_writer_settings)
def play(
self,
@@ -93,6 +95,11 @@ class CairoRenderer:
self.camera,
scene.animations,
scene.mobjects,
+ backend="cairo",
+ encoder_fingerprint=video_encoder_fingerprint(
+ scene.session_spec.video_encoder,
+ ),
+ renderer_state=(),
)
if self.file_writer.is_already_cached(hash_current_animation):
logger.info(
@@ -109,7 +116,10 @@ class CairoRenderer:
{"h": str(self.animations_hashes[:5])},
)
- self.file_writer.begin_animation(not self.skip_animations)
+ self.file_writer.begin_animation(
+ not self.skip_animations,
+ animation_index=self.num_plays,
+ )
scene.begin_animations()
# Save a static image, to avoid rendering non moving objects.
@@ -198,7 +208,7 @@ class CairoRenderer:
if self.skip_animations:
return
self.time += num_frames * dt
- self.file_writer.write_frame(frame, num_frames=num_frames)
+ self.file_writer.write_frame(frame, repeat=num_frames)
def freeze_current_frame(self, duration: float) -> None:
"""Adds a static frame to the movie for a given duration. The static frame is the current frame.
@@ -286,4 +296,4 @@ class CairoRenderer:
if self.num_plays:
self.static_image = None
self.update_frame(scene)
- self.file_writer.save_image(self.camera.get_image())
+ self.file_writer.save_image(self.get_frame())
diff --git a/manim/renderer/opengl_renderer.py b/manim/renderer/opengl_renderer.py
index d4ee03bb1..cbaee2bc9 100644
--- a/manim/renderer/opengl_renderer.py
+++ b/manim/renderer/opengl_renderer.py
@@ -51,6 +51,7 @@ if TYPE_CHECKING:
from manim.animation.animation import Animation
from manim.mobject.mobject import Mobject, _AnimationBuilder
from manim.scene.scene import Scene
+ from manim.scene.scene_file_writer import _SceneFileWriterSettings
from manim.typing import (
FloatRGBA,
PathFuncType,
@@ -520,7 +521,12 @@ class OpenGLRenderer:
self.path_to_texture_id: dict[str, int] = {}
self.background_color = config["background_color"]
- def init_scene(self, scene: Scene, session_spec: RenderSessionSpec) -> None:
+ def init_scene(
+ self,
+ scene: Scene,
+ session_spec: RenderSessionSpec,
+ file_writer_settings: _SceneFileWriterSettings,
+ ) -> None:
"""
Initializes the OpenGL rendering context and related resources
for the given scene.
@@ -538,10 +544,7 @@ class OpenGLRenderer:
"""
self.partial_movie_files: list[str | None] = []
self.file_writer: SceneFileWriter = self._file_writer_class(
- self,
- scene.__class__.__name__,
- output_spec=session_spec.output,
- output_plan=scene.output_plan,
+ file_writer_settings,
)
self.scene = scene
@@ -832,16 +835,23 @@ class OpenGLRenderer:
"""
# TODO: Handle data locking / unlocking.
self.animation_start_time = time.time()
- self.file_writer.begin_animation(not self.skip_animations)
+ self.file_writer.begin_animation(
+ not self.skip_animations,
+ animation_index=self.num_plays,
+ )
scene.compile_animation_data(*animations, **kwargs)
scene.begin_animations()
if scene.is_current_animation_frozen_frame():
self.update_frame(scene)
- if not self.skip_animations:
+ output = self.file_writer.output_spec
+ if not self.skip_animations and (
+ output.is_video or output.is_image_sequence
+ ):
self.file_writer.write_frame(
- self, num_frames=int(config.frame_rate * scene.duration)
+ self.get_frame(),
+ repeat=int(config.frame_rate * scene.duration),
)
if self.window is not None:
@@ -901,7 +911,9 @@ class OpenGLRenderer:
if self.skip_animations:
return
- self.file_writer.write_frame(self)
+ output = self.file_writer.output_spec
+ if output.is_video or output.is_image_sequence:
+ self.file_writer.write_frame(self.get_frame())
if self.window is not None:
self.window.swap_buffers()
@@ -967,7 +979,7 @@ class OpenGLRenderer:
if self.should_save_last_frame():
if self.num_plays > 0:
self.update_frame(scene)
- self.file_writer.save_image(self.get_image())
+ self.file_writer.save_image(self.get_frame())
def should_save_last_frame(self) -> bool:
"""
@@ -1127,8 +1139,7 @@ class OpenGLRenderer:
result_dimensions = (pixel_shape[1], pixel_shape[0], 4)
np_buf = np.frombuffer(raw, dtype="uint8").reshape(result_dimensions)
- np_buf = np.flipud(np_buf)
- return np_buf
+ return np.flipud(np_buf).copy()
# Returns offset from the bottom left corner in pixels.
# top_left flag should be set to True when using a GUI framework
diff --git a/manim/scene/scene.py b/manim/scene/scene.py
index 93f7e09e2..29f69d5c5 100644
--- a/manim/scene/scene.py
+++ b/manim/scene/scene.py
@@ -59,6 +59,7 @@ from ..manager import Manager
from ..renderer.cairo_renderer import CairoRenderer
from ..renderer.opengl_renderer import OpenGLCamera, OpenGLMobject, OpenGLRenderer
from ..renderer.shader import Object3D
+from ..scene.scene_file_writer import _SceneFileWriterSettings
from ..utils import opengl, space_ops
from ..utils.exceptions import RerunSceneException
from ..utils.family import extract_mobject_family_members
@@ -229,12 +230,13 @@ class Scene:
)
scene_name = type(self).__name__
module_name = resolve_module_name(config)
+ working_directory = Path.cwd()
media_layout = resolve_media_layout(
config,
self.session_spec.output,
module_name=module_name,
scene_name=scene_name,
- working_directory=Path.cwd(),
+ working_directory=working_directory,
)
self.output_plan = resolve_output_plan(
media_layout,
@@ -242,6 +244,19 @@ class Scene:
scene_name=scene_name,
requested_output_name=resolve_requested_output_name(config),
)
+ assets_dir = config.get_dir("assets_dir")
+ if assets_dir is None:
+ assets_dir = working_directory
+ elif not assets_dir.is_absolute():
+ assets_dir = working_directory / assets_dir
+ self.file_writer_settings = _SceneFileWriterSettings(
+ plan=self.output_plan,
+ video_encoder=self.session_spec.video_encoder,
+ max_inflight_encoders=config.max_inflight_encoders,
+ encoder_queue_size=config.encoder_queue_size,
+ max_files_cached=config.max_files_cached,
+ assets_dir=assets_dir.absolute(),
+ )
self._log_file_path = resolve_file_log_path(
media_layout,
module_name=module_name,
@@ -250,7 +265,11 @@ class Scene:
if self._log_file_path is not None:
self._log_file_path.parent.mkdir(parents=True, exist_ok=True)
set_file_logger(self._log_file_path)
- self.renderer.init_scene(self, self.session_spec)
+ self.renderer.init_scene(
+ self,
+ self.session_spec,
+ self.file_writer_settings,
+ )
self.mobjects: list[Mobject] = []
# TODO, remove need for foreground mobjects
diff --git a/manim/scene/scene_file_writer.py b/manim/scene/scene_file_writer.py
index 3a71e5d7d..fb55c9620 100644
--- a/manim/scene/scene_file_writer.py
+++ b/manim/scene/scene_file_writer.py
@@ -1,4 +1,4 @@
-"""The interface between scenes and ffmpeg."""
+"""Scene output coordination and media-artifact assembly."""
from __future__ import annotations
@@ -8,7 +8,8 @@ import json
import shutil
import warnings
from contextlib import suppress
-from fractions import Fraction
+from dataclasses import dataclass
+from io import BytesIO
from pathlib import Path
from queue import Queue
from tempfile import NamedTemporaryFile, _TemporaryFileWrapper
@@ -33,38 +34,17 @@ with warnings.catch_warnings():
from manim import __version__
-from .. import config, logger
-from .._config.output import OutputSpec
+from .. import logger
from .._config.output_plan import OutputPlan
-from ..constants import RendererType
+from .._config.video_encoder import VideoEncoderSpec
+from ..utils.caching import prune_segment_cache
from ..utils.file_ops import modify_atime
from ..utils.sounds import get_full_sound_file_path
from .section import DefaultSectionType, Section
+from .video_segment_encoder import VideoSegmentEncoder
if TYPE_CHECKING:
- from av.container.output import OutputContainer
- from av.stream import Stream
-
- from manim.renderer.cairo_renderer import CairoRenderer
- from manim.renderer.opengl_renderer import OpenGLRenderer
- from manim.typing import PixelArray, StrPath
-
-
-def to_av_frame_rate(fps: float) -> Fraction:
- epsilon1 = 1e-4
- epsilon2 = 0.02
-
- if isinstance(fps, int):
- (num, denom) = (fps, 1)
- elif abs(fps - round(fps)) < epsilon1:
- (num, denom) = (round(fps), 1)
- else:
- denom = 1001
- num = round(fps * denom / 1000) * 1000
- if abs(fps - num / denom) >= epsilon2:
- raise ValueError("invalid frame rate")
-
- return Fraction(num, denom)
+ from manim.typing import RGBAPixelArray, StrPath
def convert_audio(
@@ -85,28 +65,28 @@ def convert_audio(
class _PartialMovieEncodeJob:
- """Encode and write the frames for one partial movie file."""
+ """Run one segment encoder on a dedicated worker thread."""
def __init__(
self,
- path: StrPath,
+ *,
animation_index: int,
- container: OutputContainer,
- stream: Stream,
+ encoder: VideoSegmentEncoder,
frame_queue_size: int,
) -> None:
- self.path = path
+ self.path = encoder.target
self.animation_index = animation_index
- self.container = container
- self.stream = stream
+ self.encoder = encoder
# A size of 0 preserves the unbounded queue used by serial encoding.
# Parallel encoding uses a bounded queue; at the default capacity, eight
# 1080p RGBA frames occupy about 66 MB per job. The worker drains through
# the sentinel after an exception, so a bounded queue cannot deadlock.
- self.queue: Queue[tuple[int, PixelArray | None]] = Queue(
+ self.queue: Queue[tuple[int, RGBAPixelArray | None]] = Queue(
maxsize=frame_queue_size,
)
self._exception: BaseException | None = None
+ self._sealed = False
+ self._abort_requested = False
self.thread = Thread(
target=self._listen_and_write,
name=f"partial-movie-encoder-{animation_index}",
@@ -122,109 +102,140 @@ class _PartialMovieEncodeJob:
"""Whether the worker has captured an exception."""
return self._exception is not None
+ def _abort_encoder(self) -> None:
+ try:
+ self.encoder.abort()
+ except Exception as exception:
+ logger.warning(
+ "Failed to clean up incomplete segment %(path)s: %(error)s",
+ {"path": f"'{self.path}'", "error": exception},
+ )
+ self._capture_exception(exception)
+
def _listen_and_write(self) -> None:
while True:
- num_frames, frame_data = self.queue.get()
+ repeat, frame_data = self.queue.get()
if frame_data is None:
break
if self._exception is not None:
continue
try:
- self._encode_and_write_frame(frame_data, num_frames)
+ self.encoder.write_frame(frame_data, repeat=repeat)
except BaseException as exception:
self._capture_exception(exception)
+ if self._abort_requested or self._exception is not None:
+ self._abort_encoder()
+ return
+
try:
- for packet in self.stream.encode():
- self.container.mux(packet)
+ self.encoder.finish()
except BaseException as exception:
self._capture_exception(exception)
- finally:
- try:
- self.container.close()
- except BaseException as exception:
- self._capture_exception(exception)
+ self._abort_encoder()
- def _encode_and_write_frame(self, frame: PixelArray, num_frames: int) -> None:
- for _ in range(num_frames):
- # Notes: precomputing reusing packets does not work!
- # I.e., you cannot do `packets = encode(...)`
- # and reuse it, as it seems that `mux(...)`
- # consumes the packet.
- # The same issue applies for `av_frame`,
- # reusing it renders weird-looking frames.
- av_frame = av.VideoFrame.from_ndarray(frame, format="rgba")
- for packet in self.stream.encode(av_frame):
- self.container.mux(packet)
-
- def put(self, num_frames: int, frame: PixelArray) -> None:
+ def put(self, repeat: int, frame: RGBAPixelArray) -> None:
"""Add a frame to the encoding queue."""
- self.queue.put((num_frames, frame))
+ self.queue.put((repeat, frame))
def seal(self) -> None:
"""Signal that no more frames will be added."""
- self.queue.put((-1, None))
+ if not self._sealed:
+ self._sealed = True
+ self.queue.put((-1, None))
+
+ def abort(self) -> None:
+ """Signal that the segment must be discarded."""
+ self._abort_requested = True
+ self.seal()
def join(self) -> None:
"""Wait for encoding to finish and propagate worker failures."""
self.thread.join()
if self._exception is not None:
- # A failed encode may leave a structurally valid but truncated
- # file behind; remove it so a later run cannot cache-hit it.
- try:
- Path(self.path).unlink(missing_ok=True)
- except OSError as cleanup_error:
- logger.warning(
- "Failed to remove incomplete partial movie file %(path)s: "
- "%(error)s",
- {"path": f"'{self.path}'", "error": cleanup_error},
- )
raise self._exception
+ if not self._abort_requested:
+ logger.info(
+ f"Animation {self.animation_index} : Partial movie file written in %(path)s",
+ {"path": f"'{self.path}'"},
+ )
- logger.info(
- f"Animation {self.animation_index} : Partial movie file written in %(path)s",
- {"path": f"'{self.path}'"},
- )
+@dataclass(frozen=True, slots=True)
+class _SceneFileWriterSettings:
+ """Immutable inputs consumed by one :class:`SceneFileWriter`.
-class SceneFileWriter:
- """SceneFileWriter is the object that actually writes the animations
- played, into video files, using FFMPEG.
- This is mostly for Manim's internal use. You will rarely, if ever,
- have to use the methods for this class, unless tinkering with the very
- fabric of Manim's reality.
+ The settings contain resolved output paths and segment encoding, bounded
+ encoder-pool limits, cache maintenance, and the sound-asset search root.
+ """
- Attributes
- ----------
- sections : list of :class:`.Section`
- used to segment scene
+ plan: OutputPlan
+ video_encoder: VideoEncoderSpec | None
+ max_inflight_encoders: int
+ encoder_queue_size: int
+ max_files_cached: int
+ assets_dir: Path
+
+ def __post_init__(self) -> None:
+ output = self.plan.output
+ if output.is_video != (self.video_encoder is not None):
+ raise ValueError(
+ "Video output and resolved video encoder settings must be provided together.",
+ )
+ expected_segment_extension = (
+ output.segment_extension if output.is_video else None
+ )
+ if self.plan.segment_extension != expected_segment_extension:
+ raise ValueError(
+ "The output plan segment extension does not match its output specification.",
+ )
+ if (
+ self.video_encoder is not None
+ and f".{self.video_encoder.container_format}" != expected_segment_extension
+ ):
+ raise ValueError(
+ "The video encoder container does not match the output plan.",
+ )
+ if self.max_inflight_encoders <= 0:
+ raise ValueError("max_inflight_encoders must be positive.")
+ if self.encoder_queue_size <= 0:
+ raise ValueError("encoder_queue_size must be positive.")
+ if self.max_files_cached < -1:
+ raise ValueError("max_files_cached must be non-negative or -1.")
+ if not self.assets_dir.is_absolute():
+ raise ValueError("assets_dir must be absolute.")
- sections_output_dir : :class:`pathlib.Path`
- where are section videos stored
- output_name : str
- name of movie without extension and basis for section video names
+class SceneFileWriter:
+ """Coordinate segment jobs and assemble one scene's media artifacts.
+
+ The writer receives immutable resolved settings and concrete top-left-origin
+ RGBA arrays. Ownership of each array passed to
+ :meth:`write_frame` transfers to the writer; callers must not mutate or reuse
+ it afterward. For video output the writer coordinates queued
+ :class:`.VideoSegmentEncoder` jobs, then assembles their silent cached
+ segments with optional audio, sections, and subcaptions. It also writes
+ still images and PNG sequences described by the output plan.
- Some useful attributes are:
- ``output_spec``
- The immutable primary output intent for this render session.
- ``partial_movie_files``
- List of all the partial-movie files.
+ Parameters
+ ----------
+ settings
+ Resolved output, encoding, pool, cache, and asset-search settings.
+ Attributes
+ ----------
+ sections
+ Ordered section metadata for the scene.
+ partial_movie_files
+ Segment paths in animation order, including ``None`` for skipped plays.
"""
- def __init__(
- self,
- renderer: CairoRenderer | OpenGLRenderer,
- scene_name: str,
- output_spec: OutputSpec,
- output_plan: OutputPlan,
- **kwargs: Any,
- ) -> None:
- self.renderer = renderer
- self.output_spec = output_spec
- self.output_plan = output_plan
+ def __init__(self, settings: _SceneFileWriterSettings) -> None:
+ self.settings = settings
+ self.output_spec = settings.plan.output
+ self.output_plan = settings.plan
+ self.video_encoder = settings.video_encoder
self._inflight_encode_jobs: list[_PartialMovieEncodeJob] = []
self._inflight_by_path: dict[str, _PartialMovieEncodeJob] = {}
self._current_encode_job: _PartialMovieEncodeJob | None = None
@@ -323,25 +334,20 @@ class SceneFileWriter:
)
def add_partial_movie_file(self, hash_animation: str | None) -> None:
- """Adds a new partial movie file path to ``scene.partial_movie_files``
- and current section from a hash.
+ """Append a planned segment path to the writer and current section.
- This method will compute the path from the hash. In addition to that it
- adds the new animation to the current section.
+ The list retains one entry per animation so explicit animation indices
+ select the corresponding segment.
Parameters
----------
hash_animation
Hash of the animation.
"""
- if (
- not hasattr(self, "partial_movie_directory")
- or not self.output_spec.is_video
- ):
+ if not self.output_spec.is_video:
return
- # None has to be added to partial_movie_files to keep the right index with scene.num_plays.
- # i.e if an animation is skipped, scene.num_plays is still incremented and we add an element to partial_movie_file be even with num_plays.
+ # Skipped animations retain a placeholder to preserve index alignment.
if hash_animation is None:
self.partial_movie_files.append(None)
self.sections[-1].partial_movie_files.append(None)
@@ -427,7 +433,7 @@ class SceneFileWriter:
used there can be referenced here.
"""
- file_path = get_full_sound_file_path(sound_file)
+ file_path = get_full_sound_file_path(sound_file, self.settings.assets_dir)
# we assume files with .wav / .raw suffix are actually
# .wav and .raw files, respectively.
if file_path.suffix not in (".wav", ".raw"):
@@ -447,75 +453,67 @@ class SceneFileWriter:
# Writers
def begin_animation(
- self, allow_write: bool = False, file_path: StrPath | None = None
+ self,
+ allow_write: bool = False,
+ *,
+ animation_index: int,
+ file_path: StrPath | None = None,
) -> None:
- """Used internally by manim to stream the animation to FFMPEG for
- displaying or writing to a file.
+ """Start a segment job for one animation when video writing is enabled.
Parameters
----------
allow_write
- Whether or not to write to a video file.
+ Whether this animation needs a new segment.
+ animation_index
+ Scene-local animation index used to select and label the segment.
+ file_path
+ Explicit segment target, or ``None`` to use the planned cache path.
"""
if self.output_spec.is_video and allow_write:
- self.open_partial_movie_stream(file_path=file_path)
+ self.open_partial_movie_stream(
+ animation_index=animation_index,
+ file_path=file_path,
+ )
def end_animation(self, allow_write: bool = False) -> None:
- """Internally used by Manim to stop streaming to FFMPEG gracefully.
+ """Seal the current segment job when video writing is enabled.
Parameters
----------
allow_write
- Whether or not to write to a video file.
+ Whether the current animation has an open segment job.
"""
if self.output_spec.is_video and allow_write:
self.close_partial_movie_stream()
def write_frame(
- self, frame_or_renderer: PixelArray | OpenGLRenderer, num_frames: int = 1
+ self,
+ pixels: RGBAPixelArray,
+ *,
+ repeat: int = 1,
) -> None:
- """Used internally by Manim to write a frame to the FFMPEG input buffer.
+ """Take ownership of one top-left C-contiguous ``uint8`` RGBA frame.
- Parameters
- ----------
- frame_or_renderer
- Pixel array of the frame.
- num_frames
- The number of times to write frame.
+ The caller must not mutate or reuse ``pixels`` after this method returns
+ because video encoding can consume the array asynchronously.
"""
if self.output_spec.is_video:
- if isinstance(frame_or_renderer, np.ndarray):
- frame = frame_or_renderer
- else:
- frame = (
- frame_or_renderer.get_frame()
- if config.renderer == RendererType.OPENGL
- else frame_or_renderer
- )
-
job = self._current_encode_job
if job is None:
- # Interactive OpenGL rendering emits frames outside an open
- # partial movie stream; drop them silently.
+ # Presentation rendering can emit frames outside an open
+ # segment; such frames do not belong to file output.
return
if job.failed:
# Surface the failure at the first write after it was captured;
- # join() unlinks the partial and re-raises.
+ # the worker discards the partial before join() re-raises.
job.seal()
self._current_encode_job = None
job.join()
- job.put(num_frames, frame)
+ job.put(repeat, pixels)
if self.output_spec.is_image_sequence:
- if isinstance(frame_or_renderer, np.ndarray):
- image = Image.fromarray(frame_or_renderer)
- else:
- image = (
- frame_or_renderer.get_image()
- if config.renderer == RendererType.OPENGL
- else Image.fromarray(frame_or_renderer)
- )
- self.output_image(image)
+ self.output_image(Image.fromarray(pixels))
def output_image(self, image: Image.Image) -> None:
file_path = self.output_plan.image_frame_path(self.frame_count)
@@ -523,35 +521,26 @@ class SceneFileWriter:
image.save(file_path)
self.frame_count += 1
- def save_image(self, image: Image.Image) -> None:
- """This method saves the image passed to it in the default image directory.
-
- Parameters
- ----------
- image
- The pixel array of the image to save.
- """
+ def save_image(self, pixels: RGBAPixelArray) -> None:
+ """Save one RGBA frame to the planned still-image path."""
if not self.output_spec.enabled:
return
self.image_file_path.parent.mkdir(parents=True, exist_ok=True)
- image.save(self.image_file_path)
+ Image.fromarray(pixels).save(self.image_file_path)
self.print_file_ready_message(self.image_file_path)
def finish(self) -> None:
- """Finishes writing to the FFMPEG buffer or writing images to output directory.
- Combines the partial movie files into the whole scene.
- If save_last_frame is True, saves the last frame in the default image directory.
- """
+ """Drain segment jobs and assemble the configured time-based output."""
if self.output_spec.is_video:
self.join_all_encode_jobs()
self.combine_to_movie()
if self.output_spec.save_sections:
self.combine_to_section_videos()
# Cache cleanup runs after the in-flight encode jobs have been drained.
- if config["flush_cache"]:
- self.flush_cache_directory()
- else:
- self.clean_cache()
+ prune_segment_cache(
+ self.partial_movie_directory,
+ self.settings.max_files_cached,
+ )
elif self.output_spec.is_image_sequence:
target_dir = self.image_sequence_directory
self.final_file_path = target_dir
@@ -559,14 +548,25 @@ class SceneFileWriter:
if self.subcaptions:
self.write_subcaption_file()
- def open_partial_movie_stream(self, file_path: StrPath | None = None) -> None:
- """Open a container holding a video stream.
+ def _create_segment_encoder(self, target: Path) -> VideoSegmentEncoder:
+ encoder = self.video_encoder
+ if encoder is None:
+ raise RuntimeError("Video segment encoding requires resolved settings.")
+ return VideoSegmentEncoder(target=target, spec=encoder)
- This is used internally by Manim initialize the container holding
- the video stream of a partial movie file.
- """
+ def open_partial_movie_stream(
+ self,
+ *,
+ animation_index: int,
+ file_path: StrPath | None = None,
+ ) -> None:
+ """Create a queued encoder job for one planned video segment."""
+ if self._current_encode_job is not None:
+ raise RuntimeError(
+ "Cannot open a video segment while another segment is still open.",
+ )
if file_path is None:
- file_path = self.partial_movie_files[self.renderer.num_plays]
+ file_path = self.partial_movie_files[animation_index]
if file_path is None:
raise RuntimeError(
"open_partial_movie_stream() called for a play that has no "
@@ -577,45 +577,15 @@ class SceneFileWriter:
path_key = str(file_path)
if path_key in self._inflight_by_path:
self._join_job_and_drain_on_failure(self._inflight_by_path[path_key])
- self.partial_movie_file_path = file_path
-
- fps = to_av_frame_rate(config.frame_rate)
-
- partial_movie_file_codec = "libx264"
- partial_movie_file_pix_fmt = "yuv420p"
- av_options = {
- "an": "1", # ffmpeg: -an, no audio
- "crf": "23", # ffmpeg: -crf, constant rate factor (improved bitrate)
- }
-
- if self.output_spec.segment_extension == ".webm":
- partial_movie_file_codec = "libvpx-vp9"
- av_options["-auto-alt-ref"] = "1"
- if self.output_spec.transparent:
- partial_movie_file_pix_fmt = "yuva420p"
-
- elif self.output_spec.transparent:
- partial_movie_file_codec = "qtrle"
- partial_movie_file_pix_fmt = "argb"
-
- video_container = av.open(file_path, mode="w")
- stream = video_container.add_stream(
- partial_movie_file_codec,
- rate=fps,
- options=av_options,
- )
- stream.pix_fmt = partial_movie_file_pix_fmt
- stream.width = config.pixel_width
- stream.height = config.pixel_height
-
+ segment_encoder = self._create_segment_encoder(file_path)
frame_queue_size = (
- 0 if config.max_inflight_encoders == 1 else config.encoder_queue_size
+ 0
+ if self.settings.max_inflight_encoders == 1
+ else self.settings.encoder_queue_size
)
self._current_encode_job = _PartialMovieEncodeJob(
- path=file_path,
- animation_index=self.renderer.num_plays,
- container=video_container,
- stream=stream,
+ animation_index=animation_index,
+ encoder=segment_encoder,
frame_queue_size=frame_queue_size,
)
@@ -655,24 +625,18 @@ class SceneFileWriter:
raise first_exception
def abort_encode_jobs(self, reraise_encoder_failures: bool = False) -> None:
- """Tear down encode jobs after an aborted or rerun render.
-
- Seals the current job so its worker can exit (a non-daemon thread
- blocked on the queue would hang the process at exit), then deletes its
- partial file unconditionally: an aborted partial is structurally valid
- but truncated, so leaving it behind produces an erroneous cache hit on
- a later run. Sealed in-flight jobs are then drained. With
- ``reraise_encoder_failures=False`` (a render exception is already
- propagating) drain failures are logged, not raised; with ``True``
- (rerun path -- no primary exception exists) the first drain failure
- propagates so corrupt completed partials cannot be silently ignored.
+ """Discard the current segment and drain completed encode jobs.
+
+ When ``reraise_encoder_failures`` is true, the first encoder failure is
+ propagated. Otherwise failures are logged so an active render exception
+ remains primary.
"""
current_exception: BaseException | None = None
job = self._current_encode_job
if job is not None:
- # Seal before clearing: an interrupt landing between the two
- # statements must not orphan the worker.
- job.seal()
+ # Request abort before clearing: an interrupt between these
+ # statements must not orphan a worker blocked on its queue.
+ job.abort()
self._current_encode_job = None
job.thread.join()
current_exception = job._exception
@@ -682,18 +646,11 @@ class SceneFileWriter:
job.animation_index,
exc_info=current_exception,
)
- try:
- Path(job.path).unlink(missing_ok=True)
+ else:
logger.info(
"Discarded partial movie file of aborted animation %(index)d",
{"index": job.animation_index},
)
- except OSError as cleanup_error:
- logger.warning(
- "Failed to remove incomplete partial movie file %(path)s: "
- "%(error)s",
- {"path": f"'{job.path}'", "error": cleanup_error},
- )
if reraise_encoder_failures:
self.join_all_encode_jobs()
if current_exception is not None:
@@ -707,12 +664,7 @@ class SceneFileWriter:
logger.exception("Encoder failure while aborting render")
def close_partial_movie_stream(self) -> None:
- """Close the currently opened video container.
-
- Used internally by Manim to first flush the remaining packages
- in the video stream holding a partial file, and then close
- the corresponding container.
- """
+ """Seal the current segment and enforce the in-flight job limit."""
job = self._current_encode_job
if job is None:
raise RuntimeError(
@@ -724,7 +676,7 @@ class SceneFileWriter:
self._inflight_by_path[str(job.path)] = job
self._current_encode_job = None
- while len(self._inflight_encode_jobs) >= config.max_inflight_encoders:
+ while len(self._inflight_encode_jobs) >= self.settings.max_inflight_encoders:
self._join_job_and_drain_on_failure(self._inflight_encode_jobs[0])
def is_already_cached(self, hash_invocation: str) -> bool:
@@ -740,10 +692,7 @@ class SceneFileWriter:
:class:`bool`
Whether the file exists.
"""
- if (
- not hasattr(self, "partial_movie_directory")
- or not self.output_spec.is_video
- ):
+ if not self.output_spec.is_video:
return False
path = self.output_plan.segment_path(hash_invocation)
path_key = str(path)
@@ -751,6 +700,41 @@ class SceneFileWriter:
self._join_job_and_drain_on_failure(self._inflight_by_path[path_key])
return path.exists()
+ @staticmethod
+ def _concat_manifest_bytes(input_files: list[str]) -> bytes:
+ """Return a complete FFmpeg concat manifest for ``input_files``."""
+ manifest_text = (
+ "# This file records the segment order used by Manim.\n"
+ + "".join(
+ f"file 'file:{Path(file_path).as_posix()}'\n"
+ for file_path in input_files
+ )
+ )
+ return manifest_text.encode("utf-8")
+
+ def _write_concat_manifest(self, input_files: list[str]) -> None:
+ """Atomically persist the complete scene segment order for diagnostics."""
+ manifest_path = self.output_plan.concat_manifest
+ assert manifest_path is not None
+ manifest_path.parent.mkdir(parents=True, exist_ok=True)
+ temporary_path: Path | None = None
+ try:
+ with NamedTemporaryFile(
+ mode="wb",
+ dir=manifest_path.parent,
+ prefix=f".{manifest_path.name}.",
+ suffix=".tmp",
+ delete=False,
+ ) as temporary_file:
+ temporary_path = Path(temporary_file.name)
+ temporary_file.write(self._concat_manifest_bytes(input_files))
+ temporary_path.replace(manifest_path)
+ except BaseException:
+ if temporary_path is not None:
+ with suppress(OSError):
+ temporary_path.unlink(missing_ok=True)
+ raise
+
def combine_files(
self,
input_files: list[str],
@@ -758,19 +742,12 @@ class SceneFileWriter:
create_gif: bool = False,
includes_sound: bool = False,
) -> None:
- file_list = self.output_plan.concat_manifest
- assert file_list is not None
- file_list.parent.mkdir(parents=True, exist_ok=True)
output_file.parent.mkdir(parents=True, exist_ok=True)
logger.debug(
f"Partial movie files to combine ({len(input_files)} files): %(p)s",
{"p": input_files[:5]},
)
- with file_list.open("w", encoding="utf-8") as fp:
- fp.write("# This file is used internally by FFMPEG.\n")
- for pf_path in input_files:
- pf_path = Path(pf_path).as_posix()
- fp.write(f"file 'file:{pf_path}'\n")
+ manifest = BytesIO(self._concat_manifest_bytes(input_files))
av_options = {
"safe": "0", # needed to read files
@@ -780,7 +757,9 @@ class SceneFileWriter:
av_options["an"] = "1"
partial_movies_input = av.open(
- str(file_list), options=av_options, format="concat"
+ manifest,
+ options=av_options,
+ format="concat",
)
partial_movies_stream = partial_movies_input.streams.video[0]
output_container = av.open(str(output_file), mode="w")
@@ -799,9 +778,11 @@ class SceneFileWriter:
output_stream.pix_fmt = "rgb8"
if self.output_spec.transparent:
output_stream.pix_fmt = "pal8"
- output_stream.width = config.pixel_width
- output_stream.height = config.pixel_height
- output_stream.rate = to_av_frame_rate(config.frame_rate)
+ encoder = self.video_encoder
+ assert encoder is not None
+ output_stream.width = encoder.width
+ output_stream.height = encoder.height
+ output_stream.rate = encoder.frame_rate
graph = av.filter.Graph()
input_buffer = graph.add_buffer(template=partial_movies_stream)
split = graph.add("split")
@@ -862,6 +843,7 @@ class SceneFileWriter:
partial_movies_input.close()
output_container.close()
+ manifest.close()
def combine_to_movie(self) -> None:
"""Used internally by Manim to combine the separate
@@ -884,6 +866,7 @@ class SceneFileWriter:
return
logger.info("Combining to Movie file.")
+ self._write_concat_manifest(partial_movie_files)
self.combine_files(
partial_movie_files,
movie_file_path,
@@ -991,63 +974,6 @@ class SceneFileWriter:
with section_index.open("w") as file:
json.dump(sections_index, file, indent=4)
- def _cached_partial_movie_files(self) -> list[Path]:
- """Return the partial movie files currently contained in the cache.
-
- The partial movie file list is excluded (matching by file name: a
- ``Path`` never compares equal to its ``str`` name). Hidden files are
- excluded as well: on macOS, Finder leaves resource forks (``._*.mp4``)
- and ``.DS_Store`` files in the directory which must not count against
- ``max_files_cached`` and may vanish again before they could be
- deleted.
- """
- if not self.partial_movie_directory.exists():
- return []
- return [
- self.partial_movie_directory / file_name
- for file_name in self.partial_movie_directory.iterdir()
- if file_name.name != "partial_movie_file_list.txt"
- and not file_name.name.startswith(".")
- ]
-
- def clean_cache(self) -> None:
- """Will clean the cache by removing the oldest partial_movie_files."""
- cached_partial_movies = self._cached_partial_movie_files()
- if len(cached_partial_movies) > config["max_files_cached"]:
- number_files_to_delete = (
- len(cached_partial_movies) - config["max_files_cached"]
- )
-
- def access_time(path: Path) -> float:
- try:
- return path.stat().st_atime
- except FileNotFoundError:
- # The file vanished between listing the directory and
- # now; sort it first and let unlink(missing_ok=True)
- # handle its deletion without evicting an existing file.
- return float("-inf")
-
- oldest_files_to_delete = sorted(
- cached_partial_movies,
- key=access_time,
- )[:number_files_to_delete]
- for file_to_delete in oldest_files_to_delete:
- file_to_delete.unlink(missing_ok=True)
- logger.info(
- f"The partial movie directory is full (> {config['max_files_cached']} files). Therefore, manim has removed the {number_files_to_delete} oldest file(s)."
- " You can change this behaviour by changing max_files_cached in config.",
- )
-
- def flush_cache_directory(self) -> None:
- """Delete all the cached partial movie files"""
- cached_partial_movies = self._cached_partial_movie_files()
- for f in cached_partial_movies:
- f.unlink(missing_ok=True)
- logger.info(
- f"Cache flushed. {len(cached_partial_movies)} file(s) deleted in %(par_dir)s.",
- {"par_dir": self.partial_movie_directory},
- )
-
def write_subcaption_file(self) -> None:
"""Writes the subcaption file next to the primary video artifact."""
if not self.output_spec.is_video:
diff --git a/manim/scene/video_segment_encoder.py b/manim/scene/video_segment_encoder.py
new file mode 100644
index 000000000..55dd06363
--- /dev/null
+++ b/manim/scene/video_segment_encoder.py
@@ -0,0 +1,143 @@
+"""Encoding for one silent cached video segment."""
+
+from __future__ import annotations
+
+from contextlib import suppress
+from fractions import Fraction
+from pathlib import Path
+
+import av
+import numpy as np
+
+from manim._config.video_encoder import VideoEncoderSpec
+from manim.typing import RGBAPixelArray
+
+__all__ = ["VideoSegmentEncoder"]
+
+
+class VideoSegmentEncoder:
+ """Encode top-left-origin C-contiguous RGBA frames into one video segment."""
+
+ def __init__(self, *, target: Path, spec: VideoEncoderSpec) -> None:
+ self.target = target
+ self.spec = spec
+ self._next_pts = 0
+ self._closed = False
+ self.target.parent.mkdir(parents=True, exist_ok=True)
+
+ container = None
+ try:
+ container = av.open(
+ self.target,
+ mode="w",
+ format=self.spec.container_format,
+ )
+ stream = container.add_stream(
+ self.spec.codec,
+ rate=self.spec.frame_rate,
+ options=dict(self.spec.options),
+ )
+ stream.pix_fmt = self.spec.pixel_format
+ stream.width = self.spec.width
+ stream.height = self.spec.height
+ except BaseException as error:
+ if container is not None:
+ with suppress(BaseException):
+ container.close()
+ with suppress(OSError):
+ self.target.unlink(missing_ok=True)
+ raise self._operation_error("open", error) from error
+
+ self._container = container
+ self._stream = stream
+
+ def _profile_description(self) -> str:
+ return (
+ f"{self.spec.container_format}/{self.spec.codec}/"
+ f"{self.spec.pixel_format}, {self.spec.width}x{self.spec.height} "
+ f"at {self.spec.frame_rate} fps"
+ )
+
+ def _operation_error(self, operation: str, error: BaseException) -> RuntimeError:
+ return RuntimeError(
+ f"Failed to {operation} video segment {self.target} "
+ f"({self._profile_description()}): {error}",
+ )
+
+ def _validate_frame(self, pixels: RGBAPixelArray, repeat: int) -> None:
+ if self._closed:
+ raise RuntimeError(f"Video segment encoder for {self.target} is closed.")
+ if isinstance(repeat, bool) or not isinstance(repeat, int) or repeat <= 0:
+ raise ValueError("Frame repeat must be a positive integer.")
+ if not isinstance(pixels, np.ndarray):
+ raise TypeError("Video segment frames must be NumPy arrays.")
+ expected_shape = (self.spec.height, self.spec.width, 4)
+ if pixels.shape != expected_shape:
+ raise ValueError(
+ f"Video segment frames must have shape {expected_shape}; "
+ f"got {pixels.shape}.",
+ )
+ if pixels.dtype != np.uint8:
+ raise TypeError(
+ f"Video segment frames must use uint8; got {pixels.dtype}.",
+ )
+ if not pixels.flags.c_contiguous:
+ raise ValueError("Video segment frames must be C-contiguous.")
+
+ def write_frame(self, pixels: RGBAPixelArray, *, repeat: int = 1) -> None:
+ """Encode ``pixels`` at consecutive segment-local PTS values."""
+ self._validate_frame(pixels, repeat)
+ time_base = Fraction(
+ self.spec.frame_rate.denominator,
+ self.spec.frame_rate.numerator,
+ )
+ try:
+ for _ in range(repeat):
+ frame = av.VideoFrame.from_ndarray(pixels, format="rgba")
+ frame.pts = self._next_pts
+ frame.time_base = time_base
+ self._next_pts += 1
+ for packet in self._stream.encode(frame):
+ self._container.mux(packet)
+ except BaseException as error:
+ raise self._operation_error("encode", error) from error
+
+ def finish(self) -> None:
+ """Flush encoded packets and close the segment."""
+ if self._closed:
+ return
+ self._closed = True
+ first_error: Exception | None = None
+ try:
+ try:
+ for packet in self._stream.encode():
+ self._container.mux(packet)
+ except Exception as error:
+ first_error = error
+ finally:
+ try:
+ self._container.close()
+ except Exception as error:
+ if first_error is None:
+ first_error = error
+ if first_error is not None:
+ raise self._operation_error("finish", first_error) from first_error
+
+ def abort(self) -> None:
+ """Close resources and remove the incomplete target."""
+ first_error: Exception | None = None
+ try:
+ if not self._closed:
+ self._closed = True
+ try:
+ self._container.close()
+ except Exception as error:
+ first_error = error
+ finally:
+ try:
+ self.target.unlink(missing_ok=True)
+ except Exception as error:
+ if first_error is None:
+ first_error = error
+ if first_error is not None:
+ raise self._operation_error("abort", first_error) from first_error
diff --git a/manim/utils/caching.py b/manim/utils/caching.py
index 9ab7e6bfd..78e011071 100644
--- a/manim/utils/caching.py
+++ b/manim/utils/caching.py
@@ -1,12 +1,82 @@
from __future__ import annotations
from collections.abc import Callable
+from pathlib import Path
from typing import TYPE_CHECKING, Any
from .. import config, logger
+from .._config.video_encoder import video_encoder_fingerprint
from ..utils.hashing import get_hash_from_play_call
-__all__ = ["handle_caching_play"]
+__all__ = [
+ "clear_segment_cache",
+ "handle_caching_play",
+ "prune_segment_cache",
+]
+
+_SEGMENT_EXTENSIONS = frozenset({".mov", ".mp4", ".webm"})
+
+
+def _segment_cache_files(directory: Path) -> list[Path]:
+ """Return recognized segment files currently present in ``directory``."""
+ try:
+ entries = directory.iterdir()
+ return [
+ entry
+ for entry in entries
+ if not entry.name.startswith(".")
+ and entry.suffix.lower() in _SEGMENT_EXTENSIONS
+ and entry.is_file()
+ ]
+ except FileNotFoundError:
+ return []
+
+
+def prune_segment_cache(directory: Path, max_files: int) -> None:
+ """Remove the least recently accessed segments beyond ``max_files``.
+
+ ``max_files=-1`` leaves the cache unlimited. Files that disappear during
+ pruning count toward the number selected for removal, preventing a race
+ from evicting an additional live segment.
+ """
+ if max_files == -1:
+ return
+ if max_files < 0:
+ raise ValueError("max_files must be non-negative or -1 for unlimited")
+
+ cached_segments = _segment_cache_files(directory)
+ excess = len(cached_segments) - max_files
+ if excess <= 0:
+ return
+
+ def access_time(path: Path) -> float:
+ try:
+ return path.stat().st_atime
+ except FileNotFoundError:
+ return float("-inf")
+
+ for segment in sorted(cached_segments, key=access_time)[:excess]:
+ segment.unlink(missing_ok=True)
+
+ logger.info(
+ "The segment cache exceeded %d files; removed %d least recently used "
+ "segment(s). Change max_files_cached to adjust this limit.",
+ max_files,
+ excess,
+ )
+
+
+def clear_segment_cache(directory: Path) -> int:
+ """Delete recognized segment files from ``directory`` and return the count."""
+ removed = 0
+ for segment in _segment_cache_files(directory):
+ try:
+ segment.unlink()
+ except FileNotFoundError:
+ continue
+ removed += 1
+ return removed
+
if TYPE_CHECKING:
from manim.renderer.opengl_renderer import OpenGLRenderer
@@ -53,6 +123,15 @@ def handle_caching_play(func: Callable[..., None]) -> Callable[..., None]:
self.camera,
animations,
mobjects_on_scene,
+ backend="opengl",
+ encoder_fingerprint=video_encoder_fingerprint(
+ scene.session_spec.video_encoder,
+ ),
+ renderer_state={
+ "meshes": scene.meshes,
+ "background_color": self.background_color,
+ "anti_alias_width": self.anti_alias_width,
+ },
)
if self.file_writer.is_already_cached(hash_play):
logger.info(
diff --git a/manim/utils/hashing.py b/manim/utils/hashing.py
index 4dfd11c1d..3b36e0017 100644
--- a/manim/utils/hashing.py
+++ b/manim/utils/hashing.py
@@ -418,8 +418,12 @@ def get_hash_from_play_call(
camera_object: Camera | OpenGLCamera,
animations_list: Iterable[Animation],
current_mobjects_list: Iterable[Mobject],
+ *,
+ backend: str,
+ encoder_fingerprint: str,
+ renderer_state: Any,
) -> str:
- """Take the list of animations and a list of mobjects and output their hashes. This is meant to be used for `scene.play` function.
+ """Return the visual-segment cache key for one compiled play call.
Parameters
-----------
@@ -434,12 +438,23 @@ def get_hash_from_play_call(
current_mobjects_list
The list of mobjects.
+ backend
+ Stable identity of the renderer producing the segment.
+ encoder_fingerprint
+ Stable identity of the resolved segment encoder settings.
+ renderer_state
+ Additional renderer-specific state which affects segment pixels.
Returns
-------
:class:`str`
- A string concatenation of the respective hashes of `camera_object`, `animations_list` and `current_mobjects_list`, separated by `_`.
+ A filename-safe digest of all visual cache inputs.
"""
+ if backend not in {"cairo", "opengl"}:
+ raise ValueError(f"Unsupported cache backend: {backend}")
+ if not encoder_fingerprint:
+ raise ValueError("An encoder fingerprint is required for cache identity.")
+
logger.debug("Hashing ...")
t_start = perf_counter()
memoizer = _Memoizer()
@@ -453,12 +468,32 @@ def get_hash_from_play_call(
_get_json(mobject, memoizer, include_pixel_array=True)
for mobject in current_mobjects_list
]
- hash_camera, hash_animations, hash_current_mobjects = (
- zlib.crc32(repr(json_val).encode())
- for json_val in [camera_json, animations_list_json, current_mobjects_list_json]
+ renderer_state_json = _get_json(
+ renderer_state,
+ memoizer,
+ include_pixel_array=True,
)
- hash_complete = f"{hash_camera}_{hash_animations}_{hash_current_mobjects}"
+ digest = hashlib.sha256()
+
+ def feed(value: str) -> None:
+ encoded = value.encode("utf-8")
+ digest.update(len(encoded).to_bytes(8, "big"))
+ digest.update(encoded)
+
+ for name, values in (
+ ("backend", [backend]),
+ ("encoder", [encoder_fingerprint]),
+ ("camera", [camera_json]),
+ ("animations", animations_list_json),
+ ("mobjects", current_mobjects_list_json),
+ ("renderer", [renderer_state_json]),
+ ):
+ feed(name)
+ feed(str(len(values)))
+ for value in values:
+ feed(value)
+ cache_key = digest.hexdigest()
t_end = perf_counter()
logger.debug("Hashing done in %(time)s s.", {"time": str(t_end - t_start)[:8]})
- logger.debug("Hash generated : %(h)s", {"h": hash_complete})
- return hash_complete
+ logger.debug("Hash generated : %(h)s", {"h": cache_key})
+ return cache_key
diff --git a/manim/utils/sounds.py b/manim/utils/sounds.py
index de7b6be80..ae6c909f2 100644
--- a/manim/utils/sounds.py
+++ b/manim/utils/sounds.py
@@ -8,7 +8,6 @@ __all__ = [
from typing import TYPE_CHECKING
-from .. import config
from ..utils.file_ops import seek_full_path_from_defaults
if TYPE_CHECKING:
@@ -18,9 +17,10 @@ if TYPE_CHECKING:
# Still in use by add_sound() function in scene_file_writer.py
-def get_full_sound_file_path(sound_file_name: StrPath) -> Path:
+def get_full_sound_file_path(sound_file_name: StrPath, assets_dir: Path) -> Path:
+ """Locate a sound path directly or relative to ``assets_dir``."""
return seek_full_path_from_defaults(
sound_file_name,
- default_dir=config.get_dir("assets_dir"),
+ default_dir=assets_dir,
extensions=[".wav", ".mp3"],
)
diff --git a/manim/utils/testing/_test_class_makers.py b/manim/utils/testing/_test_class_makers.py
index 40897fa2f..49a1a8514 100644
--- a/manim/utils/testing/_test_class_makers.py
+++ b/manim/utils/testing/_test_class_makers.py
@@ -6,7 +6,7 @@ from typing import Any
from manim.renderer.cairo_renderer import CairoRenderer
from manim.renderer.opengl_renderer import OpenGLRenderer
from manim.scene.scene import Scene
-from manim.scene.scene_file_writer import SceneFileWriter
+from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings
from manim.typing import PixelArray, StrPath
from ._frames_testers import _FramesTester
@@ -44,20 +44,19 @@ def _make_test_renderer_class(from_renderer: type) -> Any:
class DummySceneFileWriter(SceneFileWriter):
"""Delegate of SceneFileWriter used to test the frames."""
- def __init__(
- self,
- renderer: CairoRenderer | OpenGLRenderer,
- scene_name: str,
- **kwargs: Any,
- ) -> None:
- super().__init__(renderer, scene_name, **kwargs)
+ def __init__(self, settings: _SceneFileWriterSettings) -> None:
+ super().__init__(settings)
self.i = 0
def add_partial_movie_file(self, hash_animation: str | None) -> None:
pass
def begin_animation(
- self, allow_write: bool = True, file_path: StrPath | None = None
+ self,
+ allow_write: bool = True,
+ *,
+ animation_index: int,
+ file_path: StrPath | None = None,
) -> Any:
pass
@@ -70,21 +69,14 @@ class DummySceneFileWriter(SceneFileWriter):
def combine_to_section_videos(self) -> None:
pass
- def clean_cache(self) -> None:
- pass
-
- def write_frame(
- self, frame_or_renderer: PixelArray | OpenGLRenderer, num_frames: int = 1
- ) -> None:
+ def write_frame(self, pixels: PixelArray, *, repeat: int = 1) -> None:
self.i += 1
def _make_scene_file_writer_class(tester: _FramesTester) -> type[SceneFileWriter]:
class TestSceneFileWriter(DummySceneFileWriter):
- def write_frame(
- self, frame_or_renderer: PixelArray | OpenGLRenderer, num_frames: int = 1
- ) -> None:
- tester.check_frame(self.i, frame_or_renderer)
- super().write_frame(frame_or_renderer, num_frames=num_frames)
+ def write_frame(self, pixels: PixelArray, *, repeat: int = 1) -> None:
+ tester.check_frame(self.i, pixels)
+ super().write_frame(pixels, repeat=repeat)
return TestSceneFileWriter
diff --git a/tests/interface/test_cache_command.py b/tests/interface/test_cache_command.py
new file mode 100644
index 000000000..177eb8b02
--- /dev/null
+++ b/tests/interface/test_cache_command.py
@@ -0,0 +1,116 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+from click.testing import CliRunner
+
+from manim.__main__ import main
+
+
+def _segment_directory(
+ media_dir: Path,
+ *,
+ module_name: str,
+ quality: str,
+ scene_name: str,
+) -> Path:
+ return (
+ media_dir
+ / "videos"
+ / module_name
+ / quality
+ / "partial_movie_files"
+ / scene_name
+ )
+
+
+def test_cache_clear_resolves_each_scene_without_importing_file(tmp_path):
+ scene_file = tmp_path / "example.py"
+ scene_file.write_text("raise RuntimeError('must not be imported')\n")
+ media_dir = tmp_path / "media"
+
+ for scene_name in ("FirstScene", "SecondScene"):
+ directory = _segment_directory(
+ media_dir,
+ module_name="example",
+ quality="480p15",
+ scene_name=scene_name,
+ )
+ directory.mkdir(parents=True)
+ (directory / "segment.mp4").touch()
+
+ result = CliRunner().invoke(
+ main,
+ [
+ "cache",
+ "clear",
+ "--media-dir",
+ str(media_dir),
+ "-q",
+ "l",
+ str(scene_file),
+ "FirstScene",
+ "SecondScene",
+ ],
+ prog_name="manim",
+ )
+
+ assert result.exception is None, result.output
+ for scene_name in ("FirstScene", "SecondScene"):
+ directory = _segment_directory(
+ media_dir,
+ module_name="example",
+ quality="480p15",
+ scene_name=scene_name,
+ )
+ assert not (directory / "segment.mp4").exists()
+ assert "Removed 1 cached segment(s) for FirstScene" in result.output
+ assert "Removed 1 cached segment(s) for SecondScene" in result.output
+
+
[email protected]("override_media_dir", [False, True])
+def test_cache_clear_applies_config_file_and_path_overrides(
+ tmp_path,
+ override_media_dir,
+):
+ scene_file = tmp_path / "configured_scene.py"
+ scene_file.touch()
+ configured_media_dir = tmp_path / "configured-media"
+ selected_media_dir = (
+ tmp_path / "overridden-media" if override_media_dir else configured_media_dir
+ )
+ config_file = tmp_path / "cache.cfg"
+ config_file.write_text(f"[CLI]\nmedia_dir = {configured_media_dir}\n")
+
+ directory = _segment_directory(
+ selected_media_dir,
+ module_name="configured_scene",
+ quality="360p12",
+ scene_name="ConfiguredScene",
+ )
+ directory.mkdir(parents=True)
+ segment = directory / "segment.mp4"
+ segment.touch()
+
+ options = ["--config-file", str(config_file)]
+ if override_media_dir:
+ options.extend(["--media-dir", str(selected_media_dir)])
+ result = CliRunner().invoke(
+ main,
+ [
+ "cache",
+ "clear",
+ *options,
+ "--resolution",
+ "640,360",
+ "--fps",
+ "12",
+ str(scene_file),
+ "ConfiguredScene",
+ ],
+ prog_name="manim",
+ )
+
+ assert result.exception is None, result.output
+ assert not segment.exists()
diff --git a/tests/module/test_output_path_behavior.py b/tests/module/test_output_path_behavior.py
index c71f18c6a..1d1e49804 100644
--- a/tests/module/test_output_path_behavior.py
+++ b/tests/module/test_output_path_behavior.py
@@ -3,8 +3,8 @@ from __future__ import annotations
from pathlib import Path
from unittest.mock import Mock
+import numpy as np
import pytest
-from PIL import Image
from manim import __version__
from manim._config.output import OutputFormat, OutputSpec
@@ -14,7 +14,8 @@ from manim._config.output_plan import (
resolve_output_plan,
resolve_requested_output_name,
)
-from manim.scene.scene_file_writer import SceneFileWriter
+from manim._config.video_encoder import resolve_video_encoder
+from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings
def _make_writer(
@@ -53,14 +54,20 @@ def _make_writer(
scene_name="ExampleScene",
requested_output_name=resolve_requested_output_name(config),
)
- renderer = Mock()
- renderer.num_plays = 0
- return SceneFileWriter(
- renderer,
- "ExampleScene",
- output,
- output_plan,
+ settings = _SceneFileWriterSettings(
+ plan=output_plan,
+ video_encoder=resolve_video_encoder(
+ output,
+ width=config.pixel_width,
+ height=config.pixel_height,
+ frame_rate=config.frame_rate,
+ ),
+ max_inflight_encoders=config.max_inflight_encoders,
+ encoder_queue_size=config.encoder_queue_size,
+ max_files_cached=config.max_files_cached,
+ assets_dir=Path.cwd(),
)
+ return SceneFileWriter(settings)
@pytest.mark.parametrize(
@@ -125,7 +132,7 @@ def test_default_png_and_automatic_video_fallback_paths(config, tmp_path):
/ f"ExampleScene_ManimCE_v{__version__}.png"
)
- png_writer.save_image(Image.new("RGBA", (1, 1)))
+ png_writer.save_image(np.zeros((1, 1, 4), dtype=np.uint8))
assert png_writer.final_file_path == expected
video_writer = _make_writer(
@@ -134,7 +141,7 @@ def test_default_png_and_automatic_video_fallback_paths(config, tmp_path):
OutputFormat.MP4,
fallback_to_still=True,
)
- video_writer.save_image(Image.new("RGBA", (1, 1)))
+ video_writer.save_image(np.zeros((1, 1, 4), dtype=np.uint8))
assert video_writer.final_file_path == expected
@@ -145,7 +152,10 @@ def test_png_sequence_path_and_zero_padding(config, tmp_path):
expected_dir = tmp_path / "images" / "example.scene" / "ExampleScene"
assert writer.image_sequence_directory == expected_dir
- writer.output_image(Image.new("RGBA", (1, 1)))
+ pixels = np.zeros((1, 1, 4), dtype=np.uint8)
+ writer.write_frame(pixels)
+
+ assert not hasattr(writer, "renderer")
assert (expected_dir / "000.png").is_file()
@@ -218,6 +228,53 @@ def test_nested_and_absolute_output_names_do_not_relocate_sections(
)
+def test_diagnostic_concat_manifest_records_the_full_scene_only(
+ config,
+ tmp_path,
+ monkeypatch,
+):
+ writer = _make_writer(
+ config,
+ tmp_path,
+ OutputFormat.MP4,
+ save_sections=True,
+ )
+ partial_directory = writer.partial_movie_directory
+ partial_directory.mkdir(parents=True)
+ first = partial_directory / "first.mp4"
+ second = partial_directory / "second.mp4"
+ first.touch()
+ second.touch()
+ writer.partial_movie_files = [str(first), None, str(second)]
+
+ manifest_path = writer.output_plan.concat_manifest
+ assert manifest_path is not None
+ manifest_path.write_text("stale")
+ combine_files = Mock()
+ monkeypatch.setattr(writer, "combine_files", combine_files)
+
+ writer.combine_to_movie()
+
+ expected_manifest = (
+ "# This file records the segment order used by Manim.\n"
+ f"file 'file:{first.as_posix()}'\n"
+ f"file 'file:{second.as_posix()}'\n"
+ )
+ assert manifest_path.read_text() == expected_manifest
+ assert not list(partial_directory.glob(".partial_movie_file_list.txt.*.tmp"))
+
+ section = writer.sections[-1]
+ section.partial_movie_files = [str(second)]
+ monkeypatch.setattr(
+ section,
+ "get_dict",
+ Mock(return_value={"name": section.name}),
+ )
+ writer.combine_to_section_videos()
+
+ assert manifest_path.read_text() == expected_manifest
+
+
def test_no_output_plans_no_media_directories(config, tmp_path):
media_root = tmp_path / "unused-media"
writer = _make_writer(config, media_root, OutputFormat.NONE)
diff --git a/tests/module/test_output_plan.py b/tests/module/test_output_plan.py
index c49660031..d7a72534b 100644
--- a/tests/module/test_output_plan.py
+++ b/tests/module/test_output_plan.py
@@ -56,13 +56,15 @@ def _output(
def test_resolve_video_plan(tmp_path, output_format, extension):
layout = _layout(tmp_path)
+ output = _output(output_format)
plan = resolve_output_plan(
layout,
- _output(output_format),
+ output,
scene_name="ExampleScene",
requested_output_name=None,
)
+ assert plan.output is output
assert plan.primary_artifact == layout.video_dir / f"ExampleScene{extension}"
assert plan.fallback_image is None
assert plan.segment_cache_dir == layout.partial_movie_dir
diff --git a/tests/module/test_scene_file_writer_settings.py b/tests/module/test_scene_file_writer_settings.py
new file mode 100644
index 000000000..2d8680120
--- /dev/null
+++ b/tests/module/test_scene_file_writer_settings.py
@@ -0,0 +1,76 @@
+from dataclasses import replace
+from pathlib import Path
+from unittest.mock import Mock
+
+import pytest
+
+import manim
+from manim import Scene, config, tempconfig
+from manim.scene import scene_file_writer as writer_module
+
+
+def test_scene_resolves_consistent_writer_settings_with_empty_assets_root():
+ with tempconfig(
+ {
+ "format": "none",
+ "assets_dir": "",
+ "max_inflight_encoders": 3,
+ "encoder_queue_size": 5,
+ "max_files_cached": -1,
+ },
+ ):
+ scene = Scene()
+ settings = scene.file_writer_settings
+
+ assert scene.renderer.file_writer.settings is settings
+ assert settings.plan is scene.output_plan
+ assert settings.plan.output is scene.session_spec.output
+ assert settings.video_encoder is scene.session_spec.video_encoder
+ assert settings.max_inflight_encoders == 3
+ assert settings.encoder_queue_size == 5
+ assert settings.max_files_cached == -1
+ assert settings.assets_dir == Path.cwd().absolute()
+
+
+def test_writer_settings_are_private_and_not_exported_from_top_level_manim():
+ assert not hasattr(writer_module, "SceneFileWriterSettings")
+ assert not hasattr(manim, "SceneFileWriterSettings")
+ assert not hasattr(manim, "_SceneFileWriterSettings")
+
+
+def test_writer_settings_reject_plan_output_mismatches():
+ with tempconfig({"format": "mp4"}):
+ settings = Scene().file_writer_settings
+
+ mismatched_plan = replace(settings.plan, segment_extension=".webm")
+ with pytest.raises(ValueError, match="plan segment extension"):
+ replace(settings, plan=mismatched_plan)
+
+ assert settings.video_encoder is not None
+ mismatched_encoder = replace(settings.video_encoder, container_format="webm")
+ with pytest.raises(ValueError, match="encoder container"):
+ replace(settings, video_encoder=mismatched_encoder)
+
+
+def test_writer_uses_captured_assets_root(tmp_path, monkeypatch):
+ captured_assets = tmp_path / "captured"
+ changed_assets = tmp_path / "changed"
+ captured_assets.mkdir()
+ changed_assets.mkdir()
+ sound_path = captured_assets / "tone.wav"
+ sound_path.touch()
+
+ with tempconfig({"format": "none", "assets_dir": captured_assets}):
+ scene = Scene()
+ writer = scene.renderer.file_writer
+ config.assets_dir = changed_assets
+
+ decoded = Mock()
+ from_file = Mock(return_value=decoded)
+ monkeypatch.setattr(writer_module.AudioSegment, "from_file", from_file)
+ writer.add_audio_segment = Mock()
+
+ writer.add_sound("tone")
+
+ from_file.assert_called_once_with(sound_path)
+ writer.add_audio_segment.assert_called_once_with(decoded, None)
diff --git a/tests/module/test_video_encoder.py b/tests/module/test_video_encoder.py
new file mode 100644
index 000000000..063c50866
--- /dev/null
+++ b/tests/module/test_video_encoder.py
@@ -0,0 +1,217 @@
+from __future__ import annotations
+
+from dataclasses import replace
+from fractions import Fraction
+
+import pytest
+
+from manim._config.output import OutputFormat, OutputSpec
+from manim._config.video_encoder import (
+ VideoEncoderSpec,
+ resolve_video_encoder,
+ to_av_frame_rate,
+ video_encoder_fingerprint,
+)
+
+
+def _output(format: OutputFormat, *, transparent: bool = False) -> OutputSpec:
+ return OutputSpec(
+ format=format,
+ transparent=transparent,
+ save_sections=False,
+ fallback_to_still=False,
+ )
+
+
+def _resolve(
+ format=OutputFormat.MP4,
+ *,
+ transparent=False,
+ width=1920,
+ height=1080,
+ frame_rate=30,
+ codec="auto",
+ pixel_format="auto",
+ options=None,
+):
+ return resolve_video_encoder(
+ _output(format, transparent=transparent),
+ width=width,
+ height=height,
+ frame_rate=frame_rate,
+ codec=codec,
+ pixel_format=pixel_format,
+ options=options,
+ )
+
+
+ (
+ "format",
+ "transparent",
+ "container",
+ "codec",
+ "pixel_format",
+ "options",
+ ),
+ [
+ (OutputFormat.MP4, False, "mp4", "libx264", "yuv420p", (("crf", "23"),)),
+ (OutputFormat.MOV, False, "mov", "libx264", "yuv420p", (("crf", "23"),)),
+ (OutputFormat.MOV, True, "mov", "qtrle", "argb", ()),
+ (
+ OutputFormat.WEBM,
+ False,
+ "webm",
+ "libvpx-vp9",
+ "yuv420p",
+ (("crf", "23"),),
+ ),
+ (
+ OutputFormat.WEBM,
+ True,
+ "webm",
+ "libvpx-vp9",
+ "yuva420p",
+ (("crf", "23"),),
+ ),
+ (OutputFormat.GIF, False, "mp4", "libx264", "yuv420p", (("crf", "23"),)),
+ (OutputFormat.GIF, True, "mov", "qtrle", "argb", ()),
+ ],
+)
+def test_default_video_encoder_profiles(
+ format,
+ transparent,
+ container,
+ codec,
+ pixel_format,
+ options,
+):
+ spec = _resolve(format, transparent=transparent, frame_rate=60)
+
+ assert spec == VideoEncoderSpec(
+ container_format=container,
+ codec=codec,
+ pixel_format=pixel_format,
+ width=1920,
+ height=1080,
+ frame_rate=Fraction(60, 1),
+ options=options,
+ )
+
+
+ "format",
+ [OutputFormat.NONE, OutputFormat.PNG, OutputFormat.PNG_SEQUENCE],
+)
+def test_non_video_output_has_no_encoder(format):
+ assert _resolve(format, width=0, height=0, frame_rate=0) is None
+
+
+ ("value", "expected"),
+ [
+ (25, Fraction(25, 1)),
+ (24.0, Fraction(24, 1)),
+ (23.976, Fraction(24_000, 1001)),
+ (23.98, Fraction(24_000, 1001)),
+ (59.94, Fraction(60_000, 1001)),
+ (12.5, Fraction(25, 2)),
+ (Fraction(30000, 1001), Fraction(30000, 1001)),
+ ],
+)
+def test_frame_rate_resolution(value, expected):
+ assert to_av_frame_rate(value) == expected
+
+
[email protected]("value", [True, 0, -1, float("inf"), float("nan"), "60"])
+def test_invalid_frame_rates(value):
+ with pytest.raises(ValueError, match="frame rate"):
+ to_av_frame_rate(value)
+
+
+ ("width", "height"), [(0, 10), (10, -1), (1.5, 10), (True, 10)]
+)
+def test_invalid_video_geometry(width, height):
+ with pytest.raises(ValueError, match="dimensions"):
+ _resolve(width=width, height=height)
+
+
+def test_custom_options_overlay_defaults_and_are_sorted():
+ spec = _resolve(
+ codec="libx264",
+ options={"preset": "slow", "crf": "18"},
+ )
+
+ assert spec is not None
+ assert spec.options == (("crf", "18"), ("preset", "slow"))
+
+
+def test_different_explicit_codec_does_not_inherit_default_options():
+ spec = _resolve(codec="qtrle", pixel_format="argb")
+
+ assert spec is not None
+ assert spec.options == ()
+
+
[email protected]("option", ["codec", "pixel_format", "width", "rate"])
+def test_stream_fields_cannot_be_supplied_as_codec_options(option):
+ with pytest.raises(ValueError, match="explicit stream settings"):
+ _resolve(options={option: "value"})
+
+
+def test_encoder_options_must_be_strings():
+ with pytest.raises(TypeError, match="keys and values must be strings"):
+ _resolve(options={"crf": 18})
+
+
+def test_unknown_encoder_is_rejected():
+ with pytest.raises(ValueError, match="Unknown video encoder"):
+ _resolve(codec="not-a-codec")
+
+
+def test_encoder_pixel_format_mismatch_is_rejected():
+ with pytest.raises(ValueError, match="not supported"):
+ _resolve(codec="qtrle", pixel_format="yuv420p")
+
+
+def test_video_encoder_fingerprint_is_canonical_and_byte_sensitive():
+ spec = VideoEncoderSpec(
+ container_format="mp4",
+ codec="libx264",
+ pixel_format="yuv420p",
+ width=1920,
+ height=1080,
+ frame_rate=Fraction(30, 1),
+ options=(("preset", "slow"), ("crf", "18")),
+ )
+ token = video_encoder_fingerprint(spec)
+
+ assert len(token) == 64
+ assert (
+ video_encoder_fingerprint(
+ replace(spec, options=tuple(reversed(spec.options))),
+ )
+ == token
+ )
+
+ changed_specs = [
+ replace(spec, container_format="mov"),
+ replace(spec, codec="libx265"),
+ replace(spec, pixel_format="yuv444p"),
+ replace(spec, width=1280),
+ replace(spec, height=720),
+ replace(spec, frame_rate=Fraction(60, 1)),
+ replace(spec, options=(("crf", "19"), ("preset", "slow"))),
+ ]
+ assert all(video_encoder_fingerprint(changed) != token for changed in changed_specs)
+ assert video_encoder_fingerprint(None) == "none"
+
+
+def test_transparent_output_requires_alpha_pixel_format():
+ with pytest.raises(ValueError, match="alpha-bearing"):
+ _resolve(
+ OutputFormat.WEBM,
+ transparent=True,
+ pixel_format="yuv420p",
+ )
diff --git a/tests/module/test_video_segment_encoder.py b/tests/module/test_video_segment_encoder.py
new file mode 100644
index 000000000..7cd0435f5
--- /dev/null
+++ b/tests/module/test_video_segment_encoder.py
@@ -0,0 +1,183 @@
+from fractions import Fraction
+from unittest.mock import Mock, call
+
+import av
+import numpy as np
+import pytest
+
+from manim._config.video_encoder import VideoEncoderSpec
+from manim.scene.video_segment_encoder import VideoSegmentEncoder
+
+
+def _spec(*, width=4, height=2):
+ return VideoEncoderSpec(
+ container_format="mp4",
+ codec="libx264",
+ pixel_format="yuv420p",
+ width=width,
+ height=height,
+ frame_rate=Fraction(30, 1),
+ options=(("crf", "23"),),
+ )
+
+
+def _detached_encoder(tmp_path, *, stream=None, container=None):
+ encoder = object.__new__(VideoSegmentEncoder)
+ encoder.target = tmp_path / "segment.mp4"
+ encoder.spec = _spec()
+ encoder._next_pts = 0
+ encoder._closed = False
+ encoder._stream = Mock() if stream is None else stream
+ encoder._container = Mock() if container is None else container
+ return encoder
+
+
+def _frame():
+ return np.full((2, 4, 4), 127, dtype=np.uint8)
+
+
+def test_write_frame_owns_repeat_and_segment_local_pts(tmp_path):
+ packet = object()
+ encoded_frames = []
+ stream = Mock()
+
+ def encode(frame):
+ encoded_frames.append(frame)
+ return [packet]
+
+ stream.encode.side_effect = encode
+ container = Mock()
+ encoder = _detached_encoder(tmp_path, stream=stream, container=container)
+
+ encoder.write_frame(_frame(), repeat=3)
+
+ assert [frame.pts for frame in encoded_frames] == [0, 1, 2]
+ assert [frame.time_base for frame in encoded_frames] == [Fraction(1, 30)] * 3
+ assert encoder._next_pts == 3
+ assert container.mux.call_args_list == [call(packet)] * 3
+
+
+ ("frame", "repeat", "exception", "message"),
+ [
+ (np.zeros((3, 4, 4), dtype=np.uint8), 1, ValueError, "shape"),
+ (np.zeros((2, 4, 3), dtype=np.uint8), 1, ValueError, "shape"),
+ (np.zeros((2, 4, 4), dtype=np.float32), 1, TypeError, "uint8"),
+ (np.zeros((2, 4, 4), dtype=np.uint8)[:, ::-1], 1, ValueError, "C-contiguous"),
+ (np.zeros((2, 4, 4), dtype=np.uint8), 0, ValueError, "positive integer"),
+ (np.zeros((2, 4, 4), dtype=np.uint8), True, ValueError, "positive integer"),
+ ],
+)
+def test_write_frame_validates_boundary(
+ tmp_path,
+ frame,
+ repeat,
+ exception,
+ message,
+):
+ encoder = _detached_encoder(tmp_path)
+
+ with pytest.raises(exception, match=message):
+ encoder.write_frame(frame, repeat=repeat)
+
+ encoder._stream.encode.assert_not_called()
+
+
+def test_write_frame_rejects_closed_encoder(tmp_path):
+ encoder = _detached_encoder(tmp_path)
+ encoder._closed = True
+
+ with pytest.raises(RuntimeError, match="is closed"):
+ encoder.write_frame(_frame())
+
+
+def test_encode_failure_has_target_profile_and_original_cause(tmp_path):
+ expected_exception = RuntimeError("codec exploded")
+ stream = Mock()
+ stream.encode.side_effect = expected_exception
+ encoder = _detached_encoder(tmp_path, stream=stream)
+
+ with pytest.raises(
+ RuntimeError, match=r"segment\.mp4.*mp4/libx264/yuv420p"
+ ) as exc_info:
+ encoder.write_frame(_frame())
+
+ assert exc_info.value.__cause__ is expected_exception
+
+
+def test_finish_flushes_closes_and_is_idempotent(tmp_path):
+ packet = object()
+ stream = Mock()
+ stream.encode.return_value = [packet]
+ container = Mock()
+ encoder = _detached_encoder(tmp_path, stream=stream, container=container)
+
+ encoder.finish()
+ encoder.finish()
+
+ stream.encode.assert_called_once_with()
+ container.mux.assert_called_once_with(packet)
+ container.close.assert_called_once_with()
+
+
+def test_finish_preserves_flush_failure_when_close_also_fails(tmp_path):
+ flush_exception = RuntimeError("flush failed")
+ close_exception = RuntimeError("close failed")
+ stream = Mock()
+ stream.encode.side_effect = flush_exception
+ container = Mock()
+ container.close.side_effect = close_exception
+ encoder = _detached_encoder(tmp_path, stream=stream, container=container)
+
+ with pytest.raises(RuntimeError, match="Failed to finish") as exc_info:
+ encoder.finish()
+
+ assert exc_info.value.__cause__ is flush_exception
+ container.close.assert_called_once_with()
+
+
+def test_finish_closes_before_propagating_control_flow_exception(tmp_path):
+ stream = Mock()
+ stream.encode.side_effect = KeyboardInterrupt
+ container = Mock()
+ encoder = _detached_encoder(tmp_path, stream=stream, container=container)
+
+ with pytest.raises(KeyboardInterrupt):
+ encoder.finish()
+
+ container.close.assert_called_once_with()
+
+
+def test_abort_closes_removes_target_and_is_idempotent(tmp_path):
+ encoder = _detached_encoder(tmp_path)
+ encoder.target.write_bytes(b"incomplete")
+
+ encoder.abort()
+ encoder.abort()
+
+ encoder._container.close.assert_called_once_with()
+ assert not encoder.target.exists()
+
+
+def test_abort_removes_target_before_propagating_control_flow_exception(tmp_path):
+ container = Mock()
+ container.close.side_effect = KeyboardInterrupt
+ encoder = _detached_encoder(tmp_path, container=container)
+ encoder.target.write_bytes(b"incomplete")
+
+ with pytest.raises(KeyboardInterrupt):
+ encoder.abort()
+
+ assert not encoder.target.exists()
+
+
+def test_open_failure_has_target_profile_and_original_cause(tmp_path, monkeypatch):
+ expected_exception = RuntimeError("open failed")
+ monkeypatch.setattr(av, "open", Mock(side_effect=expected_exception))
+
+ with pytest.raises(
+ RuntimeError, match=r"segment\.mp4.*mp4/libx264/yuv420p"
+ ) as exc_info:
+ VideoSegmentEncoder(target=tmp_path / "segment.mp4", spec=_spec())
+
+ assert exc_info.value.__cause__ is expected_exception
diff --git a/tests/module/utils/test_caching.py b/tests/module/utils/test_caching.py
new file mode 100644
index 000000000..8f73a6178
--- /dev/null
+++ b/tests/module/utils/test_caching.py
@@ -0,0 +1,181 @@
+from __future__ import annotations
+
+import os
+from unittest.mock import Mock, call
+
+import numpy as np
+
+from manim import tempconfig
+from manim.utils import caching
+from manim.utils.caching import (
+ clear_segment_cache,
+ handle_caching_play,
+ prune_segment_cache,
+)
+
+
+def test_prune_segment_cache_ignores_non_segments_and_hidden_files(tmp_path):
+ segment = tmp_path / "00001.mp4"
+ segment.touch()
+ ignored = [
+ tmp_path / ".DS_Store",
+ tmp_path / "._00001.mp4",
+ tmp_path / "partial_movie_file_list.txt",
+ tmp_path / "notes.txt",
+ ]
+ for path in ignored:
+ path.touch()
+
+ prune_segment_cache(tmp_path, 1)
+ assert segment.exists()
+
+ prune_segment_cache(tmp_path, 0)
+ assert not segment.exists()
+ assert all(path.exists() for path in ignored)
+
+
+def test_prune_segment_cache_removes_least_recently_accessed(tmp_path):
+ oldest = tmp_path / "oldest.mp4"
+ newest = tmp_path / "newest.mp4"
+ oldest.touch()
+ newest.touch()
+ os.utime(oldest, (1, 1))
+ os.utime(newest, (2, 2))
+
+ prune_segment_cache(tmp_path, 1)
+
+ assert not oldest.exists()
+ assert newest.exists()
+
+
+def test_prune_segment_cache_treats_minus_one_as_unlimited(tmp_path):
+ segments = [tmp_path / f"{index}.mp4" for index in range(3)]
+ for segment in segments:
+ segment.touch()
+
+ prune_segment_cache(tmp_path, -1)
+
+ assert all(segment.exists() for segment in segments)
+
+
+def test_prune_segment_cache_tolerates_vanishing_files(tmp_path, monkeypatch):
+ survivor = tmp_path / "00001.mp4"
+ survivor.touch()
+ ghost = tmp_path / "00002.mp4"
+ monkeypatch.setattr(
+ caching,
+ "_segment_cache_files",
+ lambda directory: [survivor, ghost],
+ )
+
+ prune_segment_cache(tmp_path, 0)
+
+ assert not survivor.exists()
+
+
+def test_prune_segment_cache_does_not_over_evict_for_vanished_file(
+ tmp_path,
+ monkeypatch,
+):
+ survivors = [tmp_path / f"{index:05}.mp4" for index in range(2)]
+ for survivor in survivors:
+ survivor.touch()
+ ghost = tmp_path / "00002.mp4"
+ monkeypatch.setattr(
+ caching,
+ "_segment_cache_files",
+ lambda directory: [*survivors, ghost],
+ )
+
+ prune_segment_cache(tmp_path, len(survivors))
+
+ assert all(survivor.exists() for survivor in survivors)
+
+
+def test_clear_segment_cache_removes_only_recognized_segments(tmp_path):
+ segments = [tmp_path / name for name in ("one.mp4", "two.mov", "three.webm")]
+ ignored = [
+ tmp_path / ".hidden.mp4",
+ tmp_path / "partial_movie_file_list.txt",
+ tmp_path / "unrelated.mkv",
+ ]
+ for path in [*segments, *ignored]:
+ path.touch()
+
+ assert clear_segment_cache(tmp_path) == 3
+ assert all(not segment.exists() for segment in segments)
+ assert all(path.exists() for path in ignored)
+
+
+def test_clear_segment_cache_accepts_missing_directory(tmp_path):
+ assert clear_segment_cache(tmp_path / "missing") == 0
+
+
+def test_opengl_cache_inputs_and_per_play_policy(monkeypatch):
+ encoder = object()
+ fingerprint = Mock(return_value="encoder-token")
+ hash_play = Mock(return_value="cache-key")
+ monkeypatch.setattr(caching, "video_encoder_fingerprint", fingerprint)
+ monkeypatch.setattr(caching, "get_hash_from_play_call", hash_play)
+
+ class FakeScene:
+ def __init__(self):
+ self.mobjects = [object()]
+ self.meshes = [object()]
+ self.session_spec = Mock(video_encoder=encoder)
+
+ def compile_animations(self, *args, **kwargs):
+ return []
+
+ def add_mobjects_from_animations(self, animations):
+ pass
+
+ class FakeRenderer:
+ _original_skipping_status = False
+ skip_animations = False
+ num_plays = 0
+ animations_hashes = []
+ camera = object()
+ background_color = np.array([0.1, 0.2, 0.3, 1.0])
+ anti_alias_width = 1.5
+ file_writer = Mock()
+
+ def update_skipping_status(self):
+ pass
+
+ @handle_caching_play
+ def play(self, scene, *args, **kwargs):
+ self.num_plays += 1
+
+ scene = FakeScene()
+ renderer = FakeRenderer()
+ renderer.animations_hashes = []
+ renderer.file_writer.is_already_cached.side_effect = [False, True]
+
+ with tempconfig({"disable_caching": False}):
+ renderer.play(scene)
+ with tempconfig({"disable_caching": True}):
+ renderer.play(scene)
+ renderer.play(scene)
+
+ assert fingerprint.call_args_list == [call(encoder), call(encoder)]
+ assert hash_play.call_count == 2
+ assert hash_play.call_args.kwargs == {
+ "backend": "opengl",
+ "encoder_fingerprint": "encoder-token",
+ "renderer_state": {
+ "meshes": scene.meshes,
+ "background_color": renderer.background_color,
+ "anti_alias_width": renderer.anti_alias_width,
+ },
+ }
+ assert renderer.animations_hashes == [
+ "cache-key",
+ "uncached_00001",
+ "cache-key",
+ ]
+ assert renderer.file_writer.add_partial_movie_file.call_args_list == [
+ call("cache-key"),
+ call("uncached_00001"),
+ call("cache-key"),
+ ]
diff --git a/tests/module/utils/test_hashing.py b/tests/module/utils/test_hashing.py
index 788d86a78..3f63ff053 100644
--- a/tests/module/utils/test_hashing.py
+++ b/tests/module/utils/test_hashing.py
@@ -11,6 +11,15 @@ import manim.utils.hashing as hashing
from manim import ImageMobject, Square
ALREADY_PROCESSED_PLACEHOLDER = hashing._Memoizer.ALREADY_PROCESSED_PLACEHOLDER
+_CACHE_IDENTITY = {
+ "backend": "cairo",
+ "encoder_fingerprint": "encoder-token",
+ "renderer_state": (),
+}
+
+
+def _play_hash(*args):
+ return hashing.get_hash_from_play_call(*args, **_CACHE_IDENTITY)
def test_JSON_basic():
@@ -254,13 +263,13 @@ def test_play_hash_includes_mobject_pixels_but_not_camera_pixels():
camera = HashableObject("camera", np.arange(8, dtype=np.uint8))
mobject = ImageMobject(np.zeros((8, 8, 4), dtype=np.uint8))
- original = hashing.get_hash_from_play_call(scene, camera, [], [mobject])
+ original = _play_hash(scene, camera, [], [mobject])
mobject.pixel_array[4, 4, 0] ^= 1
- assert hashing.get_hash_from_play_call(scene, camera, [], [mobject]) != original
+ assert _play_hash(scene, camera, [], [mobject]) != original
mobject.pixel_array[4, 4, 0] ^= 1
camera.pixel_array[0] ^= 1
- assert hashing.get_hash_from_play_call(scene, camera, [], [mobject]) == original
+ assert _play_hash(scene, camera, [], [mobject]) == original
def test_play_hash_keeps_distinct_mobjects_with_equal_python_hashes():
@@ -278,9 +287,51 @@ def test_play_hash_keeps_distinct_mobjects_with_equal_python_hashes():
camera = CollidingObject("camera")
mobjects = [CollidingObject("first"), CollidingObject("second")]
- original = hashing.get_hash_from_play_call(scene, camera, [], mobjects)
+ original = _play_hash(scene, camera, [], mobjects)
mobjects[1].name = "changed"
- assert hashing.get_hash_from_play_call(scene, camera, [], mobjects) != original
+ assert _play_hash(scene, camera, [], mobjects) != original
+
+
+def test_play_hash_includes_backend_encoder_and_renderer_state():
+ class HashableObject:
+ def __init__(self, value):
+ self.value = value
+
+ scene = HashableObject("scene")
+ camera = HashableObject("camera")
+
+ def cache_key(*, backend="cairo", encoder="encoder", **renderer_state):
+ return hashing.get_hash_from_play_call(
+ scene,
+ camera,
+ [],
+ [],
+ backend=backend,
+ encoder_fingerprint=encoder,
+ renderer_state=renderer_state,
+ )
+
+ renderer_state = {
+ "meshes": [HashableObject("mesh")],
+ "background_color": np.array([0.0, 0.0, 0.0, 1.0]),
+ "anti_alias_width": 1.5,
+ }
+ original = cache_key(**renderer_state)
+ changed_inputs = [
+ cache_key(backend="opengl", **renderer_state),
+ cache_key(encoder="other", **renderer_state),
+ cache_key(**{**renderer_state, "meshes": [HashableObject("changed")]}),
+ cache_key(
+ **{
+ **renderer_state,
+ "background_color": np.array([1.0, 0.0, 0.0, 1.0]),
+ },
+ ),
+ cache_key(**{**renderer_state, "anti_alias_width": 2.0}),
+ ]
+
+ assert len(original) == 64
+ assert all(changed != original for changed in changed_inputs)
def test_JSON_with_tuple():
diff --git a/tests/test_config.py b/tests/test_config.py
index aab4270dc..ec8f967ae 100644
--- a/tests/test_config.py
+++ b/tests/test_config.py
@@ -54,6 +54,14 @@ def test_tempconfig(config):
assert config[k] == v
+def test_max_files_cached_uses_minus_one_for_unlimited(config):
+ config.max_files_cached = -1
+ assert config.max_files_cached == -1
+
+ with pytest.raises(ValueError, match="non-negative integer or -1"):
+ config.max_files_cached = float("inf")
+
+
def test_tempconfig_restores_renderer_class_bases(config):
with tempconfig({"renderer": "opengl"}):
assert config.renderer == RendererType.OPENGL
@@ -224,6 +232,143 @@ def test_format_is_loaded_from_config_file(tmp_path, config):
assert config.format == "png-sequence"
+def test_render_session_resolves_configured_encoder_profile(config):
+ config.format = "mov"
+ config.video_codec = "qtrle"
+ config.pixel_format = "argb"
+ config.video_encoder_options = {"threads": "1"}
+
+ encoder = _resolve_session(config).video_encoder
+
+ assert encoder is not None
+ assert encoder.codec == "qtrle"
+ assert encoder.pixel_format == "argb"
+ assert encoder.options == (("threads", "1"),)
+
+
+def test_video_encoder_profile_is_loaded_from_config_file(tmp_path, config):
+ config_file = tmp_path / "encoder.cfg"
+ config_file.write_text(
+ """\
+[video_encoder]
+codec = libx264
+pixel_format = yuv444p
+
+[video_encoder.options]
+crf = 18
+preset = veryslow
+tune = animation, film: grain
+""",
+ )
+
+ config.digest_file(config_file)
+
+ assert config.video_codec == "libx264"
+ assert config.pixel_format == "yuv444p"
+ assert config.video_encoder_options == {
+ "crf": "18",
+ "preset": "veryslow",
+ "tune": "animation, film: grain",
+ }
+
+
+def test_media_loglevel_is_loaded_from_media_section(tmp_path, config):
+ config_file = tmp_path / "media.cfg"
+ config_file.write_text("[media]\nloglevel = DEBUG\n")
+
+ config.digest_file(config_file)
+
+ assert config.media_loglevel == "DEBUG"
+ with pytest.raises(AttributeError):
+ _ = config.ffmpeg_loglevel
+
+
+def test_video_encoder_options_are_copied(config):
+ supplied = {"crf": "18"}
+ config.video_encoder_options = supplied
+ supplied["crf"] = "30"
+ assert config.video_encoder_options == {"crf": "18"}
+
+ returned = config.video_encoder_options
+ returned["crf"] = "40"
+ assert config.video_encoder_options == {"crf": "18"}
+
+
+def test_encoder_cli_options_replace_config_file_map(tmp_path, config):
+ scene_file = tmp_path / "trivial_scene.py"
+ scene_file.write_text("# --jupyter returns before importing this file\n")
+ config_file = tmp_path / "encoder.cfg"
+ config_file.write_text(
+ """\
+[video_encoder]
+codec = qtrle
+pixel_format = argb
+
+[video_encoder.options]
+predictor = 1
+""",
+ )
+ runner = CliRunner()
+ common_args = [
+ str(scene_file),
+ "--jupyter",
+ "--config_file",
+ str(config_file),
+ ]
+
+ result = runner.invoke(
+ render,
+ [
+ *common_args,
+ "--video-codec",
+ "libx264",
+ "--pixel-format",
+ "yuv420p",
+ "--encoder-option",
+ "crf=18",
+ "--encoder-option",
+ "tune=animation=film",
+ ],
+ standalone_mode=False,
+ )
+ assert result.exit_code == 0, result.output
+ with tempconfig({}):
+ config.digest_args(result.return_value)
+ assert config.video_codec == "libx264"
+ assert config.pixel_format == "yuv420p"
+ assert config.video_encoder_options == {
+ "crf": "18",
+ "tune": "animation=film",
+ }
+
+ result = runner.invoke(render, common_args, standalone_mode=False)
+ assert result.exit_code == 0, result.output
+ with tempconfig({}):
+ config.digest_args(result.return_value)
+ assert config.video_codec == "qtrle"
+ assert config.pixel_format == "argb"
+ assert config.video_encoder_options == {"predictor": "1"}
+
+
+ "options",
+ [
+ ["--encoder-option", "missing-separator"],
+ ["--encoder-option", "=missing-key"],
+ ["--encoder-option", "missing-value="],
+ ["--encoder-option", "crf=18", "--encoder-option", "crf=20"],
+ ],
+)
+def test_encoder_cli_options_reject_malformed_or_duplicate_entries(tmp_path, options):
+ scene_file = tmp_path / "trivial_scene.py"
+ scene_file.touch()
+
+ result = CliRunner().invoke(render, [str(scene_file), *options])
+
+ assert result.exit_code == 2
+ assert "Invalid value" in result.output
+
+
def test_cli_distinguishes_preview_from_live_preview(tmp_path):
scene_file = tmp_path / "scene.py"
scene_file.write_text("# --jupyter returns before loading this file\n")
diff --git a/tests/test_scene_rendering/opengl/test_opengl_renderer.py b/tests/test_scene_rendering/opengl/test_opengl_renderer.py
index b0b9cded6..c2925b737 100644
--- a/tests/test_scene_rendering/opengl/test_opengl_renderer.py
+++ b/tests/test_scene_rendering/opengl/test_opengl_renderer.py
@@ -72,6 +72,8 @@ def test_get_frame_with_live_preview_disabled(config, using_opengl_renderer):
# height and width are flipped
assert renderer.get_pixel_shape()[0] == frame.shape[1]
assert renderer.get_pixel_shape()[1] == frame.shape[0]
+ assert frame.dtype == np.uint8
+ assert frame.flags.c_contiguous
@pytest.mark.slow
@@ -92,9 +94,26 @@ def test_get_frame_with_live_preview_enabled(config, using_opengl_renderer):
# height and width are flipped
assert renderer.get_pixel_shape()[0] == frame.shape[1]
assert renderer.get_pixel_shape()[1] == frame.shape[0]
+ assert frame.dtype == np.uint8
+ assert frame.flags.c_contiguous
renderer.window.close()
+def test_render_without_frame_output_skips_gpu_readback(
+ config,
+ using_opengl_renderer,
+):
+ config.format = "none"
+ config.live_preview = False
+ scene = SquareToCircle()
+ renderer = scene.renderer
+ renderer.get_frame = Mock(wraps=renderer.get_frame)
+
+ renderer.render(scene, 0, [])
+
+ renderer.get_frame.assert_not_called()
+
+
def test_pixel_coords_to_space_coords(config, using_opengl_renderer):
config.live_preview = False
diff --git a/tests/test_scene_rendering/test_cairo_renderer.py b/tests/test_scene_rendering/test_cairo_renderer.py
index 4a5229e96..0a2f38d28 100644
--- a/tests/test_scene_rendering/test_cairo_renderer.py
+++ b/tests/test_scene_rendering/test_cairo_renderer.py
@@ -105,3 +105,5 @@ def test_hash_logic_is_called_when_caching_is_enabled(using_temp_config):
scene = SquareToCircle()
scene.render()
mocked.assert_called_once()
+ assert mocked.call_args.kwargs["backend"] == "cairo"
+ assert mocked.call_args.kwargs["encoder_fingerprint"] != "none"
diff --git a/tests/test_scene_rendering/test_file_writer.py b/tests/test_scene_rendering/test_file_writer.py
index c44bbbed7..413f668a1 100644
--- a/tests/test_scene_rendering/test_file_writer.py
+++ b/tests/test_scene_rendering/test_file_writer.py
@@ -1,22 +1,11 @@
import sys
-from fractions import Fraction
from pathlib import Path
-from unittest.mock import Mock
import av
import numpy as np
import pytest
from manim import DR, Circle, Create, Scene, Star, tempconfig
-from manim._config import config
-from manim._config.output import OutputFormat, OutputSpec
-from manim._config.output_plan import (
- resolve_media_layout,
- resolve_module_name,
- resolve_output_plan,
- resolve_requested_output_name,
-)
-from manim.scene.scene_file_writer import SceneFileWriter, to_av_frame_rate
from manim.utils.commands import capture, get_video_metadata
@@ -186,129 +175,3 @@ def test_unicode_partial_movie(config, tmpdir, simple_scenes_path):
_, err, exit_code = capture(command)
assert exit_code == 0, err
-
-
-def test_frame_rates():
- assert to_av_frame_rate(25) == Fraction(25, 1)
- assert to_av_frame_rate(24.0) == Fraction(24, 1)
- assert to_av_frame_rate(23.976) == Fraction(24 * 1000, 1001)
- assert to_av_frame_rate(23.98) == Fraction(24 * 1000, 1001)
- assert to_av_frame_rate(59.94) == Fraction(60 * 1000, 1001)
-
-
-def _new_file_writer(scene_name: str) -> SceneFileWriter:
- renderer = Mock()
- renderer.num_plays = 0
- output = OutputSpec(
- OutputFormat.MP4,
- transparent=False,
- save_sections=False,
- fallback_to_still=False,
- )
- module_name = resolve_module_name(config)
- layout = resolve_media_layout(
- config,
- output,
- module_name=module_name,
- scene_name=scene_name,
- working_directory=Path.cwd(),
- )
- plan = resolve_output_plan(
- layout,
- output,
- scene_name=scene_name,
- requested_output_name=resolve_requested_output_name(config),
- )
- return SceneFileWriter(renderer, scene_name, output, plan)
-
-
-def test_clean_cache_ignores_hidden_files(config, tmp_path):
- # macOS leaves resource forks (._*.mp4) and .DS_Store files in the
- # partial movie directory; they must not be counted against
- # max_files_cached nor be deleted, see issue #3234.
- with tempconfig({"media_dir": tmp_path, "format": "mp4"}):
- writer = _new_file_writer("CacheCleaningScene")
- cache_dir = writer.partial_movie_directory
- cache_dir.mkdir(parents=True)
-
- for name in ["00001.mp4", ".DS_Store", "._00001.mp4"]:
- (cache_dir / name).touch()
-
- config.max_files_cached = 1
- # The hidden files must not count towards the limit: with a single
- # real partial movie file cached, nothing must be evicted.
- writer.clean_cache()
-
- assert (cache_dir / "00001.mp4").exists()
- assert (cache_dir / ".DS_Store").exists()
- assert (cache_dir / "._00001.mp4").exists()
-
- config.max_files_cached = 0
- writer.clean_cache()
-
- assert not (cache_dir / "00001.mp4").exists()
- assert (cache_dir / ".DS_Store").exists()
- assert (cache_dir / "._00001.mp4").exists()
-
-
-def test_flush_cache_directory_ignores_hidden_files(config, tmp_path):
- with tempconfig({"media_dir": tmp_path, "format": "mp4"}):
- writer = _new_file_writer("CacheFlushingScene")
- cache_dir = writer.partial_movie_directory
- cache_dir.mkdir(parents=True)
-
- for name in ["00001.mp4", "00002.mp4", ".DS_Store", "._00001.mp4"]:
- (cache_dir / name).touch()
- (cache_dir / "partial_movie_file_list.txt").touch()
-
- writer.flush_cache_directory()
-
- assert not (cache_dir / "00001.mp4").exists()
- assert not (cache_dir / "00002.mp4").exists()
- assert (cache_dir / "partial_movie_file_list.txt").exists()
- assert (cache_dir / ".DS_Store").exists()
- assert (cache_dir / "._00001.mp4").exists()
-
-
-def test_clean_cache_tolerates_vanishing_files(config, tmp_path, monkeypatch):
- # A file can disappear between listing the directory and unlinking it
- # (e.g. Finder removing a transient resource fork); clean_cache must
- # not raise FileNotFoundError in that case.
- with tempconfig({"media_dir": tmp_path, "format": "mp4"}):
- writer = _new_file_writer("VanishingFileScene")
- cache_dir = writer.partial_movie_directory
- cache_dir.mkdir(parents=True)
-
- survivor = cache_dir / "00001.mp4"
- survivor.touch()
- ghost = cache_dir / "00002.mp4"
- monkeypatch.setattr(
- writer, "_cached_partial_movie_files", lambda: [survivor, ghost]
- )
-
- config.max_files_cached = 0
- writer.clean_cache()
-
- assert not survivor.exists()
-
-
-def test_clean_cache_does_not_evict_for_vanished_file(config, tmp_path, monkeypatch):
- with tempconfig({"media_dir": tmp_path, "format": "mp4"}):
- writer = _new_file_writer("VanishedFileEvictionScene")
- cache_dir = writer.partial_movie_directory
- cache_dir.mkdir(parents=True)
-
- survivors = [cache_dir / f"{index:05}.mp4" for index in range(2)]
- for survivor in survivors:
- survivor.touch()
- ghost = cache_dir / "00002.mp4"
- monkeypatch.setattr(
- writer,
- "_cached_partial_movie_files",
- lambda: [*survivors, ghost],
- )
-
- config.max_files_cached = len(survivors)
- writer.clean_cache()
-
- assert all(survivor.exists() for survivor in survivors)
diff --git a/tests/test_scene_rendering/test_parallel_encoding.py b/tests/test_scene_rendering/test_parallel_encoding.py
index acf0347dc..a3eb47ce2 100644
--- a/tests/test_scene_rendering/test_parallel_encoding.py
+++ b/tests/test_scene_rendering/test_parallel_encoding.py
@@ -6,7 +6,7 @@ import textwrap
import threading
import time
from pathlib import Path
-from unittest.mock import ANY, Mock, call
+from unittest.mock import ANY, Mock
import av
import numpy as np
@@ -22,8 +22,9 @@ from manim._config.output_plan import (
resolve_output_plan,
resolve_requested_output_name,
)
+from manim._config.video_encoder import resolve_video_encoder
from manim.cli.render.commands import render
-from manim.scene.scene_file_writer import SceneFileWriter
+from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings
from manim.utils.exceptions import RerunSceneException
_ENCODER_THREAD_PREFIX = "partial-movie-encoder-"
@@ -61,8 +62,20 @@ def _make_writer(
scene_name=scene_name,
requested_output_name=resolve_requested_output_name(config),
)
- renderer = Mock(num_plays=0)
- return SceneFileWriter(renderer, scene_name, output, plan)
+ settings = _SceneFileWriterSettings(
+ plan=plan,
+ video_encoder=resolve_video_encoder(
+ output,
+ width=config.pixel_width,
+ height=config.pixel_height,
+ frame_rate=config.frame_rate,
+ ),
+ max_inflight_encoders=config.max_inflight_encoders,
+ encoder_queue_size=config.encoder_queue_size,
+ max_files_cached=config.max_files_cached,
+ assets_dir=Path.cwd(),
+ )
+ return SceneFileWriter(settings)
_SCENE_NAME = "ParallelEncodingCacheScene"
@@ -143,6 +156,39 @@ def test_parallel_encoding_cache_behavior(tmp_path):
assert second_snapshot == partial_snapshot
+def test_disable_caching_is_evaluated_per_play(config, tmp_path):
+ class DynamicCachePolicyScene(Scene):
+ def construct(self):
+ self.play(FadeIn(Square()), run_time=0.1)
+ self.clear()
+ with tempconfig({"disable_caching": True}):
+ self.play(FadeIn(Square()), run_time=0.1)
+ self.clear()
+ self.play(FadeIn(Square()), run_time=0.1)
+ self.clear()
+ self.play(FadeIn(Square()), run_time=0.1)
+
+ with tempconfig(
+ {
+ "format": "mp4",
+ "media_dir": tmp_path,
+ "quality": "low_quality",
+ "disable_caching": False,
+ },
+ ):
+ scene = DynamicCachePolicyScene()
+ scene.render()
+
+ initial_hash, uncached_hash, restored_hash, cached_hash = (
+ scene.renderer.animations_hashes
+ )
+ assert initial_hash is not None
+ assert uncached_hash == "uncached_00001"
+ assert restored_hash is not None
+ assert cached_hash == restored_hash
+
+
@pytest.mark.slow
def test_no_encoder_threads_survive_render(config, tmp_path):
class ThreadSweepScene(Scene):
@@ -170,26 +216,23 @@ def _frame():
return np.zeros((4, 4, 4), dtype=np.uint8)
-def _new_encode_job(
- tmp_path,
- monkeypatch,
- name,
- stream,
- container,
- frame_queue_size=8,
-):
+def _fake_segment_encoder(tmp_path, name):
+ target = tmp_path / f"{name}.mp4"
+ encoder = Mock(target=target)
+ encoder.abort.side_effect = lambda: target.unlink(missing_ok=True)
+ return encoder
+
+
+def _new_encode_job(tmp_path, name, encoder=None, frame_queue_size=8):
from manim.scene.scene_file_writer import _PartialMovieEncodeJob
- job = _PartialMovieEncodeJob(
- path=tmp_path / f"{name}.mp4",
+ if encoder is None:
+ encoder = _fake_segment_encoder(tmp_path, name)
+ return _PartialMovieEncodeJob(
animation_index=0,
- stream=Mock(),
- container=Mock(),
+ encoder=encoder,
frame_queue_size=frame_queue_size,
)
- monkeypatch.setattr(job, "stream", stream)
- monkeypatch.setattr(job, "container", container)
- return job
def _assert_failed_join(job, expected_exception):
@@ -205,22 +248,18 @@ def _assert_failed_join(job, expected_exception):
def test_encode_failure_propagates_and_drains_bounded_queue(
tmp_path,
- monkeypatch,
manim_caplog,
):
expected_exception = RuntimeError("encode failed")
encode_failed = threading.Event()
- stream = Mock()
- container = Mock()
+ encoder = _fake_segment_encoder(tmp_path, "encode_failure")
- def encode(*args):
- if args:
- encode_failed.set()
- raise expected_exception
- return []
+ def fail_write(*args, **kwargs):
+ encode_failed.set()
+ raise expected_exception
- stream.encode.side_effect = encode
- job = _new_encode_job(tmp_path, monkeypatch, "encode_failure", stream, container)
+ encoder.write_frame.side_effect = fail_write
+ job = _new_encode_job(tmp_path, "encode_failure", encoder)
job.put(1, _frame())
assert encode_failed.wait(timeout=2), "Encode failure was not triggered"
@@ -235,155 +274,61 @@ def test_encode_failure_propagates_and_drains_bounded_queue(
assert not producer.is_alive(), "Producer deadlocked on the bounded queue"
_assert_failed_join(job, expected_exception)
- container.close.assert_called_once_with()
- assert "Partial movie file written" not in manim_caplog.text
-
-
-def test_flush_failure_propagates_and_closes_container(
- tmp_path,
- monkeypatch,
- manim_caplog,
-):
- expected_exception = RuntimeError("flush failed")
- stream = Mock()
- container = Mock()
-
- def encode(*args):
- if args:
- return []
- raise expected_exception
-
- stream.encode.side_effect = encode
- job = _new_encode_job(tmp_path, monkeypatch, "flush_failure", stream, container)
- job.put(1, _frame())
- job.seal()
-
- _assert_failed_join(job, expected_exception)
- container.close.assert_called_once_with()
+ encoder.abort.assert_called_once_with()
assert "Partial movie file written" not in manim_caplog.text
-def test_close_failure_propagates_after_close_attempt(
- tmp_path,
- monkeypatch,
- manim_caplog,
-):
- expected_exception = RuntimeError("close failed")
- stream = Mock()
- stream.encode.return_value = []
- container = Mock()
- container.close.side_effect = expected_exception
- job = _new_encode_job(tmp_path, monkeypatch, "close_failure", stream, container)
- job.put(1, _frame())
- job.seal()
-
- _assert_failed_join(job, expected_exception)
- container.close.assert_called_once_with()
- assert "Partial movie file written" not in manim_caplog.text
-
-
-def test_encode_failure_precedes_close_failure_and_removes_partial(
- tmp_path,
- monkeypatch,
- manim_caplog,
-):
- expected_exception = RuntimeError("encode failed")
- close_exception = RuntimeError("close failed")
- stream = Mock()
- container = Mock()
-
- def encode(*args):
- if args:
- raise expected_exception
- return []
-
- stream.encode.side_effect = encode
- container.close.side_effect = close_exception
- job = _new_encode_job(
- tmp_path,
- monkeypatch,
- "encode_and_close_failure",
- stream,
- container,
- )
- job.path.write_bytes(b"stale")
+def test_finish_failure_aborts_segment(tmp_path, manim_caplog):
+ expected_exception = RuntimeError("finish failed")
+ encoder = _fake_segment_encoder(tmp_path, "finish_failure")
+ encoder.finish.side_effect = expected_exception
+ job = _new_encode_job(tmp_path, "finish_failure", encoder)
job.put(1, _frame())
job.seal()
_assert_failed_join(job, expected_exception)
- container.close.assert_called_once_with()
- assert not job.path.exists()
+ encoder.abort.assert_called_once_with()
assert "Partial movie file written" not in manim_caplog.text
-def test_partial_cleanup_failure_does_not_mask_encode_failure(
- tmp_path,
- monkeypatch,
- manim_caplog,
-):
+def test_encode_failure_precedes_abort_failure(tmp_path, manim_caplog):
expected_exception = RuntimeError("encode failed")
- cleanup_exception = PermissionError("cannot remove partial")
- stream = Mock()
- container = Mock()
-
- def encode(*args):
- if args:
- raise expected_exception
- return []
-
- stream.encode.side_effect = encode
- job = _new_encode_job(tmp_path, monkeypatch, "cleanup_failure", stream, container)
- job.path.write_bytes(b"stale")
+ abort_exception = RuntimeError("abort failed")
+ encoder = _fake_segment_encoder(tmp_path, "encode_and_abort_failure")
+ encoder.write_frame.side_effect = expected_exception
+ encoder.abort.side_effect = abort_exception
+ job = _new_encode_job(tmp_path, "encode_and_abort_failure", encoder)
job.put(1, _frame())
job.seal()
- unlink = Mock(side_effect=cleanup_exception)
- monkeypatch.setattr(Path, "unlink", unlink)
_assert_failed_join(job, expected_exception)
-
- unlink.assert_called_once_with(missing_ok=True)
- assert "Failed to remove incomplete partial movie file" in manim_caplog.text
- assert "cannot remove partial" in manim_caplog.text
+ assert "Failed to clean up incomplete segment" in manim_caplog.text
+ assert "abort failed" in manim_caplog.text
assert "Partial movie file written" not in manim_caplog.text
-def test_successful_encode_job_logs_partial_movie_written(
- tmp_path,
- monkeypatch,
- manim_caplog,
-):
- stream = Mock()
- stream.encode.return_value = []
- container = Mock()
- job = _new_encode_job(tmp_path, monkeypatch, "encode_success", stream, container)
+def test_successful_encode_job_logs_partial_movie_written(tmp_path, manim_caplog):
+ encoder = _fake_segment_encoder(tmp_path, "encode_success")
+ job = _new_encode_job(tmp_path, "encode_success", encoder)
job.put(1, _frame())
job.seal()
job.join()
- container.close.assert_called_once_with()
+ encoder.write_frame.assert_called_once_with(ANY, repeat=1)
+ encoder.finish.assert_called_once_with()
+ encoder.abort.assert_not_called()
assert "Partial movie file written" in manim_caplog.text
assert not _alive_encoder_threads()
-def test_write_frame_fails_fast_after_encoder_failure(
- config,
- tmp_path,
- monkeypatch,
-):
+def test_write_frame_fails_fast_after_encoder_failure(config, tmp_path):
expected_exception = RuntimeError("encode failed")
- stream = Mock()
- container = Mock()
-
- def encode(*args):
- if args:
- raise expected_exception
- return []
-
- stream.encode.side_effect = encode
+ encoder = _fake_segment_encoder(tmp_path, "fail_fast")
+ encoder.write_frame.side_effect = expected_exception
config.media_dir = str(tmp_path)
writer = _make_writer("FailFastScene")
- job = _new_encode_job(tmp_path, monkeypatch, "fail_fast", stream, container)
+ job = _new_encode_job(tmp_path, "fail_fast", encoder)
job.path.write_bytes(b"stale")
writer._current_encode_job = job
@@ -403,10 +348,8 @@ def test_write_frame_fails_fast_after_encoder_failure(
assert not job.path.exists()
assert not _alive_encoder_threads()
finally:
- # An assertion failure above must not leave an unsealed non-daemon
- # worker behind: it would hang pytest at exit.
if writer._current_encode_job is not None:
- job.seal()
+ job.abort()
writer._current_encode_job = None
job.thread.join(timeout=5)
@@ -426,7 +369,10 @@ def test_frame_queue_configuration(
config.encoder_queue_size = encoder_queue_size
writer = _make_writer("FrameQueueSizeScene")
- writer.open_partial_movie_stream(tmp_path / "partial.mp4")
+ writer.open_partial_movie_stream(
+ animation_index=0,
+ file_path=tmp_path / "partial.mp4",
+ )
job = writer._current_encode_job
assert job is not None
assert job.queue.maxsize == expected_queue_size
@@ -436,6 +382,43 @@ def test_frame_queue_configuration(
assert not _alive_encoder_threads()
+def test_pool_settings_are_not_read_from_mutated_config(config, tmp_path):
+ config.max_inflight_encoders = 2
+ config.encoder_queue_size = 3
+ writer = _make_writer("CapturedPoolSettingsScene")
+ config.max_inflight_encoders = 1
+ config.encoder_queue_size = 9
+
+ writer.open_partial_movie_stream(
+ animation_index=0,
+ file_path=tmp_path / "partial.mp4",
+ )
+ job = writer._current_encode_job
+ assert job is not None
+ assert job.queue.maxsize == 3
+
+ writer.close_partial_movie_stream()
+ assert writer._inflight_encode_jobs == [job]
+ writer.join_all_encode_jobs()
+ assert not _alive_encoder_threads()
+
+
+def test_cache_limit_is_not_read_from_mutated_config(config, monkeypatch):
+ config.max_files_cached = -1
+ writer = _make_writer("CapturedCacheLimitScene")
+ config.max_files_cached = 0
+ writer.combine_to_movie = Mock()
+ prune = Mock()
+ monkeypatch.setattr(
+ "manim.scene.scene_file_writer.prune_segment_cache",
+ prune,
+ )
+
+ writer.finish()
+
+ prune.assert_called_once_with(writer.partial_movie_directory, -1)
+
+
@pytest.mark.parametrize("max_inflight_encoders", [1, 2, 3])
def test_close_partial_movie_stream_respects_cap_and_joins_fifo(
config,
@@ -549,7 +532,7 @@ def test_open_partial_movie_stream_joins_same_path_inflight_job(config, tmp_path
writer._inflight_encode_jobs.append(inflight_job)
writer._inflight_by_path[str(path)] = inflight_job
- writer.open_partial_movie_stream(file_path=path)
+ writer.open_partial_movie_stream(animation_index=0, file_path=path)
current_job = writer._current_encode_job
assert current_job is not None
try:
@@ -596,10 +579,8 @@ def _new_writer(config, tmp_path, scene_name):
return _make_writer(scene_name)
-def _healthy_current_job(tmp_path, monkeypatch, name):
- stream = Mock()
- stream.encode.return_value = []
- job = _new_encode_job(tmp_path, monkeypatch, name, stream, Mock())
+def _healthy_current_job(tmp_path, name):
+ job = _new_encode_job(tmp_path, name)
job.path.write_bytes(b"stale")
return job
@@ -612,11 +593,10 @@ def _add_inflight_job(writer, job):
def test_abort_encode_jobs_unlinks_current_and_drains_inflight(
config,
tmp_path,
- monkeypatch,
manim_caplog,
):
writer = _new_writer(config, tmp_path, "AbortScene")
- job = _healthy_current_job(tmp_path, monkeypatch, "abort_current")
+ job = _healthy_current_job(tmp_path, "abort_current")
writer._current_encode_job = job
failing_inflight = Mock(path=tmp_path / "inflight.mp4")
failing_inflight.join.side_effect = RuntimeError("in-flight join failed")
@@ -662,7 +642,7 @@ def test_abort_encode_jobs_cleanup_failure_logs_warning(
manim_caplog,
):
writer = _new_writer(config, tmp_path, "AbortCleanupFailureScene")
- job = _healthy_current_job(tmp_path, monkeypatch, "abort_cleanup_failure")
+ job = _healthy_current_job(tmp_path, "abort_cleanup_failure")
writer._current_encode_job = job
unlink = Mock(side_effect=PermissionError("cannot remove partial"))
monkeypatch.setattr(Path, "unlink", unlink)
@@ -671,7 +651,7 @@ def test_abort_encode_jobs_cleanup_failure_logs_warning(
assert writer._current_encode_job is None
unlink.assert_called_once_with(missing_ok=True)
- assert "Failed to remove incomplete partial movie file" in manim_caplog.text
+ assert "Failed to clean up incomplete segment" in manim_caplog.text
assert "cannot remove partial" in manim_caplog.text
assert "Discarded partial movie file" not in manim_caplog.text
assert not _alive_encoder_threads()
@@ -723,18 +703,12 @@ def test_rerun_propagates_encoder_failure(config, tmp_path):
assert writer._inflight_by_path == {}
-def test_rerun_propagates_failed_current_job(config, tmp_path, monkeypatch):
+def test_rerun_propagates_failed_current_job(config, tmp_path):
expected_exception = RuntimeError("encode failed")
- stream = Mock()
-
- def encode(*args):
- if args:
- raise expected_exception
- return []
-
- stream.encode.side_effect = encode
+ encoder = _fake_segment_encoder(tmp_path, "rerun_current")
+ encoder.write_frame.side_effect = expected_exception
writer = _new_writer(config, tmp_path, "RerunCurrentFailureScene")
- job = _new_encode_job(tmp_path, monkeypatch, "rerun_current", stream, Mock())
+ job = _new_encode_job(tmp_path, "rerun_current", encoder)
job.path.write_bytes(b"stale")
writer._current_encode_job = job
job.put(1, _frame())
@@ -765,7 +739,7 @@ def test_rerun_propagates_failed_current_job(config, tmp_path, monkeypatch):
# An assertion failure above must not leave an unsealed non-daemon
# worker behind: it would hang pytest at exit.
if writer._current_encode_job is not None:
- job.seal()
+ job.abort()
writer._current_encode_job = None
job.thread.join(timeout=5)
@@ -980,33 +954,17 @@ def test_parallel_encoding_output_matches_serial(tmp_path):
)
-def test_encode_job_repeats_frame_num_frames_times(tmp_path):
- """``put(n, frame)`` must encode the frame n times and mux every packet.
-
- Every other test in this module uses ``num_frames=1``, leaving the
- repetition loop in ``_encode_and_write_frame`` uncovered.
- """
- from manim.scene.scene_file_writer import _PartialMovieEncodeJob
+def test_encode_job_forwards_frame_repeat(tmp_path):
+ encoder = _fake_segment_encoder(tmp_path, "freeze_frame")
+ job = _new_encode_job(tmp_path, "freeze_frame", encoder)
- packet = object()
- stream = Mock()
- stream.encode.side_effect = lambda *args: [packet] if args else []
- container = Mock()
-
- job = _PartialMovieEncodeJob(
- path=tmp_path / "freeze_frame.mp4",
- animation_index=0,
- container=container,
- stream=stream,
- frame_queue_size=8,
- )
- job.put(3, _frame())
+ frame = _frame()
+ job.put(3, frame)
job.seal()
job.join()
- # Three encode calls with a frame, then the argless flush.
- assert stream.encode.call_args_list == [call(ANY)] * 3 + [call()]
- assert container.mux.call_args_list == [call(packet)] * 3
+ encoder.write_frame.assert_called_once_with(frame, repeat=3)
+ encoder.finish.assert_called_once_with()
def test_is_already_cached_false_after_joining_failed_path(config, tmp_path):
@@ -1058,15 +1016,23 @@ def test_open_partial_movie_stream_without_path_raises(config, tmp_path):
writer.partial_movie_files = [None]
with pytest.raises(RuntimeError, match="partial movie file path"):
- writer.open_partial_movie_stream()
+ writer.open_partial_movie_stream(animation_index=0)
-def test_write_frame_without_open_stream_drops_frame(config, tmp_path):
- """Interactive OpenGL emits frames with no open stream; they are dropped.
+def test_open_partial_movie_stream_rejects_nested_current_segment(config, tmp_path):
+ with tempconfig({"media_dir": str(tmp_path)}):
+ writer = _make_writer("GuardScene")
+ writer._current_encode_job = Mock()
- Video output is enabled under the default test config, so the call reaches
- the drop branch in ``write_frame``.
- """
+ with pytest.raises(RuntimeError, match="another segment is still open"):
+ writer.open_partial_movie_stream(
+ animation_index=1,
+ file_path=tmp_path / "nested.mp4",
+ )
+
+
+def test_write_frame_without_open_stream_drops_frame(config, tmp_path):
+ """Frames outside an open segment are ignored by the transitional scheduler."""
with tempconfig({"media_dir": str(tmp_path)}):
writer = _make_writer("DropFrameScene")
assert writer._current_encode_job is None
diff --git a/tests/test_scene_rendering/test_video_segment_profiles.py b/tests/test_scene_rendering/test_video_segment_profiles.py
new file mode 100644
index 000000000..9ff96a15c
--- /dev/null
+++ b/tests/test_scene_rendering/test_video_segment_profiles.py
@@ -0,0 +1,204 @@
+from __future__ import annotations
+
+from fractions import Fraction
+from pathlib import Path
+
+import av
+import numpy as np
+import pytest
+from av.codec.context import CodecContext
+
+from manim import tempconfig
+from manim._config import config
+from manim._config.output import OutputFormat, OutputSpec
+from manim._config.output_plan import resolve_media_layout, resolve_output_plan
+from manim._config.video_encoder import resolve_video_encoder
+from manim.scene.scene_file_writer import SceneFileWriter, _SceneFileWriterSettings
+
+_WIDTH = 16
+_HEIGHT = 12
+_FRAME_RATE = 5
+_FRAME_COUNT = 3
+
+
+def _output(format: str, transparent: bool) -> OutputSpec:
+ return OutputSpec(
+ format=OutputFormat(format),
+ transparent=transparent,
+ save_sections=False,
+ fallback_to_still=False,
+ )
+
+
+def _writer(output: OutputSpec) -> SceneFileWriter:
+ layout = resolve_media_layout(
+ config,
+ output,
+ module_name="segment_profiles",
+ scene_name="SegmentProfileScene",
+ working_directory=Path.cwd(),
+ )
+ plan = resolve_output_plan(
+ layout,
+ output,
+ scene_name="SegmentProfileScene",
+ requested_output_name=None,
+ )
+ video_encoder = resolve_video_encoder(
+ output,
+ width=config.pixel_width,
+ height=config.pixel_height,
+ frame_rate=config.frame_rate,
+ codec=config.video_codec,
+ pixel_format=config.pixel_format,
+ options=config.video_encoder_options,
+ )
+ settings = _SceneFileWriterSettings(
+ plan=plan,
+ video_encoder=video_encoder,
+ max_inflight_encoders=config.max_inflight_encoders,
+ encoder_queue_size=config.encoder_queue_size,
+ max_files_cached=config.max_files_cached,
+ assets_dir=Path.cwd(),
+ )
+ return SceneFileWriter(settings)
+
+
+def _asymmetric_rgba_frame() -> np.ndarray:
+ frame = np.empty((_HEIGHT, _WIDTH, 4), dtype=np.uint8)
+ frame[: _HEIGHT // 2, : _WIDTH // 2] = [255, 0, 0, 32]
+ frame[: _HEIGHT // 2, _WIDTH // 2 :] = [0, 255, 0, 96]
+ frame[_HEIGHT // 2 :, : _WIDTH // 2] = [0, 0, 255, 160]
+ frame[_HEIGHT // 2 :, _WIDTH // 2 :] = [255, 255, 255, 224]
+ return frame
+
+
+def _decode_frames(
+ path: Path,
+ *,
+ transparent_vp9: bool,
+) -> tuple[str, str, Fraction | None, list[np.ndarray], int]:
+ with av.open(path) as container:
+ stream = container.streams.video[0]
+ codec = stream.codec_context.name
+ pixel_format = stream.codec_context.format.name
+ rate = stream.average_rate
+ audio_streams = len(container.streams.audio)
+
+ if transparent_vp9:
+ decoder = CodecContext.create("libvpx-vp9", "r")
+ decoded = []
+ for packet in container.demux(video=0):
+ decoded.extend(decoder.decode(packet))
+ pixel_format = decoded[0].format.name
+ else:
+ decoded = list(container.decode(video=0))
+
+ frames = [frame.to_ndarray(format="rgba") for frame in decoded]
+ return codec, pixel_format, rate, frames, audio_streams
+
+
+ (
+ "format",
+ "transparent",
+ "segment_extension",
+ "codec",
+ "pixel_format",
+ ),
+ [
+ ("mp4", False, ".mp4", "h264", "yuv420p"),
+ ("mov", False, ".mov", "h264", "yuv420p"),
+ ("mov", True, ".mov", "qtrle", "argb"),
+ ("webm", False, ".webm", "vp9", "yuv420p"),
+ ("webm", True, ".webm", "vp9", "yuva420p"),
+ ],
+)
+def test_cached_segment_profile_and_pixel_orientation(
+ tmp_path,
+ format,
+ transparent,
+ segment_extension,
+ codec,
+ pixel_format,
+):
+ output = _output(format, transparent)
+ target = tmp_path / f"segment{segment_extension}"
+ source = _asymmetric_rgba_frame()
+
+ with tempconfig(
+ {
+ "media_dir": tmp_path,
+ "pixel_width": _WIDTH,
+ "pixel_height": _HEIGHT,
+ "frame_rate": _FRAME_RATE,
+ },
+ ):
+ writer = _writer(output)
+ writer.open_partial_movie_stream(animation_index=0, file_path=target)
+ writer.write_frame(source, repeat=_FRAME_COUNT)
+ writer.close_partial_movie_stream()
+ writer.join_all_encode_jobs()
+
+ (
+ actual_codec,
+ actual_pixel_format,
+ actual_rate,
+ decoded_frames,
+ audio_streams,
+ ) = _decode_frames(
+ target,
+ transparent_vp9=format == "webm" and transparent,
+ )
+
+ assert output.segment_extension == segment_extension
+ assert actual_codec == codec
+ assert actual_pixel_format == pixel_format
+ assert actual_rate == Fraction(_FRAME_RATE, 1)
+ assert len(decoded_frames) == _FRAME_COUNT
+ assert audio_streams == 0
+
+ sample_points = (
+ ((_HEIGHT // 4, _WIDTH // 4), np.array([255, 0, 0, 32])),
+ ((_HEIGHT // 4, 3 * _WIDTH // 4), np.array([0, 255, 0, 96])),
+ ((3 * _HEIGHT // 4, _WIDTH // 4), np.array([0, 0, 255, 160])),
+ ((3 * _HEIGHT // 4, 3 * _WIDTH // 4), np.array([255, 255, 255, 224])),
+ )
+ decoded = decoded_frames[0]
+ for (row, column), expected in sample_points:
+ expected = expected.copy()
+ if not transparent:
+ expected[3] = 255
+ np.testing.assert_allclose(decoded[row, column], expected, atol=15)
+
+
+def test_configured_encoder_profile_is_used_for_cached_segment(tmp_path):
+ output = _output("mov", transparent=False)
+ target = tmp_path / "custom.mov"
+
+ with tempconfig(
+ {
+ "media_dir": tmp_path,
+ "pixel_width": _WIDTH,
+ "pixel_height": _HEIGHT,
+ "frame_rate": _FRAME_RATE,
+ "video_codec": "qtrle",
+ "pixel_format": "argb",
+ "video_encoder_options": {},
+ },
+ ):
+ writer = _writer(output)
+ writer.open_partial_movie_stream(animation_index=0, file_path=target)
+ writer.write_frame(_asymmetric_rgba_frame())
+ writer.close_partial_movie_stream()
+ writer.join_all_encode_jobs()
+
+ codec, pixel_format, rate, frames, audio_streams = _decode_frames(
+ target,
+ transparent_vp9=False,
+ )
+ assert codec == "qtrle"
+ assert pixel_format == "argb"
+ assert rate == Fraction(_FRAME_RATE, 1)
+ assert len(frames) == 1
+ assert audio_streams == 0