Skip to content

ReShot — copy the shot, not the actors. Video → depth map, OpenPose skeleton or canny lines for Seedance 2.0/2.5 reference video, MiniMax H3 Fun ControlNet and Wan VACE. Apache-2.0. Open-sourced by Maosika 猫斯卡 (www.maosika.com), the AI short-drama production system.

0.3.2: --max-res no longer lowers inference resolution; batch mode (run_many); to_gray and decode memory

wangfeng143018a92d5b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1a767ae..6ae9d4b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,17 @@
# Changelog
+## 0.3.2 — 2026-09-12
+- **Fix: `--max-res` no longer lowers the inference resolution.** It capped the processing
+ size too, so `--max-res 320` fed the model a thumbnail; inference now always runs at model
+ resolution and `--max-res` only sizes the output (regression test added).
+- **Batch mode**: `reshot a.mp4 b.mp4 … -o dir/` (or any `-o` that is a directory) runs
+ every clip on one model load; `--metrics`/`--npz` become per-clip directories; a failed clip
+ is skipped and reported. Python: `run_many()`.
+- Memory: `to_gray` converts frame by frame (−1.2 GB peak for a 12 s clip, byte-identical
+ output); decoding writes into one preallocated buffer instead of list + `np.stack` (−400 MB).
+- `--npz` / `--metrics` create their parent directories. `torch.load(weights_only=True)`;
+ inference under `torch.inference_mode()`; `extract()` honours `RESHOT_FAKE_BACKEND`.
+
## 0.3.1 — 2026-09-12
- Fix: the package failed to import on Windows since 0.2.0 (`import resource` is Unix-only); peak-memory metric now uses the Win32 counter there.
- README rewritten as a manual: the problem → what it does → four steps with the real prompts and character sheets from the demo (`docs/prompts/`, `docs/refs/`) → what transfers and what doesn't → presets → developers.
diff --git a/README.md b/README.md
index 6654b68..430d4ac 100644
--- a/README.md
+++ b/README.md
@@ -49,7 +49,7 @@ No ffmpeg? `pip install "reshot[ffmpeg] @ git+https://github.com/maosika-ai/resh
reshot reference.mp4 -o depth.mp4 --target seedance
```
-`--target seedance` sets 24 fps, H.264, a frame size that is a multiple of 16 and at least 407,696 pixels, up to 15 seconds — the reference-video rules of the Seedance API. For MiniMax H3 use `--target h3` (multiples of 32). On an RTX 4090 a 12-second clip takes about 20 seconds; on a MacBook a few minutes.
+`--target seedance` sets 24 fps, H.264, a frame size that is a multiple of 16 and at least 407,696 pixels, up to 15 seconds — the reference-video rules of the Seedance API. For MiniMax H3 use `--target h3` (multiples of 32). On an RTX 4090 a 12-second clip takes about 20 seconds; on a MacBook a few minutes. A whole folder at once, one model load: `reshot clips/*.mp4 -o depth/ --target seedance`.
### 3. Give it to the video model
diff --git a/README.zh-CN.md b/README.zh-CN.md
index f67a544..3eb5b08 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -49,7 +49,7 @@ pip install git+https://github.com/maosika-ai/reshot # 需要 PATH 里有
reshot 参考片.mp4 -o 深度图.mp4 --target seedance
```
-`--target seedance` 会把输出设成 24 fps、H.264、边长 16 的倍数、不少于 407,696 像素、最长 15 秒——这就是 Seedance 接口对参考视频的要求。给 MiniMax H3 用 `--target h3`(边长 32 的倍数)。RTX 4090 上 12 秒的片约 20 秒跑完,MacBook 要几分钟。
+`--target seedance` 会把输出设成 24 fps、H.264、边长 16 的倍数、不少于 407,696 像素、最长 15 秒——这就是 Seedance 接口对参考视频的要求。给 MiniMax H3 用 `--target h3`(边长 32 的倍数)。RTX 4090 上 12 秒的片约 20 秒跑完,MacBook 要几分钟。整个文件夹一起来、模型只加载一次:`reshot clips/*.mp4 -o depth/ --target seedance`。
### 3. 交给视频模型
diff --git a/demo/requirements.txt b/demo/requirements.txt
index 842369e..38ed5a0 100644
--- a/demo/requirements.txt
+++ b/demo/requirements.txt
@@ -1,5 +1,5 @@
--extra-index-url https://download.pytorch.org/whl/cpu
torch
torchvision
-reshot[ffmpeg] @ git+https://github.com/maosika-ai/[email protected]
+reshot[ffmpeg] @ git+https://github.com/maosika-ai/[email protected]
gradio>=4.0
diff --git a/docs/USAGE.md b/docs/USAGE.md
index 00c6268..8f1be48 100644
--- a/docs/USAGE.md
+++ b/docs/USAGE.md
@@ -91,7 +91,7 @@ Frame size is **cropped**, never padded: a padded black border reads to the gene
```python
from pathlib import Path
-from reshot import RunConfig, run, plan
+from reshot import RunConfig, run, run_many, plan
cfg = RunConfig(input=Path("in.mp4"), output=Path("out.mp4"), target="h3", metrics=Path("m.json"))
print(plan(cfg)) # sizes, frame counts, RAM estimate — no decoding yet
@@ -158,20 +158,24 @@ verified end-to-end yet.
### Batch
+Give `reshot` several inputs and a directory; the model loads once for all of them and
+each clip comes out as `<name>_depth.mp4`. `--metrics` / `--npz` become directories too
+(`<name>.json` / `<name>.npz`). A clip that fails is reported and skipped; the exit code
+is that of the first failure.
+
```bash
-for f in clips/*.mp4; do reshot "$f" -o "out/$(basename "$f" .mp4)_depth.mp4" --target h3 --metrics "out/$(basename "$f" .mp4).json"; done
+reshot clips/*.mp4 -o depth/ --target seedance --metrics depth/metrics/
```
-The model loads once per process; for hundreds of clips write a short Python loop with
-`extract()` and reuse the backend object:
+From Python, `run_many()` is the same thing:
```python
-from reshot.backends import get_backend
-from reshot import read_video, to_gray, write_gray_video
-b = get_backend("vda", model="small")
-for src in sources:
- frames, fps = read_video(src, max_res=900)
- write_gray_video(to_gray(b.infer(frames, fps)), src.with_suffix(".depth.mp4"), fps)
+from pathlib import Path
+from reshot import RunConfig, run_many
+
+cfgs = [RunConfig(input=p, output=Path("depth") / f"{p.stem}_depth.mp4", target="seedance") for p in Path("clips").glob("*.mp4")]
+for r in run_many(cfgs): # RunResult, or the ReshotError for a clip that failed
+ print(r)
```
### Long videos
diff --git a/docs/USAGE.zh-CN.md b/docs/USAGE.zh-CN.md
index d21f079..f80036a 100644
--- a/docs/USAGE.zh-CN.md
+++ b/docs/USAGE.zh-CN.md
@@ -90,7 +90,7 @@ reshot 输入 -o 输出 [选项]
```python
from pathlib import Path
-from reshot import RunConfig, run, plan
+from reshot import RunConfig, run, run_many, plan
cfg = RunConfig(input=Path("in.mp4"), output=Path("out.mp4"), target="h3", metrics=Path("m.json"))
print(plan(cfg)) # 尺寸、帧数、内存估算——此时还没解码
@@ -152,19 +152,23 @@ write_gray_video(upsample_frames(gray, 1280, 736), "out.mp4", fps, size=(1280, 7
### 批量
+一次给 `reshot` 多个输入和一个目录,模型只加载一次,每条片输出为 `<原名>_depth.mp4`。
+`--metrics` / `--npz` 此时也当目录用(`<原名>.json` / `<原名>.npz`)。某条片失败会报出来并跳过,
+退出码取第一个失败的。
+
```bash
-for f in clips/*.mp4; do reshot "$f" -o "out/$(basename "$f" .mp4)_depth.mp4" --target h3 --metrics "out/$(basename "$f" .mp4).json"; done
+reshot clips/*.mp4 -o depth/ --target seedance --metrics depth/metrics/
```
-每个进程载一次模型;几百条片子的话写个 Python 循环复用后端对象:
+Python 里对应 `run_many()`:
```python
-from reshot.backends import get_backend
-from reshot import read_video, to_gray, write_gray_video
-b = get_backend("vda", model="small")
-for src in sources:
- frames, fps = read_video(src, max_res=900)
- write_gray_video(to_gray(b.infer(frames, fps)), src.with_suffix(".depth.mp4"), fps)
+from pathlib import Path
+from reshot import RunConfig, run_many
+
+cfgs = [RunConfig(input=p, output=Path("depth") / f"{p.stem}_depth.mp4", target="seedance") for p in Path("clips").glob("*.mp4")]
+for r in run_many(cfgs): # 成功是 RunResult,失败是那条片的 ReshotError
+ print(r)
```
### 长视频
diff --git a/reshot/__init__.py b/reshot/__init__.py
index 724c930..40c3f23 100644
--- a/reshot/__init__.py
+++ b/reshot/__init__.py
@@ -23,7 +23,7 @@ from ._version import __version__
from .config import RunConfig
from .errors import BackendError, InputError, RamBudgetError, ReshotError, ToolMissingError
from .io import probe_video, read_video, write_gray_video
-from .pipeline import Plan, RunResult, extract, plan, run
+from .pipeline import Plan, RunResult, extract, plan, run, run_many
from .planning import memory_verdict, processing_max_res
from .postprocess import to_gray, upsample_frames
from .targets import TARGETS, Target, center_crop, fit_dimensions
@@ -49,6 +49,7 @@ __all__ = [
"processing_max_res",
"read_video",
"run",
+ "run_many",
"to_gray",
"upsample_frames",
"write_gray_video",
diff --git a/reshot/_version.py b/reshot/_version.py
index 260c070..f9aa3e1 100644
--- a/reshot/_version.py
+++ b/reshot/_version.py
@@ -1 +1 @@
-__version__ = "0.3.1"
+__version__ = "0.3.2"
diff --git a/reshot/backends/vda.py b/reshot/backends/vda.py
index cc6e70e..c91e68c 100644
--- a/reshot/backends/vda.py
+++ b/reshot/backends/vda.py
@@ -103,7 +103,7 @@ class VDABackend:
) from exc
log.info("loading %s from %s on %s (fp32=%s)", variant, path, self.device, self.fp32)
model = VideoDepthAnything(**_VARIANTS[variant])
- model.load_state_dict(torch.load(path, map_location="cpu"), strict=True)
+ model.load_state_dict(torch.load(path, map_location="cpu", weights_only=True), strict=True)
self.model = model.to(self.device).eval()
def infer(self, frames: np.ndarray, fps: float, *, input_size: int = 518) -> np.ndarray:
@@ -112,7 +112,10 @@ class VDABackend:
# Upstream's infer_video_depth already does 32-frame windows with 10-frame overlap,
# keyframe-based scale/shift alignment across windows and interpolation over the
# seam. That is the temporal-consistency machinery — do not re-chunk outside it.
- depths, _ = self.model.infer_video_depth(frames, fps, input_size=input_size, device=self.device, fp32=self.fp32)
+ with torch.inference_mode(): # upstream uses no_grad; inference_mode also skips version counters
+ depths, _ = self.model.infer_video_depth(
+ frames, fps, input_size=input_size, device=self.device, fp32=self.fp32
+ )
return np.asarray(depths, dtype=np.float32)
diff --git a/reshot/cli.py b/reshot/cli.py
index 6c7ea80..acc50cc 100644
--- a/reshot/cli.py
+++ b/reshot/cli.py
@@ -1,8 +1,13 @@
-"""`reshot` command line: argv → RunConfig → pipeline.run().
+"""`reshot` command line: argv → RunConfig(s) → pipeline.run() / run_many().
reshot in.mp4 -o out.mp4 # Apache-2.0 Small model, auto device
reshot in.mp4 -o out.mp4 --target h3 # 24 fps, ×32 dims, ≤15 s for MiniMax H3
reshot in.mp4 -o out.mp4 --npz d.npz --metrics run.json
+ reshot clips/*.mp4 -o depth/ --target seedance # batch: one model load, out/<name>_depth.mp4
+
+Batch mode (several inputs, or `-o` naming a directory): `--npz` / `--metrics`, if given,
+are directories too and get `<name>.npz` / `<name>.json` per clip. One bad clip is
+reported and skipped; the exit code is that of the first failure.
Exit codes: 0 ok · 1 unexpected · 2 bad input/arguments · 3 over RAM budget ·
4 ffmpeg missing · 5 backend/weights problem.
@@ -18,7 +23,7 @@ from pathlib import Path
from ._version import __version__
from .config import BACKENDS, MODEL_VARIANTS, RunConfig
from .errors import ReshotError
-from .pipeline import run
+from .pipeline import run, run_many
from .reporter import StderrReporter
from .targets import TARGETS
@@ -29,8 +34,10 @@ def build_parser() -> argparse.ArgumentParser:
description="Video → depth-map video for video-generation ControlNets.",
epilog="Docs: https://github.com/maosika-ai/reshot",
)
- p.add_argument("input", type=Path, help="input video (anything ffmpeg/OpenCV can read)")
- p.add_argument("-o", "--output", type=Path, required=True, help="output .mp4")
+ p.add_argument("input", type=Path, nargs="+", help="input video(s) (anything ffmpeg/OpenCV can read)")
+ p.add_argument(
+ "-o", "--output", type=Path, required=True, help="output .mp4, or a directory when there are several inputs"
+ )
g = p.add_argument_group("model")
g.add_argument(
"--model",
@@ -69,27 +76,52 @@ def build_parser() -> argparse.ArgumentParser:
return p
+def _is_batch(args: argparse.Namespace) -> bool:
+ """Several inputs, or an output that is (or is spelled like) a directory."""
+ o = args.output
+ return len(args.input) > 1 or o.is_dir() or str(o).endswith(("/", "\\"))
+
+
+def configs_from_args(args: argparse.Namespace) -> list[RunConfig]:
+ """One RunConfig per input. In batch mode the outputs are derived from the input names."""
+ batch = _is_batch(args)
+
+ def per_clip(option: Path | None, stem: str, ext: str) -> Path | None:
+ if option is None:
+ return None
+ return option / f"{stem}{ext}" if batch else option
+
+ cfgs = []
+ for src in args.input:
+ stem = src.stem
+ cfgs.append(
+ RunConfig(
+ input=src,
+ output=args.output / f"{stem}_depth.mp4" if batch else args.output,
+ model=args.model,
+ backend=args.backend,
+ device=args.device,
+ target=args.target,
+ fps=args.fps,
+ max_res=args.max_res,
+ max_frames=args.max_frames,
+ input_size=args.input_size,
+ invert=args.invert,
+ clip_percent=args.clip,
+ gamma=args.gamma,
+ crf=args.crf,
+ npz=per_clip(args.npz, stem, ".npz"),
+ metrics=per_clip(args.metrics, stem, ".json"),
+ checkpoint=args.checkpoint,
+ force=args.force,
+ )
+ )
+ return cfgs
+
+
def config_from_args(args: argparse.Namespace) -> RunConfig:
- return RunConfig(
- input=args.input,
- output=args.output,
- model=args.model,
- backend=args.backend,
- device=args.device,
- target=args.target,
- fps=args.fps,
- max_res=args.max_res,
- max_frames=args.max_frames,
- input_size=args.input_size,
- invert=args.invert,
- clip_percent=args.clip,
- gamma=args.gamma,
- crf=args.crf,
- npz=args.npz,
- metrics=args.metrics,
- checkpoint=args.checkpoint,
- force=args.force,
- )
+ """Single-clip form, kept for callers that build a Namespace themselves."""
+ return configs_from_args(args)[0]
def main(argv: list[str] | None = None) -> int:
@@ -98,14 +130,22 @@ def main(argv: list[str] | None = None) -> int:
level=logging.INFO if args.verbose else logging.WARNING, format="%(levelname)s %(name)s: %(message)s"
)
try:
- run(config_from_args(args), StderrReporter())
+ cfgs = configs_from_args(args)
+ if len(cfgs) == 1 and not _is_batch(args):
+ run(cfgs[0], StderrReporter())
+ return 0
+ results = run_many(cfgs, StderrReporter())
+ failures = [r for r in results if isinstance(r, ReshotError)]
+ if failures:
+ print(f"reshot: {len(failures)} of {len(results)} clips failed", file=sys.stderr)
+ return failures[0].exit_code
+ return 0
except ReshotError as exc:
print(f"reshot: {exc}", file=sys.stderr)
return exc.exit_code
except KeyboardInterrupt:
print("reshot: interrupted", file=sys.stderr)
return 130
- return 0
if __name__ == "__main__": # pragma: no cover
diff --git a/reshot/io.py b/reshot/io.py
index efb31e9..1778222 100644
--- a/reshot/io.py
+++ b/reshot/io.py
@@ -54,7 +54,18 @@ def read_video(
src_fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
out_fps = src_fps if target_fps <= 0 or target_fps >= src_fps else float(target_fps)
- frames: list[np.ndarray] = []
+ # Decode straight into one preallocated array. Collecting frames in a list and
+ # np.stack-ing them at the end held two copies of the clip for a moment (≈400 MB
+ # extra for 12 s at model resolution). The header's frame count can be off for some
+ # containers, so the buffer grows if the file turns out longer than announced.
+ header_n = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+ expect = header_n if header_n > 0 else 1024
+ if out_fps < src_fps:
+ expect = int(expect * out_fps / src_fps) + 2
+ if max_frames > 0:
+ expect = min(expect, max_frames)
+ buf: np.ndarray | None = None
+ n = 0
src_index = 0
next_pick_time = 0.0
while True:
@@ -72,13 +83,18 @@ def read_video(
new_w = int(round(bgr.shape[1] * scale / 2) * 2)
new_h = int(round(bgr.shape[0] * scale / 2) * 2)
bgr = cv2.resize(bgr, (new_w, new_h), interpolation=cv2.INTER_AREA)
- frames.append(cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB))
- if 0 < max_frames <= len(frames):
+ if buf is None:
+ buf = np.empty((max(expect, 1), *bgr.shape), dtype=np.uint8)
+ elif n == buf.shape[0]:
+ buf = np.concatenate([buf, np.empty_like(buf[: max(16, n // 4)])])
+ cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB, dst=buf[n])
+ n += 1
+ if 0 < max_frames <= n:
break
cap.release()
- if not frames:
+ if buf is None or n == 0:
raise ValueError(f"no frames decoded from {path}")
- return np.stack(frames), out_fps
+ return buf[:n], out_fps
def ffmpeg_binary() -> str:
diff --git a/reshot/pipeline.py b/reshot/pipeline.py
index 34d776c..f090071 100644
--- a/reshot/pipeline.py
+++ b/reshot/pipeline.py
@@ -29,7 +29,7 @@ import numpy as np
from ._version import __version__
from .backends import get_backend, pick_device
from .config import RunConfig
-from .errors import InputError, RamBudgetError
+from .errors import InputError, RamBudgetError, ReshotError
from .io import probe_video, read_video, write_gray_video
from .planning import gib, memory_verdict, processing_max_res
from .postprocess import to_gray, upsample_frames
@@ -93,8 +93,13 @@ def plan(cfg: RunConfig) -> Plan:
frames = round(frames * out_fps / src_fps)
frames = max(frames, 1)
+ # Inference always runs at the model's working resolution. `max_res` caps the OUTPUT
+ # only — until 0.3.2 it also capped the processing size, so `--max-res 320` fed the
+ # model a 320-px thumbnail that it then upsampled internally: a depth map guessed
+ # from a thumbnail. The three demo takes were made from full-resolution depth
+ # downscaled afterwards, which is what this now does.
cap = cfg.max_res if cfg.max_res > 0 else 10**9
- proc_max_res = min(processing_max_res(src_w, src_h, cfg.input_size), cap)
+ proc_max_res = processing_max_res(src_w, src_h, cfg.input_size)
scale = min(1.0, proc_max_res / max(src_w, src_h))
proc_w, proc_h = (int(round(src_w * scale / 2) * 2), int(round(src_h * scale / 2) * 2))
@@ -119,9 +124,60 @@ def plan(cfg: RunConfig) -> Plan:
)
+def _make_backend(cfg: RunConfig):
+ """Build the depth backend a config asks for (the fake one when the env var is set)."""
+ backend_name = "fake" if os.environ.get("RESHOT_FAKE_BACKEND") else cfg.backend
+ if backend_name == "fake":
+ return get_backend("fake"), backend_name
+ device = pick_device(cfg.device)
+ backend = get_backend(
+ "vda", model=cfg.model, device=device, checkpoint=str(cfg.checkpoint) if cfg.checkpoint else None
+ )
+ return backend, backend_name
+
+
+def run_many(cfgs: list[RunConfig], reporter: Reporter | None = None) -> list[RunResult | ReshotError]:
+ """Run several configs with ONE model load.
+
+ A shell loop over `reshot` pays torch import + CUDA context + weight load (3–6 s) per
+ clip; here that happens once. Every config must ask for the same model / device /
+ checkpoint. A clip that fails with a user-fixable `ReshotError` does not stop the
+ batch — its error is returned in place of a result so the caller can report it.
+ """
+ if not cfgs:
+ return []
+ rep = reporter or NullReporter()
+ key = {(c.model, c.backend, c.device, c.checkpoint) for c in cfgs}
+ if len(key) != 1:
+ raise InputError("batch: every input must use the same --model / --backend / --device / --checkpoint")
+ backend, backend_name = _make_backend(cfgs[0])
+ results: list[RunResult | ReshotError] = []
+ try:
+ for i, cfg in enumerate(cfgs, 1):
+ rep.step("file", f"[{i}/{len(cfgs)}] {cfg.input} → {cfg.output}")
+ try:
+ results.append(_run_one(cfg, rep, backend, backend_name))
+ except ReshotError as exc:
+ rep.step("error", f"{cfg.input}: {exc}")
+ results.append(exc)
+ finally:
+ _release_accelerator(backend.device)
+ del backend
+ return results
+
+
def run(cfg: RunConfig, reporter: Reporter | None = None) -> RunResult:
"""Execute a full run. See module docstring."""
- rep = reporter or NullReporter()
+ backend, backend_name = _make_backend(cfg)
+ try:
+ return _run_one(cfg, reporter or NullReporter(), backend, backend_name)
+ finally:
+ _release_accelerator(backend.device)
+ del backend
+
+
+def _run_one(cfg: RunConfig, rep: Reporter, backend, backend_name: str) -> RunResult:
+ """One clip through plan → read → depth → grey → encode on an already-built backend."""
t0 = time.time()
target = TARGETS[cfg.target]
@@ -147,15 +203,7 @@ def run(cfg: RunConfig, reporter: Reporter | None = None) -> RunResult:
rep.step("warn", f"clip is {t / fps:.1f}s; {target.key} accepts ≤ {target.max_seconds:.0f}s — trim it")
# ── depth ─────────────────────────────────────────────────────────────────
- backend_name = "fake" if os.environ.get("RESHOT_FAKE_BACKEND") else cfg.backend
- device = pick_device(cfg.device)
- backend = (
- get_backend("fake")
- if backend_name == "fake"
- else get_backend(
- "vda", model=cfg.model, device=device, checkpoint=str(cfg.checkpoint) if cfg.checkpoint else None
- )
- )
+ device = backend.device
precision = "fp32" if backend.fp32 else "fp16"
rep.step(
"model",
@@ -165,11 +213,11 @@ def run(cfg: RunConfig, reporter: Reporter | None = None) -> RunResult:
depths = backend.infer(frames, fps, input_size=cfg.input_size)
depth_seconds = time.time() - t1
del frames
- _release_accelerator(backend.device)
- del backend
+ _release_accelerator(backend.device) # the model stays; only cached activations go
rep.step("depth", f"{depth_seconds:.1f}s = {depth_seconds / t * 1000:.0f} ms/frame")
if cfg.npz:
+ Path(cfg.npz).parent.mkdir(parents=True, exist_ok=True)
np.savez_compressed(cfg.npz, depths=depths, fps=fps)
rep.step("npz", str(cfg.npz))
@@ -208,6 +256,7 @@ def run(cfg: RunConfig, reporter: Reporter | None = None) -> RunResult:
"output_bytes": os.path.getsize(out),
}
if cfg.metrics:
+ Path(cfg.metrics).parent.mkdir(parents=True, exist_ok=True)
with open(cfg.metrics, "w", encoding="utf-8") as fh:
json.dump(metrics, fh, ensure_ascii=False, indent=2)
rep.step("metrics", str(cfg.metrics))
@@ -232,11 +281,16 @@ def extract(
) -> tuple[np.ndarray, float]:
"""Convenience: decode → depth, no encoding. Returns `(depths[T,H,W] float32, fps)`.
- Same resolution policy as `run()`: frames are read at model resolution."""
+ Frames are read at model resolution, like `run()`. Unlike `run()`, `max_res` here caps
+ the *processing* size (there is no separate output to cap) — pass a smaller value
+ only when you deliberately want cheaper, coarser depth."""
w, h, _, _ = probe_video(video)
proc = min(processing_max_res(w, h, input_size), max_res if max_res > 0 else 10**9)
frames, fps = read_video(video, max_frames=max_frames, target_fps=target_fps, max_res=proc)
- backend = get_backend("vda", model=model, device=device, checkpoint=checkpoint)
+ if os.environ.get("RESHOT_FAKE_BACKEND"): # same escape hatch as run(), for tests
+ backend = get_backend("fake")
+ else:
+ backend = get_backend("vda", model=model, device=device, checkpoint=checkpoint)
return backend.infer(frames, fps, input_size=input_size), fps
diff --git a/reshot/postprocess.py b/reshot/postprocess.py
index 347bbdd..92f372b 100644
--- a/reshot/postprocess.py
+++ b/reshot/postprocess.py
@@ -40,12 +40,20 @@ def to_gray(
lo, hi = float(d.min()), float(d.max())
if hi - lo < 1e-6:
return np.zeros(d.shape, dtype=np.uint8)
- n = np.clip((d - lo) / (hi - lo), 0.0, 1.0)
- if gamma != 1.0:
- n = n ** float(gamma)
- if invert:
- n = 1.0 - n
- return (n * 255.0 + 0.5).astype(np.uint8)
+ # One scale for the clip (lo/hi above), but convert frame by frame: doing the
+ # arithmetic on the whole [T, H, W] array allocated four float32 temporaries of the
+ # clip's size — 1.2 GB extra for a 12 s clip, 3 GB for 30 s — which is what decided
+ # whether a 16 GB laptop could run a long clip at all. Same operations in the same
+ # order, so the bytes are identical to the whole-array version.
+ out = np.empty(d.shape, dtype=np.uint8)
+ for i in range(d.shape[0]):
+ n = np.clip((d[i] - lo) / (hi - lo), 0.0, 1.0)
+ if gamma != 1.0:
+ n = n ** float(gamma)
+ if invert:
+ n = 1.0 - n
+ out[i] = (n * 255.0 + 0.5).astype(np.uint8)
+ return out
def upsample_frames(gray: np.ndarray, height: int, width: int):
diff --git a/tests/test_io.py b/tests/test_io.py
index 0d3c021..65d6e81 100644
--- a/tests/test_io.py
+++ b/tests/test_io.py
@@ -79,3 +79,24 @@ def test_probe_video(tmp_path):
w, h, fps, n = probe_video(p)
assert (w, h, n) == (64, 48, 5) and abs(fps - 25) < 0.01
+
+
+def test_read_video_grows_when_header_frame_count_is_wrong(tmp_path, monkeypatch):
+ """The decode buffer is sized from the container's frame count; a wrong header must
+ not truncate the clip."""
+ import reshot.io as io
+
+ src = tmp_path / "src.mp4"
+ vw = cv2.VideoWriter(str(src), cv2.VideoWriter_fourcc(*"mp4v"), 30, (64, 48))
+ for i in range(40):
+ vw.write(np.full((48, 64, 3), i * 6 % 256, np.uint8))
+ vw.release()
+ real_get = cv2.VideoCapture.get
+
+ def lying_get(self, prop):
+ return 3 if prop == cv2.CAP_PROP_FRAME_COUNT else real_get(self, prop)
+
+ monkeypatch.setattr(cv2.VideoCapture, "get", lying_get)
+ frames, _ = io.read_video(src)
+ assert frames.shape[0] == 40
+ assert abs(int(frames[39, 0, 0, 0]) - 39 * 6 % 256) < 12 # lossy codec; frames arrived in order
diff --git a/tests/test_pipeline.py b/tests/test_pipeline.py
index 2d8425a..214b3c9 100644
--- a/tests/test_pipeline.py
+++ b/tests/test_pipeline.py
@@ -79,3 +79,48 @@ def test_cli_exit_codes(tmp_path):
src = _clip(tmp_path / "src.mp4")
assert main([str(src), "-o", str(tmp_path / "o.mp4"), "--backend", "fake"]) == 0
assert main([str(tmp_path / "nope.mp4"), "-o", str(tmp_path / "o.mp4"), "--backend", "fake"]) == 2
+
+
+def test_max_res_caps_output_not_inference(tmp_path):
+ """`--max-res` must not shrink what the model sees (0.3.2 regression lock: a 320-px cap
+ used to feed the model a thumbnail that it upsampled internally)."""
+ src = _clip(tmp_path / "src.mp4", frames=4, w=1280, h=736)
+ p = plan(RunConfig(input=src, output=tmp_path / "o.mp4", backend="fake", target="h3", max_res=320))
+ assert p.proc_h == 518 and p.proc_w == 900 # model resolution
+ assert (p.out_w, p.out_h) == (320, 160) # output capped, multiple of 32
+
+
+def test_cli_batch_loads_model_once_and_skips_bad_clip(tmp_path, monkeypatch):
+ """`reshot a.mp4 b.mp4 missing.mp4 -o dir/`: one backend for the whole batch, per-clip
+ outputs named after the inputs, a bad clip reported and skipped, exit code of the failure."""
+ import reshot.pipeline as pl
+ from reshot.cli import main
+
+ a, b = _clip(tmp_path / "a.mp4"), _clip(tmp_path / "b.mp4", frames=8)
+ calls = []
+ real = pl.get_backend
+
+ def counting(name, **kw):
+ calls.append(name)
+ return real(name, **kw)
+
+ monkeypatch.setattr(pl, "get_backend", counting)
+ out = tmp_path / "depth"
+ code = main(
+ [
+ str(a),
+ str(b),
+ str(tmp_path / "missing.mp4"),
+ "-o",
+ str(out),
+ "--backend",
+ "fake",
+ "--metrics",
+ str(tmp_path / "m"),
+ ]
+ )
+ assert code == 2 # InputError for the missing clip
+ assert calls == ["fake"] # one model for three clips
+ assert (out / "a_depth.mp4").exists() and (out / "b_depth.mp4").exists()
+ assert (tmp_path / "m" / "a.json").exists() and (tmp_path / "m" / "b.json").exists()
+ assert not (out / "missing_depth.mp4").exists()
diff --git a/tests/test_postprocess.py b/tests/test_postprocess.py
index 670a4a9..f7f99dc 100644
--- a/tests/test_postprocess.py
+++ b/tests/test_postprocess.py
@@ -65,3 +65,22 @@ def test_h3_preset_matches_published_constraints():
def test_seedance_preset():
t = TARGETS["seedance"]
assert (t.fps, t.multiple, t.max_seconds, t.min_pixels) == (24.0, 16, 15.0, 407_696)
+
+
+def test_to_gray_matches_whole_array_reference_bit_for_bit():
+ """to_gray converts frame by frame to save memory; the bytes must equal the
+ straightforward whole-array formula for every option combination."""
+ rng = np.random.default_rng(0)
+ d = rng.random((6, 40, 64), dtype=np.float32) * 10
+
+ def ref(x, invert=False, gamma=1.0):
+ lo, hi = float(x.min()), float(x.max())
+ n = np.clip((x - lo) / (hi - lo), 0.0, 1.0)
+ if gamma != 1.0:
+ n = n ** float(gamma)
+ if invert:
+ n = 1.0 - n
+ return (n * 255.0 + 0.5).astype(np.uint8)
+
+ for kw in ({}, {"gamma": 1.6}, {"invert": True}, {"gamma": 0.8, "invert": True}):
+ assert np.array_equal(to_gray(d, **kw), ref(d, **kw)), kw