ReShot — copy the shot, not the actors. Video → depth map, OpenPose skeleton or canny lines for Seedance 2.0/2.5 reference video, MiniMax H3 Fun ControlNet and Wan VACE. Apache-2.0. Open-sourced by Maosika 猫斯卡 (www.maosika.com), the AI short-drama production system.
Python ai-short-dramaai-videocannycomfyuicontrolnetdepth-anythingdepth-estimationdwposemaosikaminimaxopenposepose-controlnetpose-estimationseedanceshort-dramavideo-generationvideo-to-videowan
0.3.4: 8 GB GPUs get input-size 364 automatically (measured on a 3080 Ti); GPU peak in metrics; 'What you need' table
wangfeng1ddb763c5dae
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 33bcb35..b9ccd9f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,14 @@# Changelog+## 0.3.4 — 2026-09-12+- **Small GPUs work out of the box.** Measured on an RTX 3080 Ti: the default `--input-size 518`+ peaks at 7.4 GiB allocated / 10.9 GiB reserved and OOMs on an 8 GB card; 364 peaks at+ 2.25 / 3.0 GiB. Cards under 11.5 GB now get 364 automatically (reported as a `vram` step,+ recorded as `input_size_used` in metrics); an explicit `--input-size` is always respected.+- Metrics gain `gpu_peak_allocated_bytes` / `gpu_peak_reserved_bytes`; hidden+ `--cuda-memory-fraction` emulates a smaller card for verification.+- README: a "What you need" table with the measured numbers.+## 0.3.3 — 2026-09-12- Memory estimate is now a measured line, not a guess: peak RSS = 2.0 GiB + 20 B × frames ×processing pixels (fitted on six RTX 4090 runs, base 1.73 GiB / slope 16.9 B, residualdiff --git a/README.md b/README.mdindex 0884914..60ad24d 100644--- a/README.md+++ b/README.md@@ -98,6 +98,18 @@ Things we learned making the demo:- **Change the species, keep the size ratio.** The bear take works because the prompt says the bear is "about 1.3× the rabbit, never more than 1.5×". The depth map already says who is bigger; the prompt must not contradict it.- **Plain clothes on extras, no logos.** Whatever the prompt leaves open, the model fills with text and badges.+## What you need++| | Runs? | Measured |+|---|---|---|+| **NVIDIA, 12 GB or more** (RTX 3080 Ti, 4070, 4090 …) | Yes, full quality | 4090: 62 ms/frame, 11 GB VRAM peak · 3080 Ti: 83 ms/frame |+| **NVIDIA, 8 GB** (RTX 3070, 4060 …) | Yes — the tool switches to `--input-size 364` by itself | 3 GB VRAM, 34 ms/frame on a 3080 Ti; depth is slightly softer |+| **Apple Silicon** | Yes, slower | M2 Max: ~500 ms/frame, a 12 s clip in about 2.5 min |+| **CPU only** | Yes, slow | ~1.8 s/frame |+| **Host RAM** | 16 GB covers clips up to ~27 s at 720p | peak = 2 GB + 224 MB per second of 720p; the tool refuses before starting if a clip won't fit |++Verified on rented cards on 2026-09-12; the numbers are in `--metrics` output of those runs.+## Presets| `--target` | fps | frame size | length | for |@@ -124,7 +136,7 @@ Three details make the output something a video model will actually follow:- **Frames picked by timestamp.** 30 fps → 24 fps really is 24; nothing is duplicated or dropped in a pattern the model could learn.- **Cropped, never padded.** Frame size is trimmed to the model's grid. A black border would read as a far wall.-The model is Video Depth Anything Small (ByteDance, CVPR 2025). It works on overlapping 32-frame windows and aligns them, so depth doesn't jitter between frames. A 12-second 720p clip peaks at 3.9 GB of host RAM (measured; the estimate the tool shows before starting is a fitted line, within 0.05 GB of measurements) and it refuses up front if a clip won't fit. A `fake` backend runs the whole pipeline without a model for your own tests. Errors that are yours to fix are `ReshotError` subclasses with an exit code and a concrete fix in the message.+The model is Video Depth Anything Small (ByteDance, CVPR 2025). It works on overlapping 32-frame windows and aligns them, so depth doesn't jitter between frames. A 12-second 720p clip peaks at 3.9 GB of host RAM and 11 GB of VRAM (3 GB with `--input-size 364`) (measured; the estimate the tool shows before starting is a fitted line, within 0.05 GB of measurements) and it refuses up front if a clip won't fit. A `fake` backend runs the whole pipeline without a model for your own tests. Errors that are yours to fix are `ReshotError` subclasses with an exit code and a concrete fix in the message.## Licensediff --git a/README.zh-CN.md b/README.zh-CN.mdindex 8ad99cc..8c9688f 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -98,6 +98,18 @@ is transferred onto <Subject 1> and <Subject 2>; its grey depth look is not tran- **换物种可以,体量比例别动。** 熊那一段能成,是因为提示词写了熊「约为兔子的 1.3 倍,不超过 1.5 倍」。谁大谁小深度图已经定了,提示词不能跟它打架。- **配角穿素色衣服,不留标志。** 提示词没写死的地方,模型会自己往上填字和徽章。+## 什么机器能跑++| | 能不能跑 | 实测 |+|---|---|---|+| **N 卡 12 GB 及以上**(RTX 3080 Ti、4070、4090……) | 能,全画质 | 4090:62 ms/帧、显存峰值 11 GB;3080 Ti:83 ms/帧 |+| **N 卡 8 GB**(RTX 3070、4060……) | 能——工具自动切到 `--input-size 364` | 显存 3 GB、3080 Ti 上 34 ms/帧;深度图略软 |+| **Apple 芯片** | 能,慢 | M2 Max 约 500 ms/帧,12 秒的片约 2.5 分钟 |+| **纯 CPU** | 能,很慢 | 约 1.8 秒/帧 |+| **内存** | 16 GB 能跑 720p 约 27 秒 | 峰值 = 2 GB + 每秒 720p 约 224 MB;放不下会在开始之前拒绝 |++2026-09-12 在租来的显卡上实测;数字都在那几次运行的 `--metrics` 输出里。+## 预设| `--target` | fps | 尺寸 | 时长 | 给谁用 |@@ -124,7 +136,7 @@ run(RunConfig(input=Path("参考片.mp4"), output=Path("深度图.mp4"), target=- **按时间戳选帧。** 30 fps 转 24 fps 就真的是 24,不会以某种规律重复或丢帧让模型学了去。- **只裁不补。** 尺寸裁到模型要的网格上,绝不补黑边——黑边会被当成远处的墙。-模型是 Video Depth Anything Small(字节跳动,CVPR 2025),以 32 帧为一个窗口重叠推理并对齐,深度不会一帧一帧地跳。12 秒 720p 一段片峰值占 3.9 GB 内存(实测;开跑前显示的估算是拟合出来的直线,与实测误差 0.05 GB 以内),放不下会在开始之前拒绝。`fake` 后端不加载模型就能跑完整条流水线,方便你写测试。属于「用户该修」的错误都是 `ReshotError` 的子类,带退出码和具体的修法。+模型是 Video Depth Anything Small(字节跳动,CVPR 2025),以 32 帧为一个窗口重叠推理并对齐,深度不会一帧一帧地跳。12 秒 720p 一段片峰值占 3.9 GB 内存、11 GB 显存(`--input-size 364` 时 3 GB)(实测;开跑前显示的估算是拟合出来的直线,与实测误差 0.05 GB 以内),放不下会在开始之前拒绝。`fake` 后端不加载模型就能跑完整条流水线,方便你写测试。属于「用户该修」的错误都是 `ReshotError` 的子类,带退出码和具体的修法。## 协议diff --git a/demo/requirements.txt b/demo/requirements.txtindex 9eb0827..4caf100 100644--- a/demo/requirements.txt+++ b/demo/requirements.txt@@ -1,5 +1,5 @@--extra-index-url https://download.pytorch.org/whl/cputorchtorchvision-reshot[ffmpeg] @ git+https://github.com/maosika-ai/[email protected]+reshot[ffmpeg] @ git+https://github.com/maosika-ai/[email protected]gradio>=4.0diff --git a/reshot/_version.py b/reshot/_version.pyindex e19434e..334b899 100644--- a/reshot/_version.py+++ b/reshot/_version.py@@ -1 +1 @@-__version__ = "0.3.3"+__version__ = "0.3.4"diff --git a/reshot/pipeline.py b/reshot/pipeline.pyindex a4e8b77..6002758 100644--- a/reshot/pipeline.py+++ b/reshot/pipeline.py@@ -213,8 +213,9 @@ def _run_one(cfg: RunConfig, rep: Reporter, backend, backend_name: str) -> RunRe"model",f"{backend.info.name}/{backend.info.variant} on {backend.device} ({precision}), licence {backend.info.license}",)+ input_size = _fit_input_size_to_vram(cfg.input_size, backend.device, rep)t1 = time.time()- depths = backend.infer(frames, fps, input_size=cfg.input_size)+ depths = backend.infer(frames, fps, input_size=input_size)depth_seconds = time.time() - t1del framesgpu_peak = backend.peak_memory_bytes()@@ -257,6 +258,7 @@ def _run_one(cfg: RunConfig, rep: Reporter, backend, backend_name: str) -> RunRe"precision": precision,"frames": t,"fps": fps,+ "input_size_used": input_size,"depth_seconds": round(depth_seconds, 3),"ms_per_frame": round(depth_seconds / t * 1000, 1),"total_seconds": round(total, 2),@@ -305,6 +307,43 @@ def extract(return backend.infer(frames, fps, input_size=input_size), fps+#: Measured 2026-09-12 on an RTX 3080 Ti (12 GB), 294 frames at 518×900: input_size 518 peaks at+#: 7.4 GiB allocated / 10.9 GiB reserved and dies with "tried to allocate 4.02 GiB" when the+#: card is capped at 8 GB; input_size 364 peaks at 2.25 / 3.0 GiB and runs with a 6 GB cap.+#: So cards below this line get 364 by default instead of an OOM traceback.+VRAM_FULL_SIZE_MIN_BYTES = int(11.5 * 2**30)+VRAM_SMALL_INPUT_SIZE = 364+++def _cuda_total_bytes(device: str) -> int:+ """Total memory of the CUDA device in use, 0 for anything else. Split out so tests can fake it."""+ if device != "cuda":+ return 0+ try:+ import torch++ return int(torch.cuda.get_device_properties(torch.cuda.current_device()).total_memory)+ except Exception:+ return 0+++def _fit_input_size_to_vram(input_size: int, device: str, rep: Reporter) -> int:+ """Drop the model's working size on small GPUs (see VRAM_FULL_SIZE_MIN_BYTES).++ Only the default is touched: a user who passed `--input-size` explicitly above 364+ on a small card asked for it and gets the OOM (and the message that comes with it).+ """+ total = _cuda_total_bytes(device)+ if total and total < VRAM_FULL_SIZE_MIN_BYTES and input_size == 518:+ rep.step(+ "vram",+ f"{gib(total)} GPU: using --input-size {VRAM_SMALL_INPUT_SIZE} (518 needs ~11 GB; "+ f"364 needs ~3 GB and is 2× faster, slightly softer depth)",+ )+ return VRAM_SMALL_INPUT_SIZE+ return input_size++def _license_ok(metrics: dict) -> bool:return metrics["config"]["model"] == "small" or metrics["config"]["backend"] == "fake"diff --git a/tests/test_pipeline.py b/tests/test_pipeline.pyindex 214b3c9..43dab95 100644--- a/tests/test_pipeline.py+++ b/tests/test_pipeline.py@@ -124,3 +124,22 @@ def test_cli_batch_loads_model_once_and_skips_bad_clip(tmp_path, monkeypatch):assert (out / "a_depth.mp4").exists() and (out / "b_depth.mp4").exists()assert (tmp_path / "m" / "a.json").exists() and (tmp_path / "m" / "b.json").exists()assert not (out / "missing_depth.mp4").exists()+++def test_small_gpu_drops_default_input_size_but_respects_explicit(tmp_path, monkeypatch):+ """An 8 GB card gets input_size 364 instead of an OOM; an explicit --input-size is kept."""+ import reshot.pipeline as pl++ monkeypatch.setattr(pl, "_cuda_total_bytes", lambda device: 8 * 2**30)+ steps = []++ class Rep:+ def step(self, tag, msg):+ steps.append((tag, msg))++ assert pl._fit_input_size_to_vram(518, "cuda", Rep()) == 364+ assert steps and steps[0][0] == "vram"+ assert pl._fit_input_size_to_vram(518 + 14, "cuda", Rep()) == 532 # explicit, left alone+ monkeypatch.setattr(pl, "_cuda_total_bytes", lambda device: 24 * 2**30)+ assert pl._fit_input_size_to_vram(518, "cuda", Rep()) == 518+ assert pl._fit_input_size_to_vram(518, "cpu", Rep()) == 518