Skip to content

ReShot — copy the shot, not the actors. Video → depth map, OpenPose skeleton or canny lines for Seedance 2.0/2.5 reference video, MiniMax H3 Fun ControlNet and Wan VACE. Apache-2.0. Open-sourced by Maosika 猫斯卡 (www.maosika.com), the AI short-drama production system.

9.3 KB
Raw Blame

Changelog

0.5.0 — 2026-09-14

  • Skeleton output — --control pose (closes #6). DWPose (YOLOX-L person detector + RTMPose whole-body estimator, Apache-2.0 code and weights) in ONNX Runtime, drawn in the OpenPose layout the video ControlNets were trained on: 18 body joints + 21-point hands, faces off by default. Across frames: identity tracking (IoU), visibility hysteresis (on > 0.3, off < 0.2) and a One-Euro filter per joint, so the skeleton does not flicker. --keypoints x.json writes every skeleton (reshot-pose/1: stable id per person, 134 points + scores per frame). pip install "reshot[pose]"; weights (~340 MB) download once from yzd-v/DWPose, or --checkpoint-dir. Measured: 306 ms/frame on an M2 Max CPU, the 12 s demo in 90 s.
  • --pose-detect-every N (default 3). The detector is 80 % of the pose cost, so it runs every third frame and the previous skeletons supply the boxes in between; a cut or an unreliable skeleton re-detects at once. Against every-frame detection on the 289-frame demo: 280 frames identical people count, 80 % of joints within 2 px, 91 % within 5 px, 35 % faster.
  • Line output — --control canny, thresholds --canny LOW,HIGH. No model.
  • --control depth,pose,canny writes all three in one go (-o a directory); one model load per kind.
  • Web page: a third choice, What to make — depth map / skeleton / lines; quality and the advanced fields show only for the type they apply to; the result panel names the file, offers the keypoints JSON for skeletons, and prints the prompt line for your target × type.
  • Python: write_rgb_video, reshot.pose (PoseClip, skeleton.render_frame, tracking.stabilise), reshot.edges.canny_frame; RunConfig.control and the pose / canny fields.
  • Colab notebook and docs/AGENT_INSTALL.md cover the new types.
  • Pose and canny are read at the output size (no "model resolution"), with their own, much smaller host-RAM estimate.
  • Verified on MiniMax H3: the three demo takes re-made with the skeleton as <Video 1> (same prompts and sheets) reproduce the six beats like the depth takes — docs/demo-pose.gif, prompts with seeds in docs/prompts/*_pose.txt.
  • Verified on MiniMax H3: the three demo takes re-made with the skeleton as <Video 1> (same prompts and sheets) reproduce the six beats like the depth takes; prompts + seeds in docs/prompts/*_pose.txt, comparison in docs/img/pose_vs_depth_takes.jpg.

0.4.1 — 2026-09-14

  • Colab notebook. examples/reshot_colab.ipynb — a free GPU, upload a clip, download depth.mp4, no install. "Open in Colab" badge in both READMEs.
  • run(cfg, backend=...) accepts a pre-built backend, so a resident process (a server, a Space) loads the weights once instead of per clip. Without it nothing changes.
  • demo/space/: a ZeroGPU Gradio app, ready for a Hugging Face account with billing enabled (Gradio Spaces are no longer free to host). demo/landing/: the static Space.
  • Discussions are open; the README has a Community section.

0.4.0 — 2026-09-12

  • Web page. reshot with no arguments (or reshot web) starts a local UI in the browser: drop a clip, pick the target model and quality, run, preview reference and depth side by side, download, copy the prompt line for Seedance / MiniMax H3. Studio Air look (the Maosika design system), Chinese and English. Standard library only — no new dependencies; listens on 127.0.0.1; results in ~/ReShot. Verified in Chrome with the real model.

0.3.8 — 2026-09-12

  • Fix: on Apple Silicon with torch ≥ 2.14 the model failed to load with invalid low watermark ratio 1.4 — the MPS low watermark is now set alongside the high one.
  • docs/AGENT_INSTALL.md: a step-by-step install guide written for AI coding agents, and a copy-paste prompt in the README that points them at it.

0.3.7 — 2026-09-12

  • On PyPI: pip install reshot (and uvx reshot …). No git needed any more.

0.3.6 — 2026-09-12

  • --quality fast is the default (the model sees 644×364 for 16:9, 364×644 for 9:16; ~3 GB VRAM, 34 ms/frame on a 3080 Ti). full (924×518 / 518×924, ~11 GB) is the user's choice; asking for it on a card under 11.5 GB prints a warning first. auto is gone.
  • The run prints and records (model_input_resolution in metrics) the exact resolution the model works at; model_input_resolution() is public.

0.3.5 — 2026-09-12

  • --quality auto|full|fast replaces the silent small-GPU fallback of 0.3.4: full is 518 px, fast 364 px, auto (default) picks by VRAM and says which it chose. --input-size stays as the expert override. Measured difference full vs fast on the same clip: mean 5.3/255 grey levels, 95 % of pixels within 15, edge energy −4.5 % — fine detail only.

0.3.4 — 2026-09-12

  • Small GPUs work out of the box. Measured on an RTX 3080 Ti: the default --input-size 518 peaks at 7.4 GiB allocated / 10.9 GiB reserved and OOMs on an 8 GB card; 364 peaks at 2.25 / 3.0 GiB. Cards under 11.5 GB now get 364 automatically (reported as a vram step, recorded as input_size_used in metrics); an explicit --input-size is always respected.
  • Metrics gain gpu_peak_allocated_bytes / gpu_peak_reserved_bytes; hidden --cuda-memory-fraction emulates a smaller card for verification.
  • README: a "What you need" table with the measured numbers.

0.3.3 — 2026-09-12

  • Memory estimate is now a measured line, not a guess: peak RSS = 2.0 GiB + 20 B × frames × processing pixels (fitted on six RTX 4090 runs, base 1.73 GiB / slope 16.9 B, residual ≤ 0.05 GiB, padded ~15 %). The old flat 32 B/pixel-frame over-refused long clips and under-estimated short ones; --max-frames suggestions now account for the base.
  • Measured on the same runs: 61–62 ms/frame on a 4090 for clips ≥ 190 frames (77–92 ms/frame for very short clips, where model load and the first window dominate).

0.3.2 — 2026-09-12

  • Fix: --max-res no longer lowers the inference resolution. It capped the processing size too, so --max-res 320 fed the model a thumbnail; inference now always runs at model resolution and --max-res only sizes the output (regression test added).
  • Batch mode: reshot a.mp4 b.mp4 … -o dir/ (or any -o that is a directory) runs every clip on one model load; --metrics/--npz become per-clip directories; a failed clip is skipped and reported. Python: run_many().
  • Memory: to_gray converts frame by frame (−1.2 GB peak for a 12 s clip, byte-identical output); decoding writes into one preallocated buffer instead of list + np.stack (−400 MB).
  • --npz / --metrics create their parent directories. torch.load(weights_only=True); inference under torch.inference_mode(); extract() honours RESHOT_FAKE_BACKEND.

0.3.1 — 2026-09-12

  • Fix: the package failed to import on Windows since 0.2.0 (import resource is Unix-only); peak-memory metric now uses the Win32 counter there.
  • README rewritten as a manual: the problem → what it does → four steps with the real prompts and character sheets from the demo (docs/prompts/, docs/refs/) → what transfers and what doesn't → presets → developers.
  • New demo: a corridor fight → depth map → three generated takes (two humans, one bear), docs/demo-fight.gif + docs/demo-fight.mp4; social preview card updated.
  • Wording: the output is called what it is — a depth map (relative inverse depth, monocular video depth estimation), no longer a "blockout" (that term means untextured 3D geometry). Docs, CLI help, Space page and demo labels updated; demo.make_blockout → make_depth_map.

0.3.0 — 2026-09-12

  • Renamed: depthcat → ReShot. Package, CLI, env var and error class follow: pip install git+https://github.com/maosika-ai/reshot, reshot in.mp4 -o out.mp4, RESHOT_FAKE_BACKEND, reshot.ReshotError. No behaviour change.
  • README/Space copy: who makes it — Maosika 猫斯卡, the AI short-drama production system.

0.2.1 — 2026-09-11

  • --target seedance: 24 fps, frame size a multiple of 16, ≤ 15 s, and a warning when the frame falls below the 407,696-pixel floor — all from the official Seedance reference-video spec.
  • Target.min_pixels and the matching pipeline warning.
  • README rewritten as a product page (Seedance first); Seedance recipe in the usage guide; demo default preset is now seedance.

0.2.0 — 2026-09-11

  • One pipeline: reshot.run(RunConfig) is the single code path for the CLI and the Python API.
  • Backend registry (vda, fake); --backend fake runs the whole pipeline without a model.
  • Typed errors with exit codes (2 input · 3 RAM budget · 4 ffmpeg · 5 weights) and concrete fixes in the message.
  • RunConfig validates arguments before any work; plan() exposes what a run will do.
  • Metrics JSON now records the full config and plan.
  • Lint (ruff) in CI; 28 tests without a model plus a real-model CPU smoke job.

0.1.0 — 2026-09-11

  • First release: reshot in.mp4 -o out.mp4 on Video Depth Anything Small (Apache-2.0).
  • Depth estimated at model resolution, whole-clip normalisation, near = white, x264 CRF 12.
  • --target h3 / wan presets (fps, size multiple, max length), --npz, --metrics.
  • Host-RAM estimate with refusal over 50 % of physical RAM; MPS allocator cap on Apple Silicon.
  • Measured: RTX 4090 fp16 70 ms/frame; Apple M2 Max fp32 ~500 ms/frame.