ReShot — copy the shot, not the actors. Video → depth map, OpenPose skeleton or canny lines for Seedance 2.0/2.5 reference video, MiniMax H3 Fun ControlNet and Wan VACE. Apache-2.0. Open-sourced by Maosika 猫斯卡 (www.maosika.com), the AI short-drama production system.
Python ai-short-dramaai-videocannycomfyuicontrolnetdepth-anythingdepth-estimationdwposemaosikaminimaxopenposepose-controlnetpose-estimationseedanceshort-dramavideo-generationvideo-to-videowan
docs: README as a manual — problem, what it does, four steps with the real demo prompts and sheets
wangfeng2bafb340fc4b
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a711777..d235496 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,6 +1,7 @@# Changelog## 0.3.1 — 2026-09-12+- README rewritten as a manual: the problem → what it does → four steps with the real prompts and character sheets from the demo (`docs/prompts/`, `docs/refs/`) → what transfers and what doesn't → presets → developers.- New demo: a corridor fight → depth map → three generated takes (two humans, one bear), `docs/demo-fight.gif` + `docs/demo-fight.mp4`; social preview card updated.- Wording: the output is called what it is — a **depth map** (relative inverse depth, monocularvideo depth estimation), no longer a "blockout" (that term means untextured 3D geometry).diff --git a/README.md b/README.mdindex 18d52fa..6654b68 100644--- a/README.md+++ b/README.md@@ -1,9 +1,9 @@<h1 align="center">ReShot</h1><p align="center"><b>Copy the shot, not the actors.</b></p>-<p align="center"><sub>Open-source depth-map tool from <a href="https://www.maosika.com">Maosika 猫斯卡</a>, the AI short-drama production system.</sub></p>+<p align="center">ReShot turns a reference video into a depth map, so Seedance or MiniMax H3 can repeat its choreography and camera moves — with your own characters in it.</p><p align="center"><img src="docs/demo-fight.gif" width="720" alt="a fight scene, its depth map, and three new takes generated from it"></p>-<p align="center"><sub>A fight scene. Its depth map. Three new takes — different fighters (one of them a bear), same choreography, same camera. <a href="docs/demo-fight.mp4">Full-resolution clip</a>.</sub></p>+<p align="center"><sub>Top: the reference and its depth map. Bottom: three takes generated from that depth map — two women, one rabbit. Same moves, same camera. <a href="docs/demo-fight.mp4">Full-resolution clip</a>.</sub></p><p align="center"><a href="README.zh-CN.md">中文</a> · <a href="docs/USAGE.md">User guide</a> · <a href="https://huggingface.co/spaces/maosika/reshot">Hugging Face</a> · <a href="CHANGELOG.md">Changelog</a></p><p align="center">@@ -14,81 +14,102 @@----## Every shot is two things.+## The problem-Where everyone stands. How big they are. The way they move. The way the camera moves.-That's the **staging** — and it's what makes a great shot great.+You have a clip whose fight, dance or camera move is exactly what you want in your own AI video. There are two ways to get it, and both fail:-Then there's who they are, what they wear, how the light falls. That's the **look**.+- **Feed the clip to the video model as a reference.** It copies the faces, the clothes and the look along with the moves. If the clip has real people in it, the platform's content check may refuse it outright.+- **Describe the moves in words.** "She kicks off the wall, grabs a pipe, throws the big guy over her shoulder" — the model gives you a different fight every time, and the camera never does what you said.-Until now you couldn't take one without the other. Describe the staging in words and the-model gives you something else every time. Hand it the footage and it takes the faces,-the wardrobe, the style — everything you didn't ask for.+## What ReShot does-## Keep the choreography. Change everything else.+ReShot takes an `.mp4` in and writes an `.mp4` out. The output is a **depth map video**: every frame is grey, near things are white, far things are black. It keeps where everyone stands, how big they are, how they move and how the camera moves. It throws away faces, clothes, lighting and style.-ReShot turns a reference clip into a depth map: a clean, silent record of the-staging and nothing else. Near is white, far is black, every frame in step with the last.-No faces. No costumes. No style. Just the shot.+<p align="center"><img src="docs/img/step1_reference.jpg" width="360" alt="reference frame"> <img src="docs/img/step2_depth.jpg" width="360" alt="the same frame as a depth map"></p>-Technically: monocular video depth estimation. The model predicts relative inverse depth-for every frame; ReShot normalises it once over the whole clip to 8-bit grey (near = white)-and encodes it as a standard depth-map video.+You give that grey video to your video model as the reference and describe the people and the look in the prompt. The model takes the moves from the video and everything else from your words.-Give the depth map to your video model along with a prompt for the look, and you get the-reference's blocking and camera with your cast in it.+Technically: monocular video depth estimation. The model predicts relative inverse depth for every frame; ReShot normalises it once over the whole clip to 8-bit grey (near = white) and encodes it as a standard depth-map video.++## How to use it++The three takes in the demo were made exactly this way. Every file involved is in this repo, so you can repeat it.++### 1. Install++```bash+pip install git+https://github.com/maosika-ai/reshot # needs ffmpeg on PATH+```++No ffmpeg? `pip install "reshot[ffmpeg] @ git+https://github.com/maosika-ai/reshot"` bundles one. The model weights (111 MB) download on first run.++### 2. Make the depth map```bash-pip install git+https://github.com/maosika-ai/reshotreshot reference.mp4 -o depth.mp4 --target seedance```-That's the whole thing.+`--target seedance` sets 24 fps, H.264, a frame size that is a multiple of 16 and at least 407,696 pixels, up to 15 seconds — the reference-video rules of the Seedance API. For MiniMax H3 use `--target h3` (multiples of 32). On an RTX 4090 a 12-second clip takes about 20 seconds; on a MacBook a few minutes.-## Works with the models you already use.+### 3. Give it to the video model-**Seedance 2.0 and 2.5.** Add the depth map as a reference video and ask for its motion-and camera. The depth map meets the API's reference-video requirements out of the box —-24 fps, H.264, sized for the model — and because it carries no likeness, it passes the-content checks that stop real footage.+**Seedance 2.0 / 2.5.** Upload `depth.mp4` as a reference video. In the prompt, point at it and describe the people and the look:```-参考@视频1的动作与运镜。两名武者在雨夜屋顶对决,黑色劲装,冷蓝月光,电影感。+参考@视频1的动作与运镜,顺序与视频保持一致。+一名穿深绿色丝绒旗袍的女子在狭窄的金属走廊里与三名黑衣守卫搏斗,冷蓝走廊光,红色警示灯,电影感。```-**MiniMax H3 Fun ControlNet.** Use the depth map as the depth condition. 24 fps, frame size-a multiple of 32, up to 15 seconds — the preset handles it.+**MiniMax H3.** Attach `depth.mp4` as `<Video 1>`. If you want a specific face, attach a character sheet as `<Picture 1>`. These are the three sheets used for the demo:-**Wan VACE, and any depth ControlNet.** It's a standard near-white depth video. If your-model reads depth, it reads this.+<p align="center"><img src="docs/img/step3_sheets.jpg" width="720" alt="the three character sheets used as Picture 1"></p>-| `--target` | fps | frame size | length | |-|---|---|---|---|---|-| `seedance` | 24 | ×16, ≥ 407,696 px | ≤ 15 s | Seedance 2.0 / 2.5 reference video |-| `h3` | 24 | ×32 | ≤ 15 s | MiniMax H3 Fun ControlNet |-| `wan` | 16 | ×16 | – | Wan 2.1 VACE |-| `none` | source | even | – | anything |+MiniMax H3 wants its prompt in a fixed six-section format. The full prompts for all three takes are in [`docs/prompts/`](docs/prompts/). The part that does the work is how `<Video 1>` is defined and what it is allowed to transfer:++```+<Subject 3> is the fight choreography and camera movement shown in <Video 1>, a grey depth map+in which near objects are white and far objects are black: one fighter leans on a corridor wall+in close-up, kicks off it to tear down a pipe, fights several opponents, is grabbed from behind+by the largest and throws him, slams the last one into a wall panel, wipes the mouth in close-up,+then walks away through a door past the fallen opponents.++<Subject 3>: attribute_transfer - every action, position, timing and camera move of <Video 1>+is transferred onto <Subject 1> and <Subject 2>; its grey depth look is not transferred.+```++Two things in there matter. **Say in words what happens in the grey clip** — the model reads the depth map far better when the prompt tells it what the blobs are doing. **Say that the grey look is not to be copied**, or you may get a grey film back.++### 4. What comes out++<p align="center"><img src="docs/img/step4_takes.jpg" width="720" alt="the three takes"></p>-## Steady by design.+Left to right: [Jiang Xue](docs/prompts/take1_jiangxue_armor.txt) in bronze armour, [Su Wan](docs/prompts/take2_suwan_qipao.txt) in a green velvet qipao, and a [rabbit boxer](docs/prompts/take3_rabbit_boxer.txt) against a wolf, a tiger and a bear as a 3D animated feature. Same six shots, same close-up at the start, same walk out through the door at the end. The reference clip itself was a MiniMax H3 text-to-video render, so nobody's likeness was involved at any step.-A depth map is only useful if the model trusts it. So ReShot is built around the details-that make a depth video hold still.+## What transfers, and what doesn't-- **It sees the whole clip.** The model works on overlapping windows of 32 frames and- aligns them, so depth doesn't jitter from frame to frame.-- **One scale for the entire shot.** Brightness means distance, and it means the same- distance in frame 1 and frame 300. A wall doesn't pulse because someone walked past it.-- **Nothing the model didn't ask for.** Frames are picked by timestamp, so 24 fps is- really 24. Frame size is trimmed to the model's grid, never padded with black. The- encode is clean enough that the depth gradient has no banding.+**Transfers:** who stands where, how big they are relative to each other, every move and its timing, the cuts, and the camera — push-ins, tracking, handheld shake.-## Made for the pace of a studio.+**Doesn't:** faces (use a character sheet), clothes, lighting, colour, props in detail, and anything smaller than a hand. Those come from your prompt and your reference images.-A 12-second shot takes about 20 seconds on an RTX 4090 — 70 ms a frame, 31 with the-fast setting. It runs on a MacBook. It needs about 4 GB of memory for a 720p clip, and it-tells you before it starts if a clip won't fit.+Things we learned making the demo:-Batch a folder of references in a shell loop, or call it from Python:+- **Keep the reference under 15 seconds** (Seedance's limit) and cut it to the shots you want before running ReShot. Everything in the clip gets copied, including the boring part at the end.+- **For MiniMax H3, make the depth map small: `--target h3 --max-res 320`** (that is 320×176 for a 16:9 clip). A full-size grey silhouette starts to pull the character's face shape towards the person in the reference; a small one carries the moves without the shape.+- **Change the species, keep the size ratio.** The bear take works because the prompt says the bear is "about 1.3× the rabbit, never more than 1.5×". The depth map already says who is bigger; the prompt must not contradict it.+- **Plain clothes on extras, no logos.** Whatever the prompt leaves open, the model fills with text and badges.++## Presets++| `--target` | fps | frame size | length | for |+|---|---|---|---|---|+| `seedance` | 24 | ×16, ≥ 407,696 px | ≤ 15 s | Seedance 2.0 / 2.5 reference video |+| `h3` | 24 | ×32 | ≤ 15 s | MiniMax H3 (reference video or Fun ControlNet depth) |+| `wan` | 16 | ×16 | – | Wan 2.1 VACE |+| `none` | source | even | – | anything that reads a depth video |++`reshot --help` lists every option; the [user guide](docs/USAGE.md) explains them.++## For developers```pythonfrom pathlib import Path@@ -97,26 +118,19 @@ from reshot import RunConfig, runrun(RunConfig(input=Path("reference.mp4"), output=Path("depth.mp4"), target="seedance"))```-## Yours to ship.+Three details make the output something a video model will actually follow:-ReShot is Apache-2.0, and so is the model it runs by default — Video Depth Anything-Small, from ByteDance (CVPR 2025), vendored under `reshot/third_party/`. Use it in a-product, a pipeline, a service. The larger research-only variants exist and are clearly-labelled; they never load unless you ask.+- **One scale for the whole clip.** Depth is normalised once over all frames, never per frame, so a wall keeps the same grey when someone walks past it. Per-frame normalisation makes the scene "breathe".+- **Frames picked by timestamp.** 30 fps → 24 fps really is 24; nothing is duplicated or dropped in a pattern the model could learn.+- **Cropped, never padded.** Frame size is trimmed to the model's grid. A black border would read as a far wall.-## Get started+The model is Video Depth Anything Small (ByteDance, CVPR 2025). It works on overlapping 32-frame windows and aligns them, so depth doesn't jitter between frames. It needs about 4 GB of RAM for a 720p clip and refuses up front if a clip won't fit. A `fake` backend runs the whole pipeline without a model for your own tests. Errors that are yours to fix are `ReshotError` subclasses with an exit code and a concrete fix in the message.-```bash-pip install git+https://github.com/maosika-ai/reshot # ffmpeg on PATH-pip install "reshot[ffmpeg] @ git+https://github.com/maosika-ai/reshot" # or bundle one-reshot --help-```+## License-Weights (111 MB) download on first run. Behind a firewall: `export HF_ENDPOINT=https://hf-mirror.com`.+Apache-2.0, and so is the default model. The vendored model code is under `reshot/third_party/`. Use it in a product, a pipeline, a service. The larger research-only weights (Base, Large) are CC-BY-NC and never load unless you ask for them.-Everything else — every option, presets, recipes for each model, troubleshooting — is in-the **[user guide](docs/USAGE.md)**. Issues and pull requests are welcome; see-[CONTRIBUTING.md](CONTRIBUTING.md).+Issues and pull requests are welcome; see [CONTRIBUTING.md](CONTRIBUTING.md).## About Maosika 猫斯卡diff --git a/README.zh-CN.md b/README.zh-CN.mdindex b4a6a55..f67a544 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -1,9 +1,9 @@<h1 align="center">ReShot</h1><p align="center"><b>复制走位,不复制演员。</b></p>-<p align="center"><sub>由 AI 短剧生产系统 <a href="https://www.maosika.com">猫斯卡</a> 开源的深度图工具。</sub></p>+<p align="center">ReShot 把一段参考视频变成深度图,让 Seedance 或 MiniMax H3 照着它的动作和运镜再拍一遍——人物换成你的。</p><p align="center"><img src="docs/demo-fight.gif" width="720" alt="一段武打参考片、它的深度图、用深度图生成的三段新武打片"></p>-<p align="center"><sub>一段武打。它的深度图。三段新片——换了人(其中一段换成了熊),招式和镜头一模一样。<a href="docs/demo-fight.mp4">高清原片</a>。</sub></p>+<p align="center"><sub>上排:参考片和它的深度图。下排:用这张深度图生成的三段片——两个女人、一只兔子。动作一样,镜头一样。<a href="docs/demo-fight.mp4">高清原片</a>。</sub></p><p align="center"><a href="README.md">English</a> · <a href="docs/USAGE.zh-CN.md">使用手册</a> · <a href="https://huggingface.co/spaces/maosika/reshot">Hugging Face</a> · <a href="CHANGELOG.md">更新日志</a></p><p align="center">@@ -14,70 +14,102 @@----## 一个镜头,其实是两件事。+## 你遇到的问题-谁站在哪。多大。怎么动。镜头怎么跟。-这是**走位**——一个好镜头之所以好,就好在这里。+你看中了一段片子:那场打戏、那段舞、那个运镜,正是你的 AI 视频想要的。想拿过来用,有两条路,都走不通:-然后才是他们是谁、穿什么、光怎么打。这是**长相**。+- **把片子直接当参考视频喂给模型。** 它会把脸、衣服、画风连同动作一起抄走。片子里如果是真人,平台审核可能直接拒收。+- **用文字描述动作。**「她蹬墙而起,扯下一根管子,把大个子过肩摔」——模型每次给你的打法都不一样,镜头更是从来不听话。-以前这两件事分不开。用文字描述走位,模型每次给你的都不一样;把片子直接喂进去,-它把人脸、服装、画风全拿走了——你根本没要这些。+## ReShot 做什么-## 招式留下,其余全换。+ReShot 吃进一个 `.mp4`,吐出一个 `.mp4`。输出是一段**深度图视频**:每一帧都是灰的,近的地方白,远的地方黑。它留下的是谁站在哪、谁大谁小、怎么动、镜头怎么走;扔掉的是脸、衣服、光线和画风。-ReShot 把一段参考片变成深度图:只记录走位、别的什么都没有。近处白、远处黑,-每一帧都和上一帧严丝合缝。没有脸,没有衣服,没有风格。只有这个镜头本身。+<p align="center"><img src="docs/img/step1_reference.jpg" width="360" alt="参考片的一帧"> <img src="docs/img/step2_depth.jpg" width="360" alt="同一帧的深度图"></p>-严格地说,这是单目视频深度估计:模型对每一帧预测相对逆深度,ReShot 对整段视频做一次归一化,-变成 8 位灰度(近白远黑),再编码成标准的深度图视频。+你把这段灰片当参考视频交给视频模型,提示词里写人物和画风。动作和镜头模型从灰片里读,其余全听你的。-把深度图交给视频模型,提示词里写你要的长相,出来的就是参考片的走位和运镜,演的是你的人。+严格地说,这是单目视频深度估计:模型对每一帧预测相对逆深度,ReShot 对整段视频做一次归一化,变成 8 位灰度(近白远黑),再编码成标准的深度图视频。++## 怎么用++演示里那三段片就是这么做出来的,用到的每一个文件都在这个仓库里,你可以照着复现。++### 1. 装++```bash+pip install git+https://github.com/maosika-ai/reshot # 需要 PATH 里有 ffmpeg+```++没有 ffmpeg 就装 `pip install "reshot[ffmpeg] @ git+https://github.com/maosika-ai/reshot"`,自带一个。模型权重 111 MB,首次运行自动下载;国内先 `export HF_ENDPOINT=https://hf-mirror.com`。++### 2. 抽深度图```bash-pip install git+https://github.com/maosika-ai/reshotreshot 参考片.mp4 -o 深度图.mp4 --target seedance```-就这么多。+`--target seedance` 会把输出设成 24 fps、H.264、边长 16 的倍数、不少于 407,696 像素、最长 15 秒——这就是 Seedance 接口对参考视频的要求。给 MiniMax H3 用 `--target h3`(边长 32 的倍数)。RTX 4090 上 12 秒的片约 20 秒跑完,MacBook 要几分钟。-## 你在用的模型,它都接得上。+### 3. 交给视频模型-**Seedance 2.0 和 2.5。** 把深度图当参考视频传进去,让它参考动作和运镜。-深度图出厂就符合官方参考视频的要求——24 fps、H.264、尺寸合规——而且因为它不含任何真人特征,-真人素材过不了的审核,它能过。+**Seedance 2.0 / 2.5。** 把 `深度图.mp4` 当参考视频上传,提示词里点名它,再写人物和画风:```-参考@视频1的动作与运镜。两名武者在雨夜屋顶对决,黑色劲装,冷蓝月光,电影感。+参考@视频1的动作与运镜,顺序与视频保持一致。+一名穿深绿色丝绒旗袍的女子在狭窄的金属走廊里与三名黑衣守卫搏斗,冷蓝走廊光,红色警示灯,电影感。```-**MiniMax H3 Fun ControlNet。** 深度图作为 depth 条件。24 fps、边长 32 的倍数、最长 15 秒,预设都替你算好了。+**MiniMax H3。** 把 `深度图.mp4` 挂成 `<Video 1>`。想要固定的脸,再把定妆图挂成 `<Picture 1>`。演示用的三张定妆图:-**Wan VACE 和任何深度 ControlNet。** 它就是标准的近白远黑深度视频。模型认深度,就认它。+<p align="center"><img src="docs/img/step3_sheets.jpg" width="720" alt="演示用的三张定妆图"></p>-| `--target` | fps | 尺寸 | 时长 | |-|---|---|---|---|---|-| `seedance` | 24 | 16 的倍数,≥ 407,696 像素 | ≤ 15 秒 | Seedance 2.0 / 2.5 参考视频 |-| `h3` | 24 | 32 的倍数 | ≤ 15 秒 | MiniMax H3 Fun ControlNet |-| `wan` | 16 | 16 的倍数 | – | Wan 2.1 VACE |-| `none` | 原片 | 偶数 | – | 通用 |+MiniMax H3 的提示词有固定的六段格式,三段片的完整提示词都在 [`docs/prompts/`](docs/prompts/)。真正起作用的是这两句——怎么定义 `<Video 1>`,允许它转移什么:++```+<Subject 3> is the fight choreography and camera movement shown in <Video 1>, a grey depth map+in which near objects are white and far objects are black: one fighter leans on a corridor wall+in close-up, kicks off it to tear down a pipe, fights several opponents, is grabbed from behind+by the largest and throws him, slams the last one into a wall panel, wipes the mouth in close-up,+then walks away through a door past the fallen opponents.++<Subject 3>: attribute_transfer - every action, position, timing and camera move of <Video 1>+is transferred onto <Subject 1> and <Subject 2>; its grey depth look is not transferred.+```++两个要点。**用文字把灰片里发生的事写一遍**——提示词告诉它那些灰影在干什么,模型读深度图会准得多。**明说灰色外观不要抄**,不然可能给你出一部灰片。++### 4. 出来的是什么++<p align="center"><img src="docs/img/step4_takes.jpg" width="720" alt="三段生成片"></p>-## 稳,是设计出来的。+从左到右:穿青铜甲的[江雪](docs/prompts/take1_jiangxue_armor.txt)、穿墨绿丝绒旗袍的[苏晚](docs/prompts/take2_suwan_qipao.txt)、和狼、虎、熊打成一团的[拳手兔子](docs/prompts/take3_rabbit_boxer.txt)(院线 3D 动画风)。同样的六个镜头,开头同样的特写,结尾同样走出那扇门。参考片本身也是 MiniMax H3 用文字生成的,全程没有任何真人肖像。-深度图只有让模型信得过才有用。所以 ReShot 围绕"深度视频怎么才能不抖"来做。+## 能复刻什么,不能复刻什么-- **它看的是整段片子。** 模型以 32 帧为一个窗口重叠推理并对齐,深度不会一帧一帧地跳。-- **整个镜头只用一把尺子。** 亮度就是距离,第 1 帧和第 300 帧里同样的亮度是同样的距离。- 有人从墙前走过,墙不会跟着呼吸。-- **不给模型任何它没要的东西。** 帧按时间戳选,24 fps 就是 24;尺寸裁到模型的网格上,绝不补黑边;- 编码干净到深度渐变没有色带。+**能:** 谁站在哪、彼此谁大谁小、每个动作和它的节奏、切镜、运镜——推进、跟拍、手持抖动。-## 跟得上一个工作室的节奏。+**不能:** 脸(用定妆图)、衣服、光线、颜色、道具细节,以及比手还小的东西。这些全靠你的提示词和参考图。-12 秒的镜头,RTX 4090 上约 20 秒——每帧 70 毫秒,快速档 31 毫秒。MacBook 也能跑。-720p 一段片约 4 GB 内存,放不下会在开始之前告诉你。+做演示时踩出来的几条:-一个 shell 循环批量处理整个文件夹,或者从 Python 里调:+- **参考片控制在 15 秒以内**(Seedance 的上限),跑 ReShot 之前先把片子剪到你要的那几个镜头。片子里有什么就会复刻什么,包括结尾没用的那段。+- **给 MiniMax H3 的深度图要缩小:`--target h3 --max-res 320`**(16:9 就是 320×176)。全尺寸的灰色人影会把角色的脸型往参考片里那个人上带;缩小后只带动作、不带脸型。+- **换物种可以,体量比例别动。** 熊那一段能成,是因为提示词写了熊「约为兔子的 1.3 倍,不超过 1.5 倍」。谁大谁小深度图已经定了,提示词不能跟它打架。+- **配角穿素色衣服,不留标志。** 提示词没写死的地方,模型会自己往上填字和徽章。++## 预设++| `--target` | fps | 尺寸 | 时长 | 给谁用 |+|---|---|---|---|---|+| `seedance` | 24 | 16 的倍数,≥ 407,696 像素 | ≤ 15 秒 | Seedance 2.0 / 2.5 参考视频 |+| `h3` | 24 | 32 的倍数 | ≤ 15 秒 | MiniMax H3(参考视频或 Fun ControlNet 深度条件) |+| `wan` | 16 | 16 的倍数 | – | Wan 2.1 VACE |+| `none` | 原片 | 偶数 | – | 任何认深度视频的模型 |++`reshot --help` 列出全部参数,[使用手册](docs/USAGE.zh-CN.md)逐条解释。++## 给开发者```pythonfrom pathlib import Path@@ -86,23 +118,18 @@ from reshot import RunConfig, runrun(RunConfig(input=Path("参考片.mp4"), output=Path("深度图.mp4"), target="seedance"))```-## 放心用在产品里。+三个细节决定了视频模型愿不愿意跟着这段灰片走:-ReShot 是 Apache-2.0,它默认用的模型也是——字节跳动的 Video Depth Anything Small(CVPR 2025),-vendored 在 `reshot/third_party/`。用在产品、流水线、服务里都行。更大的研究用变体也在,-标得清清楚楚,不主动指定就不会加载。+- **整段只用一把尺子。** 深度对全部帧归一化一次,绝不逐帧归一化,有人从墙前走过,墙的灰度不变。逐帧归一化会让整个场景「呼吸」。+- **按时间戳选帧。** 30 fps 转 24 fps 就真的是 24,不会以某种规律重复或丢帧让模型学了去。+- **只裁不补。** 尺寸裁到模型要的网格上,绝不补黑边——黑边会被当成远处的墙。-## 开始+模型是 Video Depth Anything Small(字节跳动,CVPR 2025),以 32 帧为一个窗口重叠推理并对齐,深度不会一帧一帧地跳。720p 一段片约需 4 GB 内存,放不下会在开始之前拒绝。`fake` 后端不加载模型就能跑完整条流水线,方便你写测试。属于「用户该修」的错误都是 `ReshotError` 的子类,带退出码和具体的修法。-```bash-pip install git+https://github.com/maosika-ai/reshot # 自己有 ffmpeg-pip install "reshot[ffmpeg] @ git+https://github.com/maosika-ai/reshot" # 或者自带一个-reshot --help-```+## 协议-权重 111 MB,首次运行自动下载。国内:`export HF_ENDPOINT=https://hf-mirror.com`。+Apache-2.0,默认模型也是。模型代码 vendored 在 `reshot/third_party/`。用在产品、流水线、服务里都行。更大的研究用权重(Base、Large)是 CC-BY-NC,不主动指定不会加载。-其余的——全部参数、预设、每个模型的接法、排错——都在**[使用手册](docs/USAGE.zh-CN.md)**。欢迎 issue 和 PR,见 [CONTRIBUTING.md](CONTRIBUTING.md)。## 关于猫斯卡diff --git a/demo/space/index.html b/demo/space/index.htmlindex d878601..37b28d1 100644--- a/demo/space/index.html+++ b/demo/space/index.html@@ -29,11 +29,11 @@</head><body><main><h1>Re<span>Shot</span></h1>-<p class="tag"><b>Copy the shot, not the actors.</b> Turn a reference video into a depth map your video model can follow — same staging, same camera, your cast, your style.<br><span class="zh"><b>复制走位,不复制演员。</b>把参考视频抽成视频模型能跟着走的深度图——走位一样、镜头一样,人是你的,画风是你的。</span></p>+<p class="tag"><b>Copy the shot, not the actors.</b> ReShot turns a reference video into a depth map, so Seedance or MiniMax H3 can repeat its choreography and camera moves — with your own characters in it.<br><span class="zh"><b>复制走位,不复制演员。</b>ReShot 把参考视频变成深度图,让 Seedance 或 MiniMax H3 照着它的动作和运镜再拍一遍——人物换成你的。</span></p><p class="zh"><sub>Open-sourced by <a href="https://www.maosika.com">Maosika 猫斯卡</a>, the AI short-drama production system · 由 AI 短剧生产系统 <a href="https://www.maosika.com">猫斯卡</a> 开源</sub></p><video class="demo" src="demo.mp4" poster="demo.gif" autoplay muted loop playsinline></video>-<p class="cap">top: reference · its ReShot depth map (near is white, far is black) · bottom: three takes generated from that depth map — different fighters, same choreography, same camera</p>+<p class="cap">top: the reference and its depth map · bottom: three takes generated from that depth map — two women, one rabbit; same moves, same camera<br><span class="zh">上排:参考片和它的深度图 · 下排:用这张深度图生成的三段片——两个女人、一只兔子;动作一样,镜头一样</span></p><pre><code>pip install git+https://github.com/maosika-ai/reshotreshot reference.mp4 -o depth.mp4 --target seedance</code></pre>@@ -45,10 +45,21 @@ reshot reference.mp4 -o depth.mp4 --target seedance</code></pre><a class="ghost" href="https://www.maosika.com">Maosika 猫斯卡 · make AI short drama</a></div>-<h2>Every shot is two things. <span class="zh">/ 每个镜头都是两样东西</span></h2>-<p>Where people are and how the camera moves — and who they are and what it looks like. A reference clip gives you both at once; you only wanted the first. ReShot keeps the staging and the camera and drops everything else: no faces, no wardrobe, no style. Give the depth map to your model with a prompt for the look, and you get the reference's blocking with your cast in it.</p>-<p class="zh">走位和运镜,以及画面里的人和风格。参考片两样一起给你,你只想要前一样。ReShot 留下走位和镜头,去掉其余一切——没有脸、没有服装、没有画风。把深度图交给模型,再告诉它你要的样子,就得到参考片的走位、你的人。</p>+<h2>The problem <span class="zh">/ 你遇到的问题</span></h2>+<p>You have a clip whose fight, dance or camera move is exactly what you want in your own AI video. Feed it to the video model as a reference and it copies the faces, the clothes and the look along with the moves — and real people in the clip may not pass the content check at all. Describe the moves in words instead, and you get a different fight every time.</p>+<p class="zh">你看中了一段片子,那场打戏、那段舞、那个运镜正是你的 AI 视频想要的。直接当参考视频喂给模型,它会把脸、衣服、画风连同动作一起抄走,片子里是真人还可能过不了审;改用文字描述动作,模型每次给你的打法都不一样。</p>+<h2>What ReShot does <span class="zh">/ ReShot 做什么</span></h2>+<p>An <code>.mp4</code> in, an <code>.mp4</code> out. The output is a depth-map video: every frame grey, near is white, far is black. It keeps where everyone stands, how big they are, how they move and how the camera moves; it throws away faces, clothes, lighting and style. Give the grey video to your model as the reference and describe the people and the look in the prompt.</p>+<p class="zh">吃进一个 mp4,吐出一个 mp4。输出是深度图视频:每帧都是灰的,近白远黑。留下谁站在哪、谁大谁小、怎么动、镜头怎么走;扔掉脸、衣服、光线、画风。把灰片当参考视频交给模型,提示词里写人物和画风。</p>++<h2>How to use it <span class="zh">/ 怎么用</span></h2>+<table>+<tr><td><b>1</b></td><td><code>pip install git+https://github.com/maosika-ai/reshot</code><br><span class="zh">装</span></td></tr>+<tr><td><b>2</b></td><td><code>reshot reference.mp4 -o depth.mp4 --target seedance</code> — 24 fps, H.264, the Seedance reference-video spec out of the box; <code>--target h3</code> for MiniMax H3.<br><span class="zh">抽深度图;<code>--target seedance</code> 直接按 Seedance 参考视频规格出片</span></td></tr>+<tr><td><b>3</b></td><td><b>Seedance 2.0 / 2.5:</b> upload <code>depth.mp4</code> as a reference video and prompt <code>参考@视频1的动作与运镜,顺序与视频保持一致。</code> + the people and the look.<br><b>MiniMax H3:</b> attach it as <code><Video 1></code>, a character sheet as <code><Picture 1></code>; define <code><Video 1></code> as "the fight choreography and camera movement shown in <Video 1>, a grey depth map…" and state that "its grey depth look is not transferred". The full prompts for the three takes are in <a href="https://github.com/maosika-ai/reshot/tree/main/docs/prompts">docs/prompts</a>.<br><span class="zh">交给模型:Seedance 当参考视频上传并写「参考@视频1的动作与运镜」;MiniMax H3 挂成 <Video 1>、定妆图挂 <Picture 1>,明说灰色外观不要抄。三段片的完整提示词在 docs/prompts。</span></td></tr>+<tr><td><b>4</b></td><td>Same six shots, same camera, your cast — the three takes above. Tips that made them work: keep the reference under 15 s; for MiniMax H3 shrink the depth map (<code>--max-res 320</code>) so the grey silhouette doesn't pull the face shape; keep the size ratio between characters as the depth map has it.<br><span class="zh">出片。几条经验:参考片 15 秒以内;给 MiniMax H3 的深度图缩到 320(不然灰影会带脸型);角色体量比例别跟深度图打架。</span></td></tr>+</table><h2>Works with the models you already use. <span class="zh">/ 你在用的模型都能接</span></h2><table><tr><th><code>--target</code></th><th>fps</th><th>size</th><th>for</th></tr>diff --git a/docs/img/step1_reference.jpg b/docs/img/step1_reference.jpgnew file mode 100644index 0000000..483fda4Binary files /dev/null and b/docs/img/step1_reference.jpg differdiff --git a/docs/img/step2_depth.jpg b/docs/img/step2_depth.jpgnew file mode 100644index 0000000..8ee65e4Binary files /dev/null and b/docs/img/step2_depth.jpg differdiff --git a/docs/img/step3_sheets.jpg b/docs/img/step3_sheets.jpgnew file mode 100644index 0000000..6717521Binary files /dev/null and b/docs/img/step3_sheets.jpg differdiff --git a/docs/img/step4_takes.jpg b/docs/img/step4_takes.jpgnew file mode 100644index 0000000..083a5b1Binary files /dev/null and b/docs/img/step4_takes.jpg differdiff --git a/docs/prompts/take1_jiangxue_armor.txt b/docs/prompts/take1_jiangxue_armor.txtnew file mode 100644index 0000000..00abfd0--- /dev/null+++ b/docs/prompts/take1_jiangxue_armor.txt@@ -0,0 +1,24 @@+subject_definitions:+<Subject 1> is the young woman shown in <Picture 1> and in close-up in <Picture 2>, Jiang Xue, a warrior with a high ponytail, red flame tattoos across her chest and shoulders and a dark leather-and-bronze armored corset; both pictures are used only for her appearance and her face must match <Picture 2> exactly.+<Subject 2> is a group of four guards in plain black tactical clothing without any markings, one of them holding a steel baton, the largest of them a head taller than the others.+<Subject 3> is the fight choreography and camera movement shown in <Video 1>, a grey depth map in which near objects are white and far objects are black: one fighter leans on a corridor wall in close-up, kicks off it to tear down a pipe, fights several men, is grabbed from behind by the largest and throws him, slams the last one into a wall panel, wipes the mouth in close-up, then walks away through a door past the fallen men.+<Subject 4> is a narrow steel corridor with riveted grey walls, flickering ceiling strip lights, a blinking red alarm lamp, sprinkler mist hanging in the air, stacked metal crates along one side, a wall control panel and a heavy sliding door at the far end.+summary:+[reference generation] <Subject 1> fights <Subject 2> inside <Subject 4>, reproducing shot for shot the choreography and camera of <Subject 3>: a close-up with eyes closed, kicking off the wall to tear down a pipe, a brawl in which she throws the largest guard over her shoulder, slamming the last guard into the control panel, wiping her mouth, and walking out through the door.+retention_analysis:+<Subject 1> (appears in [Shot 1], [Shot 2], [Shot 3], [Shot 4], [Shot 5], [Shot 6]): fully_preserved - her face, ponytail, chest tattoos and armored corset are retained exactly as in <Picture 1>.+<Subject 2> (appears in [Shot 3], [Shot 4], [Shot 6]): fully_preserved - plain black-clad guards.+<Subject 3> (appears in [Shot 1], [Shot 2], [Shot 3], [Shot 4], [Shot 5], [Shot 6]): attribute_transfer - every action, position, timing and camera move of <Video 1> is transferred onto <Subject 1> and <Subject 2>; its grey depth look is not transferred.+<Subject 4> (appears in all shots): fully_preserved - the corridor, crates, panel and door layout are retained.+detailed_description:+The target video is a photorealistic action fantasy with cold cinematic corridor lighting and a teal-and-red color grade.+[Shot 1] A tight close-up frames the face of <Subject 1> leaning back against the corridor wall of <Subject 4>, eyes closed, jaw set; the flickering light plays across her face, then her eyes snap open and she lets out a short shout as the camera holds a static shot.+[Shot 2] At 00:01.000, the shot cuts to a low medium-wide shot as <Subject 1> kicks off the wall, springs up, and rips a steel pipe from the wall fixture in a burst of sparks; the camera tracks her with small amplitude at fast speed.+[Shot 3] At 00:02.000, the camera cuts to a handheld medium shot inside the brawl in <Subject 4>: <Subject 1> drives the pipe into the ribs of one guard of <Subject 2>, hooks his neck and spins him into the wall, then leaps into a flying kick that sends a second guard crashing into the crates. The largest guard charges in and wraps <Subject 1> from behind in a crushing bear hug; <Subject 1> snaps her head back into his face, twists, and throws him over her shoulder so that his body slams flat onto the floor. The camera shakes slightly and trucks right to follow the action.+[Shot 4] At 00:07.000, the shot cuts to a medium shot as <Subject 1> grabs the last guard of <Subject 2> by the collar, drags him to the control panel and smashes his head into it; sparks spray from the panel and the heavy door at the end of <Subject 4> hisses open. The camera pushes in with small amplitude.+[Shot 5] At 00:08.500, the camera cuts to a close-up of <Subject 1>, chest heaving, wiping blood from the corner of her mouth with the back of her hand and staring past the camera; the camera holds a static shot.+[Shot 6] At 00:10.000, the shot cuts to a wide shot from low behind the fallen guards of <Subject 2> on the corridor floor as <Subject 1> steps over them and walks away through the open door into darkness; the camera holds a static shot.+overall_soundscape:+Low electrical hum of the corridor, dripping sprinkler water, heavy body impacts, the clang of the pipe on metal, sparks crackling, ragged breathing, the hiss of the hydraulic door.+non_diegetic_music:+A sparse, low pulsing synth bass with a slow heartbeat-like kick that fades out at the end.diff --git a/docs/prompts/take2_suwan_qipao.txt b/docs/prompts/take2_suwan_qipao.txtnew file mode 100644index 0000000..cf0b187--- /dev/null+++ b/docs/prompts/take2_suwan_qipao.txt@@ -0,0 +1,24 @@+subject_definitions:+<Subject 1> is the young woman shown in <Picture 1> and in close-up in <Picture 2>, Su Wan, an elegant beauty with black hair in a low side bun, a pearl earring and a dark green velvet qipao; both pictures are used only for her appearance and her face must match <Picture 2> exactly.+<Subject 2> is a group of four guards in plain black tactical clothing without any markings, one of them holding a steel baton, the largest of them a head taller than the others.+<Subject 3> is the fight choreography and camera movement shown in <Video 1>, a grey depth map in which near objects are white and far objects are black: one fighter leans on a corridor wall in close-up, kicks off it to tear down a pipe, fights several men, is grabbed from behind by the largest and throws him, slams the last one into a wall panel, wipes the mouth in close-up, then walks away through a door past the fallen men.+<Subject 4> is a narrow steel corridor with riveted grey walls, flickering ceiling strip lights, a blinking red alarm lamp, sprinkler mist hanging in the air, stacked metal crates along one side, a wall control panel and a heavy sliding door at the far end.+summary:+[reference generation] <Subject 1> fights <Subject 2> inside <Subject 4>, reproducing shot for shot the choreography and camera of <Subject 3>: a close-up with eyes closed, kicking off the wall to tear down a pipe, a brawl in which she throws the largest guard over her shoulder, slamming the last guard into the control panel, wiping her mouth, and walking out through the door.+retention_analysis:+<Subject 1> (appears in [Shot 1], [Shot 2], [Shot 3], [Shot 4], [Shot 5], [Shot 6]): fully_preserved - her face, side bun, pearl earring and green velvet qipao are retained exactly as in <Picture 1>.+<Subject 2> (appears in [Shot 3], [Shot 4], [Shot 6]): fully_preserved - plain black-clad guards.+<Subject 3> (appears in [Shot 1], [Shot 2], [Shot 3], [Shot 4], [Shot 5], [Shot 6]): attribute_transfer - every action, position, timing and camera move of <Video 1> is transferred onto <Subject 1> and <Subject 2>; its grey depth look is not transferred.+<Subject 4> (appears in all shots): fully_preserved - the corridor, crates, panel and door layout are retained.+detailed_description:+The target video is a photorealistic action fantasy with cold cinematic corridor lighting and a teal-and-red color grade.+[Shot 1] A tight close-up frames the face of <Subject 1> leaning back against the corridor wall of <Subject 4>, eyes closed, jaw set; the flickering light plays across her face, then her eyes snap open and she lets out a short shout as the camera holds a static shot.+[Shot 2] At 00:01.000, the shot cuts to a low medium-wide shot as <Subject 1> kicks off the wall, springs up, and rips a steel pipe from the wall fixture in a burst of sparks; the camera tracks her with small amplitude at fast speed.+[Shot 3] At 00:02.000, the camera cuts to a handheld medium shot inside the brawl in <Subject 4>: <Subject 1> drives the pipe into the ribs of one guard of <Subject 2>, hooks his neck and spins him into the wall, then leaps into a flying kick that sends a second guard crashing into the crates. The largest guard charges in and wraps <Subject 1> from behind in a crushing bear hug; <Subject 1> snaps her head back into his face, twists, and throws him over her shoulder so that his body slams flat onto the floor. The camera shakes slightly and trucks right to follow the action.+[Shot 4] At 00:07.000, the shot cuts to a medium shot as <Subject 1> grabs the last guard of <Subject 2> by the collar, drags him to the control panel and smashes his head into it; sparks spray from the panel and the heavy door at the end of <Subject 4> hisses open. The camera pushes in with small amplitude.+[Shot 5] At 00:08.500, the camera cuts to a close-up of <Subject 1>, chest heaving, wiping blood from the corner of her mouth with the back of her hand and staring past the camera; the camera holds a static shot.+[Shot 6] At 00:10.000, the shot cuts to a wide shot from low behind the fallen guards of <Subject 2> on the corridor floor as <Subject 1> steps over them and walks away through the open door into darkness; the camera holds a static shot.+overall_soundscape:+Low electrical hum of the corridor, dripping sprinkler water, heavy body impacts, the clang of the pipe on metal, sparks crackling, ragged breathing, the hiss of the hydraulic door.+non_diegetic_music:+A sparse, low pulsing synth bass with a slow heartbeat-like kick that fades out at the end.diff --git a/docs/prompts/take3_rabbit_boxer.txt b/docs/prompts/take3_rabbit_boxer.txtnew file mode 100644index 0000000..d828a8d--- /dev/null+++ b/docs/prompts/take3_rabbit_boxer.txt@@ -0,0 +1,24 @@+subject_definitions:+<Subject 1> is the anthropomorphic rabbit shown in <Picture 1>, a female rabbit with cream-tan fur, dark-tipped upright ears, big green eyes, a tiny silver ear stud, a grey hoodie, black shorts and white hand wraps, standing upright like a person, about 1.5 meters tall (baseline 1x), her head reaching the wolf's shoulder; the picture (face on the left, full body on the right) is used only for her appearance and her face, fur color and outfit must match it exactly. She is the only rabbit in the entire film: there is exactly one rabbit, and every other character is a predator of <Subject 2>.+<Subject 2> is a group of three anthropomorphic predators standing upright like people: a grey wolf about 1.9 meters tall, an orange tiger about 2.0 meters tall and a brown bear about 2.1 meters tall (about 1.3x the rabbit, never more than 1.5x), all in plain black leather jackets without any markings, one of them holding a steel bar; none of them is a rabbit and none of them has long ears.+<Subject 3> is the fight choreography and camera movement shown in <Video 1>, a grey depth map in which near objects are white and far objects are black: one fighter leans on a corridor wall in close-up, kicks off it to tear down a pipe, fights several opponents, is grabbed from behind by the largest and throws him, slams the last one into a wall panel, wipes the mouth in close-up, then walks away through a door past the fallen opponents.+<Subject 4> is a narrow steel corridor with riveted grey walls, flickering ceiling strip lights, a blinking red alarm lamp, sprinkler mist in the air, stacked metal crates along one side, a wall control panel with round colored buttons and a heavy sliding door at the far end; no text, letters or signs anywhere.+summary:+[reference generation] <Subject 1> fights <Subject 2> inside <Subject 4>, reproducing shot for shot the choreography and camera of <Subject 3>: a close-up with eyes closed, kicking off the wall to tear down a pipe, a brawl in which she throws the bear over her shoulder, slamming the last predator into the control panel, wiping her mouth, and walking out through the door. The whole film is a cartoon: no blood, no text anywhere, and exactly one rabbit on screen at any time.+retention_analysis:+<Subject 1> (appears in [Shot 1], [Shot 2], [Shot 3], [Shot 4], [Shot 5], [Shot 6]): fully_preserved - her cream-tan fur, green eyes, ear stud, grey hoodie and hand wraps are retained exactly as in <Picture 1>; she appears once per shot, never duplicated.+<Subject 2> (appears in [Shot 3], [Shot 4], [Shot 6]): fully_preserved - wolf, tiger and bear in plain black jackets.+<Subject 3> (appears in [Shot 1], [Shot 2], [Shot 3], [Shot 4], [Shot 5], [Shot 6]): attribute_transfer - every action, position, timing and camera move of <Video 1> is transferred onto <Subject 1> and <Subject 2>; its grey depth look is not transferred.+<Subject 4> (appears in all shots): fully_preserved - the corridor, crates, panel and door layout are retained.+detailed_description:+The target video is a cinematic 3D feature animation with cute stylized animal characters, soft fluffy fur, soft key light with bounce fill, and volumetric light.+[Shot 1] A tight close-up frames the face of <Subject 1> leaning back against the corridor wall of <Subject 4>, eyes closed, ears flat; the flickering light plays across her fur, then her big eyes snap open and she lets out a short shout as the camera holds a static shot.+[Shot 2] At 00:01.000, the shot cuts to a low medium-wide shot as <Subject 1> kicks off the wall, springs up, and rips a steel pipe from the wall fixture in a burst of sparks; the camera tracks her with small amplitude at fast speed.+[Shot 3] At 00:02.000, the camera cuts to a handheld medium shot inside the brawl in <Subject 4>: <Subject 1> drives the pipe into the ribs of the wolf of <Subject 2>, hooks his neck and spins him into the wall, then leaps into a flying kick that sends the tiger crashing into the crates. The bear charges in and wraps <Subject 1> from behind in a crushing bear hug; she snaps her head back into his snout, twists, and throws him over her shoulder so that his huge body slams flat onto the floor. The camera shakes slightly and trucks right to follow the action.+[Shot 4] At 00:07.000, the shot cuts to a medium shot as <Subject 1> grabs the wolf of <Subject 2> by the collar, drags him to the control panel and bonks his head into it; sparks spray from the panel and the heavy door at the end of <Subject 4> hisses open. The camera pushes in with small amplitude.+[Shot 5] At 00:08.500, the camera cuts to a close-up of <Subject 1>, chest heaving, wiping her mouth with the back of her paw and staring past the camera; the camera holds a static shot.+[Shot 6] At 00:10.000, the shot cuts to a wide shot from low behind the fallen wolf, tiger and bear of <Subject 2> lying on the corridor floor as <Subject 1>, the rabbit with her long upright ears clearly visible from behind, steps over them and walks away alone through the open door into the light; none of the predators stands up; the camera holds a static shot.+overall_soundscape:+Low electrical hum, dripping sprinkler water, soft cartoon body impacts, the clang of the pipe on metal, sparks crackling, quick breathing, the hiss of the door.+non_diegetic_music:+A playful but driving orchestral action cue with brass hits on the impacts, fading to a soft heartbeat at the end.diff --git a/docs/refs/take1_jiangxue_sheet.jpg b/docs/refs/take1_jiangxue_sheet.jpgnew file mode 100644index 0000000..567d3cfBinary files /dev/null and b/docs/refs/take1_jiangxue_sheet.jpg differdiff --git a/docs/refs/take2_suwan_sheet.jpg b/docs/refs/take2_suwan_sheet.jpgnew file mode 100644index 0000000..39fb1f5Binary files /dev/null and b/docs/refs/take2_suwan_sheet.jpg differdiff --git a/docs/refs/take3_rabbit_sheet.jpg b/docs/refs/take3_rabbit_sheet.jpgnew file mode 100644index 0000000..a6d16bfBinary files /dev/null and b/docs/refs/take3_rabbit_sheet.jpg differ