ReShot — copy the shot, not the actors. Video → depth map, OpenPose skeleton or canny lines for Seedance 2.0/2.5 reference video, MiniMax H3 Fun ControlNet and Wan VACE. Apache-2.0. Open-sourced by Maosika 猫斯卡 (www.maosika.com), the AI short-drama production system.
Python ai-short-dramaai-videocannycomfyuicontrolnetdepth-anythingdepth-estimationdwposemaosikaminimaxopenposepose-controlnetpose-estimationseedanceshort-dramavideo-generationvideo-to-videowan
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": "# ReShot — copy the shot, not the actors\n\nTurn a reference video into a **depth map, skeleton or line video** on a free Colab GPU, then give that grey clip to\nSeedance or MiniMax H3 as the reference: it repeats the choreography and the camera moves with\nyour own characters in it.\n\n**Before you run:** menu *Runtime → Change runtime type → T4 GPU*. Then *Runtime → Run all*.\nFour steps: check the GPU · install · pick a clip · run and download.\n\nCode and docs: [github.com/maosika-ai/reshot](https://github.com/maosika-ai/reshot) · Apache-2.0 ·\nmade at [Maosika 猫斯卡](https://www.maosika.com), the AI short-drama production system.\n\n在免费的 Colab GPU 上把参考视频变成深度图视频,再交给 Seedance / MiniMax H3 当参考。\n先在菜单 *代码执行程序 → 更改运行时类型* 选 **T4 GPU**,然后 *全部运行*。"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1 · GPU"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "form"
},
"execution_count": null,
"outputs": [],
"source": [
"#@title Check that this runtime has a GPU\n",
"import torch, subprocess\n",
"print(subprocess.run([\"nvidia-smi\", \"-L\"], capture_output=True, text=True).stdout.strip() or \"no nvidia-smi\")\n",
"print(\"torch\", torch.__version__, \"· CUDA available:\", torch.cuda.is_available())\n",
"assert torch.cuda.is_available(), \"No GPU. Runtime → Change runtime type → T4 GPU, then run again.\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2 · Install"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "form"
},
"execution_count": null,
"outputs": [],
"source": "#@title Install ReShot (about a minute; Colab already has a CUDA build of PyTorch)\nin_china = False #@param {type:\"boolean\"}\nimport os\nif in_china:\n os.environ[\"HF_ENDPOINT\"] = \"https://hf-mirror.com\" # the 111 MB weights come from Hugging Face\n%pip install -q \"reshot[ffmpeg,pose]\"\nimport reshot\nprint(\"reshot\", reshot.__version__)"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3 · Pick a clip"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "form"
},
"execution_count": null,
"outputs": [],
"source": [
"#@title Use the demo clip, or upload your own (mp4 / mov / webm, best ≤ 15 s)\n",
"source = \"demo clip (12 s corridor fight)\" #@param [\"demo clip (12 s corridor fight)\", \"upload my own\"]\n",
"import urllib.request, os\n",
"if source.startswith(\"upload\"):\n",
" from google.colab import files\n",
" up = files.upload()\n",
" assert up, \"nothing uploaded\"\n",
" INPUT = next(iter(up))\n",
"else:\n",
" INPUT = \"city_night.mp4\"\n",
" if not os.path.exists(INPUT):\n",
" urllib.request.urlretrieve(\"https://github.com/maosika-ai/reshot/raw/main/examples/city_night.mp4\", INPUT)\n",
"import cv2\n",
"cap = cv2.VideoCapture(INPUT); n, fps = cap.get(cv2.CAP_PROP_FRAME_COUNT), cap.get(cv2.CAP_PROP_FPS)\n",
"print(f\"{INPUT}: {int(cap.get(3))}×{int(cap.get(4))} · {fps:.0f} fps · {n/fps:.1f} s\"); cap.release()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## 4 · Make the control video"
},
{
"cell_type": "code",
"metadata": {
"cellView": "form"
},
"execution_count": null,
"outputs": [],
"source": "#@title Run (measured on a free T4: the 12 s demo clip takes about a minute at `fast`; `full` is ~2× that; `pose` is CPU-bound, a few minutes)\ncontrol = \"depth\" #@param [\"depth\", \"pose\", \"canny\"]\ntarget = \"seedance\" #@param [\"seedance\", \"h3\", \"wan\", \"none\"]\nquality = \"fast\" #@param [\"fast\", \"full\"]\nimport json, os, shutil, subprocess, time\nt0 = time.time()\nOUTPUT = f\"{control}.mp4\"\nr = subprocess.run([shutil.which(\"reshot\") or \"reshot\", INPUT, \"-o\", OUTPUT, \"--control\", control,\n \"--target\", target, \"--quality\", quality, \"--metrics\", \"metrics.json\", \"--keypoints\", \"pose.json\"],\n capture_output=True, text=True)\nprint(r.stderr[-3000:])\nassert r.returncode == 0 and os.path.exists(OUTPUT), \"run failed — read the message above\"\nm = json.load(open(\"metrics.json\"))\nprint(f\"done in {time.time()-t0:.0f} s · {m['frames']} frames @ {m['fps']} fps · \"\n f\"{m['output_size'][0]}×{m['output_size'][1]} · {m['ms_per_frame']:.0f} ms/frame on {m['device']}\")"
},
{
"cell_type": "code",
"metadata": {
"cellView": "form"
},
"execution_count": null,
"outputs": [],
"source": "#@title Look at it: reference and result side by side, then the two clips\nimport base64, cv2\nfrom IPython.display import HTML, display\ndef frame(path, idx=0):\n cap = cv2.VideoCapture(path); cap.set(cv2.CAP_PROP_POS_FRAMES, idx); ok, bgr = cap.read(); cap.release()\n h = 360; bgr = cv2.resize(bgr, (int(bgr.shape[1] * h / bgr.shape[0]), h))\n return base64.b64encode(cv2.imencode(\".jpg\", bgr)[1]).decode()\ndef video(path):\n return f'<video controls loop muted autoplay width=\"360\" src=\"data:video/mp4;base64,{base64.b64encode(open(path,\"rb\").read()).decode()}\"></video>'\ndisplay(HTML(f'<div style=\"display:flex;gap:12px\"><img src=\"data:image/jpeg;base64,{frame(INPUT)}\"><img src=\"data:image/jpeg;base64,{frame(OUTPUT)}\"></div>'\n f'<div style=\"display:flex;gap:12px;margin-top:12px\">{video(INPUT)}{video(OUTPUT)}</div>'))"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5 · Download, and what to do with it"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "form"
},
"execution_count": null,
"outputs": [],
"source": "#@title Download the result and print the prompt line for your video model\nfrom google.colab import files\nLOOK = {\n \"depth\": (\"a grey depth map in which near objects are white and far objects are black\", \"its grey depth look is not transferred\"),\n \"pose\": (\"an OpenPose skeleton video — coloured stick figures on black, one per person\", \"the stick-figure look is not transferred\"),\n \"canny\": (\"a white-line edge drawing on black\", \"the line-drawing look is not transferred\"),\n}[control]\nSEEDANCE = {\n \"depth\": \"参考@视频1的动作与运镜,顺序与视频保持一致。\",\n \"pose\": \"参考@视频1里火柴人的动作与运镜,顺序与视频保持一致;只取动作和机位,画面里不要出现骨架线条。\",\n \"canny\": \"参考@视频1的构图、动作与运镜,顺序与视频保持一致;只取构图和动作,不要画成线稿。\",\n}[control]\nNEXT = {\n \"seedance\": (f\"Upload {OUTPUT} to Seedance 2.0 / 2.5 as a reference video and start the prompt with:\",\n SEEDANCE + \"\\n(then describe your people, wardrobe, set and look)\"),\n \"h3\": (f\"In MiniMax H3 attach {OUTPUT} as <Video 1> (a character sheet as <Picture 1>); define it and say its look is not to be copied:\",\n f\"<Subject 3> is the choreography and camera movement shown in <Video 1>, {LOOK[0]}: (one sentence on what happens in the clip)\\n\"\n f\"<Subject 3>: attribute_transfer - every action, position, timing and camera move of <Video 1> is transferred onto <Subject 1>; {LOOK[1]}.\"),\n \"wan\": (f\"Connect {OUTPUT} to WanVaceToVideo → control_video in ComfyUI.\", \"\"),\n \"none\": (f\"Hand {OUTPUT} to any model that reads a {control} video.\", \"\"),\n}\nhow, prompt = NEXT[target]\nprint(how); print(); print(prompt)\nfiles.download(OUTPUT)\nif control == \"pose\" and os.path.exists(\"pose.json\"):\n files.download(\"pose.json\") # the skeletons as JSON, in case you want to edit or retarget them"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"Made something? Post the reference, the depth map, the take and your prompt line in\n",
"[Show and tell](https://github.com/maosika-ai/reshot/discussions/2). Something broke?\n",
"[Open an issue](https://github.com/maosika-ai/reshot/issues/new/choose) with the message from step 4.\n",
"\n",
"For whole folders, longer clips and scripting, run it on your own machine: `pip install reshot`,\n",
"then `reshot` for the local web page or `reshot clip.mp4 -o depth.mp4 --target seedance`."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "reshot_colab.ipynb",
"provenance": [],
"gpuType": "T4"
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}