Motion capture for any creature from one AI video: two cameras in one frame, a DINOv2 joint reader, the rig fitted in both views, a GLB out.
Load the released wolf LoRA from its URL
rehan-remadeb24df9fc77a7
diff --git a/mocap/fal.py b/mocap/fal.pyindex b6a6b6b..8df330e 100644--- a/mocap/fal.py+++ b/mocap/fal.py@@ -76,7 +76,7 @@ def run(endpoint, payload, **kw):TYPES = {"png": "image/png", "jpg": "image/jpeg", "jpeg": "image/jpeg", "webp": "image/webp", "mp4": "video/mp4",- "zip": "application/zip"}+ "zip": "application/zip", "safetensors": "application/octet-stream"}def upload(path):diff --git a/mocap/lora.py b/mocap/lora.pyindex ff72b79..1315baf 100644--- a/mocap/lora.py+++ b/mocap/lora.py@@ -52,14 +52,21 @@ def resume(run_dir):print("done", json.dumps(res)[:600], info, flush=True)+RELEASE_LORA = "https://github.com/Patch-gg/diptych/releases/download/examples-v1/lora-wolf-twocam-h3.safetensors"++def lora_path(src):- """The LoRA for fal to load: a training run's directory (the URL in its lora.json), a URL, or a local .safetensors- (uploaded to fal first, e.g. the wolf LoRA from the examples release)."""+ """The LoRA for fal to load: a training run's directory (the URL in its lora.json), a URL (such as the wolf LoRA in+ the examples release, RELEASE_LORA), or a small local .safetensors (uploaded to fal first)."""if str(src).startswith(("http://", "https://")):return str(src)p = pathlib.Path(src)if p.suffix == ".safetensors":- return fal.upload(p)+ try:+ return fal.upload(p)+ except Exception as e: # fal's simple upload refuses big files (the wolf LoRA is 131 MB): use a URL instead+ raise SystemExit(f"could not upload {p.name} to fal ({e.__class__.__name__}); pass the LoRA's URL instead, "+ "e.g. " + RELEASE_LORA) from Nonereturn json.loads((p / "lora.json").read_text())["response"]["lora_file"]["url"]diff --git a/skills/diptych-lora/SKILL.md b/skills/diptych-lora/SKILL.mdindex 308aaba..da88096 100644--- a/skills/diptych-lora/SKILL.md+++ b/skills/diptych-lora/SKILL.md@@ -45,12 +45,15 @@ its licence is non-commercial; say which you're using.```bash.venv/bin/python -m mocap.lora generate runs/<c>/lora/v1 runs/<c>/ref/ref.png prompts/<c>/pounce.txt runs/<c>/videos/pounce/lora.mp4 1.0 42```- The wolf's trained LoRA is in the examples release, so it can be tried without training (about $0.38 a clip):+ The wolf's trained LoRA is in the examples release, so it can be tried without training (about $0.38 a clip);+ fal loads it straight from the release:```bash- tools/fetch_examples.sh wolf --clips --lora- .venv/bin/python -m mocap.lora generate runs/wolf/lora/twocam_wolf_h3_lora.safetensors runs/wolf/ref/ref.png \- prompts/wolf/heldout/pounce.txt runs/wolf/videos/pounce/lora.mp4 1.0 42+ tools/fetch_examples.sh wolf --clips+ .venv/bin/python -m mocap.lora generate \+ https://github.com/Patch-gg/diptych/releases/download/examples-v1/lora-wolf-twocam-h3.safetensors \+ runs/wolf/ref/ref.png prompts/wolf/heldout/pounce.txt runs/wolf/videos/pounce/lora.mp4 1.0 42```+ `tools/fetch_examples.sh wolf --lora` downloads the file itself (131 MB), to run H3 with it yourself.6. A/B test on held-out moves against plain H3 with the same seed, then capture both and compare paw agreement andreprojection error (diptych-capture).