Retarget human motion from video to an Adobe Mixamo-rigged 3D character.
Python
Init project
MaxLiuc64e29dcecba
diff --git a/.gitignore b/.gitignorenew file mode 100644index 0000000..b126c58--- /dev/null+++ b/.gitignore@@ -0,0 +1,32 @@+# ── User-supplied assets: never committed ───────────────────+# Ignore every file under assets/; re-include directories (so git+# can descend) plus the docs and .gitkeep placeholders. This keeps+# SMPLX_NEUTRAL.npz / *.fbx / videos out of git no matter what.+assets/**+!assets/**/+!assets/README.md+!assets/**/.gitkeep++# ── Auto-downloaded inference weights (GVHMR / 2D pose, etc.) ─+weights/**+!weights/.gitkeep++# ── Run outputs ──────────────────────────────────────────────+outputs/**+!outputs/.gitkeep++# ── Python ───────────────────────────────────────────────────+__pycache__/+*.py[cod]+.venv/+venv/+*.egg-info/+build/+dist/++# ── Local secrets / environment variables ────────────────────+.env+.env.*++# ── OS junk ──────────────────────────────────────────────────+.DS_Storediff --git a/AGENTS.md b/AGENTS.mdnew file mode 100644index 0000000..21d5d59--- /dev/null+++ b/AGENTS.md@@ -0,0 +1,55 @@+# AGENTS.md++Motion2MixamoRig drives an Adobe Mixamo-rigged 3D character with human motion+extracted from a monocular video. CLI: `m2mr`.++## Setup++```bash+# chumpy (legacy dep of gvhmr) breaks under PEP 517 build isolation+# ("No module named 'pip'"); install it without isolation first.+pip install --upgrade pip setuptools wheel \+ && pip install "numpy>=1.26" \+ && pip install --no-build-isolation "chumpy==0.70" \+ && pip install -e .+```++Three things must be supplied by the user (all git-ignored, licensing-gated):++| What | Where to put it | Source |+| --- | --- | --- |+| SMPL-X body model | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | https://smpl-x.is.tue.mpg.de/ (registration) |+| Mixamo character | `assets/mixamo/<Name>.fbx` (FBX Binary, T-pose) | https://www.mixamo.com/ (Adobe account) |+| Input video | `assets/video/<name>.mp4` | one clearly visible person (group shots fail preflight) |++Inference weights (~5 GB) auto-download into `weights/` on the first run.++## Commands++```bash+m2mr doctor # verify assets + dependencies; prints what is missing and where to get it+m2mr run # latest video in assets/video/ + assets/mixamo/Y_Bot.fbx+m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx+m2mr run --skeleton outputs/<run>/skeleton_motion.npz --rig ... # reuse extraction, swap rig+```++`--video` and `--rig` combine freely. Extraction is the slow step; reuse it+with `--skeleton` when only the rig changes.++## Outputs++Each run writes `outputs/<YYYYMMDD_HHMMSS>_<video>/`:++- `run.json` — inputs, timings, output listing+- `skeleton_motion.npz` — 3D human skeleton (24 SMPL joints, Y-up meters)+- `mixamo_rotations.npz` — per-bone animation rotations (quaternion wxyz; convention string inside)+- `mixamo_character.glb` — skinned character + clip (glTF 2.0; Blender: File → Import → glTF 2.0)+- `videos/human_skeleton.mp4`, `videos/mixamo_skeleton.mp4`, `videos/mixamo_character.mp4`, `videos/compare.mp4` (2x2: original / mixamo skeleton / human skeleton / character)++## Code map++- `motion2mixamorig/paths.py` — all filesystem locations+- `motion2mixamorig/extract.py` — video -> 3D human skeleton (GVHMR)+- `motion2mixamorig/mixamo/` — FBX parsing, FK, retargeting, skinning, rendering+- `motion2mixamorig/pipeline.py` — one run end to end+- `motion2mixamorig/cli.py` — `m2mr doctor` / `m2mr run`diff --git a/NOTICE.md b/NOTICE.mdnew file mode 100644index 0000000..eb479e8--- /dev/null+++ b/NOTICE.md@@ -0,0 +1,41 @@+# NOTICE++This project's own code is released under the license in `LICENSE`. It builds+on the following upstream components, each governed by its own terms. None of+their gated assets are redistributed here; users obtain them directly from the+sources below.++## GVHMR++World-Grounded Human Motion Recovery via Gravity-View Coordinates+(Shen et al., SIGGRAPH Asia 2024). Installed as the `gvhmr` Python package.+GVHMR is released for **non-commercial research purposes only** — see its+license before any commercial use.+https://github.com/zju3dv/GVHMR++## SMPL-X body model++The SMPL-X body model is registration-gated and licensed by the Max Planck+Institute for Intelligent Systems for non-commercial scientific research.+Users must register and download `SMPLX_NEUTRAL.npz` themselves.+https://smpl-x.is.tue.mpg.de/++## Adobe Mixamo characters++Mixamo characters (Y Bot and others) are provided by Adobe under the Mixamo+terms of use. Download them with your own Adobe account; the FBX files are+never committed to this repository.+https://www.mixamo.com/++## Inference weights++The extraction step auto-downloads model checkpoints (GVHMR, HMR2, RTMPose /+ViTPose 2D pose, YOLOv8 person detector) into `weights/`. Note that YOLOv8+(Ultralytics) is licensed under **AGPL-3.0**, and the pose/HMR checkpoints+carry their respective research licenses.++## Summary++The end-to-end pipeline is effectively restricted to non-commercial /+research use by its upstream components, regardless of this repository's own+code license.diff --git a/README.md b/README.mdindex 96c83ca..aaca8cb 100644--- a/README.md+++ b/README.md@@ -1,2 +1,122 @@# Motion2MixamoRig-Retarget human motion from video to an Adobe Mixamo-rigged 3D character.++[中文](README.zh-CN.md) · English++Transfer the motion of a person in a video onto an Adobe Mixamo-rigged 3D character.++For game developers: give it a **single-person** action video and a Mixamo character; it writes preview videos plus a skinned character file (`.glb`) you can drop into Blender or Unity.++Agents should start with [`AGENTS.md`](AGENTS.md).++## Prerequisites++Python 3.10+. After cloning, create a virtualenv and install dependencies; the `m2mr` command is then available:++```bash+git clone https://github.com/MaxLiu22/Motion2MixamoRig.git+cd Motion2MixamoRig+python -m venv .venv && source .venv/bin/activate++# Install dependencies and this project. chumpy (a legacy package on the+# dependency chain) imports pip in its setup script, which fails with+# "No module named 'pip'" under pip's default isolated build. Install it+# first with isolation turned off, then install this project.+pip install --upgrade pip setuptools wheel \+ && pip install "numpy>=1.26" \+ && pip install --no-build-isolation "chumpy==0.70" \+ && pip install -e .+```++Inference weights (GVHMR and so on) are not downloaded here. The first `m2mr run` pulls them into `weights/` (~5 GB, once only).++ffmpeg is recommended (macOS: `brew install ffmpeg`, Ubuntu: `apt install ffmpeg`).+The pipeline runs without it, but the output videos will be **silent** and will not play in a browser.++## Before you run: put three things in `assets/`++This repository does not ship these files; you have to fetch them. The SMPL-X filename must match the table exactly; FBX and video names can be anything.++| What | Where | Where to get it |+|---|---|---|+| SMPL-X body model | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | Register at [SMPL-X](https://smpl-x.is.tue.mpg.de/) and download it |+| Mixamo character FBX | `assets/mixamo/Y_Bot.fbx` | With an Adobe account, download Y Bot (or any other Mixamo-rigged character) from [Mixamo](https://www.mixamo.com) |+| Action video | `assets/video/<your_clip>.mp4` | A clip with **exactly one** clearly visible person (you may keep several files; each run uses one) |++Note: Mixamo's download is named `Y Bot.fbx` (**space**), one character off from `Y_Bot.fbx` (**underscore**) in the table.+Rename it to the underscore form and it becomes the default rig (`m2mr run` without `--rig`). Leaving the space is fine too — if `Y_Bot.fbx` is missing, the first `.fbx` in `assets/mixamo/` is used.++`assets/video/` can hold several videos. Without `--video`, the run uses the file **most recently added** to that folder.+Each clip must show only one person: `m2mr doctor` / `m2mr run` sample the frames and stop before extraction if two people are in view.++## Quick Start++Results land in `outputs/<run-time>_<video>/`. Open that folder's `videos/` to watch them.++### 1. Check assets and the environment++Anything missing is printed with where to download it and where to put it.++```bash+m2mr doctor+```++### 2. First run: get something on screen++With no flags, this uses the **latest file in `assets/video/`** and `assets/mixamo/Y_Bot.fbx`.++```bash+m2mr run+```++This is the slow step. The first run downloads ~5 GB of inference weights (once only), then extracts human motion from the video.+On CPU, a ~30 second clip takes about 8–15 minutes to extract; later runs of similar length, with weights already there, take about 3–5 minutes.+Retargeting and rendering are usually a minute or two. Reusing a skeleton with `--skeleton` in the next section skips extraction entirely and finishes in tens of seconds.++### 3. Same motion, different character++Download another character from Mixamo into `assets/mixamo/`. **Do not rerun with only `--rig`** — that repeats the slow extraction.+Pass `--skeleton` to reuse the skeleton from a previous run; minutes become tens of seconds:++```bash+m2mr run --skeleton outputs/<previous-run-dir>/skeleton_motion.npz --rig assets/mixamo/Vampire.fbx+```++### 4. A different video++`--video` picks the clip. It combines freely with `--rig`.++```bash+m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx+```++## Outputs++Each `m2mr run` creates a directory named by **the time the command started**:++```+outputs/20260829_193205_dance/+├── run.json # start time, video / rig used, full command+├── skeleton_motion.npz # 3D human skeleton (reuse when swapping rigs; no re-extract)+├── mixamo_rotations.npz # per-bone Mixamo rotations+├── mixamo_character.glb # skinned character + animation, import into Blender / Unity+└── videos/ # same aspect ratio as the input+ ├── human_skeleton.mp4 # human skeleton+ ├── mixamo_skeleton.mp4 # Mixamo rig skeleton+ ├── mixamo_character.mp4 # Mixamo rig character+ └── compare.mp4 # 2×2: original (TL) / Mixamo skeleton (TR) / human skeleton (BL) / character (BR)+```++Open `mixamo_character.glb` in Blender: File → Import → glTF 2.0 (.glb/.gltf). If the camera is framed on empty space, select the character and use View → Frame Selected. The cluster of spheres on the head and hands is the bone display, not the mesh: select the armature and turn off Shapes under Armature → Viewport Display. Set the timeline end frame to the clip length, then play. Compare against `videos/mixamo_character.mp4` from the same run.++## License++This repository's code is Apache-2.0. The assets you download have their own terms and must not be redistributed with this repo:++- **SMPL-X**: MPI-gated; default non-commercial research / education / art; commercial use needs a separate deal+- **Mixamo FBX**: fine in a shipped product; do not redistribute the raw FBX as an asset pack+- **GVHMR and other inference weights**: auto-downloaded into `weights/` on the first `m2mr run`, under each upstream license++## Contact++If you run into a problem, need help finishing a local setup, or would like to talk about working together, email:+[[email protected]](mailto:[email protected])diff --git a/README.zh-CN.md b/README.zh-CN.mdnew file mode 100644index 0000000..3490159--- /dev/null+++ b/README.zh-CN.md@@ -0,0 +1,122 @@+# Motion2MixamoRig++中文 · [English](README.md)++把视频里人物的动作,转到 Adobe Mixamo rig 的 3D 角色上。++面向游戏开发者:输入一段**单人**动作视频和一个 Mixamo 角色,输出预览视频,以及可直接丢进 Blender / Unity 的带皮角色文件(`.glb`)。++Agent 请先读 [`AGENTS.md`](AGENTS.md)。++## 前置条件++需要 Python 3.10+。克隆后建虚拟环境并安装依赖,`m2mr` 命令随之可用:++```bash+git clone https://github.com/MaxLiu22/Motion2MixamoRig.git+cd Motion2MixamoRig+python -m venv .venv && source .venv/bin/activate++# 安装依赖和本项目。chumpy(依赖链里的老包)的构建脚本会 import pip,+# 在 pip 默认的隔离构建环境里必然报 "No module named 'pip'",+# 所以先关掉隔离(--no-build-isolation)单独装它,再装本项目。+pip install --upgrade pip setuptools wheel \+ && pip install "numpy>=1.26" \+ && pip install --no-build-isolation "chumpy==0.70" \+ && pip install -e .+```++推理权重(GVHMR 等)不在这一步下载,首次 `m2mr run` 时自动拉到 `weights/`(约 5 GB,只下这一次)。++建议再装上 ffmpeg(macOS:`brew install ffmpeg`,Ubuntu:`apt install ffmpeg`)。+没有它也能跑,但输出的视频会**没有声音**,编码格式也没法在浏览器里直接播放。++## 使用前:把三样东西放到 `assets/`++本仓库不附带这些文件,需要你自己准备。SMPL-X 的文件名必须和下表一字不差;FBX 和视频叫什么都行。++| 放什么 | 放到哪 | 去哪拿 |+|---|---|---|+| SMPL-X 人体模型 | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | 注册 [SMPL-X](https://smpl-x.is.tue.mpg.de/) 后下载 |+| Mixamo 角色 FBX | `assets/mixamo/Y_Bot.fbx` | 用 Adobe 账号从 [Mixamo](https://www.mixamo.com) 下载 Y Bot(或其他 Mixamo rig 角色) |+| 动作视频 | `assets/video/<your_clip>.mp4` | **只有一个人**、人清晰可见的动作视频(可放多个文件,每次跑用其中一个) |++注意:Mixamo 下载下来的文件叫 `Y Bot.fbx`(中间是**空格**),和上表的 `Y_Bot.fbx`(**下划线**)差一个字符。+改成下划线它就是默认 rig,`m2mr run` 不带 `--rig` 时自动用它;不想改名也行——找不到 `Y_Bot.fbx` 时+会自动用 `assets/mixamo/` 里的第一个 `.fbx`。++`assets/video/` 可以放多个视频文件。不指定 `--video` 时,会用**最后放进这个文件夹**的那一个。+每个视频画面里只能有一个人:`m2mr doctor` / `m2mr run` 会抽样检测,看到两个人就直接停,不会进入提取。++## Quick Start++结果都在 `outputs/<运行时间>_<视频名>/`,看视频打开该目录下的 `videos/`。++### 1. 检查资产和环境++缺什么会告诉你去哪下、放到哪。++```bash+m2mr doctor+```++### 2. 第一次跑:先拿到一个能看的结果++不带参数时,用**最后放进 `assets/video/` 的视频** + `assets/mixamo/Y_Bot.fbx`。++```bash+m2mr run+```++这步最慢。第一次跑会先下载约 5 GB 推理权重(只这一次),然后从视频里提取人体动作。+一段约 30 秒的视频,CPU 上提取大约 8–15 分钟;权重下完之后再跑同类长度,大约 3–5 分钟。+后面的重定向和渲染通常一两分钟。用下一节的 `--skeleton` 复用骨架时,提取整段跳过,几十秒就能出结果。++### 3. 同一段动作,换个角色++从 Mixamo 下载别的角色丢进 `assets/mixamo/`。**别直接只改 `--rig` 重跑**——那样最慢的+动作提取会原样再来一遍。用 `--skeleton` 复用上一次运行提取好的骨架,几分钟的事变成几十秒:++```bash+m2mr run --skeleton outputs/<上次的运行目录>/skeleton_motion.npz --rig assets/mixamo/Vampire.fbx+```++### 4. 换视频++`--video` 指定用哪一段,和 `--rig` 可任意组合。++```bash+m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx+```++## 输出++每次 `m2mr run` 按**命令启动时间**新建一个目录:++```+outputs/20260829_193205_dance/+├── run.json # 启动时间、用的视频 / rig、完整命令+├── skeleton_motion.npz # 人体 3D 骨架(换 rig 可复用,不必重跑提取)+├── mixamo_rotations.npz # Mixamo 骨逐帧旋转+├── mixamo_character.glb # 带皮角色 + 动画,Blender / Unity 可直接导入+└── videos/ # 比例跟输入视频一致+ ├── human_skeleton.mp4 # 人体骨架+ ├── mixamo_skeleton.mp4 # Mixamo rig 骨架+ ├── mixamo_character.mp4 # Mixamo rig 角色+ └── compare.mp4 # 四宫格:左上原片 / 右上 Mixamo 骨架 / 左下人体骨架 / 右下 Mixamo 角色+```++`mixamo_character.glb` 用 Blender 打开:File → Import → glTF 2.0 (.glb/.gltf)。导入后若镜头对不准,先选中角色,再 View → Frame Selected。头和手上那堆球是骨头的显示形状,不是模型:选中骨架,在 Armature → Viewport Display 里去掉 Shapes。时间轴结束帧改成和视频一样长,再点播放;对照同一次运行的 `videos/mixamo_character.mp4`。++## 许可证++本仓库代码为 Apache-2.0。你自取的资产各有条款,不能随本仓库再分发:++- **SMPL-X**:MPI 门禁,默认非商业科研 / 教育 / 艺术;商用需另谈+- **Mixamo FBX**:可用在成品里,不能把原始 FBX 当资源包发出去+- **GVHMR 等推理权重**:首次 `m2mr run` 自动下载到 `weights/`,遵循各自上游许可++## 联系++若使用中遇到问题、需要协助完成本地部署,或希望洽谈合作,欢迎通过邮箱联系我:+[[email protected]](mailto:[email protected])diff --git a/assets/README.md b/assets/README.mdnew file mode 100644index 0000000..11d83d1--- /dev/null+++ b/assets/README.md@@ -0,0 +1,18 @@+# assets++Everything in this folder is supplied by you and is never committed to git+(see the repository `.gitignore`). Licensing is why these cannot ship with the+project: SMPL-X and Mixamo characters must be obtained under your own account.++Run `m2mr doctor` at any time to check what is present and what is missing.++## What goes where++| Path | What | Where to get it |+| --- | --- | --- |+| `body_models/smplx/SMPLX_NEUTRAL.npz` | SMPL-X neutral body model, needed to lift the video into a 3D skeleton | Register at <https://smpl-x.is.tue.mpg.de/>, download the model archive, copy `SMPLX_NEUTRAL.npz` here |+| `mixamo/<Character>.fbx` | The character to drive, e.g. `Y_Bot.fbx` (the default) | <https://www.mixamo.com/> — pick a character, download as **FBX Binary**, **T-pose** |+| `video/<your_clip>.mp4` | The video whose human motion you want to transfer | One clearly visible person only (`.mp4`, `.mov`, …). Group shots are rejected before extraction |++`m2mr run` without flags uses the most recently added file in `video/` and+`mixamo/Y_Bot.fbx`; use `--video` / `--rig` to pick explicitly.diff --git a/assets/body_models/smplx/.gitkeep b/assets/body_models/smplx/.gitkeepnew file mode 100644index 0000000..e69de29diff --git a/assets/mixamo/.gitkeep b/assets/mixamo/.gitkeepnew file mode 100644index 0000000..e69de29diff --git a/assets/video/.gitkeep b/assets/video/.gitkeepnew file mode 100644index 0000000..e69de29diff --git a/motion2mixamorig/__init__.py b/motion2mixamorig/__init__.pynew file mode 100644index 0000000..c9f970f--- /dev/null+++ b/motion2mixamorig/__init__.py@@ -0,0 +1 @@+"""Motion2MixamoRig: retarget human motion from video to a Mixamo-rigged 3D character."""diff --git a/motion2mixamorig/checks.py b/motion2mixamorig/checks.pynew file mode 100644index 0000000..09a6c81--- /dev/null+++ b/motion2mixamorig/checks.py@@ -0,0 +1,174 @@+"""Content checks for user-supplied files, each returning a fix-it hint.++The CLI already checks that files exist. These look *inside* them, because the+common failure mode is not a missing file but a wrong one — an ASCII FBX, an+SMPL model renamed to SMPL-X, a clip OpenCV cannot decode — which would+otherwise surface minutes into a run as a deep traceback.++Each check returns None when the file is usable, else one line saying what is+wrong and what to do about it.+"""++from __future__ import annotations++from pathlib import Path++# Vertex counts identify the body-model family regardless of the file name.+SMPLX_VERTS = 10475+SMPL_VERTS = 6890++FBX_BINARY_MAGIC = b"Kaydara FBX Binary"++# Sampled frames for the person-count check. The pipeline maps one body to one+# rig, so two people in frame is a hard stop — we do not wait for extraction.+_PERSON_SAMPLES = 16+_PERSON_CONF = 0.5+# Ignore postage-stamp extras (audience, posters). A second dancer is larger.+_PERSON_MIN_AREA = 0.015++_yolo = None+++def _person_detector():+ """Ultralytics YOLO, reused across checks. Same family GVHMR uses to find people."""+ global _yolo+ if _yolo is not None:+ return _yolo+ from ultralytics import YOLO++ from .paths import WEIGHTS, export_gvhmr_env++ export_gvhmr_env()+ ckpt = WEIGHTS / "yolo" / "yolov8x.pt"+ _yolo = YOLO(str(ckpt) if ckpt.exists() else "yolov8n.pt")+ return _yolo+++def _count_people(frame) -> int:+ h, w = frame.shape[:2]+ min_area = _PERSON_MIN_AREA * h * w+ result = _person_detector().predict(+ frame, conf=_PERSON_CONF, classes=[0], verbose=False+ )[0]+ n = 0+ if result.boxes is None:+ return 0+ for box in result.boxes:+ x1, y1, x2, y2 = (float(v) for v in box.xyxy[0])+ if (x2 - x1) * (y2 - y1) >= min_area:+ n += 1+ return n+++def check_video(path: Path) -> str | None:+ """Decodable, and a single person on the sampled frames — not a group."""+ import cv2++ cap = cv2.VideoCapture(str(path))+ if not cap.isOpened():+ return (+ "cannot be decoded as video — convert it to a plain mp4 first, e.g. "+ "ffmpeg -i input -c:v libx264 output.mp4"+ )+ n_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))+ if n_frames <= 0:+ ok, _ = cap.read()+ cap.release()+ if not ok:+ return (+ "cannot be decoded as video — convert it to a plain mp4 first, e.g. "+ "ffmpeg -i input -c:v libx264 output.mp4"+ )+ return "has no readable frame count — re-export the clip as a plain mp4"+ k = min(_PERSON_SAMPLES, n_frames)+ sample = {int(round(i * (n_frames - 1) / max(k - 1, 1))) for i in range(k)}+ counts: list[int] = []+ idx = 0+ while idx <= max(sample):+ ok, frame = cap.read()+ if not ok:+ break+ if idx in sample:+ counts.append(_count_people(frame))+ idx += 1+ cap.release()+ if not counts:+ return "could not read frames to count people — re-export the clip as a plain mp4"+ crowded = [c for c in counts if c >= 2]+ if crowded:+ peak = max(counts)+ return (+ f"shows about {peak} people in frame — this pipeline maps one body "+ "to one Mixamo rig. Use a clip with a single, clearly visible person"+ )+ if max(counts) < 1:+ return (+ "no person was detected — use a clip with one clearly visible dancer, "+ "not an empty or heavily cropped shot"+ )+ return None+++def check_smplx(path: Path) -> str | None:+ """SMPLX_NEUTRAL.npz must be the npz variant of SMPL-X (not SMPL, not .pkl)."""+ import numpy as np++ try:+ with np.load(path, allow_pickle=True) as data:+ v_template = data["v_template"]+ except KeyError:+ return "is an .npz but not a body model (no v_template) — re-download SMPLX_NEUTRAL.npz"+ except Exception:+ return (+ "is not a valid .npz — you may have the pickle (.pkl) variant; "+ "download the *npz* SMPL-X model from https://smpl-x.is.tue.mpg.de/"+ )+ n = int(v_template.shape[0])+ if n == SMPL_VERTS:+ return (+ "is the older SMPL model, not SMPL-X — download SMPLX_NEUTRAL.npz "+ "from https://smpl-x.is.tue.mpg.de/"+ )+ if n != SMPLX_VERTS:+ return f"has {n} template vertices, expected {SMPLX_VERTS} (SMPL-X) — re-download SMPLX_NEUTRAL.npz"+ return None+++def check_rig(path: Path) -> str | None:+ """The rig must be a binary FBX containing a Mixamo skeleton."""+ from .mixamo.fbx_skeleton import extract_skeleton++ with open(path, "rb") as f:+ head = f.read(len(FBX_BINARY_MAGIC))+ if head != FBX_BINARY_MAGIC:+ return (+ "is not a *binary* FBX — on mixamo.com pick Format: 'FBX Binary(.fbx)' "+ "when downloading the character"+ )+ try:+ bones = {str(b["name"]) for b in extract_skeleton(path)["bones"]}+ except Exception:+ return "could not be parsed as an FBX scene — re-download the character from mixamo.com"+ if "mixamorig:Hips" not in bones:+ return (+ "has no Mixamo skeleton (mixamorig:* bones) — download a *character* "+ "from mixamo.com, T-pose, FBX Binary"+ )+ return None+++def check_skeleton_npz(path: Path) -> str | None:+ """--skeleton must point at a skeleton_motion.npz from a previous run."""+ import numpy as np++ try:+ with np.load(path, allow_pickle=True) as data:+ missing = {"joints_3d", "joint_names", "fps"} - set(data.keys())+ except Exception:+ return "is not a valid .npz file"+ if missing:+ return (+ f"is missing {sorted(missing)} — pass the skeleton_motion.npz "+ "found inside a previous run's output directory"+ )+ return Nonediff --git a/motion2mixamorig/cli.py b/motion2mixamorig/cli.pynew file mode 100644index 0000000..5efe0f7--- /dev/null+++ b/motion2mixamorig/cli.py@@ -0,0 +1,240 @@+"""The `m2mr` command-line entry point: doctor / run.++ m2mr doctor check assets, weights and dependencies+ m2mr run latest video in assets/video/ + default Y Bot+ m2mr run --video V --rig R pick the inputs explicitly (flags combine freely)++`m2mr run` performs the same asset checks as `m2mr doctor` before doing any+work, and reports everything that is missing in one pass.+"""++from __future__ import annotations++import argparse+import sys+from pathlib import Path++from . import checks, paths++OK = "[ok] "+MISS = "[missing]"+BAD = "[invalid]"++SMPLX_HINT = (+ "register at https://smpl-x.is.tue.mpg.de/ , download SMPLX_NEUTRAL.npz, "+ "place it at assets/body_models/smplx/SMPLX_NEUTRAL.npz"+)+RIG_HINT = (+ "download a character from https://www.mixamo.com/ (FBX Binary, T-pose) "+ "into assets/mixamo/ , e.g. Y_Bot.fbx"+)+VIDEO_HINT = (+ "put a *single-person* motion video (.mp4/.mov/...) into assets/video/ "+ "— group shots are rejected before extraction"+)+++def _check(label: str, present: bool, hint: str) -> bool:+ print(f"{OK if present else MISS} {label}")+ if not present:+ print(f" -> {hint}")+ return present+++def _check_content(label: str, path: Path, checker) -> bool:+ """Existence is assumed; `checker` inspects the file and returns a hint or None."""+ problem = checker(path)+ print(f"{OK if problem is None else BAD} {label}")+ if problem is not None:+ print(f" -> {path.name} {problem}")+ return problem is None+++def _module_available(name: str) -> bool:+ import importlib.util++ return importlib.util.find_spec(name) is not None+++def _available_rigs() -> list[Path]:+ return sorted(paths.MIXAMO_DIR.glob("*.fbx")) if paths.MIXAMO_DIR.is_dir() else []+++def doctor() -> int:+ """Check every user-supplied asset and dependency; say where to get what's missing."""+ print(f"project root: {paths.PROJECT_ROOT}\n")++ ok = True+ smplx_label = f"SMPL-X body model {paths.SMPLX_NEUTRAL.relative_to(paths.PROJECT_ROOT)}"+ if paths.SMPLX_NEUTRAL.exists():+ ok &= _check_content(smplx_label, paths.SMPLX_NEUTRAL, checks.check_smplx)+ else:+ ok &= _check(smplx_label, False, SMPLX_HINT)+ rigs = _available_rigs()+ ok &= _check(+ f"Mixamo rig .fbx assets/mixamo/ ({len(rigs)} found"+ + (f": {', '.join(r.name for r in rigs[:5])})" if rigs else ")"),+ bool(rigs),+ RIG_HINT,+ )+ for rig in rigs:+ ok &= _check_content(f" - {rig.name}", rig, checks.check_rig)+ videos = paths.list_videos()+ ok &= _check(+ f"input video assets/video/ ({len(videos)} found"+ + (f", latest: {videos[0].name})" if videos else ")"),+ bool(videos),+ VIDEO_HINT,+ )+ for video in videos:+ ok &= _check_content(f" - {video.name}", video, checks.check_video)++ print()+ deps_ok = True+ for module, hint in (+ ("numpy", "pip install -e ."),+ ("cv2", "pip install -e ."),+ ("torch", "pip install -e ."),+ ("gvhmr", "pip install -e ."),+ ):+ deps_ok &= _check(f"python module {module}", _module_available(module), hint)+ import shutil as _shutil++ _check(+ "ffmpeg (optional) H.264 output with the source video's audio",+ _shutil.which("ffmpeg") is not None,+ "brew install ffmpeg / apt install ffmpeg — runs fine without, but videos are silent mp4v",+ )++ weights_note = (+ "downloaded automatically into weights/ on the first `m2mr run` (~5 GB)"+ )+ print(f"\ninference weights: {weights_note}")++ if ok and deps_ok:+ print("\neverything in place — try: m2mr run")+ return 0+ print("\nfix the [missing] items above, then re-run: m2mr doctor")+ return 1+++def preflight(args: argparse.Namespace) -> tuple[Path, Path, Path | None]:+ """Resolve run inputs, checking everything doctor checks for this run.++ All problems are collected and reported together (same format as doctor)+ instead of failing on the first one. Exits with code 1 if anything is+ missing.+ """+ problems = 0++ def check(label: str, present: bool, hint: str) -> bool:+ nonlocal problems+ if not _check(label, present, hint):+ problems += 1+ return present++ def check_content(label: str, path: Path | None, checker) -> None:+ """Inspect the file's contents, but only when it exists at all."""+ nonlocal problems+ if path is not None and path.exists() and not _check_content(label, path, checker):+ problems += 1++ # Input video: --video wins, else the most recently added file in assets/video/.+ if args.video:+ video = Path(args.video)+ check(f"input video {video}", video.exists(), "check the --video path")+ else:+ video = paths.latest_video()+ check(+ "input video assets/video/"+ + (f" (latest: {video.name})" if video else " (0 found)"),+ video is not None,+ VIDEO_HINT + ", or pass --video",+ )+ check_content(f"video is usable {video.name if video else ''}", video, checks.check_video)++ # Rig: --rig wins, else Y_Bot.fbx, else the first .fbx in assets/mixamo/.+ if args.rig:+ rig = Path(args.rig)+ check(f"Mixamo rig .fbx {rig}", rig.exists(), "check the --rig path")+ else:+ rig = paths.DEFAULT_RIG+ if not rig.exists():+ rigs = _available_rigs()+ rig = rigs[0] if rigs else None+ check(+ "Mixamo rig .fbx " + (str(rig) if rig else "assets/mixamo/ (0 found)"),+ rig is not None,+ RIG_HINT + ", or pass --rig",+ )+ check_content(f"rig is Mixamo {rig.name if rig else ''}", rig, checks.check_rig)++ # Skeleton reuse skips extraction; only then is SMPL-X not needed.+ skeleton = Path(args.skeleton) if args.skeleton else None+ if skeleton is not None:+ check(+ f"skeleton npz {skeleton}",+ skeleton.exists(),+ "check the --skeleton path (a previous run's skeleton_motion.npz)",+ )+ check_content(f"skeleton loads {skeleton.name}", skeleton, checks.check_skeleton_npz)+ else:+ check(+ f"SMPL-X body model {paths.SMPLX_NEUTRAL.relative_to(paths.PROJECT_ROOT)}",+ paths.SMPLX_NEUTRAL.exists(),+ SMPLX_HINT,+ )+ check_content(+ "SMPL-X loads SMPLX_NEUTRAL.npz", paths.SMPLX_NEUTRAL, checks.check_smplx+ )++ if problems:+ print(f"\n{problems} problem(s) — nothing was run. `m2mr doctor` checks everything.")+ sys.exit(1)+ return video, rig, skeleton+++def run(args: argparse.Namespace) -> int:+ video, rig, skeleton = preflight(args)+ print()++ from .pipeline import run_pipeline++ run_dir = run_pipeline(video, rig, device=args.device, skeleton_npz=skeleton)+ print(f"\nresults: {run_dir / 'videos'}")+ return 0+++def main() -> None:+ parser = argparse.ArgumentParser(+ prog="m2mr",+ description="Drive a Mixamo-rigged character with human motion from a video.",+ )+ sub = parser.add_subparsers(dest="command", required=True)++ sub.add_parser("doctor", help="check assets, weights and dependencies")++ p_run = sub.add_parser("run", help="video -> Mixamo rig, results in outputs/")+ p_run.add_argument(+ "--video",+ help="input video (default: the file most recently added to assets/video/)",+ )+ p_run.add_argument(+ "--rig",+ help="Mixamo character .fbx (default: assets/mixamo/Y_Bot.fbx)",+ )+ p_run.add_argument(+ "--skeleton",+ help="reuse a previous run's skeleton_motion.npz and skip video extraction",+ )+ p_run.add_argument(+ "--device",+ default="cpu",+ choices=["cpu", "cuda", "mps"],+ help="inference device for the extraction step (default: cpu)",+ )++ args = parser.parse_args()+ if args.command == "doctor":+ sys.exit(doctor())+ sys.exit(run(args))diff --git a/motion2mixamorig/extract.py b/motion2mixamorig/extract.pynew file mode 100644index 0000000..9513706--- /dev/null+++ b/motion2mixamorig/extract.py@@ -0,0 +1,173 @@+"""Video -> 3D human skeleton (GVHMR world-grounded motion recovery).++Produces skeleton_motion.npz with 24 SMPL joints per frame in a Y-up,+meter-scaled world, plus the root translation/orientation and fps.+"""++from __future__ import annotations++from pathlib import Path++import numpy as np++from .paths import export_gvhmr_env++# SMPL 24-joint order used by GVHMR (J_regressor).+SMPL_JOINT_NAMES = [+ "pelvis",+ "left_hip",+ "right_hip",+ "spine1",+ "left_knee",+ "right_knee",+ "spine2",+ "left_ankle",+ "right_ankle",+ "spine3",+ "left_foot",+ "right_foot",+ "neck",+ "left_collar",+ "right_collar",+ "head",+ "left_shoulder",+ "right_shoulder",+ "left_elbow",+ "right_elbow",+ "left_wrist",+ "right_wrist",+ "left_hand",+ "right_hand",+]+++def _to_numpy(x):+ if hasattr(x, "detach"):+ return x.detach().cpu().numpy()+ return np.asarray(x)+++def joints_world_from_smplx(params, device: str = "cpu") -> np.ndarray:+ """Convert GVHMR SMPL-X params to 24 SMPL joints without the gated SMPL .pkl.++ Official ``result.joints_world`` also loads SMPL faces from+ ``SMPL_NEUTRAL.pkl``. The joint math itself only needs SMPL-X plus the+ shipped ``smplx2smpl`` / ``J_regressor`` maps.+ """+ import torch+ from einops import einsum+ from gvhmr import PROJ_ROOT+ from gvhmr.utils.device import to_device+ from gvhmr.utils.smplx_utils import make_smplx++ if device == "mps":+ device = "cpu"+ torch_device = torch.device(device)+ smplx = make_smplx("supermotion").to(torch_device)+ smplx2smpl = torch.load(+ PROJ_ROOT / "gvhmr/utils/body_model/smplx2smpl_sparse.pt",+ weights_only=False,+ ).to(torch_device)+ j_regressor = torch.load(+ PROJ_ROOT / "gvhmr/utils/body_model/smpl_neutral_J_regressor.pt",+ weights_only=False,+ ).to(torch_device)+ with torch.no_grad():+ out = smplx(**to_device(dict(params), torch_device))+ verts = torch.stack([torch.matmul(smplx2smpl, v) for v in out.vertices])+ joints = einsum(j_regressor, verts, "j v, l v i -> l j i")+ return joints.cpu().numpy().astype(np.float32)+++def _stabilize_pose_cache(pose_cache: Path) -> bool:+ """Stabilize GVHMR's 2D-pose cache in place. True if the file changed.++ Idempotent: a cache that was already stabilized comes back unchanged. The+ pre-stabilization data is kept next to it (*_raw) so estimator-vs-+ stabilizer questions stay answerable after the fact.+ """+ import shutil++ import torch++ from .stabilize_kp2d import stabilize_coco17++ current = torch.load(pose_cache, weights_only=False)+ fixed, swaps = stabilize_coco17(current)+ if np.allclose(fixed, _to_numpy(current).astype(np.float32)):+ return False+ raw_backup = pose_cache.with_name(pose_cache.stem + "_raw" + pose_cache.suffix)+ if not raw_backup.exists():+ shutil.copy2(pose_cache, raw_backup)+ torch.save(torch.from_numpy(fixed), pose_cache)+ print(f"stabilized {pose_cache.name}: corrected {int(swaps.sum())}/{len(swaps)} left/right flips")+ return True+++def extract_motion(+ video: Path,+ output_npz: Path,+ *,+ work_dir: Path,+ device: str = "cpu",+ static_camera: bool = True,+ verbose: bool = True,+) -> Path:+ """Run GVHMR on `video` and save skeleton_motion.npz to `output_npz`.++ `work_dir` holds GVHMR's per-video intermediates (2D pose cache, 3D lift).+ Re-running with the same work_dir reuses the expensive passes.+ """+ export_gvhmr_env()+ import gvhmr++ video = Path(video).resolve()+ if not video.exists():+ raise FileNotFoundError(video)++ output_dir = Path(work_dir) / video.stem++ def recover():+ return gvhmr.recover(+ str(video),+ static_camera=static_camera,+ device=device,+ pose2d="rtmpose",+ output_dir=str(output_dir),+ output_root=str(work_dir),+ progress=verbose,+ )++ result = recover()++ # RTMPose labels left/right per frame and can swap them when the subject+ # turns away, which spins the recovered 3D heading. Stabilize the 2D cache+ # and redo the 3D lift once if anything was corrected.+ pose_cache = output_dir / "preprocess" / "vitpose.pt"+ if pose_cache.exists() and _stabilize_pose_cache(pose_cache):+ motion_cache = output_dir / "hmr4d_results.pt"+ motion_cache.unlink(missing_ok=True)+ result = recover()++ world = result.smpl_params_world+ joints_3d = joints_world_from_smplx(world, device=device)+ root_translation = _to_numpy(world["transl"]).astype(np.float32)+ root_orientation = _to_numpy(world["global_orient"]).astype(np.float32)+ timestamps = np.arange(len(joints_3d), dtype=np.float64) / float(result.fps)++ output_npz = Path(output_npz)+ output_npz.parent.mkdir(parents=True, exist_ok=True)+ np.savez_compressed(+ output_npz,+ joints_3d=joints_3d,+ joint_names=np.array(SMPL_JOINT_NAMES),+ fps=np.float32(result.fps),+ timestamps=timestamps,+ root_translation=root_translation,+ root_orientation=root_orientation,+ camera=np.array(result.camera),+ video=np.array(str(video)),+ )+ if verbose:+ print(f"saved {output_npz}")+ return output_npzdiff --git a/motion2mixamorig/human_motion.py b/motion2mixamorig/human_motion.pynew file mode 100644index 0000000..2e3a389--- /dev/null+++ b/motion2mixamorig/human_motion.py@@ -0,0 +1,74 @@+"""Load the skeleton_motion.npz produced by extract.py.++Human world (GVHMR gravity frame):+ +Y up, +X right, +Z forward (right-handed), units = meters.+"""++from __future__ import annotations++from dataclasses import dataclass+from pathlib import Path++import numpy as np+++@dataclass+class HumanMotion:+ joints_3d: np.ndarray # (F, J, 3) meters, Y-up world+ joint_names: list[str]+ fps: float+ timestamps: np.ndarray+ root_translation: np.ndarray | None+ path: Path++ @property+ def n_frames(self) -> int:+ return int(self.joints_3d.shape[0])++ def index(self, name: str) -> int:+ try:+ return self.joint_names.index(name)+ except ValueError as exc:+ raise KeyError(f"joint {name!r} not in {self.joint_names}") from exc++ def joint(self, frame: int, name: str) -> np.ndarray:+ return np.asarray(self.joints_3d[frame, self.index(name)], dtype=np.float64)+++def load_human_motion(path: Path) -> HumanMotion:+ path = Path(path)+ if not path.exists():+ raise FileNotFoundError(f"Missing {path}. Run the extraction step first.")+ data = np.load(path, allow_pickle=True)+ joints = np.asarray(data["joints_3d"], dtype=np.float64)+ if joints.ndim != 3 or joints.shape[-1] != 3:+ raise ValueError(f"joints_3d must be (F, J, 3), got {joints.shape}")+ names = [str(x) for x in data["joint_names"]]+ fps = float(data["fps"]) if "fps" in data.files else 30.0+ if "timestamps" in data.files:+ timestamps = np.asarray(data["timestamps"], dtype=np.float64)+ else:+ timestamps = np.arange(len(joints), dtype=np.float64) / fps+ root = (+ np.asarray(data["root_translation"], dtype=np.float64)+ if "root_translation" in data.files+ else None+ )+ return HumanMotion(+ joints_3d=joints,+ joint_names=names,+ fps=fps,+ timestamps=timestamps,+ root_translation=root,+ path=path,+ )+++def source_video_path(motion: HumanMotion) -> Path | None:+ """The input video recorded inside skeleton_motion.npz, if still on disk."""+ npz = np.load(motion.path, allow_pickle=True)+ if "video" in npz.files:+ path = Path(str(npz["video"]))+ if path.exists():+ return path+ return Nonediff --git a/motion2mixamorig/mixamo/__init__.py b/motion2mixamorig/mixamo/__init__.pynew file mode 100644index 0000000..6f5d98a--- /dev/null+++ b/motion2mixamorig/mixamo/__init__.py@@ -0,0 +1,12 @@+"""Human skeleton -> Mixamo rig retargeting.++Modules:+ fbx_skeleton binary FBX parser, core-bone skeleton extraction+ kinematics FK with FBX PreRotation, rotation utilities+ retarget per-frame Human -> Mixamo Skeleton swing/aim retarget+ tpose_calibration measured T-pose alignment skeleton <-> character rig+ ybot_retarget rest-relative rotation transfer onto the full rig+ skinned_mesh FBX meshes, skin clusters, linear blend skinning+ animation whole-clip retargeting driver+ render software cameras, skeleton/mesh drawing, video writing+"""diff --git a/motion2mixamorig/mixamo/animation.py b/motion2mixamorig/mixamo/animation.pynew file mode 100644index 0000000..6ac158b--- /dev/null+++ b/motion2mixamorig/mixamo/animation.py@@ -0,0 +1,113 @@+"""Per-frame retarget of a whole clip: Human -> Mixamo Skeleton -> character rig.++ human skeleton_motion.npz+ | retarget_one_frame (swing/aim, no bone scaling, no IK)+ Mixamo Skeleton R_anim+ | rest-relative transfer from the T-pose calibration+ character rig R_anim -> FK -> linear blend skinning+"""++from __future__ import annotations++from dataclasses import dataclass, field++import numpy as np++from .kinematics import forward_kinematics, world_positions_m+from .retarget import joint_mixamo, retarget_one_frame+from .skinned_mesh import YBotAsset, bone_palette, skin_mesh+from .ybot_retarget import YBotMapping+++@dataclass+class RetargetedAnimation:+ frames: list[int]+ fps: float+ r_anim_skeleton: list[dict[str, np.ndarray]]+ r_anim_rig: list[dict[str, np.ndarray]]+ hips_cm_skeleton: list[np.ndarray]+ hips_cm_rig: list[np.ndarray]+ mixamo_pos: list[dict[str, np.ndarray]]+ rig_pos: list[dict[str, np.ndarray]]+ human_pos: list[dict[str, np.ndarray]]+ twist_events: list[dict] = field(default_factory=list)+ sign_events: list[dict] = field(default_factory=list)++ @property+ def n(self) -> int:+ return len(self.frames)+++def build_animation(+ motion, mapping: YBotMapping, frames: list[int], *, verbose: bool = True+) -> RetargetedAnimation:+ """Retarget Human -> Mixamo, then carry the rest-relative motion onto the rig."""+ skel = mapping.skeleton_rig+ anim = RetargetedAnimation(+ frames=frames,+ fps=float(motion.fps),+ r_anim_skeleton=[],+ r_anim_rig=[],+ hips_cm_skeleton=[],+ hips_cm_rig=[],+ mixamo_pos=[],+ rig_pos=[],+ human_pos=[],+ )+ r_prev = q_prev = None+ for i, frame in enumerate(frames):+ result = retarget_one_frame(skel, motion, frame, prev_r_anim=r_prev, prev_q_anim=q_prev)+ hips_cm = joint_mixamo(motion, frame, "pelvis") / skel.meters_per_unit++ mix_pos = world_positions_m(+ skel, forward_kinematics(skel, result.r_anim, hips_translation_cm=hips_cm)+ )+ r_rig = mapping.apply_rest_relative_rotations(result.r_anim)+ hips_rig = mapping.hips_translation_cm(hips_cm)+ rig_pos = mapping.ybot_positions_m(r_rig, hips_rig)++ anim.r_anim_skeleton.append(result.r_anim)+ anim.r_anim_rig.append(r_rig)+ anim.hips_cm_skeleton.append(hips_cm)+ anim.hips_cm_rig.append(hips_rig)+ anim.mixamo_pos.append(mix_pos)+ anim.rig_pos.append(rig_pos)+ anim.human_pos.append({n: joint_mixamo(motion, frame, n) for n in motion.joint_names})+ if result.twist_flips:+ anim.twist_events.append({"frame": frame, "bones": result.twist_flips})+ if result.sign_flips:+ anim.sign_events.append({"frame": frame, "bones": result.sign_flips})++ r_prev, q_prev = result.r_anim_raw, result.q_anim+ if verbose and (i == 0 or (i + 1) % 200 == 0 or i == len(frames) - 1):+ print(f" retarget {i + 1}/{len(frames)}")+ return anim+++def skin_frames(+ asset: YBotAsset, anim: RetargetedAnimation, indices: list[int]+) -> list[list[np.ndarray]]:+ """Linear blend skinning of the character mesh (meters) for the given indices."""+ out: list[list[np.ndarray]] = []+ scale = asset.rig.meters_per_unit+ for i in indices:+ world = forward_kinematics(+ asset.rig, anim.r_anim_rig[i], hips_translation_cm=anim.hips_cm_rig[i]+ )+ out.append(+ [+ skin_mesh(mesh, bone_palette(world, mesh, asset.rig.order)) * scale+ for mesh in asset.meshes+ ]+ )+ return out+++def core_bones_only(pos: dict[str, np.ndarray], edges) -> dict[str, np.ndarray]:+ """Drop the finger chains so the rig panel draws the same graph as the edges."""+ keep = {name for edge in edges for name in edge}+ return {name: p for name, p in pos.items() if name in keep}+++def all_points(seq: list[dict[str, np.ndarray]]) -> np.ndarray:+ return np.stack([p for pos in seq for p in pos.values()])diff --git a/motion2mixamorig/mixamo/fbx_skeleton.py b/motion2mixamorig/mixamo/fbx_skeleton.pynew file mode 100644index 0000000..2c726f9--- /dev/null+++ b/motion2mixamorig/mixamo/fbx_skeleton.py@@ -0,0 +1,628 @@+"""Parse a binary Mixamo FBX and extract the core skeleton.++Reads the binary FBX directly (PreRotation is a Maya/FBX property, so Assimp+cannot be used) and returns the core-bone skeleton as a plain dict — the same+structure `kinematics.rig_from_skeleton_data` consumes.+"""++from __future__ import annotations++import re+import struct+import zlib+from pathlib import Path+from typing import Any++import numpy as np++# Mixamo body chain only: no finger bones. End sites keep bone directions.+CORE_BONE_SUFFIXES = (+ "Hips",+ "Spine",+ "Spine1",+ "Spine2",+ "Neck",+ "Head",+ "HeadTop_End",+ "LeftShoulder",+ "LeftArm",+ "LeftForeArm",+ "LeftHand",+ "RightShoulder",+ "RightArm",+ "RightForeArm",+ "RightHand",+ "LeftUpLeg",+ "LeftLeg",+ "LeftFoot",+ "LeftToeBase",+ "LeftToe_End",+ "RightUpLeg",+ "RightLeg",+ "RightFoot",+ "RightToeBase",+ "RightToe_End",+)++# SMPL-24 (GVHMR skeleton_motion.npz) → Mixamo Y Bot.+HUMAN_TO_MIXAMO = {+ "pelvis": "mixamorig:Hips",+ "left_hip": "mixamorig:LeftUpLeg",+ "right_hip": "mixamorig:RightUpLeg",+ "spine1": "mixamorig:Spine",+ "left_knee": "mixamorig:LeftLeg",+ "right_knee": "mixamorig:RightLeg",+ "spine2": "mixamorig:Spine1",+ "left_ankle": "mixamorig:LeftFoot",+ "right_ankle": "mixamorig:RightFoot",+ "spine3": "mixamorig:Spine2",+ "left_foot": "mixamorig:LeftToeBase",+ "right_foot": "mixamorig:RightToeBase",+ "neck": "mixamorig:Neck",+ "left_collar": "mixamorig:LeftShoulder",+ "right_collar": "mixamorig:RightShoulder",+ "head": "mixamorig:Head",+ "left_shoulder": "mixamorig:LeftArm",+ "right_shoulder": "mixamorig:RightArm",+ "left_elbow": "mixamorig:LeftForeArm",+ "right_elbow": "mixamorig:RightForeArm",+ "left_wrist": "mixamorig:LeftHand",+ "right_wrist": "mixamorig:RightHand",+ "left_hand": "mixamorig:LeftHand",+ "right_hand": "mixamorig:RightHand",+}++# Mixamo bones that have no 1:1 SMPL-24 joint (chain tips / extra spine).+MIXAMO_WITHOUT_HUMAN = (+ "mixamorig:HeadTop_End",+ "mixamorig:LeftToe_End",+ "mixamorig:RightToe_End",+)++_RIG_PREFIX_RE = re.compile(r"^mixamorig\d+:")++FBX_HEADER = b"Kaydara FBX Binary \x00\x1a\x00"+ARRAY_DTYPE = {+ b"f": ("<f", 4),+ b"d": ("<d", 8),+ b"l": ("<q", 8),+ b"i": ("<i", 4),+ b"b": ("<?", 1),+}+++class FbxError(ValueError):+ pass+++def _read(buf: memoryview, offset: int, fmt: str) -> tuple[Any, int]:+ size = struct.calcsize(fmt)+ return struct.unpack_from(fmt, buf, offset)[0], offset + size+++def _read_array(buf: memoryview, offset: int, type_code: bytes) -> tuple[list[Any], int]:+ length, offset = _read(buf, offset, "<I")+ encoding, offset = _read(buf, offset, "<I")+ compressed_len, offset = _read(buf, offset, "<I")+ dtype, item_size = ARRAY_DTYPE[type_code]+ raw = bytes(buf[offset : offset + compressed_len])+ offset += compressed_len+ if encoding == 1:+ raw = zlib.decompress(raw)+ elif encoding != 0:+ raise FbxError(f"unsupported FBX array encoding {encoding}")+ expected = length * item_size+ if len(raw) != expected:+ raise FbxError(f"array size mismatch: {len(raw)} != {expected}")+ values = list(struct.unpack_from(f"<{length}{dtype[-1]}", raw, 0)) if length else []+ return values, offset+++def _read_property(buf: memoryview, offset: int) -> tuple[Any, int]:+ type_code = bytes(buf[offset : offset + 1])+ offset += 1+ if type_code == b"Y":+ return _read(buf, offset, "<h")+ if type_code == b"C":+ value, offset = _read(buf, offset, "<B")+ return bool(value), offset+ if type_code == b"I":+ return _read(buf, offset, "<i")+ if type_code == b"F":+ return _read(buf, offset, "<f")+ if type_code == b"D":+ return _read(buf, offset, "<d")+ if type_code == b"L":+ return _read(buf, offset, "<q")+ if type_code in ARRAY_DTYPE:+ return _read_array(buf, offset, type_code)+ if type_code in (b"S", b"R"):+ length, offset = _read(buf, offset, "<I")+ data = bytes(buf[offset : offset + length])+ offset += length+ if type_code == b"S":+ return data.split(b"\x00")[0].decode("utf-8", errors="replace"), offset+ return data, offset+ raise FbxError(f"unknown FBX property type {type_code!r}")+++def _is_null_record(buf: memoryview, offset: int, version: int) -> bool:+ size = 25 if version >= 7500 else 13+ return offset + size <= len(buf) and all(b == 0 for b in buf[offset : offset + size])+++def _read_node(buf: memoryview, offset: int, version: int) -> tuple[dict[str, Any] | None, int]:+ start = offset+ if version >= 7500:+ end_offset, offset = _read(buf, offset, "<Q")+ n_props, offset = _read(buf, offset, "<Q")+ _prop_len, offset = _read(buf, offset, "<Q")+ else:+ end_offset, offset = _read(buf, offset, "<I")+ n_props, offset = _read(buf, offset, "<I")+ _prop_len, offset = _read(buf, offset, "<I")+ if end_offset == 0:+ return None, start + (25 if version >= 7500 else 13)+ name_len = buf[offset]+ offset += 1+ name = bytes(buf[offset : offset + name_len]).decode("ascii", errors="replace")+ offset += name_len+ props: list[Any] = []+ for _ in range(n_props):+ value, offset = _read_property(buf, offset)+ props.append(value)+ children: list[dict[str, Any]] = []+ while offset < end_offset:+ if _is_null_record(buf, offset, version):+ offset = end_offset+ break+ child, offset = _read_node(buf, offset, version)+ if child is None:+ break+ children.append(child)+ return {"name": name, "props": props, "children": children}, end_offset+++def parse_fbx(path: Path) -> tuple[int, list[dict[str, Any]]]:+ data = path.read_bytes()+ if not data.startswith(FBX_HEADER):+ raise FbxError(f"{path} is not a binary FBX file")+ version = struct.unpack_from("<I", data, 23)[0]+ buf = memoryview(data)+ offset = 27+ nodes: list[dict[str, Any]] = []+ while offset < len(buf):+ if _is_null_record(buf, offset, version):+ break+ node, offset = _read_node(buf, offset, version)+ if node is None:+ break+ nodes.append(node)+ return version, nodes+++def _children_named(node: dict[str, Any], name: str) -> list[dict[str, Any]]:+ return [c for c in node["children"] if c["name"] == name]+++def _child_named(node: dict[str, Any], name: str) -> dict[str, Any] | None:+ for child in node["children"]:+ if child["name"] == name:+ return child+ return None+++def _vec3(value: Any, default: tuple[float, float, float] = (0.0, 0.0, 0.0)) -> np.ndarray:+ if value is None:+ return np.array(default, dtype=np.float64)+ arr = np.asarray(value, dtype=np.float64).reshape(-1)+ if arr.size < 3:+ out = np.array(default, dtype=np.float64)+ out[: arr.size] = arr+ return out+ return arr[:3].copy()+++def parse_properties70(node: dict[str, Any]) -> dict[str, Any]:+ props: dict[str, Any] = {}+ block = _child_named(node, "Properties70")+ if block is None:+ return props+ for child in block["children"]:+ if child["name"] != "P" or not child["props"]:+ continue+ raw = child["props"]+ key = str(raw[0])+ values = raw[4:] if len(raw) > 4 else []+ if len(values) == 1:+ props[key] = values[0]+ elif len(values) >= 3 and all(isinstance(v, (int, float)) for v in values[:3]):+ props[key] = [float(v) for v in values[:3]]+ else:+ props[key] = values+ return props+++def euler_xyz_deg_to_matrix(euler_deg: np.ndarray) -> np.ndarray:+ """FBX/Maya Euler XYZ: R = Rz @ Ry @ Rx (column vectors)."""+ x, y, z = np.deg2rad(np.asarray(euler_deg, dtype=np.float64)[:3])+ cx, sx = np.cos(x), np.sin(x)+ cy, sy = np.cos(y), np.sin(y)+ cz, sz = np.cos(z), np.sin(z)+ rx = np.array([[1.0, 0.0, 0.0], [0.0, cx, -sx], [0.0, sx, cx]])+ ry = np.array([[cy, 0.0, sy], [0.0, 1.0, 0.0], [-sy, 0.0, cy]])+ rz = np.array([[cz, -sz, 0.0], [sz, cz, 0.0], [0.0, 0.0, 1.0]])+ return rz @ ry @ rx+++def matrix_to_quat_wxyz(rmat: np.ndarray) -> np.ndarray:+ m = np.asarray(rmat, dtype=np.float64)+ t = float(np.trace(m))+ if t > 0.0:+ s = 0.5 / np.sqrt(t + 1.0)+ w = 0.25 / s+ x = (m[2, 1] - m[1, 2]) * s+ y = (m[0, 2] - m[2, 0]) * s+ z = (m[1, 0] - m[0, 1]) * s+ else:+ i = int(np.argmax([m[0, 0], m[1, 1], m[2, 2]]))+ if i == 0:+ s = 2.0 * np.sqrt(1.0 + m[0, 0] - m[1, 1] - m[2, 2])+ w = (m[2, 1] - m[1, 2]) / s+ x = 0.25 * s+ y = (m[0, 1] + m[1, 0]) / s+ z = (m[0, 2] + m[2, 0]) / s+ elif i == 1:+ s = 2.0 * np.sqrt(1.0 + m[1, 1] - m[0, 0] - m[2, 2])+ w = (m[0, 2] - m[2, 0]) / s+ x = (m[0, 1] + m[1, 0]) / s+ y = 0.25 * s+ z = (m[1, 2] + m[2, 1]) / s+ else:+ s = 2.0 * np.sqrt(1.0 + m[2, 2] - m[0, 0] - m[1, 1])+ w = (m[1, 0] - m[0, 1]) / s+ x = (m[0, 2] + m[2, 0]) / s+ y = (m[1, 2] + m[2, 1]) / s+ z = 0.25 * s+ q = np.array([w, x, y, z], dtype=np.float64)+ n = float(np.linalg.norm(q))+ if n < 1e-12:+ return np.array([1.0, 0.0, 0.0, 0.0])+ if q[0] < 0:+ q = -q+ return q / n+++def make_transform(+ translation: np.ndarray,+ pre_rot_deg: np.ndarray,+ lcl_rot_deg: np.ndarray,+ post_rot_deg: np.ndarray,+ scale: np.ndarray,+ rotation_offset: np.ndarray | None = None,+ rotation_pivot: np.ndarray | None = None,+) -> np.ndarray:+ """Maya/FBX local TRS with PreRotation (pivots/offsets identity on Mixamo).++ M = T * Roff * Rp * Rpre * R * Rpost^{-1} * Rp^{-1} * S+ """+ t = _vec3(translation)+ s = _vec3(scale, (1.0, 1.0, 1.0))+ roff = _vec3(rotation_offset)+ rp = _vec3(rotation_pivot)+ rpre = euler_xyz_deg_to_matrix(pre_rot_deg)+ r = euler_xyz_deg_to_matrix(lcl_rot_deg)+ rpost = euler_xyz_deg_to_matrix(post_rot_deg)+ rot = rpre @ r @ rpost.T++ def translate(vec: np.ndarray) -> np.ndarray:+ m = np.eye(4)+ m[:3, 3] = vec+ return m++ t_mat = translate(t)+ roff_mat = translate(roff)+ rp_mat = translate(rp)+ rp_inv = translate(-rp)+ r_mat = np.eye(4)+ r_mat[:3, :3] = rot+ s_mat = np.diag([float(s[0]), float(s[1]), float(s[2]), 1.0])+ return t_mat @ roff_mat @ rp_mat @ r_mat @ rp_inv @ s_mat+++def _tolist(arr: np.ndarray, ndigits: int = 12) -> list[Any]:+ def _round(x: float) -> float:+ v = round(float(x), ndigits)+ return 0.0 if abs(v) < 10 ** (-ndigits) else v++ if arr.ndim == 1:+ return [_round(x) for x in arr]+ return [[_round(x) for x in row] for row in arr]+++def _describe_path(path: Path) -> str:+ return str(path)+++def normalize_rig_prefix(name: str) -> str:+ """`mixamorig8:Hips` → `mixamorig:Hips`.++ Mixamo appends a digit to the namespace when a character is exported from a+ scene that already held a rig, so the same bone arrives under a different+ name per file. Every bone table in the project keys off the bare prefix.+ """+ return _RIG_PREFIX_RE.sub("mixamorig:", name, count=1)+++def bone_suffix(name: str) -> str:+ return name.split(":")[-1]+++def is_core_bone(name: str) -> bool:+ return bone_suffix(name) in CORE_BONE_SUFFIXES+++def extract_objects(nodes: list[dict[str, Any]]) -> dict[int, dict[str, Any]]:+ objects: dict[int, dict[str, Any]] = {}+ for node in nodes:+ if node["name"] != "Objects":+ continue+ for child in node["children"]:+ if not child["props"]:+ continue+ uid = int(child["props"][0])+ name = normalize_rig_prefix(str(child["props"][1])) if len(child["props"]) > 1 else ""+ class_name = str(child["props"][2]) if len(child["props"]) > 2 else ""+ objects[uid] = {+ "uid": uid,+ "node_name": child["name"],+ "name": name,+ "class": class_name,+ "props": parse_properties70(child),+ "raw": child,+ }+ return objects+++def extract_connections(nodes: list[dict[str, Any]]) -> list[tuple[str, int, int]]:+ links: list[tuple[str, int, int]] = []+ for node in nodes:+ if node["name"] != "Connections":+ continue+ for child in node["children"]:+ if child["name"] != "C" or len(child["props"]) < 3:+ continue+ kind = str(child["props"][0])+ src = int(child["props"][1])+ dst = int(child["props"][2])+ links.append((kind, src, dst))+ return links+++def extract_global_settings(nodes: list[dict[str, Any]]) -> dict[str, Any]:+ for node in nodes:+ if node["name"] == "GlobalSettings":+ return parse_properties70(node)+ return {}+++def extract_bind_pose(objects: dict[int, dict[str, Any]]) -> dict[int, np.ndarray]:+ poses: dict[int, np.ndarray] = {}+ for obj in objects.values():+ if obj["node_name"] != "Pose":+ continue+ for pose_node in _children_named(obj["raw"], "PoseNode"):+ node_elem = _child_named(pose_node, "Node")+ matrix_elem = _child_named(pose_node, "Matrix")+ if node_elem is None or matrix_elem is None or not matrix_elem["props"]:+ continue+ uid = int(node_elem["props"][0])+ values = matrix_elem["props"][0]+ mat = np.asarray(values, dtype=np.float64).reshape(4, 4).T+ poses[uid] = mat+ return poses+++def collect_limb_nodes(objects: dict[int, dict[str, Any]]) -> dict[int, dict[str, Any]]:+ limbs: dict[int, dict[str, Any]] = {}+ for uid, obj in objects.items():+ if obj["node_name"] != "Model":+ continue+ if obj["class"] != "LimbNode":+ continue+ limbs[uid] = obj+ return limbs+++def build_hierarchy(+ limbs: dict[int, dict[str, Any]],+ connections: list[tuple[str, int, int]],+) -> dict[int, int | None]:+ parent_of: dict[int, int | None] = {uid: None for uid in limbs}+ for kind, src, dst in connections:+ if kind != "OO":+ continue+ if src in limbs and dst in limbs:+ parent_of[src] = dst+ return parent_of+++def bone_local_matrix(props: dict[str, Any]) -> tuple[np.ndarray, dict[str, np.ndarray]]:+ translation = _vec3(props.get("Lcl Translation"))+ lcl_rot = _vec3(props.get("Lcl Rotation"))+ pre_rot = _vec3(props.get("PreRotation"))+ post_rot = _vec3(props.get("PostRotation"))+ scale = _vec3(props.get("Lcl Scaling"), (1.0, 1.0, 1.0))+ rotation_offset = _vec3(props.get("RotationOffset"))+ rotation_pivot = _vec3(props.get("RotationPivot"))+ local = make_transform(+ translation,+ pre_rot,+ lcl_rot,+ post_rot,+ scale,+ rotation_offset=rotation_offset,+ rotation_pivot=rotation_pivot,+ )+ parts = {+ "lcl_translation": translation,+ "lcl_rotation_euler_deg": lcl_rot,+ "pre_rotation_euler_deg": pre_rot,+ "post_rotation_euler_deg": post_rot,+ "lcl_scaling": scale,+ "rotation_offset": rotation_offset,+ "rotation_pivot": rotation_pivot,+ }+ return local, parts+++def compose_world(+ parent_of: dict[int, int | None],+ local_of: dict[int, np.ndarray],+) -> dict[int, np.ndarray]:+ world: dict[int, np.ndarray] = {}++ def eval_uid(uid: int) -> np.ndarray:+ if uid in world:+ return world[uid]+ parent = parent_of.get(uid)+ local = local_of[uid]+ if parent is None:+ world[uid] = local.copy()+ else:+ world[uid] = eval_uid(parent) @ local+ return world[uid]++ for uid in local_of:+ eval_uid(uid)+ return world+++def unit_to_meters(settings: dict[str, Any]) -> float:+ """FBX default unit is centimeters when UnitScaleFactor == 1."""+ scale = float(settings.get("UnitScaleFactor", 1.0) or 1.0)+ return 0.01 * scale+++def axis_name(index: int, sign: int) -> str:+ names = {0: "X", 1: "Y", 2: "Z"}+ prefix = "+" if int(sign) >= 0 else "-"+ return f"{prefix}{names.get(int(index), '?')}"+++def extract_skeleton(fbx_path: Path) -> dict[str, Any]:+ version, nodes = parse_fbx(fbx_path)+ objects = extract_objects(nodes)+ connections = extract_connections(nodes)+ settings = extract_global_settings(nodes)+ bind_pose = extract_bind_pose(objects)+ limbs = collect_limb_nodes(objects)+ parent_of = build_hierarchy(limbs, connections)++ local_of: dict[int, np.ndarray] = {}+ parts_of: dict[int, dict[str, np.ndarray]] = {}+ for uid, obj in limbs.items():+ local, parts = bone_local_matrix(obj["props"])+ local_of[uid] = local+ parts_of[uid] = parts+ world_of = compose_world(parent_of, local_of)++ core_uids = [uid for uid, obj in limbs.items() if is_core_bone(obj["name"])]+ children_of: dict[int, list[int]] = {uid: [] for uid in core_uids}+ for uid in core_uids:+ parent = parent_of[uid]+ if parent in children_of:+ children_of[parent].append(uid)++ def sort_children(uids: list[int]) -> list[int]:+ order = {name: i for i, name in enumerate(CORE_BONE_SUFFIXES)}+ return sorted(uids, key=lambda u: order.get(bone_suffix(limbs[u]["name"]), 999))++ roots = [uid for uid in core_uids if parent_of[uid] not in set(core_uids)]+ ordered: list[int] = []++ def walk(uid: int) -> None:+ ordered.append(uid)+ for child in sort_children(children_of[uid]):+ walk(child)++ for root in sort_children(roots):+ walk(root)++ meters_per_fbx = unit_to_meters(settings)+ bones = []+ for uid in ordered:+ obj = limbs[uid]+ parts = parts_of[uid]+ local = local_of[uid]+ world = world_of[uid]+ parent_uid = parent_of[uid]+ parent_name = limbs[parent_uid]["name"] if parent_uid in limbs else None+ child_names = [limbs[c]["name"] for c in sort_children(children_of[uid])]+ bind = bind_pose.get(uid)+ bind_err = None+ if bind is not None:+ bind_err = float(np.linalg.norm(world - bind))+ pre = parts["pre_rotation_euler_deg"]+ bones.append(+ {+ "name": obj["name"],+ "parent": parent_name,+ "children": child_names,+ "pre_rotation_euler_deg": _tolist(pre),+ "pre_rotation_matrix": _tolist(euler_xyz_deg_to_matrix(pre)),+ "lcl_translation": _tolist(parts["lcl_translation"]),+ "lcl_rotation_euler_deg": _tolist(parts["lcl_rotation_euler_deg"]),+ "lcl_scaling": _tolist(parts["lcl_scaling"]),+ "post_rotation_euler_deg": _tolist(parts["post_rotation_euler_deg"]),+ "rest_local": {+ "matrix": _tolist(local),+ "translation": _tolist(local[:3, 3]),+ "rotation_matrix": _tolist(local[:3, :3]),+ "quaternion_wxyz": _tolist(matrix_to_quat_wxyz(local[:3, :3])),+ "scale": _tolist(np.array([np.linalg.norm(local[:3, i]) for i in range(3)])),+ },+ "rest_world": {+ "matrix": _tolist(world),+ "translation": _tolist(world[:3, 3]),+ "translation_m": _tolist(world[:3, 3] * meters_per_fbx),+ "rotation_matrix": _tolist(world[:3, :3]),+ "quaternion_wxyz": _tolist(matrix_to_quat_wxyz(world[:3, :3])),+ },+ "bind_pose_world_matrix": _tolist(bind) if bind is not None else None,+ "bind_pose_match_error": bind_err,+ }+ )++ hierarchy = [+ {+ "name": limbs[uid]["name"],+ "parent": limbs[parent_of[uid]]["name"] if parent_of[uid] in limbs else None,+ "children": [limbs[c]["name"] for c in sort_children(children_of[uid])],+ }+ for uid in ordered+ ]++ return {+ "source_fbx": _describe_path(fbx_path),+ "fbx_version": version,+ "character": fbx_path.stem,+ "coord_system": {+ "up": axis_name(int(settings.get("UpAxis", 1)), int(settings.get("UpAxisSign", 1))),+ "front": axis_name(int(settings.get("FrontAxis", 2)), int(settings.get("FrontAxisSign", 1))),+ "coord": axis_name(int(settings.get("CoordAxis", 0)), int(settings.get("CoordAxisSign", 1))),+ "handedness": "right",+ "rotation_order": "XYZ",+ "fbx_unit": "centimeter" if abs(float(settings.get("UnitScaleFactor", 1.0)) - 1.0) < 1e-6 else "custom",+ "unit_scale_factor": float(settings.get("UnitScaleFactor", 1.0)),+ "meters_per_fbx_unit": meters_per_fbx,+ },+ "transform_convention": (+ "Maya/FBX: M_local = T * Roff * Rp * Rpre * R * Rpost^{-1} * Rp^{-1} * S; "+ "M_world = M_parent_world * M_local. Euler XYZ in degrees, column-vector matrices."+ ),+ "root": limbs[roots[0]]["name"] if roots else None,+ "bone_count": len(bones),+ "hierarchy": hierarchy,+ "bones": bones,+ }diff --git a/motion2mixamorig/mixamo/gltf.py b/motion2mixamorig/mixamo/gltf.pynew file mode 100644index 0000000..694cfb8--- /dev/null+++ b/motion2mixamorig/mixamo/gltf.py@@ -0,0 +1,240 @@+"""Write a skinned Mixamo character + its animation as a .glb (glTF 2.0).++Blender opens this in one import: mesh, skin weights, and the clip. Node TRS+is the full local Mixamo transform `T * Rpre * R_anim` (glTF has no+PreRotation), so the importer's linear-blend skin matches this repo's.++Units are metres (Y-up). Vertex influences are the top four weights.+"""++from __future__ import annotations++import json+import struct+from pathlib import Path++import numpy as np++from .fbx_skeleton import matrix_to_quat_wxyz+from .kinematics import continue_quaternion, local_matrix+from .skinned_mesh import YBotAsset++_GLTF_MAGIC = 0x46546C67+_JSON_CHUNK = 0x4E4F534A+_BIN_CHUNK = 0x004E4942+++class _Bin:+ def __init__(self) -> None:+ self._parts: list[bytes] = []+ self._cursor = 0++ def add(self, data: bytes) -> tuple[int, int]:+ offset = self._cursor+ self._parts.append(data)+ self._cursor += len(data)+ pad = (4 - len(data) % 4) % 4+ if pad:+ self._parts.append(b"\x00" * pad)+ self._cursor += pad+ return offset, len(data)++ def add_array(self, array: np.ndarray) -> tuple[int, int]:+ return self.add(np.ascontiguousarray(array).tobytes())++ def dumps(self) -> bytes:+ return b"".join(self._parts)+++def _quat_xyzw(rmat: np.ndarray) -> np.ndarray:+ wxyz = matrix_to_quat_wxyz(rmat)+ return np.array([wxyz[1], wxyz[2], wxyz[3], wxyz[0]], dtype=np.float32)+++def _scale_affine(mat: np.ndarray, scale: float) -> np.ndarray:+ """Metres from centimetres: R stays, translation is multiplied by `scale`."""+ out = np.asarray(mat, dtype=np.float64).reshape(4, 4).copy()+ out[:3, 3] *= scale+ return out+++def _top4_weights(weights: np.ndarray) -> tuple[np.ndarray, np.ndarray]:+ n_bones = weights.shape[1]+ take = min(4, n_bones)+ order = np.argpartition(-weights, take - 1, axis=1)[:, :take]+ w = np.take_along_axis(weights, order, axis=1)+ w = w / np.maximum(w.sum(axis=1, keepdims=True), 1e-8)+ if take < 4:+ pad_i = np.zeros((weights.shape[0], 4 - take), dtype=order.dtype)+ pad_w = np.zeros((weights.shape[0], 4 - take), dtype=w.dtype)+ order = np.concatenate([order, pad_i], axis=1)+ w = np.concatenate([w, pad_w], axis=1)+ return order.astype(np.uint16), w.astype(np.float32)+++def _vertex_normals(vertices: np.ndarray, faces: np.ndarray) -> np.ndarray:+ normals = np.zeros_like(vertices, dtype=np.float64)+ p0, p1, p2 = vertices[faces[:, 0]], vertices[faces[:, 1]], vertices[faces[:, 2]]+ face_n = np.cross(p1 - p0, p2 - p0)+ for i in range(3):+ np.add.at(normals, faces[:, i], face_n)+ length = np.linalg.norm(normals, axis=1, keepdims=True)+ return (normals / np.where(length < 1e-12, 1.0, length)).astype(np.float32)+++def write_glb(+ path: Path,+ asset: YBotAsset,+ r_anim_frames: list[dict[str, np.ndarray]],+ hips_cm_frames: list[np.ndarray],+ fps: float,+) -> Path:+ """Skinned character + clip → a single `.glb` file."""+ rig = asset.rig+ scale = float(rig.meters_per_unit)+ identity = np.eye(3)+ n_bones = len(rig.order)+ bone_index = {name: i for i, name in enumerate(rig.order)}++ blob = _Bin()+ accessors: list[dict] = []+ views: list[dict] = []++ def accessor(array: np.ndarray, typ: str, component: int, *, target: int | None = None,+ normalized: bool = False) -> int:+ offset, length = blob.add_array(array)+ views.append({"buffer": 0, "byteOffset": offset, "byteLength": length,+ **({"target": target} if target is not None else {})})+ acc: dict = {+ "bufferView": len(views) - 1,+ "componentType": component,+ "count": int(array.shape[0]),+ "type": typ,+ }+ if normalized:+ acc["normalized"] = True+ if typ in {"VEC3", "VEC4", "SCALAR"} and array.dtype.kind == "f":+ flat = array.reshape(array.shape[0], -1)+ acc["min"] = [float(v) for v in flat.min(axis=0)]+ acc["max"] = [float(v) for v in flat.max(axis=0)]+ accessors.append(acc)+ return len(accessors) - 1++ nodes: list[dict] = []+ for name in rig.order:+ bone = rig.bones[name]+ rest = local_matrix(bone, identity)+ kids = [bone_index[c] for c in bone.children if c in bone_index]+ node = {+ "name": name,+ "translation": [float(v) * scale for v in rest[:3, 3]],+ "rotation": [float(v) for v in _quat_xyzw(rest[:3, :3])],+ }+ if kids:+ node["children"] = kids+ nodes.append(node)++ mesh_node = len(nodes)+ nodes.append({"name": "skinned_mesh", "mesh": 0, "skin": 0})++ ibm = np.stack(+ [_scale_affine(asset.meshes[0].ibm[i], scale) for i in range(n_bones)],+ axis=0,+ ).astype(np.float32)+ # glTF stores matrices column-major.+ ibm_acc = accessor(np.ascontiguousarray(ibm.transpose(0, 2, 1)), "MAT4", 5126)++ primitives = []+ materials = []+ for mesh in asset.meshes:+ verts = (mesh.vertices * scale).astype(np.float32)+ faces = np.ascontiguousarray(mesh.faces, dtype=np.uint32)+ joints, weights = _top4_weights(mesh.weights)+ normals = _vertex_normals(verts.astype(np.float64), mesh.faces)+ color = np.clip(np.asarray(mesh.color, dtype=np.float64).reshape(3), 0, 1)+ materials.append({+ "name": mesh.name,+ "pbrMetallicRoughness": {+ "baseColorFactor": [float(color[0]), float(color[1]), float(color[2]), 1.0],+ "metallicFactor": 0.0,+ "roughnessFactor": 0.7,+ },+ })+ primitives.append({+ "attributes": {+ "POSITION": accessor(verts, "VEC3", 5126, target=34962),+ "NORMAL": accessor(normals, "VEC3", 5126, target=34962),+ "JOINTS_0": accessor(joints, "VEC4", 5123, target=34962),+ "WEIGHTS_0": accessor(weights, "VEC4", 5126, target=34962),+ },+ "indices": accessor(faces.reshape(-1), "SCALAR", 5125, target=34963),+ "material": len(materials) - 1,+ })++ n = len(r_anim_frames)+ times = (np.arange(n, dtype=np.float32) / max(float(fps), 1e-6))+ time_acc = accessor(times, "SCALAR", 5126)++ channels = []+ samplers = []+ prev_q: dict[str, np.ndarray] = {}+ rot_tracks = {name: np.zeros((n, 4), dtype=np.float32) for name in rig.order}+ hips_track = np.zeros((n, 3), dtype=np.float32)+ for i, (r_anim, hips) in enumerate(zip(r_anim_frames, hips_cm_frames)):+ hips_track[i] = np.asarray(hips, dtype=np.float64).reshape(3) * scale+ for name in rig.order:+ bone = rig.bones[name]+ local = local_matrix(+ bone,+ r_anim.get(name, identity),+ hips if name == rig.root else None,+ )+ q_wxyz, _ = continue_quaternion(matrix_to_quat_wxyz(local[:3, :3]), prev_q.get(name))+ prev_q[name] = q_wxyz+ rot_tracks[name][i] = [q_wxyz[1], q_wxyz[2], q_wxyz[3], q_wxyz[0]]++ def add_sampler(values: np.ndarray, typ: str) -> int:+ samplers.append({"input": time_acc, "output": accessor(values, typ, 5126), "interpolation": "LINEAR"})+ return len(samplers) - 1++ hips_s = add_sampler(hips_track.astype(np.float32), "VEC3")+ channels.append({"sampler": hips_s, "target": {"node": bone_index[rig.root], "path": "translation"}})+ for name in rig.order:+ sid = add_sampler(rot_tracks[name], "VEC4")+ channels.append({"sampler": sid, "target": {"node": bone_index[name], "path": "rotation"}})++ gltf = {+ "asset": {"version": "2.0", "generator": "motion2mixamorig"},+ "scene": 0,+ "scenes": [{"nodes": [bone_index[rig.root], mesh_node]}],+ "nodes": nodes,+ "meshes": [{"name": "character", "primitives": primitives}],+ "materials": materials,+ "skins": [{+ "name": "mixamo",+ "joints": list(range(n_bones)),+ "skeleton": bone_index[rig.root],+ "inverseBindMatrices": ibm_acc,+ }],+ "animations": [{"name": "mixamo_clip", "samplers": samplers, "channels": channels}],+ "accessors": accessors,+ "bufferViews": views,+ "buffers": [{"byteLength": blob._cursor}],+ }++ json_bytes = json.dumps(gltf, separators=(",", ":")).encode("utf-8")+ json_pad = (4 - len(json_bytes) % 4) % 4+ json_bytes = json_bytes + b" " * json_pad+ bin_bytes = blob.dumps()+ bin_pad = (4 - len(bin_bytes) % 4) % 4+ bin_bytes = bin_bytes + b"\x00" * bin_pad++ total = 12 + 8 + len(json_bytes) + 8 + len(bin_bytes)+ header = struct.pack("<III", _GLTF_MAGIC, 2, total)+ json_chunk = struct.pack("<II", len(json_bytes), _JSON_CHUNK) + json_bytes+ bin_chunk = struct.pack("<II", len(bin_bytes), _BIN_CHUNK) + bin_bytes++ path = Path(path)+ path.parent.mkdir(parents=True, exist_ok=True)+ path.write_bytes(header + json_chunk + bin_chunk)+ return pathdiff --git a/motion2mixamorig/mixamo/kinematics.py b/motion2mixamorig/mixamo/kinematics.pynew file mode 100644index 0000000..a5dcd41--- /dev/null+++ b/motion2mixamorig/mixamo/kinematics.py@@ -0,0 +1,420 @@+"""Mixamo Y Bot FK that keeps FBX PreRotation as a separate node.++Maya/FBX (and Assimp's extra PreRotation node) compose as:++ M_local = T * Rpre * R_anim+ M_world = M_parent_world * M_local++R_anim is the animated Lcl Rotation. Rpre is rest-only and is never baked+into R_anim. Bone lengths live in T (Lcl Translation) and stay constant.+"""++from __future__ import annotations++import json+from dataclasses import dataclass, field+from pathlib import Path+from typing import Any, Iterable++import numpy as np++from .fbx_skeleton import matrix_to_quat_wxyz+++def _as_vec3(value: Iterable[float]) -> np.ndarray:+ return np.asarray(list(value)[:3], dtype=np.float64)+++def _as_mat3(value) -> np.ndarray:+ return np.asarray(value, dtype=np.float64).reshape(3, 3)+++def _as_mat4(value) -> np.ndarray:+ return np.asarray(value, dtype=np.float64).reshape(4, 4)+++def normalize(v: np.ndarray, fallback: np.ndarray | None = None) -> np.ndarray:+ v = np.asarray(v, dtype=np.float64).reshape(3)+ n = float(np.linalg.norm(v))+ if n < 1e-10:+ if fallback is None:+ raise ValueError("cannot normalize a near-zero vector")+ return np.asarray(fallback, dtype=np.float64).reshape(3)+ return v / n+++def rotation_aligning(src: np.ndarray, dst: np.ndarray) -> np.ndarray:+ """Shortest rotation R with R @ src = dst (column vectors). Twist-free swing."""+ a = normalize(src, np.array([0.0, 1.0, 0.0]))+ b = normalize(dst, np.array([0.0, 1.0, 0.0]))+ cosine = float(np.clip(a @ b, -1.0, 1.0))+ axis = np.cross(a, b)+ sine = float(np.linalg.norm(axis))+ if cosine > 0.999999:+ return np.eye(3)+ if cosine < -0.999999:+ helper = np.array([1.0, 0.0, 0.0]) if abs(a[0]) < 0.9 else np.array([0.0, 1.0, 0.0])+ axis = normalize(np.cross(a, helper))+ return 2.0 * np.outer(axis, axis) - np.eye(3)+ skew = np.array(+ [[0.0, -axis[2], axis[1]], [axis[2], 0.0, -axis[0]], [-axis[1], axis[0], 0.0]],+ dtype=np.float64,+ )+ return np.eye(3) + skew + skew @ skew * ((1.0 - cosine) / (sine * sine))+++def orthonormal_frame(left: np.ndarray, up: np.ndarray) -> np.ndarray:+ """Right-handed Mixamo axes: columns = (left=+X, up=+Y, forward=+Z)."""+ up_u = normalize(up, np.array([0.0, 1.0, 0.0]))+ left_rej = left - up_u * float(left @ up_u)+ if float(np.linalg.norm(left_rej)) < 1e-8:+ helper = np.array([1.0, 0.0, 0.0]) if abs(up_u[0]) < 0.9 else np.array([0.0, 0.0, 1.0])+ left_rej = np.cross(helper, up_u)+ left_u = normalize(left_rej)+ fwd = normalize(np.cross(left_u, up_u))+ left_u = normalize(np.cross(up_u, fwd))+ return np.column_stack([left_u, up_u, fwd])+++def angle_deg(a: np.ndarray, b: np.ndarray) -> float:+ ua = normalize(a, np.array([0.0, 1.0, 0.0]))+ ub = normalize(b, np.array([0.0, 1.0, 0.0]))+ return float(np.degrees(np.arccos(np.clip(ua @ ub, -1.0, 1.0))))+++def mat4(rotation: np.ndarray, translation: np.ndarray) -> np.ndarray:+ out = np.eye(4)+ out[:3, :3] = rotation+ out[:3, 3] = translation+ return out+++def rh_axes(lateral: np.ndarray, up: np.ndarray) -> np.ndarray:+ """Right-handed columns (lateral, up, lateral × up)."""+ up_u = normalize(up, np.array([0.0, 1.0, 0.0]))+ lat_rej = lateral - up_u * float(lateral @ up_u)+ if float(np.linalg.norm(lat_rej)) < 1e-8:+ helper = np.array([1.0, 0.0, 0.0]) if abs(up_u[0]) < 0.9 else np.array([0.0, 0.0, 1.0])+ lat_rej = np.cross(helper, up_u)+ lat_u = normalize(lat_rej)+ fwd = normalize(np.cross(lat_u, up_u))+ lat_u = normalize(np.cross(up_u, fwd))+ up_u = normalize(np.cross(fwd, lat_u))+ return np.column_stack([lat_u, up_u, fwd])+++def human_body_frame(joints_or_right, up: np.ndarray | None = None) -> np.ndarray:+ """Human torso columns (right, up, chest-forward).++ chest-forward = right × up (matches the source video: f253 faces +X).+ up × right is the back.+ """+ if up is None:+ raise TypeError("human_body_frame(right, up)")+ return rh_axes(joints_or_right, up)+++def mixamo_bone_frame(left: np.ndarray, up: np.ndarray) -> np.ndarray:+ """Mixamo rest bone columns (left=+X, up=+Y, chest-forward=+Z)."""+ return rh_axes(left, up)+++def body_frame_from_landmarks(right: np.ndarray, up: np.ndarray, chest_hint: np.ndarray) -> np.ndarray:+ """Columns (right, up, forward). Forward is whichever of right×up / up×right matches chest_hint.++ Does not recross `right` from up×forward. That would force det +1 by flipping+ Mixamo's anatomical right onto character left.+ """+ up_u = normalize(up, np.array([0.0, 1.0, 0.0]))+ right_rej = right - up_u * float(np.asarray(right, dtype=np.float64) @ up_u)+ right_u = normalize(right_rej)+ fwd_ru = np.cross(right_u, up_u)+ fwd_ur = np.cross(up_u, right_u)+ hint = np.asarray(chest_hint, dtype=np.float64).reshape(3)+ fwd = fwd_ur if float(hint @ fwd_ur) >= float(hint @ fwd_ru) else fwd_ru+ return np.column_stack([right_u, up_u, normalize(fwd)])+++def body_align_from_bases(r_human: np.ndarray, r_mixamo: np.ndarray, *, proper: bool = False) -> np.ndarray:+ """Linear map R with R @ r_human ≈ r_mixamo.++ The anatomical Human→Mixamo map (right→right, up→up, chest→chest) has det −1+ because Mixamo rest +X is character left. `proper=True` polar-projects onto+ SO(3). Identity and rotY(180) are equally close; neither matches all three axes.+ """+ a = np.asarray(r_mixamo, dtype=np.float64) @ np.asarray(r_human, dtype=np.float64).T+ if not proper:+ return a+ u, _, vt = np.linalg.svd(a)+ r = u @ vt+ if float(np.linalg.det(r)) < 0.0:+ u = u.copy()+ u[:, -1] *= -1.0+ r = u @ vt+ return r+++# Maps the capture's anatomical triad onto Mixamo's:+# Human (right, up, chest) = (+X, +Y, −Z)+# Mixamo (right, up, chest) = (−X, +Y, +Z) # rest +X is character left+# Anatomy obeys chest = up × right, so both triads are left-handed and the map+# between them is a proper rotation: rotY(180), det +1, no reflection.+#+# The capture's chest axis is −Z, measured rather than assumed: over all frames+# the toes-anterior and knees-bow-anterior directions agree to 5.4 deg mean and+# both sit ~167 deg from +Z. Taking chest as +Z makes this map the reflection+# diag(-1, 1, 1), which lands the body's front on Mixamo's back and inverts+# laterality — the torso-backward bug.+R_BODY_ALIGN = np.diag([-1.0, 1.0, -1.0])+++def human_to_mixamo(p: np.ndarray) -> np.ndarray:+ """Human world (m) → Mixamo FBX world by one global body-frame map.++ Human: +X = anatomical right, +Y = up, −Z = chest-forward.+ Mixamo rest: +X = anatomical left, +Y = up, +Z = chest-forward (toes / FBX front).++ p_mixamo = R_BODY_ALIGN @ p_human = [-x, y, -z]+ """+ p = np.asarray(p, dtype=np.float64)+ return np.asarray(p, dtype=np.float64) @ R_BODY_ALIGN.T+++@dataclass+class MixamoBone:+ name: str+ parent: str | None+ children: list[str]+ pre_rotation: np.ndarray+ lcl_translation_cm: np.ndarray+ rest_world: np.ndarray+ rest_local: np.ndarray+ pre_rotation_euler_deg: np.ndarray+++@dataclass+class MixamoRig:+ bones: dict[str, MixamoBone]+ order: list[str]+ meters_per_unit: float+ source: Path+ coord_system: dict = field(default_factory=dict)++ @property+ def root(self) -> str:+ return self.order[0]++ def child_offset_cm(self, parent: str, child: str) -> np.ndarray:+ return self.bones[child].lcl_translation_cm.copy()+++def rig_from_skeleton_data(data: dict[str, Any], source: Path) -> MixamoRig:+ """Build a MixamoRig from the dict `fbx_skeleton.extract_skeleton` returns."""+ bones: dict[str, MixamoBone] = {}+ for raw in data["bones"]:+ name = str(raw["name"])+ bones[name] = MixamoBone(+ name=name,+ parent=raw["parent"],+ children=list(raw["children"]),+ pre_rotation=_as_mat3(raw["pre_rotation_matrix"]),+ lcl_translation_cm=_as_vec3(raw["lcl_translation"]),+ rest_world=_as_mat4(raw["rest_world"]["matrix"]),+ rest_local=_as_mat4(raw["rest_local"]["matrix"]),+ pre_rotation_euler_deg=_as_vec3(raw["pre_rotation_euler_deg"]),+ )+ return MixamoRig(+ bones=bones,+ order=[b["name"] for b in data["bones"]],+ meters_per_unit=float(data["coord_system"]["meters_per_fbx_unit"]),+ source=Path(source),+ coord_system=dict(data["coord_system"]),+ )+++def load_mixamo_rig(path: Path) -> MixamoRig:+ """Load a core-bone rig from a previously exported skeleton JSON."""+ path = Path(path)+ data = json.loads(path.read_text(encoding="utf-8"))+ return rig_from_skeleton_data(data, path)+++def local_matrix(bone: MixamoBone, r_anim: np.ndarray, translation_cm: np.ndarray | None = None) -> np.ndarray:+ """Assimp-equivalent local node: T * Rpre * R_anim."""+ t = bone.lcl_translation_cm if translation_cm is None else np.asarray(translation_cm, dtype=np.float64)+ return mat4(bone.pre_rotation @ r_anim, t)+++def forward_kinematics(+ rig: MixamoRig,+ r_anim: dict[str, np.ndarray] | None = None,+ hips_translation_cm: np.ndarray | None = None,+ lcl_translations_cm: dict[str, np.ndarray] | None = None,+) -> dict[str, np.ndarray]:+ """World 4x4 for every bone. Missing R_anim entries stay identity (rest)."""+ anim = r_anim or {}+ extra_t = lcl_translations_cm or {}+ world: dict[str, np.ndarray] = {}+ for name in rig.order:+ bone = rig.bones[name]+ rot = np.asarray(anim.get(name, np.eye(3)), dtype=np.float64)+ if name == rig.root and hips_translation_cm is not None:+ trans = hips_translation_cm+ elif name in extra_t:+ trans = extra_t[name]+ else:+ trans = None+ local = local_matrix(bone, rot, trans)+ if bone.parent is None:+ world[name] = local+ else:+ world[name] = world[bone.parent] @ local+ return world+++def world_positions_m(rig: MixamoRig, world: dict[str, np.ndarray]) -> dict[str, np.ndarray]:+ scale = rig.meters_per_unit+ return {name: mat[:3, 3] * scale for name, mat in world.items()}+++def bone_direction_world(rig: MixamoRig, world: dict[str, np.ndarray], parent: str, child: str) -> np.ndarray:+ """World direction of parent→child. Independent of child's own rotation."""+ offset = rig.child_offset_cm(parent, child)+ return world[parent][:3, :3] @ offset+++def anim_from_world_rotation(bone: MixamoBone, r_parent_world: np.ndarray, r_world: np.ndarray) -> np.ndarray:+ """R_anim such that R_parent * Rpre * R_anim = R_world."""+ return bone.pre_rotation.T @ r_parent_world.T @ r_world+++def swing_anim_to_direction(+ rig: MixamoRig,+ world: dict[str, np.ndarray],+ bone_name: str,+ child_name: str | None,+ target_dir_world: np.ndarray,+ d_local: np.ndarray | None = None,+) -> np.ndarray:+ """Lcl Rotation that aims this bone's rest axis at target_dir_world.++ Rest axis is the child's Lcl Translation (Mixamo local +Y) after PreRotation:+ R_parent * Rpre * R_anim * d_local = d_target+ """+ bone = rig.bones[bone_name]+ parent_r = np.eye(3) if bone.parent is None else world[bone.parent][:3, :3]+ if d_local is None:+ if child_name is None:+ raise ValueError(f"{bone_name}: need child_name or d_local")+ d_local = rig.child_offset_cm(bone_name, child_name)+ d_in_anim = bone.pre_rotation.T @ parent_r.T @ normalize(target_dir_world)+ return rotation_aligning(d_local, d_in_anim)+++def frame_from_primary_secondary(primary: np.ndarray, secondary: np.ndarray) -> np.ndarray:+ """Orthonormal basis: columns = (primary, secondary_ortho, primary × secondary_ortho)."""+ p = normalize(primary, np.array([0.0, 1.0, 0.0]))+ s = secondary - p * float(secondary @ p)+ if float(np.linalg.norm(s)) < 1e-8:+ helper = np.array([0.0, 1.0, 0.0]) if abs(p[1]) < 0.9 else np.array([1.0, 0.0, 0.0])+ s = np.cross(helper, p)+ s = normalize(s)+ t = np.cross(p, s)+ return np.column_stack([p, s, t])+++def frame_anim_from_two_vectors(+ bone: MixamoBone,+ r_parent_world: np.ndarray,+ rest_primary_local: np.ndarray,+ rest_secondary_local: np.ndarray,+ tgt_primary_world: np.ndarray,+ tgt_secondary_world: np.ndarray,+) -> np.ndarray:+ """Lcl Rotation mapping rest local (primary, secondary) onto world targets.++ Used for feet: primary = ankle→toe (forward), secondary = world up (twist).+ """+ r_rest_world = r_parent_world @ bone.pre_rotation+ r_rest_frame = frame_from_primary_secondary(+ r_rest_world @ rest_primary_local, r_rest_world @ rest_secondary_local+ )+ r_tgt_frame = frame_from_primary_secondary(tgt_primary_world, tgt_secondary_world)+ r_world = (r_tgt_frame @ r_rest_frame.T) @ r_rest_world+ return bone.pre_rotation.T @ r_parent_world.T @ r_world+++def frame_anim_from_axes(+ bone: MixamoBone,+ r_parent_world: np.ndarray,+ rest_left_local: np.ndarray,+ rest_up_local: np.ndarray,+ tgt_left_world: np.ndarray,+ tgt_up_world: np.ndarray,+) -> np.ndarray:+ """Lcl Rotation that maps rest local (left, up) onto the world target axes.++ Used for Hips and Spine2 so clavicle attachments keep a chest twist, not a+ free swing around the spine.+ """+ r_rest_world = r_parent_world @ bone.pre_rotation+ r_rest_frame = orthonormal_frame(r_rest_world @ rest_left_local, r_rest_world @ rest_up_local)+ r_tgt_frame = orthonormal_frame(tgt_left_world, tgt_up_world)+ r_world = (r_tgt_frame @ r_rest_frame.T) @ r_rest_world+ return bone.pre_rotation.T @ r_parent_world.T @ r_world++++def quat_wxyz(rmat: np.ndarray) -> np.ndarray:+ return matrix_to_quat_wxyz(rmat)+++def local_aim_axis(rig: MixamoRig, bone_name: str) -> np.ndarray:+ """Rest bone axis in the bone's local (pre-R_anim) frame. Mixamo +Y, or the first child offset."""+ bone = rig.bones[bone_name]+ if bone.children:+ return rig.child_offset_cm(bone_name, bone.children[0])+ return np.array([0.0, 1.0, 0.0])+++def geodesic_angle_deg(r_a: np.ndarray, r_b: np.ndarray) -> float:+ rel = np.asarray(r_a, dtype=np.float64).T @ np.asarray(r_b, dtype=np.float64)+ cosine = (float(np.trace(rel)) - 1.0) * 0.5+ return float(np.degrees(np.arccos(np.clip(cosine, -1.0, 1.0))))+++def rotation_180_around(axis: np.ndarray) -> np.ndarray:+ n = normalize(axis, np.array([0.0, 1.0, 0.0]))+ return 2.0 * np.outer(n, n) - np.eye(3)+++def pick_closest_aim_twist(+ r_curr: np.ndarray,+ r_prev: np.ndarray | None,+ axis: np.ndarray,+) -> tuple[np.ndarray, bool]:+ """Keep the independent aim. If a 180° twist around the bone axis is closer to r_prev, use it.++ Does not blend or smooth: only the discrete antipodal twist that still aims the same way.+ """+ r_curr = np.asarray(r_curr, dtype=np.float64)+ if r_prev is None:+ return r_curr, False+ r_prev = np.asarray(r_prev, dtype=np.float64)+ r_flip = r_curr @ rotation_180_around(axis)+ if geodesic_angle_deg(r_flip, r_prev) + 1e-9 < geodesic_angle_deg(r_curr, r_prev):+ return r_flip, True+ return r_curr, False+++def continue_quaternion(q_curr: np.ndarray, q_prev: np.ndarray | None) -> tuple[np.ndarray, bool]:+ """If dot(q_t, q_{t-1}) < 0, use -q_t. Quaternion double-cover only; same rotation matrix."""+ q = np.asarray(q_curr, dtype=np.float64).reshape(4)+ n = float(np.linalg.norm(q))+ if n < 1e-12:+ q = np.array([1.0, 0.0, 0.0, 0.0])+ else:+ q = q / n+ if q_prev is not None and float(np.dot(q, np.asarray(q_prev, dtype=np.float64).reshape(4))) < 0.0:+ return -q, True+ return q, Falsediff --git a/motion2mixamorig/mixamo/render.py b/motion2mixamorig/mixamo/render.pynew file mode 100644index 0000000..8c53e07--- /dev/null+++ b/motion2mixamorig/mixamo/render.py@@ -0,0 +1,609 @@+"""Rendering for the Y Bot rig and mesh, with cameras taken from the calibration.++The anatomical FRONT / BACK / LEFT / RIGHT cameras are built from the validated+Y Bot Right / Up / Forward axes using the same construction that produced the+accepted orientation-check images. There is no image flip and no "camera on −Z"+rule compensating for a mapping error.+"""++from __future__ import annotations++from dataclasses import dataclass++import numpy as np++from .kinematics import normalize++BG_BGR = (18, 18, 20)+GRID_BGR = (58, 56, 52)+SHADOW_BGR = (8, 8, 9)+++@dataclass(frozen=True)+class Theme:+ """Panel palette. The floor runs far -> near so it reads as receding."""++ bg: tuple[int, int, int]+ floor_far: tuple[int, int, int]+ floor_near: tuple[int, int, int]+ grid: tuple[int, int, int]+ shadow: tuple[int, int, int]+ shadow_alpha: float+ # White joint dots vanish on a pale floor and leave the limbs looking dashed.+ joint: tuple[int, int, int]+++# Black backdrop with the floor only just lifted off it, so the grid still reads+# as a surface while everything but the dancer stays out of the way.+DARK_THEME = Theme((0, 0, 0), (10, 10, 11), (22, 22, 25), (52, 51, 48), (0, 0, 0), 0.75,+ (242, 242, 242))+# Sampled off the Mixamo viewport: flat 160 backdrop, floor 193 at the far edge+# rising to 204 underfoot, grid lines a little darker than the floor.+LIGHT_THEME = Theme((160, 160, 160), (193, 193, 193), (208, 208, 208), (168, 168, 168),+ (108, 108, 108), 0.42, (48, 48, 48))+# For the character grid. Same mood as DARK_THEME but off black, because a+# navy or black outfit against a black backdrop has no silhouette to read.+GRID_THEME = Theme((30, 29, 27), (44, 43, 41), (66, 65, 62), (94, 92, 88), (0, 0, 0), 0.50,+ (242, 242, 242))+PANEL_BAR_BGR = (16, 16, 18)+TEXT_BGR = (240, 240, 240)+MUTED_BGR = (136, 136, 136)+FLAG_BGR = (60, 60, 220)+LEFT_BGR = (232, 155, 76)+RIGHT_BGR = (90, 90, 224)+TORSO_BGR = (76, 200, 230)++MESH_COLORS_BGR = {+ "Alpha_Surface": np.array([200, 188, 180], dtype=np.float64),+ "Alpha_Joints": np.array([112, 104, 98], dtype=np.float64),+}++# The teal the Mixamo viewport gives the Y Bot, sampled off a screenshot.+MESH_COLORS_MIXAMO_BGR = {+ "Alpha_Surface": np.array([205, 180, 140], dtype=np.float64),+ "Alpha_Joints": np.array([120, 104, 84], dtype=np.float64),+}++# White shell over the stock grey joints. The white stops just under 255 on+# purpose: at 255 the lit faces clip and lose their shading.+MESH_COLORS_WHITE_BGR = {+ "Alpha_Surface": np.array([246, 246, 248], dtype=np.float64),+ "Alpha_Joints": np.array([112, 104, 98], dtype=np.float64),+}++VIEWS = ("front", "back", "left", "right")+++@dataclass(frozen=True)+class Camera:+ """Orthographic anatomical view derived from the calibrated rig axes."""++ view: str+ look: np.ndarray+ cam_right: np.ndarray+ cam_up: np.ndarray+ center: np.ndarray+ scale: float+ width: int+ height: int++ def project(self, pts: np.ndarray) -> np.ndarray:+ d = np.asarray(pts, dtype=np.float64).reshape(-1, 3) - self.center+ u = d @ self.cam_right+ v = d @ self.cam_up+ x = self.width * 0.5 + u * self.scale+ y = self.height * 0.5 - v * self.scale+ return np.column_stack([x, y])++ def depth(self, pts: np.ndarray) -> np.ndarray:+ d = np.asarray(pts, dtype=np.float64).reshape(-1, 3) - self.center+ return d @ self.look++ def clip_near(self, poly: np.ndarray) -> np.ndarray:+ """Nothing is behind an orthographic camera, so this is the identity."""+ return np.asarray(poly, dtype=np.float64).reshape(-1, 3)++ def clip_segment(self, a: np.ndarray, b: np.ndarray):+ return a, b+++@dataclass(frozen=True)+class PerspectiveCamera:+ """Pinhole view. Same project / depth / clip_near surface as Camera.++ An orthographic camera cannot show a floor receding to a vanishing point;+ every horizontal line stays parallel and evenly spaced. This one can.+ """++ eye: np.ndarray+ look: np.ndarray+ cam_right: np.ndarray+ cam_up: np.ndarray+ focal: float+ width: int+ height: int+ near: float = 0.05++ def project(self, pts: np.ndarray) -> np.ndarray:+ d = np.asarray(pts, dtype=np.float64).reshape(-1, 3) - self.eye+ z = np.maximum(d @ self.look, self.near)+ u = (d @ self.cam_right) / z * self.focal+ v = (d @ self.cam_up) / z * self.focal+ return np.column_stack([self.width * 0.5 + u, self.height * 0.5 - v])++ def depth(self, pts: np.ndarray) -> np.ndarray:+ d = np.asarray(pts, dtype=np.float64).reshape(-1, 3) - self.eye+ return d @ self.look++ def clip_near(self, poly: np.ndarray) -> np.ndarray:+ """Sutherland-Hodgman against z = near, so nothing behind the eye wraps."""+ p = np.asarray(poly, dtype=np.float64).reshape(-1, 3)+ z = self.depth(p) - self.near+ out = []+ for i in range(len(p)):+ j = (i + 1) % len(p)+ if z[i] >= 0.0:+ out.append(p[i])+ if (z[i] >= 0.0) != (z[j] >= 0.0):+ out.append(p[i] + z[i] / (z[i] - z[j]) * (p[j] - p[i]))+ return np.asarray(out, dtype=np.float64).reshape(-1, 3)++ def clip_segment(self, a: np.ndarray, b: np.ndarray):+ """Trim a world segment to the half-space in front of the eye, or drop it."""+ za, zb = (float(v) for v in self.depth(np.stack([a, b])) - self.near)+ if za < 0.0 and zb < 0.0:+ return None+ if za < 0.0:+ a = a + za / (za - zb) * (b - a)+ elif zb < 0.0:+ b = b + zb / (zb - za) * (a - b)+ return a, b++ @property+ def fov_deg(self) -> float:+ return float(2.0 * np.degrees(np.arctan(self.height * 0.5 / self.focal)))+++def view_basis(axes: dict, view: str) -> tuple[np.ndarray, np.ndarray, np.ndarray]:+ """Same anatomical camera construction as the accepted orientation check.++ FRONT: stand on +Forward and look at the face/chest.+ BACK: stand on −Forward. LEFT / RIGHT: stand on that anatomical side.+ """+ right = np.asarray(axes["right"], dtype=np.float64)+ up = np.asarray(axes["up"], dtype=np.float64)+ fwd = np.asarray(axes["forward"], dtype=np.float64)+ eye_dir = {"front": fwd, "back": -fwd, "left": -right, "right": right}[view]+ look = normalize(-eye_dir)+ cam_right = np.cross(look, up)+ if float(np.linalg.norm(cam_right)) < 1e-8:+ cam_right = np.cross(look, fwd)+ cam_right = normalize(cam_right)+ cam_up = normalize(np.cross(cam_right, look))+ return look, cam_right, cam_up+++def pitch_down(look: np.ndarray, cam_up: np.ndarray, deg: float) -> tuple[np.ndarray, np.ndarray]:+ """Rotate a view basis down by `deg` about the unchanged camera right axis.++ A level orthographic camera sees a horizontal plane exactly edge-on, so a+ floor drawn under the character collapses to a line. Pitching down opens the+ plane out without touching the horizontal framing.+ """+ t = np.radians(deg)+ return (+ normalize(look * np.cos(t) - cam_up * np.sin(t)),+ normalize(cam_up * np.cos(t) + look * np.sin(t)),+ )+++def build_camera(+ axes: dict,+ view: str,+ points: np.ndarray,+ width: int,+ height: int,+ *,+ fill: float = 0.82,+ pitch_deg: float = 0.0,+) -> Camera:+ """Fixed camera framing every point of the whole sequence.++ `points` is the full set of world points over all frames, so the framing is+ computed once. Nothing is re-centred per frame.+ """+ look, cam_right, cam_up = view_basis(axes, view)+ if pitch_deg:+ look, cam_up = pitch_down(look, cam_up, pitch_deg)+ pts = np.asarray(points, dtype=np.float64).reshape(-1, 3)+ u = pts @ cam_right+ v = pts @ cam_up+ center_u = 0.5 * (float(u.min()) + float(u.max()))+ center_v = 0.5 * (float(v.min()) + float(v.max()))+ center = center_u * cam_right + center_v * cam_up+ span_u = max(float(u.max() - u.min()), 1e-3)+ span_v = max(float(v.max() - v.min()), 1e-3)+ scale = fill * min(width / span_u, height / span_v)+ return Camera(+ view=view,+ look=look,+ cam_right=cam_right,+ cam_up=cam_up,+ center=center,+ scale=scale,+ width=width,+ height=height,+ )+++def build_perspective_camera(+ axes: dict,+ points: np.ndarray,+ width: int,+ height: int,+ *,+ fov_deg: float = 28.0,+ azimuth_deg: float = 0.0,+ elevation_deg: float = 10.0,+ fill: float = 0.94,+ iterations: int = 12,+) -> PerspectiveCamera:+ """Pinhole camera orbiting the sequence, framed on every point of it.++ azimuth turns the eye around the character's up axis from dead ahead;+ elevation lifts it above the floor. The eye distance and the aim point are+ then solved together so the whole clip fits, since with perspective the+ framing depends on where the eye is and not on a scale alone.+ """+ up = normalize(np.asarray(axes["up"], dtype=np.float64))+ fwd = normalize(np.asarray(axes["forward"], dtype=np.float64))+ right = normalize(np.asarray(axes["right"], dtype=np.float64))++ az, el = np.radians(azimuth_deg), np.radians(elevation_deg)+ offset = normalize(np.cos(el) * (np.cos(az) * fwd + np.sin(az) * right) + np.sin(el) * up)+ look = -offset+ cam_right = normalize(np.cross(look, up))+ cam_up = normalize(np.cross(cam_right, look))+ focal = (height * 0.5) / np.tan(np.radians(fov_deg) * 0.5)++ pts = np.asarray(points, dtype=np.float64).reshape(-1, 3)+ target = 0.5 * (pts.min(axis=0) + pts.max(axis=0))+ dist = float(np.linalg.norm(pts.max(axis=0) - pts.min(axis=0))) * 3.0++ def place(t: np.ndarray, d: float) -> PerspectiveCamera:+ return PerspectiveCamera(t + offset * d, look, cam_right, cam_up,+ float(focal), width, height)++ for _ in range(iterations):+ cam = place(target, dist)+ uv = cam.project(pts)+ lo, hi = uv.min(axis=0), uv.max(axis=0)+ mid = 0.5 * (lo + hi)+ # The world point on the aim plane that lands on `mid`; aim there instead.+ target = target + ((mid[0] - width * 0.5) * cam_right+ - (mid[1] - height * 0.5) * cam_up) * dist / focal+ dist *= max((hi[0] - lo[0]) / (fill * width), (hi[1] - lo[1]) / (fill * height))++ # Re-aiming and rescaling interact, so guarantee the fit rather than assume it.+ for _ in range(40):+ cam = place(target, dist)+ uv = cam.project(pts)+ if (uv.min(axis=0) >= 0.0).all() and (uv.max(axis=0) < (width, height)).all():+ break+ dist *= 1.02+ return place(target, dist)+++def content_panel_size(+ axes: dict,+ view: str,+ points: np.ndarray,+ height: int,+ *,+ min_w: int = 560,+ max_w: int = 1280,+) -> tuple[int, int]:+ """Panel shape matching what the sequence actually sweeps out.++ The camera stays fixed; this only stops a wandering character from being+ rendered into a panel whose aspect leaves most of it empty.+ """+ _, cam_right, cam_up = view_basis(axes, view)+ pts = np.asarray(points, dtype=np.float64).reshape(-1, 3)+ span_u = max(float(np.ptp(pts @ cam_right)), 1e-3)+ span_v = max(float(np.ptp(pts @ cam_up)), 1e-3)+ width = int(round(height * span_u / span_v))+ width = int(np.clip(width, min_w, max_w))+ return width + width % 2, height + height % 2+++def edge_bgr(a: str, b: str) -> tuple[int, int, int]:+ token = f"{a}_{b}".lower()+ if "left" in token:+ return LEFT_BGR+ if "right" in token:+ return RIGHT_BGR+ return TORSO_BGR+++def blank_panel(width: int, height: int) -> np.ndarray:+ return np.full((height, width, 3), BG_BGR, dtype=np.uint8)+++@dataclass(frozen=True)+class Ground:+ """A square patch of floor the character is standing on.++ `y` is a world height, so panels sharing a camera and a Ground put their+ floors on the same pixel rows and read as one surface. The patch is wide+ enough that under perspective its far edge crowds the vanishing point.+ """++ y: float+ cx: float+ cz: float+ extent: float = 9.0+ step: float = 0.5++ def corners(self) -> np.ndarray:+ e = self.extent+ return np.array([+ (self.cx - e, self.y, self.cz - e),+ (self.cx + e, self.y, self.cz - e),+ (self.cx + e, self.y, self.cz + e),+ (self.cx - e, self.y, self.cz + e),+ ], dtype=np.float64)++ def lines(self) -> np.ndarray:+ """Grid lines as world endpoint pairs, shape (n, 2, 3)."""+ e = self.extent+ ticks = np.arange(-e, e + 1e-9, self.step)+ segs = [((self.cx + t, self.y, self.cz - e), (self.cx + t, self.y, self.cz + e))+ for t in ticks]+ segs += [((self.cx - e, self.y, self.cz + t), (self.cx + e, self.y, self.cz + t))+ for t in ticks]+ return np.asarray(segs, dtype=np.float64)+++def ground_backdrop(cam, ground: Ground, width: int, height: int,+ theme: Theme = DARK_THEME) -> np.ndarray:+ """The floor drawn once. Camera and floor are both fixed, so frames copy this."""+ import cv2++ img = np.full((height, width, 3), theme.bg, dtype=np.uint8)++ quad = cam.clip_near(ground.corners())+ if len(quad) >= 3:+ poly = np.rint(cam.project(quad)).astype(np.int32)+ mask = np.zeros((height, width), dtype=np.uint8)+ cv2.fillPoly(mask, [poly], 255, cv2.LINE_8)+ top, bottom = float(poly[:, 1].min()), float(poly[:, 1].max())+ t = np.clip((np.arange(height) - top) / max(bottom - top, 1.0), 0.0, 1.0)[:, None]+ ramp = (np.asarray(theme.floor_far, dtype=np.float64) * (1.0 - t)+ + np.asarray(theme.floor_near, dtype=np.float64) * t)+ img = np.where(mask[:, :, None] > 0, ramp[:, None, :], img).astype(np.uint8)++ segs = ground.lines()+ d = cam.depth(segs.mean(axis=1))+ span = max(float(np.ptp(d)), 1e-6)+ drawn: list[float] = []+ for i in np.argsort(-d):+ clipped = cam.clip_segment(segs[i, 0], segs[i, 1])+ if clipped is None:+ continue+ a, b = (cam.project(p)[0] for p in clipped)+ if max(abs(a[0] - b[0]), abs(a[1] - b[1])) < 3.0:+ continue+ # Lines crowding the vanishing point would fuse into a solid band.+ if abs(a[1] - b[1]) < 2.0:+ row = 0.5 * (a[1] + b[1])+ if any(abs(row - r) < 2.0 for r in drawn):+ continue+ drawn.append(row)+ near = float(np.clip((d[i] - d.min()) / span, 0.0, 1.0))+ far = np.asarray(theme.floor_far, dtype=np.float64)+ colour = far + (np.asarray(theme.grid, dtype=np.float64) - far) * (0.35 + 0.65 * near)+ cv2.line(img, tuple(np.rint(a).astype(int)), tuple(np.rint(b).astype(int)),+ tuple(int(round(v)) for v in colour), 1, cv2.LINE_AA)+ return img+++def draw_shadow(img: np.ndarray, cam, pts: np.ndarray, ground: Ground,+ theme: Theme = DARK_THEME) -> None:+ """The figure flattened onto the floor: an overhead light's cast shadow."""+ import cv2++ flat = np.asarray(pts, dtype=np.float64).reshape(-1, 3).copy()+ flat[:, 1] = ground.y+ hull = cv2.convexHull(np.rint(cam.project(flat)).astype(np.int32))+ mask = np.zeros(img.shape[:2], dtype=np.uint8)+ cv2.fillConvexPoly(mask, hull, 255, cv2.LINE_8)+ mask = cv2.GaussianBlur(mask, (0, 0), 9)+ m = (mask.astype(np.float64) / 255.0 * theme.shadow_alpha)[:, :, None]+ blended = (img.astype(np.float64) * (1.0 - m)+ + np.asarray(theme.shadow, dtype=np.float64) * m)+ img[:] = np.clip(blended, 0, 255).astype(np.uint8)+++def draw_skeleton(+ img: np.ndarray,+ pos: dict[str, np.ndarray],+ edges: list[tuple[str, str]],+ cam,+ thickness: int = 3,+ joint_bgr: tuple[int, int, int] = (242, 242, 242),+) -> None:+ import cv2++ names = list(pos.keys())+ uv = cam.project(np.stack([pos[n] for n in names]))+ px = {n: (int(round(uv[i, 0])), int(round(uv[i, 1]))) for i, n in enumerate(names)}+ for a, b in edges:+ if a in px and b in px:+ cv2.line(img, px[a], px[b], edge_bgr(a, b), thickness, cv2.LINE_AA)+ for p in px.values():+ cv2.circle(img, p, 3, joint_bgr, -1, cv2.LINE_AA)+++def draw_mesh(+ img: np.ndarray,+ vertices: list[np.ndarray],+ faces: list[np.ndarray],+ colors: list[np.ndarray],+ cam: Camera,+ ambient: float = 0.30,+) -> None:+ """Back-to-front painter fill with Lambert shading from a head-on key light.++ A colour entry is either one BGR for the whole mesh or one per triangle.+ `ambient` is the floor of the shading term: on a near-black texture the lit+ side already sits low, so too little of it crushes the whole part to black.+ """+ import cv2++ polys: list[np.ndarray] = []+ shades: list[np.ndarray] = []+ depths: list[np.ndarray] = []+ light = -cam.look++ for verts, face, base in zip(vertices, faces, colors):+ uv = cam.project(verts)+ z = cam.depth(verts)+ v0, v1, v2 = face[:, 0], face[:, 1], face[:, 2]+ p0, p1, p2 = verts[v0], verts[v1], verts[v2]+ normals = np.cross(p1 - p0, p2 - p0)+ nlen = np.linalg.norm(normals, axis=1, keepdims=True)+ normals = normals / np.where(nlen < 1e-12, 1.0, nlen)+ facing = normals @ light+ keep = facing > 0.0+ if not np.any(keep):+ continue+ lambert = np.clip(np.abs(facing[keep]), 0.0, 1.0)+ shade = ambient + (1.0 - ambient) * lambert+ tri = np.stack([uv[v0[keep]], uv[v1[keep]], uv[v2[keep]]], axis=1)+ polys.append(np.rint(tri).astype(np.int32))+ tint = np.asarray(base, dtype=np.float64)+ tint = tint[None, :] if tint.ndim == 1 else tint[keep]+ shades.append(shade[:, None] * tint)+ depths.append((z[v0[keep]] + z[v1[keep]] + z[v2[keep]]) / 3.0)++ if not polys:+ return+ tri = np.concatenate(polys, axis=0)+ rgb = np.clip(np.concatenate(shades, axis=0), 0, 255)+ depth = np.concatenate(depths, axis=0)++ order = np.argsort(-depth)+ tri = tri[order]+ rgb = rgb[order].astype(np.int32)+ for i in range(tri.shape[0]):+ cv2.fillConvexPoly(img, tri[i], (int(rgb[i, 0]), int(rgb[i, 1]), int(rgb[i, 2])), cv2.LINE_8)+++def label_panel(img: np.ndarray, text: str, *, flagged: bool = False, height: int = 32) -> None:+ import cv2++ w = img.shape[1]+ cv2.rectangle(img, (0, 0), (w, height), PANEL_BAR_BGR, -1)+ color = FLAG_BGR if flagged else TEXT_BGR+ cv2.putText(img, text, (10, height - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.55, color, 1, cv2.LINE_AA)+++def footer(img: np.ndarray, text: str, height: int = 28) -> None:+ import cv2++ h, w = img.shape[:2]+ cv2.rectangle(img, (0, h - height), (w, h), PANEL_BAR_BGR, -1)+ cv2.putText(img, text, (10, h - 9), cv2.FONT_HERSHEY_SIMPLEX, 0.48, MUTED_BGR, 1, cv2.LINE_AA)+++def hstack_panels(panels: list[np.ndarray]) -> np.ndarray:+ import cv2++ grid = np.hstack(panels)+ x = 0+ for panel in panels[:-1]:+ x += panel.shape[1]+ cv2.line(grid, (x, 0), (x, grid.shape[0]), (40, 40, 44), 1)+ return grid+++def fit_letterbox(frame: np.ndarray, width: int, height: int) -> np.ndarray:+ import cv2++ out = np.full((height, width, 3), BG_BGR, dtype=np.uint8)+ if frame is None:+ return out+ h, w = frame.shape[:2]+ s = min(width / w, height / h)+ nw, nh = max(1, int(round(w * s))), max(1, int(round(h * s)))+ resized = cv2.resize(frame, (nw, nh), interpolation=cv2.INTER_AREA)+ ox, oy = (width - nw) // 2, (height - nh) // 2+ out[oy:oy + nh, ox:ox + nw] = resized+ return out+++COVER_SHIFT_X = 0.0+++def cover_crop_box(+ src_w: int, src_h: int, width: int, height: int, shift_x: float = 0.0+) -> tuple[int, int, int, int]:+ """Crop of a src_w x src_h frame carrying the panel's aspect: x, y, w, h.++ shift_x slides the window by that fraction of its own width, negative = left,+ clamped so the window stays inside the frame.+ """+ crop_w = min(src_w, max(1, int(round(src_h * width / height))))+ crop_h = min(src_h, max(1, int(round(crop_w * height / width))))+ x0 = (src_w - crop_w) // 2 + int(round(shift_x * crop_w))+ y0 = (src_h - crop_h) // 2+ return max(0, min(src_w - crop_w, x0)), max(0, min(src_h - crop_h, y0)), crop_w, crop_h+++def fit_cover(+ frame: np.ndarray, width: int, height: int, *, shift_x: float = COVER_SHIFT_X+) -> np.ndarray:+ """Crop to the panel's aspect, then fill it edge to edge.++ Letterboxing a 16:9 clip into a tall panel spends most of the panel on bars+ and leaves the dancer small next to the rendered panels beside it.+ """+ import cv2++ out = np.full((height, width, 3), BG_BGR, dtype=np.uint8)+ if frame is None:+ return out+ h, w = frame.shape[:2]+ x0, y0, crop_w, crop_h = cover_crop_box(w, h, width, height, shift_x)+ crop = frame[y0:y0 + crop_h, x0:x0 + crop_w]+ return cv2.resize(crop, (width, height), interpolation=cv2.INTER_AREA)+++def write_video(path, frames_bgr: list[np.ndarray], fps: float) -> None:+ import cv2++ path.parent.mkdir(parents=True, exist_ok=True)+ h, w = frames_bgr[0].shape[:2]+ writer = cv2.VideoWriter(str(path), cv2.VideoWriter_fourcc(*"mp4v"), float(fps), (w, h))+ if not writer.isOpened():+ raise RuntimeError(f"Cannot open VideoWriter for {path}")+ for frame in frames_bgr:+ writer.write(frame)+ writer.release()+++def encode_h264(src, dst) -> bool:+ """Re-encode with ffmpeg when available so the clip plays in browsers."""+ import shutil+ import subprocess++ if shutil.which("ffmpeg") is None:+ return False+ cmd = [+ "ffmpeg", "-y", "-loglevel", "error",+ "-i", str(src),+ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "20",+ str(dst),+ ]+ return subprocess.run(cmd, check=False).returncode == 0diff --git a/motion2mixamorig/mixamo/retarget.py b/motion2mixamorig/mixamo/retarget.pynew file mode 100644index 0000000..9ee1a1e--- /dev/null+++ b/motion2mixamorig/mixamo/retarget.py@@ -0,0 +1,524 @@+"""Human → Mixamo per-frame retarget.++FK is T * Rpre * R_anim. Mixamo rest bone lengths are never scaled.+Optional previous-frame rotations pick the 180-degree twist closest to the+last frame and enforce quaternion sign continuity. No temporal smoothing.+"""++from __future__ import annotations++from dataclasses import dataclass, field++import numpy as np++from .kinematics import (+ MixamoRig,+ anim_from_world_rotation,+ continue_quaternion,+ forward_kinematics,+ frame_anim_from_axes,+ frame_anim_from_two_vectors,+ human_to_mixamo,+ local_aim_axis,+ normalize,+ orthonormal_frame,+ pick_closest_aim_twist,+ quat_wxyz,+ swing_anim_to_direction,+ world_positions_m,+)++CONTROLLED = (+ "mixamorig:Hips",+ "mixamorig:Spine",+ "mixamorig:Spine1",+ "mixamorig:Spine2",+ "mixamorig:Neck",+ "mixamorig:Head",+ "mixamorig:LeftShoulder",+ "mixamorig:RightShoulder",+ "mixamorig:LeftArm",+ "mixamorig:RightArm",+ "mixamorig:LeftForeArm",+ "mixamorig:RightForeArm",+ "mixamorig:LeftHand",+ "mixamorig:RightHand",+ "mixamorig:LeftUpLeg",+ "mixamorig:RightUpLeg",+ "mixamorig:LeftLeg",+ "mixamorig:RightLeg",+ "mixamorig:LeftFoot",+ "mixamorig:RightFoot",+ "mixamorig:LeftToeBase",+ "mixamorig:RightToeBase",+)++# Mixamo bone → (human parent joint, human child joint) used as the aim vector.+# Spine2 uses a chest frame (not a single swing) so clavicle attachments keep the right twist.+#+# Neck/Head must not aim along SMPL neck→head. That vector is a short (~9 cm)+# skull-center offset, often 50–70° from vertical, and flops Mixamo's longer+# neck/head bones. spine3→head is the stable head-up (skull above the chest)+# while still using the Human head joint. Joint mapping remains+# neck→mixamorig:Neck and head→mixamorig:Head.+#+# Mixamo Neck→Head and Head→HeadTop rest translations are not along local +Y+# (they include a forward +Z offset). Aiming that offset at head-up tilts the+# neck/head bones. Neck/Head therefore swing local +Y.+NECK_HEAD = ("mixamorig:Neck", "mixamorig:Head")+NECK_HEAD_AIM_LOCAL = np.array([0.0, 1.0, 0.0])++SWING_TARGETS = {+ "mixamorig:Spine": ("spine1", "spine2"),+ "mixamorig:Spine1": ("spine2", "spine3"),+ "mixamorig:Neck": ("spine3", "head"),+ "mixamorig:Head": ("spine3", "head"),+ "mixamorig:LeftShoulder": ("left_collar", "left_shoulder"),+ "mixamorig:LeftArm": ("left_shoulder", "left_elbow"),+ "mixamorig:LeftForeArm": ("left_elbow", "left_wrist"),+ "mixamorig:RightShoulder": ("right_collar", "right_shoulder"),+ "mixamorig:RightArm": ("right_shoulder", "right_elbow"),+ "mixamorig:RightForeArm": ("right_elbow", "right_wrist"),+ "mixamorig:LeftUpLeg": ("left_hip", "left_knee"),+ "mixamorig:LeftLeg": ("left_knee", "left_ankle"),+ "mixamorig:RightUpLeg": ("right_hip", "right_knee"),+ "mixamorig:RightLeg": ("right_knee", "right_ankle"),+}++SWING_TARGETS_NO_CLAVICLE = {+ "mixamorig:Spine": ("spine1", "spine2"),+ "mixamorig:Spine1": ("spine2", "spine3"),+ "mixamorig:Spine2": ("spine3", "neck"),+ "mixamorig:LeftArm": ("left_shoulder", "left_elbow"),+ "mixamorig:LeftForeArm": ("left_elbow", "left_wrist"),+ "mixamorig:RightArm": ("right_shoulder", "right_elbow"),+ "mixamorig:RightForeArm": ("right_elbow", "right_wrist"),+}++# Child that defines this bone's rest axis (local +Y after PreRotation).+AIM_CHILD = {+ "mixamorig:Spine": "mixamorig:Spine1",+ "mixamorig:Spine1": "mixamorig:Spine2",+ "mixamorig:Spine2": "mixamorig:Neck",+ "mixamorig:Neck": "mixamorig:Head",+ "mixamorig:Head": "mixamorig:HeadTop_End",+ "mixamorig:LeftShoulder": "mixamorig:LeftArm",+ "mixamorig:LeftArm": "mixamorig:LeftForeArm",+ "mixamorig:LeftForeArm": "mixamorig:LeftHand",+ "mixamorig:RightShoulder": "mixamorig:RightArm",+ "mixamorig:RightArm": "mixamorig:RightForeArm",+ "mixamorig:RightForeArm": "mixamorig:RightHand",+ "mixamorig:LeftUpLeg": "mixamorig:LeftLeg",+ "mixamorig:LeftLeg": "mixamorig:LeftFoot",+ "mixamorig:LeftFoot": "mixamorig:LeftToeBase",+ "mixamorig:LeftToeBase": "mixamorig:LeftToe_End",+ "mixamorig:RightUpLeg": "mixamorig:RightLeg",+ "mixamorig:RightLeg": "mixamorig:RightFoot",+ "mixamorig:RightFoot": "mixamorig:RightToeBase",+ "mixamorig:RightToeBase": "mixamorig:RightToe_End",+}+++def swing_aim_local(rig: MixamoRig, bone_name: str) -> np.ndarray:+ """Rest axis that swing_anim_to_direction aligns with the Human target."""+ if bone_name in NECK_HEAD:+ return NECK_HEAD_AIM_LOCAL.copy()+ return rig.child_offset_cm(bone_name, AIM_CHILD[bone_name])+++MIXAMO_TO_HUMAN = {+ "mixamorig:Hips": "pelvis",+ "mixamorig:Spine": "spine1",+ "mixamorig:Spine1": "spine2",+ "mixamorig:Spine2": "spine3",+ "mixamorig:Neck": "neck",+ "mixamorig:Head": "head",+ "mixamorig:LeftShoulder": "left_collar",+ "mixamorig:LeftArm": "left_shoulder",+ "mixamorig:LeftForeArm": "left_elbow",+ "mixamorig:LeftHand": "left_wrist",+ "mixamorig:RightShoulder": "right_collar",+ "mixamorig:RightArm": "right_shoulder",+ "mixamorig:RightForeArm": "right_elbow",+ "mixamorig:RightHand": "right_wrist",+ "mixamorig:LeftUpLeg": "left_hip",+ "mixamorig:LeftLeg": "left_knee",+ "mixamorig:LeftFoot": "left_ankle",+ "mixamorig:LeftToeBase": "left_foot",+ "mixamorig:RightUpLeg": "right_hip",+ "mixamorig:RightLeg": "right_knee",+ "mixamorig:RightFoot": "right_ankle",+ "mixamorig:RightToeBase": "right_foot",+}++ARM_POSITION_MARKERS = {+ "left_elbow": "mixamorig:LeftForeArm",+ "left_wrist": "mixamorig:LeftHand",+ "right_elbow": "mixamorig:RightForeArm",+ "right_wrist": "mixamorig:RightHand",+}++LEG_POSITION_MARKERS = {+ "left_hip": "mixamorig:LeftUpLeg",+ "left_knee": "mixamorig:LeftLeg",+ "left_ankle": "mixamorig:LeftFoot",+ "right_hip": "mixamorig:RightUpLeg",+ "right_knee": "mixamorig:RightLeg",+ "right_ankle": "mixamorig:RightFoot",+}++LEG_BONES = (+ "mixamorig:LeftUpLeg",+ "mixamorig:LeftLeg",+ "mixamorig:LeftFoot",+ "mixamorig:LeftToeBase",+ "mixamorig:LeftToe_End",+ "mixamorig:RightUpLeg",+ "mixamorig:RightLeg",+ "mixamorig:RightFoot",+ "mixamorig:RightToeBase",+ "mixamorig:RightToe_End",+)++WORLD_UP = np.array([0.0, 1.0, 0.0])+HAND_LOCAL_AXIS = np.array([0.0, 1.0, 0.0])++MIXAMO_EDGES = [+ ("mixamorig:Hips", "mixamorig:Spine"),+ ("mixamorig:Spine", "mixamorig:Spine1"),+ ("mixamorig:Spine1", "mixamorig:Spine2"),+ ("mixamorig:Spine2", "mixamorig:Neck"),+ ("mixamorig:Neck", "mixamorig:Head"),+ ("mixamorig:Head", "mixamorig:HeadTop_End"),+ ("mixamorig:Spine2", "mixamorig:LeftShoulder"),+ ("mixamorig:LeftShoulder", "mixamorig:LeftArm"),+ ("mixamorig:LeftArm", "mixamorig:LeftForeArm"),+ ("mixamorig:LeftForeArm", "mixamorig:LeftHand"),+ ("mixamorig:Spine2", "mixamorig:RightShoulder"),+ ("mixamorig:RightShoulder", "mixamorig:RightArm"),+ ("mixamorig:RightArm", "mixamorig:RightForeArm"),+ ("mixamorig:RightForeArm", "mixamorig:RightHand"),+ ("mixamorig:Hips", "mixamorig:LeftUpLeg"),+ ("mixamorig:LeftUpLeg", "mixamorig:LeftLeg"),+ ("mixamorig:LeftLeg", "mixamorig:LeftFoot"),+ ("mixamorig:LeftFoot", "mixamorig:LeftToeBase"),+ ("mixamorig:LeftToeBase", "mixamorig:LeftToe_End"),+ ("mixamorig:Hips", "mixamorig:RightUpLeg"),+ ("mixamorig:RightUpLeg", "mixamorig:RightLeg"),+ ("mixamorig:RightLeg", "mixamorig:RightFoot"),+ ("mixamorig:RightFoot", "mixamorig:RightToeBase"),+ ("mixamorig:RightToeBase", "mixamorig:RightToe_End"),+]++HUMAN_EDGES = [+ ("pelvis", "spine1"),+ ("spine1", "spine2"),+ ("spine2", "spine3"),+ ("spine3", "neck"),+ ("neck", "head"),+ ("spine3", "left_collar"),+ ("left_collar", "left_shoulder"),+ ("left_shoulder", "left_elbow"),+ ("left_elbow", "left_wrist"),+ ("left_wrist", "left_hand"),+ ("spine3", "right_collar"),+ ("right_collar", "right_shoulder"),+ ("right_shoulder", "right_elbow"),+ ("right_elbow", "right_wrist"),+ ("right_wrist", "right_hand"),+ ("pelvis", "left_hip"),+ ("left_hip", "left_knee"),+ ("left_knee", "left_ankle"),+ ("left_ankle", "left_foot"),+ ("pelvis", "right_hip"),+ ("right_hip", "right_knee"),+ ("right_knee", "right_ankle"),+ ("right_ankle", "right_foot"),+]++LENGTH_EDGES = [+ ("mixamorig:Spine2", "mixamorig:LeftShoulder"),+ ("mixamorig:LeftShoulder", "mixamorig:LeftArm"),+ ("mixamorig:LeftArm", "mixamorig:LeftForeArm"),+ ("mixamorig:LeftForeArm", "mixamorig:LeftHand"),+ ("mixamorig:Spine2", "mixamorig:RightShoulder"),+ ("mixamorig:RightShoulder", "mixamorig:RightArm"),+ ("mixamorig:RightArm", "mixamorig:RightForeArm"),+ ("mixamorig:RightForeArm", "mixamorig:RightHand"),+ ("mixamorig:Hips", "mixamorig:Spine"),+ ("mixamorig:Spine", "mixamorig:Spine1"),+ ("mixamorig:Spine1", "mixamorig:Spine2"),+ ("mixamorig:Spine2", "mixamorig:Neck"),+ ("mixamorig:Neck", "mixamorig:Head"),+ ("mixamorig:Head", "mixamorig:HeadTop_End"),+ ("mixamorig:Hips", "mixamorig:LeftUpLeg"),+ ("mixamorig:LeftUpLeg", "mixamorig:LeftLeg"),+ ("mixamorig:LeftLeg", "mixamorig:LeftFoot"),+ ("mixamorig:LeftFoot", "mixamorig:LeftToeBase"),+ ("mixamorig:Hips", "mixamorig:RightUpLeg"),+ ("mixamorig:RightUpLeg", "mixamorig:RightLeg"),+ ("mixamorig:RightLeg", "mixamorig:RightFoot"),+ ("mixamorig:RightFoot", "mixamorig:RightToeBase"),+]+++@dataclass+class FrameRetarget:+ r_anim: dict[str, np.ndarray]+ q_anim: dict[str, np.ndarray]+ r_anim_raw: dict[str, np.ndarray] = field(default_factory=dict)+ twist_flips: list[str] = field(default_factory=list)+ sign_flips: list[str] = field(default_factory=list)+++def joint_mixamo(motion, frame: int, name: str) -> np.ndarray:+ return human_to_mixamo(motion.joint(frame, name))+++def human_dir(motion, frame: int, start: str, end: str) -> np.ndarray:+ return joint_mixamo(motion, frame, end) - joint_mixamo(motion, frame, start)+++def verify_rest_fk(rig: MixamoRig, atol: float = 1e-4) -> float:+ """Identity R_anim must reproduce the FBX rest pose (PreRotation included)."""+ world = forward_kinematics(rig, {})+ err = 0.0+ for name, bone in rig.bones.items():+ err = max(err, float(np.linalg.norm(world[name] - bone.rest_world)))+ if err > atol:+ raise RuntimeError(f"rest FK mismatch {err:.4g} > {atol} — PreRotation composition is wrong")+ return err+++def mixamo_proportioned_from(+ rig: MixamoRig,+ motion,+ frame: int,+ rest_pos: dict[str, np.ndarray],+ root_name: str,+ root_pos_m: np.ndarray,+) -> dict[str, np.ndarray]:+ """Rebuild descendants of root_name with Human directions and Mixamo rest lengths."""+ target: dict[str, np.ndarray] = {root_name: np.asarray(root_pos_m, dtype=np.float64).copy()}+ for name in rig.order:+ if name == root_name:+ continue+ parent = rig.bones[name].parent+ if parent is None or parent not in target:+ continue+ length = float(np.linalg.norm(rest_pos[name] - rest_pos[parent]))+ if name in MIXAMO_TO_HUMAN and parent in MIXAMO_TO_HUMAN:+ direction = human_dir(motion, frame, MIXAMO_TO_HUMAN[parent], MIXAMO_TO_HUMAN[name])+ else:+ direction = rest_pos[name] - rest_pos[parent]+ target[name] = target[parent] + normalize(direction) * length+ return target+++def position_errors(+ posed: dict[str, np.ndarray],+ target: dict[str, np.ndarray],+ markers: dict[str, str],+) -> dict[str, dict[str, float]]:+ out: dict[str, dict[str, float]] = {}+ for label, bone in markers.items():+ delta = posed[bone] - target[bone]+ out[label] = {+ "error_m": float(np.linalg.norm(delta)),+ "dx": float(delta[0]),+ "dy": float(delta[1]),+ "dz": float(delta[2]),+ "mixamo_bone": bone,+ }+ return out+++def _hips_rotation(rig: MixamoRig, motion, frame: int) -> np.ndarray:+ rest_world = forward_kinematics(rig, {})+ rest_pos = world_positions_m(rig, rest_world)+ r_rest_frame = orthonormal_frame(+ rest_pos["mixamorig:LeftUpLeg"] - rest_pos["mixamorig:RightUpLeg"],+ rest_pos["mixamorig:Neck"] - rest_pos["mixamorig:Hips"],+ )+ r_tgt_frame = orthonormal_frame(+ joint_mixamo(motion, frame, "left_hip") - joint_mixamo(motion, frame, "right_hip"),+ joint_mixamo(motion, frame, "neck") - joint_mixamo(motion, frame, "pelvis"),+ )+ r_hips_world = (r_tgt_frame @ r_rest_frame.T) @ rest_world["mixamorig:Hips"][:3, :3]+ return anim_from_world_rotation(rig.bones["mixamorig:Hips"], np.eye(3), r_hips_world)+++def _spine2_chest_rotation(rig: MixamoRig, world: dict[str, np.ndarray], motion, frame: int) -> np.ndarray:+ bone = rig.bones["mixamorig:Spine2"]+ parent_r = world[bone.parent][:3, :3] if bone.parent else np.eye(3)+ rest_left = rig.child_offset_cm("mixamorig:Spine2", "mixamorig:LeftShoulder") - rig.child_offset_cm(+ "mixamorig:Spine2", "mixamorig:RightShoulder"+ )+ rest_up = rig.child_offset_cm("mixamorig:Spine2", "mixamorig:Neck")+ return frame_anim_from_axes(+ bone,+ parent_r,+ rest_left,+ rest_up,+ joint_mixamo(motion, frame, "left_collar") - joint_mixamo(motion, frame, "right_collar"),+ joint_mixamo(motion, frame, "neck") - joint_mixamo(motion, frame, "spine3"),+ )+++def _foot_rotation(rig: MixamoRig, world: dict[str, np.ndarray], motion, frame: int, side: str) -> np.ndarray:+ foot = f"mixamorig:{side}Foot"+ toe = f"mixamorig:{side}ToeBase"+ bone = rig.bones[foot]+ parent_r = world[bone.parent][:3, :3] if bone.parent else np.eye(3)+ along_local = rig.child_offset_cm(foot, toe)+ r_rest = parent_r @ bone.pre_rotation+ up_local = r_rest.T @ WORLD_UP+ ankle = "left_ankle" if side == "Left" else "right_ankle"+ foot_j = "left_foot" if side == "Left" else "right_foot"+ knee = "left_knee" if side == "Left" else "right_knee"+ tgt_along = human_dir(motion, frame, ankle, foot_j)+ tgt_up = WORLD_UP.copy()+ if abs(float(normalize(tgt_along) @ normalize(tgt_up))) > 0.95:+ tgt_up = human_dir(motion, frame, knee, ankle)+ if bone.parent:+ up_local = r_rest.T @ (parent_r @ rig.child_offset_cm(bone.parent, foot))+ return frame_anim_from_two_vectors(+ bone,+ parent_r,+ along_local,+ up_local,+ tgt_along,+ tgt_up,+ )+++def _hand_rotation(rig: MixamoRig, world: dict[str, np.ndarray], motion, frame: int, side: str) -> np.ndarray:+ """Aim Mixamo Hand local +Y along Human wrist→hand. No finger bones in the core rig."""+ hand = f"mixamorig:{side}Hand"+ wrist_j = "left_wrist" if side == "Left" else "right_wrist"+ palm_j = "left_hand" if side == "Left" else "right_hand"+ tgt = human_dir(motion, frame, wrist_j, palm_j)+ if float(np.linalg.norm(tgt)) < 1e-4:+ return np.eye(3)+ return swing_anim_to_direction(rig, world, hand, None, tgt, d_local=HAND_LOCAL_AXIS)+++# Bones whose rotation is built from two independent human directions (or is+# constant). They are fully determined by the current frame's data, so there is+# no twist ambiguity for the continuity rule to resolve. Letting the rule+# prefer the 180-degree twist variant anyway lets one transient glitch in the+# extracted data latch permanently: once the previous frame is in the flipped+# state, the flipped variant of every later (correct) rotation stays closest+# to it, and the character finishes the clip facing backwards.+FULLY_DETERMINED = frozenset({+ "mixamorig:Hips",+ "mixamorig:Spine2",+ "mixamorig:LeftFoot",+ "mixamorig:RightFoot",+ "mixamorig:LeftToeBase",+ "mixamorig:RightToeBase",+})+++def retarget_one_frame(+ rig: MixamoRig,+ motion,+ frame: int,+ *,+ clavicles: bool = True,+ legs: bool = True,+ prev_r_anim: dict[str, np.ndarray] | None = None,+ prev_q_anim: dict[str, np.ndarray] | None = None,+) -> FrameRetarget:+ hips_t_cm = joint_mixamo(motion, frame, "pelvis") / rig.meters_per_unit+ r_anim = {name: np.eye(3) for name in rig.order}+ q_anim = {name: np.array([1.0, 0.0, 0.0, 0.0]) for name in rig.order}+ twist_flips: list[str] = []+ sign_flips: list[str] = []++ def fk() -> dict[str, np.ndarray]:+ return forward_kinematics(rig, r_anim, hips_translation_cm=hips_t_cm)++ def commit(name: str, rotation: np.ndarray) -> None:+ q_prev = None if prev_q_anim is None else prev_q_anim.get(name)+ if name in FULLY_DETERMINED:+ # Trust the data over history; only the aim-ambiguous bones below+ # get the anti-twist continuity.+ r_pick, did_twist = np.asarray(rotation, dtype=np.float64), False+ else:+ axis = swing_aim_local(rig, name) if name in AIM_CHILD or name in NECK_HEAD else local_aim_axis(rig, name)+ r_prev = None if prev_r_anim is None else prev_r_anim.get(name)+ r_pick, did_twist = pick_closest_aim_twist(rotation, r_prev, axis)+ q = quat_wxyz(r_pick)+ q, did_sign = continue_quaternion(q, q_prev)+ r_anim[name] = r_pick+ q_anim[name] = q+ if did_twist:+ twist_flips.append(name)+ if did_sign:+ sign_flips.append(name)++ def apply_swings(targets: dict[str, tuple[str, str]]) -> None:+ for bone_name, (h_a, h_b) in targets.items():+ rotation = swing_anim_to_direction(+ rig,+ fk(),+ bone_name,+ AIM_CHILD.get(bone_name),+ human_dir(motion, frame, h_a, h_b),+ d_local=swing_aim_local(rig, bone_name),+ )+ commit(bone_name, rotation)++ commit("mixamorig:Hips", _hips_rotation(rig, motion, frame))+ if clavicles:+ apply_swings(+ {+ "mixamorig:Spine": SWING_TARGETS["mixamorig:Spine"],+ "mixamorig:Spine1": SWING_TARGETS["mixamorig:Spine1"],+ }+ )+ commit("mixamorig:Spine2", _spine2_chest_rotation(rig, fk(), motion, frame))+ apply_swings(+ {+ "mixamorig:Neck": SWING_TARGETS["mixamorig:Neck"],+ "mixamorig:Head": SWING_TARGETS["mixamorig:Head"],+ "mixamorig:LeftShoulder": SWING_TARGETS["mixamorig:LeftShoulder"],+ "mixamorig:LeftArm": SWING_TARGETS["mixamorig:LeftArm"],+ "mixamorig:LeftForeArm": SWING_TARGETS["mixamorig:LeftForeArm"],+ "mixamorig:RightShoulder": SWING_TARGETS["mixamorig:RightShoulder"],+ "mixamorig:RightArm": SWING_TARGETS["mixamorig:RightArm"],+ "mixamorig:RightForeArm": SWING_TARGETS["mixamorig:RightForeArm"],+ }+ )+ commit("mixamorig:LeftHand", _hand_rotation(rig, fk(), motion, frame, "Left"))+ commit("mixamorig:RightHand", _hand_rotation(rig, fk(), motion, frame, "Right"))+ else:+ apply_swings(SWING_TARGETS_NO_CLAVICLE)+ apply_swings(+ {+ "mixamorig:Neck": SWING_TARGETS["mixamorig:Neck"],+ "mixamorig:Head": SWING_TARGETS["mixamorig:Head"],+ }+ )+ if legs:+ apply_swings(+ {+ "mixamorig:LeftUpLeg": SWING_TARGETS["mixamorig:LeftUpLeg"],+ "mixamorig:LeftLeg": SWING_TARGETS["mixamorig:LeftLeg"],+ "mixamorig:RightUpLeg": SWING_TARGETS["mixamorig:RightUpLeg"],+ "mixamorig:RightLeg": SWING_TARGETS["mixamorig:RightLeg"],+ }+ )+ commit("mixamorig:LeftFoot", _foot_rotation(rig, fk(), motion, frame, "Left"))+ commit("mixamorig:LeftToeBase", np.eye(3))+ commit("mixamorig:RightFoot", _foot_rotation(rig, fk(), motion, frame, "Right"))+ commit("mixamorig:RightToeBase", np.eye(3))+ r_raw = {name: np.asarray(rot, dtype=np.float64).copy() for name, rot in r_anim.items()}+ return FrameRetarget(+ r_anim=r_anim,+ q_anim=q_anim,+ r_anim_raw=r_raw,+ twist_flips=twist_flips,+ sign_flips=sign_flips,+ )diff --git a/motion2mixamorig/mixamo/skinned_mesh.py b/motion2mixamorig/mixamo/skinned_mesh.pynew file mode 100644index 0000000..6ae9f73--- /dev/null+++ b/motion2mixamorig/mixamo/skinned_mesh.py@@ -0,0 +1,543 @@+"""Load Mixamo character meshes + skin clusters from binary FBX.++Does not change retargeting. FK is the existing T * Rpre * R_anim chain;+finger bones stay at rest (identity R_anim) and follow the hands.+"""++from __future__ import annotations++from dataclasses import dataclass, field+from pathlib import Path++import numpy as np++from .fbx_skeleton import (+ _child_named,+ bone_local_matrix,+ build_hierarchy,+ collect_limb_nodes,+ compose_world,+ euler_xyz_deg_to_matrix,+ extract_connections,+ extract_global_settings,+ extract_objects,+ parse_fbx,+ unit_to_meters,+)+from .kinematics import MixamoBone, MixamoRig, forward_kinematics+++def _fbx_name(raw: str) -> str:+ return str(raw).split("\x00", 1)[0]+++def _mat4_fbx(values) -> np.ndarray:+ return np.asarray(values, dtype=np.float64).reshape(4, 4).T+++def _triangulate(indices: np.ndarray) -> tuple[np.ndarray, np.ndarray, np.ndarray]:+ """Fan-triangulate PolygonVertexIndex.++ Returns the vertex ids per triangle, the *flat* polygon-vertex slots they+ came from, and the polygon each triangle belongs to. The slots index the+ per-polygon-vertex layers (UV), and the polygon id indexes the per-polygon+ ones (material), so both can be read back after triangulation.+ """+ faces: list[tuple[int, int, int]] = []+ slots: list[tuple[int, int, int]] = []+ polys: list[int] = []+ corner: list[int] = []+ corner_slot: list[int] = []+ poly_id = 0+ for flat, raw in enumerate(indices):+ value = int(raw)+ corner_slot.append(flat)+ if value < 0:+ corner.append(~value)+ for i in range(1, len(corner) - 1):+ faces.append((corner[0], corner[i], corner[i + 1]))+ slots.append((corner_slot[0], corner_slot[i], corner_slot[i + 1]))+ polys.append(poly_id)+ corner = []+ corner_slot = []+ poly_id += 1+ else:+ corner.append(value)+ if not faces:+ return (np.zeros((0, 3), dtype=np.int32),+ np.zeros((0, 3), dtype=np.int64),+ np.zeros((0,), dtype=np.int64))+ return (np.asarray(faces, dtype=np.int32),+ np.asarray(slots, dtype=np.int64),+ np.asarray(polys, dtype=np.int64))+++# Mixamo ships diffuse alongside normal/spec/emission maps under one material,+# and only the file names tell them apart.+_DIFFUSE_HINTS = ("diffuse", "albedo", "basecolor", "base_color")+_OTHER_MAP_HINTS = ("normal", "spec", "gloss", "rough", "metal", "emission",+ "occlusion", "_ao", "bump", "displace", "opacity", "transparen")+++def _texture_rank(filename: str) -> int:+ low = filename.lower()+ if any(h in low for h in _DIFFUSE_HINTS):+ return 0+ if any(h in low for h in _OTHER_MAP_HINTS):+ return 2+ return 1+++def _decode_video(obj: dict) -> "np.ndarray | None":+ """Decoded texture, keeping any alpha channel.++ Alpha matters: the facial decals and hair/cape cards are masked art painted+ on an opaque backing, so discarding alpha stamps that backing onto the model.+ """+ import cv2++ content = _child_named(obj["raw"], "Content")+ if content is None or not content["props"]:+ return None+ blob = content["props"][0]+ if not isinstance(blob, (bytes, bytearray)):+ return None+ if len(blob) == 0:+ return None+ image = cv2.imdecode(np.frombuffer(bytes(blob), dtype=np.uint8), cv2.IMREAD_UNCHANGED)+ if image is None:+ return None+ if image.ndim == 2:+ image = cv2.cvtColor(image, cv2.COLOR_GRAY2BGR)+ return image+++def _videos_by_filename(objects: dict) -> dict[str, list[int]]:+ """Several Video nodes can share one file with only one holding the bytes."""+ out: dict[str, list[int]] = {}+ for uid, obj in objects.items():+ if obj["node_name"] != "Video":+ continue+ node = _child_named(obj["raw"], "RelativeFilename")+ if node is None or not node["props"]:+ continue+ key = _fbx_name(str(node["props"][0]))+ if key:+ out.setdefault(key, []).append(uid)+ return out+++def _material_textures(objects: dict, connections: list) -> dict[int, list[tuple[int, str]]]:+ """material uid → its textures as (video uid, filename), best diffuse first."""+ tex_uids = {u for u, o in objects.items() if o["node_name"] == "Texture"}+ video_uids = {u for u, o in objects.items() if o["node_name"] == "Video"}+ mat_uids = {u for u, o in objects.items() if o["node_name"] == "Material"}++ tex_video: dict[int, int] = {}+ for kind, src, dst in connections:+ if kind == "OO" and src in video_uids and dst in tex_uids:+ tex_video[dst] = src++ out: dict[int, list[tuple[int, str]]] = {}+ for kind, src, dst in connections:+ if src not in tex_uids or dst not in mat_uids:+ continue+ node = _child_named(objects[src]["raw"], "RelativeFilename")+ filename = _fbx_name(str(node["props"][0])) if node is not None and node["props"] else ""+ video = tex_video.get(src)+ if video is None:+ continue+ entry = (video, filename)+ bucket = out.setdefault(dst, [])+ if entry not in bucket:+ bucket.append(entry)+ for uid, bucket in out.items():+ bucket.sort(key=lambda e: _texture_rank(e[1]))+ return out+++def _geometry_materials(objects: dict, connections: list) -> dict[int, list[int]]:+ """geometry uid → material uids in the order the Model declares them.++ A `ByPolygon` material layer indexes this list, and FBX defines the order by+ the sequence of material→model connections.+ """+ geom_uids = {u for u, o in objects.items() if o["node_name"] == "Geometry"}+ mat_uids = {u for u, o in objects.items() if o["node_name"] == "Material"}+ model_uids = {u for u, o in objects.items()+ if o["node_name"] == "Model" and o.get("class") == "Mesh"}++ geom_model: dict[int, int] = {}+ model_mats: dict[int, list[int]] = {}+ for kind, src, dst in connections:+ if kind != "OO":+ continue+ if src in geom_uids and dst in model_uids:+ geom_model[src] = dst+ if src in mat_uids and dst in model_uids:+ model_mats.setdefault(dst, []).append(src)+ return {g: model_mats.get(m, []) for g, m in geom_model.items()}+++def _uv_per_slot(geom_raw: dict) -> np.ndarray | None:+ """(n_polygon_vertex, 2) UVs, resolved through UVIndex when present."""+ layer = _child_named(geom_raw, "LayerElementUV")+ if layer is None:+ return None+ uv_node = _child_named(layer, "UV")+ if uv_node is None or not uv_node["props"]:+ return None+ uvs = np.asarray(uv_node["props"][0], dtype=np.float64).reshape(-1, 2)+ idx_node = _child_named(layer, "UVIndex")+ if idx_node is not None and idx_node["props"]:+ idx = np.asarray(idx_node["props"][0], dtype=np.int64)+ idx = np.clip(idx, 0, uvs.shape[0] - 1)+ return uvs[idx]+ return uvs+++def _material_per_polygon(geom_raw: dict, n_poly: int) -> np.ndarray:+ layer = _child_named(geom_raw, "LayerElementMaterial")+ if layer is None:+ return np.zeros(n_poly, dtype=np.int64)+ arr = _child_named(layer, "Materials")+ if arr is None or not arr["props"]:+ return np.zeros(n_poly, dtype=np.int64)+ mats = np.asarray(arr["props"][0], dtype=np.int64)+ if mats.size == 1:+ return np.full(n_poly, int(mats[0]), dtype=np.int64)+ if mats.size < n_poly:+ mats = np.pad(mats, (0, n_poly - mats.size), mode="edge")+ return mats[:n_poly]+++def _sample(image: np.ndarray, uv: np.ndarray) -> np.ndarray:+ """Nearest-texel BGR at UV. FBX UV origin is bottom-left, images are top-down.++ UVs are wrapped, not clamped: Mixamo reuses one half of an atlas by offsetting+ the other half a whole tile in U, so clamping folds every one of those faces+ onto the texture's edge column.+ """+ h, w = image.shape[:2]+ u = np.mod(uv[:, 0], 1.0)+ v = np.mod(1.0 - uv[:, 1], 1.0)+ xs = np.clip((u * (w - 1)).astype(np.int64), 0, w - 1)+ ys = np.clip((v * (h - 1)).astype(np.int64), 0, h - 1)+ return image[ys, xs].astype(np.float64)+++# Barycentric weights of points strictly inside a triangle. One texel at the+# centroid lands in the black gutter between UV islands often enough to speckle+# a whole character, so each triangle is probed at several interior points and+# the median is kept, which ignores a stray sample or two.+_PROBES = np.array([+ [1 / 3, 1 / 3, 1 / 3],+ [0.6, 0.2, 0.2], [0.2, 0.6, 0.2], [0.2, 0.2, 0.6],+ [0.45, 0.45, 0.10], [0.45, 0.10, 0.45], [0.10, 0.45, 0.45],+])+++def _sample_triangles(image: np.ndarray, corners: np.ndarray) -> np.ndarray:+ """(T, 3, 2) triangle UVs -> (T, C) sample, median over the interior probes."""+ channels = image.shape[2]+ stack = np.empty((_PROBES.shape[0], corners.shape[0], channels), dtype=np.float64)+ for i, w in enumerate(_PROBES):+ uv = (corners[:, 0] * w[0] + corners[:, 1] * w[1] + corners[:, 2] * w[2])+ stack[i] = _sample(image, uv)+ return np.median(stack, axis=0)+++def face_colors_from_textures(+ objects: dict,+ connections: list,+ geom_uid: int,+ geom_raw: dict,+ slots: np.ndarray,+ poly_ids: np.ndarray,+ lift: float = 1.0,+) -> tuple[np.ndarray, np.ndarray] | None:+ """One BGR per triangle, read from the diffuse map at the UV centroid.++ UVs never move, so this is paid once per character rather than per frame,+ and the painter fill stays exactly as cheap as a flat colour.++ `lift` below 1 is a gamma on the sampled colour. Several of these characters+ wear near-black clothing that no amount of ambient light separates from a+ dark backdrop; the curve raises those without touching the bright end.+ """+ uv_slots = _uv_per_slot(geom_raw)+ if uv_slots is None or slots.shape[0] == 0:+ return None+ mats = _geometry_materials(objects, connections).get(geom_uid, [])+ if not mats:+ return None+ textures = _material_textures(objects, connections)+ shared = _videos_by_filename(objects)++ n_poly = int(poly_ids.max()) + 1 if poly_ids.size else 0+ poly_mat = _material_per_polygon(geom_raw, n_poly)+ tri_mat = poly_mat[poly_ids]++ slot_clip = np.clip(slots, 0, uv_slots.shape[0] - 1)+ corners = np.stack([uv_slots[slot_clip[:, 0]], uv_slots[slot_clip[:, 1]],+ uv_slots[slot_clip[:, 2]]], axis=1)++ out = np.full((slots.shape[0], 3), np.nan, dtype=np.float64)+ opacity = np.full(slots.shape[0], 255.0, dtype=np.float64)+ for local_idx, mat_uid in enumerate(mats):+ picked = tri_mat == local_idx+ if not np.any(picked):+ continue+ image = None+ for video_uid, filename in textures.get(mat_uid, []):+ for candidate in [video_uid, *shared.get(filename, [])]:+ image = _decode_video(objects[candidate])+ if image is not None:+ break+ if image is not None:+ break+ if image is None:+ continue+ sampled = _sample_triangles(image, corners[picked])+ out[picked] = sampled[:, :3]+ if sampled.shape[1] == 4:+ opacity[picked] = sampled[:, 3]+ del image+ if np.all(np.isnan(out)):+ return None+ fallback = np.nanmean(out.reshape(-1, 3), axis=0)+ bad = np.isnan(out[:, 0])+ out[bad] = fallback+ if abs(lift - 1.0) > 1e-9:+ out = 255.0 * np.power(np.clip(out / 255.0, 0.0, 1.0), lift)+ return out, opacity >= 128.0+++@dataclass+class SkinnedMesh:+ name: str+ vertices: np.ndarray # (V, 3) bind pose, FBX cm+ faces: np.ndarray # (F, 3)+ weights: np.ndarray # (V, B)+ ibm: np.ndarray # (B, 4, 4) inverse bind = Cluster.Transform+ color: np.ndarray+ face_colors: np.ndarray | None = None # (F, 3) BGR from the diffuse map+++@dataclass+class YBotAsset:+ rig: MixamoRig+ meshes: list[SkinnedMesh]+ bone_index: dict[str, int] = field(default_factory=dict)++ @property+ def n_bones(self) -> int:+ return len(self.rig.order)+++def load_full_rig(fbx_path: Path) -> MixamoRig:+ """Every Mixamo limb, same PreRotation local-matrix convention as the core rig."""+ fbx_path = Path(fbx_path)+ version, nodes = parse_fbx(fbx_path)+ objects = extract_objects(nodes)+ connections = extract_connections(nodes)+ settings = extract_global_settings(nodes)+ limbs = collect_limb_nodes(objects)+ parent_of = build_hierarchy(limbs, connections)+ local_of: dict[int, np.ndarray] = {}+ parts_of: dict[int, dict[str, np.ndarray]] = {}+ for uid, obj in limbs.items():+ local, parts = bone_local_matrix(obj["props"])+ local_of[uid] = local+ parts_of[uid] = parts+ world_of = compose_world(parent_of, local_of)++ children_of: dict[int, list[int]] = {uid: [] for uid in limbs}+ for uid, parent in parent_of.items():+ if parent in children_of:+ children_of[parent].append(uid)+ roots = [uid for uid, parent in parent_of.items() if parent is None]+ ordered: list[int] = []++ def walk(uid: int) -> None:+ ordered.append(uid)+ for child in children_of[uid]:+ walk(child)++ for root in roots:+ walk(root)++ bones: dict[str, MixamoBone] = {}+ order: list[str] = []+ for uid in ordered:+ obj = limbs[uid]+ name = _fbx_name(obj["name"])+ parts = parts_of[uid]+ parent_uid = parent_of[uid]+ parent_name = _fbx_name(limbs[parent_uid]["name"]) if parent_uid in limbs else None+ child_names = [_fbx_name(limbs[c]["name"]) for c in children_of[uid]]+ bones[name] = MixamoBone(+ name=name,+ parent=parent_name,+ children=child_names,+ pre_rotation=euler_xyz_deg_to_matrix(parts["pre_rotation_euler_deg"]),+ lcl_translation_cm=parts["lcl_translation"].copy(),+ rest_world=world_of[uid],+ rest_local=local_of[uid],+ pre_rotation_euler_deg=parts["pre_rotation_euler_deg"].copy(),+ )+ order.append(name)+ return MixamoRig(+ bones=bones,+ order=order,+ meters_per_unit=unit_to_meters(settings),+ source=fbx_path,+ coord_system={"fbx_unit": "centimeter", "meters_per_fbx_unit": unit_to_meters(settings)},+ )+++def load_character_asset(+ fbx_path: Path, *, textures: bool = False, lift: float = 1.0+) -> YBotAsset:+ """Rig + skinned meshes (optionally with diffuse-sampled face colors)."""+ fbx_path = Path(fbx_path)+ rig = load_full_rig(fbx_path)+ _, nodes = parse_fbx(fbx_path)+ objects = extract_objects(nodes)+ connections = extract_connections(nodes)+ bone_index = {name: i for i, name in enumerate(rig.order)}+ n_bones = len(rig.order)++ uid_name = {uid: _fbx_name(obj["name"]) for uid, obj in objects.items()}++ limb_uids = {uid for uid, obj in objects.items() if obj["node_name"] == "Model" and obj.get("class") == "LimbNode"}+ skin_uids = {uid for uid, obj in objects.items() if obj["node_name"] == "Deformer" and obj.get("class") == "Skin"}+ geom_uids = {uid for uid, obj in objects.items() if obj["node_name"] == "Geometry"}+ cluster_uids = {uid for uid, obj in objects.items() if obj["node_name"] == "Deformer" and obj.get("class") == "Cluster"}++ skin_to_geom: dict[int, int] = {}+ cluster_to_skin: dict[int, int] = {}+ cluster_to_limb: dict[int, int] = {}+ for kind, src, dst in connections:+ if kind != "OO":+ continue+ if src in cluster_uids and dst in skin_uids:+ cluster_to_skin[src] = dst+ if src in limb_uids and dst in cluster_uids:+ cluster_to_limb[dst] = src+ if src in skin_uids and dst in geom_uids:+ skin_to_geom[src] = dst+ if src in geom_uids and dst in skin_uids:+ skin_to_geom[dst] = src++ colors = {+ "Alpha_Surface": np.array([0.72, 0.74, 0.78]),+ "Alpha_Joints": np.array([0.38, 0.40, 0.44]),+ }+ meshes: list[SkinnedMesh] = []+ for geom_uid in geom_uids:+ geom = objects[geom_uid]+ verts_node = _child_named(geom["raw"], "Vertices")+ idx_node = _child_named(geom["raw"], "PolygonVertexIndex")+ if verts_node is None or idx_node is None:+ continue+ vertices = np.asarray(verts_node["props"][0], dtype=np.float64).reshape(-1, 3)+ faces, slots, poly_ids = _triangulate(np.asarray(idx_node["props"][0], dtype=np.int64))+ tinted = None+ if textures:+ sampled = face_colors_from_textures(+ objects, connections, geom_uid, geom["raw"], slots, poly_ids, lift+ )+ if sampled is not None:+ tinted, visible = sampled+ # Masked-out faces are dropped outright rather than blended, so+ # the painter fill stays a single opaque pass.+ if not np.all(visible):+ faces = faces[visible]+ tinted = tinted[visible]+ n_verts = vertices.shape[0]+ weights = np.zeros((n_verts, n_bones), dtype=np.float64)+ ibm = np.repeat(np.eye(4)[None, ...], n_bones, axis=0)+ for cluster_uid, skin_uid in cluster_to_skin.items():+ if skin_to_geom.get(skin_uid) != geom_uid:+ continue+ limb_uid = cluster_to_limb.get(cluster_uid)+ if limb_uid is None:+ continue+ bone_name = uid_name[limb_uid]+ if bone_name not in bone_index:+ continue+ cluster = objects[cluster_uid]+ idx_elem = _child_named(cluster["raw"], "Indexes")+ w_elem = _child_named(cluster["raw"], "Weights")+ t_elem = _child_named(cluster["raw"], "Transform")+ b = bone_index[bone_name]+ if t_elem is not None and t_elem["props"]:+ ibm[b] = _mat4_fbx(t_elem["props"][0])+ if idx_elem is None or w_elem is None:+ continue+ indexes = np.asarray(idx_elem["props"][0], dtype=np.int64)+ wts = np.asarray(w_elem["props"][0], dtype=np.float64)+ weights[indexes, b] = wts+ row = weights.sum(axis=1, keepdims=True)+ row = np.where(row < 1e-8, 1.0, row)+ weights = weights / row+ meshes.append(+ SkinnedMesh(+ name=_fbx_name(geom["name"]),+ vertices=vertices,+ faces=faces,+ weights=weights,+ ibm=ibm,+ color=colors.get(_fbx_name(geom["name"]), np.array([0.6, 0.6, 0.6])),+ face_colors=tinted,+ )+ )+ return YBotAsset(rig=rig, meshes=meshes, bone_index=bone_index)+++def bone_palette(world: dict[str, np.ndarray], mesh: SkinnedMesh, order: list[str]) -> np.ndarray:+ mats = np.stack([world[name] for name in order], axis=0)+ return mats @ mesh.ibm+++def skin_mesh(mesh: SkinnedMesh, palette: np.ndarray) -> np.ndarray:+ """Linear blend skinning. palette is (B, 4, 4) = M_world @ IBM."""+ ones = np.ones((mesh.vertices.shape[0], 1), dtype=np.float64)+ vh = np.concatenate([mesh.vertices, ones], axis=1)+ posed = np.zeros((mesh.vertices.shape[0], 4), dtype=np.float64)+ for b in range(palette.shape[0]):+ w = mesh.weights[:, b]+ if float(w.max()) < 1e-12:+ continue+ posed += w[:, None] * (vh @ palette[b].T)+ return posed[:, :3]+++def posed_vertices(+ asset: YBotAsset,+ r_anim: dict[str, np.ndarray],+ hips_translation_cm: np.ndarray,+ lcl_translations_cm: dict[str, np.ndarray] | None = None,+) -> list[np.ndarray]:+ world = forward_kinematics(+ asset.rig,+ r_anim,+ hips_translation_cm=hips_translation_cm,+ lcl_translations_cm=lcl_translations_cm,+ )+ out: list[np.ndarray] = []+ for mesh in asset.meshes:+ palette = bone_palette(world, mesh, asset.rig.order)+ out.append(skin_mesh(mesh, palette))+ return out+++def rest_skin_error(asset: YBotAsset) -> float:+ """Identity R_anim must reproduce bind vertices (PreRotation FK + IBM)."""+ world = forward_kinematics(asset.rig, {})+ err = 0.0+ for mesh in asset.meshes:+ posed = skin_mesh(mesh, bone_palette(world, mesh, asset.rig.order))+ err = max(err, float(np.linalg.norm(posed - mesh.vertices, axis=1).max()))+ return errdiff --git a/motion2mixamorig/mixamo/tpose_calibration.py b/motion2mixamorig/mixamo/tpose_calibration.pynew file mode 100644index 0000000..44ea203--- /dev/null+++ b/motion2mixamorig/mixamo/tpose_calibration.py@@ -0,0 +1,382 @@+"""The single source of truth for the Mixamo Skeleton ↔ character Rig T-pose relationship.++ Mixamo Skeleton T-pose+ ↕ one global calibration+ character Rig rest T-pose++Both the T-pose inspection programs and the animation retargeting import this+module, so there is exactly one Front / Right / Up convention in the project.++Nothing here is a manual correction: the anatomical axes are measured from bone+positions, and the alignment is `B_rig @ B_skel.T` with no polar/SVD det fix and+no per-bone 90/180 patches.+"""++from __future__ import annotations++from dataclasses import dataclass+from pathlib import Path++import numpy as np++from .fbx_skeleton import extract_skeleton+from .kinematics import (+ MixamoRig,+ forward_kinematics,+ normalize,+ rig_from_skeleton_data,+ world_positions_m,+)++# Mixamo skeleton world (validated Human → Mixamo space):+# +X = anatomical LEFT, +Y = UP, +Z = chest-forward (toes / FBX front).+LEFT_DIR = np.array([1.0, 0.0, 0.0])+RIGHT_DIR = np.array([-1.0, 0.0, 0.0])+UP_DIR = np.array([0.0, 1.0, 0.0])+DOWN_DIR = np.array([0.0, -1.0, 0.0])+FWD_DIR = np.array([0.0, 0.0, 1.0])++REQUIRED_BONES = (+ "mixamorig:Hips",+ "mixamorig:Spine",+ "mixamorig:Spine1",+ "mixamorig:Spine2",+ "mixamorig:Neck",+ "mixamorig:Head",+ "mixamorig:LeftShoulder",+ "mixamorig:LeftArm",+ "mixamorig:LeftForeArm",+ "mixamorig:LeftHand",+ "mixamorig:RightShoulder",+ "mixamorig:RightArm",+ "mixamorig:RightForeArm",+ "mixamorig:RightHand",+ "mixamorig:LeftUpLeg",+ "mixamorig:LeftLeg",+ "mixamorig:LeftFoot",+ "mixamorig:RightUpLeg",+ "mixamorig:RightLeg",+ "mixamorig:RightFoot",+)++SPINE_CHAIN = (+ "mixamorig:Spine",+ "mixamorig:Spine1",+ "mixamorig:Spine2",+ "mixamorig:Neck",+ "mixamorig:Head",+ "mixamorig:HeadTop_End",+)+LEFT_ARM_CHAIN = (+ "mixamorig:LeftShoulder",+ "mixamorig:LeftArm",+ "mixamorig:LeftForeArm",+ "mixamorig:LeftHand",+)+RIGHT_ARM_CHAIN = (+ "mixamorig:RightShoulder",+ "mixamorig:RightArm",+ "mixamorig:RightForeArm",+ "mixamorig:RightHand",+)+LEFT_LEG_CHAIN = ("mixamorig:LeftUpLeg", "mixamorig:LeftLeg", "mixamorig:LeftFoot")+RIGHT_LEG_CHAIN = ("mixamorig:RightUpLeg", "mixamorig:RightLeg", "mixamorig:RightFoot")+LEFT_TOE_CHAIN = ("mixamorig:LeftToeBase", "mixamorig:LeftToe_End")+RIGHT_TOE_CHAIN = ("mixamorig:RightToeBase", "mixamorig:RightToe_End")++# Anatomical bone correspondence. Left stays Left, Right stays Right.+# Human joint → Mixamo/Y Bot bone; the two rigs share these bone names.+ANATOMICAL_BONE_MAP = {+ "pelvis": "mixamorig:Hips",+ "spine1": "mixamorig:Spine",+ "spine2": "mixamorig:Spine1",+ "spine3": "mixamorig:Spine2",+ "neck": "mixamorig:Neck",+ "head": "mixamorig:Head",+ "left_collar": "mixamorig:LeftShoulder",+ "left_shoulder": "mixamorig:LeftArm",+ "left_elbow": "mixamorig:LeftForeArm",+ "left_wrist": "mixamorig:LeftHand",+ "right_collar": "mixamorig:RightShoulder",+ "right_shoulder": "mixamorig:RightArm",+ "right_elbow": "mixamorig:RightForeArm",+ "right_wrist": "mixamorig:RightHand",+ "left_hip": "mixamorig:LeftUpLeg",+ "left_knee": "mixamorig:LeftLeg",+ "left_ankle": "mixamorig:LeftFoot",+ "left_foot": "mixamorig:LeftToeBase",+ "right_hip": "mixamorig:RightUpLeg",+ "right_knee": "mixamorig:RightLeg",+ "right_ankle": "mixamorig:RightFoot",+ "right_foot": "mixamorig:RightToeBase",+}+++def rest_length(rest: dict[str, np.ndarray], parent: str, child: str) -> float:+ return float(np.linalg.norm(rest[child] - rest[parent]))+++def canonical_mixamo_tpose(rig: MixamoRig) -> dict[str, np.ndarray]:+ """Explicit Mixamo-skeleton T-pose. Not an animation frame.++ Uses Mixamo rest bone *lengths* and the validated Mixamo world axes:+ arms along ±X (left = +X), spine +Y, legs −Y, feet +Z. Hip / clavicle+ widths keep the rest lateral offsets so proportions stay Mixamo's.+ """+ rest = world_positions_m(rig, forward_kinematics(rig, {}))+ pos: dict[str, np.ndarray] = {"mixamorig:Hips": np.zeros(3)}++ def place(child: str, parent: str, direction: np.ndarray) -> None:+ length = rest_length(rest, parent, child)+ pos[child] = pos[parent] + normalize(direction) * length++ for child in SPINE_CHAIN:+ parent = rig.bones[child].parent+ place(child, parent, UP_DIR)++ def lateral_then_chain(first: str, chain: tuple[str, ...], side_sign: float, along: np.ndarray) -> None:+ parent = rig.bones[first].parent+ offset = rest[first] - rest[parent]+ lateral = np.array([side_sign * abs(float(offset[0])), float(offset[1]), 0.0])+ if float(np.linalg.norm(lateral)) < 1e-8:+ lateral = np.array([side_sign, 0.0, 0.0])+ place(first, parent, lateral)+ for child in chain[1:]:+ p = rig.bones[child].parent+ place(child, p, along)++ lateral_then_chain("mixamorig:LeftShoulder", LEFT_ARM_CHAIN, +1.0, LEFT_DIR)+ for child in LEFT_ARM_CHAIN[1:]:+ place(child, rig.bones[child].parent, LEFT_DIR)+ lateral_then_chain("mixamorig:RightShoulder", RIGHT_ARM_CHAIN, -1.0, RIGHT_DIR)+ for child in RIGHT_ARM_CHAIN[1:]:+ place(child, rig.bones[child].parent, RIGHT_DIR)++ lateral_then_chain("mixamorig:LeftUpLeg", LEFT_LEG_CHAIN, +1.0, DOWN_DIR)+ for child in LEFT_LEG_CHAIN[1:]:+ place(child, rig.bones[child].parent, DOWN_DIR)+ lateral_then_chain("mixamorig:RightUpLeg", RIGHT_LEG_CHAIN, -1.0, DOWN_DIR)+ for child in RIGHT_LEG_CHAIN[1:]:+ place(child, rig.bones[child].parent, DOWN_DIR)++ for child in LEFT_TOE_CHAIN:+ place(child, rig.bones[child].parent, FWD_DIR)+ for child in RIGHT_TOE_CHAIN:+ place(child, rig.bones[child].parent, FWD_DIR)++ return pos+++def rig_rest_from_skeleton_data(data: dict, fbx_path: Path) -> tuple[dict[str, np.ndarray], dict]:+ """Core Mixamo bones from the character FBX rest pose.++ PreRotation is already inside the local matrix. Assimp+ `$AssimpFbx$_PreRotation` helpers are not anatomical bones; extract_skeleton+ applies Maya/FBX `T * Rpre * R * ...` so those extra nodes are not listed.+ """+ pos: dict[str, np.ndarray] = {}+ for bone in data["bones"]:+ pos[str(bone["name"])] = np.asarray(bone["rest_world"]["translation_m"], dtype=np.float64)+ missing = [name for name in REQUIRED_BONES if name not in pos]+ if missing:+ raise RuntimeError(f"{fbx_path.name} is missing required Mixamo bones: {missing}")+ meta = {+ "source_fbx": data.get("source_fbx"),+ "fbx_version": data.get("fbx_version"),+ "coord_system": data.get("coord_system"),+ "transform_convention": data.get("transform_convention"),+ "bone_count": data.get("bone_count"),+ "character": data.get("character"),+ }+ return pos, meta+++def hips_origin(pos: dict[str, np.ndarray]) -> dict[str, np.ndarray]:+ origin = np.asarray(pos["mixamorig:Hips"], dtype=np.float64)+ return {name: np.asarray(p, dtype=np.float64) - origin for name, p in pos.items()}+++def body_height(pos: dict[str, np.ndarray]) -> float:+ return float(np.linalg.norm(pos["mixamorig:Head"] - pos["mixamorig:Hips"]))+++def scale_to_height(pos: dict[str, np.ndarray], height: float) -> dict[str, np.ndarray]:+ h = body_height(pos)+ if h < 1e-8:+ raise RuntimeError("degenerate pelvis→head height")+ s = float(height / h)+ return {name: p * s for name, p in pos.items()}+++def anatomical_axes(pos: dict[str, np.ndarray], label: str, *, verbose: bool = True) -> dict:+ """Right = right_hip − left_hip, Up = neck − pelvis. Front from the two crosses."""+ left_hip = pos["mixamorig:LeftUpLeg"]+ right_hip = pos["mixamorig:RightUpLeg"]+ pelvis = pos["mixamorig:Hips"]+ neck = pos["mixamorig:Neck"]+ mid_sh = 0.5 * (pos["mixamorig:LeftArm"] + pos["mixamorig:RightArm"])+ toe = pos["mixamorig:LeftToeBase"] - pos["mixamorig:LeftFoot"]+ chest_hint = mid_sh - pelvis+ if float(np.linalg.norm(chest_hint)) < 1e-6:+ chest_hint = toe++ right_raw = right_hip - left_hip+ up_raw = neck - pelvis+ up = normalize(up_raw, UP_DIR)+ right = right_raw - up * float(right_raw @ up)+ right = normalize(right, RIGHT_DIR)++ cross_right_up = np.cross(right, up)+ cross_up_right = np.cross(up, right)+ n_ru = float(np.linalg.norm(cross_right_up))+ n_ur = float(np.linalg.norm(cross_up_right))+ fwd_ru = cross_right_up / n_ru if n_ru > 1e-12 else np.zeros(3)+ fwd_ur = cross_up_right / n_ur if n_ur > 1e-12 else np.zeros(3)++ hint = normalize(chest_hint, FWD_DIR)+ toe_u = normalize(toe, FWD_DIR)+ score_ru = float(hint @ fwd_ru) + float(toe_u @ fwd_ru)+ score_ur = float(hint @ fwd_ur) + float(toe_u @ fwd_ur)+ use_up_cross_right = score_ur >= score_ru+ forward = fwd_ur if use_up_cross_right else fwd_ru+ formula = "up × right" if use_up_cross_right else "right × up"++ basis = np.column_stack([right, up, forward])+ det = float(np.linalg.det(basis))+ if verbose:+ print(f"\n=== {label} anatomical axes ===")+ print(f" right (RightUpLeg − LeftUpLeg, ortho to up) {np.round(right, 6)}")+ print(f" up (Neck − Hips) {np.round(up, 6)}")+ print(f" right × up {np.round(fwd_ru, 6)} chest· = {float(hint @ fwd_ru):+.4f} toe· = {float(toe_u @ fwd_ru):+.4f}")+ print(f" up × right {np.round(fwd_ur, 6)} chest· = {float(hint @ fwd_ur):+.4f} toe· = {float(toe_u @ fwd_ur):+.4f}")+ print(f" FRONT chosen {formula}")+ print(f" forward {np.round(forward, 6)}")+ print(f" basis columns [Right | Up | Forward]")+ print(f" det(basis) {det:+.6f}")+ if det < 0.0:+ print(" FLAG det < 0 — anatomical [Right|Up|Forward] is left-handed in this world.")+ print(" Expected for Mixamo: anatomical right is world −X (world +X is character left).")+ print(" This labels the existing world; it is not a transform we apply.")+ elif abs(det - 1.0) > 1e-3:+ print(f" FLAG |det| not 1 — basis is not orthonormal-proper.")+ else:+ print(" det ≈ +1 right-handed orthonormal frame.")+ return {+ "right": right,+ "up": up,+ "forward": forward,+ "basis": basis,+ "det": det,+ "front_formula": formula,+ "cross_right_up": fwd_ru,+ "cross_up_right": fwd_ur,+ "chest_hint": hint,+ "toe_hint": toe_u,+ }+++def global_alignment(skel: dict, rig: dict, *, verbose: bool = True) -> dict:+ """R such that R @ skeleton_axis ≈ rig_axis for Right, Up, Forward.++ R = B_rig @ B_skel.T. Not polar-projected. A det −1 result is a reflection+ and is flagged, not silently corrected with a 180° yaw.+ """+ b_s = skel["basis"]+ b_r = rig["basis"]+ r = b_r @ b_s.T+ det = float(np.linalg.det(r))+ if verbose:+ print("\n=== candidate global alignment R_skeleton_to_rig ===")+ print(" maps Skeleton Right/Up/Forward → Rig Right/Up/Forward")+ print(r)+ print(f" det(R_skeleton_to_rig) = {det:+.6f}")+ if det < 0.0:+ print(" FLAG det < 0 — this is a REFLECTION / mirror. Not applied as a rotation.")+ print(" No SVD/polar flip, no rotY(180) substitute.")+ elif abs(det - 1.0) > 5e-3:+ print(" FLAG det is not +1 — not a proper rotation.")+ else:+ print(" det ≈ +1 proper rotation (no reflection).")+ mapped = r @ b_s+ print(" R @ skeleton_right ", np.round(mapped[:, 0], 6), " vs rig", np.round(b_r[:, 0], 6))+ print(" R @ skeleton_up ", np.round(mapped[:, 1], 6), " vs rig", np.round(b_r[:, 1], 6))+ print(" R @ skeleton_forward ", np.round(mapped[:, 2], 6), " vs rig", np.round(b_r[:, 2], 6))+ return {"R": r, "det": det}+++def apply_global(+ pos: dict[str, np.ndarray],+ r: np.ndarray,+ scale: float,+ translation: np.ndarray,+) -> dict[str, np.ndarray]:+ return {name: scale * (r @ p) + translation for name, p in pos.items()}+++@dataclass(frozen=True)+class TposeCalibration:+ """Frozen result of the accepted T-pose orientation check.++ `skeleton_axes` / `rig_axes` are the anatomical [Right | Up | Forward] bases+ that produced the accepted FRONT/BACK/LEFT/RIGHT images. `r_skeleton_to_rig`+ maps one onto the other. `height_scale` stretches the Mixamo pelvis→head+ length onto the character rig's for visualisation only.+ """++ skeleton_rig: MixamoRig+ skeleton_tpose: dict[str, np.ndarray]+ rig_rest: dict[str, np.ndarray]+ skeleton_axes: dict+ rig_axes: dict+ r_skeleton_to_rig: np.ndarray+ height_scale: float+ fbx_meta: dict+ fbx_path: Path++ @property+ def forward(self) -> np.ndarray:+ """Anatomical FRONT in the character rig world. Used by the fixed render camera."""+ return np.asarray(self.rig_axes["forward"], dtype=np.float64)++ @property+ def up(self) -> np.ndarray:+ return np.asarray(self.rig_axes["up"], dtype=np.float64)++ @property+ def right(self) -> np.ndarray:+ return np.asarray(self.rig_axes["right"], dtype=np.float64)+++def load_calibration(fbx_path: Path, *, verbose: bool = False) -> TposeCalibration:+ """T-pose calibration between the Mixamo core skeleton and the character rig.++ Both sides come from the same character FBX: the core-bone skeleton drives+ the retarget, the full rig receives the motion. Everything is measured from+ the file; nothing is hand-authored.+ """+ fbx_path = Path(fbx_path)+ data = extract_skeleton(fbx_path)+ rig = rig_from_skeleton_data(data, fbx_path)+ skel = hips_origin(canonical_mixamo_tpose(rig))+ rest_raw, fbx_meta = rig_rest_from_skeleton_data(data, fbx_path)+ rest = hips_origin(rest_raw)++ ref_h = body_height(rest)+ skel_n = scale_to_height(skel, ref_h)+ height_scale = float(ref_h / body_height(skel))++ skel_axes = anatomical_axes(skel_n, "Mixamo skeleton T-pose", verbose=verbose)+ rig_axes = anatomical_axes(rest, f"{fbx_path.stem} FBX rest rig", verbose=verbose)+ align = global_alignment(skel_axes, rig_axes, verbose=verbose)++ return TposeCalibration(+ skeleton_rig=rig,+ skeleton_tpose=skel_n,+ rig_rest=rest,+ skeleton_axes=skel_axes,+ rig_axes=rig_axes,+ r_skeleton_to_rig=align["R"],+ height_scale=height_scale,+ fbx_meta=fbx_meta,+ fbx_path=fbx_path,+ )diff --git a/motion2mixamorig/mixamo/ybot_retarget.py b/motion2mixamorig/mixamo/ybot_retarget.pynew file mode 100644index 0000000..63c9aca--- /dev/null+++ b/motion2mixamorig/mixamo/ybot_retarget.py@@ -0,0 +1,234 @@+"""Mixamo Skeleton motion → Y Bot Rig motion, derived from the T-pose calibration.++ validated_mixamo_motion+ ↓+ tpose_calibration+ ↓+ apply_rest_relative_rotations()+ ↓+ ybot_rig_fk++For every controlled bone we measure how the Mixamo Skeleton bone moved away+from its own rest/T-pose, and apply that same rest-relative motion to the+corresponding Y Bot bone:++ delta_skeleton = inverse(skeleton_rest_local_rotation) * skeleton_frame_local_rotation+ ybot_frame_local = ybot_rest_local_rotation * (C * delta_skeleton * Cᵀ)++`C` is the per-bone rest-frame basis conversion. It is *derived* from the two+validated T-pose bone frames (see `bone_basis_change`), never authored by hand.+No world-space Euler angles are copied, and there are no per-bone 90°/180°+constants, laterality swaps or camera-based assumptions anywhere in this module.+"""++from __future__ import annotations++from dataclasses import dataclass+from pathlib import Path++import numpy as np++from .kinematics import MixamoRig, forward_kinematics, world_positions_m+from .skinned_mesh import load_full_rig+from .tpose_calibration import (+ TposeCalibration,+ anatomical_axes,+ global_alignment,+ hips_origin,+)++# Bones whose motion is authored by the validated Human → Mixamo retarget.+# Anatomical LEFT stays LEFT and RIGHT stays RIGHT.+CONTROLLED_BONES = (+ "mixamorig:Hips",+ "mixamorig:Spine",+ "mixamorig:Spine1",+ "mixamorig:Spine2",+ "mixamorig:Neck",+ "mixamorig:Head",+ "mixamorig:LeftShoulder",+ "mixamorig:LeftArm",+ "mixamorig:LeftForeArm",+ "mixamorig:LeftHand",+ "mixamorig:RightShoulder",+ "mixamorig:RightArm",+ "mixamorig:RightForeArm",+ "mixamorig:RightHand",+ "mixamorig:LeftUpLeg",+ "mixamorig:LeftLeg",+ "mixamorig:LeftFoot",+ "mixamorig:LeftToeBase",+ "mixamorig:RightUpLeg",+ "mixamorig:RightLeg",+ "mixamorig:RightFoot",+ "mixamorig:RightToeBase",+)+++def rest_world_rotations(rig: MixamoRig) -> dict[str, np.ndarray]:+ """World rotation of each bone's rest frame.++ FK is `M_local = T * Rpre * R_anim`, so at rest (`R_anim = I`) the bone's+ world rotation is `R_parent_world @ Rpre`. PreRotation therefore stays part+ of the rest transform and is never folded into the animation channel.+ """+ world = forward_kinematics(rig, {})+ return {name: np.asarray(mat[:3, :3], dtype=np.float64) for name, mat in world.items()}+++def bone_basis_change(+ skel_rest_rot: dict[str, np.ndarray],+ ybot_rest_rot: dict[str, np.ndarray],+ bone: str,+ r_world_align: np.ndarray,+) -> np.ndarray:+ """Rest-frame basis conversion `C` for one bone, from the two T-pose frames.++ A rest-relative local rotation `D` acts on the bone's own rest frame. In the+ Mixamo Skeleton that frame sits in the world as `F = skel_rest_rot[bone]`;+ in the Y Bot as `F' = ybot_rest_rot[bone]`. Requiring the *world* motion to+ match (up to the calibrated world alignment `R`) gives++ F' D' F'ᵀ = R (F D Fᵀ) Rᵀ ⇒ D' = C D Cᵀ, C = F'ᵀ R F++ so `C` falls straight out of the two validated rest poses.+ """+ f_skel = np.asarray(skel_rest_rot[bone], dtype=np.float64)+ f_ybot = np.asarray(ybot_rest_rot[bone], dtype=np.float64)+ return f_ybot.T @ np.asarray(r_world_align, dtype=np.float64) @ f_skel+++@dataclass(frozen=True)+class YBotMapping:+ """Frozen Mixamo Skeleton → Y Bot transfer, built once from the calibration."""++ skeleton_rig: MixamoRig+ ybot_rig: MixamoRig+ basis_change: dict[str, np.ndarray]+ translation_scale: float+ r_world_align: np.ndarray+ max_basis_deviation_deg: float++ @property+ def controlled(self) -> tuple[str, ...]:+ return tuple(name for name in CONTROLLED_BONES if name in self.ybot_rig.bones)++ def apply_rest_relative_rotations(+ self, r_anim_skeleton: dict[str, np.ndarray]+ ) -> dict[str, np.ndarray]:+ """Transfer one frame of rest-relative local rotations onto the Y Bot rig.++ Input and output are both animation-channel rotations, i.e. the `R_anim`+ of `M_local = T * Rpre * R_anim`. Bones the Mixamo retarget does not+ author (the finger chains) are left at rest.+ """+ out: dict[str, np.ndarray] = {}+ for bone, delta in r_anim_skeleton.items():+ change = self.basis_change.get(bone)+ if change is None:+ continue+ out[bone] = change @ np.asarray(delta, dtype=np.float64) @ change.T+ return out++ def hips_translation_cm(self, pelvis_translation_cm: np.ndarray) -> np.ndarray:+ """Root translation carried across with the calibrated scale and world alignment."""+ p = np.asarray(pelvis_translation_cm, dtype=np.float64).reshape(3)+ return self.translation_scale * (self.r_world_align @ p)++ def ybot_rig_fk(+ self,+ r_anim_ybot: dict[str, np.ndarray],+ hips_translation_cm: np.ndarray,+ ) -> dict[str, np.ndarray]:+ """World 4x4 per Y Bot bone. Raw FBX rest hierarchy, animation on top."""+ return forward_kinematics(+ self.ybot_rig,+ r_anim_ybot,+ hips_translation_cm=np.asarray(hips_translation_cm, dtype=np.float64),+ )++ def ybot_positions_m(+ self,+ r_anim_ybot: dict[str, np.ndarray],+ hips_translation_cm: np.ndarray,+ ) -> dict[str, np.ndarray]:+ return world_positions_m(self.ybot_rig, self.ybot_rig_fk(r_anim_ybot, hips_translation_cm))+++def _rotation_angle_deg(rot: np.ndarray) -> float:+ cosine = (float(np.trace(np.asarray(rot, dtype=np.float64))) - 1.0) * 0.5+ return float(np.degrees(np.arccos(np.clip(cosine, -1.0, 1.0))))+++def build_mapping(+ calibration: TposeCalibration,+ fbx_path: Path | None = None,+ *,+ verbose: bool = True,+) -> YBotMapping:+ """Derive the Mixamo Skeleton → character-rig transfer from the calibration."""+ fbx_path = Path(fbx_path) if fbx_path is not None else calibration.fbx_path+ skel_rig = calibration.skeleton_rig+ ybot_rig = load_full_rig(fbx_path)++ skel_rest_rot = rest_world_rotations(skel_rig)+ ybot_rest_rot = rest_world_rotations(ybot_rig)++ # World alignment between the two rigs, measured with the same+ # `anatomical_axes` / `global_alignment` that produced the accepted+ # orientation-check images — so the animation cannot invent a different+ # Front/Right/Up convention. Both sides are the rigs' own rest poses.+ skel_rest_pos = hips_origin(world_positions_m(skel_rig, forward_kinematics(skel_rig, {})))+ ybot_rest_pos = hips_origin(world_positions_m(ybot_rig, forward_kinematics(ybot_rig, {})))+ skel_rest_axes = anatomical_axes(skel_rest_pos, "Mixamo Skeleton rest", verbose=False)+ ybot_rest_axes = anatomical_axes(ybot_rest_pos, "Y Bot rig rest", verbose=False)+ r_world_align = global_alignment(skel_rest_axes, ybot_rest_axes, verbose=False)["R"]++ # The Y Bot anatomical frame used here must be the one the calibration+ # accepted; otherwise the render camera and the mapping would disagree.+ front_drift = float(+ np.degrees(+ np.arccos(+ np.clip(float(np.asarray(ybot_rest_axes["forward"]) @ calibration.forward), -1.0, 1.0)+ )+ )+ )+ if front_drift > 1e-6:+ raise RuntimeError(+ f"rig FRONT disagrees with the accepted calibration by {front_drift:.6f} deg"+ )++ basis_change: dict[str, np.ndarray] = {}+ worst = 0.0+ for bone in CONTROLLED_BONES:+ if bone not in skel_rest_rot or bone not in ybot_rest_rot:+ continue+ change = bone_basis_change(skel_rest_rot, ybot_rest_rot, bone, r_world_align)+ basis_change[bone] = change+ worst = max(worst, _rotation_angle_deg(change))++ translation_scale = float(skel_rig.meters_per_unit / ybot_rig.meters_per_unit)++ if verbose:+ print("\n=== Mixamo Skeleton → Y Bot rest-relative mapping ===")+ print(f" skeleton rig {skel_rig.source.name} {len(skel_rig.order)} bones")+ print(f" Y Bot rig {Path(fbx_path).name} {len(ybot_rig.order)} bones")+ print(f" controlled bones {len(basis_change)}")+ print(f" untouched (rest) bones {len(ybot_rig.order) - len(basis_change)} (finger chains)")+ print(f" translation scale {translation_scale:.6f}")+ print(f" world alignment {_rotation_angle_deg(r_world_align):.6f} deg (measured, det={np.linalg.det(r_world_align):+.6f})")+ print(f" max |C| rotation {worst:.6f} deg (0 = identical rest frames)")+ if worst < 1e-6:+ print(" the two validated T-pose bone frames coincide, so C = I for every bone:")+ print(" the Mixamo rest-relative rotation is applied to the Y Bot unchanged.")+ else:+ print(" C is a measured rest-frame basis change, not a hand-authored offset.")++ return YBotMapping(+ skeleton_rig=skel_rig,+ ybot_rig=ybot_rig,+ basis_change=basis_change,+ translation_scale=translation_scale,+ r_world_align=r_world_align,+ max_basis_deviation_deg=worst,+ )diff --git a/motion2mixamorig/paths.py b/motion2mixamorig/paths.pynew file mode 100644index 0000000..f9b9998--- /dev/null+++ b/motion2mixamorig/paths.py@@ -0,0 +1,59 @@+"""Single source of truth for project paths: assets, weights, outputs."""++from __future__ import annotations++import os+from datetime import datetime+from pathlib import Path++PROJECT_ROOT = Path(__file__).resolve().parents[1]++ASSETS = PROJECT_ROOT / "assets"+BODY_MODELS = ASSETS / "body_models"+SMPLX_NEUTRAL = BODY_MODELS / "smplx" / "SMPLX_NEUTRAL.npz"+MIXAMO_DIR = ASSETS / "mixamo"+DEFAULT_RIG = MIXAMO_DIR / "Y_Bot.fbx"+VIDEO_DIR = ASSETS / "video"++WEIGHTS = PROJECT_ROOT / "weights"+OUTPUTS = PROJECT_ROOT / "outputs"++VIDEO_SUFFIXES = (".mp4", ".mov", ".avi", ".mkv", ".webm")+++def list_videos() -> list[Path]:+ """User-supplied videos in assets/video/, most recently added first."""+ if not VIDEO_DIR.is_dir():+ return []+ videos = [+ p+ for p in VIDEO_DIR.iterdir()+ if p.suffix.lower() in VIDEO_SUFFIXES and not p.name.startswith(".")+ ]+ return sorted(videos, key=lambda p: p.stat().st_mtime, reverse=True)+++def latest_video() -> Path | None:+ """The video most recently placed into assets/video/ (by file mtime)."""+ videos = list_videos()+ return videos[0] if videos else None+++def new_run_dir(video: Path, now: datetime | None = None) -> Path:+ """Create outputs/<YYYYMMDD_HHMMSS>_<video-stem>/ for one `m2mr run`."""+ stamp = (now or datetime.now()).strftime("%Y%m%d_%H%M%S")+ run_dir = OUTPUTS / f"{stamp}_{video.stem}"+ run_dir.mkdir(parents=True, exist_ok=True)+ (run_dir / "videos").mkdir(exist_ok=True)+ return run_dir+++def export_gvhmr_env() -> None:+ """Point GVHMR at this project's weights/ and assets/ before importing it.++ The gvhmr package resolves its asset roots from these variables at import+ time. Auto-downloaded checkpoints land in weights/; the gated SMPL-X body+ model is read from assets/body_models/ where the user placed it.+ """+ os.environ.setdefault("GVHMR_CHECKPOINTS", str(WEIGHTS))+ os.environ.setdefault("GVHMR_BODY_MODELS", str(BODY_MODELS))diff --git a/motion2mixamorig/pipeline.py b/motion2mixamorig/pipeline.pynew file mode 100644index 0000000..aff185b--- /dev/null+++ b/motion2mixamorig/pipeline.py@@ -0,0 +1,158 @@+"""One `m2mr run`: video -> human skeleton -> Mixamo rig -> result videos.++Everything a run produces lands in its own timestamped folder:++ outputs/<YYYYMMDD_HHMMSS>_<video>/+ ├── run.json what ran, with what inputs, producing what+ ├── skeleton_motion.npz 3D human skeleton (24 SMPL joints per frame)+ ├── mixamo_rotations.npz per-bone animation rotations for the rig+ ├── mixamo_character.glb skinned character + animation (Blender / Unity)+ ├── cache/ GVHMR intermediates (2D pose, 3D lift)+ └── videos/ the four result videos+"""++from __future__ import annotations++import json+import sys+import time+from datetime import datetime+from pathlib import Path++import numpy as np++from . import paths+from .human_motion import HumanMotion, load_human_motion+from .mixamo.animation import build_animation+from .mixamo.gltf import write_glb+from .mixamo.kinematics import quat_wxyz+from .mixamo.retarget import verify_rest_fk+from .mixamo.skinned_mesh import load_character_asset, rest_skin_error+from .mixamo.tpose_calibration import load_calibration+from .mixamo.ybot_retarget import build_mapping+from .videos import render_run_videos++ROTATION_CONVENTION = (+ "Animation-channel rotations: M_local = T * Rpre * R_anim, world = parent @ local. "+ "Quaternions are wxyz. FBX rest translations (T, Rpre) come from the rig file."+)+++def save_mixamo_rotations(anim, mapping, rig_path: Path, out_npz: Path) -> Path:+ """Per-bone animation rotations, both the portable Mixamo Skeleton channel+ and the channel already converted for this rig file."""+ bones = list(mapping.controlled)+ n = anim.n+ q_skel = np.zeros((n, len(bones), 4), dtype=np.float64)+ q_rig = np.zeros((n, len(bones), 4), dtype=np.float64)+ for i in range(n):+ for b, name in enumerate(bones):+ q_skel[i, b] = quat_wxyz(anim.r_anim_skeleton[i].get(name, np.eye(3)))+ q_rig[i, b] = quat_wxyz(anim.r_anim_rig[i].get(name, np.eye(3)))+ out_npz = Path(out_npz)+ out_npz.parent.mkdir(parents=True, exist_ok=True)+ np.savez_compressed(+ out_npz,+ bone_names=np.array(bones),+ quat_wxyz_skeleton=q_skel,+ quat_wxyz_rig=q_rig,+ hips_translation_cm_skeleton=np.stack(anim.hips_cm_skeleton),+ hips_translation_cm_rig=np.stack(anim.hips_cm_rig),+ frames=np.array(anim.frames, dtype=np.int64),+ fps=np.float64(anim.fps),+ rig=np.array(str(rig_path)),+ convention=np.array(ROTATION_CONVENTION),+ )+ return out_npz+++def run_pipeline(+ video: Path,+ rig: Path,+ *,+ device: str = "cpu",+ skeleton_npz: Path | None = None,+ verbose: bool = True,+) -> Path:+ """Full pipeline for one video + one rig. Returns the run directory.++ `skeleton_npz` reuses a previous run's skeleton_motion.npz and skips the+ (slow) video extraction entirely.+ """+ video = Path(video).resolve()+ rig = Path(rig).resolve()+ started = datetime.now()+ t0 = time.time()+ run_dir = paths.new_run_dir(video, started)+ print(f"run directory {run_dir}")++ motion_npz = run_dir / "skeleton_motion.npz"+ if skeleton_npz is not None:+ import shutil++ shutil.copy2(skeleton_npz, motion_npz)+ print(f"reusing skeleton from {skeleton_npz}")+ else:+ from .extract import extract_motion++ print(f"extracting human motion from {video.name}")+ extract_motion(+ video,+ motion_npz,+ work_dir=run_dir / "cache",+ device=device,+ verbose=verbose,+ )+ motion: HumanMotion = load_human_motion(motion_npz)+ print(f"human motion {motion.n_frames} frames @ {motion.fps:g} fps")++ print(f"calibrating rig {rig.name}")+ calibration = load_calibration(rig)+ rest_err = verify_rest_fk(calibration.skeleton_rig)+ mapping = build_mapping(calibration, verbose=verbose)+ asset = load_character_asset(rig, textures=True)+ skin_err = rest_skin_error(asset)+ if verbose:+ print(f"rest FK error {rest_err:.3e} rest skin error {skin_err:.3e} cm")++ print("retargeting")+ frames = list(range(motion.n_frames))+ anim = build_animation(motion, mapping, frames, verbose=verbose)++ rotations_npz = run_dir / "mixamo_rotations.npz"+ save_mixamo_rotations(anim, mapping, rig, rotations_npz)+ print(f"saved {rotations_npz}")++ glb_path = run_dir / "mixamo_character.glb"+ write_glb(glb_path, asset, anim.r_anim_rig, anim.hips_cm_rig, anim.fps)+ print(f"saved {glb_path}")++ print("rendering result videos")+ videos = render_run_videos(+ anim, asset, calibration, video, run_dir / "videos", verbose=verbose+ )++ report = {+ "started": started.isoformat(timespec="seconds"),+ "elapsed_s": round(time.time() - t0, 1),+ "command": sys.argv,+ "video": str(video),+ "rig": str(rig),+ "device": device,+ "fps": float(motion.fps),+ "n_frames": motion.n_frames,+ "skeleton_reused_from": str(skeleton_npz) if skeleton_npz else None,+ "rest_fk_error": rest_err,+ "rest_skin_error_cm": skin_err,+ "twist_events": len(anim.twist_events),+ "quat_sign_flips": len(anim.sign_events),+ "outputs": {+ "skeleton_motion": motion_npz.name,+ "mixamo_rotations": rotations_npz.name,+ "mixamo_character_glb": glb_path.name,+ "videos": {k: f"videos/{v.name}" for k, v in videos.items()},+ },+ }+ (run_dir / "run.json").write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")+ print(f"done in {report['elapsed_s']}s -> {run_dir}")+ return run_dirdiff --git a/motion2mixamorig/stabilize_kp2d.py b/motion2mixamorig/stabilize_kp2d.pynew file mode 100644index 0000000..19199ae--- /dev/null+++ b/motion2mixamorig/stabilize_kp2d.py@@ -0,0 +1,96 @@+"""Keep COCO-17 left/right identity consistent across time.++RTMPose/ViTPose label left/right independently per frame. When the subject+turns to face away, those labels often swap. GVHMR then reads a jumping 2D+skeleton and spins the 3D heading. This assigns each left/right pair to the+previous frame's positions so anatomical identity is tracked, not re-guessed.+"""++from __future__ import annotations++import shutil+from pathlib import Path++import numpy as np++# Full COCO-17 left/right permutation (nose stays).+COCO17_SWAP = np.array([0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15], dtype=np.int64)+++def _as_numpy(x) -> np.ndarray:+ if hasattr(x, "detach"):+ return x.detach().cpu().numpy()+ return np.asarray(x)+++def _lr_x(frame: np.ndarray) -> float:+ """Shoulder left-minus-right in image x. Hips if shoulders are collapsed."""+ sh = float(frame[5, 0] - frame[6, 0])+ if abs(sh) >= 40.0:+ return sh+ return float(frame[11, 0] - frame[12, 0])+++def stabilize_coco17(kp2d: np.ndarray) -> tuple[np.ndarray, np.ndarray]:+ """Return (stabilized (F,17,3), per-frame swap flags (F,)).++ A real 180-degree turn drives left-right x through a narrow side view. A+ label flip jumps the sign while the width stays large. Those jumps are+ un-swapped so GVHMR sees one anatomical identity.++ "Narrow" is relative to how wide this subject usually reads: on a mid-spin+ frame the estimator can report the shoulders 50-60 px apart instead of ~0+ (one such borderline frame per turn is common), and a fixed 40 px cut then+ misreads the whole genuine turn as a label flip and un-swaps it away. The+ width is invariant under label swaps, so the scale can be measured on the+ raw input up front.+ """+ src = np.asarray(_as_numpy(kp2d), dtype=np.float64)+ if src.ndim != 3 or src.shape[1] < 17:+ raise ValueError(f"expected (F, 17, 3) COCO-17, got {src.shape}")+ out = src.copy()+ n = len(out)+ narrow = float(np.clip(0.45 * np.median([abs(_lr_x(f)) for f in src]), 40.0, 80.0))+ swapped = np.zeros(n, dtype=np.int32)+ last_sign = np.sign(_lr_x(out[0])) or 1.0+ last_turn = -(10**9)+ for i in range(n):+ raw = src[i]+ width = abs(_lr_x(raw))+ sign = np.sign(_lr_x(raw)) or last_sign+ if width >= 40.0 and sign != last_sign:+ start = max(0, last_turn + 1, i - 18)+ widths = [abs(_lr_x(f)) for f in out[start:i]]+ run = best = 0+ for w in widths:+ run = run + 1 if w < narrow else 0+ best = max(best, run)+ if best >= (1 if last_turn < 0 else 4):+ last_sign = sign+ last_turn = i+ else:+ raw = raw[COCO17_SWAP]+ swapped[i] = 1+ out[i] = raw+ sh = out[i, 5, 0] - out[i, 6, 0]+ hip = out[i, 11, 0] - out[i, 12, 0]+ if abs(sh) >= 40.0 and abs(hip) >= 40.0 and np.sign(sh) != np.sign(hip):+ out[i, [11, 12, 13, 14, 15, 16]] = out[i, [12, 11, 14, 13, 16, 15]]+ return out.astype(np.float32), swapped+++def stabilize_cache(path: Path) -> np.ndarray:+ """Stabilize a GVHMR 2D-pose cache file in place (keeping a *_raw backup)."""+ import torch++ raw = path.with_name(path.stem + "_raw" + path.suffix)+ src = raw if raw.exists() else path+ if not src.exists():+ raise FileNotFoundError(path)+ if src == path and not raw.exists():+ shutil.copy2(path, raw)+ tensor = torch.load(src, weights_only=False)+ fixed, swaps = stabilize_coco17(tensor)+ print(f"stabilized {path.name}: corrected {int(swaps.sum())}/{len(swaps)} left/right flips")+ torch.save(torch.from_numpy(fixed), path)+ return fixeddiff --git a/motion2mixamorig/videos.py b/motion2mixamorig/videos.pynew file mode 100644index 0000000..c31d50f--- /dev/null+++ b/motion2mixamorig/videos.py@@ -0,0 +1,312 @@+"""Render the four result videos of a run, all in the input video's aspect ratio.++ videos/human_skeleton.mp4 3D human skeleton recovered from the video+ videos/mixamo_skeleton.mp4 retargeted Mixamo Skeleton+ videos/mixamo_character.mp4 skinned character mesh+ videos/compare.mp4 2x2 grid: original | mixamo skeleton+ human | character++The three rendered panels share one perspective camera and one floor grid, so+they read as views into the same space: an orthographic camera cannot show a+floor receding to a vanishing point, and without a floor and a contact shadow+the figure just hangs in black. All of it is fitted once over the whole clip;+nothing is re-centred per frame.++Frames are streamed straight into ffmpeg (which also muxes the source video's+audio into every output), so memory stays flat no matter how long the clip is.+When ffmpeg is missing the videos still get written via OpenCV, just silent.+"""++from __future__ import annotations++import shutil+import subprocess+from pathlib import Path++import numpy as np++from .mixamo.animation import RetargetedAnimation, all_points, skin_frames+from .mixamo.render import (+ DARK_THEME,+ GRID_THEME,+ MESH_COLORS_BGR,+ Ground,+ blank_panel,+ build_perspective_camera,+ draw_mesh,+ draw_shadow,+ draw_skeleton,+ fit_letterbox,+ ground_backdrop,+ label_panel,+)+from .mixamo.retarget import HUMAN_EDGES, MIXAMO_EDGES++# Rendered panel height in pixels; width follows the input video's aspect.+PANEL_H = 720+MAX_PANEL_W = 1280++# Frames sampled only to bound the mesh for the camera fit and the floor+# height; never all kept, or the streaming render would gain nothing.+CAMERA_MESH_SAMPLES = 60+# Fraction of the panel the whole clip's motion is fitted into.+CAMERA_FILL = 0.94+# Pinhole view matching the Mixamo viewport the look was taken from. Level+# (elevation 0): a pinhole camera standing above the floor still sees it+# recede, and tilting down makes the chest overhang the hips.+CAMERA_FOV_DEG = 28.0+CAMERA_AZIMUTH_DEG = 0.0+CAMERA_ELEVATION_DEG = 0.0+# The sole wanders over ~20 cm across a clip, mostly on lifted-leg frames, so+# the floor goes at a low percentile of it rather than the minimum. Higher and+# the character sinks in on many frames; lower and it visibly floats on all.+FLOOR_PERCENTILE = 10.0+# The character keeps its own texture colours, and a navy or black outfit+# against a black backdrop has no silhouette: its panel gets the lifted+# backdrop and a little more ambient fill than the white-on-black skeletons.+CHARACTER_AMBIENT = 0.34+++def video_size(video: Path) -> tuple[int, int]:+ import cv2++ cap = cv2.VideoCapture(str(video))+ if not cap.isOpened():+ raise RuntimeError(f"Cannot open video: {video}")+ w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))+ h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))+ cap.release()+ return w, h+++def panel_size(video: Path) -> tuple[int, int]:+ """Panel with the same aspect ratio as the input video, even dimensions."""+ w, h = video_size(video)+ pw = int(round(PANEL_H * w / max(h, 1)))+ pw = min(max(pw, 2), MAX_PANEL_W)+ return pw + pw % 2, PANEL_H + PANEL_H % 2+++class _OriginalReader:+ """Sequential reader that letterboxes each requested source frame."""++ def __init__(self, src: Path, width: int, height: int):+ import cv2++ self.blank = blank_panel(width, height)+ self.width, self.height = width, height+ self.cap = cv2.VideoCapture(str(src))+ self.next_idx = 0++ def frame(self, idx: int) -> np.ndarray:+ out = None+ while self.cap.isOpened() and self.next_idx <= idx:+ ok, frame = self.cap.read()+ if not ok:+ break+ if self.next_idx == idx:+ out = fit_letterbox(frame, self.width, self.height)+ self.next_idx += 1+ return self.blank.copy() if out is None else out++ def close(self) -> None:+ self.cap.release()+++class _StreamWriter:+ """Raw BGR frames piped into ffmpeg, which encodes H.264 and muxes the+ source video's audio over the rendered frame window.++ Falls back to a plain (silent) OpenCV writer when ffmpeg is unavailable.+ """++ def __init__(+ self,+ out_path: Path,+ fps: float,+ width: int,+ height: int,+ *,+ audio_src: Path | None = None,+ audio_start_s: float = 0.0,+ audio_dur_s: float | None = None,+ ):+ self.out_path = Path(out_path)+ self.out_path.parent.mkdir(parents=True, exist_ok=True)+ self.proc = None+ self.writer = None+ ffmpeg = shutil.which("ffmpeg")+ if ffmpeg is not None:+ cmd = [+ ffmpeg, "-y", "-loglevel", "error",+ "-f", "rawvideo", "-pix_fmt", "bgr24",+ "-s", f"{width}x{height}", "-r", f"{fps}",+ "-i", "-",+ ]+ if audio_src is not None and Path(audio_src).exists():+ cmd += ["-ss", f"{audio_start_s:.6f}"]+ if audio_dur_s is not None:+ cmd += ["-t", f"{audio_dur_s:.6f}"]+ # `0:v` is the rendered frames, `1:a:0?` the source audio if+ # it has any; -shortest stops the audio with the last frame.+ cmd += ["-i", str(audio_src), "-map", "0:v:0", "-map", "1:a:0?",+ "-c:a", "aac", "-b:a", "192k", "-shortest"]+ cmd += ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "18",+ "-preset", "medium", "-movflags", "+faststart",+ str(self.out_path)]+ self.proc = subprocess.Popen(cmd, stdin=subprocess.PIPE)+ else:+ import cv2++ self.writer = cv2.VideoWriter(+ str(self.out_path), cv2.VideoWriter_fourcc(*"mp4v"),+ float(fps), (width, height),+ )+ if not self.writer.isOpened():+ raise RuntimeError(f"Cannot open VideoWriter for {self.out_path}")++ def write(self, frame: np.ndarray) -> None:+ if self.proc is not None:+ self.proc.stdin.write(frame.tobytes())+ else:+ self.writer.write(frame)++ def close(self) -> Path:+ if self.proc is not None:+ self.proc.stdin.close()+ if self.proc.wait() != 0:+ raise RuntimeError(f"ffmpeg failed encoding {self.out_path}")+ if self.writer is not None:+ self.writer.release()+ return self.out_path+++def _grid2x2(tl: np.ndarray, tr: np.ndarray, bl: np.ndarray, br: np.ndarray) -> np.ndarray:+ return np.vstack([np.hstack([tl, tr]), np.hstack([bl, br])])+++def render_run_videos(+ anim: RetargetedAnimation,+ asset,+ calibration,+ source_video: Path,+ out_dir: Path,+ *,+ verbose: bool = True,+) -> dict[str, Path]:+ """Write the four result videos into `out_dir` (videos/ of a run)."""+ out_dir = Path(out_dir)+ out_dir.mkdir(parents=True, exist_ok=True)+ width, height = panel_size(source_video)+ fps = anim.fps++ # One camera and one floor over human joints, Mixamo joints and the mesh,+ # so the three rendered panels agree on where everything stands. Human,+ # Mixamo Skeleton and character all live in the calibrated rig world, so+ # the shared anatomical axes come from the calibration.+ axes = calibration.rig_axes+ stride = max(1, anim.n // CAMERA_MESH_SAMPLES)+ probe = skin_frames(asset, anim, list(range(0, anim.n, stride)))+ frames_pts = [np.concatenate(v, axis=0) for v in probe]+ del probe+ soles = np.array([float(pts[:, 1].min()) for pts in frames_pts])+ mesh_pts = np.concatenate(frames_pts, axis=0)+ del frames_pts++ cam = build_perspective_camera(+ axes,+ np.concatenate(+ [all_points(anim.human_pos), all_points(anim.mixamo_pos), mesh_pts],+ axis=0,+ ),+ width,+ height,+ fov_deg=CAMERA_FOV_DEG,+ azimuth_deg=CAMERA_AZIMUTH_DEG,+ elevation_deg=CAMERA_ELEVATION_DEG,+ fill=CAMERA_FILL,+ )+ del mesh_pts+ hips = np.stack([anim.mixamo_pos[i]["mixamorig:Hips"] for i in range(anim.n)])+ ground = Ground(+ y=float(np.percentile(soles, FLOOR_PERCENTILE)),+ cx=float(np.median(hips[:, 0])),+ cz=float(np.median(hips[:, 2])),+ )+ if verbose:+ print(f" camera eye {np.round(cam.eye, 3)} fov {cam.fov_deg:.1f} deg"+ f" floor y {ground.y:+.4f} m")++ # Camera and floor are both fixed, so the floor is drawn once per theme+ # and every frame starts as a copy of it.+ backdrop_dark = ground_backdrop(cam, ground, width, height, DARK_THEME)+ backdrop_grid = ground_backdrop(cam, ground, width, height, GRID_THEME)++ faces = [m.faces for m in asset.meshes]+ colors = [+ m.face_colors+ if m.face_colors is not None+ else MESH_COLORS_BGR.get(m.name, np.array([150.0, 150.0, 150.0]))+ for m in asset.meshes+ ]++ audio_start = anim.frames[0] / fps+ audio_dur = anim.n / fps++ def writer(name: str, w: int, h: int) -> _StreamWriter:+ return _StreamWriter(+ out_dir / name, fps, w, h,+ audio_src=source_video,+ audio_start_s=audio_start,+ audio_dur_s=audio_dur,+ )++ character = calibration.fbx_path.stem+ writers = {+ "human_skeleton": writer("human_skeleton.mp4", width, height),+ "mixamo_skeleton": writer("mixamo_skeleton.mp4", width, height),+ "mixamo_character": writer("mixamo_character.mp4", width, height),+ "compare": writer("compare.mp4", 2 * width, 2 * height),+ }+ originals = _OriginalReader(source_video, width, height)+ try:+ for i in range(anim.n):+ t = anim.frames[i] / fps+ stamp = f"f{anim.frames[i]} t={t:.2f}s"++ p_hum = backdrop_dark.copy()+ pos = anim.human_pos[i]+ draw_shadow(p_hum, cam, np.stack(list(pos.values())), ground, DARK_THEME)+ draw_skeleton(p_hum, pos, HUMAN_EDGES, cam, 2, DARK_THEME.joint)+ label_panel(p_hum, f"Human Skeleton {stamp}")++ p_mix = backdrop_dark.copy()+ pos = anim.mixamo_pos[i]+ draw_shadow(p_mix, cam, np.stack(list(pos.values())), ground, DARK_THEME)+ draw_skeleton(p_mix, pos, MIXAMO_EDGES, cam, 3, DARK_THEME.joint)+ label_panel(p_mix, f"Mixamo Skeleton {stamp}")++ p_char = backdrop_grid.copy()+ verts = skin_frames(asset, anim, [i])[0]+ draw_shadow(p_char, cam, np.concatenate(verts, axis=0), ground, GRID_THEME)+ draw_mesh(p_char, verts, faces, colors, cam, CHARACTER_AMBIENT)+ label_panel(p_char, f"{character} {stamp}")++ p_orig = originals.frame(anim.frames[i])+ label_panel(p_orig, f"Original Video {stamp}")++ writers["human_skeleton"].write(p_hum)+ writers["mixamo_skeleton"].write(p_mix)+ writers["mixamo_character"].write(p_char)+ writers["compare"].write(_grid2x2(p_orig, p_mix, p_hum, p_char))++ if verbose and (i == 0 or (i + 1) % 100 == 0 or i == anim.n - 1):+ print(f" render {i + 1}/{anim.n}")+ finally:+ originals.close()++ out = {name: w.close() for name, w in writers.items()}+ if verbose:+ for path in out.values():+ print(f"saved {path}")+ return outdiff --git a/pyproject.toml b/pyproject.tomlnew file mode 100644index 0000000..379490b--- /dev/null+++ b/pyproject.toml@@ -0,0 +1,23 @@+[project]+name = "motion2mixamorig"+version = "0.1.0"+description = "Retarget human motion from video to an Adobe Mixamo-rigged 3D character."+requires-python = ">=3.10"+dependencies = [+ "numpy",+ "opencv-python",+ "einops",+ # Video -> 3D human skeleton. `preproc` = person detection/tracking,+ # `rtmpose` = the ungated 2D-pose backend the pipeline uses.+ "gvhmr[preproc,rtmpose]>=1.6",+]++[project.scripts]+m2mr = "motion2mixamorig.cli:main"++[build-system]+requires = ["setuptools>=68"]+build-backend = "setuptools.build_meta"++[tool.setuptools.packages.find]+include = ["motion2mixamorig*"]diff --git a/weights/.gitkeep b/weights/.gitkeepnew file mode 100644index 0000000..e69de29