Skip to content

Retarget human motion from video to an Adobe Mixamo-rigged 3D character.

Python

Add still-image pose input via --image

MaxLiu631d752f925a
diff --git a/AGENTS.md b/AGENTS.md
index 21d5d59..4c0d52f 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,7 +1,8 @@
# AGENTS.md
Motion2MixamoRig drives an Adobe Mixamo-rigged 3D character with human motion
-extracted from a monocular video. CLI: `m2mr`.
+extracted from a monocular video, or a static 3D pose from a still photo.
+CLI: `m2mr`.
## Setup
@@ -14,13 +15,15 @@ pip install --upgrade pip setuptools wheel \
&& pip install -e .
```
-Three things must be supplied by the user (all git-ignored, licensing-gated):
+These must be supplied by the user (all git-ignored, licensing-gated).
+A run needs SMPL-X, a Mixamo character, and either a video or a still:
| What | Where to put it | Source |
| --- | --- | --- |
| SMPL-X body model | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | https://smpl-x.is.tue.mpg.de/ (registration) |
| Mixamo character | `assets/mixamo/<Name>.fbx` (FBX Binary, T-pose) | https://www.mixamo.com/ (Adobe account) |
| Input video | `assets/video/<name>.mp4` | one clearly visible person (group shots fail preflight) |
+| Input still | `assets/image/<name>.jpg` | one clearly visible person, head to toe (optional; `--image`) |
Inference weights (~5 GB) auto-download into `weights/` on the first run.
@@ -30,26 +33,28 @@ Inference weights (~5 GB) auto-download into `weights/` on the first run.
m2mr doctor # verify assets + dependencies; prints what is missing and where to get it
m2mr run # latest video in assets/video/ + assets/mixamo/Y_Bot.fbx
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Y_Bot.fbx
m2mr run --skeleton outputs/<run>/skeleton_motion.npz --rig ... # reuse extraction, swap rig
```
-`--video` and `--rig` combine freely. Extraction is the slow step; reuse it
-with `--skeleton` when only the rig changes.
+`--video` / `--image` and `--rig` combine freely (not `--video` with `--image`).
+Extraction is the slow step; reuse it with `--skeleton` when only the rig changes.
## Outputs
-Each run writes `outputs/<YYYYMMDD_HHMMSS>_<video>/`:
+Each run writes `outputs/<YYYYMMDD_HHMMSS>_<source>/`:
- `run.json` — inputs, timings, output listing
- `skeleton_motion.npz` — 3D human skeleton (24 SMPL joints, Y-up meters)
- `mixamo_rotations.npz` — per-bone animation rotations (quaternion wxyz; convention string inside)
- `mixamo_character.glb` — skinned character + clip (glTF 2.0; Blender: File → Import → glTF 2.0)
-- `videos/human_skeleton.mp4`, `videos/mixamo_skeleton.mp4`, `videos/mixamo_character.mp4`, `videos/compare.mp4` (2x2: original / mixamo skeleton / human skeleton / character)
+- Video in: `videos/human_skeleton.mp4`, `videos/mixamo_skeleton.mp4`, `videos/mixamo_character.mp4`, `videos/compare.mp4` (2x2: original / mixamo skeleton / human skeleton / character)
+- Photo in: the same four views as PNG under `images/`, plus `images/before_after_360_compare.mp4` (left original / right 10° orbit of the rig)
## Code map
- `motion2mixamorig/paths.py` — all filesystem locations
-- `motion2mixamorig/extract.py` — video -> 3D human skeleton (GVHMR)
+- `motion2mixamorig/extract.py` — video or still -> 3D human skeleton (GVHMR)
- `motion2mixamorig/mixamo/` — FBX parsing, FK, retargeting, skinning, rendering
- `motion2mixamorig/pipeline.py` — one run end to end
- `motion2mixamorig/cli.py` — `m2mr doctor` / `m2mr run`
diff --git a/README.ar.md b/README.ar.md
index ed76b60..8cd780b 100644
--- a/README.ar.md
+++ b/README.ar.md
@@ -6,9 +6,9 @@
<a href="README.zh-CN.md">中文</a> · <a href="README.md">English</a> · <a href="README.ko.md">한국어</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.ru.md">Русский</a> · العربية
</p>
-ينقل حركة شخص في مقطع فيديو إلى شخصية ثلاثية الأبعاد بهيكل تحريك Adobe Mixamo.
+ينقل حركة شخص في مقطع فيديو — أو الوضعية في صورة ثابتة — إلى شخصية ثلاثية الأبعاد بهيكل تحريك Adobe Mixamo.
-للمطوّرين في مجال الألعاب: قدّم فيديو حركة لـ**شخص واحد** وشخصية Mixamo؛ يكتب مقاطع معاينة وملف شخصية مكسوّة (`.glb`) يمكن وضعه في Blender أو Unity.
+للمطوّرين في مجال الألعاب: قدّم فيديو حركة لـ**شخص واحد** (أو صورة) وشخصية Mixamo؛ يكتب معاينات وملف شخصية مكسوّة (`.glb`) يمكن وضعه في Blender أو Unity.
ينبغي للوكلاء البرمجيين (Agents) البدء من [`AGENTS.md`](AGENTS.md).
@@ -46,25 +46,26 @@ pip install --upgrade pip setuptools wheel \
يُفضَّل تثبيت ffmpeg (macOS: `brew install ffmpeg`، Ubuntu: `apt install ffmpeg`).
يعمل مسار المعالجة بدونه، لكن مقاطع الإخراج ستكون **صامتة** ولن تُشغَّل في المتصفح.
-## قبل التشغيل: ضع ثلاثة أشياء في `assets/`
+## قبل التشغيل: ضع هذه في `assets/`
-هذا المستودع لا يتضمّن هذه الملفات؛ ينبغي جلبها بنفسك. يجب أن يطابق اسم ملف SMPL-X الجدول حرفيًا؛ أما أسماء FBX والفيديو فحرة.
+هذا المستودع لا يتضمّن هذه الملفات؛ ينبغي جلبها بنفسك. يجب أن يطابق اسم ملف SMPL-X الجدول حرفيًا؛ أما أسماء FBX والفيديو والصورة فحرة. يحتاج التشغيل إلى SMPL-X وشخصية Mixamo و**إما** فيديو **أو** صورة.
| ماذا | أين | من أين |
|---|---|---|
| نموذج جسم SMPL-X | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | سجّل في [SMPL-X](https://smpl-x.is.tue.mpg.de/) ثم نزّله |
| ملف FBX لشخصية Mixamo | `assets/mixamo/Y_Bot.fbx` | بحساب Adobe نزّل Y Bot (أو أي شخصية بهيكل Mixamo) من [Mixamo](https://www.mixamo.com) |
| فيديو الحركة | `assets/video/<your_clip>.mp4` | مقطع فيه **شخص واحد بالضبط** ظاهر بوضوح، **من الرأس إلى القدمين داخل الإطار طوال المقطع** (يمكن وضع عدة ملفات؛ كل تشغيل يستخدم آخر ملف أُضيف) |
+| صورة وضعية (اختياري) | `assets/image/<your_photo>.jpg` | صورة فيها **شخص واحد بالضبط** ظاهر بوضوح، **من الرأس إلى القدمين داخل الإطار**. مع `--image` |
ملاحظة: ملف التنزيل من Mixamo اسمه `Y Bot.fbx` (**مسافة**)، ويختلف بحرف واحد عن `Y_Bot.fbx` (**شرطة سفلية**) في الجدول.
أعد تسميته بالشرطة السفلية فيصبح الهيكل الافتراضي (`m2mr run` دون `--rig`). الإبقاء على المسافة جائز أيضًا — إن غاب `Y_Bot.fbx` يُستخدم أول `.fbx` في `assets/mixamo/`.
-يمكن وضع عدة فيديوهات في `assets/video/`. دون `--video` يستخدم التشغيل **آخر ملف أُضيف إلى هذا المجلد**.
-يجب أن يظهر في كل مقطع شخص واحد فقط، **من الرأس إلى القدمين داخل الإطار طوال المقطع** — دون قص عند الحواف. يأخذ `m2mr doctor` / `m2mr run` عيّنات من الإطارات ويتوقفان قبل الاستخراج إن ظهر شخصان.
+يمكن وضع عدة فيديوهات في `assets/video/`. دون `--video` أو `--image` يستخدم التشغيل **آخر ملف أُضيف إلى `assets/video/`**؛ إن كان المجلد فارغًا يعود إلى أحدث صورة في `assets/image/`.
+يجب أن يظهر في كل مقطع أو صورة شخص واحد فقط، **من الرأس إلى القدمين داخل الإطار** — دون قص عند الحواف. يتوقف `m2mr doctor` / `m2mr run` قبل الاستخراج إن ظهر شخصان.
## البدء السريع
-النتائج في `outputs/<وقت-التشغيل>_<الفيديو>/`. افتح مجلد `videos/` لمشاهدتها.
+النتائج في `outputs/<وقت-التشغيل>_<المصدر>/`. تشغيل الفيديو يكتب `videos/`؛ تشغيل الصورة يكتب `images/`.
### 1. تحقق من الأصول والبيئة
@@ -103,6 +104,17 @@ m2mr run --skeleton outputs/<previous-run-dir>/skeleton_motion.npz --rig assets/
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
```
+### 5. صورة ثابتة (وضعية ساكنة)
+
+`--image` يستعيد **وضعية ثلاثية الأبعاد ساكنة** من صورة لشخص واحد، ثم ينقلها إلى شخصية Mixamo كما يفعل تشغيل الفيديو. المعاينات أربع صور في `images/` (نفس المناظر كتشغيل الفيديو) إضافة إلى مقطع المدار يمين/يسار `before_after_360_compare.mp4`. لا تُجمع مع `--video`.
+
+```bash
+m2mr run --image assets/image/pose.jpg
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
+```
+
+`--image` دون مسار يستخدم آخر ملف أُضيف إلى `assets/image/`.
+
## المخرجات
كل `m2mr run` ينشئ مجلدًا باسم **وقت بدء الأمر**:
@@ -113,11 +125,15 @@ outputs/20260829_193205_dance/
├── skeleton_motion.npz # هيكل إنسان ثلاثي الأبعاد (يُعاد استخدامه عند تبديل الهيكل؛ بلا إعادة استخراج)
├── mixamo_rotations.npz # دورانات عظام Mixamo لكل عظم
├── mixamo_character.glb # شخصية مكسوّة + تحريك، استيراد إلى Blender / Unity
-└── videos/ # نسبة العرض إلى الارتفاع نفسها كفيديو الإدخال
- ├── human_skeleton.mp4 # هيكل الإنسان
- ├── mixamo_skeleton.mp4 # هيكل Mixamo
- ├── mixamo_character.mp4 # شخصية Mixamo
+└── videos/ # إدخال فيديو: نسبة العرض إلى الارتفاع نفسها كالمقطع
+ ├── human_skeleton.mp4
+ ├── mixamo_skeleton.mp4
+ ├── mixamo_character.mp4
└── compare.mp4 # 2×2: الأصل (أعلى اليسار) / هيكل Mixamo (أعلى اليمين) / هيكل الإنسان (أسفل اليسار) / الشخصية (أسفل اليمين)
+
+# إدخال الصورة يكتب images/ بنفس المناظر الأربعة:
+# human_skeleton.png / mixamo_skeleton.png / mixamo_character.png / compare.png
+# إضافة إلى before_after_360_compare.mp4 (اليسار الأصل / اليمين مدار 10°)
```
افتح `mixamo_character.glb` في Blender: File → Import → glTF 2.0 (.glb/.gltf). إن وُجّهت الكاميرا إلى فراغ، حدّد الشخصية ثم View → Frame Selected. مجموعة الكرات على الرأس واليدين هي عرض للعظام وليست الشبكة: حدّد الهيكل (Armature) وأوقف Shapes تحت Armature → Viewport Display. اضبط إطار نهاية الخط الزمني على طول المقطع ثم شغّل. قارن مع `videos/mixamo_character.mp4` من التشغيل نفسه.
diff --git a/README.de.md b/README.de.md
index adc0d8a..c008a1c 100644
--- a/README.de.md
+++ b/README.de.md
@@ -6,9 +6,9 @@
<a href="README.zh-CN.md">中文</a> · <a href="README.md">English</a> · <a href="README.ko.md">한국어</a> · <a href="README.ja.md">日本語</a> · Deutsch · <a href="README.ru.md">Русский</a> · <a href="README.ar.md">العربية</a>
</p>
-Überträgt die Bewegung einer Person aus einem Video auf einen Adobe-Mixamo-geriggten 3D-Charakter.
+Überträgt die Bewegung einer Person aus einem Video — oder die Pose aus einem Foto — auf einen Adobe-Mixamo-geriggten 3D-Charakter.
-Für Spieleentwickler: ein Video mit **genau einer Person** und ein Mixamo-Charakter; Ausgabe sind Vorschauvideos plus eine Charakterdatei mit Skinning (`.glb`) für Blender oder Unity.
+Für Spieleentwickler: ein Video mit **genau einer Person** (oder ein Still) und ein Mixamo-Charakter; Ausgabe sind Vorschauen plus eine Charakterdatei mit Skinning (`.glb`) für Blender oder Unity.
Agents beginnen mit [`AGENTS.md`](AGENTS.md).
@@ -46,25 +46,26 @@ Inferenzgewichte (GVHMR usw.) werden hier nicht geladen. Der erste `m2mr run` zi
ffmpeg wird empfohlen (macOS: `brew install ffmpeg`, Ubuntu: `apt install ffmpeg`).
Ohne ffmpeg läuft die Pipeline, aber die Ausgabevideos sind **stumm** und spielen im Browser nicht.
-## Vor dem Lauf: drei Dinge nach `assets/`
+## Vor dem Lauf: dies nach `assets/` legen
-Dieses Repository enthält diese Dateien nicht; sie müssen selbst besorgt werden. Der SMPL-X-Dateiname muss exakt zur Tabelle passen; FBX- und Videonamen sind frei.
+Dieses Repository enthält diese Dateien nicht; sie müssen selbst besorgt werden. Der SMPL-X-Dateiname muss exakt zur Tabelle passen; FBX-, Video- und Bildnamen sind frei. Ein Lauf braucht SMPL-X, einen Mixamo-Charakter und **entweder** ein Video **oder** ein Still.
| Was | Wohin | Woher |
|---|---|---|
| SMPL-X-Körpermodell | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | Bei [SMPL-X](https://smpl-x.is.tue.mpg.de/) registrieren und herunterladen |
| Mixamo-Charakter-FBX | `assets/mixamo/Y_Bot.fbx` | Mit Adobe-Konto Y Bot (oder einen anderen Mixamo-geriggten Charakter) von [Mixamo](https://www.mixamo.com) laden |
| Bewegungsvideo | `assets/video/<your_clip>.mp4` | Clip mit **genau einer** klar sichtbaren Person, **Kopf bis Fuß die ganze Zeit im Bild** (mehrere Dateien möglich; jeder Lauf nimmt die zuletzt hinzugefügte) |
+| Pose-Foto (optional) | `assets/image/<your_photo>.jpg` | Foto mit **genau einer** klar sichtbaren Person, **Kopf bis Fuß im Bild**. Mit `--image` |
Hinweis: Der Download von Mixamo heißt `Y Bot.fbx` (**Leerzeichen**) und weicht um ein Zeichen von `Y_Bot.fbx` (**Unterstrich**) in der Tabelle ab.
Umbenennen zum Unterstrich macht es zum Standard-Rig (`m2mr run` ohne `--rig`). Der Name mit Leerzeichen ist ebenfalls in Ordnung — fehlt `Y_Bot.fbx`, wird die erste `.fbx` in `assets/mixamo/` verwendet.
-In `assets/video/` können mehrere Videos liegen. Ohne `--video` nimmt der Lauf die **zuletzt in diesen Ordner gelegte** Datei.
-Jeder Clip darf nur eine Person zeigen, **Kopf bis Fuß die ganze Zeit im Bild** — nichts am Rand abgeschnitten. `m2mr doctor` / `m2mr run` prüfen Frames stichprobenartig und stoppen vor der Extraktion, wenn zwei Personen zu sehen sind.
+In `assets/video/` können mehrere Videos liegen. Ohne `--video` oder `--image` nimmt der Lauf die **zuletzt in `assets/video/` gelegte** Datei; ist der Ordner leer, das neueste Still in `assets/image/`.
+Jeder Clip oder jedes Foto darf nur eine Person zeigen, **Kopf bis Fuß im Bild** — nichts am Rand abgeschnitten. `m2mr doctor` / `m2mr run` stoppen vor der Extraktion, wenn zwei Personen zu sehen sind.
## Schnellstart
-Ergebnisse liegen in `outputs/<Laufzeit>_<Video>/`. Videos im Unterordner `videos/` ansehen.
+Ergebnisse liegen in `outputs/<Laufzeit>_<Quelle>/`. Videoläufe schreiben `videos/`; Fotoläufe schreiben `images/`.
### 1. Assets und Umgebung prüfen
@@ -103,6 +104,17 @@ m2mr run --skeleton outputs/<previous-run-dir>/skeleton_motion.npz --rig assets/
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
```
+### 5. Ein Still (statische Pose)
+
+`--image` rekonstruiert eine **statische 3D-Pose** aus einem Foto mit einer Person und überträgt sie wie ein Videolauf auf den Mixamo-Charakter. Vorschauen sind vier Stills in `images/` (gleiche Ansichten wie der Videolauf) plus der Links/Rechts-Orbit-Clip `before_after_360_compare.mp4`. Nicht zusammen mit `--video` verwenden.
+
+```bash
+m2mr run --image assets/image/pose.jpg
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
+```
+
+`--image` ohne Pfad nimmt die zuletzt nach `assets/image/` gelegte Datei.
+
## Ausgaben
Jeder `m2mr run` legt ein Verzeichnis an, benannt nach **Startzeit des Befehls**:
@@ -113,11 +125,15 @@ outputs/20260829_193205_dance/
├── skeleton_motion.npz # 3D-Körperskelett (beim Rig-Wechsel wiederverwenden; keine Neu-Extraktion)
├── mixamo_rotations.npz # Mixamo-Rotationen pro Bone
├── mixamo_character.glb # Charakterdatei mit Skinning + Animation, Import in Blender / Unity
-└── videos/ # gleiches Seitenverhältnis wie das Eingangsvideo
- ├── human_skeleton.mp4 # Körperskelett
- ├── mixamo_skeleton.mp4 # Mixamo-Rig-Skelett
- ├── mixamo_character.mp4 # Mixamo-Rig-Charakter
+└── videos/ # Videoeingabe: gleiches Seitenverhältnis wie der Clip
+ ├── human_skeleton.mp4
+ ├── mixamo_skeleton.mp4
+ ├── mixamo_character.mp4
└── compare.mp4 # 2×2: Original (oben links) / Mixamo-Skelett (oben rechts) / Körperskelett (unten links) / Charakter (unten rechts)
+
+# Fotoeingabe schreibt images/ mit denselben vier Ansichten:
+# human_skeleton.png / mixamo_skeleton.png / mixamo_character.png / compare.png
+# plus before_after_360_compare.mp4 (links Original / rechts 10°-Orbit)
```
`mixamo_character.glb` in Blender öffnen: File → Import → glTF 2.0 (.glb/.gltf). Zeigt die Kamera ins Leere, den Charakter wählen und View → Frame Selected. Die Kugeln an Kopf und Händen sind die Knochenanzeige, nicht das Mesh: Armature wählen und unter Armature → Viewport Display die Shapes ausschalten. Das letzte Timeline-Frame auf die Cliplänge setzen, dann abspielen. Mit `videos/mixamo_character.mp4` desselben Laufs vergleichen.
diff --git a/README.ja.md b/README.ja.md
index 90ce7b4..8f1bfd0 100644
--- a/README.ja.md
+++ b/README.ja.md
@@ -6,9 +6,9 @@
<a href="README.zh-CN.md">中文</a> · <a href="README.md">English</a> · <a href="README.ko.md">한국어</a> · 日本語 · <a href="README.de.md">Deutsch</a> · <a href="README.ru.md">Русский</a> · <a href="README.ar.md">العربية</a>
</p>
-動画の人物の動きを、Adobe Mixamo rig の 3D キャラクターに移します。
+動画の人物の動き — または静止画のポーズ — を、Adobe Mixamo rig の 3D キャラクターに移します。
-ゲーム開発者向け:入力は**一人**の動作動画と Mixamo キャラクター。プレビュー動画と、Blender / Unity にそのまま入れられるスキン付きキャラクターファイル(`.glb`)を書き出します。
+ゲーム開発者向け:入力は**一人**の動作動画(または写真)と Mixamo キャラクター。プレビューと、Blender / Unity にそのまま入れられるスキン付きキャラクターファイル(`.glb`)を書き出します。
Agent は先に [`AGENTS.md`](AGENTS.md) を読んでください。
@@ -46,25 +46,26 @@ pip install --upgrade pip setuptools wheel \
ffmpeg も推奨です(macOS: `brew install ffmpeg`、Ubuntu: `apt install ffmpeg`)。
なくても動きますが、出力動画は**無音**になり、ブラウザでは再生できません。
-## 実行前:`assets/` に 3 つのファイルを入れてください
+## 実行前:これらを `assets/` に入れてください
-このリポジトリにはこれらのファイルは入っていません。自分で用意してください。SMPL-X のファイル名は下表と一字一句同じである必要があります。FBX と動画の名前は何でも構いません。
+このリポジトリにはこれらのファイルは入っていません。自分で用意してください。SMPL-X のファイル名は下表と一字一句同じである必要があります。FBX、動画、写真の名前は何でも構いません。1 回の実行には SMPL-X、Mixamo キャラクター、そして**動画か写真のどちらか**が必要です。
| 何を | どこへ | どこから |
|---|---|---|
| SMPL-X ボディモデル | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | [SMPL-X](https://smpl-x.is.tue.mpg.de/) で登録してダウンロード |
| Mixamo キャラクター FBX | `assets/mixamo/Y_Bot.fbx` | Adobe アカウントで [Mixamo](https://www.mixamo.com) から Y Bot(または他の Mixamo rig キャラクター)をダウンロード |
| 動作動画 | `assets/video/<your_clip>.mp4` | **一人だけ**、はっきり見え、**頭から足まで全編フレーム内**(複数可、実行ごとに最後に入れたファイルを使用) |
+| ポーズ写真(任意) | `assets/image/<your_photo>.jpg` | **一人だけ**、はっきり見え、**頭から足までフレーム内**。`--image` と一緒に使う |
注意:Mixamo のダウンロード名は `Y Bot.fbx`(**スペース**)で、表の `Y_Bot.fbx`(**アンダースコア**)と一字違います。
アンダースコアにすればデフォルト rig になり、`m2mr run` は `--rig` なしでそれを使います。名前を変えなくても構いません——`Y_Bot.fbx` が無いときは `assets/mixamo/` の最初の `.fbx` を使います。
-`assets/video/` には動画を複数置けます。`--video` を指定しないと、**このフォルダに最後に入れた**ファイルを使います。
-各クリップは一人だけ、**頭から足まで全編フレーム内**に収まっている必要があります。はみ出してはいけません。`m2mr doctor` / `m2mr run` はフレームをサンプリングし、二人が写っていれば抽出に入らず止まります。
+`assets/video/` には動画を複数置けます。`--video` も `--image` も指定しないと、**`assets/video/` に最後に入れた**ファイルを使い、そのフォルダが空なら `assets/image/` の最新の写真に戻ります。
+各クリップや写真は一人だけ、**頭から足までフレーム内**に収まっている必要があります。はみ出してはいけません。`m2mr doctor` / `m2mr run` は二人が写っていれば抽出に入らず止まります。
## クイックスタート
-結果は `outputs/<実行時刻>_<動画名>/` です。動画を見るにはその中の `videos/` を開きます。
+結果は `outputs/<実行時刻>_<入力名>/` です。動画入力は `videos/`、写真入力は `images/` に書き出します。
### 1. アセットと環境を確認
@@ -103,6 +104,17 @@ m2mr run --skeleton outputs/<previous-run-dir>/skeleton_motion.npz --rig assets/
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
```
+### 5. 静止画(静的ポーズ)
+
+`--image` は**一人**の写真から**静的な 3D ポーズ**を復元し、動画実行と同じ経路で Mixamo キャラクターに載せます。プレビューは `images/` の 4 枚(動画実行と同じ視点)と、左右比較の周回クリップ `before_after_360_compare.mp4` です。`--video` と同時に指定しないでください。
+
+```bash
+m2mr run --image assets/image/pose.jpg
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
+```
+
+パスなしの `--image` は、`assets/image/` に最後に入れたファイルを使います。
+
## 出力
各 `m2mr run` は**コマンド開始時刻**でディレクトリを作ります:
@@ -113,11 +125,15 @@ outputs/20260829_193205_dance/
├── skeleton_motion.npz # 人体 3D スケルトン(rig 差し替え時に再利用、再抽出不要)
├── mixamo_rotations.npz # Mixamo ボーンごとの回転
├── mixamo_character.glb # スキン付きキャラクター + アニメーション、Blender / Unity に直接インポート
-└── videos/ # 入力動画と同じアスペクト比
- ├── human_skeleton.mp4 # 人体スケルトン
- ├── mixamo_skeleton.mp4 # Mixamo rig スケルトン
- ├── mixamo_character.mp4 # Mixamo rig キャラクター
+└── videos/ # 動画入力:クリップと同じアスペクト比
+ ├── human_skeleton.mp4
+ ├── mixamo_skeleton.mp4
+ ├── mixamo_character.mp4
└── compare.mp4 # 2×2:元映像(左上)/ Mixamo スケルトン(右上)/ 人体スケルトン(左下)/ キャラクター(右下)
+
+# 写真入力は images/ に同じ 4 ビューを書き出します:
+# human_skeleton.png / mixamo_skeleton.png / mixamo_character.png / compare.png
+# 加えて before_after_360_compare.mp4(左が原画 / 右が 10° 周回)
```
`mixamo_character.glb` は Blender で File → Import → glTF 2.0 (.glb/.gltf) で開きます。カメラが空を向いているときはキャラクターを選び、View → Frame Selected。頭と手の球の塊はメッシュではなくボーン表示です:アーマチュアを選び、Armature → Viewport Display で Shapes をオフにします。タイムラインの終了フレームをクリップ長に合わせて再生。同じ実行の `videos/mixamo_character.mp4` と見比べてください。
diff --git a/README.ko.md b/README.ko.md
index 1eb8a52..10d87aa 100644
--- a/README.ko.md
+++ b/README.ko.md
@@ -6,9 +6,9 @@
<a href="README.zh-CN.md">中文</a> · <a href="README.md">English</a> · 한국어 · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.ru.md">Русский</a> · <a href="README.ar.md">العربية</a>
</p>
-영상 속 사람의 동작을 Adobe Mixamo rig 3D 캐릭터로 옮깁니다.
+영상 속 사람의 동작 — 또는 사진 속 자세 — 를 Adobe Mixamo rig 3D 캐릭터로 옮깁니다.
-게임 개발자를 위한 도구입니다. **한 사람만** 나오는 동작 영상과 Mixamo 캐릭터를 넣으면 미리보기 영상과, Blender / Unity에 바로 넣을 수 있는 스키닝된 캐릭터 파일(`.glb`)을 생성합니다.
+게임 개발자를 위한 도구입니다. **한 사람만** 나오는 동작 영상(또는 사진)과 Mixamo 캐릭터를 넣으면 미리보기와, Blender / Unity에 바로 넣을 수 있는 스키닝된 캐릭터 파일(`.glb`)을 생성합니다.
Agent는 먼저 [`AGENTS.md`](AGENTS.md)를 읽어 주세요.
@@ -46,25 +46,26 @@ pip install --upgrade pip setuptools wheel \
ffmpeg도 권장합니다(macOS: `brew install ffmpeg`, Ubuntu: `apt install ffmpeg`).
없어도 실행되지만, 출력 영상에 **소리가 없고** 브라우저에서 바로 재생되지 않습니다.
-## 실행 전: `assets/`에 세 가지를 넣으세요
+## 실행 전: 이것을 `assets/`에 넣으세요
-이 저장소는 해당 파일을 포함하지 않습니다. 직접 준비해야 합니다. SMPL-X 파일명은 아래 표와 한 글자도 같아야 합니다. FBX와 영상 이름은 상관없습니다.
+이 저장소는 해당 파일을 포함하지 않습니다. 직접 준비해야 합니다. SMPL-X 파일명은 아래 표와 한 글자도 같아야 합니다. FBX, 영상, 사진 이름은 상관없습니다. 한 번의 실행에는 SMPL-X, Mixamo 캐릭터, 그리고 **영상 또는 사진 중 하나**가 필요합니다.
| 무엇 | 어디에 | 어디서 |
|---|---|---|
| SMPL-X 바디 모델 | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | [SMPL-X](https://smpl-x.is.tue.mpg.de/)에서 등록한 뒤 다운로드 |
| Mixamo 캐릭터 FBX | `assets/mixamo/Y_Bot.fbx` | Adobe 계정으로 [Mixamo](https://www.mixamo.com)에서 Y Bot(또는 다른 Mixamo rig 캐릭터) 다운로드 |
| 동작 영상 | `assets/video/<your_clip>.mp4` | **사람 한 명만**, 잘 보이고, **머리부터 발끝까지 영상 내내 화면 안에** (여러 파일 가능, 실행마다 가장 최근에 넣은 것 사용) |
+| 자세 사진(선택) | `assets/image/<your_photo>.jpg` | **사람 한 명만**, 잘 보이고, **머리부터 발끝까지 화면 안에**. `--image`와 함께 사용 |
주의: Mixamo에서 받은 파일은 `Y Bot.fbx`(**공백**)이고, 표의 `Y_Bot.fbx`(**밑줄**)과 한 글자 다릅니다.
밑줄로 바꾸면 기본 rig가 되어 `m2mr run`에 `--rig` 없이 씁니다. 이름을 안 바꿔도 됩니다 — `Y_Bot.fbx`가 없으면 `assets/mixamo/`의 첫 `.fbx`를 씁니다.
-`assets/video/`에는 영상을 여러 개 넣을 수 있습니다. `--video`를 지정하지 않으면 **이 폴더에 가장 마지막에 넣은** 파일을 씁니다.
-각 영상에는 사람이 한 명만 나와야 하고, **머리부터 발끝까지 영상 내내 화면 안**에 있어야 하며 화면 밖으로 나가면 안 됩니다. `m2mr doctor` / `m2mr run`이 프레임을 샘플링해 두 명이 보이면 추출 전에 멈춥니다.
+`assets/video/`에는 영상을 여러 개 넣을 수 있습니다. `--video`나 `--image`를 지정하지 않으면 **`assets/video/`에 가장 마지막에 넣은** 파일을 쓰고, 그 폴더가 비어 있으면 `assets/image/`의 최신 사진을 씁니다.
+각 영상이나 사진에는 사람이 한 명만 나와야 하고, **머리부터 발끝까지 화면 안**에 있어야 하며 화면 밖으로 나가면 안 됩니다. `m2mr doctor` / `m2mr run`이 두 명이 보이면 추출 전에 멈춥니다.
## 빠른 시작
-결과는 `outputs/<실행시각>_<영상명>/`에 있습니다. 영상은 그 안의 `videos/`를 여세요.
+결과는 `outputs/<실행시각>_<입력명>/`에 있습니다. 영상 입력은 `videos/`, 사진 입력은 `images/`에 씁니다.
### 1. 자산과 환경 확인
@@ -103,6 +104,17 @@ m2mr run --skeleton outputs/<previous-run-dir>/skeleton_motion.npz --rig assets/
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
```
+### 5. 사진(정적 자세)
+
+`--image`는 **한 사람** 사진에서 **정적 3D 자세**를 복원한 뒤, 영상 실행과 같은 경로로 Mixamo 캐릭터에 옮깁니다. 미리보기는 `images/`의 네 장(영상 실행과 같은 뷰)과 좌우 비교 궤도 클립 `before_after_360_compare.mp4`입니다. `--video`와 함께 쓰지 마세요.
+
+```bash
+m2mr run --image assets/image/pose.jpg
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
+```
+
+경로 없이 `--image`만 쓰면 `assets/image/`에 가장 최근에 넣은 파일을 씁니다.
+
## 출력
각 `m2mr run`은 **명령이 시작된 시각**으로 디렉터리를 만듭니다:
@@ -113,11 +125,15 @@ outputs/20260829_193205_dance/
├── skeleton_motion.npz # 인체 3D 스켈레톤 (rig 교체 시 재사용, 추출 생략)
├── mixamo_rotations.npz # Mixamo 본별 회전
├── mixamo_character.glb # 스키닝된 캐릭터 + 애니메이션, Blender / Unity에 바로 임포트
-└── videos/ # 입력 영상과 같은 화면비
- ├── human_skeleton.mp4 # 인체 스켈레톤
- ├── mixamo_skeleton.mp4 # Mixamo rig 스켈레톤
- ├── mixamo_character.mp4 # Mixamo rig 캐릭터
+└── videos/ # 영상 입력: 클립과 같은 화면비
+ ├── human_skeleton.mp4
+ ├── mixamo_skeleton.mp4
+ ├── mixamo_character.mp4
└── compare.mp4 # 2×2: 원본(좌상) / Mixamo 스켈레톤(우상) / 인체 스켈레톤(좌하) / 캐릭터(우하)
+
+# 사진 입력은 images/에 같은 네 뷰를 씁니다:
+# human_skeleton.png / mixamo_skeleton.png / mixamo_character.png / compare.png
+# 그리고 before_after_360_compare.mp4 (왼쪽 원본 / 오른쪽 10° 궤도)
```
`mixamo_character.glb`는 Blender에서 File → Import → glTF 2.0 (.glb/.gltf)로 엽니다. 카메라가 빈 곳을 보고 있으면 캐릭터를 선택한 뒤 View → Frame Selected. 머리와 손의 구체들은 메시가 아니라 본 표시입니다. Armature를 선택하고 Armature → Viewport Display에서 Shapes를 끄세요. 타임라인 끝 프레임을 클립 길이에 맞춘 뒤 재생하세요. 같은 실행의 `videos/mixamo_character.mp4`와 비교하면 됩니다.
diff --git a/README.md b/README.md
index c1e09a0..8f76537 100644
--- a/README.md
+++ b/README.md
@@ -6,9 +6,9 @@
<a href="README.zh-CN.md">中文</a> · English · <a href="README.ko.md">한국어</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.ru.md">Русский</a> · <a href="README.ar.md">العربية</a>
</p>
-Transfer the motion of a person in a video onto an Adobe Mixamo-rigged 3D character.
+Transfer the motion of a person in a video — or the pose in a still photo — onto an Adobe Mixamo-rigged 3D character.
-For game developers: give it a **single-person** action video and a Mixamo character; it writes preview videos plus a skinned character file (`.glb`) you can drop into Blender or Unity.
+For game developers: give it a **single-person** action video (or a still) and a Mixamo character; it writes preview videos plus a skinned character file (`.glb`) you can drop into Blender or Unity.
Agents should start with [`AGENTS.md`](AGENTS.md).
@@ -46,25 +46,26 @@ Inference weights (GVHMR and so on) are not downloaded here. The first `m2mr run
ffmpeg is recommended (macOS: `brew install ffmpeg`, Ubuntu: `apt install ffmpeg`).
The pipeline runs without it, but the output videos will be **silent** and will not play in a browser.
-## Before you run: put three things in `assets/`
+## Before you run: put these in `assets/`
-This repository does not ship these files; you have to fetch them. The SMPL-X filename must match the table exactly; FBX and video names can be anything.
+This repository does not ship these files; you have to fetch them. The SMPL-X filename must match the table exactly; FBX, video, and image names can be anything. A run needs SMPL-X, a Mixamo character, and **either** a video **or** a still.
| What | Where | Where to get it |
|---|---|---|
| SMPL-X body model | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | Register at [SMPL-X](https://smpl-x.is.tue.mpg.de/) and download it |
| Mixamo character FBX | `assets/mixamo/Y_Bot.fbx` | With an Adobe account, download Y Bot (or any other Mixamo-rigged character) from [Mixamo](https://www.mixamo.com) |
| Action video | `assets/video/<your_clip>.mp4` | A clip with **exactly one** clearly visible person, **head to toe in frame the whole time** (you may keep several files; each run uses the most recently added) |
+| Pose still (optional) | `assets/image/<your_photo>.jpg` | A photo with **exactly one** clearly visible person, **head to toe in frame**. Use with `--image` |
Note: Mixamo's download is named `Y Bot.fbx` (**space**), one character off from `Y_Bot.fbx` (**underscore**) in the table.
Rename it to the underscore form and it becomes the default rig (`m2mr run` without `--rig`). Leaving the space is fine too — if `Y_Bot.fbx` is missing, the first `.fbx` in `assets/mixamo/` is used.
-`assets/video/` can hold several videos. Without `--video`, the run uses the file **most recently added** to that folder.
-Each clip must show only one person, **head to toe in frame for the whole clip** — nothing cropped at the edges. `m2mr doctor` / `m2mr run` sample the frames and stop before extraction if two people are in view.
+`assets/video/` can hold several videos. Without `--video` or `--image`, the run uses the file **most recently added** to `assets/video/`; if that folder is empty it falls back to the latest still in `assets/image/`.
+Each clip or photo must show only one person, **head to toe in frame** — nothing cropped at the edges. `m2mr doctor` / `m2mr run` stop before extraction if two people are in view.
## Quick Start
-Results land in `outputs/<run-time>_<video>/`. Open that folder's `videos/` to watch them.
+Results land in `outputs/<run-time>_<source>/`. Video runs write `videos/`; still runs write `images/`.
### 1. Check assets and the environment
@@ -103,6 +104,17 @@ m2mr run --skeleton outputs/<previous-run-dir>/skeleton_motion.npz --rig assets/
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
```
+### 5. A still photo (static pose)
+
+`--image` recovers a **static 3D pose** from a photo of one person, then retargets it onto the Mixamo character the same way a video run does. Previews are four stills in `images/` (same views as the video run), plus a left/right orbit clip `before_after_360_compare.mp4`. Do not pass `--video` at the same time.
+
+```bash
+m2mr run --image assets/image/pose.jpg
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
+```
+
+`--image` with no path uses the file most recently added to `assets/image/`.
+
## Outputs
Each `m2mr run` creates a directory named by **the time the command started**:
@@ -113,11 +125,15 @@ outputs/20260829_193205_dance/
├── skeleton_motion.npz # 3D human skeleton (reuse when swapping rigs; no re-extract)
├── mixamo_rotations.npz # per-bone Mixamo rotations
├── mixamo_character.glb # skinned character + animation, import into Blender / Unity
-└── videos/ # same aspect ratio as the input
- ├── human_skeleton.mp4 # human skeleton
- ├── mixamo_skeleton.mp4 # Mixamo rig skeleton
- ├── mixamo_character.mp4 # Mixamo rig character
+└── videos/ # video input: same aspect ratio as the clip
+ ├── human_skeleton.mp4
+ ├── mixamo_skeleton.mp4
+ ├── mixamo_character.mp4
└── compare.mp4 # 2×2: original (TL) / Mixamo skeleton (TR) / human skeleton (BL) / character (BR)
+
+# still input writes images/ instead, same four views:
+# human_skeleton.png / mixamo_skeleton.png / mixamo_character.png / compare.png
+# plus before_after_360_compare.mp4 (left original / right 10° orbit of the rig)
```
Open `mixamo_character.glb` in Blender: File → Import → glTF 2.0 (.glb/.gltf). If the camera is framed on empty space, select the character and use View → Frame Selected. The cluster of spheres on the head and hands is the bone display, not the mesh: select the armature and turn off Shapes under Armature → Viewport Display. Set the timeline end frame to the clip length, then play. Compare against `videos/mixamo_character.mp4` from the same run.
diff --git a/README.ru.md b/README.ru.md
index 84760f9..6d536dd 100644
--- a/README.ru.md
+++ b/README.ru.md
@@ -6,9 +6,9 @@
<a href="README.zh-CN.md">中文</a> · <a href="README.md">English</a> · <a href="README.ko.md">한국어</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · Русский · <a href="README.ar.md">العربية</a>
</p>
-Переносит движение человека из видео на 3D-персонажа с ригом Adobe Mixamo.
+Переносит движение человека из видео — или позу с фото — на 3D-персонажа с ригом Adobe Mixamo.
-Для разработчиков игр: видео с **одним человеком** и персонаж Mixamo; на выходе превью-видео и файл персонажа со скином (`.glb`) для Blender или Unity.
+Для разработчиков игр: видео с **одним человеком** (или фото) и персонаж Mixamo; на выходе превью и файл персонажа со скином (`.glb`) для Blender или Unity.
Агентам следует начать с [`AGENTS.md`](AGENTS.md).
@@ -46,25 +46,26 @@ pip install --upgrade pip setuptools wheel \
Рекомендуется ffmpeg (macOS: `brew install ffmpeg`, Ubuntu: `apt install ffmpeg`).
Без него пайплайн работает, но выходные видео будут **без звука** и не откроются в браузере.
-## Перед запуском: положите три вещи в `assets/`
+## Перед запуском: положите это в `assets/`
-Репозиторий эти файлы не содержит — их нужно получить самостоятельно. Имя файла SMPL-X должно совпадать с таблицей буква в букву; имена FBX и видео любые.
+Репозиторий эти файлы не содержит — их нужно получить самостоятельно. Имя файла SMPL-X должно совпадать с таблицей буква в букву; имена FBX, видео и фото любые. Для запуска нужны SMPL-X, персонаж Mixamo и **либо** видео, **либо** фото.
| Что | Куда | Откуда |
|---|---|---|
| Модель тела SMPL-X | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | Зарегистрироваться на [SMPL-X](https://smpl-x.is.tue.mpg.de/) и скачать |
| FBX-персонаж Mixamo | `assets/mixamo/Y_Bot.fbx` | С Adobe-аккаунтом скачать Y Bot (или другого персонажа с ригом Mixamo) с [Mixamo](https://www.mixamo.com) |
| Видео с действием | `assets/video/<your_clip>.mp4` | Ролик с **ровно одним** хорошо видимым человеком, **с головы до ног весь ролик в кадре** (можно несколько файлов; каждый запуск берёт последний добавленный) |
+| Фото позы (необязательно) | `assets/image/<your_photo>.jpg` | Фото с **ровно одним** хорошо видимым человеком, **с головы до ног в кадре**. Вместе с `--image` |
Замечание: скачанный файл Mixamo называется `Y Bot.fbx` (**пробел**), на один символ отличается от `Y_Bot.fbx` (**подчёркивание**) в таблице.
Переименуйте с подчёркиванием — это станет ригом по умолчанию (`m2mr run` без `--rig`). Можно не переименовывать: если `Y_Bot.fbx` нет, берётся первый `.fbx` в `assets/mixamo/`.
-В `assets/video/` можно класть несколько видео. Без `--video` запуск берёт файл, **последним положенный в эту папку**.
-В каждом ролике должен быть один человек, **с головы до ног весь ролик в кадре** — ничего не обрезано по краям. `m2mr doctor` / `m2mr run` выборочно проверяют кадры и останавливаются до извлечения, если в кадре двое.
+В `assets/video/` можно класть несколько видео. Без `--video` или `--image` запуск берёт файл, **последним положенный в `assets/video/`**; если папка пуста — последнее фото в `assets/image/`.
+В каждом ролике или фото должен быть один человек, **с головы до ног в кадре** — ничего не обрезано по краям. `m2mr doctor` / `m2mr run` останавливаются до извлечения, если в кадре двое.
## Быстрый старт
-Результаты в `outputs/<время-запуска>_<видео>/`. Видео смотреть в `videos/`.
+Результаты в `outputs/<время-запуска>_<источник>/`. Видео-запуск пишет в `videos/`; фото-запуск — в `images/`.
### 1. Проверить ассеты и окружение
@@ -103,6 +104,17 @@ m2mr run --skeleton outputs/<previous-run-dir>/skeleton_motion.npz --rig assets/
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
```
+### 5. Фото (статическая поза)
+
+`--image` восстанавливает **статическую 3D-позу** с фото одного человека и переносит её на персонажа Mixamo так же, как видео-запуск. Превью — четыре кадра в `images/` (те же виды, что у видео) плюс клип орбиты слева/справа `before_after_360_compare.mp4`. Не сочетать с `--video`.
+
+```bash
+m2mr run --image assets/image/pose.jpg
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
+```
+
+`--image` без пути берёт файл, последним положенный в `assets/image/`.
+
## Результаты
Каждый `m2mr run` создаёт каталог по **времени старта команды**:
@@ -113,11 +125,15 @@ outputs/20260829_193205_dance/
├── skeleton_motion.npz # 3D-скелет человека (переиспользовать при смене рига; без повторного извлечения)
├── mixamo_rotations.npz # вращения костей Mixamo
├── mixamo_character.glb # персонаж со скином + анимация, импорт в Blender / Unity
-└── videos/ # то же соотношение сторон, что у входного видео
- ├── human_skeleton.mp4 # скелет человека
- ├── mixamo_skeleton.mp4 # скелет рига Mixamo
- ├── mixamo_character.mp4 # персонаж с ригом Mixamo
+└── videos/ # видео-вход: то же соотношение сторон, что у клипа
+ ├── human_skeleton.mp4
+ ├── mixamo_skeleton.mp4
+ ├── mixamo_character.mp4
└── compare.mp4 # 2×2: оригинал (верхний левый) / скелет Mixamo (верхний правый) / скелет человека (нижний левый) / персонаж (нижний правый)
+
+# фото-вход пишет images/ с теми же четырьмя видами:
+# human_skeleton.png / mixamo_skeleton.png / mixamo_character.png / compare.png
+# плюс before_after_360_compare.mp4 (слева оригинал / справа орбита 10°)
```
`mixamo_character.glb` в Blender: File → Import → glTF 2.0 (.glb/.gltf). Если камера смотрит в пустоту, выберите персонажа и View → Frame Selected. Скопление сфер на голове и руках — отображение костей, не меш: выберите арматуру и отключите Shapes в Armature → Viewport Display. Конечный кадр таймлайна поставьте по длине клипа, затем воспроизведите. Сверяйте с `videos/mixamo_character.mp4` того же запуска.
diff --git a/README.zh-CN.md b/README.zh-CN.md
index edf1cb3..7549727 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -6,9 +6,9 @@
中文 · <a href="README.md">English</a> · <a href="README.ko.md">한국어</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.ru.md">Русский</a> · <a href="README.ar.md">العربية</a>
</p>
-把视频里人物的动作,转到 Adobe Mixamo rig 的 3D 角色上。
+把视频里人物的动作,或一张照片里的姿态,转到 Adobe Mixamo rig 的 3D 角色上。
-面向游戏开发者:输入一段**单人**动作视频和一个 Mixamo 角色,输出预览视频,以及可直接丢进 Blender / Unity 的带皮角色文件(`.glb`)。
+面向游戏开发者:输入一段**单人**动作视频(或一张人物照片)和一个 Mixamo 角色,输出预览视频,以及可直接丢进 Blender / Unity 的带皮角色文件(`.glb`)。
Agent 请先读 [`AGENTS.md`](AGENTS.md)。
@@ -45,26 +45,27 @@ pip install --upgrade pip setuptools wheel \
建议再装上 ffmpeg(macOS:`brew install ffmpeg`,Ubuntu:`apt install ffmpeg`)。
没有它也能跑,但输出的视频会**没有声音**,编码格式也没法在浏览器里直接播放。
-## 使用前:把三样东西放到 `assets/`
+## 使用前:把这些东西放到 `assets/`
-本仓库不附带这些文件,需要你自己准备。SMPL-X 的文件名必须和下表一字不差;FBX 和视频叫什么都行。
+本仓库不附带这些文件,需要你自己准备。SMPL-X 的文件名必须和下表一字不差;FBX、视频和照片叫什么都行。一次运行需要 SMPL-X、一个 Mixamo 角色,以及**视频或照片二选一**。
| 放什么 | 放到哪 | 去哪拿 |
|---|---|---|
| SMPL-X 人体模型 | `assets/body_models/smplx/SMPLX_NEUTRAL.npz` | 注册 [SMPL-X](https://smpl-x.is.tue.mpg.de/) 后下载 |
| Mixamo 角色 FBX | `assets/mixamo/Y_Bot.fbx` | 用 Adobe 账号从 [Mixamo](https://www.mixamo.com) 下载 Y Bot(或其他 Mixamo rig 角色) |
| 动作视频 | `assets/video/<your_clip>.mp4` | **只有一个人**、人清晰可见、**从头到脚全程都在画框内**(可放多个文件,每次跑用最新的那一个) |
+| 姿态照片(可选) | `assets/image/<your_photo>.jpg` | **只有一个人**、人清晰可见、**从头到脚都在画框内**。配合 `--image` 使用 |
注意:Mixamo 下载下来的文件叫 `Y Bot.fbx`(中间是**空格**),和上表的 `Y_Bot.fbx`(**下划线**)差一个字符。
改成下划线它就是默认 rig,`m2mr run` 不带 `--rig` 时自动用它;不想改名也行——找不到 `Y_Bot.fbx` 时
会自动用 `assets/mixamo/` 里的第一个 `.fbx`。
-`assets/video/` 可以放多个视频文件。不指定 `--video` 时,会用**最后放进这个文件夹**的那一个。
-每个视频画面里只能有一个人,而且人物必须**从头到脚全程都在画框内**,不能出画。`m2mr doctor` / `m2mr run` 会抽样检测人数,看到两个人就直接停,不会进入提取。
+`assets/video/` 可以放多个视频文件。不指定 `--video` 或 `--image` 时,会用**最后放进 `assets/video/` 的视频**;这个文件夹是空的,就退回到 `assets/image/` 里最新的一张照片。
+每个视频或照片画面里只能有一个人,而且人物必须**从头到脚都在画框内**,不能出画。`m2mr doctor` / `m2mr run` 检测到两个人就直接停,不会进入提取。
## Quick Start
-结果都在 `outputs/<运行时间>_<视频名>/`,看视频打开该目录下的 `videos/`。
+结果都在 `outputs/<运行时间>_<输入名>/`。视频输入写 `videos/`,照片输入写 `images/`。
### 1. 检查资产和环境
@@ -103,6 +104,17 @@ m2mr run --skeleton outputs/<上次的运行目录>/skeleton_motion.npz --rig as
m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
```
+### 5. 用一张照片(静态姿态)
+
+`--image` 从**单人照片**恢复一个静态 3D 姿态,再按和视频相同的路径转到 Mixamo 角色上。预览是 `images/` 里的四张静帧(种类和视频 run 一样),外加左右对比绕转视频 `before_after_360_compare.mp4`。不要和 `--video` 一起用。
+
+```bash
+m2mr run --image assets/image/pose.jpg
+m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
+```
+
+`--image` 不带路径时,用 `assets/image/` 里最后放进去的那一张。
+
## 输出
每次 `m2mr run` 按**命令启动时间**新建一个目录:
@@ -113,11 +125,15 @@ outputs/20260829_193205_dance/
├── skeleton_motion.npz # 人体 3D 骨架(换 rig 可复用,不必重跑提取)
├── mixamo_rotations.npz # Mixamo 骨骼逐帧旋转
├── mixamo_character.glb # 带皮角色 + 动画,Blender / Unity 可直接导入
-└── videos/ # 比例跟输入视频一致
- ├── human_skeleton.mp4 # 人体骨架
- ├── mixamo_skeleton.mp4 # Mixamo rig 骨架
- ├── mixamo_character.mp4 # Mixamo rig 角色
+└── videos/ # 视频输入:比例跟输入视频一致
+ ├── human_skeleton.mp4
+ ├── mixamo_skeleton.mp4
+ ├── mixamo_character.mp4
└── compare.mp4 # 四宫格:左上原片 / 右上 Mixamo 骨架 / 左下人体骨架 / 右下 Mixamo 角色
+
+# 照片输入则写 images/,四种图和上面一一对应:
+# human_skeleton.png / mixamo_skeleton.png / mixamo_character.png / compare.png
+# 另外还有 before_after_360_compare.mp4(左原图 / 右 10° 俯视绕角色转一圈)
```
`mixamo_character.glb` 用 Blender 打开:File → Import → glTF 2.0 (.glb/.gltf)。导入后若镜头对不准,先选中角色,再 View → Frame Selected。头和手上那堆球是骨头的显示形状,不是模型:选中骨架,在 Armature → Viewport Display 里去掉 Shapes。时间轴结束帧改成和视频一样长,再点播放;对照同一次运行的 `videos/mixamo_character.mp4`。
diff --git a/assets/README.md b/assets/README.md
index 11d83d1..ed386a7 100644
--- a/assets/README.md
+++ b/assets/README.md
@@ -13,6 +13,7 @@ Run `m2mr doctor` at any time to check what is present and what is missing.
| `body_models/smplx/SMPLX_NEUTRAL.npz` | SMPL-X neutral body model, needed to lift the video into a 3D skeleton | Register at <https://smpl-x.is.tue.mpg.de/>, download the model archive, copy `SMPLX_NEUTRAL.npz` here |
| `mixamo/<Character>.fbx` | The character to drive, e.g. `Y_Bot.fbx` (the default) | <https://www.mixamo.com/> — pick a character, download as **FBX Binary**, **T-pose** |
| `video/<your_clip>.mp4` | The video whose human motion you want to transfer | One clearly visible person only (`.mp4`, `.mov`, …). Group shots are rejected before extraction |
+| `image/<your_photo>.jpg` | A still whose pose you want to transfer | One clearly visible person (`.jpg`, `.png`, …). Group photos are rejected before extraction |
`m2mr run` without flags uses the most recently added file in `video/` and
-`mixamo/Y_Bot.fbx`; use `--video` / `--rig` to pick explicitly.
+`mixamo/Y_Bot.fbx`; use `--video` / `--image` / `--rig` to pick explicitly.
diff --git a/assets/image/.gitkeep b/assets/image/.gitkeep
new file mode 100644
index 0000000..e69de29
diff --git a/motion2mixamorig/__init__.py b/motion2mixamorig/__init__.py
index c9f970f..823aa74 100644
--- a/motion2mixamorig/__init__.py
+++ b/motion2mixamorig/__init__.py
@@ -1 +1 @@
-"""Motion2MixamoRig: retarget human motion from video to a Mixamo-rigged 3D character."""
+"""Motion2MixamoRig: retarget human motion (or a still pose) to a Mixamo-rigged 3D character."""
diff --git a/motion2mixamorig/checks.py b/motion2mixamorig/checks.py
index 09a6c81..017dcdf 100644
--- a/motion2mixamorig/checks.py
+++ b/motion2mixamorig/checks.py
@@ -109,6 +109,29 @@ def check_video(path: Path) -> str | None:
return None
+def check_image(path: Path) -> str | None:
+ """Decodable still, and exactly one person — same rule as check_video."""
+ import cv2
+
+ frame = cv2.imread(str(path), cv2.IMREAD_COLOR)
+ if frame is None:
+ return (
+ "cannot be decoded as an image — use a jpg/png/webp still of one person"
+ )
+ n = _count_people(frame)
+ if n >= 2:
+ return (
+ f"shows about {n} people — this pipeline maps one body "
+ "to one Mixamo rig. Use a photo with a single, clearly visible person"
+ )
+ if n < 1:
+ return (
+ "no person was detected — use a photo with one clearly visible person, "
+ "head to toe in frame"
+ )
+ return None
+
+
def check_smplx(path: Path) -> str | None:
"""SMPLX_NEUTRAL.npz must be the npz variant of SMPL-X (not SMPL, not .pkl)."""
import numpy as np
diff --git a/motion2mixamorig/cli.py b/motion2mixamorig/cli.py
index 5efe0f7..72ccd06 100644
--- a/motion2mixamorig/cli.py
+++ b/motion2mixamorig/cli.py
@@ -3,6 +3,7 @@
m2mr doctor check assets, weights and dependencies
m2mr run latest video in assets/video/ + default Y Bot
m2mr run --video V --rig R pick the inputs explicitly (flags combine freely)
+ m2mr run --image I --rig R still photo -> 3D pose on the same Mixamo path
`m2mr run` performs the same asset checks as `m2mr doctor` before doing any
work, and reports everything that is missing in one pass.
@@ -32,6 +33,11 @@ VIDEO_HINT = (
"put a *single-person* motion video (.mp4/.mov/...) into assets/video/ "
"— group shots are rejected before extraction"
)
+IMAGE_HINT = (
+ "put a *single-person* still (.jpg/.png/...) into assets/image/ "
+ "— group photos are rejected before extraction"
+)
+SOURCE_HINT = VIDEO_HINT + "; or " + IMAGE_HINT
def _check(label: str, present: bool, hint: str) -> bool:
@@ -80,14 +86,21 @@ def doctor() -> int:
for rig in rigs:
ok &= _check_content(f" - {rig.name}", rig, checks.check_rig)
videos = paths.list_videos()
+ images = paths.list_images()
ok &= _check(
- f"input video assets/video/ ({len(videos)} found"
- + (f", latest: {videos[0].name})" if videos else ")"),
- bool(videos),
- VIDEO_HINT,
+ "input source assets/video/ + assets/image/ "
+ f"({len(videos)} videos"
+ + (f", latest {videos[0].name}" if videos else "")
+ + f"; {len(images)} images"
+ + (f", latest {images[0].name}" if images else "")
+ + ")",
+ bool(videos) or bool(images),
+ SOURCE_HINT,
)
for video in videos:
ok &= _check_content(f" - {video.name}", video, checks.check_video)
+ for image in images:
+ ok &= _check_content(f" - {image.name}", image, checks.check_image)
print()
deps_ok = True
@@ -118,12 +131,12 @@ def doctor() -> int:
return 1
-def preflight(args: argparse.Namespace) -> tuple[Path, Path, Path | None]:
+def preflight(args: argparse.Namespace) -> tuple[Path | None, Path, Path | None, Path | None]:
"""Resolve run inputs, checking everything doctor checks for this run.
All problems are collected and reported together (same format as doctor)
instead of failing on the first one. Exits with code 1 if anything is
- missing.
+ missing. Returns (video, rig, skeleton, image).
"""
problems = 0
@@ -139,19 +152,67 @@ def preflight(args: argparse.Namespace) -> tuple[Path, Path, Path | None]:
if path is not None and path.exists() and not _check_content(label, path, checker):
problems += 1
- # Input video: --video wins, else the most recently added file in assets/video/.
- if args.video:
+ video: Path | None = None
+ image: Path | None = None
+ want_image = getattr(args, "image", None) is not None
+ want_video = bool(getattr(args, "video", None))
+
+ if want_image and want_video:
+ check(
+ "input source --video / --image",
+ False,
+ "pass only one of --video or --image",
+ )
+ elif want_image:
+ if args.image:
+ image = Path(args.image)
+ check(f"input image {image}", image.exists(), "check the --image path")
+ else:
+ image = paths.latest_image()
+ check(
+ "input image assets/image/"
+ + (f" (latest: {image.name})" if image else " (0 found)"),
+ image is not None,
+ IMAGE_HINT + ", or pass --image PATH",
+ )
+ if image is not None and image.exists() and not paths.is_image(image):
+ check(
+ f"image suffix {image.name}",
+ False,
+ "use a still (.jpg/.png/webp/bmp), or pass a clip with --video",
+ )
+ check_content(f"image is usable {image.name if image else ''}", image, checks.check_image)
+ elif want_video:
video = Path(args.video)
- check(f"input video {video}", video.exists(), "check the --video path")
+ if video.exists() and paths.is_image(video):
+ image = video
+ video = None
+ print(f"{OK} input image {image} (detected from --video path)")
+ check_content(f"image is usable {image.name}", image, checks.check_image)
+ else:
+ check(f"input video {video}", video.exists(), "check the --video path")
+ check_content(f"video is usable {video.name}", video, checks.check_video)
else:
video = paths.latest_video()
- check(
- "input video assets/video/"
- + (f" (latest: {video.name})" if video else " (0 found)"),
- video is not None,
- VIDEO_HINT + ", or pass --video",
- )
- check_content(f"video is usable {video.name if video else ''}", video, checks.check_video)
+ if video is None:
+ image = paths.latest_image()
+ check(
+ "input image assets/image/"
+ + (f" (latest: {image.name})" if image else " (0 found)"),
+ image is not None,
+ SOURCE_HINT + ", or pass --video / --image",
+ )
+ check_content(
+ f"image is usable {image.name if image else ''}", image, checks.check_image
+ )
+ else:
+ check(
+ "input video assets/video/"
+ + f" (latest: {video.name})",
+ True,
+ VIDEO_HINT + ", or pass --video",
+ )
+ check_content(f"video is usable {video.name}", video, checks.check_video)
# Rig: --rig wins, else Y_Bot.fbx, else the first .fbx in assets/mixamo/.
if args.rig:
@@ -191,41 +252,57 @@ def preflight(args: argparse.Namespace) -> tuple[Path, Path, Path | None]:
if problems:
print(f"\n{problems} problem(s) — nothing was run. `m2mr doctor` checks everything.")
sys.exit(1)
- return video, rig, skeleton
+ return video, rig, skeleton, image
def run(args: argparse.Namespace) -> int:
- video, rig, skeleton = preflight(args)
+ video, rig, skeleton, image = preflight(args)
print()
from .pipeline import run_pipeline
- run_dir = run_pipeline(video, rig, device=args.device, skeleton_npz=skeleton)
- print(f"\nresults: {run_dir / 'videos'}")
+ run_dir = run_pipeline(
+ video, rig, image=image, device=args.device, skeleton_npz=skeleton
+ )
+ preview = "images" if image is not None else "videos"
+ print(f"\nresults: {run_dir / preview}")
return 0
def main() -> None:
parser = argparse.ArgumentParser(
prog="m2mr",
- description="Drive a Mixamo-rigged character with human motion from a video.",
+ description=(
+ "Drive a Mixamo-rigged character with human motion from a video, "
+ "or a static 3D pose from a still photo."
+ ),
)
sub = parser.add_subparsers(dest="command", required=True)
sub.add_parser("doctor", help="check assets, weights and dependencies")
- p_run = sub.add_parser("run", help="video -> Mixamo rig, results in outputs/")
+ p_run = sub.add_parser("run", help="video or still -> Mixamo rig, results in outputs/")
p_run.add_argument(
"--video",
help="input video (default: the file most recently added to assets/video/)",
)
+ p_run.add_argument(
+ "--image",
+ nargs="?",
+ const="",
+ default=None,
+ help=(
+ "still photo of one person (default: the file most recently added "
+ "to assets/image/). Recovers a static 3D pose; do not combine with --video"
+ ),
+ )
p_run.add_argument(
"--rig",
help="Mixamo character .fbx (default: assets/mixamo/Y_Bot.fbx)",
)
p_run.add_argument(
"--skeleton",
- help="reuse a previous run's skeleton_motion.npz and skip video extraction",
+ help="reuse a previous run's skeleton_motion.npz and skip extraction",
)
p_run.add_argument(
"--device",
diff --git a/motion2mixamorig/extract.py b/motion2mixamorig/extract.py
index 9513706..3351fdf 100644
--- a/motion2mixamorig/extract.py
+++ b/motion2mixamorig/extract.py
@@ -1,16 +1,21 @@
-"""Video -> 3D human skeleton (GVHMR world-grounded motion recovery).
+"""Video or still -> 3D human skeleton (GVHMR world-grounded motion recovery).
Produces skeleton_motion.npz with 24 SMPL joints per frame in a Y-up,
meter-scaled world, plus the root translation/orientation and fps.
+
+A still photo is repeated into a short hold clip first: GVHMR is a temporal
+model and a one-frame "video" is too short to recover a stable pose.
"""
from __future__ import annotations
+from contextlib import contextmanager
from pathlib import Path
+from typing import Iterator
import numpy as np
-from .paths import export_gvhmr_env
+from .paths import VIDEO_SUFFIXES, export_gvhmr_env
# SMPL 24-joint order used by GVHMR (J_regressor).
SMPL_JOINT_NAMES = [
@@ -40,6 +45,15 @@ SMPL_JOINT_NAMES = [
"right_hand",
]
+# GVHMR is a temporal model: a one-frame "video" is too short and often
+# unstable. Repeating the still for about a second gives it a static clip
+# without inventing motion, and the preview / .glb then hold the pose.
+HOLD_FRAMES = 32
+HOLD_FPS = 30.0
+# Phone photos are often 4000px+; YOLO / RTMPose do not need that, and a
+# huge still makes the hold-video encode the slow part of an image run.
+HOLD_MAX_SIDE = 1920
+
def _to_numpy(x):
if hasattr(x, "detach"):
@@ -104,6 +118,103 @@ def _stabilize_pose_cache(pose_cache: Path) -> bool:
return True
+def still_to_hold_video(
+ image: Path,
+ out_mp4: Path,
+ *,
+ n_frames: int = HOLD_FRAMES,
+ fps: float = HOLD_FPS,
+) -> Path:
+ """Repeat a still as a short silent mp4 so GVHMR can treat it as a clip."""
+ import shutil
+ import subprocess
+
+ import cv2
+
+ image = Path(image)
+ frame = cv2.imread(str(image), cv2.IMREAD_COLOR)
+ if frame is None:
+ raise ValueError(f"cannot decode image: {image}")
+
+ h, w = frame.shape[:2]
+ longest = max(h, w)
+ if longest > HOLD_MAX_SIDE:
+ scale = HOLD_MAX_SIDE / longest
+ w = int(round(w * scale))
+ h = int(round(h * scale))
+ frame = cv2.resize(frame, (w, h), interpolation=cv2.INTER_AREA)
+ h, w = frame.shape[:2]
+ w -= w % 2
+ h -= h % 2
+ if w < 2 or h < 2:
+ raise ValueError(f"image is too small: {image}")
+ frame = frame[:h, :w]
+
+ out_mp4 = Path(out_mp4)
+ out_mp4.parent.mkdir(parents=True, exist_ok=True)
+
+ ffmpeg = shutil.which("ffmpeg")
+ if ffmpeg is not None:
+ cmd = [
+ ffmpeg, "-y", "-loglevel", "error",
+ "-f", "rawvideo", "-pix_fmt", "bgr24",
+ "-s", f"{w}x{h}", "-r", f"{fps:g}",
+ "-i", "-",
+ "-frames:v", str(n_frames),
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "18",
+ "-preset", "veryfast", "-movflags", "+faststart",
+ str(out_mp4),
+ ]
+ proc = subprocess.Popen(cmd, stdin=subprocess.PIPE)
+ assert proc.stdin is not None
+ for _ in range(n_frames):
+ proc.stdin.write(frame.tobytes())
+ proc.stdin.close()
+ if proc.wait() != 0:
+ raise RuntimeError(f"ffmpeg failed writing {out_mp4}")
+ return out_mp4
+
+ writer = cv2.VideoWriter(
+ str(out_mp4), cv2.VideoWriter_fourcc(*"mp4v"), float(fps), (w, h)
+ )
+ if not writer.isOpened():
+ raise RuntimeError(f"Cannot open VideoWriter for {out_mp4}")
+ for _ in range(n_frames):
+ writer.write(frame)
+ writer.release()
+ return out_mp4
+
+
+@contextmanager
+def temp_hold_video(image: Path) -> Iterator[Path]:
+ """Write the hold clip under the system temp dir; delete it afterwards.
+
+ GVHMR needs a real video file, but that file is not a run artifact and
+ must not land in cache/.
+ """
+ import shutil
+ import tempfile
+
+ tmp = Path(tempfile.mkdtemp(prefix="m2mr-hold-"))
+ try:
+ yield still_to_hold_video(image, tmp / f"{Path(image).stem}.mp4")
+ finally:
+ shutil.rmtree(tmp, ignore_errors=True)
+
+
+def _strip_cache_videos(root: Path) -> None:
+ """Drop staged / overlay mp4s from a still-image extract.
+
+ GVHMR always stages ``0_input_video.mp4`` (and may write overlay clips
+ under preprocess/). Those are only useful for a video run.
+ """
+ if not root.is_dir():
+ return
+ for path in root.rglob("*"):
+ if path.suffix.lower() in VIDEO_SUFFIXES and (path.is_file() or path.is_symlink()):
+ path.unlink(missing_ok=True)
+
+
def extract_motion(
video: Path,
output_npz: Path,
@@ -112,11 +223,13 @@ def extract_motion(
device: str = "cpu",
static_camera: bool = True,
verbose: bool = True,
+ source_image: Path | None = None,
) -> Path:
"""Run GVHMR on `video` and save skeleton_motion.npz to `output_npz`.
`work_dir` holds GVHMR's per-video intermediates (2D pose cache, 3D lift).
Re-running with the same work_dir reuses the expensive passes.
+ `source_image` is recorded when the clip was built from a still.
"""
export_gvhmr_env()
import gvhmr
@@ -157,8 +270,7 @@ def extract_motion(
output_npz = Path(output_npz)
output_npz.parent.mkdir(parents=True, exist_ok=True)
- np.savez_compressed(
- output_npz,
+ payload = dict(
joints_3d=joints_3d,
joint_names=np.array(SMPL_JOINT_NAMES),
fps=np.float32(result.fps),
@@ -168,6 +280,10 @@ def extract_motion(
camera=np.array(result.camera),
video=np.array(str(video)),
)
+ if source_image is not None:
+ payload["image"] = np.array(str(source_image))
+ _strip_cache_videos(output_dir)
+ np.savez_compressed(output_npz, **payload)
if verbose:
print(f"saved {output_npz}")
return output_npz
diff --git a/motion2mixamorig/paths.py b/motion2mixamorig/paths.py
index f9b9998..45f2aba 100644
--- a/motion2mixamorig/paths.py
+++ b/motion2mixamorig/paths.py
@@ -14,23 +14,33 @@ SMPLX_NEUTRAL = BODY_MODELS / "smplx" / "SMPLX_NEUTRAL.npz"
MIXAMO_DIR = ASSETS / "mixamo"
DEFAULT_RIG = MIXAMO_DIR / "Y_Bot.fbx"
VIDEO_DIR = ASSETS / "video"
+IMAGE_DIR = ASSETS / "image"
WEIGHTS = PROJECT_ROOT / "weights"
OUTPUTS = PROJECT_ROOT / "outputs"
VIDEO_SUFFIXES = (".mp4", ".mov", ".avi", ".mkv", ".webm")
+IMAGE_SUFFIXES = (".jpg", ".jpeg", ".png", ".webp", ".bmp")
-def list_videos() -> list[Path]:
- """User-supplied videos in assets/video/, most recently added first."""
- if not VIDEO_DIR.is_dir():
+def _list_media(folder: Path, suffixes: tuple[str, ...]) -> list[Path]:
+ if not folder.is_dir():
return []
- videos = [
+ files = [
p
- for p in VIDEO_DIR.iterdir()
- if p.suffix.lower() in VIDEO_SUFFIXES and not p.name.startswith(".")
+ for p in folder.iterdir()
+ if p.suffix.lower() in suffixes and not p.name.startswith(".")
]
- return sorted(videos, key=lambda p: p.stat().st_mtime, reverse=True)
+ return sorted(files, key=lambda p: p.stat().st_mtime, reverse=True)
+
+
+def is_image(path: Path) -> bool:
+ return Path(path).suffix.lower() in IMAGE_SUFFIXES
+
+
+def list_videos() -> list[Path]:
+ """User-supplied videos in assets/video/, most recently added first."""
+ return _list_media(VIDEO_DIR, VIDEO_SUFFIXES)
def latest_video() -> Path | None:
@@ -39,12 +49,22 @@ def latest_video() -> Path | None:
return videos[0] if videos else None
-def new_run_dir(video: Path, now: datetime | None = None) -> Path:
- """Create outputs/<YYYYMMDD_HHMMSS>_<video-stem>/ for one `m2mr run`."""
+def list_images() -> list[Path]:
+ """User-supplied stills in assets/image/, most recently added first."""
+ return _list_media(IMAGE_DIR, IMAGE_SUFFIXES)
+
+
+def latest_image() -> Path | None:
+ """The still most recently placed into assets/image/ (by file mtime)."""
+ images = list_images()
+ return images[0] if images else None
+
+
+def new_run_dir(source: Path, now: datetime | None = None) -> Path:
+ """Create outputs/<YYYYMMDD_HHMMSS>_<source-stem>/ for one `m2mr run`."""
stamp = (now or datetime.now()).strftime("%Y%m%d_%H%M%S")
- run_dir = OUTPUTS / f"{stamp}_{video.stem}"
+ run_dir = OUTPUTS / f"{stamp}_{source.stem}"
run_dir.mkdir(parents=True, exist_ok=True)
- (run_dir / "videos").mkdir(exist_ok=True)
return run_dir
diff --git a/motion2mixamorig/pipeline.py b/motion2mixamorig/pipeline.py
index aff185b..a5669a5 100644
--- a/motion2mixamorig/pipeline.py
+++ b/motion2mixamorig/pipeline.py
@@ -1,14 +1,14 @@
-"""One `m2mr run`: video -> human skeleton -> Mixamo rig -> result videos.
+"""One `m2mr run`: video or still -> human skeleton -> Mixamo rig -> result videos.
Everything a run produces lands in its own timestamped folder:
- outputs/<YYYYMMDD_HHMMSS>_<video>/
+ outputs/<YYYYMMDD_HHMMSS>_<source>/
├── run.json what ran, with what inputs, producing what
├── skeleton_motion.npz 3D human skeleton (24 SMPL joints per frame)
├── mixamo_rotations.npz per-bone animation rotations for the rig
├── mixamo_character.glb skinned character + animation (Blender / Unity)
├── cache/ GVHMR intermediates (2D pose, 3D lift)
- └── videos/ the four result videos
+ └── videos/ or images/ four result clips (video in) or four stills (photo in)
"""
from __future__ import annotations
@@ -30,7 +30,7 @@ from .mixamo.retarget import verify_rest_fk
from .mixamo.skinned_mesh import load_character_asset, rest_skin_error
from .mixamo.tpose_calibration import load_calibration
from .mixamo.ybot_retarget import build_mapping
-from .videos import render_run_videos
+from .videos import render_run_images, render_run_videos
ROTATION_CONVENTION = (
"Animation-channel rotations: M_local = T * Rpre * R_anim, world = parent @ local. "
@@ -67,37 +67,61 @@ def save_mixamo_rotations(anim, mapping, rig_path: Path, out_npz: Path) -> Path:
def run_pipeline(
- video: Path,
+ video: Path | None,
rig: Path,
*,
+ image: Path | None = None,
device: str = "cpu",
skeleton_npz: Path | None = None,
verbose: bool = True,
) -> Path:
- """Full pipeline for one video + one rig. Returns the run directory.
+ """Full pipeline for one video or still + one rig. Returns the run directory.
`skeleton_npz` reuses a previous run's skeleton_motion.npz and skips the
- (slow) video extraction entirely.
+ (slow) extraction entirely. A still is turned into a short hold clip so
+ the same GVHMR path can recover a static 3D pose.
"""
- video = Path(video).resolve()
+ if image is None and video is None:
+ raise ValueError("run_pipeline needs a video or an image")
+ if image is not None:
+ image = Path(image).resolve()
+ if video is not None:
+ video = Path(video).resolve()
rig = Path(rig).resolve()
started = datetime.now()
t0 = time.time()
- run_dir = paths.new_run_dir(video, started)
+ source = image if image is not None else video
+ assert source is not None
+ run_dir = paths.new_run_dir(source, started)
print(f"run directory {run_dir}")
+ extract_video = video
motion_npz = run_dir / "skeleton_motion.npz"
if skeleton_npz is not None:
import shutil
shutil.copy2(skeleton_npz, motion_npz)
print(f"reusing skeleton from {skeleton_npz}")
+ elif image is not None:
+ from .extract import extract_motion, temp_hold_video
+
+ print(f"extracting human pose from {image.name}")
+ with temp_hold_video(image) as hold:
+ extract_motion(
+ hold,
+ motion_npz,
+ work_dir=run_dir / "cache",
+ device=device,
+ verbose=verbose,
+ source_image=image,
+ )
else:
from .extract import extract_motion
- print(f"extracting human motion from {video.name}")
+ assert extract_video is not None
+ print(f"extracting human motion from {extract_video.name}")
extract_motion(
- video,
+ extract_video,
motion_npz,
work_dir=run_dir / "cache",
device=device,
@@ -127,16 +151,34 @@ def run_pipeline(
write_glb(glb_path, asset, anim.r_anim_rig, anim.hips_cm_rig, anim.fps)
print(f"saved {glb_path}")
- print("rendering result videos")
- videos = render_run_videos(
- anim, asset, calibration, video, run_dir / "videos", verbose=verbose
- )
+ if image is not None:
+ print("rendering result images")
+ previews = render_run_images(
+ anim, asset, calibration, image, run_dir / "images", verbose=verbose
+ )
+ preview_key = "images"
+ preview_dir = "images"
+ else:
+ print("rendering result videos")
+ assert extract_video is not None
+ previews = render_run_videos(
+ anim,
+ asset,
+ calibration,
+ extract_video,
+ run_dir / "videos",
+ original_label="Original Video",
+ verbose=verbose,
+ )
+ preview_key = "videos"
+ preview_dir = "videos"
report = {
"started": started.isoformat(timespec="seconds"),
"elapsed_s": round(time.time() - t0, 1),
"command": sys.argv,
- "video": str(video),
+ "video": None if image is not None else str(video),
+ "image": str(image) if image is not None else None,
"rig": str(rig),
"device": device,
"fps": float(motion.fps),
@@ -150,7 +192,7 @@ def run_pipeline(
"skeleton_motion": motion_npz.name,
"mixamo_rotations": rotations_npz.name,
"mixamo_character_glb": glb_path.name,
- "videos": {k: f"videos/{v.name}" for k, v in videos.items()},
+ preview_key: {k: f"{preview_dir}/{v.name}" for k, v in previews.items()},
},
}
(run_dir / "run.json").write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
diff --git a/motion2mixamorig/videos.py b/motion2mixamorig/videos.py
index c31d50f..ae183e8 100644
--- a/motion2mixamorig/videos.py
+++ b/motion2mixamorig/videos.py
@@ -1,10 +1,14 @@
-"""Render the four result videos of a run, all in the input video's aspect ratio.
+"""Render the four result views of a run, all in the input's aspect ratio.
- videos/human_skeleton.mp4 3D human skeleton recovered from the video
- videos/mixamo_skeleton.mp4 retargeted Mixamo Skeleton
- videos/mixamo_character.mp4 skinned character mesh
- videos/compare.mp4 2x2 grid: original | mixamo skeleton
- human | character
+Video input writes `videos/*.mp4`; a still writes the same four views as
+`images/*.png`:
+
+ human_skeleton 3D human skeleton recovered from the source
+ mixamo_skeleton retargeted Mixamo Skeleton
+ mixamo_character skinned character mesh
+ compare 2x2: original | mixamo skeleton
+ human | character
+ before_after_360_compare.mp4 image runs only: original | orbiting rig
The three rendered panels share one perspective camera and one floor grid, so
they read as views into the same space: an orthographic camera cannot show a
@@ -12,9 +16,10 @@ floor receding to a vanishing point, and without a floor and a contact shadow
the figure just hangs in black. All of it is fitted once over the whole clip;
nothing is re-centred per frame.
-Frames are streamed straight into ffmpeg (which also muxes the source video's
-audio into every output), so memory stays flat no matter how long the clip is.
-When ffmpeg is missing the videos still get written via OpenCV, just silent.
+Video frames are streamed straight into ffmpeg (which also muxes the source
+video's audio into every output), so memory stays flat no matter how long the
+clip is. When ffmpeg is missing the videos still get written via OpenCV, just
+silent.
"""
from __future__ import annotations
@@ -26,11 +31,13 @@ from pathlib import Path
import numpy as np
from .mixamo.animation import RetargetedAnimation, all_points, skin_frames
+from .mixamo.kinematics import normalize
from .mixamo.render import (
DARK_THEME,
GRID_THEME,
MESH_COLORS_BGR,
Ground,
+ PerspectiveCamera,
blank_panel,
build_perspective_camera,
draw_mesh,
@@ -66,6 +73,17 @@ FLOOR_PERCENTILE = 10.0
# backdrop and a little more ambient fill than the white-on-black skeletons.
CHARACTER_AMBIENT = 0.34
+# Image-only turntable: original still on the left, orbiting rig on the right.
+# Elevation 10° looks down from about the character's own height; a slightly
+# wider FOV and looser fill keep hands and feet inside the frame as the
+# camera goes around.
+ORBIT_ELEVATION_DEG = 10.0
+ORBIT_FOV_DEG = 38.0
+ORBIT_FILL = 0.86
+ORBIT_SECONDS = 6.0
+ORBIT_FPS = 30.0
+ORBIT_PAD = 1.06
+
def video_size(video: Path) -> tuple[int, int]:
import cv2
@@ -79,9 +97,25 @@ def video_size(video: Path) -> tuple[int, int]:
return w, h
-def panel_size(video: Path) -> tuple[int, int]:
- """Panel with the same aspect ratio as the input video, even dimensions."""
- w, h = video_size(video)
+def image_size(image: Path) -> tuple[int, int]:
+ import cv2
+
+ frame = cv2.imread(str(image), cv2.IMREAD_COLOR)
+ if frame is None:
+ raise RuntimeError(f"Cannot open image: {image}")
+ h, w = frame.shape[:2]
+ return w, h
+
+
+def source_size(path: Path) -> tuple[int, int]:
+ from .paths import is_image
+
+ return image_size(path) if is_image(path) else video_size(path)
+
+
+def panel_size(source: Path) -> tuple[int, int]:
+ """Panel with the same aspect ratio as the input, even dimensions."""
+ w, h = source_size(source)
pw = int(round(PANEL_H * w / max(h, 1)))
pw = min(max(pw, 2), MAX_PANEL_W)
return pw + pw % 2, PANEL_H + PANEL_H % 2
@@ -185,25 +219,27 @@ def _grid2x2(tl: np.ndarray, tr: np.ndarray, bl: np.ndarray, br: np.ndarray) ->
return np.vstack([np.hstack([tl, tr]), np.hstack([bl, br])])
-def render_run_videos(
- anim: RetargetedAnimation,
- asset,
- calibration,
- source_video: Path,
- out_dir: Path,
- *,
- verbose: bool = True,
-) -> dict[str, Path]:
- """Write the four result videos into `out_dir` (videos/ of a run)."""
- out_dir = Path(out_dir)
- out_dir.mkdir(parents=True, exist_ok=True)
- width, height = panel_size(source_video)
- fps = anim.fps
+def _write_png(path: Path, bgr: np.ndarray) -> Path:
+ import cv2
- # One camera and one floor over human joints, Mixamo joints and the mesh,
- # so the three rendered panels agree on where everything stands. Human,
- # Mixamo Skeleton and character all live in the calibrated rig world, so
- # the shared anatomical axes come from the calibration.
+ path = Path(path)
+ path.parent.mkdir(parents=True, exist_ok=True)
+ if not cv2.imwrite(str(path), bgr):
+ raise RuntimeError(f"Cannot write {path}")
+ return path
+
+
+def _letterbox_still(image: Path, width: int, height: int) -> np.ndarray:
+ import cv2
+
+ frame = cv2.imread(str(image), cv2.IMREAD_COLOR)
+ if frame is None:
+ raise RuntimeError(f"Cannot open image: {image}")
+ return fit_letterbox(frame, width, height)
+
+
+def _setup_view(anim: RetargetedAnimation, asset, calibration, width: int, height: int):
+ """Shared camera, floor and backdrops for the three rendered panels."""
axes = calibration.rig_axes
stride = max(1, anim.n // CAMERA_MESH_SAMPLES)
probe = skin_frames(asset, anim, list(range(0, anim.n, stride)))
@@ -233,14 +269,201 @@ def render_run_videos(
cx=float(np.median(hips[:, 0])),
cz=float(np.median(hips[:, 2])),
)
+ backdrop_dark = ground_backdrop(cam, ground, width, height, DARK_THEME)
+ backdrop_grid = ground_backdrop(cam, ground, width, height, GRID_THEME)
+ faces = [m.faces for m in asset.meshes]
+ colors = [
+ m.face_colors
+ if m.face_colors is not None
+ else MESH_COLORS_BGR.get(m.name, np.array([150.0, 150.0, 150.0]))
+ for m in asset.meshes
+ ]
+ return cam, ground, backdrop_dark, backdrop_grid, faces, colors
+
+
+def _draw_panels(
+ anim: RetargetedAnimation,
+ asset,
+ calibration,
+ i: int,
+ *,
+ original: np.ndarray,
+ original_label: str,
+ stamp: str,
+ cam,
+ ground,
+ backdrop_dark,
+ backdrop_grid,
+ faces,
+ colors,
+) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
+ character = calibration.fbx_path.stem
+
+ p_hum = backdrop_dark.copy()
+ pos = anim.human_pos[i]
+ draw_shadow(p_hum, cam, np.stack(list(pos.values())), ground, DARK_THEME)
+ draw_skeleton(p_hum, pos, HUMAN_EDGES, cam, 2, DARK_THEME.joint)
+ label_panel(p_hum, f"Human Skeleton{stamp}")
+
+ p_mix = backdrop_dark.copy()
+ pos = anim.mixamo_pos[i]
+ draw_shadow(p_mix, cam, np.stack(list(pos.values())), ground, DARK_THEME)
+ draw_skeleton(p_mix, pos, MIXAMO_EDGES, cam, 3, DARK_THEME.joint)
+ label_panel(p_mix, f"Mixamo Skeleton{stamp}")
+
+ p_char = backdrop_grid.copy()
+ verts = skin_frames(asset, anim, [i])[0]
+ draw_shadow(p_char, cam, np.concatenate(verts, axis=0), ground, GRID_THEME)
+ draw_mesh(p_char, verts, faces, colors, cam, CHARACTER_AMBIENT)
+ label_panel(p_char, f"{character}{stamp}")
+
+ p_orig = original.copy()
+ label_panel(p_orig, f"{original_label}{stamp}")
+ return p_orig, p_mix, p_hum, p_char
+
+
+def render_run_images(
+ anim: RetargetedAnimation,
+ asset,
+ calibration,
+ source_image: Path,
+ out_dir: Path,
+ *,
+ original_label: str = "Original Photo",
+ verbose: bool = True,
+) -> dict[str, Path]:
+ """Write the four stills into `out_dir` (images/ of a still run).
+
+ Uses the middle hold frame: GVHMR is temporal, so the centre of the
+ repeated still is the most settled pose.
+ """
+ out_dir = Path(out_dir)
+ out_dir.mkdir(parents=True, exist_ok=True)
+ width, height = panel_size(source_image)
+ cam, ground, backdrop_dark, backdrop_grid, faces, colors = _setup_view(
+ anim, asset, calibration, width, height
+ )
if verbose:
print(f" camera eye {np.round(cam.eye, 3)} fov {cam.fov_deg:.1f} deg"
f" floor y {ground.y:+.4f} m")
- # Camera and floor are both fixed, so the floor is drawn once per theme
- # and every frame starts as a copy of it.
- backdrop_dark = ground_backdrop(cam, ground, width, height, DARK_THEME)
- backdrop_grid = ground_backdrop(cam, ground, width, height, GRID_THEME)
+ i = anim.n // 2
+ p_orig, p_mix, p_hum, p_char = _draw_panels(
+ anim, asset, calibration, i,
+ original=_letterbox_still(source_image, width, height),
+ original_label=original_label,
+ stamp="",
+ cam=cam, ground=ground,
+ backdrop_dark=backdrop_dark, backdrop_grid=backdrop_grid,
+ faces=faces, colors=colors,
+ )
+ out = {
+ "human_skeleton": _write_png(out_dir / "human_skeleton.png", p_hum),
+ "mixamo_skeleton": _write_png(out_dir / "mixamo_skeleton.png", p_mix),
+ "mixamo_character": _write_png(out_dir / "mixamo_character.png", p_char),
+ "compare": _write_png(out_dir / "compare.png", _grid2x2(p_orig, p_mix, p_hum, p_char)),
+ }
+ if verbose:
+ print("rendering 360 compare")
+ out["before_after_360_compare"] = render_orbit_compare(
+ anim, asset, calibration, source_image, out_dir, verbose=verbose
+ )
+ if verbose:
+ for path in out.values():
+ print(f"saved {path}")
+ return out
+
+
+def _orbit_camera(
+ axes: dict,
+ target: np.ndarray,
+ dist: float,
+ azimuth_deg: float,
+ width: int,
+ height: int,
+ *,
+ elevation_deg: float = ORBIT_ELEVATION_DEG,
+ fov_deg: float = ORBIT_FOV_DEG,
+) -> PerspectiveCamera:
+ """Pinhole eye on a circle around `target`, looking down at `elevation_deg`."""
+ up = normalize(np.asarray(axes["up"], dtype=np.float64))
+ fwd = normalize(np.asarray(axes["forward"], dtype=np.float64))
+ right = normalize(np.asarray(axes["right"], dtype=np.float64))
+ az, el = np.radians(azimuth_deg), np.radians(elevation_deg)
+ offset = normalize(np.cos(el) * (np.cos(az) * fwd + np.sin(az) * right) + np.sin(el) * up)
+ look = -offset
+ cam_right = normalize(np.cross(look, up))
+ cam_up = normalize(np.cross(cam_right, look))
+ focal = (height * 0.5) / np.tan(np.radians(fov_deg) * 0.5)
+ return PerspectiveCamera(
+ target + offset * dist, look, cam_right, cam_up, float(focal), width, height
+ )
+
+
+def _fit_orbit_distance(
+ axes: dict,
+ points: np.ndarray,
+ target: np.ndarray,
+ width: int,
+ height: int,
+) -> float:
+ """One radius that keeps the whole mesh on screen at every heading.
+
+ Starts at the distance that puts the eye at about the character's height
+ (elevation 10° looking at the torso centre), then grows if any heading
+ would clip a hand or a foot.
+ """
+ pts = np.asarray(points, dtype=np.float64).reshape(-1, 3)
+ up = normalize(np.asarray(axes["up"], dtype=np.float64))
+ along = pts @ up
+ head = float(along.max())
+ mid = float(target @ up)
+ el = np.radians(ORBIT_ELEVATION_DEG)
+ dist = max((head - mid) / max(np.sin(el), 1e-3), 1e-3)
+ center = pts.mean(axis=0)
+ padded = center + (pts - center) * ORBIT_PAD
+ for az in np.linspace(0.0, 360.0, 12, endpoint=False):
+ for _ in range(60):
+ cam = _orbit_camera(axes, target, dist, float(az), width, height)
+ uv = cam.project(padded)
+ lo, hi = uv.min(axis=0), uv.max(axis=0)
+ need = max(
+ (hi[0] - lo[0]) / (ORBIT_FILL * width),
+ (hi[1] - lo[1]) / (ORBIT_FILL * height),
+ 1.0,
+ )
+ inside = (lo >= 0.0).all() and (hi < (width, height)).all()
+ if need <= 1.0 and inside:
+ break
+ dist *= max(need, 1.02)
+ return float(dist)
+
+
+def render_orbit_compare(
+ anim: RetargetedAnimation,
+ asset,
+ calibration,
+ source_image: Path,
+ out_dir: Path,
+ *,
+ verbose: bool = True,
+) -> Path:
+ """Left = original photo, right = 10° orbit of the skinned rig. Image runs only."""
+ out_dir = Path(out_dir)
+ out_dir.mkdir(parents=True, exist_ok=True)
+ width, height = panel_size(source_image)
+ i = anim.n // 2
+ verts = skin_frames(asset, anim, [i])[0]
+ mesh_pts = np.concatenate(verts, axis=0)
+ axes = calibration.rig_axes
+ target = 0.5 * (mesh_pts.min(axis=0) + mesh_pts.max(axis=0))
+ dist = _fit_orbit_distance(axes, mesh_pts, target, width, height)
+ hips = anim.mixamo_pos[i]["mixamorig:Hips"]
+ ground = Ground(
+ y=float(np.percentile(mesh_pts[:, 1], FLOOR_PERCENTILE)),
+ cx=float(hips[0]),
+ cz=float(hips[2]),
+ )
faces = [m.faces for m in asset.meshes]
colors = [
@@ -249,6 +472,52 @@ def render_run_videos(
else MESH_COLORS_BGR.get(m.name, np.array([150.0, 150.0, 150.0]))
for m in asset.meshes
]
+ left = _letterbox_still(source_image, width, height)
+ label_panel(left, "Original Photo")
+ character = calibration.fbx_path.stem
+ n = max(2, int(round(ORBIT_SECONDS * ORBIT_FPS)))
+ out_path = out_dir / "before_after_360_compare.mp4"
+ writer = _StreamWriter(out_path, ORBIT_FPS, 2 * width, height)
+ if verbose:
+ print(f" orbit dist {dist:.3f} m {n} frames @ {ORBIT_FPS:g} fps"
+ f" elev {ORBIT_ELEVATION_DEG:g} deg")
+ try:
+ for k in range(n):
+ az = 360.0 * k / n
+ cam = _orbit_camera(axes, target, dist, az, width, height)
+ right = ground_backdrop(cam, ground, width, height, GRID_THEME)
+ draw_shadow(right, cam, mesh_pts, ground, GRID_THEME)
+ draw_mesh(right, verts, faces, colors, cam, CHARACTER_AMBIENT)
+ label_panel(right, character)
+ writer.write(np.hstack([left, right]))
+ if verbose and (k == 0 or (k + 1) % 30 == 0 or k == n - 1):
+ print(f" orbit {k + 1}/{n}")
+ finally:
+ writer.close()
+ return out_path
+
+
+def render_run_videos(
+ anim: RetargetedAnimation,
+ asset,
+ calibration,
+ source_video: Path,
+ out_dir: Path,
+ *,
+ original_label: str = "Original Video",
+ verbose: bool = True,
+) -> dict[str, Path]:
+ """Write the four result videos into `out_dir` (videos/ of a run)."""
+ out_dir = Path(out_dir)
+ out_dir.mkdir(parents=True, exist_ok=True)
+ width, height = panel_size(source_video)
+ fps = anim.fps
+ cam, ground, backdrop_dark, backdrop_grid, faces, colors = _setup_view(
+ anim, asset, calibration, width, height
+ )
+ if verbose:
+ print(f" camera eye {np.round(cam.eye, 3)} fov {cam.fov_deg:.1f} deg"
+ f" floor y {ground.y:+.4f} m")
audio_start = anim.frames[0] / fps
audio_dur = anim.n / fps
@@ -261,7 +530,6 @@ def render_run_videos(
audio_dur_s=audio_dur,
)
- character = calibration.fbx_path.stem
writers = {
"human_skeleton": writer("human_skeleton.mp4", width, height),
"mixamo_skeleton": writer("mixamo_skeleton.mp4", width, height),
@@ -272,29 +540,15 @@ def render_run_videos(
try:
for i in range(anim.n):
t = anim.frames[i] / fps
- stamp = f"f{anim.frames[i]} t={t:.2f}s"
-
- p_hum = backdrop_dark.copy()
- pos = anim.human_pos[i]
- draw_shadow(p_hum, cam, np.stack(list(pos.values())), ground, DARK_THEME)
- draw_skeleton(p_hum, pos, HUMAN_EDGES, cam, 2, DARK_THEME.joint)
- label_panel(p_hum, f"Human Skeleton {stamp}")
-
- p_mix = backdrop_dark.copy()
- pos = anim.mixamo_pos[i]
- draw_shadow(p_mix, cam, np.stack(list(pos.values())), ground, DARK_THEME)
- draw_skeleton(p_mix, pos, MIXAMO_EDGES, cam, 3, DARK_THEME.joint)
- label_panel(p_mix, f"Mixamo Skeleton {stamp}")
-
- p_char = backdrop_grid.copy()
- verts = skin_frames(asset, anim, [i])[0]
- draw_shadow(p_char, cam, np.concatenate(verts, axis=0), ground, GRID_THEME)
- draw_mesh(p_char, verts, faces, colors, cam, CHARACTER_AMBIENT)
- label_panel(p_char, f"{character} {stamp}")
-
- p_orig = originals.frame(anim.frames[i])
- label_panel(p_orig, f"Original Video {stamp}")
-
+ p_orig, p_mix, p_hum, p_char = _draw_panels(
+ anim, asset, calibration, i,
+ original=originals.frame(anim.frames[i]),
+ original_label=original_label,
+ stamp=f" f{anim.frames[i]} t={t:.2f}s",
+ cam=cam, ground=ground,
+ backdrop_dark=backdrop_dark, backdrop_grid=backdrop_grid,
+ faces=faces, colors=colors,
+ )
writers["human_skeleton"].write(p_hum)
writers["mixamo_skeleton"].write(p_mix)
writers["mixamo_character"].write(p_char)
diff --git a/pyproject.toml b/pyproject.toml
index 379490b..8e2aec7 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,7 +1,7 @@
[project]
name = "motion2mixamorig"
version = "0.1.0"
-description = "Retarget human motion from video to an Adobe Mixamo-rigged 3D character."
+description = "Retarget human motion from video or a still pose to an Adobe Mixamo-rigged 3D character."
requires-python = ">=3.10"
dependencies = [
"numpy",