Skip to content

Retarget human motion from video to an Adobe Mixamo-rigged 3D character.

Python

Add self-intersection diagnostics to the CLI and the Blender add-on

MaxLiu418facf9bca3
diff --git a/AGENTS.md b/AGENTS.md
index a5960d1..3d90182 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -36,6 +36,7 @@ m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbx
m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Y_Bot.fbx
m2mr run --skeleton outputs/<run>/skeleton_motion.npz --rig ... # reuse extraction, swap rig
m2mr run --output-dir DIR --progress-jsonl FILE --no-preview # Blender Extension / automation
+m2mr diagnose outputs/<run> [--visualize] [--threshold-cm N] # re-check self-intersection -> diagnostics.json (+ .html)
```
`--video` / `--image` and `--rig` combine freely (not `--video` with `--image`).
@@ -85,7 +86,10 @@ When helping a user:
6. Jobs write `<project>/outputs/blender_jobs/<job-id>/`. After import, hide
**Icosphere** and the **armature** in the Outliner; leave the
**skinned_mesh_…** objects (one per FBX geometry) visible. FPS must match the source clip duration (a 60 fps video tagged as
- 30 fps plays at half speed).
+ 30 fps plays at half speed). The timeline runs 0..n-1, the frames
+ `diagnostics.json` uses; the **Diagnostics** box under Progress reads it
+ (frame bar, Previous / Next, **Open HTML Report** builds `diagnostics.html`
+ on first click).
Smoke-test checklist: [`docs/blender-extension.md`](docs/blender-extension.md).
@@ -97,6 +101,7 @@ Each run writes `outputs/<YYYYMMDD_HHMMSS>_<source>/`:
- `skeleton_motion.npz` — 3D human skeleton (24 SMPL joints, Y-up meters)
- `mixamo_rotations.npz` — per-bone animation rotations (quaternion wxyz; convention string inside)
- `mixamo_character.glb` — skinned character + clip (glTF 2.0; Blender: File → Import → glTF 2.0)
+- `diagnostics.json` — self-intersection report on the GLB (body-part pairs, frame ranges, depth in cm; frames counted from 0); summary also in `run.json`. Parts joined at a joint are not compared. No floor check. `m2mr diagnose <run> [--visualize] [--threshold-cm N]` re-checks a run; `--visualize` adds `diagnostics.html`.
- Video in: `videos/human_skeleton.mp4`, `videos/mixamo_skeleton.mp4`, `videos/mixamo_character.mp4`, `videos/compare.mp4` (2x2: original / mixamo skeleton / human skeleton / character)
- Photo in: the same four views as PNG under `images/`, plus `images/before_after_360_compare.mp4` (left original / right 10° orbit of the rig)
@@ -105,6 +110,8 @@ Each run writes `outputs/<YYYYMMDD_HHMMSS>_<source>/`:
- `motion2mixamorig/paths.py` — all filesystem locations
- `motion2mixamorig/extract.py` — video or still -> 3D human skeleton (GVHMR)
- `motion2mixamorig/mixamo/` — FBX parsing, FK, retargeting, skinning, rendering
+- `motion2mixamorig/diagnose.py` — self-intersection check on the GLB -> `diagnostics.json`
+- `motion2mixamorig/diagnose_report.py` — `diagnostics.html` (`--visualize`)
- `motion2mixamorig/pipeline.py` — one run end to end
- `motion2mixamorig/cli.py` — `m2mr doctor` / `m2mr run`
- `blender_extension/motion2mixamorig_blender/` — Blender 4.2+ Extension (UI + external process)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6cc54e8..f7f00eb 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,8 +4,15 @@ All notable changes to this project will be documented in this file.
## [Unreleased]
+### Added
+
+- Self-intersection diagnostics: every `m2mr run` checks the finished `mixamo_character.glb` for one body part going into another and writes `diagnostics.json` (body-part pairs, frame ranges, worst frame, depth in cm) plus a one-line summary; the summary also goes into `run.json`. A failed check does not fail the run. `m2mr diagnose <run>` re-checks a run; `--visualize` also writes `diagnostics.html` with the source video beside the character, a timeline, and a clickable issue list; `--threshold-cm` sets the reporting depth (default 1 cm). Parts joined at a joint are not compared, and stretches of one pair at most 3 frames apart are reported as one range. The floor is not checked.
+- Blender add-on: a **Diagnostics** box under Progress shows the job's self-intersections as a 48-cell frame bar (red = intersecting; click a cell to go there), the parts intersecting on the current frame with their depth, and Previous / Next buttons that jump between problem ranges. **Open HTML Report** opens the interactive report in the browser; the first click builds it in the background (`m2mr diagnose --visualize`, about 15 s). Results generated before this version show a note to update the runtime and generate again.
+
### Fixed
+- Blender add-on: after import the timeline covers the clip as frames 0 to n-1 (was 1 to n), the frame numbers the glTF importer and `diagnostics.json` use.
+- Blender add-on: the external Python now finds Homebrew tools such as `ffmpeg` (`/opt/homebrew/bin`, `/usr/local/bin`) when Blender was opened from the Dock or Finder, whose PATH leaves them out. Before, the HTML report could not embed the source video.
- A subject who turned away and back could stay facing away in the recovered 3D motion for the rest of the clip. The 2D left/right stabilizer decided "real turn or label flip" at the single transition frame (it required four consecutive side-view frames after the first turn) and never revisited the decision, so one mislabelled frame or one fast spin locked the wrong facing in. `motion2mixamorig/stabilize_kp2d.py` now cuts the clip into segments at side-view frames and takes the majority of RTMPose's labels inside each segment; only runs of the opposite sign shorter than ~0.33 s count as flips. No facial keypoints are used (hats and masks do not matter) and each person track is handled on its own. On a dance clip with a fast half-second turn, wrong-facing frames went from 320 of 642 to 3. Re-running on an existing cache re-stabilizes from the `vitpose_raw.pt` backup, not from the previous rule's output.
## [0.1.7] - 2026-10-17
diff --git a/README.md b/README.md
index 62cdb56..0382ef9 100644
--- a/README.md
+++ b/README.md
@@ -139,9 +139,23 @@ m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx
`--image` with no path uses the file most recently added to `assets/image/`.
+### 6. Check for self-intersection
+
+Every run also checks the finished character for **self-intersection** — one body part going into another, like a hand inside the head — and writes `diagnostics.json`. The terminal prints one line, e.g. `diagnostics: 14 self-intersection(s) over 116 of 719 frames, deepest 7.87 cm`. The floor is not checked.
+
+To re-check a run, or to see the problems:
+
+```bash
+m2mr diagnose outputs/<run> # rewrite diagnostics.json
+m2mr diagnose outputs/<run> --visualize # also diagnostics.html: source video beside the character, timeline, clickable issue list
+m2mr diagnose outputs/<run> --threshold-cm 2 # report only penetrations deeper than 2 cm (default 1)
+```
+
+Each issue in `diagnostics.json` is a pair of body parts with the frame ranges where they intersect, the worst frame, and its depth. Frame numbers count from 0, as the Blender timeline does after a glTF import. Parts joined at a joint (upper arm and forearm, torso and thigh, …) are not compared: their skins overlap around the joint in any pose. `diagnostics.html` embeds the source video (needs `ffmpeg`) and loads three.js from a CDN, so its 3D view needs an internet connection.
+
## Outputs
-Each `m2mr run` creates a directory named by **the time the command started**. The four files at the root are the same for video and still; only the preview folder changes.
+Each `m2mr run` creates a directory named by **the time the command started**. The files at the root are the same for video and still; only the preview folder changes.
```
outputs/20260829_193205_dance/ # or …_pose/ for a still
@@ -149,6 +163,8 @@ outputs/20260829_193205_dance/ # or …_pose/ for a still
├── skeleton_motion.npz # 3D human skeleton (reuse with --skeleton)
├── mixamo_rotations.npz # per-bone Mixamo rotations
├── mixamo_character.glb # skinned character, import into Blender / Unity
+├── diagnostics.json # self-intersection report (see Quick Start step 6)
+├── diagnostics.html # only after `m2mr diagnose <run> --visualize`
└── videos/ or images/ # previews — which one depends on the input
```
@@ -176,7 +192,7 @@ images/
# right: 10° orbit of the rig (one full turn)
```
-Open `mixamo_character.glb` in Blender: File → Import → glTF 2.0 (.glb/.gltf). If the camera is framed on empty space, select the character mesh and use View → Frame Selected. The cluster of spheres on the head and hands is bone display, not the mesh: in the Outliner, hide **Icosphere** and the **armature**; leave the skinned mesh visible. Set scene FPS and the timeline end frame from `run.json` (`fps`, `n_frames`) so playback matches the source. Compare a video run against `videos/mixamo_character.mp4`, a still run against `images/mixamo_character.png` or the orbit clip.
+Open `mixamo_character.glb` in Blender: File → Import → glTF 2.0 (.glb/.gltf). If the camera is framed on empty space, select the character mesh and use View → Frame Selected. The cluster of spheres on the head and hands is bone display, not the mesh: in the Outliner, hide **Icosphere** and the **armature**; leave the skinned mesh visible. Set scene FPS from `run.json` (`fps`) and the timeline to frames 0 to `n_frames` − 1 so playback matches the source and frame numbers match `diagnostics.json`. Compare a video run against `videos/mixamo_character.mp4`, a still run against `images/mixamo_character.png` or the orbit clip.
## License
diff --git a/blender_extension/motion2mixamorig_blender/i18n.py b/blender_extension/motion2mixamorig_blender/i18n.py
index c20f9fa..c9db47c 100644
--- a/blender_extension/motion2mixamorig_blender/i18n.py
+++ b/blender_extension/motion2mixamorig_blender/i18n.py
@@ -463,6 +463,34 @@ _add("dl_eta_unknown", en="measuring speed…", **{"zh-CN": "正在测速…"})
_add("dl_eta_under_minute", en="under 1 min left", **{"zh-CN": "不到 1 分钟"})
_add("dl_eta_minutes", en="~{minutes} min left", **{"zh-CN": "约剩 {minutes} 分钟"})
+# Diagnostics frame bar (other languages: locales/*.py).
+_add("section_diagnostics", en="Diagnostics", **{"zh-CN": "诊断"})
+_add("diag_summary", en="{n} pairs · deepest {cm} cm", **{"zh-CN": "{n} 处穿模 · 最深 {cm} cm"})
+_add("diag_clean", en="No self-intersection deeper than {cm} cm", **{"zh-CN": "没有超过 {cm} cm 的穿模"})
+_add("diag_missing", en="No diagnostics for this result. Click Update Runtime, then generate again.", **{"zh-CN": "这个结果没有诊断报告。请点“更新运行环境”,再重新生成。"})
+_add("diag_frame_bad", en="Frame {f} intersects:", **{"zh-CN": "第 {f} 帧有穿模:"})
+_add("diag_frame_ok", en="Frame {f}: no intersection", **{"zh-CN": "第 {f} 帧:没有穿模"})
+_add("diag_prev", en="Previous", **{"zh-CN": "上一个问题"})
+_add("diag_next", en="Next", **{"zh-CN": "下一个问题"})
+_add("diag_no_more", en="No more problems in that direction", **{"zh-CN": "这个方向没有更多问题了"})
+_add("diag_report", en="Open HTML Report", **{"zh-CN": "打开 HTML 报告"})
+_add("diag_report_building", en="Building report… (about 15 s)", **{"zh-CN": "正在生成报告…(约 15 秒)"})
+_add("diag_report_failed", en="Report failed. See diagnose.log in the output folder.", **{"zh-CN": "报告生成失败,请看输出文件夹里的 diagnose.log。"})
+_add("part_head", en="head", **{"zh-CN": "头"})
+_add("part_torso", en="torso", **{"zh-CN": "躯干"})
+_add("part_left_upper_arm", en="left upper arm", **{"zh-CN": "左上臂"})
+_add("part_left_forearm", en="left forearm", **{"zh-CN": "左前臂"})
+_add("part_left_hand", en="left hand", **{"zh-CN": "左手"})
+_add("part_right_upper_arm", en="right upper arm", **{"zh-CN": "右上臂"})
+_add("part_right_forearm", en="right forearm", **{"zh-CN": "右前臂"})
+_add("part_right_hand", en="right hand", **{"zh-CN": "右手"})
+_add("part_left_thigh", en="left thigh", **{"zh-CN": "左大腿"})
+_add("part_left_shin", en="left shin", **{"zh-CN": "左小腿"})
+_add("part_left_foot", en="left foot", **{"zh-CN": "左脚"})
+_add("part_right_thigh", en="right thigh", **{"zh-CN": "右大腿"})
+_add("part_right_shin", en="right shin", **{"zh-CN": "右小腿"})
+_add("part_right_foot", en="right foot", **{"zh-CN": "右脚"})
+
def _load_locales() -> None:
"""Merge locales/*.py over English, so a key missing from a locale shows in English."""
diff --git a/blender_extension/motion2mixamorig_blender/operators/__init__.py b/blender_extension/motion2mixamorig_blender/operators/__init__.py
index 6896c1c..9c8f093 100644
--- a/blender_extension/motion2mixamorig_blender/operators/__init__.py
+++ b/blender_extension/motion2mixamorig_blender/operators/__init__.py
@@ -1,4 +1,4 @@
-from . import check_environment, import_result, open_output, run_pipeline, runtime, setup_runtime
+from . import check_environment, diagnostics, import_result, open_output, run_pipeline, runtime, setup_runtime
def register() -> None:
@@ -8,10 +8,12 @@ def register() -> None:
run_pipeline.register()
import_result.register()
open_output.register()
+ diagnostics.register()
def unregister() -> None:
runtime.unregister()
+ diagnostics.unregister()
open_output.unregister()
import_result.unregister()
run_pipeline.unregister()
diff --git a/blender_extension/motion2mixamorig_blender/operators/diagnostics.py b/blender_extension/motion2mixamorig_blender/operators/diagnostics.py
new file mode 100644
index 0000000..bba02a5
--- /dev/null
+++ b/blender_extension/motion2mixamorig_blender/operators/diagnostics.py
@@ -0,0 +1,193 @@
+"""Diagnostics box actions: jump to a frame, step between problems, open the HTML report."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import bpy
+from bpy.app.handlers import persistent
+from bpy.props import IntProperty
+from bpy.types import Context, Operator
+
+from ..i18n import t
+from ..preferences import get_preferences
+from ..registration import safe_register_class, safe_unregister_class
+from ..services.backend import (
+ build_diagnose_argv,
+ directory_exists,
+ launch_process,
+ open_path,
+ python_exists,
+ subprocess_env,
+ terminate_process,
+)
+from ..services.diagnostics_reader import load_diagnostics, step_problem
+
+
+def part_label(part: str) -> str:
+ return t("part_" + part)
+
+
+def pair_label(issue: dict) -> str:
+ return " ↔ ".join(part_label(p) for p in issue.get("parts", []))
+
+
+class M2MR_OT_diag_jump(Operator):
+ bl_idname = "m2mr.diag_jump"
+ bl_label = "Go to Frame"
+ bl_description = "Go to this part of the clip"
+ bl_options = {"INTERNAL"}
+
+ frame: IntProperty(default=0)
+
+ def execute(self, context: Context):
+ context.scene.frame_set(self.frame)
+ return {"FINISHED"}
+
+
+class M2MR_OT_diag_step(Operator):
+ bl_idname = "m2mr.diag_step"
+ bl_label = "Next / Previous Problem"
+ bl_description = "Go to the next or previous frame range with an intersection"
+ bl_options = {"INTERNAL"}
+
+ direction: IntProperty(default=1)
+
+ def execute(self, context: Context):
+ report = load_diagnostics(context.scene.m2mr.job_output_dir)
+ target = step_problem(report, context.scene.frame_current, self.direction) if report else None
+ if target is None:
+ self.report({"INFO"}, t("diag_no_more"))
+ return {"CANCELLED"}
+ context.scene.frame_set(target)
+ return {"FINISHED"}
+
+
+_REPORT_HTML = "diagnostics.html"
+_REPORT_JSON = "diagnostics.json"
+_REPORT_LOG = "diagnose.log"
+_REPORT_POLL_S = 0.5
+
+# The one `m2mr diagnose --visualize` this add-on is waiting on (separate from
+# the generate job, so it does not lock the panel).
+_report: dict = {}
+
+
+def report_state(run_dir: str) -> str:
+ """'building', 'failed' or '' for the job folder shown in the panel."""
+ if not run_dir or Path(run_dir).expanduser() != _report.get("dir"):
+ return ""
+ return _report.get("state", "")
+
+
+def _report_is_current(run_dir: Path) -> bool:
+ html, data = run_dir / _REPORT_HTML, run_dir / _REPORT_JSON
+ return html.is_file() and (not data.is_file() or html.stat().st_mtime >= data.stat().st_mtime)
+
+
+class M2MR_OT_diag_report(Operator):
+ bl_idname = "m2mr.diag_report"
+ bl_label = "Open HTML Report"
+ bl_description = (
+ "Open the interactive self-intersection report in the web browser. "
+ "The first time, it is built in the background (about 15 seconds)"
+ )
+ bl_options = {"INTERNAL"}
+
+ def execute(self, context: Context):
+ run_dir = Path(context.scene.m2mr.job_output_dir).expanduser()
+ if not run_dir.is_dir():
+ self.report({"ERROR"}, t("err_output_missing"))
+ return {"CANCELLED"}
+ if _report_is_current(run_dir):
+ open_path(run_dir / _REPORT_HTML)
+ return {"FINISHED"}
+ if _report.get("state") == "building":
+ self.report({"INFO"}, t("diag_report_building"))
+ return {"CANCELLED"}
+
+ prefs = get_preferences(context)
+ if not python_exists(prefs.python_path):
+ self.report({"ERROR"}, t("err_python"))
+ return {"CANCELLED"}
+ try:
+ process, log_file = launch_process(
+ build_diagnose_argv(prefs.python_path, run_dir),
+ cwd=prefs.project_dir if directory_exists(prefs.project_dir) else run_dir,
+ log_path=run_dir / _REPORT_LOG,
+ env=subprocess_env(prefs.project_dir),
+ )
+ except OSError as exc:
+ self.report({"ERROR"}, f"{t('err_python')}: {exc}")
+ return {"CANCELLED"}
+ _report.clear()
+ _report.update(dir=run_dir, process=process, log_file=log_file, state="building")
+ if not bpy.app.timers.is_registered(_poll_report):
+ bpy.app.timers.register(_poll_report, first_interval=_REPORT_POLL_S, persistent=True)
+ self.report({"INFO"}, t("diag_report_building"))
+ return {"FINISHED"}
+
+
+def _poll_report() -> float | None:
+ process = _report.get("process")
+ if process is None:
+ return None
+ if process.poll() is None:
+ return _REPORT_POLL_S
+ try:
+ _report["log_file"].close()
+ except OSError:
+ pass
+ run_dir = _report["dir"]
+ ok = process.returncode == 0 and (run_dir / _REPORT_HTML).is_file()
+ _report.update(process=None, log_file=None, state="" if ok else "failed")
+ if ok:
+ open_path(run_dir / _REPORT_HTML)
+ _redraw_sidebar(None)
+ return None
+
+
+def _stop_report() -> None:
+ if bpy.app.timers.is_registered(_poll_report):
+ bpy.app.timers.unregister(_poll_report)
+ process = _report.get("process")
+ if process is not None and process.poll() is None:
+ terminate_process(process)
+ if _report.get("log_file") is not None:
+ try:
+ _report["log_file"].close()
+ except OSError:
+ pass
+ _report.clear()
+
+
+@persistent
+def _redraw_sidebar(scene, *_args) -> None:
+ """The frame bar marks the current frame; keep it in step with playback and scrubbing."""
+ wm = bpy.context.window_manager
+ if wm is None:
+ return
+ for window in wm.windows:
+ for area in window.screen.areas:
+ if area.type == "VIEW_3D":
+ for region in area.regions:
+ if region.type == "UI":
+ region.tag_redraw()
+
+
+CLASSES = (M2MR_OT_diag_jump, M2MR_OT_diag_step, M2MR_OT_diag_report)
+
+
+def register() -> None:
+ for cls in CLASSES:
+ safe_register_class(cls)
+ if _redraw_sidebar not in bpy.app.handlers.frame_change_post:
+ bpy.app.handlers.frame_change_post.append(_redraw_sidebar)
+
+
+def unregister() -> None:
+ _stop_report()
+ if _redraw_sidebar in bpy.app.handlers.frame_change_post:
+ bpy.app.handlers.frame_change_post.remove(_redraw_sidebar)
+ for cls in reversed(CLASSES):
+ safe_unregister_class(cls)
diff --git a/blender_extension/motion2mixamorig_blender/panels/main_panel.py b/blender_extension/motion2mixamorig_blender/panels/main_panel.py
index 027ee43..57f9fda 100644
--- a/blender_extension/motion2mixamorig_blender/panels/main_panel.py
+++ b/blender_extension/motion2mixamorig_blender/panels/main_panel.py
@@ -48,6 +48,8 @@ class M2MR_PT_main(Panel):
self._draw_setup(layout, prefs, props, busy)
self._draw_input(layout, prefs, props, busy)
self._draw_progress(layout, props, busy)
+ if not busy and props.job_output_dir:
+ _draw_diagnostics(layout, context.scene)
def _draw_setup(self, layout, prefs, props, busy: bool) -> None:
box = layout.box()
@@ -172,6 +174,93 @@ class M2MR_PT_main(Panel):
box.label(text=t("no_job_yet"))
+_CELLS = 48
+
+
+def _draw_playhead(box, cells, now: int) -> None:
+ """Triangle under the bar cell that holds the current frame."""
+ # Same buttons as the bar so the cells line up. The icon has to be on a
+ # button too: an icon label keeps its full width and pushes the row right.
+ row = box.row(align=True)
+ row.scale_y = 0.6
+ for cell in cells:
+ here = cell.start <= now <= cell.end
+ slot = row.row(align=True)
+ slot.operator(
+ "m2mr.diag_jump", text="", icon="TRIA_UP" if here else "NONE", emboss=False
+ ).frame = cell.target
+
+
+def _draw_diagnostics(layout, scene) -> None:
+ """Frame bar of the job's self-intersections: red cells are frames to fix."""
+ from ..operators.diagnostics import pair_label
+ from ..services.diagnostics_reader import frame_cells, issues_at, load_diagnostics
+
+ report = load_diagnostics(scene.m2mr.job_output_dir)
+ box = layout.box()
+ box.label(text=t("section_diagnostics"))
+ if report is None:
+ box.label(text=t("diag_missing"), icon="INFO")
+ return
+ summary = report["summary"]["self_intersection"]
+ if not report["issues"]:
+ box.label(text=t("diag_clean", cm=f"{report['threshold_cm']:g}"), icon="CHECKMARK")
+ _draw_report_button(box, scene.m2mr.job_output_dir)
+ return
+ box.label(text=t("diag_summary", n=summary["issue_count"], cm=f"{summary['max_depth_cm']:g}"), icon="ERROR")
+
+ # Frame bar: one button per cell, red where any frame in it intersects.
+ # Click a cell to go there.
+ now = scene.frame_current
+ cells = frame_cells(report, _CELLS)
+ bar = box.row(align=True)
+ bar.scale_y = 1.5
+ for cell in cells:
+ slot = bar.row(align=True)
+ slot.alert = cell.hit
+ slot.operator("m2mr.diag_jump", text="", depress=cell.start <= now <= cell.end).frame = cell.target
+ _draw_playhead(box, cells, now)
+
+ n = int(report["frame_count"])
+ # Thirds of the bar width, so the middle tick sits at the centre.
+ axis = box.split(factor=1 / 3)
+ axis.scale_y = 0.6
+ axis.label(text="0")
+ rest = axis.split(factor=0.5)
+ for value, align in (((n - 1) // 2, "CENTER"), (n - 1, "RIGHT")):
+ c = rest.row()
+ c.alignment = align
+ c.label(text=str(value))
+
+ here = issues_at(report, now)
+ status = box.column(align=True)
+ if here:
+ status.alert = True
+ status.label(text=t("diag_frame_bad", f=now), icon="ERROR")
+ for issue in sorted(here, key=lambda i: -i["max_depth_cm"]):
+ status.label(text=f" {pair_label(issue)} {issue['max_depth_cm']:.1f} cm")
+ else:
+ status.label(text=t("diag_frame_ok", f=now), icon="CHECKMARK")
+ nav = box.row(align=True)
+ nav.operator("m2mr.diag_step", text=t("diag_prev"), icon="TRIA_LEFT").direction = -1
+ nav.operator("m2mr.diag_step", text=t("diag_next"), icon="TRIA_RIGHT").direction = 1
+ _draw_report_button(box, scene.m2mr.job_output_dir)
+
+
+def _draw_report_button(box, run_dir: str) -> None:
+ from ..operators.diagnostics import report_state
+
+ state = report_state(run_dir)
+ row = box.row()
+ if state == "building":
+ row.enabled = False
+ row.operator("m2mr.diag_report", text=t("diag_report_building"), icon="SORTTIME")
+ return
+ row.operator("m2mr.diag_report", text=t("diag_report"), icon="URL")
+ if state == "failed":
+ box.label(text=t("diag_report_failed"), icon="ERROR")
+
+
def _first_run(prefs) -> bool:
"""Model weights not downloaded yet (only known once the project directory is set)."""
from ..services.weights_download import weights_complete, weights_dir
diff --git a/blender_extension/motion2mixamorig_blender/services/backend.py b/blender_extension/motion2mixamorig_blender/services/backend.py
index 3e4acec..b771e04 100644
--- a/blender_extension/motion2mixamorig_blender/services/backend.py
+++ b/blender_extension/motion2mixamorig_blender/services/backend.py
@@ -39,6 +39,18 @@ def build_doctor_argv(python: str | Path) -> list[str]:
return [os.fspath(as_user_path(python)), "-m", "motion2mixamorig.cli", "doctor"]
+def build_diagnose_argv(python: str | Path, run_dir: str | Path) -> list[str]:
+ """`m2mr diagnose <run> --visualize`: rewrites diagnostics.json and writes diagnostics.html."""
+ return [
+ os.fspath(as_user_path(python)),
+ "-m",
+ "motion2mixamorig.cli",
+ "diagnose",
+ os.fspath(as_user_path(run_dir)),
+ "--visualize",
+ ]
+
+
def build_run_argv(
python: str | Path,
*,
@@ -92,6 +104,9 @@ _BLENDER_ENV_DROP = (
)
+_EXTRA_TOOL_DIRS = ("/opt/homebrew/bin", "/usr/local/bin") if sys.platform != "win32" else ()
+
+
def subprocess_env(project_dir: str | Path) -> dict[str, str]:
"""Build an env for External Python that does not inherit Blender's interpreter."""
env = os.environ.copy()
@@ -103,6 +118,14 @@ def subprocess_env(project_dir: str | Path) -> dict[str, str]:
for part in env["PATH"].split(os.pathsep)
if "Blender.app" not in part and "Blender/" not in part
)
+ # Blender started from the Dock / Finder gets launchd's PATH
+ # (/usr/bin:/bin:/usr/sbin:/sbin), not the shell's, so Homebrew tools such
+ # as ffmpeg would be missing.
+ parts = [p for p in env.get("PATH", "").split(os.pathsep) if p]
+ for extra in _EXTRA_TOOL_DIRS:
+ if extra not in parts and os.path.isdir(extra):
+ parts.append(extra)
+ env["PATH"] = os.pathsep.join(parts)
if "LD_LIBRARY_PATH" in env:
env["LD_LIBRARY_PATH"] = os.pathsep.join(
part
diff --git a/blender_extension/motion2mixamorig_blender/services/diagnostics_reader.py b/blender_extension/motion2mixamorig_blender/services/diagnostics_reader.py
new file mode 100644
index 0000000..a0ddb00
--- /dev/null
+++ b/blender_extension/motion2mixamorig_blender/services/diagnostics_reader.py
@@ -0,0 +1,87 @@
+"""Read a job's `diagnostics.json` (written by `m2mr run`) and lay it out on a frame axis.
+
+No Blender imports. Frame numbers in the report are Blender frame numbers after
+import (the clip starts at frame 0), so they work with `scene.frame_set` as is.
+"""
+
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass
+from pathlib import Path
+
+DIAGNOSTICS_JSON = "diagnostics.json"
+
+_cache: dict[str, tuple[float, dict | None]] = {}
+
+
+def load_diagnostics(run_dir: str | Path) -> dict | None:
+ """The parsed report, or None when the run has none or it is unreadable.
+
+ Cached by modification time, so it is cheap to call from a panel draw.
+ """
+ if not run_dir:
+ return None
+ path = Path(run_dir).expanduser() / DIAGNOSTICS_JSON
+ try:
+ mtime = path.stat().st_mtime
+ except OSError:
+ return None
+ key = str(path)
+ hit = _cache.get(key)
+ if hit is not None and hit[0] == mtime:
+ return hit[1]
+ try:
+ data = json.loads(path.read_text(encoding="utf-8"))
+ report = data if isinstance(data, dict) and isinstance(data.get("issues"), list) else None
+ except (OSError, ValueError):
+ report = None
+ if report is not None:
+ report["_problem_frames"] = problem_frames(report)
+ _cache[key] = (mtime, report)
+ return report
+
+
+def problem_frames(report: dict) -> list[int]:
+ """Every frame with at least one intersection, sorted."""
+ frames: set[int] = set()
+ for issue in report.get("issues", []):
+ for a, b in issue.get("ranges", []):
+ frames.update(range(int(a), int(b) + 1))
+ return sorted(frames)
+
+
+def issues_at(report: dict, frame: int) -> list[dict]:
+ return [i for i in report.get("issues", []) if any(a <= frame <= b for a, b in i.get("ranges", []))]
+
+
+@dataclass
+class Cell:
+ start: int # first frame this cell covers
+ end: int # last frame this cell covers
+ target: int # frame to jump to: the first problem frame in the cell, else its start
+ hit: bool # any intersection in the cell
+
+
+def frame_cells(report: dict, count: int) -> list[Cell]:
+ """Split frames 0 .. n-1 into `count` equal cells for the panel's frame bar."""
+ n = max(1, int(report.get("frame_count", 1)))
+ count = max(1, min(count, n))
+ bad = report.get("_problem_frames") or problem_frames(report)
+ cells, k = [], 0
+ for c in range(count):
+ start, end = c * n // count, (c + 1) * n // count - 1
+ while k < len(bad) and bad[k] < start:
+ k += 1
+ first = bad[k] if k < len(bad) and bad[k] <= end else None
+ cells.append(Cell(start, end, first if first is not None else start, first is not None))
+ return cells
+
+
+def step_problem(report: dict, frame: int, direction: int) -> int | None:
+ """Start of the next (+1) or previous (-1) problem stretch from `frame`, or None."""
+ bad = report.get("_problem_frames") or problem_frames(report)
+ starts = [f for i, f in enumerate(bad) if i == 0 or bad[i - 1] != f - 1]
+ if direction > 0:
+ return next((f for f in starts if f > frame), None)
+ return next((f for f in reversed(starts) if f < frame), None)
diff --git a/blender_extension/motion2mixamorig_blender/services/result_reader.py b/blender_extension/motion2mixamorig_blender/services/result_reader.py
index b9d0f39..2b42afa 100644
--- a/blender_extension/motion2mixamorig_blender/services/result_reader.py
+++ b/blender_extension/motion2mixamorig_blender/services/result_reader.py
@@ -31,7 +31,11 @@ def resolve_run_path(run_dir: Path, value: object) -> Path | None:
def blender_frame_settings(fps: float, n_frames: int) -> dict[str, float | int]:
- """Map a fractional fps onto Blender's integer `fps` + `fps_base`."""
+ """Map a fractional fps onto Blender's integer `fps` + `fps_base`.
+
+ The glTF importer keys animation frame k at Blender frame k (time x fps),
+ so the clip spans frames 0 .. n-1; the timeline matches that exactly.
+ """
rate = float(fps)
frames = max(1, int(n_frames))
if abs(rate - round(rate)) < 1e-6:
@@ -43,8 +47,8 @@ def blender_frame_settings(fps: float, n_frames: int) -> dict[str, float | int]:
return {
"fps": fps_int,
"fps_base": float(fps_base),
- "frame_start": 1,
- "frame_end": frames,
+ "frame_start": 0,
+ "frame_end": frames - 1,
}
diff --git a/docs/blender-extension.md b/docs/blender-extension.md
index f965bd2..d8da3fd 100644
--- a/docs/blender-extension.md
+++ b/docs/blender-extension.md
@@ -44,10 +44,15 @@ Run this on a clean user state: move `~/Motion2MixamoRig` aside first
6. Pick a single-person still, a Mixamo FBX, Device **MPS** (or CPU).
7. Generate with preview off. The UI must stay usable (orbit the viewport).
8. Confirm stage/progress/log update, then auto-import of the GLB.
-9. Timeline start is 1; end matches `n_frames` in `run.json`; FPS matches.
+9. Timeline start is 0; end is `n_frames` − 1 from `run.json`; FPS matches.
10. **Open Output Folder** and **View Full Log** work.
-11. Start a second job and click **Cancel**. Only that job should stop.
-12. Disable and re-enable the add-on without a crash.
+11. **Diagnostics** box: red cells match `diagnostics.json`; clicking a cell,
+ **Previous** and **Next** move the playhead; during playback the triangle
+ stays under the current cell. **Open HTML Report** builds the report on the
+ first click (Blender opened from the Dock, so the source video is embedded
+ via Homebrew `ffmpeg`) and opens it at once on the second.
+12. Start a second job and click **Cancel**. Only that job should stop.
+13. Disable and re-enable the add-on without a crash.
### Windows 10/11 (not tested yet)
diff --git a/docs/user-guide.md b/docs/user-guide.md
index 8196f26..3b88720 100644
--- a/docs/user-guide.md
+++ b/docs/user-guide.md
@@ -127,6 +127,13 @@ shown next to the add-on in **Get Extensions**.
- **Input**: what to animate.
- **Progress / Result**: progress, recent log, and after a job
**Import Generated Character**, **Open Output Folder**, **View Full Log**.
+- **Diagnostics**: where the character's body parts go into each other (a
+ hand inside the head, crossed arms). Red cells in the bar are frames with a
+ problem; click one to go there. The triangle under the bar follows the
+ current frame, and below it are the parts that intersect and how deep.
+ **Previous** / **Next** jump between problems. **Open HTML Report** opens a
+ report in your web browser with the source video beside the character (the
+ first time it takes about 15 seconds to build).
The add-on speaks 18 languages. **Auto** follows Blender's language; change it
in the Setup box under **UI Language**.
diff --git a/motion2mixamorig/cli.py b/motion2mixamorig/cli.py
index 6426c85..3d541cf 100644
--- a/motion2mixamorig/cli.py
+++ b/motion2mixamorig/cli.py
@@ -357,14 +357,50 @@ def build_parser() -> argparse.ArgumentParser:
action="store_true",
help="skip preview videos/images; still write NPZ, GLB, and run.json",
)
+
+ p_diag = sub.add_parser(
+ "diagnose",
+ help="check a run's mixamo_character.glb for self-intersection -> diagnostics.json",
+ )
+ p_diag.add_argument("run_dir", help="a run folder from `m2mr run` (contains mixamo_character.glb)")
+ p_diag.add_argument(
+ "--visualize",
+ action="store_true",
+ help="also write diagnostics.html: 3D view, timeline, and clickable issue list",
+ )
+ p_diag.add_argument(
+ "--threshold-cm",
+ type=float,
+ default=None,
+ help="report penetrations deeper than this (default: 1.0)",
+ )
return parser
+def diagnose(args: argparse.Namespace) -> int:
+ from .diagnose import DEFAULT_THRESHOLD_CM, DIAGNOSTICS_HTML, DIAGNOSTICS_JSON, diagnose_run, summary_line
+
+ run_dir = Path(args.run_dir)
+ threshold = DEFAULT_THRESHOLD_CM if args.threshold_cm is None else args.threshold_cm
+ try:
+ report = diagnose_run(run_dir, threshold_cm=threshold, visualize=args.visualize)
+ except (FileNotFoundError, ValueError) as exc:
+ print(f"error: {exc}", file=sys.stderr)
+ return 1
+ print(summary_line(report))
+ print(f"report: {run_dir / DIAGNOSTICS_JSON}")
+ if args.visualize:
+ print(f"view: {run_dir / DIAGNOSTICS_HTML}")
+ return 0
+
+
def main() -> None:
parser = build_parser()
args = parser.parse_args()
if args.command == "doctor":
sys.exit(doctor())
+ if args.command == "diagnose":
+ sys.exit(diagnose(args))
sys.exit(run(args))
diff --git a/motion2mixamorig/diagnose.py b/motion2mixamorig/diagnose.py
new file mode 100644
index 0000000..6dbd937
--- /dev/null
+++ b/motion2mixamorig/diagnose.py
@@ -0,0 +1,437 @@
+"""Self-intersection diagnostics for a skinned, animated character GLB.
+
+Reads `mixamo_character.glb` itself (any skinned glTF with Mixamo bone names),
+poses every frame with the file's own skin (linear blend skinning, as a glTF
+viewer would) and reports where one body part goes into another:
+
+ diagnostics.json
+ issues: one entry per body-part pair, with the frame ranges where the
+ penetration exceeds the threshold, the worst frame, and its depth.
+
+Only self-intersection is checked. Floor contact needs a defined floor, which
+this tool does not have yet.
+
+Frame numbers are Blender frame numbers after import: frame = time x fps, so
+the first animation frame is frame 0.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+import struct
+from dataclasses import dataclass, field
+from datetime import datetime
+from pathlib import Path
+
+import numpy as np
+
+SCHEMA = "m2mr-diagnostics-v1"
+DEFAULT_THRESHOLD_CM = 1.0
+DIAGNOSTICS_JSON = "diagnostics.json"
+DIAGNOSTICS_HTML = "diagnostics.html"
+
+# Mixamo bone (namespace stripped) -> body part. Fingers go with the hand.
+_PART_OF_BONE = {
+ "Hips": "torso", "Spine": "torso", "Spine1": "torso", "Spine2": "torso",
+ "LeftShoulder": "torso", "RightShoulder": "torso",
+ "Neck": "head", "Head": "head", "HeadTop_End": "head",
+ "LeftEye": "head", "RightEye": "head",
+ "LeftArm": "left_upper_arm", "LeftForeArm": "left_forearm", "LeftHand": "left_hand",
+ "RightArm": "right_upper_arm", "RightForeArm": "right_forearm", "RightHand": "right_hand",
+ "LeftUpLeg": "left_thigh", "LeftLeg": "left_shin",
+ "LeftFoot": "left_foot", "LeftToeBase": "left_foot", "LeftToe_End": "left_foot",
+ "RightUpLeg": "right_thigh", "RightLeg": "right_shin",
+ "RightFoot": "right_foot", "RightToeBase": "right_foot", "RightToe_End": "right_foot",
+}
+PARTS = (
+ "head", "torso",
+ "left_upper_arm", "left_forearm", "left_hand",
+ "right_upper_arm", "right_forearm", "right_hand",
+ "left_thigh", "left_shin", "left_foot",
+ "right_thigh", "right_shin", "right_foot",
+)
+# Parts joined at a joint are not compared: their skins meet, blend, or (on
+# segmented rigs such as Y Bot) overlap at the joint even in the rest pose.
+_ADJACENT = {
+ frozenset(p) for p in [
+ ("head", "torso"),
+ ("torso", "left_upper_arm"), ("left_upper_arm", "left_forearm"), ("left_forearm", "left_hand"),
+ ("torso", "right_upper_arm"), ("right_upper_arm", "right_forearm"), ("right_forearm", "right_hand"),
+ ("torso", "left_thigh"), ("left_thigh", "left_shin"), ("left_shin", "left_foot"),
+ ("torso", "right_thigh"), ("right_thigh", "right_shin"), ("right_shin", "right_foot"),
+ ]
+}
+# A vertex counts as inside another part when it sits behind that part's
+# surface (against its normal) and roughly in front of the nearest surface
+# point, not off to the side of a thin edge.
+_MAX_REACH_M = 0.15
+_MIN_LATERAL_M = 0.02
+# Two stretches of one pair this many frames apart or fewer are reported as one.
+_MERGE_GAP = 3
+
+
+def part_of_bone(name: str) -> str | None:
+ """'mixamorig:LeftHandIndex2' / 'mixamorig1:LeftArm' / 'LeftArm' -> body part."""
+ base = re.sub(r"^mixamorig\d*[:_]?", "", name, flags=re.IGNORECASE)
+ if base in _PART_OF_BONE:
+ return _PART_OF_BONE[base]
+ for side in ("Left", "Right"):
+ if base.startswith(side + "Hand"):
+ return f"{side.lower()}_hand"
+ return None
+
+
+# --------------------------------------------------------------------------- GLB
+
+
+_COMPONENT = {5120: np.int8, 5121: np.uint8, 5122: np.int16, 5123: np.uint16, 5125: np.uint32, 5126: np.float32}
+_WIDTH = {"SCALAR": 1, "VEC2": 2, "VEC3": 3, "VEC4": 4, "MAT4": 16}
+
+
+def read_glb(path: Path) -> tuple[dict, bytes]:
+ data = Path(path).read_bytes()
+ magic, version, _ = struct.unpack_from("<III", data, 0)
+ if magic != 0x46546C67 or version != 2:
+ raise ValueError(f"{path} is not a glTF 2.0 binary (.glb)")
+ offset, gltf, binary = 12, None, b""
+ while offset < len(data):
+ length, kind = struct.unpack_from("<II", data, offset)
+ chunk = data[offset + 8: offset + 8 + length]
+ if kind == 0x4E4F534A:
+ gltf = json.loads(chunk)
+ elif kind == 0x004E4942:
+ binary = chunk
+ offset += 8 + length
+ if gltf is None:
+ raise ValueError(f"{path} has no JSON chunk")
+ return gltf, binary
+
+
+def _accessor(gltf: dict, binary: bytes, index: int) -> np.ndarray:
+ acc = gltf["accessors"][index]
+ view = gltf["bufferViews"][acc["bufferView"]]
+ dtype = np.dtype(_COMPONENT[acc["componentType"]])
+ width = _WIDTH[acc["type"]]
+ count = acc["count"]
+ start = view.get("byteOffset", 0) + acc.get("byteOffset", 0)
+ stride = view.get("byteStride") or dtype.itemsize * width
+ raw = np.frombuffer(binary, dtype=np.uint8, count=stride * (count - 1) + dtype.itemsize * width, offset=start)
+ rows = np.lib.stride_tricks.as_strided(raw, shape=(count, dtype.itemsize * width), strides=(stride, 1))
+ out = np.ascontiguousarray(rows).view(dtype).reshape(count, width)
+ if acc.get("normalized") and dtype.kind in "iu":
+ out = out.astype(np.float64) / np.iinfo(dtype).max
+ if acc["type"] == "MAT4":
+ out = out.reshape(count, 4, 4).transpose(0, 2, 1) # column-major -> row-major
+ return out
+
+
+def _quat_xyzw_to_mat(q: np.ndarray) -> np.ndarray:
+ x, y, z, w = (q / np.linalg.norm(q, axis=-1, keepdims=True)).T
+ return np.stack([
+ 1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w),
+ 2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w),
+ 2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y),
+ ], axis=-1).reshape(q.shape[:-1] + (3, 3))
+
+
+@dataclass
+class _Primitive:
+ mesh_node: int
+ skin: int
+ positions: np.ndarray # (V, 3) bind space, metres
+ joints: np.ndarray # (V, 4) skin-joint indices
+ weights: np.ndarray # (V, 4)
+ faces: np.ndarray # (F, 3)
+ part: np.ndarray = field(default_factory=lambda: np.zeros(0, dtype=np.int16)) # index into PARTS, -1 = none
+
+
+class SkinnedClip:
+ """The node hierarchy, skins, skinned primitives, and first animation of a GLB."""
+
+ def __init__(self, path: Path) -> None:
+ gltf, binary = read_glb(path)
+ self.path = Path(path)
+ nodes = gltf.get("nodes", [])
+ self.names = [n.get("name", f"node_{i}") for i, n in enumerate(nodes)]
+ self.parent = [-1] * len(nodes)
+ for i, n in enumerate(nodes):
+ for c in n.get("children", []):
+ self.parent[c] = i
+ self.order = self._topological()
+ self.rest_t = np.array([n.get("translation", [0, 0, 0]) for n in nodes], dtype=np.float64).reshape(-1, 3)
+ self.rest_r = np.array([n.get("rotation", [0, 0, 0, 1]) for n in nodes], dtype=np.float64).reshape(-1, 4)
+ self.rest_s = np.array([n.get("scale", [1, 1, 1]) for n in nodes], dtype=np.float64).reshape(-1, 3)
+ for i, n in enumerate(nodes):
+ if "matrix" in n:
+ raise ValueError(f"node {self.names[i]!r} uses a matrix; only TRS nodes are supported")
+
+ self.skins = []
+ for skin in gltf.get("skins", []):
+ ibm = (_accessor(gltf, binary, skin["inverseBindMatrices"]).astype(np.float64)
+ if "inverseBindMatrices" in skin else np.repeat(np.eye(4)[None], len(skin["joints"]), axis=0))
+ self.skins.append((list(skin["joints"]), ibm))
+
+ self.primitives: list[_Primitive] = []
+ for ni, node in enumerate(nodes):
+ if "mesh" not in node or "skin" not in node:
+ continue
+ for prim in gltf["meshes"][node["mesh"]]["primitives"]:
+ attrs = prim["attributes"]
+ if "JOINTS_0" not in attrs or "WEIGHTS_0" not in attrs:
+ continue
+ pos = _accessor(gltf, binary, attrs["POSITION"]).astype(np.float64)
+ faces = (_accessor(gltf, binary, prim["indices"]).reshape(-1, 3).astype(np.int64)
+ if "indices" in prim else np.arange(len(pos)).reshape(-1, 3))
+ self.primitives.append(_Primitive(
+ mesh_node=ni,
+ skin=node["skin"],
+ positions=pos,
+ joints=_accessor(gltf, binary, attrs["JOINTS_0"]).astype(np.int64),
+ weights=_accessor(gltf, binary, attrs["WEIGHTS_0"]).astype(np.float64),
+ faces=faces,
+ ))
+ if not self.primitives:
+ raise ValueError(f"{path} has no skinned mesh")
+
+ self.times = np.zeros(1)
+ self.tracks: dict[tuple[int, str], np.ndarray] = {}
+ anims = gltf.get("animations", [])
+ if anims:
+ anim = anims[0]
+ inputs = {s["input"] for s in anim["samplers"]}
+ self.times = max((_accessor(gltf, binary, i).reshape(-1).astype(np.float64) for i in inputs), key=len)
+ for ch in anim["channels"]:
+ path_ = ch["target"]["path"]
+ if path_ not in ("translation", "rotation", "scale") or "node" not in ch["target"]:
+ continue
+ sampler = anim["samplers"][ch["sampler"]]
+ t = _accessor(gltf, binary, sampler["input"]).reshape(-1).astype(np.float64)
+ v = _accessor(gltf, binary, sampler["output"]).astype(np.float64)
+ if len(t) != len(self.times) or not np.allclose(t, self.times):
+ v = np.stack([np.interp(self.times, t, v[:, k]) for k in range(v.shape[1])], axis=1)
+ self.tracks[(ch["target"]["node"], path_)] = v
+ self.n_frames = len(self.times)
+ dt = np.diff(self.times)
+ self.fps = float(1.0 / np.median(dt)) if len(dt) and np.median(dt) > 0 else 30.0
+ self._label_parts()
+
+ def _topological(self) -> list[int]:
+ order, seen = [], set()
+
+ def visit(i: int) -> None:
+ if i in seen:
+ return
+ if self.parent[i] >= 0:
+ visit(self.parent[i])
+ seen.add(i)
+ order.append(i)
+
+ for i in range(len(self.parent)):
+ visit(i)
+ return order
+
+ def _label_parts(self) -> None:
+ for prim in self.primitives:
+ joint_nodes, _ = self.skins[prim.skin]
+ part_of_joint = np.array(
+ [PARTS.index(p) if (p := part_of_bone(self.names[j])) else -1 for j in joint_nodes], dtype=np.int16
+ )
+ share = np.zeros((len(prim.positions), len(PARTS)))
+ parts = part_of_joint[prim.joints]
+ for k in range(prim.joints.shape[1]):
+ ok = parts[:, k] >= 0
+ np.add.at(share, (np.nonzero(ok)[0], parts[ok, k]), prim.weights[ok, k])
+ prim.part = np.where(share.max(axis=1) > 0, share.argmax(axis=1), -1).astype(np.int16)
+
+ def node_world(self, frame: int | None) -> np.ndarray:
+ """(N, 4, 4) world matrices; frame None = rest pose (no animation)."""
+ n = len(self.names)
+ t, r, s = self.rest_t.copy(), self.rest_r.copy(), self.rest_s.copy()
+ if frame is not None:
+ for (node, path_), values in self.tracks.items():
+ {"translation": t, "rotation": r, "scale": s}[path_][node] = values[frame]
+ local = np.repeat(np.eye(4)[None], n, axis=0)
+ local[:, :3, :3] = _quat_xyzw_to_mat(r) * s[:, None, :]
+ local[:, :3, 3] = t
+ world = np.empty_like(local)
+ for i in self.order:
+ p = self.parent[i]
+ world[i] = local[i] if p < 0 else world[p] @ local[i]
+ return world
+
+ def skinned_positions(self, frame: int | None) -> list[np.ndarray]:
+ world = self.node_world(frame)
+ out = []
+ for prim in self.primitives:
+ joint_nodes, ibm = self.skins[prim.skin]
+ palette = world[joint_nodes] @ ibm # (J, 4, 4)
+ m = np.einsum("vk,vkij->vij", prim.weights, palette[prim.joints])
+ out.append(np.einsum("vij,vj->vi", m[:, :3, :3], prim.positions) + m[:, :3, 3])
+ return out
+
+
+# ---------------------------------------------------------------- intersection
+
+
+def _vertex_normals(vertices: np.ndarray, faces: np.ndarray) -> np.ndarray:
+ p0, p1, p2 = vertices[faces[:, 0]], vertices[faces[:, 1]], vertices[faces[:, 2]]
+ face_n = np.cross(p1 - p0, p2 - p0)
+ normals = np.zeros_like(vertices)
+ for i in range(3):
+ np.add.at(normals, faces[:, i], face_n)
+ return normals / np.maximum(np.linalg.norm(normals, axis=1, keepdims=True), 1e-12)
+
+
+def _frame_depths(clip: SkinnedClip, frame: int | None) -> dict[tuple[int, int], float]:
+ """Deepest penetration (metres) per unordered part pair in one frame."""
+ from scipy.spatial import cKDTree
+
+ posed = clip.skinned_positions(frame)
+ pts = np.concatenate(posed)
+ nrm = np.concatenate([_vertex_normals(v, p.faces) for v, p in zip(posed, clip.primitives)])
+ part = np.concatenate([p.part for p in clip.primitives])
+
+ groups = {}
+ for k in range(len(PARTS)):
+ idx = np.nonzero(part == k)[0]
+ if len(idx):
+ groups[k] = {"idx": idx, "lo": pts[idx].min(0), "hi": pts[idx].max(0)}
+ trees: dict[int, cKDTree] = {}
+
+ def inside(a: int, b: int) -> float:
+ """How deep part a's vertices go into part b's surface."""
+ src, dst = groups[a]["idx"], groups[b]["idx"]
+ lo, hi = groups[b]["lo"] - 0.01, groups[b]["hi"] + 0.01
+ near = src[np.all((pts[src] >= lo) & (pts[src] <= hi), axis=1)]
+ if not len(near):
+ return 0.0
+ if b not in trees:
+ trees[b] = cKDTree(pts[dst])
+ dist, j = trees[b].query(pts[near], distance_upper_bound=_MAX_REACH_M)
+ ok = np.isfinite(dist)
+ if not ok.any():
+ return 0.0
+ p, q = pts[near[ok]], pts[dst[j[ok]]]
+ n = nrm[dst[j[ok]]]
+ d = p - q
+ along = np.einsum("ij,ij->i", d, n)
+ lateral = np.linalg.norm(d - along[:, None] * n, axis=1)
+ depth = -along
+ hit = (depth > 0) & (lateral <= np.maximum(_MIN_LATERAL_M, depth))
+ return float(depth[hit].max()) if hit.any() else 0.0
+
+ out: dict[tuple[int, int], float] = {}
+ keys = sorted(groups)
+ for i, a in enumerate(keys):
+ for b in keys[i + 1:]:
+ ga, gb = groups[a], groups[b]
+ if np.any(ga["hi"] < gb["lo"] - 0.01) or np.any(gb["hi"] < ga["lo"] - 0.01):
+ continue
+ if frozenset((PARTS[a], PARTS[b])) in _ADJACENT:
+ continue
+ depth = max(inside(a, b), inside(b, a))
+ if depth > 0:
+ out[(a, b)] = depth
+ return out
+
+
+def _ranges(frames: list[int]) -> list[list[int]]:
+ """Sorted frames -> [first, last] stretches; gaps of up to _MERGE_GAP frames are bridged."""
+ out: list[list[int]] = []
+ for f in frames:
+ if out and f - out[-1][1] <= _MERGE_GAP + 1:
+ out[-1][1] = f
+ else:
+ out.append([f, f])
+ return out
+
+
+def diagnose_glb(glb_path: Path, *, threshold_cm: float = DEFAULT_THRESHOLD_CM, verbose: bool = False) -> dict:
+ """Self-intersection report for every frame of the GLB's first animation."""
+ clip = SkinnedClip(glb_path)
+ limit_m = threshold_cm / 100.0
+ rest = {pair for pair, d in _frame_depths(clip, None).items() if d > limit_m}
+
+ per_pair: dict[tuple[int, int], dict[int, float]] = {}
+ for f in range(clip.n_frames):
+ for pair, depth in _frame_depths(clip, f).items():
+ if depth > limit_m:
+ per_pair.setdefault(pair, {})[f] = depth
+ if verbose and ((f + 1) % 200 == 0 or f + 1 == clip.n_frames):
+ print(f" diagnose {f + 1}/{clip.n_frames}")
+
+ issues = []
+ for (a, b), depths in per_pair.items():
+ frames = sorted(depths)
+ worst = max(frames, key=depths.get)
+ issues.append({
+ "kind": "self_intersection",
+ "parts": [PARTS[a], PARTS[b]],
+ "ranges": _ranges(frames),
+ "frame_count": len(frames),
+ "worst_frame": worst,
+ "max_depth_cm": round(depths[worst] * 100.0, 2),
+ "also_in_rest_pose": (a, b) in rest,
+ })
+ issues.sort(key=lambda i: (-i["max_depth_cm"], i["parts"]))
+ all_frames = sorted({f for (_, _), depths in per_pair.items() for f in depths})
+
+ return {
+ "schema": SCHEMA,
+ "source": clip.path.name,
+ "generated": datetime.now().isoformat(timespec="seconds"),
+ "frame_count": clip.n_frames,
+ "fps": round(clip.fps, 3),
+ "threshold_cm": threshold_cm,
+ "summary": {
+ "self_intersection": {
+ "issue_count": len(issues),
+ "frame_count": len(all_frames),
+ "ranges": _ranges(all_frames),
+ "max_depth_cm": max((i["max_depth_cm"] for i in issues), default=0.0),
+ }
+ },
+ "issues": issues,
+ "method": {
+ "checks": "Self-intersection only. Floor contact is not checked.",
+ "geometry": "Every frame of the GLB's first animation, posed with the file's own skin "
+ "(top-4 joint weights, linear blend skinning) — the mesh a glTF viewer shows.",
+ "parts": "Vertices are grouped into 14 body parts by the Mixamo bones that carry most of "
+ "their skin weight; fingers belong to the hand.",
+ "test": "A vertex is inside another part when it lies behind that part's nearest surface "
+ "vertex, against its normal. Depth is that distance, so it is approximate to about "
+ "the mesh's vertex spacing.",
+ "neighbours": "Parts joined at a joint (e.g. upper arm and forearm, torso and thigh) are "
+ "not compared with each other: their skins meet at the joint.",
+ "ranges": f"Stretches of one pair at most {_MERGE_GAP} frames apart are reported as one range.",
+ "rest_pose": "Pairs that already intersect in the rest pose are labelled "
+ "also_in_rest_pose, not removed.",
+ "frames": "Frame numbers match Blender after import (frame = time x fps); the first "
+ "animation frame is frame 0.",
+ },
+ }
+
+
+def summary_line(report: dict) -> str:
+ s = report["summary"]["self_intersection"]
+ if not s["issue_count"]:
+ return f"diagnostics: no self-intersection deeper than {report['threshold_cm']:g} cm"
+ return (f"diagnostics: {s['issue_count']} self-intersection(s) over {s['frame_count']} of "
+ f"{report['frame_count']} frames, deepest {s['max_depth_cm']:g} cm")
+
+
+def diagnose_run(run_dir: Path, *, threshold_cm: float = DEFAULT_THRESHOLD_CM,
+ visualize: bool = False, verbose: bool = True) -> dict:
+ """Diagnose <run_dir>/mixamo_character.glb; write diagnostics.json (and .html)."""
+ run_dir = Path(run_dir)
+ glb = run_dir / "mixamo_character.glb"
+ if not glb.is_file():
+ raise FileNotFoundError(f"{glb} not found — pass a run folder from `m2mr run`")
+ report = diagnose_glb(glb, threshold_cm=threshold_cm, verbose=verbose)
+ (run_dir / DIAGNOSTICS_JSON).write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
+ if visualize:
+ from .diagnose_report import write_html_report
+
+ write_html_report(run_dir / DIAGNOSTICS_HTML, glb, report, SkinnedClip(glb))
+ return report
diff --git a/motion2mixamorig/diagnose_report.py b/motion2mixamorig/diagnose_report.py
new file mode 100644
index 0000000..766fe6c
--- /dev/null
+++ b/motion2mixamorig/diagnose_report.py
@@ -0,0 +1,602 @@
+"""Interactive HTML view of diagnostics.json (`m2mr diagnose <run> --visualize`).
+
+One self-contained file next to the GLB: the source video (or photo) beside
+the character, both embedded and kept on the same frame, a timeline of every
+issue, and a list you can click to jump to the worst frame. Parts in an active
+issue turn red. three.js loads from a CDN, so the 3D view needs an internet
+connection; the source, list, and timeline work offline.
+"""
+
+from __future__ import annotations
+
+import base64
+import json
+import shutil
+import subprocess
+import tempfile
+from pathlib import Path
+
+from .diagnose import PARTS, SkinnedClip
+
+_THREE = "https://cdn.jsdelivr.net/npm/[email protected]"
+# The embedded copy of the source: small enough to keep the HTML a few MB.
+_SOURCE_HEIGHT = 640
+_SOURCE_FPS = 30
+
+
+def _source_video(video: Path) -> tuple[dict, str] | None:
+ """Small H.264 + AAC copy of the source, same timeline (frame k is at k / fps)."""
+ ffmpeg = shutil.which("ffmpeg")
+ if ffmpeg is None:
+ return None
+ with tempfile.TemporaryDirectory() as tmp:
+ out = Path(tmp) / "source.mp4"
+ cmd = [
+ ffmpeg, "-v", "error", "-y", "-i", str(video),
+ "-vf", f"fps={_SOURCE_FPS},scale=-2:'min({_SOURCE_HEIGHT},ih)'",
+ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "28", "-preset", "veryfast",
+ "-g", "15", "-c:a", "aac", "-b:a", "96k", "-movflags", "+faststart", str(out),
+ ]
+ if subprocess.run(cmd, capture_output=True).returncode != 0 or not out.is_file():
+ return None
+ return {"kind": "video", "mime": "video/mp4"}, base64.b64encode(out.read_bytes()).decode("ascii")
+
+
+def _source_image(image: Path) -> tuple[dict, str] | None:
+ import cv2
+
+ img = cv2.imread(str(image))
+ if img is None:
+ return None
+ h, w = img.shape[:2]
+ if h > _SOURCE_HEIGHT:
+ img = cv2.resize(img, (round(w * _SOURCE_HEIGHT / h), _SOURCE_HEIGHT), interpolation=cv2.INTER_AREA)
+ ok, buf = cv2.imencode(".jpg", img, [cv2.IMWRITE_JPEG_QUALITY, 85])
+ return ({"kind": "image", "mime": "image/jpeg"}, base64.b64encode(buf.tobytes()).decode("ascii")) if ok else None
+
+
+def source_media(run_dir: Path) -> tuple[dict, str]:
+ """The run's input video or photo (from run.json), ready to embed; or a note why not."""
+ try:
+ run = json.loads((Path(run_dir) / "run.json").read_text(encoding="utf-8"))
+ except (OSError, ValueError):
+ return {"kind": "none", "note": "run.json not found, so the source video is unknown."}, ""
+ for key, load in (("image", _source_image), ("video", _source_video)):
+ src = run.get(key)
+ if not src:
+ continue
+ if not Path(src).is_file():
+ return {"kind": "none", "note": f"Source {key} not found: {src}"}, ""
+ got = load(Path(src))
+ if got is None:
+ note = "ffmpeg is needed to embed the source video." if key == "video" else "Could not read the photo."
+ return {"kind": "none", "note": note}, ""
+ meta, b64 = got
+ meta["name"] = Path(src).name
+ return meta, b64
+ return {"kind": "none", "note": "run.json lists no source video or photo."}, ""
+
+
+def write_html_report(path: Path, glb_path: Path, report: dict, clip: SkinnedClip) -> Path:
+ media, media_b64 = source_media(Path(glb_path).parent)
+ payload = {
+ "report": report,
+ "parts": list(PARTS),
+ "media": media,
+ # Body part per vertex, per skinned mesh node, in GLB vertex order.
+ "meshes": [
+ {"name": clip.names[p.mesh_node], "count": int(len(p.part)), "part": p.part.tolist()}
+ for p in clip.primitives
+ ],
+ }
+ glb_b64 = base64.b64encode(Path(glb_path).read_bytes()).decode("ascii")
+ html = (_TEMPLATE
+ .replace("__THREE__", _THREE)
+ .replace("__DATA__", json.dumps(payload, separators=(",", ":")))
+ .replace("__MEDIA__", media_b64)
+ .replace("__GLB__", glb_b64))
+ path = Path(path)
+ path.write_text(html, encoding="utf-8")
+ return path
+
+
+_TEMPLATE = r"""<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>Motion2MixamoRig · Self-Intersection Diagnostics</title>
+<style>
+ :root {
+ --bg:#121212; --panel:#1f1f1f; --head:#2a2a2a; --field:#151515; --control:#333333; --control-hi:#3d3d3d;
+ --border:#000000; --edge:#3a3a3a; --line:#2c2c2c;
+ --text:#cfcfcf; --bright:#f0f0f0; --muted:#8a8a8a; --dim:#5f5f5f;
+ --green:#76b900; --green-hi:#8fd400; --green-sel:rgba(118,185,0,.16);
+ --red:#f0524f; --red-sel:rgba(240,82,79,.16);
+ --sans: "NVIDIA Sans", "Helvetica Neue", Arial, sans-serif;
+ --mono: ui-monospace, "SF Mono", Menlo, Consolas, monospace;
+ }
+ * { box-sizing:border-box; }
+ html, body { height:100%; }
+ body { margin:0; font:12px/1.4 var(--sans); color:var(--text); background:var(--bg); display:flex; flex-direction:column; overflow:hidden; }
+ button, input { font:inherit; color:inherit; }
+ button:focus { outline:none; }
+ button:focus-visible { outline:1px solid var(--green); outline-offset:1px; }
+
+ /* Application bar */
+ #titlebar { flex:0 0 auto; height:40px; display:flex; align-items:stretch; background:#000; border-bottom:1px solid var(--edge); }
+ .brand { display:flex; align-items:center; gap:10px; padding:0 16px 0 0; white-space:nowrap; }
+ .brand .bar { width:4px; align-self:stretch; background:var(--green); }
+ .brand .name { font-weight:700; letter-spacing:.06em; color:var(--bright); font-size:13px; }
+ .brand .sep { width:1px; height:16px; background:var(--edge); }
+ .brand .app { color:var(--muted); font-size:12px; letter-spacing:.02em; }
+ #meta { display:flex; align-items:stretch; flex:1; min-width:0; overflow:hidden; border-left:1px solid var(--line); }
+ .field { display:flex; flex-direction:column; justify-content:center; padding:0 14px; border-right:1px solid var(--line); white-space:nowrap; }
+ .field .k { font-size:9.5px; letter-spacing:.1em; text-transform:uppercase; color:var(--dim); }
+ .field .v { font-family:var(--mono); font-size:11.5px; color:var(--text); }
+ #status { display:flex; align-items:center; gap:8px; padding:0 16px; white-space:nowrap; font-weight:700; letter-spacing:.04em; text-transform:uppercase; font-size:11px; border-left:1px solid var(--line); }
+ #status::before { content:""; width:8px; height:8px; background:currentColor; }
+ #status.alert { color:var(--red); }
+ #status.ok { color:var(--green); }
+
+ /* Docked panels, 1 px apart */
+ #work { flex:1 1 auto; min-height:0; display:grid; grid-template-columns:minmax(0,1fr) 340px; gap:1px; background:var(--border); }
+ #left { min-width:0; min-height:0; display:flex; flex-direction:column; gap:1px; }
+ .panel { background:var(--panel); min-height:0; overflow:hidden; }
+ .phead { flex:0 0 auto; height:26px; display:flex; align-items:center; gap:8px; padding:0 10px; background:var(--head); border-bottom:1px solid var(--border); font-size:10.5px; font-weight:700; letter-spacing:.1em; text-transform:uppercase; color:var(--text); user-select:none; white-space:nowrap; }
+ .phead .sub { font-weight:400; letter-spacing:.02em; text-transform:none; color:var(--muted); font-size:11px; overflow:hidden; text-overflow:ellipsis; }
+ .phead[data-toggle] { cursor:pointer; }
+ .phead[data-toggle]:hover { background:#303030; }
+ .phead .tri { color:var(--muted); width:9px; font-size:9px; }
+ .phead .count { margin-left:auto; font-family:var(--mono); font-weight:400; letter-spacing:0; color:var(--muted); }
+
+ #viewports { flex:1 1 auto; min-height:0; display:flex; gap:1px; background:var(--border); }
+ .vp { position:relative; min-width:0; display:flex; flex-direction:column; background:var(--panel); }
+ .vp .body { position:relative; flex:1 1 auto; min-height:0; overflow:hidden; }
+ #source { flex:0 0 auto; max-width:46%; }
+ #source .body { background:#0a0a0a; display:flex; align-items:center; justify-content:center; }
+ #source video, #source img { display:block; width:100%; height:100%; object-fit:contain; }
+ #source .note { color:var(--muted); padding:16px; text-align:center; width:240px; }
+ #view { flex:1 1 auto; }
+ #view canvas { display:block; }
+ .hint { position:absolute; top:8px; right:10px; z-index:2; font-size:10.5px; color:var(--dim); pointer-events:none; }
+ .caption { position:absolute; left:10px; bottom:10px; z-index:2; pointer-events:none; padding:5px 10px; background:rgba(0,0,0,.72); border-left:3px solid var(--green); font-size:12px; }
+ .caption.bad { border-left-color:var(--red); }
+ .caption .bad { color:var(--red); font-weight:700; }
+ .caption .ok { color:var(--green); font-weight:700; }
+ #viewMsg { position:absolute; inset:0; display:flex; align-items:center; justify-content:center; color:var(--muted); text-align:center; padding:24px; }
+
+ /* Timeline */
+ #dock { flex:0 0 auto; display:flex; flex-direction:column; }
+ .toolbar { display:flex; align-items:center; gap:4px; padding:5px 8px; border-bottom:1px solid var(--border); background:var(--panel); flex-wrap:wrap; }
+ .btn { height:24px; min-width:26px; padding:0 8px; display:inline-flex; align-items:center; justify-content:center; gap:6px; background:var(--control); border:1px solid #444; border-radius:2px; color:var(--text); cursor:pointer; font-size:11.5px; }
+ .btn:hover { background:var(--control-hi); border-color:#555; color:var(--bright); }
+ .btn:disabled { opacity:.35; cursor:default; }
+ .btn svg { width:11px; height:11px; fill:currentColor; }
+ .btn.primary { background:var(--green); border-color:var(--green); color:#000; font-weight:700; min-width:72px; }
+ .btn.primary:hover { background:var(--green-hi); border-color:var(--green-hi); color:#000; }
+ .btn.on { border-color:var(--green); color:var(--green); }
+ .vsep { width:1px; height:18px; background:var(--edge); margin:0 6px; }
+ .framebox { display:inline-flex; align-items:center; gap:6px; height:24px; padding:0 8px; background:var(--field); border:1px solid #444; border-radius:2px; color:var(--muted); font-size:11px; }
+ .framebox input { width:44px; border:none; outline:none; text-align:right; font-family:var(--mono); font-size:12px; color:var(--bright); background:transparent; }
+ .framebox:focus-within { border-color:var(--green); }
+ .framebox .mono { font-family:var(--mono); }
+ .grow { flex:1; }
+ input[type=range] { -webkit-appearance:none; appearance:none; background:transparent; width:80px; }
+ input[type=range]::-webkit-slider-runnable-track { height:3px; background:#4a4a4a; }
+ input[type=range]::-webkit-slider-thumb { -webkit-appearance:none; width:8px; height:14px; margin-top:-5.5px; border-radius:1px; background:var(--text); }
+ input[type=range]::-moz-range-track { height:3px; background:#4a4a4a; }
+ input[type=range]::-moz-range-thumb { width:8px; height:14px; border-radius:1px; background:var(--text); border:none; }
+ #nowbar { display:flex; align-items:center; gap:6px; padding:0 10px; height:28px; border-bottom:1px solid var(--line); overflow:hidden; white-space:nowrap; }
+ #nowbar .lab { font-size:9.5px; letter-spacing:.1em; text-transform:uppercase; color:var(--dim); margin-right:4px; }
+ .tag { display:inline-block; padding:1px 7px; border:1px solid rgba(240,82,79,.55); background:var(--red-sel); color:#ff8a87; font-size:11px; border-radius:1px; }
+ .clean { color:var(--green); font-size:11.5px; }
+ #tlwrap { max-height:26vh; overflow-y:auto; overflow-x:hidden; background:var(--panel); }
+ #timeline { width:100%; display:block; cursor:pointer; }
+
+ /* Inspector */
+ #right { display:flex; flex-direction:column; }
+ .props { display:grid; grid-template-columns:118px 1fr; border-bottom:1px solid var(--border); }
+ .props > div { padding:5px 10px; border-bottom:1px solid var(--line); }
+ .props .k { color:var(--muted); }
+ .props .v { color:var(--bright); font-family:var(--mono); font-size:11.5px; }
+ .props .v.bad { color:var(--red); }
+ .filters { display:flex; padding:6px 8px; border-bottom:1px solid var(--line); }
+ .filters button { flex:1; height:22px; background:var(--control); border:1px solid #444; margin-left:-1px; color:var(--muted); cursor:pointer; font-size:11px; }
+ .filters button:first-child { margin-left:0; border-radius:2px 0 0 2px; }
+ .filters button:last-child { border-radius:0 2px 2px 0; }
+ .filters button:hover { color:var(--bright); }
+ .filters button.on { background:#3a4a22; border-color:var(--green); color:var(--bright); position:relative; z-index:1; }
+ #issuesBody { flex:1 1 auto; min-height:0; display:flex; flex-direction:column; }
+ #issues { flex:1 1 auto; overflow:auto; }
+ .issue { position:relative; padding:7px 10px 7px 12px; border-bottom:1px solid var(--line); border-left:3px solid transparent; cursor:pointer; }
+ .issue:hover { background:#262626; }
+ .issue.sel { background:var(--green-sel); border-left-color:var(--green); }
+ .issue .t { color:var(--bright); padding-right:70px; }
+ .issue .d { position:absolute; top:7px; right:10px; font-family:var(--mono); font-weight:700; color:var(--red); }
+ .issue.restpose .d { color:var(--muted); }
+ .issue .m { color:var(--muted); font-size:11px; margin-top:2px; }
+ .issue .r { color:var(--dim); font-size:10.5px; margin-top:2px; font-family:var(--mono); white-space:nowrap; overflow:hidden; text-overflow:ellipsis; }
+ .issue .badge { display:inline-block; margin-top:5px; font-size:10px; letter-spacing:.06em; text-transform:uppercase; padding:0 5px; border:1px solid var(--edge); color:var(--muted); }
+ .empty { padding:18px; text-align:center; color:var(--muted); }
+ #method { margin:0; padding:8px 12px 10px 26px; color:var(--muted); font-size:11.5px; line-height:1.55; max-height:30vh; overflow:auto; }
+ #method li { margin:4px 0; }
+ #method b { color:var(--text); }
+ .collapsed + * { display:none !important; }
+ ::-webkit-scrollbar { width:8px; height:8px; } ::-webkit-scrollbar-track { background:var(--panel); }
+ ::-webkit-scrollbar-thumb { background:#444; } ::-webkit-scrollbar-thumb:hover { background:#555; }
+ @media (max-width:980px) { body { overflow:auto; height:auto; } #work { grid-template-columns:1fr; } #viewports { height:60vh; } #meta { display:none; } }
+</style>
+<script type="importmap">{"imports":{"three":"__THREE__/build/three.module.js","three/addons/":"__THREE__/examples/jsm/"}}</script>
+</head>
+<body>
+<div id="titlebar">
+ <div class="brand"><span class="bar"></span><span class="name">MOTION2MIXAMORIG</span><span class="sep"></span><span class="app">Self-Intersection Diagnostics</span></div>
+ <div id="meta"></div>
+ <div id="status"></div>
+</div>
+
+<div id="work">
+ <div id="left">
+ <div id="viewports">
+ <div class="vp" id="source">
+ <div class="phead">Source <span class="sub" id="srcLabel"></span></div>
+ <div class="body"></div>
+ </div>
+ <div class="vp" id="view">
+ <div class="phead">Viewport <span class="sub">Character</span></div>
+ <div class="body" id="viewBody">
+ <span class="hint">LMB orbit · Wheel zoom</span>
+ <div class="caption" id="caption"></div>
+ <div id="viewMsg">Loading the 3D view…</div>
+ </div>
+ </div>
+ </div>
+
+ <div class="panel" id="dock">
+ <div class="phead">Timeline</div>
+ <div class="toolbar">
+ <button class="btn" id="first" title="First frame"><svg viewBox="0 0 16 16"><path d="M3 2h2v12H3zM13 2v12L6 8z"/></svg></button>
+ <button class="btn" id="prev" title="Previous frame (←)"><svg viewBox="0 0 16 16"><path d="M11 2v12L4 8z"/></svg></button>
+ <button class="btn primary" id="play" title="Play / Pause (Space)"></button>
+ <button class="btn" id="next" title="Next frame (→)"><svg viewBox="0 0 16 16"><path d="M5 2v12l7-6z"/></svg></button>
+ <button class="btn" id="last" title="Last frame"><svg viewBox="0 0 16 16"><path d="M11 2h2v12h-2zM3 2v12l7-6z"/></svg></button>
+ <span class="vsep"></span>
+ <span class="framebox">FRAME <input id="frameField" inputmode="numeric" title="Type a frame and press Enter"> <span class="mono" id="frameTotal"></span> <span class="mono" id="timeInfo"></span></span>
+ <div class="grow"></div>
+ <button class="btn" id="sound" title="Sound of the original video"></button>
+ <input id="volume" type="range" min="0" max="1" step="0.05" value="0.8" title="Volume">
+ </div>
+ <div id="nowbar"><span class="lab">Active</span><span id="now"></span></div>
+ <div id="tlwrap"><canvas id="timeline"></canvas></div>
+ </div>
+ </div>
+
+ <div class="panel" id="right">
+ <div class="phead" data-toggle><span class="tri">▼</span>Summary</div>
+ <div class="props" id="summary"></div>
+ <div class="phead" data-toggle><span class="tri">▼</span>Issues<span class="count" id="issueCount"></span></div>
+ <div id="issuesBody">
+ <div class="filters" id="filters">
+ <button class="on" data-f="all">All pairs</button>
+ <button data-f="motion">From motion</button>
+ <button data-f="now">This frame</button>
+ </div>
+ <div id="issues"></div>
+ </div>
+ <div class="phead collapsed" data-toggle><span class="tri">▶</span>Method</div>
+ <ul id="method"></ul>
+ </div>
+</div>
+
+<script type="application/json" id="data">__DATA__</script>
+<script type="text/plain" id="media">__MEDIA__</script>
+<script type="text/plain" id="glb">__GLB__</script>
+<script>
+const D = JSON.parse(document.getElementById('data').textContent);
+const R = D.report, N = R.frame_count, FPS = R.fps;
+const $ = id => document.getElementById(id);
+const label = p => p.replace(/_/g, ' ');
+const cap = s => s.charAt(0).toUpperCase() + s.slice(1);
+const pairName = i => cap(i.parts.map(label).join(' ↔ '));
+const fmtRange = ([a, b]) => a === b ? `${a}` : `${a}–${b}`;
+let frame = 0, selected = -1, playing = false, filter = 'all';
+window.diag = { setFrame: f => setFrame(f), onFrame: null };
+
+const ICON = {
+ play: '<svg viewBox="0 0 16 16"><path d="M4 2l10 6-10 6z"/></svg> PLAY',
+ pause: '<svg viewBox="0 0 16 16"><path d="M3 2h4v12H3zM9 2h4v12H9z"/></svg> PAUSE',
+ on: '<svg viewBox="0 0 16 16"><path d="M2 6h3l4-3v10l-4-3H2zM11 5.5a3.5 3.5 0 010 5l-.8-.8a2.4 2.4 0 000-3.4zM12.6 3.9a5.8 5.8 0 010 8.2l-.8-.8a4.7 4.7 0 000-6.6z"/></svg> Audio',
+ off: '<svg viewBox="0 0 16 16"><path d="M2 6h3l4-3v10l-4-3H2zM11 6.2l.8-.8 1.6 1.6 1.6-1.6.8.8L14.2 8l1.6 1.6-.8.8-1.6-1.6-1.6 1.6-.8-.8L12.6 8z"/></svg> Muted',
+};
+
+// Collapsible panels.
+document.querySelectorAll('[data-toggle]').forEach(h => h.onclick = () => {
+ h.classList.toggle('collapsed'); h.querySelector('.tri').textContent = h.classList.contains('collapsed') ? '▶' : '▼';
+ setTimeout(drawTimeline, 0);
+});
+
+// Original video / photo, on the same timeline (frame k at k / fps).
+const M = D.media, src = $('source'), srcBody = src.querySelector('.body');
+let video = null;
+if (M.kind === 'video' || M.kind === 'image') {
+ const el = document.createElement(M.kind === 'video' ? 'video' : 'img');
+ el.src = `data:${M.mime};base64,` + $('media').textContent.trim();
+ if (M.kind === 'video') { video = el; el.muted = true; el.playsInline = true; el.preload = 'auto'; }
+ el.addEventListener(video ? 'loadedmetadata' : 'load', () => {
+ const w = el.videoWidth || el.naturalWidth, h = el.videoHeight || el.naturalHeight;
+ if (w && h) { src.style.aspectRatio = `${w} / ${h}`; window.dispatchEvent(new Event('resize')); }
+ if (video) video.currentTime = frame / FPS;
+ });
+ srcBody.append(el);
+ $('srcLabel').textContent = M.name;
+} else {
+ src.style.width = '260px';
+ const n = document.createElement('div'); n.className = 'note'; n.textContent = M.note || 'No source video.'; srcBody.append(n);
+}
+
+// Sound: off by default (browsers block autoplay with sound); one click turns it on.
+function showSound() { $('sound').innerHTML = video && !video.muted ? ICON.on : ICON.off; $('sound').classList.toggle('on', !!video && !video.muted); }
+if (video) {
+ video.volume = +$('volume').value;
+ $('sound').onclick = () => { video.muted = !video.muted; showSound(); };
+ $('volume').oninput = e => { video.volume = +e.target.value; if (video.muted && video.volume > 0) { video.muted = false; showSound(); } };
+} else { $('sound').disabled = true; $('volume').disabled = true; }
+showSound();
+
+// Application bar and summary.
+const S = R.summary.self_intersection;
+const field = (k, v) => `<div class="field"><span class="k">${k}</span><span class="v">${v}</span></div>`;
+$('meta').innerHTML = [field('Source', R.source), field('Frames', `${N} @ ${FPS} fps`), field('Threshold', `${R.threshold_cm} cm`)].join('');
+$('status').className = S.issue_count ? 'alert' : 'ok';
+$('status').textContent = S.issue_count ? `${S.issue_count} intersections · max ${S.max_depth_cm} cm` : 'No intersection';
+const kv = [
+ ['Check', 'Self-intersection'],
+ ['Part pairs', S.issue_count, S.issue_count > 0],
+ ['Frames affected', `${S.frame_count} / ${N} (${(100 * S.frame_count / Math.max(1, N)).toFixed(1)}%)`, S.frame_count > 0],
+ ['Max depth', `${S.max_depth_cm} cm`, S.max_depth_cm > 0],
+ ['Frame numbers', 'Blender, first = 0'],
+ ['Generated', R.generated.replace('T', ' ')],
+];
+$('summary').innerHTML = kv.map(([k, v, bad]) => `<div class="k">${k}</div><div class="v${bad ? ' bad' : ''}">${v}</div>`).join('');
+$('frameTotal').textContent = `/ ${N - 1}`;
+for (const [k, v] of Object.entries(R.method)) { const li = document.createElement('li'); li.innerHTML = `<b>${cap(k.replace(/_/g, ' '))}.</b> `; li.append(v); $('method').append(li); }
+
+const inRange = (i, f) => i.ranges.some(([a, b]) => f >= a && f <= b);
+const visible = () => R.issues.filter(i =>
+ filter === 'all' || (filter === 'motion' && !i.also_in_rest_pose) || (filter === 'now' && inRange(i, frame)));
+
+function renderList() {
+ const box = $('issues'), items = visible(); box.innerHTML = '';
+ $('issueCount').textContent = `${items.length}`;
+ if (!items.length) { box.innerHTML = `<div class="empty">${filter === 'now' ? 'No intersection on this frame.' : 'No issues to show.'}</div>`; return; }
+ items.forEach(i => {
+ const idx = R.issues.indexOf(i), el = document.createElement('div');
+ el.className = 'issue' + (i.also_in_rest_pose ? ' restpose' : '') + (idx === selected ? ' sel' : '');
+ el.innerHTML = '<div class="t"></div><div class="d"></div><div class="m"></div><div class="r"></div>';
+ el.querySelector('.t').textContent = pairName(i);
+ el.querySelector('.d').textContent = `${i.max_depth_cm.toFixed(2)} cm`;
+ el.querySelector('.m').textContent = `${i.frame_count} frame${i.frame_count === 1 ? '' : 's'} · worst at frame ${i.worst_frame}`;
+ el.querySelector('.r').textContent = i.ranges.map(fmtRange).join(', ');
+ if (i.also_in_rest_pose) { const b = document.createElement('span'); b.className = 'badge'; b.textContent = 'Also in rest pose'; el.append(b); }
+ el.onclick = () => { stop(); selected = idx; setFrame(i.worst_frame); renderList(); };
+ box.append(el);
+ });
+}
+document.querySelectorAll('#filters button').forEach(b => b.onclick = () => {
+ filter = b.dataset.f; document.querySelectorAll('#filters button').forEach(x => x.classList.toggle('on', x === b)); renderList(); drawTimeline();
+});
+
+// Timeline: label column, ruler, one track per issue, green playhead.
+const TL = { label: 180, ruler: 22, row: 20, pad: 12 };
+function drawTimeline() {
+ const c = $('timeline'), items = filter === 'now' ? R.issues : visible();
+ const w = c.clientWidth || c.parentElement.clientWidth, h = TL.ruler + Math.max(1, items.length) * TL.row + 4;
+ const dpr = window.devicePixelRatio || 1;
+ c.width = w * dpr; c.height = h * dpr; c.style.height = h + 'px';
+ const g = c.getContext('2d'); g.setTransform(dpr, 0, 0, dpr, 0, 0);
+ const x0 = TL.label + TL.pad, x1 = w - TL.pad, x = f => x0 + (x1 - x0) * (N > 1 ? f / (N - 1) : 0);
+ g.clearRect(0, 0, w, h);
+ const font = '"NVIDIA Sans", "Helvetica Neue", Arial, sans-serif', mono = 'ui-monospace, "SF Mono", Menlo, Consolas, monospace';
+ g.fillStyle = '#262626'; g.fillRect(0, 0, w, TL.ruler);
+ g.fillStyle = '#000'; g.fillRect(0, TL.ruler - 1, w, 1);
+ g.fillStyle = '#1b1b1b'; g.fillRect(0, TL.ruler, TL.label, h - TL.ruler);
+ g.fillStyle = '#000'; g.fillRect(TL.label, 0, 1, h);
+ const pxPerFrame = (x1 - x0) / Math.max(1, N - 1);
+ const major = [1, 5, 10, 25, 50, 100, 250, 500, 1000].find(s => s * pxPerFrame >= 60) || 1000;
+ const minor = major >= 10 ? major / 5 : 1;
+ g.font = `10px ${mono}`; g.textBaseline = 'top'; g.textAlign = 'center';
+ for (let f = 0; f < N; f += minor) {
+ const isMajor = f % major === 0, xx = Math.round(x(f)) + .5;
+ g.strokeStyle = isMajor ? '#6a6a6a' : '#3e3e3e'; g.beginPath(); g.moveTo(xx, TL.ruler - (isMajor ? 8 : 4)); g.lineTo(xx, TL.ruler - 1); g.stroke();
+ if (isMajor) { g.fillStyle = '#8a8a8a'; g.fillText(String(f), xx, 3); g.fillStyle = 'rgba(255,255,255,.035)'; g.fillRect(xx - .5, TL.ruler, 1, h - TL.ruler); }
+ }
+ g.textAlign = 'left';
+ items.forEach((i, k) => {
+ const y = TL.ruler + k * TL.row, idx = R.issues.indexOf(i), sel = idx === selected;
+ if (sel) { g.fillStyle = 'rgba(118,185,0,.16)'; g.fillRect(0, y, w, TL.row); g.fillStyle = '#76b900'; g.fillRect(0, y, 3, TL.row); }
+ else if (k % 2) { g.fillStyle = 'rgba(255,255,255,.02)'; g.fillRect(0, y, w, TL.row); }
+ g.fillStyle = '#2c2c2c'; g.fillRect(0, y + TL.row - 1, w, 1);
+ g.fillStyle = sel ? '#f0f0f0' : (i.also_in_rest_pose ? '#6a6a6a' : '#cfcfcf'); g.textBaseline = 'middle';
+ g.font = `${sel ? 700 : 400} 11.5px ${font}`;
+ let name = pairName(i); while (g.measureText(name).width > TL.label - 16 && name.length > 4) name = name.slice(0, -2) + '…';
+ g.fillText(name, 10, y + TL.row / 2 + .5);
+ g.fillStyle = i.also_in_rest_pose ? '#5a5a5a' : '#f0524f';
+ for (const [a, b] of i.ranges) g.fillRect(Math.round(x(a)) - 1, y + 5, Math.max(3, Math.round(x(b) - x(a)) + 2), TL.row - 11);
+ });
+ const px = Math.round(x(frame)) + .5;
+ g.strokeStyle = '#76b900'; g.lineWidth = 1; g.beginPath(); g.moveTo(px, TL.ruler - 2); g.lineTo(px, h); g.stroke();
+ g.fillStyle = '#76b900'; g.fillRect(px - 17, 1, 34, TL.ruler - 4);
+ g.fillStyle = '#000'; g.font = `700 10.5px ${mono}`; g.textAlign = 'center'; g.textBaseline = 'middle'; g.fillText(String(frame), px, TL.ruler / 2 - 1); g.textAlign = 'left';
+
+ c.onmousedown = e => {
+ stop();
+ const r = c.getBoundingClientRect(), mx = e.clientX - r.left, my = e.clientY - r.top;
+ if (mx < TL.label && my > TL.ruler) {
+ const it = items[Math.floor((my - TL.ruler) / TL.row)];
+ if (it) { selected = R.issues.indexOf(it); setFrame(it.worst_frame); renderList(); }
+ return;
+ }
+ const seek = ev => setFrame(Math.round((ev.clientX - r.left - x0) / (x1 - x0) * (N - 1)));
+ seek(e);
+ const move = ev => seek(ev), up = () => { removeEventListener('mousemove', move); removeEventListener('mouseup', up); };
+ addEventListener('mousemove', move); addEventListener('mouseup', up);
+ };
+}
+
+function setFrame(f, fromVideo = false) {
+ frame = Math.max(0, Math.min(N - 1, f | 0));
+ if (video && !fromVideo && video.readyState >= 1) video.currentTime = frame / FPS;
+ if (document.activeElement !== $('frameField')) $('frameField').value = frame;
+ $('timeInfo').textContent = `${(frame / FPS).toFixed(2)} s`;
+ const now = R.issues.filter(i => inRange(i, frame));
+ $('now').innerHTML = now.length ? '' : '<span class="clean">No intersection</span>';
+ now.forEach(i => { const t = document.createElement('span'); t.className = 'tag'; t.textContent = pairName(i); $('now').append(t); });
+ $('caption').className = 'caption' + (now.length ? ' bad' : '');
+ $('caption').innerHTML = now.length ? `<span class="bad">${now.map(pairName).join(' · ')}</span>` : '<span class="ok">No intersection on this frame</span>';
+ drawTimeline();
+ if (filter === 'now') renderList();
+ if (window.diag.onFrame) window.diag.onFrame(frame, now);
+}
+
+// Transport.
+function setPlayIcon() { $('play').innerHTML = playing ? ICON.pause : ICON.play; }
+function stop() { if (playing) togglePlay(); }
+let last = 0, acc = 0;
+function togglePlay() {
+ playing = !playing; setPlayIcon();
+ if (video) { playing ? video.play() : video.pause(); }
+ last = performance.now(); if (playing) requestAnimationFrame(tick);
+}
+function tick(t) {
+ if (!playing) return;
+ if (video) {
+ const f = Math.round(video.currentTime * FPS); // the video is the clock
+ if (video.ended || f >= N) { video.currentTime = 0; video.play(); setFrame(0); }
+ else if (f !== frame) setFrame(f, true);
+ } else {
+ acc += (t - last) / 1000 * FPS; last = t;
+ if (acc >= 1) { const step = Math.floor(acc); acc -= step; setFrame(frame + step >= N ? 0 : frame + step); }
+ }
+ requestAnimationFrame(tick);
+}
+$('play').onclick = togglePlay;
+$('prev').onclick = () => { stop(); setFrame(frame - 1); };
+$('next').onclick = () => { stop(); setFrame(frame + 1); };
+$('first').onclick = () => { stop(); setFrame(0); };
+$('last').onclick = () => { stop(); setFrame(N - 1); };
+$('frameField').onkeydown = e => { if (e.key === 'Enter') { stop(); setFrame(parseInt(e.target.value, 10) || 0); e.target.blur(); } };
+addEventListener('keydown', e => {
+ if (e.target.id === 'frameField') return;
+ if (e.code === 'Space') { e.preventDefault(); togglePlay(); }
+ else if (e.key === 'ArrowLeft') { stop(); setFrame(frame - 1); }
+ else if (e.key === 'ArrowRight') { stop(); setFrame(frame + 1); }
+});
+addEventListener('resize', drawTimeline);
+setPlayIcon();
+renderList();
+setFrame(R.issues.length ? R.issues[0].worst_frame : 0);
+</script>
+
+<script type="module">
+import * as THREE from 'three';
+import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
+import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
+
+const D = JSON.parse(document.getElementById('data').textContent);
+const FPS = D.report.fps;
+const view = document.getElementById('viewBody'), msg = document.getElementById('viewMsg');
+const bytes = Uint8Array.from(atob(document.getElementById('glb').textContent.trim()), c => c.charCodeAt(0));
+
+const renderer = new THREE.WebGLRenderer({ antialias: true });
+renderer.setPixelRatio(window.devicePixelRatio || 1);
+renderer.shadowMap.enabled = true; renderer.shadowMap.type = THREE.PCFSoftShadowMap;
+view.prepend(renderer.domElement);
+
+// Dark viewport: grey void, metre grid on the floor, soft contact shadow.
+const BG = 0x2b2b2b;
+const scene = new THREE.Scene();
+scene.background = new THREE.Color(BG);
+scene.fog = new THREE.Fog(BG, 7, 22);
+const camera = new THREE.PerspectiveCamera(35, 1, 0.01, 100);
+const controls = new OrbitControls(camera, renderer.domElement);
+controls.enableDamping = true;
+scene.add(new THREE.HemisphereLight(0xffffff, 0x3a3a3a, 1.6));
+const sun = new THREE.DirectionalLight(0xffffff, 2.0);
+sun.position.set(2.5, 6, 3); sun.castShadow = true;
+sun.shadow.mapSize.set(2048, 2048); sun.shadow.radius = 6;
+Object.assign(sun.shadow.camera, { left: -2, right: 2, top: 2, bottom: -2, near: 0.5, far: 20 });
+scene.add(sun); scene.add(sun.target);
+const floor = new THREE.Group(); scene.add(floor);
+const minor = new THREE.GridHelper(40, 400, 0x353535, 0x353535), major = new THREE.GridHelper(40, 40, 0x4a4a4a, 0x4a4a4a);
+minor.position.y = 0.0005; major.position.y = 0.001;
+const ground = new THREE.Mesh(new THREE.PlaneGeometry(40, 40), new THREE.ShadowMaterial({ opacity: 0.45 }));
+ground.rotation.x = -Math.PI / 2; ground.receiveShadow = true;
+floor.add(minor, major, ground);
+
+function size() { const w = view.clientWidth, h = view.clientHeight; renderer.setSize(w, h); camera.aspect = w / Math.max(1, h); camera.updateProjectionMatrix(); }
+window.addEventListener('resize', size); size();
+
+const BASE = new THREE.Color(0x9a9c9e), HOT = new THREE.Color(0xf0524f);
+new GLTFLoader().parse(bytes.buffer, '', gltf => {
+ msg.remove();
+ scene.add(gltf.scene);
+ const mixer = new THREE.AnimationMixer(gltf.scene);
+ if (gltf.animations[0]) mixer.clipAction(gltf.animations[0]).play();
+
+ // Matte grey character; parts in an issue on this frame turn red.
+ const targets = [];
+ gltf.scene.traverse(o => {
+ if (o.isMesh) o.castShadow = true;
+ if (!o.isSkinnedMesh) return;
+ const pos = o.geometry.attributes.position;
+ const info = D.meshes.find(m => m.name === o.name && m.count === pos.count) || D.meshes.find(m => m.count === pos.count);
+ if (!info) return;
+ const col = new Float32Array(pos.count * 3);
+ for (let v = 0; v < pos.count; v++) { col[3 * v] = BASE.r; col[3 * v + 1] = BASE.g; col[3 * v + 2] = BASE.b; }
+ o.geometry.setAttribute('color', new THREE.BufferAttribute(col, 3));
+ o.material = new THREE.MeshStandardMaterial({ vertexColors: true, roughness: 0.7, metalness: 0 });
+ o.frustumCulled = false;
+ targets.push({ attr: o.geometry.attributes.color, part: info.part });
+ });
+ if (!targets.length) { const m = document.createElement('div'); m.id = 'viewMsg'; m.textContent = 'Could not match the mesh to the report.'; view.append(m); }
+
+ function paint(active) {
+ const hot = new Set();
+ active.forEach(i => i.parts.forEach(p => hot.add(D.parts.indexOf(p))));
+ for (const t of targets) {
+ const a = t.attr.array;
+ for (let v = 0; v < t.part.length; v++) {
+ const c = hot.has(t.part[v]) ? HOT : BASE;
+ a[3 * v] = c.r; a[3 * v + 1] = c.g; a[3 * v + 2] = c.b;
+ }
+ t.attr.needsUpdate = true;
+ }
+ }
+
+ mixer.setTime(0); gltf.scene.updateMatrixWorld(true);
+ const box = new THREE.Box3().setFromObject(gltf.scene), c = box.getCenter(new THREE.Vector3()), sz = box.getSize(new THREE.Vector3());
+ floor.position.y = box.min.y;
+ controls.target.copy(c);
+ camera.position.copy(c).add(new THREE.Vector3(sz.y * 0.7, sz.y * 0.75, Math.max(sz.y, sz.x) * 2.2));
+ controls.update();
+
+ // Keep the shadow camera on the character (follow the hips bone) as it moves around.
+ let hips = null; gltf.scene.traverse(o => { if (!hips && o.isBone && /hips$/i.test(o.name)) hips = o; });
+ const follow = new THREE.Vector3();
+ window.diag.onFrame = (f, active) => {
+ mixer.setTime(f / FPS); paint(active);
+ if (hips) { gltf.scene.updateMatrixWorld(true); hips.getWorldPosition(follow); } else follow.copy(c);
+ sun.target.position.set(follow.x, box.min.y, follow.z); sun.position.set(follow.x + 2.5, box.min.y + 6, follow.z + 3);
+ };
+ window.diag.setFrame(+document.getElementById('frameField').value || 0);
+ renderer.setAnimationLoop(() => { controls.update(); renderer.render(scene, camera); });
+}, err => { msg.textContent = 'Could not load the 3D view: ' + err; });
+</script>
+<script>
+// three.js comes from a CDN; without a connection the module above never runs.
+setTimeout(() => { const m = document.getElementById('viewMsg'); if (m && m.textContent.startsWith('Loading')) m.textContent = 'The 3D view needs an internet connection to load three.js. The video, timeline, and issue list work offline.'; }, 10000);
+</script>
+</body>
+</html>
+"""
diff --git a/motion2mixamorig/pipeline.py b/motion2mixamorig/pipeline.py
index eecd82a..d58dd00 100644
--- a/motion2mixamorig/pipeline.py
+++ b/motion2mixamorig/pipeline.py
@@ -22,6 +22,7 @@ import numpy as np
from . import paths
from .errors import classify_error
+from .diagnose import DIAGNOSTICS_JSON, diagnose_run, summary_line
from .human_motion import HumanMotion, load_human_motion
from .mixamo.animation import build_animation
from .mixamo.gltf import write_glb
@@ -179,6 +180,18 @@ def run_pipeline(
"mixamo_rotations": rotations_npz.name,
"mixamo_character_glb": glb_path.name,
}
+
+ # Diagnostics are a report, not a product: a failure here must not fail the run.
+ print("diagnosing self-intersection")
+ diagnostics = None
+ try:
+ diag = diagnose_run(run_dir, verbose=False)
+ diagnostics = diag["summary"]
+ outputs["diagnostics"] = DIAGNOSTICS_JSON
+ print(summary_line(diag))
+ except Exception as exc: # noqa: BLE001
+ print(f"diagnostics skipped: {exc}")
+
if not no_preview:
stage = "preview"
writer.emit("preview", 0.90, "Rendering previews")
@@ -218,6 +231,7 @@ def run_pipeline(
"rest_skin_error_cm": skin_err,
"twist_events": len(anim.twist_events),
"quat_sign_flips": len(anim.sign_events),
+ "diagnostics": diagnostics,
"outputs": outputs,
}
write_run_json(run_dir, report)
diff --git a/pyproject.toml b/pyproject.toml
index 8359a53..16daeb3 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -7,6 +7,7 @@ dependencies = [
"numpy",
"opencv-python",
"einops",
+ "scipy", # nearest-surface search in `m2mr diagnose` (also pulled in by gvhmr)
# Video -> 3D human skeleton. `preproc` = person detection/tracking,
# `rtmpose` = the ungated 2D-pose backend the pipeline uses.
"gvhmr[preproc,rtmpose]>=1.6",