Retarget human motion from video to an Adobe Mixamo-rigged 3D character.
Python
Add self-intersection diagnostics to the CLI and the Blender add-on
MaxLiu418facf9bca3
diff --git a/AGENTS.md b/AGENTS.mdindex a5960d1..3d90182 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -36,6 +36,7 @@ m2mr run --video assets/video/dance.mp4 --rig assets/mixamo/Vampire.fbxm2mr run --image assets/image/pose.jpg --rig assets/mixamo/Y_Bot.fbxm2mr run --skeleton outputs/<run>/skeleton_motion.npz --rig ... # reuse extraction, swap rigm2mr run --output-dir DIR --progress-jsonl FILE --no-preview # Blender Extension / automation+m2mr diagnose outputs/<run> [--visualize] [--threshold-cm N] # re-check self-intersection -> diagnostics.json (+ .html)````--video` / `--image` and `--rig` combine freely (not `--video` with `--image`).@@ -85,7 +86,10 @@ When helping a user:6. Jobs write `<project>/outputs/blender_jobs/<job-id>/`. After import, hide**Icosphere** and the **armature** in the Outliner; leave the**skinned_mesh_…** objects (one per FBX geometry) visible. FPS must match the source clip duration (a 60 fps video tagged as- 30 fps plays at half speed).+ 30 fps plays at half speed). The timeline runs 0..n-1, the frames+ `diagnostics.json` uses; the **Diagnostics** box under Progress reads it+ (frame bar, Previous / Next, **Open HTML Report** builds `diagnostics.html`+ on first click).Smoke-test checklist: [`docs/blender-extension.md`](docs/blender-extension.md).@@ -97,6 +101,7 @@ Each run writes `outputs/<YYYYMMDD_HHMMSS>_<source>/`:- `skeleton_motion.npz` — 3D human skeleton (24 SMPL joints, Y-up meters)- `mixamo_rotations.npz` — per-bone animation rotations (quaternion wxyz; convention string inside)- `mixamo_character.glb` — skinned character + clip (glTF 2.0; Blender: File → Import → glTF 2.0)+- `diagnostics.json` — self-intersection report on the GLB (body-part pairs, frame ranges, depth in cm; frames counted from 0); summary also in `run.json`. Parts joined at a joint are not compared. No floor check. `m2mr diagnose <run> [--visualize] [--threshold-cm N]` re-checks a run; `--visualize` adds `diagnostics.html`.- Video in: `videos/human_skeleton.mp4`, `videos/mixamo_skeleton.mp4`, `videos/mixamo_character.mp4`, `videos/compare.mp4` (2x2: original / mixamo skeleton / human skeleton / character)- Photo in: the same four views as PNG under `images/`, plus `images/before_after_360_compare.mp4` (left original / right 10° orbit of the rig)@@ -105,6 +110,8 @@ Each run writes `outputs/<YYYYMMDD_HHMMSS>_<source>/`:- `motion2mixamorig/paths.py` — all filesystem locations- `motion2mixamorig/extract.py` — video or still -> 3D human skeleton (GVHMR)- `motion2mixamorig/mixamo/` — FBX parsing, FK, retargeting, skinning, rendering+- `motion2mixamorig/diagnose.py` — self-intersection check on the GLB -> `diagnostics.json`+- `motion2mixamorig/diagnose_report.py` — `diagnostics.html` (`--visualize`)- `motion2mixamorig/pipeline.py` — one run end to end- `motion2mixamorig/cli.py` — `m2mr doctor` / `m2mr run`- `blender_extension/motion2mixamorig_blender/` — Blender 4.2+ Extension (UI + external process)diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6cc54e8..f7f00eb 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,8 +4,15 @@ All notable changes to this project will be documented in this file.## [Unreleased]+### Added++- Self-intersection diagnostics: every `m2mr run` checks the finished `mixamo_character.glb` for one body part going into another and writes `diagnostics.json` (body-part pairs, frame ranges, worst frame, depth in cm) plus a one-line summary; the summary also goes into `run.json`. A failed check does not fail the run. `m2mr diagnose <run>` re-checks a run; `--visualize` also writes `diagnostics.html` with the source video beside the character, a timeline, and a clickable issue list; `--threshold-cm` sets the reporting depth (default 1 cm). Parts joined at a joint are not compared, and stretches of one pair at most 3 frames apart are reported as one range. The floor is not checked.+- Blender add-on: a **Diagnostics** box under Progress shows the job's self-intersections as a 48-cell frame bar (red = intersecting; click a cell to go there), the parts intersecting on the current frame with their depth, and Previous / Next buttons that jump between problem ranges. **Open HTML Report** opens the interactive report in the browser; the first click builds it in the background (`m2mr diagnose --visualize`, about 15 s). Results generated before this version show a note to update the runtime and generate again.+### Fixed+- Blender add-on: after import the timeline covers the clip as frames 0 to n-1 (was 1 to n), the frame numbers the glTF importer and `diagnostics.json` use.+- Blender add-on: the external Python now finds Homebrew tools such as `ffmpeg` (`/opt/homebrew/bin`, `/usr/local/bin`) when Blender was opened from the Dock or Finder, whose PATH leaves them out. Before, the HTML report could not embed the source video.- A subject who turned away and back could stay facing away in the recovered 3D motion for the rest of the clip. The 2D left/right stabilizer decided "real turn or label flip" at the single transition frame (it required four consecutive side-view frames after the first turn) and never revisited the decision, so one mislabelled frame or one fast spin locked the wrong facing in. `motion2mixamorig/stabilize_kp2d.py` now cuts the clip into segments at side-view frames and takes the majority of RTMPose's labels inside each segment; only runs of the opposite sign shorter than ~0.33 s count as flips. No facial keypoints are used (hats and masks do not matter) and each person track is handled on its own. On a dance clip with a fast half-second turn, wrong-facing frames went from 320 of 642 to 3. Re-running on an existing cache re-stabilizes from the `vitpose_raw.pt` backup, not from the previous rule's output.## [0.1.7] - 2026-10-17diff --git a/README.md b/README.mdindex 62cdb56..0382ef9 100644--- a/README.md+++ b/README.md@@ -139,9 +139,23 @@ m2mr run --image assets/image/pose.jpg --rig assets/mixamo/Vampire.fbx`--image` with no path uses the file most recently added to `assets/image/`.+### 6. Check for self-intersection++Every run also checks the finished character for **self-intersection** — one body part going into another, like a hand inside the head — and writes `diagnostics.json`. The terminal prints one line, e.g. `diagnostics: 14 self-intersection(s) over 116 of 719 frames, deepest 7.87 cm`. The floor is not checked.++To re-check a run, or to see the problems:++```bash+m2mr diagnose outputs/<run> # rewrite diagnostics.json+m2mr diagnose outputs/<run> --visualize # also diagnostics.html: source video beside the character, timeline, clickable issue list+m2mr diagnose outputs/<run> --threshold-cm 2 # report only penetrations deeper than 2 cm (default 1)+```++Each issue in `diagnostics.json` is a pair of body parts with the frame ranges where they intersect, the worst frame, and its depth. Frame numbers count from 0, as the Blender timeline does after a glTF import. Parts joined at a joint (upper arm and forearm, torso and thigh, …) are not compared: their skins overlap around the joint in any pose. `diagnostics.html` embeds the source video (needs `ffmpeg`) and loads three.js from a CDN, so its 3D view needs an internet connection.+## Outputs-Each `m2mr run` creates a directory named by **the time the command started**. The four files at the root are the same for video and still; only the preview folder changes.+Each `m2mr run` creates a directory named by **the time the command started**. The files at the root are the same for video and still; only the preview folder changes.```outputs/20260829_193205_dance/ # or …_pose/ for a still@@ -149,6 +163,8 @@ outputs/20260829_193205_dance/ # or …_pose/ for a still├── skeleton_motion.npz # 3D human skeleton (reuse with --skeleton)├── mixamo_rotations.npz # per-bone Mixamo rotations├── mixamo_character.glb # skinned character, import into Blender / Unity+├── diagnostics.json # self-intersection report (see Quick Start step 6)+├── diagnostics.html # only after `m2mr diagnose <run> --visualize`└── videos/ or images/ # previews — which one depends on the input```@@ -176,7 +192,7 @@ images/# right: 10° orbit of the rig (one full turn)```-Open `mixamo_character.glb` in Blender: File → Import → glTF 2.0 (.glb/.gltf). If the camera is framed on empty space, select the character mesh and use View → Frame Selected. The cluster of spheres on the head and hands is bone display, not the mesh: in the Outliner, hide **Icosphere** and the **armature**; leave the skinned mesh visible. Set scene FPS and the timeline end frame from `run.json` (`fps`, `n_frames`) so playback matches the source. Compare a video run against `videos/mixamo_character.mp4`, a still run against `images/mixamo_character.png` or the orbit clip.+Open `mixamo_character.glb` in Blender: File → Import → glTF 2.0 (.glb/.gltf). If the camera is framed on empty space, select the character mesh and use View → Frame Selected. The cluster of spheres on the head and hands is bone display, not the mesh: in the Outliner, hide **Icosphere** and the **armature**; leave the skinned mesh visible. Set scene FPS from `run.json` (`fps`) and the timeline to frames 0 to `n_frames` − 1 so playback matches the source and frame numbers match `diagnostics.json`. Compare a video run against `videos/mixamo_character.mp4`, a still run against `images/mixamo_character.png` or the orbit clip.## Licensediff --git a/blender_extension/motion2mixamorig_blender/i18n.py b/blender_extension/motion2mixamorig_blender/i18n.pyindex c20f9fa..c9db47c 100644--- a/blender_extension/motion2mixamorig_blender/i18n.py+++ b/blender_extension/motion2mixamorig_blender/i18n.py@@ -463,6 +463,34 @@ _add("dl_eta_unknown", en="measuring speed…", **{"zh-CN": "正在测速…"})_add("dl_eta_under_minute", en="under 1 min left", **{"zh-CN": "不到 1 分钟"})_add("dl_eta_minutes", en="~{minutes} min left", **{"zh-CN": "约剩 {minutes} 分钟"})+# Diagnostics frame bar (other languages: locales/*.py).+_add("section_diagnostics", en="Diagnostics", **{"zh-CN": "诊断"})+_add("diag_summary", en="{n} pairs · deepest {cm} cm", **{"zh-CN": "{n} 处穿模 · 最深 {cm} cm"})+_add("diag_clean", en="No self-intersection deeper than {cm} cm", **{"zh-CN": "没有超过 {cm} cm 的穿模"})+_add("diag_missing", en="No diagnostics for this result. Click Update Runtime, then generate again.", **{"zh-CN": "这个结果没有诊断报告。请点“更新运行环境”,再重新生成。"})+_add("diag_frame_bad", en="Frame {f} intersects:", **{"zh-CN": "第 {f} 帧有穿模:"})+_add("diag_frame_ok", en="Frame {f}: no intersection", **{"zh-CN": "第 {f} 帧:没有穿模"})+_add("diag_prev", en="Previous", **{"zh-CN": "上一个问题"})+_add("diag_next", en="Next", **{"zh-CN": "下一个问题"})+_add("diag_no_more", en="No more problems in that direction", **{"zh-CN": "这个方向没有更多问题了"})+_add("diag_report", en="Open HTML Report", **{"zh-CN": "打开 HTML 报告"})+_add("diag_report_building", en="Building report… (about 15 s)", **{"zh-CN": "正在生成报告…(约 15 秒)"})+_add("diag_report_failed", en="Report failed. See diagnose.log in the output folder.", **{"zh-CN": "报告生成失败,请看输出文件夹里的 diagnose.log。"})+_add("part_head", en="head", **{"zh-CN": "头"})+_add("part_torso", en="torso", **{"zh-CN": "躯干"})+_add("part_left_upper_arm", en="left upper arm", **{"zh-CN": "左上臂"})+_add("part_left_forearm", en="left forearm", **{"zh-CN": "左前臂"})+_add("part_left_hand", en="left hand", **{"zh-CN": "左手"})+_add("part_right_upper_arm", en="right upper arm", **{"zh-CN": "右上臂"})+_add("part_right_forearm", en="right forearm", **{"zh-CN": "右前臂"})+_add("part_right_hand", en="right hand", **{"zh-CN": "右手"})+_add("part_left_thigh", en="left thigh", **{"zh-CN": "左大腿"})+_add("part_left_shin", en="left shin", **{"zh-CN": "左小腿"})+_add("part_left_foot", en="left foot", **{"zh-CN": "左脚"})+_add("part_right_thigh", en="right thigh", **{"zh-CN": "右大腿"})+_add("part_right_shin", en="right shin", **{"zh-CN": "右小腿"})+_add("part_right_foot", en="right foot", **{"zh-CN": "右脚"})+def _load_locales() -> None:"""Merge locales/*.py over English, so a key missing from a locale shows in English."""diff --git a/blender_extension/motion2mixamorig_blender/operators/__init__.py b/blender_extension/motion2mixamorig_blender/operators/__init__.pyindex 6896c1c..9c8f093 100644--- a/blender_extension/motion2mixamorig_blender/operators/__init__.py+++ b/blender_extension/motion2mixamorig_blender/operators/__init__.py@@ -1,4 +1,4 @@-from . import check_environment, import_result, open_output, run_pipeline, runtime, setup_runtime+from . import check_environment, diagnostics, import_result, open_output, run_pipeline, runtime, setup_runtimedef register() -> None:@@ -8,10 +8,12 @@ def register() -> None:run_pipeline.register()import_result.register()open_output.register()+ diagnostics.register()def unregister() -> None:runtime.unregister()+ diagnostics.unregister()open_output.unregister()import_result.unregister()run_pipeline.unregister()diff --git a/blender_extension/motion2mixamorig_blender/operators/diagnostics.py b/blender_extension/motion2mixamorig_blender/operators/diagnostics.pynew file mode 100644index 0000000..bba02a5--- /dev/null+++ b/blender_extension/motion2mixamorig_blender/operators/diagnostics.py@@ -0,0 +1,193 @@+"""Diagnostics box actions: jump to a frame, step between problems, open the HTML report."""++from __future__ import annotations++from pathlib import Path++import bpy+from bpy.app.handlers import persistent+from bpy.props import IntProperty+from bpy.types import Context, Operator++from ..i18n import t+from ..preferences import get_preferences+from ..registration import safe_register_class, safe_unregister_class+from ..services.backend import (+ build_diagnose_argv,+ directory_exists,+ launch_process,+ open_path,+ python_exists,+ subprocess_env,+ terminate_process,+)+from ..services.diagnostics_reader import load_diagnostics, step_problem+++def part_label(part: str) -> str:+ return t("part_" + part)+++def pair_label(issue: dict) -> str:+ return " ↔ ".join(part_label(p) for p in issue.get("parts", []))+++class M2MR_OT_diag_jump(Operator):+ bl_idname = "m2mr.diag_jump"+ bl_label = "Go to Frame"+ bl_description = "Go to this part of the clip"+ bl_options = {"INTERNAL"}++ frame: IntProperty(default=0)++ def execute(self, context: Context):+ context.scene.frame_set(self.frame)+ return {"FINISHED"}+++class M2MR_OT_diag_step(Operator):+ bl_idname = "m2mr.diag_step"+ bl_label = "Next / Previous Problem"+ bl_description = "Go to the next or previous frame range with an intersection"+ bl_options = {"INTERNAL"}++ direction: IntProperty(default=1)++ def execute(self, context: Context):+ report = load_diagnostics(context.scene.m2mr.job_output_dir)+ target = step_problem(report, context.scene.frame_current, self.direction) if report else None+ if target is None:+ self.report({"INFO"}, t("diag_no_more"))+ return {"CANCELLED"}+ context.scene.frame_set(target)+ return {"FINISHED"}+++_REPORT_HTML = "diagnostics.html"+_REPORT_JSON = "diagnostics.json"+_REPORT_LOG = "diagnose.log"+_REPORT_POLL_S = 0.5++# The one `m2mr diagnose --visualize` this add-on is waiting on (separate from+# the generate job, so it does not lock the panel).+_report: dict = {}+++def report_state(run_dir: str) -> str:+ """'building', 'failed' or '' for the job folder shown in the panel."""+ if not run_dir or Path(run_dir).expanduser() != _report.get("dir"):+ return ""+ return _report.get("state", "")+++def _report_is_current(run_dir: Path) -> bool:+ html, data = run_dir / _REPORT_HTML, run_dir / _REPORT_JSON+ return html.is_file() and (not data.is_file() or html.stat().st_mtime >= data.stat().st_mtime)+++class M2MR_OT_diag_report(Operator):+ bl_idname = "m2mr.diag_report"+ bl_label = "Open HTML Report"+ bl_description = (+ "Open the interactive self-intersection report in the web browser. "+ "The first time, it is built in the background (about 15 seconds)"+ )+ bl_options = {"INTERNAL"}++ def execute(self, context: Context):+ run_dir = Path(context.scene.m2mr.job_output_dir).expanduser()+ if not run_dir.is_dir():+ self.report({"ERROR"}, t("err_output_missing"))+ return {"CANCELLED"}+ if _report_is_current(run_dir):+ open_path(run_dir / _REPORT_HTML)+ return {"FINISHED"}+ if _report.get("state") == "building":+ self.report({"INFO"}, t("diag_report_building"))+ return {"CANCELLED"}++ prefs = get_preferences(context)+ if not python_exists(prefs.python_path):+ self.report({"ERROR"}, t("err_python"))+ return {"CANCELLED"}+ try:+ process, log_file = launch_process(+ build_diagnose_argv(prefs.python_path, run_dir),+ cwd=prefs.project_dir if directory_exists(prefs.project_dir) else run_dir,+ log_path=run_dir / _REPORT_LOG,+ env=subprocess_env(prefs.project_dir),+ )+ except OSError as exc:+ self.report({"ERROR"}, f"{t('err_python')}: {exc}")+ return {"CANCELLED"}+ _report.clear()+ _report.update(dir=run_dir, process=process, log_file=log_file, state="building")+ if not bpy.app.timers.is_registered(_poll_report):+ bpy.app.timers.register(_poll_report, first_interval=_REPORT_POLL_S, persistent=True)+ self.report({"INFO"}, t("diag_report_building"))+ return {"FINISHED"}+++def _poll_report() -> float | None:+ process = _report.get("process")+ if process is None:+ return None+ if process.poll() is None:+ return _REPORT_POLL_S+ try:+ _report["log_file"].close()+ except OSError:+ pass+ run_dir = _report["dir"]+ ok = process.returncode == 0 and (run_dir / _REPORT_HTML).is_file()+ _report.update(process=None, log_file=None, state="" if ok else "failed")+ if ok:+ open_path(run_dir / _REPORT_HTML)+ _redraw_sidebar(None)+ return None+++def _stop_report() -> None:+ if bpy.app.timers.is_registered(_poll_report):+ bpy.app.timers.unregister(_poll_report)+ process = _report.get("process")+ if process is not None and process.poll() is None:+ terminate_process(process)+ if _report.get("log_file") is not None:+ try:+ _report["log_file"].close()+ except OSError:+ pass+ _report.clear()+++@persistent+def _redraw_sidebar(scene, *_args) -> None:+ """The frame bar marks the current frame; keep it in step with playback and scrubbing."""+ wm = bpy.context.window_manager+ if wm is None:+ return+ for window in wm.windows:+ for area in window.screen.areas:+ if area.type == "VIEW_3D":+ for region in area.regions:+ if region.type == "UI":+ region.tag_redraw()+++CLASSES = (M2MR_OT_diag_jump, M2MR_OT_diag_step, M2MR_OT_diag_report)+++def register() -> None:+ for cls in CLASSES:+ safe_register_class(cls)+ if _redraw_sidebar not in bpy.app.handlers.frame_change_post:+ bpy.app.handlers.frame_change_post.append(_redraw_sidebar)+++def unregister() -> None:+ _stop_report()+ if _redraw_sidebar in bpy.app.handlers.frame_change_post:+ bpy.app.handlers.frame_change_post.remove(_redraw_sidebar)+ for cls in reversed(CLASSES):+ safe_unregister_class(cls)diff --git a/blender_extension/motion2mixamorig_blender/panels/main_panel.py b/blender_extension/motion2mixamorig_blender/panels/main_panel.pyindex 027ee43..57f9fda 100644--- a/blender_extension/motion2mixamorig_blender/panels/main_panel.py+++ b/blender_extension/motion2mixamorig_blender/panels/main_panel.py@@ -48,6 +48,8 @@ class M2MR_PT_main(Panel):self._draw_setup(layout, prefs, props, busy)self._draw_input(layout, prefs, props, busy)self._draw_progress(layout, props, busy)+ if not busy and props.job_output_dir:+ _draw_diagnostics(layout, context.scene)def _draw_setup(self, layout, prefs, props, busy: bool) -> None:box = layout.box()@@ -172,6 +174,93 @@ class M2MR_PT_main(Panel):box.label(text=t("no_job_yet"))+_CELLS = 48+++def _draw_playhead(box, cells, now: int) -> None:+ """Triangle under the bar cell that holds the current frame."""+ # Same buttons as the bar so the cells line up. The icon has to be on a+ # button too: an icon label keeps its full width and pushes the row right.+ row = box.row(align=True)+ row.scale_y = 0.6+ for cell in cells:+ here = cell.start <= now <= cell.end+ slot = row.row(align=True)+ slot.operator(+ "m2mr.diag_jump", text="", icon="TRIA_UP" if here else "NONE", emboss=False+ ).frame = cell.target+++def _draw_diagnostics(layout, scene) -> None:+ """Frame bar of the job's self-intersections: red cells are frames to fix."""+ from ..operators.diagnostics import pair_label+ from ..services.diagnostics_reader import frame_cells, issues_at, load_diagnostics++ report = load_diagnostics(scene.m2mr.job_output_dir)+ box = layout.box()+ box.label(text=t("section_diagnostics"))+ if report is None:+ box.label(text=t("diag_missing"), icon="INFO")+ return+ summary = report["summary"]["self_intersection"]+ if not report["issues"]:+ box.label(text=t("diag_clean", cm=f"{report['threshold_cm']:g}"), icon="CHECKMARK")+ _draw_report_button(box, scene.m2mr.job_output_dir)+ return+ box.label(text=t("diag_summary", n=summary["issue_count"], cm=f"{summary['max_depth_cm']:g}"), icon="ERROR")++ # Frame bar: one button per cell, red where any frame in it intersects.+ # Click a cell to go there.+ now = scene.frame_current+ cells = frame_cells(report, _CELLS)+ bar = box.row(align=True)+ bar.scale_y = 1.5+ for cell in cells:+ slot = bar.row(align=True)+ slot.alert = cell.hit+ slot.operator("m2mr.diag_jump", text="", depress=cell.start <= now <= cell.end).frame = cell.target+ _draw_playhead(box, cells, now)++ n = int(report["frame_count"])+ # Thirds of the bar width, so the middle tick sits at the centre.+ axis = box.split(factor=1 / 3)+ axis.scale_y = 0.6+ axis.label(text="0")+ rest = axis.split(factor=0.5)+ for value, align in (((n - 1) // 2, "CENTER"), (n - 1, "RIGHT")):+ c = rest.row()+ c.alignment = align+ c.label(text=str(value))++ here = issues_at(report, now)+ status = box.column(align=True)+ if here:+ status.alert = True+ status.label(text=t("diag_frame_bad", f=now), icon="ERROR")+ for issue in sorted(here, key=lambda i: -i["max_depth_cm"]):+ status.label(text=f" {pair_label(issue)} {issue['max_depth_cm']:.1f} cm")+ else:+ status.label(text=t("diag_frame_ok", f=now), icon="CHECKMARK")+ nav = box.row(align=True)+ nav.operator("m2mr.diag_step", text=t("diag_prev"), icon="TRIA_LEFT").direction = -1+ nav.operator("m2mr.diag_step", text=t("diag_next"), icon="TRIA_RIGHT").direction = 1+ _draw_report_button(box, scene.m2mr.job_output_dir)+++def _draw_report_button(box, run_dir: str) -> None:+ from ..operators.diagnostics import report_state++ state = report_state(run_dir)+ row = box.row()+ if state == "building":+ row.enabled = False+ row.operator("m2mr.diag_report", text=t("diag_report_building"), icon="SORTTIME")+ return+ row.operator("m2mr.diag_report", text=t("diag_report"), icon="URL")+ if state == "failed":+ box.label(text=t("diag_report_failed"), icon="ERROR")++def _first_run(prefs) -> bool:"""Model weights not downloaded yet (only known once the project directory is set)."""from ..services.weights_download import weights_complete, weights_dirdiff --git a/blender_extension/motion2mixamorig_blender/services/backend.py b/blender_extension/motion2mixamorig_blender/services/backend.pyindex 3e4acec..b771e04 100644--- a/blender_extension/motion2mixamorig_blender/services/backend.py+++ b/blender_extension/motion2mixamorig_blender/services/backend.py@@ -39,6 +39,18 @@ def build_doctor_argv(python: str | Path) -> list[str]:return [os.fspath(as_user_path(python)), "-m", "motion2mixamorig.cli", "doctor"]+def build_diagnose_argv(python: str | Path, run_dir: str | Path) -> list[str]:+ """`m2mr diagnose <run> --visualize`: rewrites diagnostics.json and writes diagnostics.html."""+ return [+ os.fspath(as_user_path(python)),+ "-m",+ "motion2mixamorig.cli",+ "diagnose",+ os.fspath(as_user_path(run_dir)),+ "--visualize",+ ]++def build_run_argv(python: str | Path,*,@@ -92,6 +104,9 @@ _BLENDER_ENV_DROP = ()+_EXTRA_TOOL_DIRS = ("/opt/homebrew/bin", "/usr/local/bin") if sys.platform != "win32" else ()++def subprocess_env(project_dir: str | Path) -> dict[str, str]:"""Build an env for External Python that does not inherit Blender's interpreter."""env = os.environ.copy()@@ -103,6 +118,14 @@ def subprocess_env(project_dir: str | Path) -> dict[str, str]:for part in env["PATH"].split(os.pathsep)if "Blender.app" not in part and "Blender/" not in part)+ # Blender started from the Dock / Finder gets launchd's PATH+ # (/usr/bin:/bin:/usr/sbin:/sbin), not the shell's, so Homebrew tools such+ # as ffmpeg would be missing.+ parts = [p for p in env.get("PATH", "").split(os.pathsep) if p]+ for extra in _EXTRA_TOOL_DIRS:+ if extra not in parts and os.path.isdir(extra):+ parts.append(extra)+ env["PATH"] = os.pathsep.join(parts)if "LD_LIBRARY_PATH" in env:env["LD_LIBRARY_PATH"] = os.pathsep.join(partdiff --git a/blender_extension/motion2mixamorig_blender/services/diagnostics_reader.py b/blender_extension/motion2mixamorig_blender/services/diagnostics_reader.pynew file mode 100644index 0000000..a0ddb00--- /dev/null+++ b/blender_extension/motion2mixamorig_blender/services/diagnostics_reader.py@@ -0,0 +1,87 @@+"""Read a job's `diagnostics.json` (written by `m2mr run`) and lay it out on a frame axis.++No Blender imports. Frame numbers in the report are Blender frame numbers after+import (the clip starts at frame 0), so they work with `scene.frame_set` as is.+"""++from __future__ import annotations++import json+from dataclasses import dataclass+from pathlib import Path++DIAGNOSTICS_JSON = "diagnostics.json"++_cache: dict[str, tuple[float, dict | None]] = {}+++def load_diagnostics(run_dir: str | Path) -> dict | None:+ """The parsed report, or None when the run has none or it is unreadable.++ Cached by modification time, so it is cheap to call from a panel draw.+ """+ if not run_dir:+ return None+ path = Path(run_dir).expanduser() / DIAGNOSTICS_JSON+ try:+ mtime = path.stat().st_mtime+ except OSError:+ return None+ key = str(path)+ hit = _cache.get(key)+ if hit is not None and hit[0] == mtime:+ return hit[1]+ try:+ data = json.loads(path.read_text(encoding="utf-8"))+ report = data if isinstance(data, dict) and isinstance(data.get("issues"), list) else None+ except (OSError, ValueError):+ report = None+ if report is not None:+ report["_problem_frames"] = problem_frames(report)+ _cache[key] = (mtime, report)+ return report+++def problem_frames(report: dict) -> list[int]:+ """Every frame with at least one intersection, sorted."""+ frames: set[int] = set()+ for issue in report.get("issues", []):+ for a, b in issue.get("ranges", []):+ frames.update(range(int(a), int(b) + 1))+ return sorted(frames)+++def issues_at(report: dict, frame: int) -> list[dict]:+ return [i for i in report.get("issues", []) if any(a <= frame <= b for a, b in i.get("ranges", []))]+++@dataclass+class Cell:+ start: int # first frame this cell covers+ end: int # last frame this cell covers+ target: int # frame to jump to: the first problem frame in the cell, else its start+ hit: bool # any intersection in the cell+++def frame_cells(report: dict, count: int) -> list[Cell]:+ """Split frames 0 .. n-1 into `count` equal cells for the panel's frame bar."""+ n = max(1, int(report.get("frame_count", 1)))+ count = max(1, min(count, n))+ bad = report.get("_problem_frames") or problem_frames(report)+ cells, k = [], 0+ for c in range(count):+ start, end = c * n // count, (c + 1) * n // count - 1+ while k < len(bad) and bad[k] < start:+ k += 1+ first = bad[k] if k < len(bad) and bad[k] <= end else None+ cells.append(Cell(start, end, first if first is not None else start, first is not None))+ return cells+++def step_problem(report: dict, frame: int, direction: int) -> int | None:+ """Start of the next (+1) or previous (-1) problem stretch from `frame`, or None."""+ bad = report.get("_problem_frames") or problem_frames(report)+ starts = [f for i, f in enumerate(bad) if i == 0 or bad[i - 1] != f - 1]+ if direction > 0:+ return next((f for f in starts if f > frame), None)+ return next((f for f in reversed(starts) if f < frame), None)diff --git a/blender_extension/motion2mixamorig_blender/services/result_reader.py b/blender_extension/motion2mixamorig_blender/services/result_reader.pyindex b9d0f39..2b42afa 100644--- a/blender_extension/motion2mixamorig_blender/services/result_reader.py+++ b/blender_extension/motion2mixamorig_blender/services/result_reader.py@@ -31,7 +31,11 @@ def resolve_run_path(run_dir: Path, value: object) -> Path | None:def blender_frame_settings(fps: float, n_frames: int) -> dict[str, float | int]:- """Map a fractional fps onto Blender's integer `fps` + `fps_base`."""+ """Map a fractional fps onto Blender's integer `fps` + `fps_base`.++ The glTF importer keys animation frame k at Blender frame k (time x fps),+ so the clip spans frames 0 .. n-1; the timeline matches that exactly.+ """rate = float(fps)frames = max(1, int(n_frames))if abs(rate - round(rate)) < 1e-6:@@ -43,8 +47,8 @@ def blender_frame_settings(fps: float, n_frames: int) -> dict[str, float | int]:return {"fps": fps_int,"fps_base": float(fps_base),- "frame_start": 1,- "frame_end": frames,+ "frame_start": 0,+ "frame_end": frames - 1,}diff --git a/docs/blender-extension.md b/docs/blender-extension.mdindex f965bd2..d8da3fd 100644--- a/docs/blender-extension.md+++ b/docs/blender-extension.md@@ -44,10 +44,15 @@ Run this on a clean user state: move `~/Motion2MixamoRig` aside first6. Pick a single-person still, a Mixamo FBX, Device **MPS** (or CPU).7. Generate with preview off. The UI must stay usable (orbit the viewport).8. Confirm stage/progress/log update, then auto-import of the GLB.-9. Timeline start is 1; end matches `n_frames` in `run.json`; FPS matches.+9. Timeline start is 0; end is `n_frames` − 1 from `run.json`; FPS matches.10. **Open Output Folder** and **View Full Log** work.-11. Start a second job and click **Cancel**. Only that job should stop.-12. Disable and re-enable the add-on without a crash.+11. **Diagnostics** box: red cells match `diagnostics.json`; clicking a cell,+ **Previous** and **Next** move the playhead; during playback the triangle+ stays under the current cell. **Open HTML Report** builds the report on the+ first click (Blender opened from the Dock, so the source video is embedded+ via Homebrew `ffmpeg`) and opens it at once on the second.+12. Start a second job and click **Cancel**. Only that job should stop.+13. Disable and re-enable the add-on without a crash.### Windows 10/11 (not tested yet)diff --git a/docs/user-guide.md b/docs/user-guide.mdindex 8196f26..3b88720 100644--- a/docs/user-guide.md+++ b/docs/user-guide.md@@ -127,6 +127,13 @@ shown next to the add-on in **Get Extensions**.- **Input**: what to animate.- **Progress / Result**: progress, recent log, and after a job**Import Generated Character**, **Open Output Folder**, **View Full Log**.+- **Diagnostics**: where the character's body parts go into each other (a+ hand inside the head, crossed arms). Red cells in the bar are frames with a+ problem; click one to go there. The triangle under the bar follows the+ current frame, and below it are the parts that intersect and how deep.+ **Previous** / **Next** jump between problems. **Open HTML Report** opens a+ report in your web browser with the source video beside the character (the+ first time it takes about 15 seconds to build).The add-on speaks 18 languages. **Auto** follows Blender's language; change itin the Setup box under **UI Language**.diff --git a/motion2mixamorig/cli.py b/motion2mixamorig/cli.pyindex 6426c85..3d541cf 100644--- a/motion2mixamorig/cli.py+++ b/motion2mixamorig/cli.py@@ -357,14 +357,50 @@ def build_parser() -> argparse.ArgumentParser:action="store_true",help="skip preview videos/images; still write NPZ, GLB, and run.json",)++ p_diag = sub.add_parser(+ "diagnose",+ help="check a run's mixamo_character.glb for self-intersection -> diagnostics.json",+ )+ p_diag.add_argument("run_dir", help="a run folder from `m2mr run` (contains mixamo_character.glb)")+ p_diag.add_argument(+ "--visualize",+ action="store_true",+ help="also write diagnostics.html: 3D view, timeline, and clickable issue list",+ )+ p_diag.add_argument(+ "--threshold-cm",+ type=float,+ default=None,+ help="report penetrations deeper than this (default: 1.0)",+ )return parser+def diagnose(args: argparse.Namespace) -> int:+ from .diagnose import DEFAULT_THRESHOLD_CM, DIAGNOSTICS_HTML, DIAGNOSTICS_JSON, diagnose_run, summary_line++ run_dir = Path(args.run_dir)+ threshold = DEFAULT_THRESHOLD_CM if args.threshold_cm is None else args.threshold_cm+ try:+ report = diagnose_run(run_dir, threshold_cm=threshold, visualize=args.visualize)+ except (FileNotFoundError, ValueError) as exc:+ print(f"error: {exc}", file=sys.stderr)+ return 1+ print(summary_line(report))+ print(f"report: {run_dir / DIAGNOSTICS_JSON}")+ if args.visualize:+ print(f"view: {run_dir / DIAGNOSTICS_HTML}")+ return 0++def main() -> None:parser = build_parser()args = parser.parse_args()if args.command == "doctor":sys.exit(doctor())+ if args.command == "diagnose":+ sys.exit(diagnose(args))sys.exit(run(args))diff --git a/motion2mixamorig/diagnose.py b/motion2mixamorig/diagnose.pynew file mode 100644index 0000000..6dbd937--- /dev/null+++ b/motion2mixamorig/diagnose.py@@ -0,0 +1,437 @@+"""Self-intersection diagnostics for a skinned, animated character GLB.++Reads `mixamo_character.glb` itself (any skinned glTF with Mixamo bone names),+poses every frame with the file's own skin (linear blend skinning, as a glTF+viewer would) and reports where one body part goes into another:++ diagnostics.json+ issues: one entry per body-part pair, with the frame ranges where the+ penetration exceeds the threshold, the worst frame, and its depth.++Only self-intersection is checked. Floor contact needs a defined floor, which+this tool does not have yet.++Frame numbers are Blender frame numbers after import: frame = time x fps, so+the first animation frame is frame 0.+"""++from __future__ import annotations++import json+import re+import struct+from dataclasses import dataclass, field+from datetime import datetime+from pathlib import Path++import numpy as np++SCHEMA = "m2mr-diagnostics-v1"+DEFAULT_THRESHOLD_CM = 1.0+DIAGNOSTICS_JSON = "diagnostics.json"+DIAGNOSTICS_HTML = "diagnostics.html"++# Mixamo bone (namespace stripped) -> body part. Fingers go with the hand.+_PART_OF_BONE = {+ "Hips": "torso", "Spine": "torso", "Spine1": "torso", "Spine2": "torso",+ "LeftShoulder": "torso", "RightShoulder": "torso",+ "Neck": "head", "Head": "head", "HeadTop_End": "head",+ "LeftEye": "head", "RightEye": "head",+ "LeftArm": "left_upper_arm", "LeftForeArm": "left_forearm", "LeftHand": "left_hand",+ "RightArm": "right_upper_arm", "RightForeArm": "right_forearm", "RightHand": "right_hand",+ "LeftUpLeg": "left_thigh", "LeftLeg": "left_shin",+ "LeftFoot": "left_foot", "LeftToeBase": "left_foot", "LeftToe_End": "left_foot",+ "RightUpLeg": "right_thigh", "RightLeg": "right_shin",+ "RightFoot": "right_foot", "RightToeBase": "right_foot", "RightToe_End": "right_foot",+}+PARTS = (+ "head", "torso",+ "left_upper_arm", "left_forearm", "left_hand",+ "right_upper_arm", "right_forearm", "right_hand",+ "left_thigh", "left_shin", "left_foot",+ "right_thigh", "right_shin", "right_foot",+)+# Parts joined at a joint are not compared: their skins meet, blend, or (on+# segmented rigs such as Y Bot) overlap at the joint even in the rest pose.+_ADJACENT = {+ frozenset(p) for p in [+ ("head", "torso"),+ ("torso", "left_upper_arm"), ("left_upper_arm", "left_forearm"), ("left_forearm", "left_hand"),+ ("torso", "right_upper_arm"), ("right_upper_arm", "right_forearm"), ("right_forearm", "right_hand"),+ ("torso", "left_thigh"), ("left_thigh", "left_shin"), ("left_shin", "left_foot"),+ ("torso", "right_thigh"), ("right_thigh", "right_shin"), ("right_shin", "right_foot"),+ ]+}+# A vertex counts as inside another part when it sits behind that part's+# surface (against its normal) and roughly in front of the nearest surface+# point, not off to the side of a thin edge.+_MAX_REACH_M = 0.15+_MIN_LATERAL_M = 0.02+# Two stretches of one pair this many frames apart or fewer are reported as one.+_MERGE_GAP = 3+++def part_of_bone(name: str) -> str | None:+ """'mixamorig:LeftHandIndex2' / 'mixamorig1:LeftArm' / 'LeftArm' -> body part."""+ base = re.sub(r"^mixamorig\d*[:_]?", "", name, flags=re.IGNORECASE)+ if base in _PART_OF_BONE:+ return _PART_OF_BONE[base]+ for side in ("Left", "Right"):+ if base.startswith(side + "Hand"):+ return f"{side.lower()}_hand"+ return None+++# --------------------------------------------------------------------------- GLB+++_COMPONENT = {5120: np.int8, 5121: np.uint8, 5122: np.int16, 5123: np.uint16, 5125: np.uint32, 5126: np.float32}+_WIDTH = {"SCALAR": 1, "VEC2": 2, "VEC3": 3, "VEC4": 4, "MAT4": 16}+++def read_glb(path: Path) -> tuple[dict, bytes]:+ data = Path(path).read_bytes()+ magic, version, _ = struct.unpack_from("<III", data, 0)+ if magic != 0x46546C67 or version != 2:+ raise ValueError(f"{path} is not a glTF 2.0 binary (.glb)")+ offset, gltf, binary = 12, None, b""+ while offset < len(data):+ length, kind = struct.unpack_from("<II", data, offset)+ chunk = data[offset + 8: offset + 8 + length]+ if kind == 0x4E4F534A:+ gltf = json.loads(chunk)+ elif kind == 0x004E4942:+ binary = chunk+ offset += 8 + length+ if gltf is None:+ raise ValueError(f"{path} has no JSON chunk")+ return gltf, binary+++def _accessor(gltf: dict, binary: bytes, index: int) -> np.ndarray:+ acc = gltf["accessors"][index]+ view = gltf["bufferViews"][acc["bufferView"]]+ dtype = np.dtype(_COMPONENT[acc["componentType"]])+ width = _WIDTH[acc["type"]]+ count = acc["count"]+ start = view.get("byteOffset", 0) + acc.get("byteOffset", 0)+ stride = view.get("byteStride") or dtype.itemsize * width+ raw = np.frombuffer(binary, dtype=np.uint8, count=stride * (count - 1) + dtype.itemsize * width, offset=start)+ rows = np.lib.stride_tricks.as_strided(raw, shape=(count, dtype.itemsize * width), strides=(stride, 1))+ out = np.ascontiguousarray(rows).view(dtype).reshape(count, width)+ if acc.get("normalized") and dtype.kind in "iu":+ out = out.astype(np.float64) / np.iinfo(dtype).max+ if acc["type"] == "MAT4":+ out = out.reshape(count, 4, 4).transpose(0, 2, 1) # column-major -> row-major+ return out+++def _quat_xyzw_to_mat(q: np.ndarray) -> np.ndarray:+ x, y, z, w = (q / np.linalg.norm(q, axis=-1, keepdims=True)).T+ return np.stack([+ 1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w),+ 2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w),+ 2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y),+ ], axis=-1).reshape(q.shape[:-1] + (3, 3))+++@dataclass+class _Primitive:+ mesh_node: int+ skin: int+ positions: np.ndarray # (V, 3) bind space, metres+ joints: np.ndarray # (V, 4) skin-joint indices+ weights: np.ndarray # (V, 4)+ faces: np.ndarray # (F, 3)+ part: np.ndarray = field(default_factory=lambda: np.zeros(0, dtype=np.int16)) # index into PARTS, -1 = none+++class SkinnedClip:+ """The node hierarchy, skins, skinned primitives, and first animation of a GLB."""++ def __init__(self, path: Path) -> None:+ gltf, binary = read_glb(path)+ self.path = Path(path)+ nodes = gltf.get("nodes", [])+ self.names = [n.get("name", f"node_{i}") for i, n in enumerate(nodes)]+ self.parent = [-1] * len(nodes)+ for i, n in enumerate(nodes):+ for c in n.get("children", []):+ self.parent[c] = i+ self.order = self._topological()+ self.rest_t = np.array([n.get("translation", [0, 0, 0]) for n in nodes], dtype=np.float64).reshape(-1, 3)+ self.rest_r = np.array([n.get("rotation", [0, 0, 0, 1]) for n in nodes], dtype=np.float64).reshape(-1, 4)+ self.rest_s = np.array([n.get("scale", [1, 1, 1]) for n in nodes], dtype=np.float64).reshape(-1, 3)+ for i, n in enumerate(nodes):+ if "matrix" in n:+ raise ValueError(f"node {self.names[i]!r} uses a matrix; only TRS nodes are supported")++ self.skins = []+ for skin in gltf.get("skins", []):+ ibm = (_accessor(gltf, binary, skin["inverseBindMatrices"]).astype(np.float64)+ if "inverseBindMatrices" in skin else np.repeat(np.eye(4)[None], len(skin["joints"]), axis=0))+ self.skins.append((list(skin["joints"]), ibm))++ self.primitives: list[_Primitive] = []+ for ni, node in enumerate(nodes):+ if "mesh" not in node or "skin" not in node:+ continue+ for prim in gltf["meshes"][node["mesh"]]["primitives"]:+ attrs = prim["attributes"]+ if "JOINTS_0" not in attrs or "WEIGHTS_0" not in attrs:+ continue+ pos = _accessor(gltf, binary, attrs["POSITION"]).astype(np.float64)+ faces = (_accessor(gltf, binary, prim["indices"]).reshape(-1, 3).astype(np.int64)+ if "indices" in prim else np.arange(len(pos)).reshape(-1, 3))+ self.primitives.append(_Primitive(+ mesh_node=ni,+ skin=node["skin"],+ positions=pos,+ joints=_accessor(gltf, binary, attrs["JOINTS_0"]).astype(np.int64),+ weights=_accessor(gltf, binary, attrs["WEIGHTS_0"]).astype(np.float64),+ faces=faces,+ ))+ if not self.primitives:+ raise ValueError(f"{path} has no skinned mesh")++ self.times = np.zeros(1)+ self.tracks: dict[tuple[int, str], np.ndarray] = {}+ anims = gltf.get("animations", [])+ if anims:+ anim = anims[0]+ inputs = {s["input"] for s in anim["samplers"]}+ self.times = max((_accessor(gltf, binary, i).reshape(-1).astype(np.float64) for i in inputs), key=len)+ for ch in anim["channels"]:+ path_ = ch["target"]["path"]+ if path_ not in ("translation", "rotation", "scale") or "node" not in ch["target"]:+ continue+ sampler = anim["samplers"][ch["sampler"]]+ t = _accessor(gltf, binary, sampler["input"]).reshape(-1).astype(np.float64)+ v = _accessor(gltf, binary, sampler["output"]).astype(np.float64)+ if len(t) != len(self.times) or not np.allclose(t, self.times):+ v = np.stack([np.interp(self.times, t, v[:, k]) for k in range(v.shape[1])], axis=1)+ self.tracks[(ch["target"]["node"], path_)] = v+ self.n_frames = len(self.times)+ dt = np.diff(self.times)+ self.fps = float(1.0 / np.median(dt)) if len(dt) and np.median(dt) > 0 else 30.0+ self._label_parts()++ def _topological(self) -> list[int]:+ order, seen = [], set()++ def visit(i: int) -> None:+ if i in seen:+ return+ if self.parent[i] >= 0:+ visit(self.parent[i])+ seen.add(i)+ order.append(i)++ for i in range(len(self.parent)):+ visit(i)+ return order++ def _label_parts(self) -> None:+ for prim in self.primitives:+ joint_nodes, _ = self.skins[prim.skin]+ part_of_joint = np.array(+ [PARTS.index(p) if (p := part_of_bone(self.names[j])) else -1 for j in joint_nodes], dtype=np.int16+ )+ share = np.zeros((len(prim.positions), len(PARTS)))+ parts = part_of_joint[prim.joints]+ for k in range(prim.joints.shape[1]):+ ok = parts[:, k] >= 0+ np.add.at(share, (np.nonzero(ok)[0], parts[ok, k]), prim.weights[ok, k])+ prim.part = np.where(share.max(axis=1) > 0, share.argmax(axis=1), -1).astype(np.int16)++ def node_world(self, frame: int | None) -> np.ndarray:+ """(N, 4, 4) world matrices; frame None = rest pose (no animation)."""+ n = len(self.names)+ t, r, s = self.rest_t.copy(), self.rest_r.copy(), self.rest_s.copy()+ if frame is not None:+ for (node, path_), values in self.tracks.items():+ {"translation": t, "rotation": r, "scale": s}[path_][node] = values[frame]+ local = np.repeat(np.eye(4)[None], n, axis=0)+ local[:, :3, :3] = _quat_xyzw_to_mat(r) * s[:, None, :]+ local[:, :3, 3] = t+ world = np.empty_like(local)+ for i in self.order:+ p = self.parent[i]+ world[i] = local[i] if p < 0 else world[p] @ local[i]+ return world++ def skinned_positions(self, frame: int | None) -> list[np.ndarray]:+ world = self.node_world(frame)+ out = []+ for prim in self.primitives:+ joint_nodes, ibm = self.skins[prim.skin]+ palette = world[joint_nodes] @ ibm # (J, 4, 4)+ m = np.einsum("vk,vkij->vij", prim.weights, palette[prim.joints])+ out.append(np.einsum("vij,vj->vi", m[:, :3, :3], prim.positions) + m[:, :3, 3])+ return out+++# ---------------------------------------------------------------- intersection+++def _vertex_normals(vertices: np.ndarray, faces: np.ndarray) -> np.ndarray:+ p0, p1, p2 = vertices[faces[:, 0]], vertices[faces[:, 1]], vertices[faces[:, 2]]+ face_n = np.cross(p1 - p0, p2 - p0)+ normals = np.zeros_like(vertices)+ for i in range(3):+ np.add.at(normals, faces[:, i], face_n)+ return normals / np.maximum(np.linalg.norm(normals, axis=1, keepdims=True), 1e-12)+++def _frame_depths(clip: SkinnedClip, frame: int | None) -> dict[tuple[int, int], float]:+ """Deepest penetration (metres) per unordered part pair in one frame."""+ from scipy.spatial import cKDTree++ posed = clip.skinned_positions(frame)+ pts = np.concatenate(posed)+ nrm = np.concatenate([_vertex_normals(v, p.faces) for v, p in zip(posed, clip.primitives)])+ part = np.concatenate([p.part for p in clip.primitives])++ groups = {}+ for k in range(len(PARTS)):+ idx = np.nonzero(part == k)[0]+ if len(idx):+ groups[k] = {"idx": idx, "lo": pts[idx].min(0), "hi": pts[idx].max(0)}+ trees: dict[int, cKDTree] = {}++ def inside(a: int, b: int) -> float:+ """How deep part a's vertices go into part b's surface."""+ src, dst = groups[a]["idx"], groups[b]["idx"]+ lo, hi = groups[b]["lo"] - 0.01, groups[b]["hi"] + 0.01+ near = src[np.all((pts[src] >= lo) & (pts[src] <= hi), axis=1)]+ if not len(near):+ return 0.0+ if b not in trees:+ trees[b] = cKDTree(pts[dst])+ dist, j = trees[b].query(pts[near], distance_upper_bound=_MAX_REACH_M)+ ok = np.isfinite(dist)+ if not ok.any():+ return 0.0+ p, q = pts[near[ok]], pts[dst[j[ok]]]+ n = nrm[dst[j[ok]]]+ d = p - q+ along = np.einsum("ij,ij->i", d, n)+ lateral = np.linalg.norm(d - along[:, None] * n, axis=1)+ depth = -along+ hit = (depth > 0) & (lateral <= np.maximum(_MIN_LATERAL_M, depth))+ return float(depth[hit].max()) if hit.any() else 0.0++ out: dict[tuple[int, int], float] = {}+ keys = sorted(groups)+ for i, a in enumerate(keys):+ for b in keys[i + 1:]:+ ga, gb = groups[a], groups[b]+ if np.any(ga["hi"] < gb["lo"] - 0.01) or np.any(gb["hi"] < ga["lo"] - 0.01):+ continue+ if frozenset((PARTS[a], PARTS[b])) in _ADJACENT:+ continue+ depth = max(inside(a, b), inside(b, a))+ if depth > 0:+ out[(a, b)] = depth+ return out+++def _ranges(frames: list[int]) -> list[list[int]]:+ """Sorted frames -> [first, last] stretches; gaps of up to _MERGE_GAP frames are bridged."""+ out: list[list[int]] = []+ for f in frames:+ if out and f - out[-1][1] <= _MERGE_GAP + 1:+ out[-1][1] = f+ else:+ out.append([f, f])+ return out+++def diagnose_glb(glb_path: Path, *, threshold_cm: float = DEFAULT_THRESHOLD_CM, verbose: bool = False) -> dict:+ """Self-intersection report for every frame of the GLB's first animation."""+ clip = SkinnedClip(glb_path)+ limit_m = threshold_cm / 100.0+ rest = {pair for pair, d in _frame_depths(clip, None).items() if d > limit_m}++ per_pair: dict[tuple[int, int], dict[int, float]] = {}+ for f in range(clip.n_frames):+ for pair, depth in _frame_depths(clip, f).items():+ if depth > limit_m:+ per_pair.setdefault(pair, {})[f] = depth+ if verbose and ((f + 1) % 200 == 0 or f + 1 == clip.n_frames):+ print(f" diagnose {f + 1}/{clip.n_frames}")++ issues = []+ for (a, b), depths in per_pair.items():+ frames = sorted(depths)+ worst = max(frames, key=depths.get)+ issues.append({+ "kind": "self_intersection",+ "parts": [PARTS[a], PARTS[b]],+ "ranges": _ranges(frames),+ "frame_count": len(frames),+ "worst_frame": worst,+ "max_depth_cm": round(depths[worst] * 100.0, 2),+ "also_in_rest_pose": (a, b) in rest,+ })+ issues.sort(key=lambda i: (-i["max_depth_cm"], i["parts"]))+ all_frames = sorted({f for (_, _), depths in per_pair.items() for f in depths})++ return {+ "schema": SCHEMA,+ "source": clip.path.name,+ "generated": datetime.now().isoformat(timespec="seconds"),+ "frame_count": clip.n_frames,+ "fps": round(clip.fps, 3),+ "threshold_cm": threshold_cm,+ "summary": {+ "self_intersection": {+ "issue_count": len(issues),+ "frame_count": len(all_frames),+ "ranges": _ranges(all_frames),+ "max_depth_cm": max((i["max_depth_cm"] for i in issues), default=0.0),+ }+ },+ "issues": issues,+ "method": {+ "checks": "Self-intersection only. Floor contact is not checked.",+ "geometry": "Every frame of the GLB's first animation, posed with the file's own skin "+ "(top-4 joint weights, linear blend skinning) — the mesh a glTF viewer shows.",+ "parts": "Vertices are grouped into 14 body parts by the Mixamo bones that carry most of "+ "their skin weight; fingers belong to the hand.",+ "test": "A vertex is inside another part when it lies behind that part's nearest surface "+ "vertex, against its normal. Depth is that distance, so it is approximate to about "+ "the mesh's vertex spacing.",+ "neighbours": "Parts joined at a joint (e.g. upper arm and forearm, torso and thigh) are "+ "not compared with each other: their skins meet at the joint.",+ "ranges": f"Stretches of one pair at most {_MERGE_GAP} frames apart are reported as one range.",+ "rest_pose": "Pairs that already intersect in the rest pose are labelled "+ "also_in_rest_pose, not removed.",+ "frames": "Frame numbers match Blender after import (frame = time x fps); the first "+ "animation frame is frame 0.",+ },+ }+++def summary_line(report: dict) -> str:+ s = report["summary"]["self_intersection"]+ if not s["issue_count"]:+ return f"diagnostics: no self-intersection deeper than {report['threshold_cm']:g} cm"+ return (f"diagnostics: {s['issue_count']} self-intersection(s) over {s['frame_count']} of "+ f"{report['frame_count']} frames, deepest {s['max_depth_cm']:g} cm")+++def diagnose_run(run_dir: Path, *, threshold_cm: float = DEFAULT_THRESHOLD_CM,+ visualize: bool = False, verbose: bool = True) -> dict:+ """Diagnose <run_dir>/mixamo_character.glb; write diagnostics.json (and .html)."""+ run_dir = Path(run_dir)+ glb = run_dir / "mixamo_character.glb"+ if not glb.is_file():+ raise FileNotFoundError(f"{glb} not found — pass a run folder from `m2mr run`")+ report = diagnose_glb(glb, threshold_cm=threshold_cm, verbose=verbose)+ (run_dir / DIAGNOSTICS_JSON).write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")+ if visualize:+ from .diagnose_report import write_html_report++ write_html_report(run_dir / DIAGNOSTICS_HTML, glb, report, SkinnedClip(glb))+ return reportdiff --git a/motion2mixamorig/diagnose_report.py b/motion2mixamorig/diagnose_report.pynew file mode 100644index 0000000..766fe6c--- /dev/null+++ b/motion2mixamorig/diagnose_report.py@@ -0,0 +1,602 @@+"""Interactive HTML view of diagnostics.json (`m2mr diagnose <run> --visualize`).++One self-contained file next to the GLB: the source video (or photo) beside+the character, both embedded and kept on the same frame, a timeline of every+issue, and a list you can click to jump to the worst frame. Parts in an active+issue turn red. three.js loads from a CDN, so the 3D view needs an internet+connection; the source, list, and timeline work offline.+"""++from __future__ import annotations++import base64+import json+import shutil+import subprocess+import tempfile+from pathlib import Path++from .diagnose import PARTS, SkinnedClip++_THREE = "https://cdn.jsdelivr.net/npm/[email protected]"+# The embedded copy of the source: small enough to keep the HTML a few MB.+_SOURCE_HEIGHT = 640+_SOURCE_FPS = 30+++def _source_video(video: Path) -> tuple[dict, str] | None:+ """Small H.264 + AAC copy of the source, same timeline (frame k is at k / fps)."""+ ffmpeg = shutil.which("ffmpeg")+ if ffmpeg is None:+ return None+ with tempfile.TemporaryDirectory() as tmp:+ out = Path(tmp) / "source.mp4"+ cmd = [+ ffmpeg, "-v", "error", "-y", "-i", str(video),+ "-vf", f"fps={_SOURCE_FPS},scale=-2:'min({_SOURCE_HEIGHT},ih)'",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "28", "-preset", "veryfast",+ "-g", "15", "-c:a", "aac", "-b:a", "96k", "-movflags", "+faststart", str(out),+ ]+ if subprocess.run(cmd, capture_output=True).returncode != 0 or not out.is_file():+ return None+ return {"kind": "video", "mime": "video/mp4"}, base64.b64encode(out.read_bytes()).decode("ascii")+++def _source_image(image: Path) -> tuple[dict, str] | None:+ import cv2++ img = cv2.imread(str(image))+ if img is None:+ return None+ h, w = img.shape[:2]+ if h > _SOURCE_HEIGHT:+ img = cv2.resize(img, (round(w * _SOURCE_HEIGHT / h), _SOURCE_HEIGHT), interpolation=cv2.INTER_AREA)+ ok, buf = cv2.imencode(".jpg", img, [cv2.IMWRITE_JPEG_QUALITY, 85])+ return ({"kind": "image", "mime": "image/jpeg"}, base64.b64encode(buf.tobytes()).decode("ascii")) if ok else None+++def source_media(run_dir: Path) -> tuple[dict, str]:+ """The run's input video or photo (from run.json), ready to embed; or a note why not."""+ try:+ run = json.loads((Path(run_dir) / "run.json").read_text(encoding="utf-8"))+ except (OSError, ValueError):+ return {"kind": "none", "note": "run.json not found, so the source video is unknown."}, ""+ for key, load in (("image", _source_image), ("video", _source_video)):+ src = run.get(key)+ if not src:+ continue+ if not Path(src).is_file():+ return {"kind": "none", "note": f"Source {key} not found: {src}"}, ""+ got = load(Path(src))+ if got is None:+ note = "ffmpeg is needed to embed the source video." if key == "video" else "Could not read the photo."+ return {"kind": "none", "note": note}, ""+ meta, b64 = got+ meta["name"] = Path(src).name+ return meta, b64+ return {"kind": "none", "note": "run.json lists no source video or photo."}, ""+++def write_html_report(path: Path, glb_path: Path, report: dict, clip: SkinnedClip) -> Path:+ media, media_b64 = source_media(Path(glb_path).parent)+ payload = {+ "report": report,+ "parts": list(PARTS),+ "media": media,+ # Body part per vertex, per skinned mesh node, in GLB vertex order.+ "meshes": [+ {"name": clip.names[p.mesh_node], "count": int(len(p.part)), "part": p.part.tolist()}+ for p in clip.primitives+ ],+ }+ glb_b64 = base64.b64encode(Path(glb_path).read_bytes()).decode("ascii")+ html = (_TEMPLATE+ .replace("__THREE__", _THREE)+ .replace("__DATA__", json.dumps(payload, separators=(",", ":")))+ .replace("__MEDIA__", media_b64)+ .replace("__GLB__", glb_b64))+ path = Path(path)+ path.write_text(html, encoding="utf-8")+ return path+++_TEMPLATE = r"""<!doctype html>+<html lang="en">+<head>+<meta charset="utf-8">+<meta name="viewport" content="width=device-width, initial-scale=1">+<title>Motion2MixamoRig · Self-Intersection Diagnostics</title>+<style>+ :root {+ --bg:#121212; --panel:#1f1f1f; --head:#2a2a2a; --field:#151515; --control:#333333; --control-hi:#3d3d3d;+ --border:#000000; --edge:#3a3a3a; --line:#2c2c2c;+ --text:#cfcfcf; --bright:#f0f0f0; --muted:#8a8a8a; --dim:#5f5f5f;+ --green:#76b900; --green-hi:#8fd400; --green-sel:rgba(118,185,0,.16);+ --red:#f0524f; --red-sel:rgba(240,82,79,.16);+ --sans: "NVIDIA Sans", "Helvetica Neue", Arial, sans-serif;+ --mono: ui-monospace, "SF Mono", Menlo, Consolas, monospace;+ }+ * { box-sizing:border-box; }+ html, body { height:100%; }+ body { margin:0; font:12px/1.4 var(--sans); color:var(--text); background:var(--bg); display:flex; flex-direction:column; overflow:hidden; }+ button, input { font:inherit; color:inherit; }+ button:focus { outline:none; }+ button:focus-visible { outline:1px solid var(--green); outline-offset:1px; }++ /* Application bar */+ #titlebar { flex:0 0 auto; height:40px; display:flex; align-items:stretch; background:#000; border-bottom:1px solid var(--edge); }+ .brand { display:flex; align-items:center; gap:10px; padding:0 16px 0 0; white-space:nowrap; }+ .brand .bar { width:4px; align-self:stretch; background:var(--green); }+ .brand .name { font-weight:700; letter-spacing:.06em; color:var(--bright); font-size:13px; }+ .brand .sep { width:1px; height:16px; background:var(--edge); }+ .brand .app { color:var(--muted); font-size:12px; letter-spacing:.02em; }+ #meta { display:flex; align-items:stretch; flex:1; min-width:0; overflow:hidden; border-left:1px solid var(--line); }+ .field { display:flex; flex-direction:column; justify-content:center; padding:0 14px; border-right:1px solid var(--line); white-space:nowrap; }+ .field .k { font-size:9.5px; letter-spacing:.1em; text-transform:uppercase; color:var(--dim); }+ .field .v { font-family:var(--mono); font-size:11.5px; color:var(--text); }+ #status { display:flex; align-items:center; gap:8px; padding:0 16px; white-space:nowrap; font-weight:700; letter-spacing:.04em; text-transform:uppercase; font-size:11px; border-left:1px solid var(--line); }+ #status::before { content:""; width:8px; height:8px; background:currentColor; }+ #status.alert { color:var(--red); }+ #status.ok { color:var(--green); }++ /* Docked panels, 1 px apart */+ #work { flex:1 1 auto; min-height:0; display:grid; grid-template-columns:minmax(0,1fr) 340px; gap:1px; background:var(--border); }+ #left { min-width:0; min-height:0; display:flex; flex-direction:column; gap:1px; }+ .panel { background:var(--panel); min-height:0; overflow:hidden; }+ .phead { flex:0 0 auto; height:26px; display:flex; align-items:center; gap:8px; padding:0 10px; background:var(--head); border-bottom:1px solid var(--border); font-size:10.5px; font-weight:700; letter-spacing:.1em; text-transform:uppercase; color:var(--text); user-select:none; white-space:nowrap; }+ .phead .sub { font-weight:400; letter-spacing:.02em; text-transform:none; color:var(--muted); font-size:11px; overflow:hidden; text-overflow:ellipsis; }+ .phead[data-toggle] { cursor:pointer; }+ .phead[data-toggle]:hover { background:#303030; }+ .phead .tri { color:var(--muted); width:9px; font-size:9px; }+ .phead .count { margin-left:auto; font-family:var(--mono); font-weight:400; letter-spacing:0; color:var(--muted); }++ #viewports { flex:1 1 auto; min-height:0; display:flex; gap:1px; background:var(--border); }+ .vp { position:relative; min-width:0; display:flex; flex-direction:column; background:var(--panel); }+ .vp .body { position:relative; flex:1 1 auto; min-height:0; overflow:hidden; }+ #source { flex:0 0 auto; max-width:46%; }+ #source .body { background:#0a0a0a; display:flex; align-items:center; justify-content:center; }+ #source video, #source img { display:block; width:100%; height:100%; object-fit:contain; }+ #source .note { color:var(--muted); padding:16px; text-align:center; width:240px; }+ #view { flex:1 1 auto; }+ #view canvas { display:block; }+ .hint { position:absolute; top:8px; right:10px; z-index:2; font-size:10.5px; color:var(--dim); pointer-events:none; }+ .caption { position:absolute; left:10px; bottom:10px; z-index:2; pointer-events:none; padding:5px 10px; background:rgba(0,0,0,.72); border-left:3px solid var(--green); font-size:12px; }+ .caption.bad { border-left-color:var(--red); }+ .caption .bad { color:var(--red); font-weight:700; }+ .caption .ok { color:var(--green); font-weight:700; }+ #viewMsg { position:absolute; inset:0; display:flex; align-items:center; justify-content:center; color:var(--muted); text-align:center; padding:24px; }++ /* Timeline */+ #dock { flex:0 0 auto; display:flex; flex-direction:column; }+ .toolbar { display:flex; align-items:center; gap:4px; padding:5px 8px; border-bottom:1px solid var(--border); background:var(--panel); flex-wrap:wrap; }+ .btn { height:24px; min-width:26px; padding:0 8px; display:inline-flex; align-items:center; justify-content:center; gap:6px; background:var(--control); border:1px solid #444; border-radius:2px; color:var(--text); cursor:pointer; font-size:11.5px; }+ .btn:hover { background:var(--control-hi); border-color:#555; color:var(--bright); }+ .btn:disabled { opacity:.35; cursor:default; }+ .btn svg { width:11px; height:11px; fill:currentColor; }+ .btn.primary { background:var(--green); border-color:var(--green); color:#000; font-weight:700; min-width:72px; }+ .btn.primary:hover { background:var(--green-hi); border-color:var(--green-hi); color:#000; }+ .btn.on { border-color:var(--green); color:var(--green); }+ .vsep { width:1px; height:18px; background:var(--edge); margin:0 6px; }+ .framebox { display:inline-flex; align-items:center; gap:6px; height:24px; padding:0 8px; background:var(--field); border:1px solid #444; border-radius:2px; color:var(--muted); font-size:11px; }+ .framebox input { width:44px; border:none; outline:none; text-align:right; font-family:var(--mono); font-size:12px; color:var(--bright); background:transparent; }+ .framebox:focus-within { border-color:var(--green); }+ .framebox .mono { font-family:var(--mono); }+ .grow { flex:1; }+ input[type=range] { -webkit-appearance:none; appearance:none; background:transparent; width:80px; }+ input[type=range]::-webkit-slider-runnable-track { height:3px; background:#4a4a4a; }+ input[type=range]::-webkit-slider-thumb { -webkit-appearance:none; width:8px; height:14px; margin-top:-5.5px; border-radius:1px; background:var(--text); }+ input[type=range]::-moz-range-track { height:3px; background:#4a4a4a; }+ input[type=range]::-moz-range-thumb { width:8px; height:14px; border-radius:1px; background:var(--text); border:none; }+ #nowbar { display:flex; align-items:center; gap:6px; padding:0 10px; height:28px; border-bottom:1px solid var(--line); overflow:hidden; white-space:nowrap; }+ #nowbar .lab { font-size:9.5px; letter-spacing:.1em; text-transform:uppercase; color:var(--dim); margin-right:4px; }+ .tag { display:inline-block; padding:1px 7px; border:1px solid rgba(240,82,79,.55); background:var(--red-sel); color:#ff8a87; font-size:11px; border-radius:1px; }+ .clean { color:var(--green); font-size:11.5px; }+ #tlwrap { max-height:26vh; overflow-y:auto; overflow-x:hidden; background:var(--panel); }+ #timeline { width:100%; display:block; cursor:pointer; }++ /* Inspector */+ #right { display:flex; flex-direction:column; }+ .props { display:grid; grid-template-columns:118px 1fr; border-bottom:1px solid var(--border); }+ .props > div { padding:5px 10px; border-bottom:1px solid var(--line); }+ .props .k { color:var(--muted); }+ .props .v { color:var(--bright); font-family:var(--mono); font-size:11.5px; }+ .props .v.bad { color:var(--red); }+ .filters { display:flex; padding:6px 8px; border-bottom:1px solid var(--line); }+ .filters button { flex:1; height:22px; background:var(--control); border:1px solid #444; margin-left:-1px; color:var(--muted); cursor:pointer; font-size:11px; }+ .filters button:first-child { margin-left:0; border-radius:2px 0 0 2px; }+ .filters button:last-child { border-radius:0 2px 2px 0; }+ .filters button:hover { color:var(--bright); }+ .filters button.on { background:#3a4a22; border-color:var(--green); color:var(--bright); position:relative; z-index:1; }+ #issuesBody { flex:1 1 auto; min-height:0; display:flex; flex-direction:column; }+ #issues { flex:1 1 auto; overflow:auto; }+ .issue { position:relative; padding:7px 10px 7px 12px; border-bottom:1px solid var(--line); border-left:3px solid transparent; cursor:pointer; }+ .issue:hover { background:#262626; }+ .issue.sel { background:var(--green-sel); border-left-color:var(--green); }+ .issue .t { color:var(--bright); padding-right:70px; }+ .issue .d { position:absolute; top:7px; right:10px; font-family:var(--mono); font-weight:700; color:var(--red); }+ .issue.restpose .d { color:var(--muted); }+ .issue .m { color:var(--muted); font-size:11px; margin-top:2px; }+ .issue .r { color:var(--dim); font-size:10.5px; margin-top:2px; font-family:var(--mono); white-space:nowrap; overflow:hidden; text-overflow:ellipsis; }+ .issue .badge { display:inline-block; margin-top:5px; font-size:10px; letter-spacing:.06em; text-transform:uppercase; padding:0 5px; border:1px solid var(--edge); color:var(--muted); }+ .empty { padding:18px; text-align:center; color:var(--muted); }+ #method { margin:0; padding:8px 12px 10px 26px; color:var(--muted); font-size:11.5px; line-height:1.55; max-height:30vh; overflow:auto; }+ #method li { margin:4px 0; }+ #method b { color:var(--text); }+ .collapsed + * { display:none !important; }+ ::-webkit-scrollbar { width:8px; height:8px; } ::-webkit-scrollbar-track { background:var(--panel); }+ ::-webkit-scrollbar-thumb { background:#444; } ::-webkit-scrollbar-thumb:hover { background:#555; }+ @media (max-width:980px) { body { overflow:auto; height:auto; } #work { grid-template-columns:1fr; } #viewports { height:60vh; } #meta { display:none; } }+</style>+<script type="importmap">{"imports":{"three":"__THREE__/build/three.module.js","three/addons/":"__THREE__/examples/jsm/"}}</script>+</head>+<body>+<div id="titlebar">+ <div class="brand"><span class="bar"></span><span class="name">MOTION2MIXAMORIG</span><span class="sep"></span><span class="app">Self-Intersection Diagnostics</span></div>+ <div id="meta"></div>+ <div id="status"></div>+</div>++<div id="work">+ <div id="left">+ <div id="viewports">+ <div class="vp" id="source">+ <div class="phead">Source <span class="sub" id="srcLabel"></span></div>+ <div class="body"></div>+ </div>+ <div class="vp" id="view">+ <div class="phead">Viewport <span class="sub">Character</span></div>+ <div class="body" id="viewBody">+ <span class="hint">LMB orbit · Wheel zoom</span>+ <div class="caption" id="caption"></div>+ <div id="viewMsg">Loading the 3D view…</div>+ </div>+ </div>+ </div>++ <div class="panel" id="dock">+ <div class="phead">Timeline</div>+ <div class="toolbar">+ <button class="btn" id="first" title="First frame"><svg viewBox="0 0 16 16"><path d="M3 2h2v12H3zM13 2v12L6 8z"/></svg></button>+ <button class="btn" id="prev" title="Previous frame (←)"><svg viewBox="0 0 16 16"><path d="M11 2v12L4 8z"/></svg></button>+ <button class="btn primary" id="play" title="Play / Pause (Space)"></button>+ <button class="btn" id="next" title="Next frame (→)"><svg viewBox="0 0 16 16"><path d="M5 2v12l7-6z"/></svg></button>+ <button class="btn" id="last" title="Last frame"><svg viewBox="0 0 16 16"><path d="M11 2h2v12h-2zM3 2v12l7-6z"/></svg></button>+ <span class="vsep"></span>+ <span class="framebox">FRAME <input id="frameField" inputmode="numeric" title="Type a frame and press Enter"> <span class="mono" id="frameTotal"></span> <span class="mono" id="timeInfo"></span></span>+ <div class="grow"></div>+ <button class="btn" id="sound" title="Sound of the original video"></button>+ <input id="volume" type="range" min="0" max="1" step="0.05" value="0.8" title="Volume">+ </div>+ <div id="nowbar"><span class="lab">Active</span><span id="now"></span></div>+ <div id="tlwrap"><canvas id="timeline"></canvas></div>+ </div>+ </div>++ <div class="panel" id="right">+ <div class="phead" data-toggle><span class="tri">▼</span>Summary</div>+ <div class="props" id="summary"></div>+ <div class="phead" data-toggle><span class="tri">▼</span>Issues<span class="count" id="issueCount"></span></div>+ <div id="issuesBody">+ <div class="filters" id="filters">+ <button class="on" data-f="all">All pairs</button>+ <button data-f="motion">From motion</button>+ <button data-f="now">This frame</button>+ </div>+ <div id="issues"></div>+ </div>+ <div class="phead collapsed" data-toggle><span class="tri">▶</span>Method</div>+ <ul id="method"></ul>+ </div>+</div>++<script type="application/json" id="data">__DATA__</script>+<script type="text/plain" id="media">__MEDIA__</script>+<script type="text/plain" id="glb">__GLB__</script>+<script>+const D = JSON.parse(document.getElementById('data').textContent);+const R = D.report, N = R.frame_count, FPS = R.fps;+const $ = id => document.getElementById(id);+const label = p => p.replace(/_/g, ' ');+const cap = s => s.charAt(0).toUpperCase() + s.slice(1);+const pairName = i => cap(i.parts.map(label).join(' ↔ '));+const fmtRange = ([a, b]) => a === b ? `${a}` : `${a}–${b}`;+let frame = 0, selected = -1, playing = false, filter = 'all';+window.diag = { setFrame: f => setFrame(f), onFrame: null };++const ICON = {+ play: '<svg viewBox="0 0 16 16"><path d="M4 2l10 6-10 6z"/></svg> PLAY',+ pause: '<svg viewBox="0 0 16 16"><path d="M3 2h4v12H3zM9 2h4v12H9z"/></svg> PAUSE',+ on: '<svg viewBox="0 0 16 16"><path d="M2 6h3l4-3v10l-4-3H2zM11 5.5a3.5 3.5 0 010 5l-.8-.8a2.4 2.4 0 000-3.4zM12.6 3.9a5.8 5.8 0 010 8.2l-.8-.8a4.7 4.7 0 000-6.6z"/></svg> Audio',+ off: '<svg viewBox="0 0 16 16"><path d="M2 6h3l4-3v10l-4-3H2zM11 6.2l.8-.8 1.6 1.6 1.6-1.6.8.8L14.2 8l1.6 1.6-.8.8-1.6-1.6-1.6 1.6-.8-.8L12.6 8z"/></svg> Muted',+};++// Collapsible panels.+document.querySelectorAll('[data-toggle]').forEach(h => h.onclick = () => {+ h.classList.toggle('collapsed'); h.querySelector('.tri').textContent = h.classList.contains('collapsed') ? '▶' : '▼';+ setTimeout(drawTimeline, 0);+});++// Original video / photo, on the same timeline (frame k at k / fps).+const M = D.media, src = $('source'), srcBody = src.querySelector('.body');+let video = null;+if (M.kind === 'video' || M.kind === 'image') {+ const el = document.createElement(M.kind === 'video' ? 'video' : 'img');+ el.src = `data:${M.mime};base64,` + $('media').textContent.trim();+ if (M.kind === 'video') { video = el; el.muted = true; el.playsInline = true; el.preload = 'auto'; }+ el.addEventListener(video ? 'loadedmetadata' : 'load', () => {+ const w = el.videoWidth || el.naturalWidth, h = el.videoHeight || el.naturalHeight;+ if (w && h) { src.style.aspectRatio = `${w} / ${h}`; window.dispatchEvent(new Event('resize')); }+ if (video) video.currentTime = frame / FPS;+ });+ srcBody.append(el);+ $('srcLabel').textContent = M.name;+} else {+ src.style.width = '260px';+ const n = document.createElement('div'); n.className = 'note'; n.textContent = M.note || 'No source video.'; srcBody.append(n);+}++// Sound: off by default (browsers block autoplay with sound); one click turns it on.+function showSound() { $('sound').innerHTML = video && !video.muted ? ICON.on : ICON.off; $('sound').classList.toggle('on', !!video && !video.muted); }+if (video) {+ video.volume = +$('volume').value;+ $('sound').onclick = () => { video.muted = !video.muted; showSound(); };+ $('volume').oninput = e => { video.volume = +e.target.value; if (video.muted && video.volume > 0) { video.muted = false; showSound(); } };+} else { $('sound').disabled = true; $('volume').disabled = true; }+showSound();++// Application bar and summary.+const S = R.summary.self_intersection;+const field = (k, v) => `<div class="field"><span class="k">${k}</span><span class="v">${v}</span></div>`;+$('meta').innerHTML = [field('Source', R.source), field('Frames', `${N} @ ${FPS} fps`), field('Threshold', `${R.threshold_cm} cm`)].join('');+$('status').className = S.issue_count ? 'alert' : 'ok';+$('status').textContent = S.issue_count ? `${S.issue_count} intersections · max ${S.max_depth_cm} cm` : 'No intersection';+const kv = [+ ['Check', 'Self-intersection'],+ ['Part pairs', S.issue_count, S.issue_count > 0],+ ['Frames affected', `${S.frame_count} / ${N} (${(100 * S.frame_count / Math.max(1, N)).toFixed(1)}%)`, S.frame_count > 0],+ ['Max depth', `${S.max_depth_cm} cm`, S.max_depth_cm > 0],+ ['Frame numbers', 'Blender, first = 0'],+ ['Generated', R.generated.replace('T', ' ')],+];+$('summary').innerHTML = kv.map(([k, v, bad]) => `<div class="k">${k}</div><div class="v${bad ? ' bad' : ''}">${v}</div>`).join('');+$('frameTotal').textContent = `/ ${N - 1}`;+for (const [k, v] of Object.entries(R.method)) { const li = document.createElement('li'); li.innerHTML = `<b>${cap(k.replace(/_/g, ' '))}.</b> `; li.append(v); $('method').append(li); }++const inRange = (i, f) => i.ranges.some(([a, b]) => f >= a && f <= b);+const visible = () => R.issues.filter(i =>+ filter === 'all' || (filter === 'motion' && !i.also_in_rest_pose) || (filter === 'now' && inRange(i, frame)));++function renderList() {+ const box = $('issues'), items = visible(); box.innerHTML = '';+ $('issueCount').textContent = `${items.length}`;+ if (!items.length) { box.innerHTML = `<div class="empty">${filter === 'now' ? 'No intersection on this frame.' : 'No issues to show.'}</div>`; return; }+ items.forEach(i => {+ const idx = R.issues.indexOf(i), el = document.createElement('div');+ el.className = 'issue' + (i.also_in_rest_pose ? ' restpose' : '') + (idx === selected ? ' sel' : '');+ el.innerHTML = '<div class="t"></div><div class="d"></div><div class="m"></div><div class="r"></div>';+ el.querySelector('.t').textContent = pairName(i);+ el.querySelector('.d').textContent = `${i.max_depth_cm.toFixed(2)} cm`;+ el.querySelector('.m').textContent = `${i.frame_count} frame${i.frame_count === 1 ? '' : 's'} · worst at frame ${i.worst_frame}`;+ el.querySelector('.r').textContent = i.ranges.map(fmtRange).join(', ');+ if (i.also_in_rest_pose) { const b = document.createElement('span'); b.className = 'badge'; b.textContent = 'Also in rest pose'; el.append(b); }+ el.onclick = () => { stop(); selected = idx; setFrame(i.worst_frame); renderList(); };+ box.append(el);+ });+}+document.querySelectorAll('#filters button').forEach(b => b.onclick = () => {+ filter = b.dataset.f; document.querySelectorAll('#filters button').forEach(x => x.classList.toggle('on', x === b)); renderList(); drawTimeline();+});++// Timeline: label column, ruler, one track per issue, green playhead.+const TL = { label: 180, ruler: 22, row: 20, pad: 12 };+function drawTimeline() {+ const c = $('timeline'), items = filter === 'now' ? R.issues : visible();+ const w = c.clientWidth || c.parentElement.clientWidth, h = TL.ruler + Math.max(1, items.length) * TL.row + 4;+ const dpr = window.devicePixelRatio || 1;+ c.width = w * dpr; c.height = h * dpr; c.style.height = h + 'px';+ const g = c.getContext('2d'); g.setTransform(dpr, 0, 0, dpr, 0, 0);+ const x0 = TL.label + TL.pad, x1 = w - TL.pad, x = f => x0 + (x1 - x0) * (N > 1 ? f / (N - 1) : 0);+ g.clearRect(0, 0, w, h);+ const font = '"NVIDIA Sans", "Helvetica Neue", Arial, sans-serif', mono = 'ui-monospace, "SF Mono", Menlo, Consolas, monospace';+ g.fillStyle = '#262626'; g.fillRect(0, 0, w, TL.ruler);+ g.fillStyle = '#000'; g.fillRect(0, TL.ruler - 1, w, 1);+ g.fillStyle = '#1b1b1b'; g.fillRect(0, TL.ruler, TL.label, h - TL.ruler);+ g.fillStyle = '#000'; g.fillRect(TL.label, 0, 1, h);+ const pxPerFrame = (x1 - x0) / Math.max(1, N - 1);+ const major = [1, 5, 10, 25, 50, 100, 250, 500, 1000].find(s => s * pxPerFrame >= 60) || 1000;+ const minor = major >= 10 ? major / 5 : 1;+ g.font = `10px ${mono}`; g.textBaseline = 'top'; g.textAlign = 'center';+ for (let f = 0; f < N; f += minor) {+ const isMajor = f % major === 0, xx = Math.round(x(f)) + .5;+ g.strokeStyle = isMajor ? '#6a6a6a' : '#3e3e3e'; g.beginPath(); g.moveTo(xx, TL.ruler - (isMajor ? 8 : 4)); g.lineTo(xx, TL.ruler - 1); g.stroke();+ if (isMajor) { g.fillStyle = '#8a8a8a'; g.fillText(String(f), xx, 3); g.fillStyle = 'rgba(255,255,255,.035)'; g.fillRect(xx - .5, TL.ruler, 1, h - TL.ruler); }+ }+ g.textAlign = 'left';+ items.forEach((i, k) => {+ const y = TL.ruler + k * TL.row, idx = R.issues.indexOf(i), sel = idx === selected;+ if (sel) { g.fillStyle = 'rgba(118,185,0,.16)'; g.fillRect(0, y, w, TL.row); g.fillStyle = '#76b900'; g.fillRect(0, y, 3, TL.row); }+ else if (k % 2) { g.fillStyle = 'rgba(255,255,255,.02)'; g.fillRect(0, y, w, TL.row); }+ g.fillStyle = '#2c2c2c'; g.fillRect(0, y + TL.row - 1, w, 1);+ g.fillStyle = sel ? '#f0f0f0' : (i.also_in_rest_pose ? '#6a6a6a' : '#cfcfcf'); g.textBaseline = 'middle';+ g.font = `${sel ? 700 : 400} 11.5px ${font}`;+ let name = pairName(i); while (g.measureText(name).width > TL.label - 16 && name.length > 4) name = name.slice(0, -2) + '…';+ g.fillText(name, 10, y + TL.row / 2 + .5);+ g.fillStyle = i.also_in_rest_pose ? '#5a5a5a' : '#f0524f';+ for (const [a, b] of i.ranges) g.fillRect(Math.round(x(a)) - 1, y + 5, Math.max(3, Math.round(x(b) - x(a)) + 2), TL.row - 11);+ });+ const px = Math.round(x(frame)) + .5;+ g.strokeStyle = '#76b900'; g.lineWidth = 1; g.beginPath(); g.moveTo(px, TL.ruler - 2); g.lineTo(px, h); g.stroke();+ g.fillStyle = '#76b900'; g.fillRect(px - 17, 1, 34, TL.ruler - 4);+ g.fillStyle = '#000'; g.font = `700 10.5px ${mono}`; g.textAlign = 'center'; g.textBaseline = 'middle'; g.fillText(String(frame), px, TL.ruler / 2 - 1); g.textAlign = 'left';++ c.onmousedown = e => {+ stop();+ const r = c.getBoundingClientRect(), mx = e.clientX - r.left, my = e.clientY - r.top;+ if (mx < TL.label && my > TL.ruler) {+ const it = items[Math.floor((my - TL.ruler) / TL.row)];+ if (it) { selected = R.issues.indexOf(it); setFrame(it.worst_frame); renderList(); }+ return;+ }+ const seek = ev => setFrame(Math.round((ev.clientX - r.left - x0) / (x1 - x0) * (N - 1)));+ seek(e);+ const move = ev => seek(ev), up = () => { removeEventListener('mousemove', move); removeEventListener('mouseup', up); };+ addEventListener('mousemove', move); addEventListener('mouseup', up);+ };+}++function setFrame(f, fromVideo = false) {+ frame = Math.max(0, Math.min(N - 1, f | 0));+ if (video && !fromVideo && video.readyState >= 1) video.currentTime = frame / FPS;+ if (document.activeElement !== $('frameField')) $('frameField').value = frame;+ $('timeInfo').textContent = `${(frame / FPS).toFixed(2)} s`;+ const now = R.issues.filter(i => inRange(i, frame));+ $('now').innerHTML = now.length ? '' : '<span class="clean">No intersection</span>';+ now.forEach(i => { const t = document.createElement('span'); t.className = 'tag'; t.textContent = pairName(i); $('now').append(t); });+ $('caption').className = 'caption' + (now.length ? ' bad' : '');+ $('caption').innerHTML = now.length ? `<span class="bad">${now.map(pairName).join(' · ')}</span>` : '<span class="ok">No intersection on this frame</span>';+ drawTimeline();+ if (filter === 'now') renderList();+ if (window.diag.onFrame) window.diag.onFrame(frame, now);+}++// Transport.+function setPlayIcon() { $('play').innerHTML = playing ? ICON.pause : ICON.play; }+function stop() { if (playing) togglePlay(); }+let last = 0, acc = 0;+function togglePlay() {+ playing = !playing; setPlayIcon();+ if (video) { playing ? video.play() : video.pause(); }+ last = performance.now(); if (playing) requestAnimationFrame(tick);+}+function tick(t) {+ if (!playing) return;+ if (video) {+ const f = Math.round(video.currentTime * FPS); // the video is the clock+ if (video.ended || f >= N) { video.currentTime = 0; video.play(); setFrame(0); }+ else if (f !== frame) setFrame(f, true);+ } else {+ acc += (t - last) / 1000 * FPS; last = t;+ if (acc >= 1) { const step = Math.floor(acc); acc -= step; setFrame(frame + step >= N ? 0 : frame + step); }+ }+ requestAnimationFrame(tick);+}+$('play').onclick = togglePlay;+$('prev').onclick = () => { stop(); setFrame(frame - 1); };+$('next').onclick = () => { stop(); setFrame(frame + 1); };+$('first').onclick = () => { stop(); setFrame(0); };+$('last').onclick = () => { stop(); setFrame(N - 1); };+$('frameField').onkeydown = e => { if (e.key === 'Enter') { stop(); setFrame(parseInt(e.target.value, 10) || 0); e.target.blur(); } };+addEventListener('keydown', e => {+ if (e.target.id === 'frameField') return;+ if (e.code === 'Space') { e.preventDefault(); togglePlay(); }+ else if (e.key === 'ArrowLeft') { stop(); setFrame(frame - 1); }+ else if (e.key === 'ArrowRight') { stop(); setFrame(frame + 1); }+});+addEventListener('resize', drawTimeline);+setPlayIcon();+renderList();+setFrame(R.issues.length ? R.issues[0].worst_frame : 0);+</script>++<script type="module">+import * as THREE from 'three';+import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';+import { OrbitControls } from 'three/addons/controls/OrbitControls.js';++const D = JSON.parse(document.getElementById('data').textContent);+const FPS = D.report.fps;+const view = document.getElementById('viewBody'), msg = document.getElementById('viewMsg');+const bytes = Uint8Array.from(atob(document.getElementById('glb').textContent.trim()), c => c.charCodeAt(0));++const renderer = new THREE.WebGLRenderer({ antialias: true });+renderer.setPixelRatio(window.devicePixelRatio || 1);+renderer.shadowMap.enabled = true; renderer.shadowMap.type = THREE.PCFSoftShadowMap;+view.prepend(renderer.domElement);++// Dark viewport: grey void, metre grid on the floor, soft contact shadow.+const BG = 0x2b2b2b;+const scene = new THREE.Scene();+scene.background = new THREE.Color(BG);+scene.fog = new THREE.Fog(BG, 7, 22);+const camera = new THREE.PerspectiveCamera(35, 1, 0.01, 100);+const controls = new OrbitControls(camera, renderer.domElement);+controls.enableDamping = true;+scene.add(new THREE.HemisphereLight(0xffffff, 0x3a3a3a, 1.6));+const sun = new THREE.DirectionalLight(0xffffff, 2.0);+sun.position.set(2.5, 6, 3); sun.castShadow = true;+sun.shadow.mapSize.set(2048, 2048); sun.shadow.radius = 6;+Object.assign(sun.shadow.camera, { left: -2, right: 2, top: 2, bottom: -2, near: 0.5, far: 20 });+scene.add(sun); scene.add(sun.target);+const floor = new THREE.Group(); scene.add(floor);+const minor = new THREE.GridHelper(40, 400, 0x353535, 0x353535), major = new THREE.GridHelper(40, 40, 0x4a4a4a, 0x4a4a4a);+minor.position.y = 0.0005; major.position.y = 0.001;+const ground = new THREE.Mesh(new THREE.PlaneGeometry(40, 40), new THREE.ShadowMaterial({ opacity: 0.45 }));+ground.rotation.x = -Math.PI / 2; ground.receiveShadow = true;+floor.add(minor, major, ground);++function size() { const w = view.clientWidth, h = view.clientHeight; renderer.setSize(w, h); camera.aspect = w / Math.max(1, h); camera.updateProjectionMatrix(); }+window.addEventListener('resize', size); size();++const BASE = new THREE.Color(0x9a9c9e), HOT = new THREE.Color(0xf0524f);+new GLTFLoader().parse(bytes.buffer, '', gltf => {+ msg.remove();+ scene.add(gltf.scene);+ const mixer = new THREE.AnimationMixer(gltf.scene);+ if (gltf.animations[0]) mixer.clipAction(gltf.animations[0]).play();++ // Matte grey character; parts in an issue on this frame turn red.+ const targets = [];+ gltf.scene.traverse(o => {+ if (o.isMesh) o.castShadow = true;+ if (!o.isSkinnedMesh) return;+ const pos = o.geometry.attributes.position;+ const info = D.meshes.find(m => m.name === o.name && m.count === pos.count) || D.meshes.find(m => m.count === pos.count);+ if (!info) return;+ const col = new Float32Array(pos.count * 3);+ for (let v = 0; v < pos.count; v++) { col[3 * v] = BASE.r; col[3 * v + 1] = BASE.g; col[3 * v + 2] = BASE.b; }+ o.geometry.setAttribute('color', new THREE.BufferAttribute(col, 3));+ o.material = new THREE.MeshStandardMaterial({ vertexColors: true, roughness: 0.7, metalness: 0 });+ o.frustumCulled = false;+ targets.push({ attr: o.geometry.attributes.color, part: info.part });+ });+ if (!targets.length) { const m = document.createElement('div'); m.id = 'viewMsg'; m.textContent = 'Could not match the mesh to the report.'; view.append(m); }++ function paint(active) {+ const hot = new Set();+ active.forEach(i => i.parts.forEach(p => hot.add(D.parts.indexOf(p))));+ for (const t of targets) {+ const a = t.attr.array;+ for (let v = 0; v < t.part.length; v++) {+ const c = hot.has(t.part[v]) ? HOT : BASE;+ a[3 * v] = c.r; a[3 * v + 1] = c.g; a[3 * v + 2] = c.b;+ }+ t.attr.needsUpdate = true;+ }+ }++ mixer.setTime(0); gltf.scene.updateMatrixWorld(true);+ const box = new THREE.Box3().setFromObject(gltf.scene), c = box.getCenter(new THREE.Vector3()), sz = box.getSize(new THREE.Vector3());+ floor.position.y = box.min.y;+ controls.target.copy(c);+ camera.position.copy(c).add(new THREE.Vector3(sz.y * 0.7, sz.y * 0.75, Math.max(sz.y, sz.x) * 2.2));+ controls.update();++ // Keep the shadow camera on the character (follow the hips bone) as it moves around.+ let hips = null; gltf.scene.traverse(o => { if (!hips && o.isBone && /hips$/i.test(o.name)) hips = o; });+ const follow = new THREE.Vector3();+ window.diag.onFrame = (f, active) => {+ mixer.setTime(f / FPS); paint(active);+ if (hips) { gltf.scene.updateMatrixWorld(true); hips.getWorldPosition(follow); } else follow.copy(c);+ sun.target.position.set(follow.x, box.min.y, follow.z); sun.position.set(follow.x + 2.5, box.min.y + 6, follow.z + 3);+ };+ window.diag.setFrame(+document.getElementById('frameField').value || 0);+ renderer.setAnimationLoop(() => { controls.update(); renderer.render(scene, camera); });+}, err => { msg.textContent = 'Could not load the 3D view: ' + err; });+</script>+<script>+// three.js comes from a CDN; without a connection the module above never runs.+setTimeout(() => { const m = document.getElementById('viewMsg'); if (m && m.textContent.startsWith('Loading')) m.textContent = 'The 3D view needs an internet connection to load three.js. The video, timeline, and issue list work offline.'; }, 10000);+</script>+</body>+</html>+"""diff --git a/motion2mixamorig/pipeline.py b/motion2mixamorig/pipeline.pyindex eecd82a..d58dd00 100644--- a/motion2mixamorig/pipeline.py+++ b/motion2mixamorig/pipeline.py@@ -22,6 +22,7 @@ import numpy as npfrom . import pathsfrom .errors import classify_error+from .diagnose import DIAGNOSTICS_JSON, diagnose_run, summary_linefrom .human_motion import HumanMotion, load_human_motionfrom .mixamo.animation import build_animationfrom .mixamo.gltf import write_glb@@ -179,6 +180,18 @@ def run_pipeline("mixamo_rotations": rotations_npz.name,"mixamo_character_glb": glb_path.name,}++ # Diagnostics are a report, not a product: a failure here must not fail the run.+ print("diagnosing self-intersection")+ diagnostics = None+ try:+ diag = diagnose_run(run_dir, verbose=False)+ diagnostics = diag["summary"]+ outputs["diagnostics"] = DIAGNOSTICS_JSON+ print(summary_line(diag))+ except Exception as exc: # noqa: BLE001+ print(f"diagnostics skipped: {exc}")+if not no_preview:stage = "preview"writer.emit("preview", 0.90, "Rendering previews")@@ -218,6 +231,7 @@ def run_pipeline("rest_skin_error_cm": skin_err,"twist_events": len(anim.twist_events),"quat_sign_flips": len(anim.sign_events),+ "diagnostics": diagnostics,"outputs": outputs,}write_run_json(run_dir, report)diff --git a/pyproject.toml b/pyproject.tomlindex 8359a53..16daeb3 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -7,6 +7,7 @@ dependencies = ["numpy","opencv-python","einops",+ "scipy", # nearest-surface search in `m2mr diagnose` (also pulled in by gvhmr)# Video -> 3D human skeleton. `preproc` = person detection/tracking,# `rtmpose` = the ungated 2D-pose backend the pipeline uses."gvhmr[preproc,rtmpose]>=1.6",