Skip to content
Python

docs: add the 2026-09-12 architecture audit with its probe scripts and measurements

mohamdev913e089ceba4
diff --git a/docs/architecture_audit_2026-09-12.md b/docs/architecture_audit_2026-09-12.md
new file mode 100644
index 0000000..683c05a
--- /dev/null
+++ b/docs/architecture_audit_2026-09-12.md
@@ -0,0 +1,174 @@
+# Generation 7 architecture audit — 12 September 2026
+
+**The regression-space mismatch is real and worth testing. It does not establish a particular accuracy gain. The largest overlooked architectural difference is the teacher's repeated geometry-to-image feedback inside its decoder.** A full reproduction of the output head alone will leave that difference intact.
+
+This is an assessment, not a training change. No trainer, deployment code, existing TODO, or checkpoint was modified. The quoted generation-7 results are validation statistics supplied by the user; no generation-7 checkpoint was available in the local `runs/` directory. These probes establish representation compatibility and approximation errors, not improvements from retraining.
+
+The wrapper in `../sam3-biomechanics/video_to_pose_pipeline` loads the implementation in `../sam3-biomechanics/sam-3d-body`. I inspected that implementation and the actual `video_to_pose_pipeline/checkpoints/sam-3d-body-dinov3/model_config.yaml` and `model.ckpt`, rather than relying on generic architectural descriptions. Its model files have no local git modifications. The wrapper also has optional encoder replacement/precision/quantization paths; they are separate from the reference architecture discussed here.
+
+Evidence: [main measurements](audit_2026-09-12/measurements.json), [follow-up measurements](audit_2026-09-12/followup_measurements.json), [representation/geometry probe](audit_2026-09-12/probe_pathway.py), [additional geometry probe](audit_2026-09-12/probe_geometry.py). Sampling used seed 20260912: 6,000 COCO annotations and 1,000 each from AIC, MPII, Harmony4D and 3DPW, selected without replacement from sorted filenames. Full mesh probes used the first 256 selected samples per dataset, in batches of 16. All computations ran on CPU; these are not GPU latency measurements. SA-1B is not available locally and was not sampled. The 3DPW probe is a representation check on annotations, not a benchmark evaluation.
+
+**1. What reproducing the regression pathway actually means.**
+
+The current student emits 204 MHR parameters, 45 shape coefficients and 3 camera coordinates from one linear layer on a single token. The teacher also reads pose from one token. A single global token is therefore not itself a departure from the teacher.
+
+| Component | Current student | Actual local DINOv3 teacher |
+|---|---|---|
+| Pose regression head | Linear(512, 252), including shape and camera | Two linear layers with ReLU, 1024 → 1024 → 519; camera has a separate MLP |
+| Initial pose | Near-zero raw parameters | Learned initial estimate; continuous neutral-pose initialization and residual prediction |
+| Global orientation | Three raw Euler values | Six values → orthogonalized rotation matrix → Euler triple |
+| Local pose | Raw `6:136` | 260 continuous values → 133 values, with hand/jaw masking and slicing |
+| Skeleton scales | 68 freely predicted values, then bounds | 28 coefficients → fixed affine map to 68 scales |
+| Finger poses | Part of the raw 204-vector | Two separate 54-value continuous blocks, converted and inserted by joint index |
+| Expression | No expression head | 72 predicted values explicitly multiplied by zero |
+
+Source: [student model](../instanthmr_distill_train/train_distill_mhr_only.py#L886), [teacher MHR head](../../sam3-biomechanics/sam-3d-body/sam_3d_body/models/heads/mhr_head.py#L39), [actual configuration](../../sam3-biomechanics/video_to_pose_pipeline/checkpoints/sam-3d-body-dinov3/model_config.yaml).
+
+The TODO's decomposition of the 260-dimensional representation is correct: 23 triples represented with 6D, 58 scalar angles represented with sin/cos, and six raw translation-like size channels. However, it includes finger channels that the head subsequently overwrites and a jaw triple that is discarded. The six size channels reach MHR indices `130:136`; genuine local rotation parameters occupy `6:130`, which is **124 values**, not 130.
+
+The measured case for changing representation is:
+
+| Probe | Result | What it establishes |
+|---|---|---|
+| COCO root components with absolute angle > 2.8 rad | 12.84% | Large root angles are common |
+| COCO poses with an axis within 0.34 rad of ±π | 21.23% | A substantial subset is near an Euler wrap boundary |
+| COCO middle root angle within 0.1 rad of ±π/2 | 2.17% | Near-singular Euler configurations occur |
+| COCO local rotation absolute angle, p99 | 1.188 rad | Local angles usually do not approach wrap boundaries |
+| Continuous body round-trip, all 10,000 annotations | Maximum parameter difference 2.38e-7 | Existing labels can supply continuous body targets |
+| Separate continuous hand round-trip | Maximum parameter difference 1.19e-7 | Existing labels also support the teacher's distinct hand parameterization |
+| GT scales projected into checkpoint's scale basis | Maximum residual < 2.2e-8 across sampled datasets | The scale subspace does not exclude these GT skeletons |
+
+These percentages differ slightly from the TODO because the sample differs. They support its qualitative observation. They are **not** the percentage of current predictions suffering a wrap error. That requires predictions paired with targets.
+
+A prediction just below −π and a target just below +π can represent nearby orientations while incurring a large raw Euler loss. Predicting continuous rotations and using an appropriate rotation loss addresses that mechanism. [Zhou et al.](https://arxiv.org/abs/1812.07035) provides the underlying continuity result, not a promised gain for this model.
+
+The local-pose case is less directly established by the angle histogram: those angles mostly remain away from wraps. The full pathway is still a sensible controlled experiment, especially because the scale basis and distinct hand mapping are compatible with the GT. It would be unjustified to claim that a head port alone will close the benchmark gap. Global rotation is removed by PA alignment; improvements to it principally affect orientation-sensitive errors, with only indirect benefits to PA-MPJPE.
+
+**Corrections required before implementing TODO item 1:**
+
+- **Fix the root inverse convention.** The teacher calls `roma.rotmat_to_euler("ZYX", R)`. Its inverse must therefore start with `R = roma.euler_to_rotmat("ZYX", p[:, 3:6])`, then concatenate the first and second **columns** of `R`. Do not pass the stored triple directly to `batch6DFromXYZ`. A simple input `[0.3, 0.5, 0.7]` returns `[0.7, 0.5, 0.3]` through that proposed inverse/forward pair. The correct construction preserves rig geometry, including sampled noncanonical Euler targets. For a physical rotation loss, respect the rig's actual extrinsic-XYZ convention; do not silently identify a teacher latent rotation matrix with the rig rotation.
+- **Pad the body target correctly.** The inverse expects 133 values: use MHR `6:136` plus three zero jaw values. Passing the 204-vector or only 130 values is wrong. Reconstruct hands using `hand_joint_idxs_left/right`, the hand conversion, and the checkpoint's hand buffers. A 339-value head that simply retains the body representation's finger channels is a different pathway from the teacher's hand overwrite.
+- **Separate inactive outputs from useful behavior.** 519 pose outputs reproduce the literal layout. 447 pose outputs omit the identically disabled face block while retaining both hand blocks; camera still needs its own three outputs. Neither choice requires a hand-crop encoder. Predicting 72 values that are always zeroed cannot improve accuracy.
+- **Include the MLP, initial estimate and valid continuous initialization.** All-zero 6D vectors and all-zero sin/cos pairs are degenerate. Neutral rotation initialization is not the current all-zero raw-Euler initialization. The teacher also normalizes decoder outputs before its head.
+- **Keep bounds compatible with the scale subspace.** PCA coefficients are not inherently bounded. Applying the existing coordinatewise tanh to the *expanded* 68 scales generally moves the result outside the teacher's affine subspace. A faithful port needs stability constraints that respect that subspace, plus separate handling of the six flexible size channels and zero root translation. Blindly retaining the old scale remap is not an exact reproduction.
+- **Use a numerically stable scale inverse.** The 28×68 checkpoint matrix has effective rank 24 in float32, three exactly zero singular values, and one tiny singular value (~1.29e-7). Do not assume an orthonormal, full-rank PCA basis or use transpose as its inverse. In the 256-sample COCO follow-up, a pseudoinverse with relative cutoff 1e-5 reconstructs scales to 1.50e-8 and the mesh to <0.00016 mm maximum vertex difference. Coefficients in null directions are not identifiable; expanded-scale supervision avoids inventing targets for them.
+- **Test equivalence in geometry, not only raw angles.** Euler canonicalization can change an equivalent angle triple. All sampled local-body triples round-tripped numerically here, but that is not a universal identity over arbitrary angles. Check rotations, FK and gradients near chart singularities. Returning through Euler for FK does not magically remove all numerical singularities.
+- **Treat loss changes as separate experiments.** Rotation-matrix or stable angular losses avoid wrap-dependent root penalties. Canonical 6D L2 supervision is not universally invalid, as the TODO implies, but its metric differs from physical rotation error. Likewise, SmoothL1 or unsquared Euclidean losses are not bugs merely because the paper describes other losses.
+
+**2. The most consequential missing decoder mechanism: geometry guides the next image lookup.**
+
+The teacher's actual configuration enables intermediate predictions, intermediate supervision, 2D keypoint tokens, 3D keypoint tokens, and keypoint updates. Its decoder does the following between layers:
+
+1. Predict an MHR mesh and camera from the current pose token.
+2. Obtain the mesh keypoints and project them into the crop.
+3. Bilinearly sample image features at those projected locations and add them to the corresponding 2D keypoint tokens. Out-of-image or behind-camera points are masked.
+4. Embed the predicted 2D coordinates into token positional information.
+5. Subtract the pelvis midpoint from predicted 3D keypoints and embed those coordinates into 70 separate 3D keypoint tokens.
+6. Let the next decoder layer use those updated tokens to revise the pose.
+
+Your four-layer decoder only exchanges generic query features and globally attends the image memory. It predicts MHR once, after the last layer. Its SimCC coordinates are never fed back into feature sampling or the pose decoder.
+
+This provides a concrete mechanism for improving **local body articulation**: after a provisional knee or elbow estimate, the next layer can inspect the corresponding image evidence. It is my highest-priority missing architectural mechanism beyond the parameter head. The recommendation is an inference from the implementation, not an ablation result on generation 7.
+
+There is an important distinction: the teacher has no independent freely regressed XYZ pose head in the inspected inference code, but **it does have 70 auxiliary 3D tokens**. The student's comment explaining their removal conflates these two things. Restoring an unconstrained 70×3 head alone would not reproduce their function.
+
+Also, the inspected callback predicts against the initial pose estimate at each layer; it does **not** add each new regression to the previous layer's predicted pose. Do not substitute conventional cumulative iterative regression and call it identical. Intermediate outputs are observable in released inference code; the configuration requests intermediate supervision, but the actual training loss implementation is not released.
+
+Sources: [intermediate prediction callback](../../sam3-biomechanics/sam-3d-body/sam_3d_body/models/meta_arch/sam3d_body.py#L463), [decoder loop](../../sam3-biomechanics/sam-3d-body/sam_3d_body/models/decoders/promptable_decoder.py#L153), [2D and 3D token updates](../../sam3-biomechanics/sam-3d-body/sam_3d_body/models/meta_arch/sam3d_body.py#L1759).
+
+The reference decoder also uses six layers at width 1024, versus four at width 512 in the student, repeats positional information within its layers, and applies a final normalization. Attention and feed-forward dimensions differ too, so width alone does not describe the cost. Its two-way image-to-token attention is explicitly disabled: adding that is not necessary for fidelity. The MLP/final-normalization changes are inexpensive candidates; none has a measured generation-7 gain yet.
+
+Implementation is more substantial than replacing a linear head: expose decoder stages, run differentiable FK and projection between them, update query features, and supervise useful intermediate predictions. Reuse the existing image encoding. The current deploy graph returns parameters and runs rig geometry outside that graph; moving FK and feature sampling inside decoding changes that export boundary. Validate the intended ONNX/mobile runtime before committing to the full design. A smaller two-stage prototype is a useful cost/accuracy experiment, but is not full teacher reproduction.
+
+**3. Image features are much coarser, beyond the crop-resolution issue already noted.**
+
+The actual DINOv3 checkpoint uses 512×512 crops and 16-pixel patches: a 32×32 image feature grid. Both current student backbones produce a **7×7** final feature grid at 224 input; I verified their forward outputs. This is 1,024 versus 49 spatial tokens. These are memory-grid sizes, not FLOP ratios.
+
+A stored-image resolution increase and a feature-stride change solve different problems. Rebuilding crops at higher resolution restores source detail. Using a stride-16 feature map or fusing an earlier backbone stage preserves more of that detail for the decoder. At 224 input, stride 16 gives a 14×14 grid without increasing input pixels. That is a particularly relevant experiment if adding joint-local feature sampling: sampling a 7×7 map remains coarse even with a sophisticated decoder.
+
+The teacher also has large-scale DINOv3 visual pretraining and substantially greater model capacity. Exact head mechanics cannot establish that this representation advantage has disappeared. Pose-oriented pretraining or feature distillation into the compact encoder is a plausible separate experiment. Ground-truth supervision remains the current training source; feature distillation would be an additional signal.
+
+One correction to TODO item 2: upsampling a 224 crop does not add image information, but that does not make training a larger input **necessarily pointless**. It can change effective feature stride and optimization. The clean experiment separates source resolution from network sampling density. Also, the current dataset resizes to the network input **before** its affine augmentation; merely storing 320-pixel crops would not preserve their detail through a zoom. Warp/crop from the higher-resolution source directly into the final network input.
+
+**4. The skeleton-only landmark approximation hides body-surface information.**
+
+The teacher forms its 70 landmarks from both mesh vertices and skeletal joints. The student uses an affine 70×127 skeleton regressor. This approximation is accurate for most fingers but less accurate for several body-surface landmarks.
+
+| Dataset, 256 samples each | All 70 landmarks: mean error | 30 nonfinger landmarks: mean error |
+|---|---:|---:|
+| COCO | 1.263 mm | 2.933 mm |
+| AIC | 1.322 mm | 3.071 mm |
+| MPII | 1.408 mm | 3.273 mm |
+| Harmony4D | 1.616 mm | 3.750 mm |
+| 3DPW annotations | 1.421 mm | 3.300 mm |
+
+These compare two landmark operators applied to the **same GT parameters**, using the local rig and teacher checkpoint mapping. No learned student prediction enters the comparison. In COCO, the worst individual discrepancy is 38.82 mm; mean discrepancies include small-toe tips at 8.8–9.1 mm and acromion landmarks at 6.7–6.8 mm. The exact operator is also closer to the stored COCO annotations: 0.175 mm mean versus the approximation's 1.242 mm. The small nonzero exact residual means the local rig/mapping is not perfectly identical to the annotation generator.
+
+An additional distinction matters for shape: changing the 45 shape coefficients cannot move the skeleton, but **can** move the teacher's mesh-derived landmarks. Zeroing shape on these COCO samples moves the nonfinger exact landmarks by 1.05 mm mean and up to 11.56 mm. Thus “shape has no geometric gradient” is a property of the student's skeleton-only landmark path, not of the teacher's full geometry. Generation 7's vertex loss already provides a shape gradient; this proposal adds accurate landmark locations and their gradients.
+
+The checkpoint's first 70 keypoints reference only **468 distinct mesh vertices**. You could extend the existing subset-skinning mechanism to include those exact indices, plus any vertices needed by the current vertex loss, and apply the sparse teacher mapping. This avoids making all 18,439 vertices mandatory at every decoder stage. Implementation feasibility follows from the current subset-skinning code; its combined runtime was not benchmarked here.
+
+This is a measurable modeling approximation worth testing. Its error is neither a guaranteed benchmark gain nor a quantity that should be subtracted from PA-MPJPE.
+
+**5. Camera conditioning and loss definitions still differ.**
+
+The teacher supplies per-location camera-ray embeddings to the image memory, as well as CLIFF information to the pose token. The student adds one learned projection of three CLIFF values to all queries. Dense ray conditioning gives each image feature its viewing direction explicitly; it is a plausible improvement for perspective-sensitive poses. For ideal square crops with known centered intrinsics, much of that information is already derivable from CLIFF plus grid position, so it is an inductive bias, not automatically extra information.
+
+The teacher separately regresses a crop camera `(scale, tx, ty)` and converts it using focal length and bbox geometry to full-camera translation. Your model directly regresses translation in metres. That is another regression-pathway difference omitted from TODO item 1. Its most direct benefits would concern depth and placement rather than PA-MPJPE. Source: [camera head](../../sam3-biomechanics/sam-3d-body/sam_3d_body/models/heads/camera_head.py#L83), [ray conditioning](../../sam3-biomechanics/sam-3d-body/sam_3d_body/models/meta_arch/sam3d_body.py#L1027).
+
+For the specific generation-7 command, `--crop-centre-fix` is a **no-op under baseline** because `cliff_follows_aug` remains false. The affine zoom/translation therefore does not get the same conditioning update as the v2 path. This can be tested independently of adopting every stronger v2 augmentation.
+
+The [paper's training section](https://arxiv.org/html/2602.15989v1#S4) describes L1 keypoint losses with learned uncertainty, pelvis/wrist normalization, annotation-dependent hand weights, parameter L2 and joint-limit penalties. The released code cannot establish their full formulas or weights. Your generation-7 configuration already uses **unsquared Euclidean** 3D error, not the default SmoothL1 listed in the TODO; shape weight is 0.03 and finger weight 0.2.
+
+Two verified loss differences deserve attention:
+
+- **No explicit pelvis/wrist centering in the main 3D loss.** It compares `pred_kp3d - tgt_3d` directly. `fk_scale` is only a training warm-up multiplier. Rig-origin coordinates are not equivalent to subtracting each sample's predicted and target hip midpoint. In the COCO probe, the GT hip midpoint lies about 907 mm from the rig coordinate origin. Separate wrist-relative hand losses would prevent arm-placement error from dominating finger articulation supervision. Retain an additional placement-sensitive term if adding centered losses.
+- **A naive joint-limit penalty conflicts with the annotations.** All 6,000 COCO samples violate at least one stored limit among the 124 local rotation parameters, using a tolerance of 1e-5 rad. About 20.23% of sampled parameter entries violate their limits; the maximum violation is 0.817 rad. There are 24 zero-width local-rotation limits. For example, `spine0_rx_flexible`, `l_talocrural_rx_flexible` and `r_subtalar_rz_flexible` have `[0,0]` limits with positive penalty weights, yet nonzero GT values. Do not apply all these limits unmodified. Use an annotation-compatible soft prior, or exclude the conflicting constraints and assess the others.
+
+Parameter supervision also remains masked by augmentation: with the generation-7 defaults, the root parameter loss sees about 20% of samples, and local-pose, scale, shape and vertex terms see about 60%. Geometry still supervises the other samples, so this is not a complete loss of their training signal. Correctly transforming parameter targets under rotations/flips could increase coverage, but requires rig-aware transformations and is not proven to match the unreleased teacher training pipeline.
+
+Learned uncertainty and hand-availability weighting are sensible experiments, particularly on ambiguous annotations. They need a proper probabilistic objective or regularizer; simply allowing a learned weight to fall toward zero can evade the task. Matching the teacher's stated norm is a lower priority than fixing demonstrated mismatches, and should not be bundled with the head ablation.
+
+**6. A tiny hand encoder is feasible; body gains need a feedback mechanism.**
+
+The teacher **shares one image encoder** across body and hand crops and uses separate body and hand decoders. Full inference runs body estimation, recrops each hand from the original image, estimates the hands, prompts the body decoder with hand wrists and body elbows, and then merges hand parameters with wrist-frame conversion and validity checks. It reuses cached body image embeddings for the prompted body refinement. It also updates selected hand scale and shape components. Source: [full inference and fusion](../../sam3-biomechanics/sam-3d-body/sam_3d_body/models/meta_arch/sam3d_body.py#L1197).
+
+Your proposed separate HGNetV2-B0 encoder is a reasonable compact alternative, though it is not the teacher's weight-sharing design. The installed timm provides B0 through B6; B0 has **1,850,396 parameters** after replacing its classifier head with Identity, and yields `(B,1024,7,7)` features at 224 input. A 256-wide, two-layer hand decoder shared by left and right crops is a reasonable initial experiment, not a measured optimum. Batch the two hands together and normalize left/right conventions consistently.
+
+The main implementation work is:
+
+1. Obtain original-resolution hand crops and labels. In the sampled COCO body crops, the projected hand extent averages only **15.4 pixels** at 224 input; Harmony4D averages **10.5 pixels**. Enlarging those saved patches cannot recover their original detail. Extents come from annotation coordinates, not a guarantee of visibility. Use original frames or store separate hand crops, and retain mappings needed to recreate their camera coordinates.
+2. Add crop jitter and annotation/visibility gates so training does not assume perfect hand boxes. The existing 2D joints can bootstrap boxes; a learned detector and confidence output are an additional option.
+3. Predict hand articulation plus wrist orientation in a consistent frame, with wrist-relative geometry supervision. Existing MHR GT supports articulation targets; hand-only normalization, flips and wrist transforms still need to be implemented.
+4. Convert predicted global wrist orientation into the body's local wrist frame, gate unreliable estimates, and preserve the body's fallback prediction.
+5. To improve the **body pose**, feed hand evidence back into body estimation. Reproduce the teacher's wrist/elbow prompting or train a smaller body-refinement stage. Merely overwriting finger parameters does not improve hip, knee or torso estimates by kinematics.
+
+Measured check: zeroing all 54 finger articulation parameters on 256 GT COCO samples changes the teacher's 30 nonfinger landmarks by only **0.102 mm mean**, while strongly moving fingers. The student's approximate landmark operator reports 1.298 mm body movement for the same intervention, showing some artificial finger-to-body coupling in that operator. Neither number is a measured gain from a trained hand network. SMPL conversion and body-metric adapters can also introduce small indirect effects.
+
+Expect the strongest benefit in hand articulation and full-mesh quality. A body-J14 gain is plausible through better wrists, arm refinement or shared training, but is not guaranteed by adding a hand encoder. Matching the complete teacher fusion is substantial work; adding a basic finger-only refinement branch is much simpler. Hand-specific training diversity also matters, beyond branch size.
+
+**7. Suggested order of experiments.**
+
+| Order | Experiment | Main quantity it should improve or clarify |
+|---|---|---|
+| 1 | Correct full continuous head, distinct hand blocks, MLP/init and compatible scale constraints | Parameter fitting, orientation, scale consistency; establishes pathway baseline |
+| 2 | Exact mesh-derived 70 landmarks using supported vertex subset | Removes a measured landmark approximation and restores surface-sensitive gradients |
+| 3 | Geometry feedback between decoder stages, including 3D token updates | Local body articulation; strongest missing body-specific mechanism |
+| 4 | Denser image features and source-aware cropping, tested separately | Local image evidence and spatial localization |
+| 5 | Camera-ray conditioning and explicit pelvis/wrist-relative terms | Perspective handling and separation of articulation from placement |
+| 6 | Dedicated hand crops/branch, then body refinement from wrists | Hands first; body improvement depends on refinement |
+
+These are priorities based on verified differences and mechanisms, not a ranking measured by training. Keeping the head experiment separate from decoder feedback is compatible with fully porting the head; it makes the result interpretable. Preserve generation 7 as the control and compare 2–3 matched seeds at comparable optimization budgets. The head and exact-landmark tests can use existing annotations; high-detail recropping requires original imagery. Measure body and hand errors separately, plus mesh error and absolute placement where relevant.
+
+The teacher's broader training mix includes additional multiview, synthetic and hand-specific datasets. Your five-source mix does not reproduce that coverage. The provided mix weights total 0.70 and the sampler renormalizes them: **SA-1B 42.86%, AIC 25.71%, Harmony4D 14.29%, COCO 12.86%, MPII 4.29%**, assuming all five are staged. Those are the actual probabilities, not 30/18/10/9/3 percent. Unlisted folders receive zero sampling weight. Source: [mixture implementation](../instanthmr_distill_train/train_distill_jz.py#L277).
+
+Finally, the supplied numbers are 3DPW validation J14+adapter records, not published test results. The summarizer also takes independent minima across epochs/RAW/EMA, so the displayed PA/MPJPE pair may not belong to one checkpoint. Establish proximity to published methods using one selected checkpoint, the matching test split, joint/conversion convention and bbox protocol; evaluate EMDB independently. Representation round-trips and the older 62.90-versus-31.84 fitting audit do not establish the achievable improvement from this head or a numerical gap attributable to it.
+
+To rerun the two main probes in the existing environment:
+
+```sh
+python docs/audit_2026-09-12/probe_pathway.py
+python docs/audit_2026-09-12/probe_geometry.py
+```
+
+The first writes `measurements.json`; the second prints the focused follow-up results. `scale_rank` in the first JSON uses float64's default tolerance (25); the effective float32 rank (24) and stable truncated-inverse check are recorded separately. Raw parameter maxima and geometric maxima use different sample counts, as specified above. The extra limit and backbone observations are recorded in `followup_measurements.json`.
diff --git a/docs/audit_2026-09-12/followup_measurements.json b/docs/audit_2026-09-12/followup_measurements.json
new file mode 100644
index 0000000..3edaea8
--- /dev/null
+++ b/docs/audit_2026-09-12/followup_measurements.json
@@ -0,0 +1,64 @@
+{
+ "geometry_samples": 256,
+ "dataset": "coco",
+ "seed": 20260912,
+ "scale_basis_rank_float32": 24,
+ "scale_basis_rank_float64": 25,
+ "scale_basis_pinv_rtol": 1e-05,
+ "stable_scale_projection_max_abs": 1.4955233162972092e-08,
+ "stable_scale_projection_max_abs_coeff": 3.551532238553479,
+ "stable_scale_projection_pve_mm": {
+ "mean": 1.431666341034088e-08,
+ "max": 0.00015275544137693942
+ },
+ "hand_zero_exact_nonfinger_keypoint_mm": {
+ "mean": 0.10175210982561111,
+ "max": 1.5162975788116455
+ },
+ "hand_zero_approx_nonfinger_keypoint_mm": {
+ "mean": 1.2984461784362793,
+ "max": 6.655439376831055
+ },
+ "shape_zero_exact_nonfinger_keypoint_mm": {
+ "mean": 1.049141526222229,
+ "max": 11.555269241333008
+ },
+ "shape_zero_pve_mm": 2.7963504791259766,
+ "coco_pose_limits": {
+ "n": 6000,
+ "indices": "6:130",
+ "tolerance": 1e-05,
+ "parameter_violation_fraction": 0.20225538313388824,
+ "sample_violation_fraction": 1.0,
+ "max_violation_rad": 0.8167469501495361,
+ "zero_width_limits": 24,
+ "total_local_rotation_limits": 124
+ },
+ "hgnetv2_b0": {
+ "backbone_parameters_without_head": 1850396,
+ "features_at_224": [
+ 1,
+ 1024,
+ 7,
+ 7
+ ]
+ },
+ "hgnetv2_b4": {
+ "backbone_parameters_without_head": 13553376,
+ "features_at_224": [
+ 1,
+ 2048,
+ 7,
+ 7
+ ]
+ },
+ "repvit_m2_3": {
+ "backbone_parameters_without_head": 22404232,
+ "features_at_224": [
+ 1,
+ 640,
+ 7,
+ 7
+ ]
+ }
+}
diff --git a/docs/audit_2026-09-12/measurements.json b/docs/audit_2026-09-12/measurements.json
new file mode 100644
index 0000000..7caa68f
--- /dev/null
+++ b/docs/audit_2026-09-12/measurements.json
@@ -0,0 +1,506 @@
+{
+ "seed": 20260912,
+ "scale_rank": 25,
+ "scale_singular_values": [
+ 0.4216929972171783,
+ 0.3619723954052365,
+ 0.3444689810276031,
+ 0.3400789604916061,
+ 0.3044403195381164,
+ 0.26572715914133743,
+ 0.2176144868135452,
+ 0.20565041106231255,
+ 0.1840951988319296,
+ 0.17085281014442444,
+ 0.148983675210781,
+ 0.14221401512622833,
+ 0.13806758695652796,
+ 0.12133844196796417,
+ 0.11143232759453013,
+ 0.10651327160577374,
+ 0.1045430526137352,
+ 0.09462177180850188,
+ 0.05311582610011101,
+ 0.04739458113908767,
+ 0.04616745188832282,
+ 0.04183576628565788,
+ 0.009205676615238188,
+ 0.006077288184314966,
+ 1.2910352609196704e-07,
+ 0.0,
+ 0.0,
+ 0.0
+ ],
+ "datasets": {
+ "coco": {
+ "n": 6000,
+ "root_fraction_abs_gt_2_8": 0.12844444811344147,
+ "root_near_pi_sample_fraction": 0.2123333364725113,
+ "root_near_gimbal_0_1_fraction": 0.021666666492819786,
+ "body_roundtrip_param_abs": {
+ "mean": 4.847996730237163e-09,
+ "p95": 2.9802322387695312e-08,
+ "max": 2.384185791015625e-07
+ },
+ "body_roundtrip_samples_gt_1e_4": 0,
+ "scale_projection_abs": {
+ "mean": 1.7601321293363869e-09,
+ "p95": 5.676704272622457e-09,
+ "max": 1.6244783407337593e-08
+ },
+ "root_roundtrip_rig_matrix_abs": {
+ "mean": 4.301777355269425e-08,
+ "p95": 1.7881393432617188e-07,
+ "max": 6.48200511932373e-07
+ },
+ "root_naive_parameter_abs": {
+ "mean": 0.6891651266078213,
+ "p95": 5.875875496864319,
+ "max": 6.276391506195068
+ },
+ "hand_roundtrip_param_abs": {
+ "mean": 1.5643703233787295e-09,
+ "p95": 7.450580596923828e-09,
+ "max": 1.1920928955078125e-07
+ },
+ "geometry_n": 256,
+ "body_roundtrip_skel_mm": {
+ "mean": 2.0324994928157182e-05,
+ "p95": 8.294915496662693e-05,
+ "max": 0.0002288818359375
+ },
+ "scale_projection_skel_mm": {
+ "mean": 1.95310167898661e-08,
+ "p95": 0.0,
+ "max": 7.631490007042885e-05
+ },
+ "scale_projection_pve_mm": {
+ "mean": 1.4316663921023e-08,
+ "p95": 0.0,
+ "max": 0.00015275544137693942
+ },
+ "hand_only_nonhand_kp_mm": {
+ "mean": 1.2984461276149117,
+ "p95": 4.054974603652953,
+ "max": 6.655439376831055
+ },
+ "skeleton_kp_vs_exact_mm": {
+ "mean": 1.262880131083348,
+ "p95": 6.897165751457212,
+ "max": 38.820594787597656
+ },
+ "skeleton_body_kp_vs_exact_mm": {
+ "mean": 2.9331194676983565,
+ "p95": 9.563913536071777,
+ "max": 38.820594787597656
+ },
+ "exact_kp_vs_annotation_mm": {
+ "mean": 0.1747331356528752,
+ "p95": 1.8100950360298131,
+ "max": 9.01367473602295
+ },
+ "skeleton_kp_vs_annotation_mm": {
+ "mean": 1.2424783247384557,
+ "p95": 6.975071215629569,
+ "max": 37.93433380126953
+ },
+ "pelvis_midhip_mm": {
+ "mean": 907.3023588657379,
+ "p95": 914.8213043212891,
+ "max": 920.49072265625
+ },
+ "root_canonical_skel_mm": {
+ "mean": 3.776603092655997e-05,
+ "p95": 0.0001314105553319671,
+ "max": 0.00038194627268239856
+ },
+ "hand_span_in_224_crop_px": {
+ "mean": 15.433155534744262,
+ "p95": 28.704643535614004,
+ "max": 90.4273452758789
+ },
+ "sampled_image_sizes": [
+ [
+ 224,
+ 224
+ ]
+ ]
+ },
+ "aic": {
+ "n": 1000,
+ "root_fraction_abs_gt_2_8": 0.04633333161473274,
+ "root_near_pi_sample_fraction": 0.07699999958276749,
+ "root_near_gimbal_0_1_fraction": 0.017999999225139618,
+ "body_roundtrip_param_abs": {
+ "mean": 4.803188577833974e-09,
+ "p95": 2.9802322387695312e-08,
+ "max": 2.384185791015625e-07
+ },
+ "body_roundtrip_samples_gt_1e_4": 0,
+ "scale_projection_abs": {
+ "mean": 1.7736721108909556e-09,
+ "p95": 5.722091468013386e-09,
+ "max": 1.5501413652696527e-08
+ },
+ "root_roundtrip_rig_matrix_abs": {
+ "mean": 3.3997358211005726e-08,
+ "p95": 1.341104507446289e-07,
+ "max": 4.172325134277344e-07
+ },
+ "root_naive_parameter_abs": {
+ "mean": 0.31238043495535384,
+ "p95": 0.924203491210937,
+ "max": 6.295416831970215
+ },
+ "hand_roundtrip_param_abs": {
+ "mean": 1.5494886858558697e-09,
+ "p95": 7.450580596923828e-09,
+ "max": 1.1920928955078125e-07
+ },
+ "geometry_n": 256,
+ "body_roundtrip_skel_mm": {
+ "mean": 1.9502637364221968e-05,
+ "p95": 7.867813110351562e-05,
+ "max": 0.00018688123964238912
+ },
+ "scale_projection_skel_mm": {
+ "mean": 1.6573496143283542e-08,
+ "p95": 0.0,
+ "max": 0.0001537753560114652
+ },
+ "scale_projection_pve_mm": {
+ "mean": 1.0493360439691971e-08,
+ "p95": 0.0,
+ "max": 0.00030577124562114477
+ },
+ "hand_only_nonhand_kp_mm": {
+ "mean": 1.3133377586958617,
+ "p95": 4.055889153480527,
+ "max": 7.546623229980469
+ },
+ "skeleton_kp_vs_exact_mm": {
+ "mean": 1.3220521317657032,
+ "p95": 7.123849296569824,
+ "max": 46.387332916259766
+ },
+ "skeleton_body_kp_vs_exact_mm": {
+ "mean": 3.0706044515078403,
+ "p95": 9.829323673248286,
+ "max": 46.387332916259766
+ },
+ "exact_kp_vs_annotation_mm": {
+ "mean": 0.15112333263938502,
+ "p95": 1.4468539297580705,
+ "max": 9.727354049682617
+ },
+ "skeleton_kp_vs_annotation_mm": {
+ "mean": 1.2894970125568872,
+ "p95": 7.168204927444458,
+ "max": 46.3869743347168
+ },
+ "pelvis_midhip_mm": {
+ "mean": 908.8520421981812,
+ "p95": 915.5803833007812,
+ "max": 941.2371826171875
+ },
+ "root_canonical_skel_mm": {
+ "mean": 3.112303484497241e-05,
+ "p95": 0.0001027138641802594,
+ "max": 0.0003773421049118042
+ },
+ "hand_span_in_224_crop_px": {
+ "mean": 14.884918254375458,
+ "p95": 26.578073787689206,
+ "max": 50.65440368652344
+ },
+ "sampled_image_sizes": [
+ [
+ 224,
+ 224
+ ]
+ ]
+ },
+ "mpii": {
+ "n": 1000,
+ "root_fraction_abs_gt_2_8": 0.1326666623353958,
+ "root_near_pi_sample_fraction": 0.2150000035762787,
+ "root_near_gimbal_0_1_fraction": 0.024000000208616257,
+ "body_roundtrip_param_abs": {
+ "mean": 4.869290418546849e-09,
+ "p95": 2.9802322387695312e-08,
+ "max": 1.1920928955078125e-07
+ },
+ "body_roundtrip_samples_gt_1e_4": 0,
+ "scale_projection_abs": {
+ "mean": 1.727188326217252e-09,
+ "p95": 5.578587583543446e-09,
+ "max": 1.5026239752469195e-08
+ },
+ "root_roundtrip_rig_matrix_abs": {
+ "mean": 4.29708905155874e-08,
+ "p95": 1.7881393432617188e-07,
+ "max": 5.364418029785156e-07
+ },
+ "root_naive_parameter_abs": {
+ "mean": 0.6591208255854047,
+ "p95": 5.836613655090332,
+ "max": 6.284758567810059
+ },
+ "hand_roundtrip_param_abs": {
+ "mean": 1.5445617368284586e-09,
+ "p95": 7.450580596923828e-09,
+ "max": 1.1920928955078125e-07
+ },
+ "geometry_n": 256,
+ "body_roundtrip_skel_mm": {
+ "mean": 2.0649441278268837e-05,
+ "p95": 8.529922342859209e-05,
+ "max": 0.0002288818359375
+ },
+ "scale_projection_skel_mm": {
+ "mean": 1.4238418234473052e-08,
+ "p95": 0.0,
+ "max": 7.864199869800359e-05
+ },
+ "scale_projection_pve_mm": {
+ "mean": 8.68299566776343e-09,
+ "p95": 0.0,
+ "max": 0.000152587890625
+ },
+ "hand_only_nonhand_kp_mm": {
+ "mean": 1.303009856767555,
+ "p95": 4.113340353965758,
+ "max": 7.0834760665893555
+ },
+ "skeleton_kp_vs_exact_mm": {
+ "mean": 1.408206587886578,
+ "p95": 7.6829523801803585,
+ "max": 51.369140625
+ },
+ "skeleton_body_kp_vs_exact_mm": {
+ "mean": 3.272702324564984,
+ "p95": 10.91020331382751,
+ "max": 51.369140625
+ },
+ "exact_kp_vs_annotation_mm": {
+ "mean": 0.19452631403964796,
+ "p95": 2.0200669407844525,
+ "max": 10.951011657714844
+ },
+ "skeleton_kp_vs_annotation_mm": {
+ "mean": 1.3965330520900545,
+ "p95": 7.800119304656979,
+ "max": 46.11688232421875
+ },
+ "pelvis_midhip_mm": {
+ "mean": 905.5695853233337,
+ "p95": 913.4701995849609,
+ "max": 932.9898071289062
+ },
+ "root_canonical_skel_mm": {
+ "mean": 4.161703200372286e-05,
+ "p95": 0.0001537753560114652,
+ "max": 0.00032259715953841805
+ },
+ "hand_span_in_224_crop_px": {
+ "mean": 13.570606416463852,
+ "p95": 20.74840297698974,
+ "max": 33.275596618652344
+ },
+ "sampled_image_sizes": [
+ [
+ 224,
+ 224
+ ]
+ ]
+ },
+ "harmony4d": {
+ "n": 1000,
+ "root_fraction_abs_gt_2_8": 0.20666666328907013,
+ "root_near_pi_sample_fraction": 0.36899998784065247,
+ "root_near_gimbal_0_1_fraction": 0.02199999988079071,
+ "body_roundtrip_param_abs": {
+ "mean": 4.859422403169584e-09,
+ "p95": 2.9802322387695312e-08,
+ "max": 1.1920928955078125e-07
+ },
+ "body_roundtrip_samples_gt_1e_4": 0,
+ "scale_projection_abs": {
+ "mean": 2.6347890349725476e-09,
+ "p95": 8.493963740696664e-09,
+ "max": 2.1240262909616447e-08
+ },
+ "root_roundtrip_rig_matrix_abs": {
+ "mean": 5.056530547638734e-08,
+ "p95": 1.9371509552001953e-07,
+ "max": 5.662441253662109e-07
+ },
+ "root_naive_parameter_abs": {
+ "mean": 1.0471845632053447,
+ "p95": 5.9726653575897215,
+ "max": 6.24089241027832
+ },
+ "hand_roundtrip_param_abs": {
+ "mean": 1.4260116559478989e-09,
+ "p95": 7.450580596923828e-09,
+ "max": 1.1920928955078125e-07
+ },
+ "geometry_n": 256,
+ "body_roundtrip_skel_mm": {
+ "mean": 2.2661963085494054e-05,
+ "p95": 8.529922342859209e-05,
+ "max": 0.00023515372595284134
+ },
+ "scale_projection_skel_mm": {
+ "mean": 2.287333476430838e-08,
+ "p95": 0.0,
+ "max": 7.630325853824615e-05
+ },
+ "scale_projection_pve_mm": {
+ "mean": 1.970640635590727e-08,
+ "p95": 0.0,
+ "max": 0.000152587890625
+ },
+ "hand_only_nonhand_kp_mm": {
+ "mean": 1.7946685821541275,
+ "p95": 5.6304475307464585,
+ "max": 17.68724250793457
+ },
+ "skeleton_kp_vs_exact_mm": {
+ "mean": 1.6157738262989496,
+ "p95": 8.519428968429562,
+ "max": 44.049713134765625
+ },
+ "skeleton_body_kp_vs_exact_mm": {
+ "mean": 3.749637672129704,
+ "p95": 12.776697397232054,
+ "max": 44.049713134765625
+ },
+ "exact_kp_vs_annotation_mm": {
+ "mean": 0.10912871654061149,
+ "p95": 0.0007656355737708509,
+ "max": 4.679005146026611
+ },
+ "skeleton_kp_vs_annotation_mm": {
+ "mean": 1.5388319493067095,
+ "p95": 8.519144773483273,
+ "max": 44.04958724975586
+ },
+ "pelvis_midhip_mm": {
+ "mean": 912.6007525920868,
+ "p95": 921.6853790283203,
+ "max": 930.41455078125
+ },
+ "root_canonical_skel_mm": {
+ "mean": 5.116117408168774e-05,
+ "p95": 0.00017625244800001383,
+ "max": 0.0003417296102270484
+ },
+ "hand_span_in_224_crop_px": {
+ "mean": 10.482691010951996,
+ "p95": 15.397425985336302,
+ "max": 29.435840606689453
+ },
+ "sampled_image_sizes": [
+ [
+ 224,
+ 224
+ ]
+ ]
+ },
+ "3dpw": {
+ "n": 1000,
+ "root_fraction_abs_gt_2_8": 0.11433333158493042,
+ "root_near_pi_sample_fraction": 0.2070000022649765,
+ "root_near_gimbal_0_1_fraction": 0.00800000037997961,
+ "body_roundtrip_param_abs": {
+ "mean": 4.315976630643039e-09,
+ "p95": 2.9802322387695312e-08,
+ "max": 2.384185791015625e-07
+ },
+ "body_roundtrip_samples_gt_1e_4": 0,
+ "scale_projection_abs": {
+ "mean": 2.0754229214372135e-09,
+ "p95": 6.65673839800318e-09,
+ "max": 2.031111703826305e-08
+ },
+ "root_roundtrip_rig_matrix_abs": {
+ "mean": 4.3235324281138474e-08,
+ "p95": 1.7881393432617188e-07,
+ "max": 6.258487701416016e-07
+ },
+ "root_naive_parameter_abs": {
+ "mean": 0.7765007006836385,
+ "p95": 5.773379302024838,
+ "max": 6.353257179260254
+ },
+ "hand_roundtrip_param_abs": {
+ "mean": 1.2745278679943238e-09,
+ "p95": 7.450580596923828e-09,
+ "max": 1.1920928955078125e-07
+ },
+ "geometry_n": 256,
+ "body_roundtrip_skel_mm": {
+ "mean": 2.151847284910666e-05,
+ "p95": 8.529922342859209e-05,
+ "max": 0.0002157918643206358
+ },
+ "scale_projection_skel_mm": {
+ "mean": 1.4343623989342038e-08,
+ "p95": 0.0,
+ "max": 7.630325853824615e-05
+ },
+ "scale_projection_pve_mm": {
+ "mean": 9.343029776613385e-09,
+ "p95": 0.0,
+ "max": 0.000152587890625
+ },
+ "hand_only_nonhand_kp_mm": {
+ "mean": 1.1839595396388778,
+ "p95": 3.4480524182319616,
+ "max": 7.466592311859131
+ },
+ "skeleton_kp_vs_exact_mm": {
+ "mean": 1.420894855516573,
+ "p95": 7.527710986137388,
+ "max": 40.37519454956055
+ },
+ "skeleton_body_kp_vs_exact_mm": {
+ "mean": 3.2999284368693274,
+ "p95": 10.43739891052244,
+ "max": 40.37519454956055
+ },
+ "exact_kp_vs_annotation_mm": {
+ "mean": 0.3037412910945271,
+ "p95": 2.8356644988059996,
+ "max": 9.90914535522461
+ },
+ "skeleton_kp_vs_annotation_mm": {
+ "mean": 1.4251954034570269,
+ "p95": 7.732362079620358,
+ "max": 40.375099182128906
+ },
+ "pelvis_midhip_mm": {
+ "mean": 910.3145744800568,
+ "p95": 920.9322509765625,
+ "max": 930.5852661132812
+ },
+ "root_canonical_skel_mm": {
+ "mean": 3.8249140468245345e-05,
+ "p95": 0.00015843627625145018,
+ "max": 0.0004565700073726475
+ },
+ "hand_span_in_224_crop_px": {
+ "mean": 14.849500558376313,
+ "p95": 20.30525197982788,
+ "max": 35.22533416748047
+ },
+ "sampled_image_sizes": [
+ [
+ 224,
+ 224
+ ]
+ ]
+ }
+ }
+}
diff --git a/docs/audit_2026-09-12/probe_geometry.py b/docs/audit_2026-09-12/probe_geometry.py
new file mode 100644
index 0000000..8b097ab
--- /dev/null
+++ b/docs/audit_2026-09-12/probe_geometry.py
@@ -0,0 +1,51 @@
+"""Read-only CPU audit; run from any directory in the existing sam3d_video_312 environment.
+Requires the adjacent sam3-biomechanics checkout and its DINOv3 checkpoint.
+"""
+import torch,numpy as np,json
+from pathlib import Path
+import importlib.util
+
+torch.set_num_threads(4)
+root=Path(__file__).resolve().parents[2]
+s=torch.load(root.parent/'sam3-biomechanics/video_to_pose_pipeline/checkpoints/sam-3d-body-dinov3/model.ckpt',map_location='cpu',weights_only=False,mmap=True)
+rig=torch.jit.load(str(root/'checkpoints/mhr_model.pt'),map_location='cpu').eval()
+for param in rig.parameters(): param.requires_grad_(False)
+ct=rig.character_torch
+print('limits repr',ct.parameter_limits)
+print('limits attributes',ct.parameter_limits._c.dump_to_str(True,False,False)[:5000])
+paths=sorted((root/'data/sam3d_gt_coco/annotations').glob('*.npz'))
+rng=np.random.default_rng(20260912)
+paths=[paths[i] for i in rng.choice(len(paths),6000,replace=False)[:256]]
+r=[]
+for path in paths:
+ with np.load(path) as z:r.append((z['mhr_model_params'],z['shape_params']))
+p,h=[torch.from_numpy(np.stack([a[i] for a in r])).float() for i in range(2)]
+C=s['head_pose.scale_comps'].double(); mu=s['head_pose.scale_mean'].double()
+coeff=(p[:,136:].double()-mu)@torch.linalg.pinv(C,rtol=1e-5)
+print('stable_pca_max_residual',float((coeff@C+mu-p[:,136:]).abs().max()),'coeff_max',float(coeff.abs().max()))
+W=torch.from_numpy(np.load(root/'instanthmr_distill_train/assets/mhr_j127_to_kp70.npy')).float()
+K=s['head_pose.keypoint_mapping'][:70].float()
+hand=torch.cat([s['head_pose.hand_joint_idxs_left'],s['head_pose.hand_joint_idxs_right']]).long()
+body=list(range(21))+[41]+list(range(62,70))
+err=[]; exacterr=[]; approxerr=[]; shapeerr=[]; shapekp=[]; pcerr=[]; pcent=[]
+for start in range(0,len(p),16):
+ pp=p[start:start+16]; hh=h[start:start+16]
+ with torch.no_grad():
+ vv,ss=rig(hh,pp,torch.zeros(len(pp),72)); ss=ss[...,:3]; [email protected]([vv,ss],1); ka=W@ss
+ err.append((ka-kp).norm(dim=-1)*10)
+ ph=pp.clone(); ph[:,hand]=0
+ vh,sh=rig(hh,ph,torch.zeros(len(pp),72)); [email protected]([vh,sh[...,:3]],1)
+ exacterr.append((kh-kp).norm(dim=-1)*10); approxerr.append(((W@sh[...,:3])-ka).norm(dim=-1)*10)
+ v0,s0=rig(torch.zeros_like(hh),pp,torch.zeros(len(pp),72)); [email protected]([v0,s0[...,:3]],1)
+ shapeerr.append((v0-vv).norm(dim=-1)*10);shapekp.append((k0-kp).norm(dim=-1)*10)
+ pc=pp.clone();pc[:,136:]=(coeff[start:start+16]@C+mu).float()
+ vc,sc=rig(hh,pc,torch.zeros(len(pp),72));pcerr.append((vc-vv).norm(dim=-1)*10)
+ # Rig origin is not hip-normalized: measure change of midhip induced by scale only.
+ [email protected]([vc,sc[...,:3]],1)
+ pcent.append((((kp[:,9]+kp[:,10])-(kc[:,9]+kc[:,10]))/2).norm(dim=-1)*10)
+for name,vals in [('regressor error',err),('hand zero exact',exacterr),('hand zero approx',approxerr),('shape zero exactkp',shapekp)]:
+ e=torch.cat(vals);print(name,'body mean/max',e[:,body].mean().item(),e[:,body].max().item(),'top means',[(i,e[:,i].mean().item()) for i in e.mean(0).argsort(descending=True)[:12].tolist()])
+print('pca24 PVE mean/max',torch.cat(pcerr).mean().item(),torch.cat(pcerr).max().item())
+print('pca24 midhip movement max',torch.cat(pcent).max().item())
+print('shapezero pve',torch.cat(shapeerr).mean().item())
+print('student regressor row sum range',W.sum(1).min().item(),W.sum(1).max().item())
diff --git a/docs/audit_2026-09-12/probe_pathway.py b/docs/audit_2026-09-12/probe_pathway.py
new file mode 100644
index 0000000..f257674
--- /dev/null
+++ b/docs/audit_2026-09-12/probe_pathway.py
@@ -0,0 +1,102 @@
+"""Read-only CPU audit; run from any directory in the existing sam3d_video_312 environment.
+Requires the adjacent sam3-biomechanics checkout and its DINOv3 checkpoint.
+"""
+import importlib.util, json, sys
+from pathlib import Path
+import numpy as np
+import torch
+import roma
+from PIL import Image
+
+torch.set_num_threads(4)
+root=Path(__file__).resolve().parents[2]
+teacher=root.parent/'sam3-biomechanics/sam-3d-body'
+modpath=teacher/'sam_3d_body/models/modules/mhr_utils.py'
+spec=importlib.util.spec_from_file_location('mhr_utils',modpath)
+u=importlib.util.module_from_spec(spec); spec.loader.exec_module(u)
+s=torch.load(root.parent/'sam3-biomechanics/video_to_pose_pipeline/checkpoints/sam-3d-body-dinov3/model.ckpt',map_location='cpu',weights_only=False,mmap=True)
+C=s['head_pose.scale_comps'].double(); mu=s['head_pose.scale_mean'].double(); pinv=torch.linalg.pinv(C)
+print('scale singular values',torch.linalg.svdvals(C).tolist(),flush=True)
+rig=torch.jit.load(str(root/'checkpoints/mhr_model.pt'),map_location='cpu').eval()
+for p in rig.parameters(): p.requires_grad_(False)
+W=torch.from_numpy(np.load(root/'instanthmr_distill_train/assets/mhr_j127_to_kp70.npy')).float()
+K=s['head_pose.keypoint_mapping'][:70].float()
+hand_idx=torch.cat([s['head_pose.hand_joint_idxs_left'],s['head_pose.hand_joint_idxs_right']]).long()
+body_idx=[i for i in range(70) if i not in list(range(21,41))+list(range(42,62))]
+print('hand rig idx',hand_idx.tolist(),flush=True)
+print('keypoint vertex support',int((K[:,:18439].abs().sum(0)>0).sum()),flush=True)
+
+def stat(x):
+ x=torch.as_tensor(x).double().reshape(-1)
+ return {k:float(v) for k,v in [('mean',x.mean()),('p95',torch.quantile(x,.95)),('max',x.max())]}
+
+def skel(p,h):
+ ct=rig.character_torch
+ return ct.joint_parameters_to_skeleton_state(ct.model_parameters_to_joint_parameters(torch.cat([p,h],1)))[...,:3]
+
+out={'seed':20260912,'scale_rank':int(torch.linalg.matrix_rank(C)), 'scale_singular_values':torch.linalg.svdvals(C).tolist(),'datasets':{}}
+for split,n in [('coco',6000),('aic',1000),('mpii',1000),('harmony4d',1000),('3dpw',1000)]:
+ paths=sorted((root/f'data/sam3d_gt_{split}/annotations').glob('*.npz'))
+ rng=np.random.default_rng(20260912)
+ chosen=[paths[i] for i in rng.choice(len(paths),min(n,len(paths)),replace=False)]
+ records=[]
+ for path in chosen:
+ with np.load(path) as z: records.append([z[k].copy() for k in ['mhr_model_params','shape_params','joints_3d','joints_2d','bbox_square']])
+ p,h,j,j2,bbox=[torch.from_numpy(np.stack([r[i] for r in records])).float() for i in range(5)]
+ b=torch.cat([p[:,6:136],torch.zeros(len(p),3)],1)
+ bc=u.compact_model_params_to_cont_body(b); br=u.compact_cont_to_model_params_body(bc)
+ pr=p.clone(); pr[:,6:136]=br[:,:130]
+ scales=((p[:,136:].double()-mu)@pinv)@C+mu
+ rootR=roma.euler_to_rotmat('ZYX',p[:,3:6])
+ root_e=roma.rotmat_to_euler('ZYX',rootR)
+ root_rig=roma.euler_to_rotmat('xyz',p[:,3:6])
+ round_rig=roma.euler_to_rotmat('xyz',root_e)
+ naive_e=roma.rotmat_to_euler('ZYX',u.batch6DFromXYZ(p[:,3:6],True))
+ d={'n':len(p),'root_fraction_abs_gt_2_8':float((p[:,3:6].abs()>2.8).float().mean()),'root_near_pi_sample_fraction':float(((p[:,3:6].abs()-torch.pi).abs()<.34).any(1).float().mean()),'root_near_gimbal_0_1_fraction':float(((p[:,4].abs()-torch.pi/2).abs()<.1).float().mean()),'body_roundtrip_param_abs':stat((br-b).abs()),'body_roundtrip_samples_gt_1e_4':int(((br-b).abs().amax(1)>1e-4).sum()),'scale_projection_abs':stat((scales-p[:,136:]).abs()),'root_roundtrip_rig_matrix_abs':stat((round_rig-root_rig).abs()),'root_naive_parameter_abs':stat((naive_e-p[:,3:6]).abs())}
+ # Hand roundtrip through the distinct, joint-ordered conversion.
+ hr=p.clone()
+ for ids in [s['head_pose.hand_joint_idxs_left'].long(),s['head_pose.hand_joint_idxs_right'].long()]:
+ hr[:,ids]=u.compact_cont_to_model_params_hand(u.compact_model_params_to_cont_hand(p[:,ids]))
+ d['hand_roundtrip_param_abs']=stat((hr-p).abs())
+ # Full FK probes on a fixed subset; cm -> mm. Keep runtime and memory bounded.
+ errs={k:[] for k in ['body_roundtrip_skel_mm','scale_projection_skel_mm','scale_projection_pve_mm','hand_only_nonhand_kp_mm','skeleton_kp_vs_exact_mm','skeleton_body_kp_vs_exact_mm','exact_kp_vs_annotation_mm','skeleton_kp_vs_annotation_mm','pelvis_midhip_mm','root_canonical_skel_mm']}
+ for start in range(0,min(256,len(p)),16):
+ pp=p[start:start+16]; hh=h[start:start+16]
+ with torch.no_grad():
+ vv,ss=rig(hh,pp,torch.zeros(len(pp),72)); ss=ss[...,:3]
+ [email protected]([vv,ss],1)
+ kp_s=W@ss
+ errs['skeleton_kp_vs_exact_mm'].append((kp-kp_s).norm(dim=-1)*10)
+ errs['skeleton_body_kp_vs_exact_mm'].append((kp[:,body_idx]-kp_s[:,body_idx]).norm(dim=-1)*10)
+ sign=torch.tensor([1.,-1.,-1.])
+ errs['exact_kp_vs_annotation_mm'].append((kp*sign/100-j[start:start+16]).norm(dim=-1)*1000)
+ errs['skeleton_kp_vs_annotation_mm'].append((kp_s*sign/100-j[start:start+16]).norm(dim=-1)*1000)
+ errs['pelvis_midhip_mm'].append(((kp[:,9]+kp[:,10])/2).norm(dim=-1)*10)
+ errs['body_roundtrip_skel_mm'].append((skel(pr[start:start+16],hh)-ss).norm(dim=-1)*10)
+ pc=pp.clone(); pc[:,136:]=scales[start:start+16].float()
+ vc,sc=rig(hh,pc,torch.zeros(len(pp),72))
+ errs['scale_projection_skel_mm'].append((sc[...,:3]-ss).norm(dim=-1)*10)
+ errs['scale_projection_pve_mm'].append((vc-vv).norm(dim=-1)*10)
+ ph=pp.clone(); ph[:,hand_idx]=0
+ sh=skel(ph,hh)
+ errs['hand_only_nonhand_kp_mm'].append(((W@sh)[:,body_idx]-kp_s[:,body_idx]).norm(dim=-1)*10)
+ proot=pp.clone(); proot[:,3:6]=root_e[start:start+16]
+ errs['root_canonical_skel_mm'].append((skel(proot,hh)-ss).norm(dim=-1)*10)
+ d['geometry_n']=min(256,len(p))
+ d.update({k:stat(torch.cat(v).flatten()) for k,v in errs.items()})
+ # Visible hand extents within the stored 224px body crop, without claiming visibility labels exist.
+ ext=[]
+ for ids in [list(range(21,42)),list(range(42,63))]:
+ xy=j2[:,ids]; side=(xy.amax(1)-xy.amin(1)).amax(1)
+ ext.append(side/(bbox[:,2]-bbox[:,0])*224)
+ d['hand_span_in_224_crop_px']=stat(torch.cat(ext))
+ imgs=[]
+ for path in chosen[:16]:
+ im=root/f'data/sam3d_gt_{split}/images/{path.stem}.png'
+ if not im.exists(): im=im.with_suffix('.jpg')
+ if im.exists():
+ with Image.open(im) as image: imgs.append(list(image.size))
+ d['sampled_image_sizes']=sorted({tuple(i) for i in imgs})
+ out['datasets'][split]=d
+ print(split,json.dumps(d),flush=True)
+Path(__file__).with_name('measurements.json').write_text(json.dumps(out,indent=2)+'\n')