Python
docs: v3 framing-robustness spec and tickets, glossary, agent-skill config; seed-spread note updated
mohamdevdca353d95480
diff --git a/.scratch/v3-framing-robustness/issues/01-archive-aic-mpii-originals.md b/.scratch/v3-framing-robustness/issues/01-archive-aic-mpii-originals.mdnew file mode 100644index 0000000..6bcc3f6--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/01-archive-aic-mpii-originals.md@@ -0,0 +1,11 @@+# 01: Archive the AIC and MPII original frames to $STORE++**What to build:** every built split has a tar in `$STORE`, so the `$SCRATCH` 30-day purge (AIC and MPII originals last read 1–2 Sept 2026, due around 1–2 Oct) costs a restore instead of a rebuild. The archive template's placeholder account has been fixed to `vsi@v100`; job 189727 was submitted with `REPO=$PWD` on the free `archive` partition.++**Blocked by:** None (can start immediately; already queued as job 189727).++**Status:** ready-for-human++- [ ] `hmr_archive_<jobid>.out` ends with an `ls -lh` listing five tars: aic, coco, mpii, sa1b, harmony4d+- [ ] The aic and mpii tars hold real frames (hard links/copies), not links: `tar tvf` shows regular files under `original_images/`+- [ ] If the job timed out on sa1b, the partial sa1b tar was deleted before resubmitting (the script skips any existing tar, including a half-written one)diff --git a/.scratch/v3-framing-robustness/issues/02-lr-schedule-sweep.md b/.scratch/v3-framing-robustness/issues/02-lr-schedule-sweep.mdnew file mode 100644index 0000000..a6545f7--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/02-lr-schedule-sweep.md@@ -0,0 +1,12 @@+# 02: Learning-rate x schedule-length sweep on the reference model++**What to build:** find out whether the underfitting reference model (train loss 2.340 above val 2.302 at epoch 299; 3DPW val still falling as the LR reached zero) gains from a better recipe. 2x2 grid on g8h, current corpus, baseline preset, 1 seed per cell: peak LR {6e-4, 1.2e-3} x length {300, 600 epochs}. The (6e-4, 300) cell is the existing g8h_s0, so three new runs. Launched through the usual Slurm launcher with the generation-8 flags plus the LR/epoch overrides.++**Blocked by:** None (can start immediately).++**Status:** ready-for-human++- [ ] Launch commands printed in the launcher's form (`RUN_NAME=... SEED=... EPOCHS=... MIX=... EXTRA_TRAIN_ARGS=... sbatch`), with enough `afterany` links for 600 epochs (~8 x 20 h)+- [ ] Verified that the LR the trainer actually uses is the intended peak after its sqrt(world size) scaling, and that the one-cycle schedule spans the full 600 epochs (read from `history.jsonl` `lr` at epoch ~60 and at the end)+- [ ] Each finished run scored on the published protocol (EMDB-1 with `--bbox annotated`, SMPL-24 adapter) and on 3DPW test J14 with the adapter+- [ ] A 2x2 table recorded in the status doc, naming the winning recipe or stating that no cell beats g8h_s0 by more than ~0.5 mmdiff --git a/.scratch/v3-framing-robustness/issues/03-profile-cpu-tier-backbones.md b/.scratch/v3-framing-robustness/issues/03-profile-cpu-tier-backbones.mdnew file mode 100644index 0000000..445bc7b--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/03-profile-cpu-tier-backbones.md@@ -0,0 +1,12 @@+# 03: Profile candidate CPU-tier backbones on the reference phones++**What to build:** pick the CPU-tier model-family backbone from measured latency, before any training. Random-weight exports of the current architecture (continuous head, exact landmarks, bounded scales, CLIFF focal) with HGNetV2-B0, B1 and B4 (anchor) at 224 px, profiled on AI Hub.++**Blocked by:** None (can start immediately).++**Status:** ready-for-agent++- [ ] The B4 random-weight export's latency matches the trained g8h's within noise on the S23 CPU (63 ms) — the check that random weights are a valid latency proxy+- [ ] CPU-only latency and peak memory for B0, B1 and B4 on Galaxy S23, A73, A53, A14 and Pixel 8; S23 NPU latency and placement for each+- [ ] Every export places 100% of layers on the NPU on the S23 and passes fp16 parity+- [ ] A table recorded in the status doc and a recommended CPU-tier size against the 33 ms budget on mid-range phones (A73 / Pixel 8)diff --git a/.scratch/v3-framing-robustness/issues/04-context-crop-builder.md b/.scratch/v3-framing-robustness/issues/04-context-crop-builder.mdnew file mode 100644index 0000000..5cebc21--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/04-context-crop-builder.md@@ -0,0 +1,13 @@+# 04: Context-crop builder++**What to build:** the crop builder can store a context crop, a square covering a chosen multiple of the person's tight box (default 1.2, today's) at a chosen size. It records the stored square in original-frame coordinates next to the existing 1.2x square, so any window can be recomputed exactly. The rebuild setting is context 2.0 at 448 px.++**Blocked by:** None (can start immediately).++**Status:** ready-for-agent++- [ ] With default flags the builder's output (crops and npz) is byte-identical to the current builder's on a real local split+- [ ] With context 2.0 / 448, the 1.2x window cut from the context crop and resized to 224 matches today's 224 crop of the same person (PSNR reported, mean and min)+- [ ] The recorded geometry maps the annotation targets' 2D keypoints onto the context crop to 0.000 px+- [ ] Where the context square leaves the frame, the existing frame-edge padding applies and is flagged as such in the npz+- [ ] Works through the array build job and the streaming script unchanged apart from the new flagsdiff --git a/.scratch/v3-framing-robustness/issues/05-input-size-setting.md b/.scratch/v3-framing-robustness/issues/05-input-size-setting.mdnew file mode 100644index 0000000..8996d7c--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/05-input-size-setting.md@@ -0,0 +1,12 @@+# 05: Input size as a checkpoint setting (prefactor)++**What to build:** one input-size setting (default 224) drives every place that builds the network input: the training dataset, 3DPW validation, the benchmark harness and the inference package. It is saved in the run config, recovered by config-from-checkpoint (a checkpoint without it means 224), and stamped into the ONNX metadata beside the CLIFF form, so the phone app can read its crop size from the model file.++**Blocked by:** None (can start immediately).++**Status:** ready-for-agent++- [ ] baseline and v2 augmented samples are bit-identical against `git show HEAD`, compared on alternating calls+- [ ] g8h_s0 evaluated through the harness gives the same 3DPW test J14 and EMDB-1 numbers as recorded, to the decimal+- [ ] A random-weight model configured for 288 runs end to end through the 3DPW and EMDB harnesses, and its ONNX metadata reads 288+- [ ] `ddp_smoke` passes at 224 and at 288diff --git a/.scratch/v3-framing-robustness/issues/06-profile-288-input.md b/.scratch/v3-framing-robustness/issues/06-profile-288-input.mdnew file mode 100644index 0000000..49872c1--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/06-profile-288-input.md@@ -0,0 +1,11 @@+# 06: Profile a 288 px input on the reference phones++**What to build:** decide whether the 288 px arm is deployable before GPU hours are spent on it. A random-weight B4 export at 288, profiled like ticket 03.++**Blocked by:** 05++**Status:** ready-for-agent++- [ ] NPU latency on Galaxy S22, S23, S24 and S25; CPU latency on S23, A73, A53, A14 and Pixel 8+- [ ] 100% NPU placement and fp16 parity on the S23+- [ ] Result recorded next to ticket 03's table, with the 224-vs-288 latency ratio per phonediff --git a/.scratch/v3-framing-robustness/issues/07-v3-preset.md b/.scratch/v3-framing-robustness/issues/07-v3-preset.mdnew file mode 100644index 0000000..f1f6590--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/07-v3-preset.md@@ -0,0 +1,14 @@+# 07: v3 preset, proven on local data++**What to build:** `--preset v3` trains from context crops. Per sample it draws the effective person box log-uniformly between 0.6x and 2.0x the tight box, shifts its centre by up to +/-15% of the box, clamps the window inside the stored context, and maps the context crop to the network input in one resampling step (with v2's rotation and flip). The CLIFF vector (perspective-correct form) describes the sampled box. The bounds are config values with these defaults.++**Blocked by:** 04, 05++**Status:** ready-for-agent++- [ ] A small local split rebuilt with ticket 04's builder (context 2.0 / 448) trains under v3 at 224 and at 288+- [ ] Verification on real samples: 2D labels land on the right pixels; CLIFF crop-centre error at the rounding floor (mean / p95 / max px, as in the crop-centre-fix check); no black pixel inside the stored context; box-scale draws cover 0.6–2.0x; output size equals the configured input size+- [ ] baseline and v2 samples bit-identical against HEAD (alternating calls)+- [ ] The v3 dataset refuses a split without context geometry, with a message naming the split+- [ ] `ddp_smoke` passes for v3 at 224 and at 288+- [ ] Files that must be rsynced to the cluster listed in the status docdiff --git a/.scratch/v3-framing-robustness/issues/08-rebuild-four-splits.md b/.scratch/v3-framing-robustness/issues/08-rebuild-four-splits.mdnew file mode 100644index 0000000..ae23bc4--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/08-rebuild-four-splits.md@@ -0,0 +1,13 @@+# 08: Rebuild coco, sa1b, aic and mpii as context crops on Jean Zay++**What to build:** the four splits whose original frames are on the cluster, rebuilt with the array build job at context 2.0 / 448 into new directories that the baseline and v2 presets never pick up, then verified and archived.++**Blocked by:** 01, 04++**Status:** ready-for-human++- [ ] Launch commands printed in the build job's form, with the new flags and output directories+- [ ] Crop counts match the current splits minus images that are missing at source (report the difference)+- [ ] Verification passes per split (reprojection RMS ~0.000 px, zero missing or orphan)+- [ ] Each split archived as one tar to `$STORE`+- [ ] `$SCRATCH` usage recorded (expected ~250 GB total with ticket 09)diff --git a/.scratch/v3-framing-robustness/issues/09-rebuild-harmony4d.md b/.scratch/v3-framing-robustness/issues/09-rebuild-harmony4d.mdnew file mode 100644index 0000000..def5f0e--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/09-rebuild-harmony4d.md@@ -0,0 +1,11 @@+# 09: Rebuild Harmony4D as context crops++**What to build:** Harmony4D rejoins the corpus as context crops. Its original frames exist nowhere, so its zips are streamed scene by scene: download, undistort, build context crops, delete. The route depends on the `prepost` download test: HTTP 206 means the streaming job runs on Jean Zay (keeping no 4K originals); otherwise the same script runs on the laptop and the crops are rsynced as one tar.++**Blocked by:** 04, and the `prepost` curl result (job 190182)++**Status:** ready-for-human++- [ ] Route chosen from the `prepost` result and recorded+- [ ] Undistortion blockers in the undistortion notes handled before the first full scene+- [ ] 496,036 train crops (± source-missing), verified and archived to `$STORE`diff --git a/.scratch/v3-framing-robustness/issues/10-v3-ablation.md b/.scratch/v3-framing-robustness/issues/10-v3-ablation.mdnew file mode 100644index 0000000..1c98df7--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/10-v3-ablation.md@@ -0,0 +1,12 @@+# 10: The v3 ablation++**What to build:** measure what the v3 preset and a larger input buy. Three arms, 2 seeds each, on the rebuilt corpus's data mix: a g8h control (baseline preset on the same mix), v3 at 224 and v3 at 288. Current generation-8 recipe unless ticket 02 has already produced a winner.++**Blocked by:** 06, 07, 08, 09++**Status:** ready-for-human++- [ ] Launch commands printed in the launcher's form, with the rsync list from ticket 07 and a cluster-side `grep -c` check that the new trainer is in place+- [ ] Every checkpoint scored on EMDB-1 published protocol, on EMDB under 1.2x / 1.4x / 1.6x enlarged boxes, and on 3DPW test J14 with the adapter+- [ ] A table of accuracy against latency (from tickets 03 and 06), stating whether v3 reduces the EMDB published-protocol penalty by more than ~0.5 mm without losing on 3DPW+- [ ] Results recorded in the benchmark results and status docsdiff --git a/.scratch/v3-framing-robustness/issues/11-train-model-family.md b/.scratch/v3-framing-robustness/issues/11-train-model-family.mdnew file mode 100644index 0000000..4d1b394--- /dev/null+++ b/.scratch/v3-framing-robustness/issues/11-train-model-family.md@@ -0,0 +1,11 @@+# 11: Train the model family++**What to build:** the paper's accuracy-vs-latency curve. The CPU-tier backbone(s) chosen in ticket 03, trained with the winning v3 setting from ticket 10 and the winning recipe from ticket 02, beside the NPU-tier (B4) member.++**Blocked by:** 02, 03, 10++**Status:** ready-for-human++- [ ] Each member scored on EMDB-1 published protocol and 3DPW test J14, 2 seeds+- [ ] Each member profiled on the reference phones: latency, NPU placement, fp16 parity+- [ ] The accuracy-vs-latency table recorded, with each member's deployment tierdiff --git a/.scratch/v3-framing-robustness/spec.md b/.scratch/v3-framing-robustness/spec.mdnew file mode 100644index 0000000..7a9b0db--- /dev/null+++ b/.scratch/v3-framing-robustness/spec.md@@ -0,0 +1,108 @@+# Spec: framing-robust, resolution-flexible training (v3 preset) and a deployable model family++Status: ready-for-agent++## Problem Statement++The reference model (g8h) scores 51.9 mm SMPL-24 PA-MPJPE on EMDB-1 when it is given tight boxes built from the ground-truth joints, which is the framing every training crop has. Under the published protocol, which uses EMDB's own looser and off-centre boxes, it scores 64.3 / 66.9 mm (seeds 0 / 1): a penalty of 12–15 mm that puts it level with SMPLer-X-H and about 21 mm behind CameraHMR. The penalty has grown since generation 5 (+7.0 mm) and generation 6 (+8.3 mm), and it differs by 2.6 mm between two seeds that agree to 0.3 mm on tight boxes, which is the signature of behaviour that training never constrains.++Enlarging our own boxes shows where it breaks: 1.4x the tight box costs 2.2 mm, 1.6x costs 16.2 mm. 1.6x is exactly the loosest framing training can produce, and at that point training has only ever shown black borders, because every stored crop is cut at exactly 1.2x the tight box and saved at the network's 224 px input size. No pixel of real scene outside that square exists in the corpus. The v2 preset cannot fix this: it adds zoom-in (truncation), which is the opposite direction, and its zoom-in upsamples the 224 crop, training the model on blur a real crop would not have.++The same fragility hits the product. On the phone, crops come from the NanoDet detector, whose boxes are often too large, and people often stand very close to the camera and are heavily truncated.++Separately:++- The model is underfitting. Training loss sits above validation loss, and it was still falling when the learning rate reached zero. Learning rate and schedule length have never been tuned.+- The input resolution has never been varied, because the stored crops cap it at 224.+- g8h runs in 1.4–4.1 ms on Snapdragon 8 Gen 1 and newer NPUs, but in 63–196 ms on phone CPUs, against a 33 ms (30 FPS) budget. The CPU deployment tier has no model that fits.++## Solution++The corpus is rebuilt once, from original frames, with twice the context around each person and at 448 px. A new v3 preset samples each training crop from that larger stored area with a wide, bounded range of framings: loose and off-centre like EMDB's and the detector's boxes, and tight and truncated like a person close to the camera. The surrounding area is always real scene, never black. Because the stored crops have resolution to spare, the network input can be 224 or larger without upsampling.++Accuracy is judged on the published protocol, and v3 is compared against g8h on two seeds each, at 224 and at 288. Independently, and on the existing corpus, a small learning-rate x schedule-length sweep tests whether the underfitting model gains from a better recipe. The winning recipe is then trained at smaller backbone sizes. The sizes are picked beforehand by profiling random-weight exports on the same hosted phones, so the model family gives one accuracy-vs-latency curve with a member for each deployment tier.++## User Stories++1. As the researcher, I want the crop builder to accept a context factor and an output size, so that I can store crops with real scene around each person at a resolution above the network input.+2. As the researcher, I want the builder with default flags to produce byte-identical output to today's, so that the existing corpus, the baseline and v2 presets, and every recorded result stay valid.+3. As the researcher, I want each stored crop to record where the standard 1.2x window sits inside it, so that the trainer and any verification can recover today's framing exactly.+4. As the researcher, I want a verification that cuts the 1.2x window out of a rebuilt crop and compares it with today's 224 crop, so that I can prove the rebuild shows the same person in the same place.+5. As the researcher, I want the rebuilt crop's stored geometry to map the annotation targets' 2D keypoints onto the new crop to 0.000 px, so that no label is silently shifted by the rebuild.+6. As the researcher, I want the rebuilt corpus written beside the existing one rather than over it, so that runs in flight and old checkpoints keep reading the 224 corpus.+7. As the researcher, I want the four splits whose original frames are on Jean Zay (coco, sa1b, aic, mpii) rebuilt with the existing array build job, so that the rebuild reuses tooling that already works on the cluster.+8. As the researcher, I want Harmony4D rebuilt by streaming its original zips scene by scene, so that the one split without original frames rejoins the corpus at the new framing.+9. As the researcher, I want the Harmony4D streaming route chosen from the result of a download test on the `prepost` partition, so that the rebuild runs on Jean Zay when the network allows and on my laptop when it does not.+10. As the researcher, I want each rebuilt split verified (crop count, reprojection, missing images) and archived to `$STORE`, so that the 30-day `$SCRATCH` purge costs a restore, not a rebuild.+11. As the researcher, I want the AIC and MPII original frames archived before any rebuild starts, so that the purge due around 1–2 October 2026 cannot delete the only on-cluster copy.+12. As the researcher, I want a v3 preset selectable with `--preset v3`, so that it sits beside baseline and v2 with the same interface.+13. As the researcher, I want v3 to sample the effective person box over a wide range, from tight truncation to loose framing, so that the model sees the framings that EMDB boxes, detector boxes and close-up people produce.+14. As the researcher, I want v3 to shift the box centre randomly, so that off-centre framing like EMDB's 5.7% offset is part of training.+15. As the researcher, I want the v3 sampler to keep the sampled window inside the stored context, so that black borders never appear except where the real frame itself ends.+16. As the researcher, I want the v3 crop sampled directly from the stored high-resolution crop in one resampling step, so that zoom-in never upsamples a 224 intermediate.+17. As the researcher, I want the CLIFF conditioning to describe the box actually sampled in v3, so that the camera and depth placement stays correct under the new framings.+18. As the researcher, I want the network input size to be a training option (224 by default, 288 for the resolution ablation), so that I can test whether resolution buys accuracy.+19. As the researcher, I want the input size recorded in the run config and read back when a checkpoint is loaded, so that evaluation and deployment cannot silently feed a 288 model a 224 image.+20. As the researcher, I want the validation, the benchmark harness and the inference package to build their input at the checkpoint's configured size, so that the three input builders stay in agreement.+21. As the researcher, I want the ONNX export to carry the input size in its metadata, so that the phone app configures its crop size from the model file.+22. As the researcher, I want baseline and v2 samples to stay bit-identical to the current code, so that controls in flight remain valid.+23. As the researcher, I want the v3 dataset to reject a split that lacks the rebuilt geometry, with a clear message, so that a v3 run can never train silently on 224 crops with black-border zoom-out.+24. As the researcher, I want a verification of v3 samples on real data (label alignment, CLIFF correctness, no black border inside the stored context, coverage of the configured box-scale range), so that the preset is proven before any GPU hours are spent.+25. As the researcher, I want the DDP smoke test to pass for v3 at 224 and at 288, so that the cluster job cannot abort on its second step.+26. As the researcher, I want the EMDB-1 published-protocol score (EMDB's own boxes, SMPL-24 via the adapter) reported for every new checkpoint beside the 3DPW test J14 score, so that decisions use the numbers that go in the paper.+27. As the researcher, I want to score a checkpoint on EMDB under enlarged boxes (1.2x, 1.4x, 1.6x), so that I can see whether the framing cliff has moved.+28. As the researcher, I want matched g8h controls on the rebuilt corpus's data mix, so that a v3 result is attributable to the preset and not to a changed mixture.+29. As the researcher, I want v3 at 224 and v3 at 288 trained with two seeds each, so that a change above ~0.5 mm can be told from seed noise.+30. As the researcher, I want a learning-rate x schedule-length sweep (peak learning rate {6e-4, 1.2e-3} x {300, 600} epochs) on g8h with the baseline preset on the current corpus, so that I know whether the underfitting model gains from a better recipe, independently of the rebuild.+31. As the researcher, I want the existing g8h_s0 run reused as the (6e-4, 300) cell of that sweep, so that the sweep costs three new runs, not four.+32. As the researcher, I want the sweep's winning recipe carried into the v3 runs once both are known, so that the final model family uses the best recipe on the best data.+33. As the researcher, I want random-weight exports of the current architecture with HGNetV2-B0 and B1 backbones profiled on the same five hosted phones as g8h, so that the CPU-tier backbone size is chosen from measured latency before any training.+34. As the researcher, I want each model-family member profiled for latency, NPU placement and fp16 error on QNN, so that every member satisfies the portability rule before it is trained at scale.+35. As the researcher, I want the chosen CPU-tier backbone(s) trained with the v3 recipe, so that the paper's accuracy-vs-latency curve has a member for each deployment tier.+36. As the researcher, I want every architecture or input-resolution change to pass the fp16-parity and full-NPU-placement check, so that a gain on the desktop never becomes a model that falls off the NPU on the phone.+37. As the researcher, I want the published-protocol numbers, the sweep results and the family latencies recorded in the repo's results and status docs, so that the paper's tables have a single source.+38. As the phone-app developer, I want the NPU runtime libraries for Snapdragon 8 Gen 1, Gen 3 and Elite shipped alongside Gen 2's, so that the same model runs on every Snapdragon phone that can run it.+39. As the phone-app developer, I want phones without a usable fp16 NPU (Snapdragon 888 / 778G class, Exynos, Tensor) served by the CPU-tier model instead of the fake-model fallback, so that the app works on any Android phone.++## Implementation Decisions++- **Two storage parameters, one rebuild.** The builder gains a context factor (the stored square's side as a multiple of the tight box; today 1.2) and keeps its existing output-size option. The rebuild uses context 2.0 at 448 px. The standard 1.2x window then covers ~269 px of real detail. That supports 224 and 256 input without upsampling; 288 upsamples by ~7%. Defaults reproduce today's builder byte for byte.+- **Recorded geometry.** Each rebuilt annotation stores the stored square in original-frame coordinates, next to the existing 1.2x square, so any window can be recomputed exactly. Where the stored square leaves the original frame, the builder's existing padding behaviour applies, and that padding is real frame-edge truncation, not an augmentation artefact.+- **Separate corpus.** Rebuilt splits live in their own directories, named so that the baseline and v2 presets never pick them up. The pair-index fingerprint therefore does not change for runs on the old corpus.+- **v3 box sampling.** Per sample, the effective person box is drawn as a multiple of the tight box, **log-uniformly** between 0.6x (the same truncation depth as v2's 2.0 zoom-in on a 1.2x crop) and 2.0x (the full stored context). Log-uniform keeps tight and loose framings equally likely instead of over-sampling the loose end. The centre is shifted by up to +/-15% of the box on each axis. The window is clamped to stay inside the stored context, so a large box and a large shift never combine into a black border. The bounds are config values with these defaults; they are not per-run tuning knobs. v3 keeps rotation and flip as v2 has them.+- **Wide on purpose.** The zoom-out range covers EMDB's framing (average ~1.4x equivalent, with a tail past the 1.6x cliff) and the phone detector's oversized boxes. The zoom-in range covers people close to the camera. The upper bound is the stored context, so "wide" never means synthetic black margins.+- **One resampling step.** The v3 warp maps the stored crop straight to the network input, composing the box sampling with rotation and flip. No intermediate resize to 224 happens first. This is what removes v2's upsampling blur.+- **CLIFF follows the sampled box.** v3 builds the conditioning vector from the sampled box in original-frame coordinates, in the perspective-correct form (`cliff_focal`), with the crop-centre correction for zoom already verified for v2.+- **Input size is config.** A single input-size setting drives the dataset, the validation, the benchmark harness and the inference package. It is stored in the run config, recovered by the existing config-from-checkpoint path, and stamped into ONNX metadata beside the CLIFF form. A checkpoint without it means 224.+- **Presets stay orthogonal.** v3 is a value of the existing preset option. The loss-budget option is unchanged. baseline and v2 code paths are not modified beyond what is needed to route to v3.+- **Harmony4D route.** If the `prepost` download test returns HTTP 206, Harmony4D is rebuilt on Jean Zay with the existing streaming job, passing the new builder flags and keeping no 4K originals. If it fails, the same streaming script runs on the laptop, scene by scene, deleting each zip after its crops are built, and the crops are rsynced as one tar.+- **Controls and seeds.** Each v3 arm is compared against g8h retrained on the rebuilt corpus's exact data mix, with 2 seeds per arm. The acceptance target is a meaningful reduction of the EMDB published-protocol penalty with no loss on 3DPW test J14, where "meaningful" means beyond the ~0.5 mm that 2 seeds can resolve.+- **Recipe sweep runs now.** The learning-rate x schedule-length sweep uses the current corpus and the baseline preset. It shares nothing with the rebuild and starts immediately.+- **Model family sizing.** CPU-tier backbones are chosen from random-weight exports profiled on the Galaxy S23, A73, A53, A14 and Pixel 8 (CPU only), against a 33 ms budget on mid-range phones. The NPU-tier member stays at the HGNetV2-B4 size.+- **Not reopened.** RepViT stays dropped because it overflows fp16 on the NPU. Weight-decay and dropout tuning stay dropped while the model underfits; they return if training ever overfits. No large seed sweep: seed spread is ~0.2 mm at generation 8.++## Testing Decisions++A good test here observes a component from its outside, on real data, and reports a number: pixels, millimetres, milliseconds or a byte diff. It never asserts on internal variables. Four seams, three of which already exist:++- **Crop builder output (new check).** Run the builder on a small real split. Default flags: output byte-identical to the current builder's. New flags: the 1.2x window cut from the rebuilt crop and resized to 224 matches today's crop to within JPEG and resampling noise (report PSNR). The recorded geometry reprojects the annotation targets' 2D keypoints to 0.000 px. Prior art: the builder's own reprojection validation and `verify_split.py`.+- **One training sample.** For baseline and v2: a few dozen augmented samples bit-identical against `git show HEAD`, compared on alternating calls because of the ~5e-10 cold-kernel difference in the rig. For v3: 2D labels land on the right pixels; the CLIFF vector matches the exact preimage of the crop centre (as in the `--crop-centre-fix` verification: mean, p95 and max error in px); no black pixel appears inside the stored context; the drawn box scales cover the configured range; the output size equals the configured input size. Prior art: `tools/verify_cont_head.py`, `tools/verify_exact_landmarks.py`.+- **Exported model on phones.** `tools/ddp_smoke.py` for every new configuration, then ONNX export, then AI Hub profiling for latency on the five reference phones, NPU placement (must be 100% on Snapdragon 8 Gen 1+) and fp16 error. Prior art: the AI Hub profiling in the phone-app repo and this session's portability run.+- **Benchmark harness, unchanged.** EMDB-1 with `--bbox annotated` and the SMPL-24 adapter, 3DPW test J14 with the adapter, plus EMDB under enlarged boxes (1.2x / 1.4x / 1.6x) as the framing-cliff diagnostic. Evaluating a 288 checkpoint through the harness is itself the test that the input size round-trips through the checkpoint config.++## Out of Scope++- Core ML / Apple deployment. There is no Mac or iPhone yet; it is deferred to when there is.+- int8 quantization for older Snapdragon NPUs. Those phones are served by the CPU tier.+- Decoder feedback (roadmap item 4). It follows once the v3 recipe is known and is gated by the same portability check.+- Changing the loss budget, the continuous head or the exact-landmark readout.+- Weight-decay and dropout tuning, while the model underfits.+- Rebuilding EgoHumans or EgoExo4D, which are not in the current corpus.+- The phone app's own changes (stories 38–39) beyond specifying what the model family needs from it. They are tracked in the phone-app repo.++## Further Notes++- Measurements this spec rests on (2026-09-25): EMDB-1 annotated-box scores for g8h_s0/s1 are in `benchmark/results/gen8_emdb1_annotated.json`. The box-scale diagnostic: 51.95 / 54.19 / 68.16 mm at 1.2 / 1.4 / 1.6x, stride 5. Hosted-phone latencies are from AI Hub jobs on the compiled g8h model (compile job `jpvldwoz5`): NPU S22 4.1 ms, S23 2.4 ms, S24 1.7 ms, S25 1.4 ms, S21 and A73 all-CPU. CPU: S23 63 ms, Pixel 8 82 ms, A73 84 ms, A53 188 ms, A14 196 ms.+- Storage: the 448 px rebuild is roughly 4x today's crop pixels, ~250 GB on `$SCRATCH` (400 TB, 1% used), archived as one tar per split to `$STORE` (50 TiB).+- Compute: 4,385 of 50,000 V100 hours used. A 300-epoch run is at most 4 links x 20 h x 4 GPUs = 320 GPU-hours.+- Vocabulary follows `CONTEXT.md`: reference model, annotation target, baseline preset, v2 preset, published protocol, deployment tier, model family.diff --git a/CLAUDE.md b/CLAUDE.mdindex ee83451..9b51f74 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -196,9 +196,16 @@ and stops being comparable with generations 6-8 -- no error, just a differentnumber. The union is free (181.2 ms for 595 alone vs 180.9 ms for 1053 at batch64), because the cost is the forward-kinematics pass, not the vertex count.`instanthmr_distill_train/assets/mhr_landmarks70.npz` is a new file and must be-rsynced. Scoring deliberately stays on the fitted readout: `val3dpw.py` and-`benchmark/eval_3dpw_ckpt.py` both call `get_native_keypoints`, so the metric-keeps measuring the deployed path and stays comparable across generations.+rsynced. Scoring stays on the fitted readout by default and is now a switch+(`--landmarks exact` on `eval_3dpw_ckpt.py` / `eval_emdb_ckpt.py`,+`--val-landmarks exact` on the trainer). Measured 2026-09-22 it is worth+**0.13 mm** of 3DPW-test J14+adapter PA-MPJPE and 0.1 mm of EMDB-1 — and the+same 0.13 mm on `g7h_s0`, which trained WITHOUT `--exact-landmarks`, so it is a+property of the operator, not of the training flag. The mismatch it removes is+real (both adapters are fitted from the annotation `joints_3d`, i.e. the exact+readout), but it is an offset, not a re-ranking. `fitted` stays default so the+metric keeps measuring the deployed path and every recorded row stays+comparable; **never put fitted-scored and exact-scored rows in one table.****`--cont-head` adds two files that must be rsynced.**`instanthmr_distill_train/mhr_cont.py` and@@ -368,9 +375,12 @@ by 0.1 mm. Measure jitter directly (frame-to-frame acceleration on 3DPW'sconsecutive frames, and the variance of rigid bone lengths within a persontrack, which cannot change and so is pure noise).-**Seed spread is wide.** Observed 70-keypoint PA-MPJPE varies 34–49 mm across-seeds, which is larger than several of the changes on the roadmap. A single-run-comparison proves nothing; use 2–3 seeds.+**Seed spread is narrow at generation 8.** g8h_s0 vs g8h_s1 differ by 0.10 mm+on 3DPW test J14 (41.76 / 41.86) and 0.24 mm on EMDB-1 (46.93 / 46.69,+fit-free J14/H36M). The old "34–49 mm" warning was the 70-keypoint metric on+early generations and no longer applies. A seed sweep buys ~0.2 mm, so it is+not a way to gain accuracy; keep 2 seeds per arm only to tell a >0.5 mm change+from noise.**You cannot reach Jean Zay.** No SSH, no Slurm. Print commands for the user andread the logs they paste back. Files reach the cluster by `rsync` of *specific@@ -379,3 +389,19 @@ files* — never the whole tree, since some files are edited on both sides.**ONNX export constrains the model.** `torch.linalg.lstsq` and friends do notexport; use explicit small-matrix algebra. Pin `onnxruntime-gpu==1.26.0` — 1.27+is CUDA 13 and silently falls back to CPU.++---++## Agent skills++### Issue tracker++Issues and specs live as local markdown files under `.scratch/<feature>/`, not GitHub Issues (the repo is public). See `docs/agents/issue-tracker.md`.++### Triage labels++The five default roles, label string = role name (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `docs/agents/triage-labels.md`.++### Domain docs++Single-context: one `CONTEXT.md` plus `docs/adr/` at the repo root, created lazily. See `docs/agents/domain.md`.diff --git a/CONTEXT.md b/CONTEXT.mdnew file mode 100644index 0000000..e11e79e--- /dev/null+++ b/CONTEXT.md@@ -0,0 +1,55 @@+# InstantHMR++A real-time, single-image human mesh recovery model that predicts MHR rig parameters and is built to run on phones. This glossary fixes the words used about its models, training runs, evaluation and deployment.++## Language++### Models and runs++**Reference model**:+The model currently shipped to devices and used as the comparison point for every new experiment (g8h as of 2026-09-24).+_Avoid_: baseline, new baseline, the current model++**Generation**:+A numbered batch of training runs that differ from the previous batch along one or two stated axes (e.g. gen8 = continuous regression head + exact landmarks).+_Avoid_: version, round++**Arm**:+One configuration inside a generation, trained under one or more seeds (e.g. g8h = gen8 with the HGNetV2-B4 backbone).+_Avoid_: variant, experiment++**Model family**:+A set of models trained with one recipe at several backbone sizes, so that each deployment tier gets its own point on one accuracy-vs-latency curve.+_Avoid_: model zoo, variants++### Training++**Annotation target**:+The human MHR fit released for each person in `facebook/sam-3d-body-dataset`, which every current run is trained to match.+_Avoid_: reference target, teacher label, pseudo-label, GT (unqualified)++**Baseline preset**:+The original input pipeline selected by `--preset baseline`; only ever the name of a preset, never of a model.+_Avoid_: baseline (on its own), default++**v2 preset**:+The alternative input pipeline selected by `--preset v2`, whose CLIFF vector follows the augmented crop.++**v3 preset**:+The input pipeline that samples each training crop from a context crop over a wide range of framings, loose, off-centre and truncated, always surrounded by real scene.++**Context crop**:+A stored training crop covering twice the person's tight box at high resolution, from which the network's input window is sampled.+_Avoid_: big crop, raw crop, original (originals are the full source frames)++### Evaluation++**Published protocol**:+The evaluation setting whose numbers may sit beside other papers' rows: EMDB-1 with EMDB's own person boxes scored on SMPL-24 joints, and 3DPW test scored on J14 with the adapter.+_Avoid_: benchmark (unqualified), our protocol++### Deployment++**Deployment tier**:+A class of device runtime the product must support, each with its own latency target: Snapdragon NPU, Apple Core ML, and CPU (the tier that covers every other device).+_Avoid_: platform, backend, targetdiff --git a/docs/agents/domain.md b/docs/agents/domain.mdnew file mode 100644index 0000000..3524904--- /dev/null+++ b/docs/agents/domain.md@@ -0,0 +1,51 @@+# Domain Docs++How the engineering skills should consume this repo's domain documentation when exploring the codebase.++## Before exploring, read these++- **`CONTEXT.md`** at the repo root, or+- **`CONTEXT-MAP.md`** at the repo root if it exists: it points at one `CONTEXT.md` per context. Read each one relevant to the topic.+- **`docs/adr/`**: read ADRs that touch the area you're about to work in. In multi-context repos, also check `src/<context>/docs/adr/` for context-scoped decisions.++If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The `/domain-modeling` skill (reached via `/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily when terms or decisions actually get resolved.++## File structure++Single-context repo (most repos):++```+/+├── CONTEXT.md+├── docs/adr/+│ ├── 0001-event-sourced-orders.md+│ └── 0002-postgres-for-write-model.md+└── src/+```++Multi-context repo (presence of `CONTEXT-MAP.md` at the root):++```+/+├── CONTEXT-MAP.md+├── docs/adr/ ← system-wide decisions+└── src/+ ├── ordering/+ │ ├── CONTEXT.md+ │ └── docs/adr/ ← context-specific decisions+ └── billing/+ ├── CONTEXT.md+ └── docs/adr/+```++## Use the glossary's vocabulary++When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids.++If the concept you need isn't in the glossary yet, that's a signal: either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/domain-modeling`).++## Flag ADR conflicts++If your output contradicts an existing ADR, surface it explicitly rather than silently overriding:++> _Contradicts ADR-0007 (event-sourced orders), but worth reopening because…_diff --git a/docs/agents/issue-tracker.md b/docs/agents/issue-tracker.mdnew file mode 100644index 0000000..0209a19--- /dev/null+++ b/docs/agents/issue-tracker.md@@ -0,0 +1,30 @@+# Issue tracker: Local Markdown++Issues and specs for this repo live as markdown files in `.scratch/`.++## Conventions++- One feature per directory: `.scratch/<feature-slug>/`+- The spec is `.scratch/<feature-slug>/spec.md`+- Implementation issues are one file per ticket at `.scratch/<feature-slug>/issues/<NN>-<slug>.md`, numbered from `01`, never a single combined tickets file+- Triage state is recorded as a `Status:` line near the top of each issue file (see `triage-labels.md` for the role strings)+- Comments and conversation history append to the bottom of the file under a `## Comments` heading++## When a skill says "publish to the issue tracker"++Create a new file under `.scratch/<feature-slug>/` (creating the directory if needed).++## When a skill says "fetch the relevant ticket"++Read the file at the referenced path. The user will normally pass the path or the issue number directly.++## Wayfinding operations++Used by `/wayfinder`. The **map** is a file with one **child** file per ticket.++- **Map**: `.scratch/<effort>/map.md` (the Notes / Decisions-so-far / Fog body).+- **Child ticket**: `.scratch/<effort>/issues/NN-<slug>.md`, numbered from `01`, with the question in the body. A `Type:` line records the ticket type (`research`/`prototype`/`grilling`/`task`); a `Status:` line records `claimed`/`resolved`.+- **Blocking**: a `Blocked by: NN, NN` line near the top. A ticket is unblocked when every file it lists is `resolved`.+- **Frontier**: scan `.scratch/<effort>/issues/` for files that are open, unblocked, and unclaimed; first by number wins.+- **Claim**: set `Status: claimed` and save before any work.+- **Resolve**: append the answer under an `## Answer` heading, set `Status: resolved`, then append a context pointer (gist + link) to the map's Decisions-so-far in `map.md`.diff --git a/docs/agents/triage-labels.md b/docs/agents/triage-labels.mdnew file mode 100644index 0000000..b716855--- /dev/null+++ b/docs/agents/triage-labels.md@@ -0,0 +1,15 @@+# Triage Labels++The skills speak in terms of five canonical triage roles. This file maps those roles to the actual label strings used in this repo's issue tracker.++| Label in mattpocock/skills | Label in our tracker | Meaning |+| -------------------------- | -------------------- | ---------------------------------------- |+| `needs-triage` | `needs-triage` | Maintainer needs to evaluate this issue |+| `needs-info` | `needs-info` | Waiting on reporter for more information |+| `ready-for-agent` | `ready-for-agent` | Fully specified, ready for an AFK agent |+| `ready-for-human` | `ready-for-human` | Requires human implementation |+| `wontfix` | `wontfix` | Will not be actioned |++When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the corresponding label string from this table.++Edit the right-hand column to match whatever vocabulary you actually use.