Python
feat: --image-size as a checkpoint setting, context crops in build_split, CPU-tier backbone latencies
mohamdev36ae38720c00
diff --git a/.scratch/v3-framing-robustness/issues/03-profile-cpu-tier-backbones.md b/.scratch/v3-framing-robustness/issues/03-profile-cpu-tier-backbones.mdindex 445bc7b..90d24be 100644--- a/.scratch/v3-framing-robustness/issues/03-profile-cpu-tier-backbones.md+++ b/.scratch/v3-framing-robustness/issues/03-profile-cpu-tier-backbones.md@@ -4,9 +4,25 @@**Blocked by:** None (can start immediately).-**Status:** ready-for-agent+**Status:** done (2026-09-25), uncommitted-- [ ] The B4 random-weight export's latency matches the trained g8h's within noise on the S23 CPU (63 ms) — the check that random weights are a valid latency proxy-- [ ] CPU-only latency and peak memory for B0, B1 and B4 on Galaxy S23, A73, A53, A14 and Pixel 8; S23 NPU latency and placement for each-- [ ] Every export places 100% of layers on the NPU on the S23 and passes fp16 parity-- [ ] A table recorded in the status doc and a recommended CPU-tier size against the 33 ms budget on mid-range phones (A73 / Pixel 8)+- [x] The B4 random-weight export's latency matches the trained g8h's within noise on the S23 CPU (63 ms) — the check that random weights are a valid latency proxy+- [x] CPU-only latency and peak memory for B0, B1 and B4 on Galaxy S23, A73, A53, A14 and Pixel 8; S23 NPU latency and placement for each+- [~] Every export places 100% of layers on the NPU on the S23 (yes, all four); fp16 parity NOT measured: it is meaningless on random weights, so it moves to ticket 11 on the trained members+- [x] A table recorded in the status doc and a recommended CPU-tier size against the 33 ms budget on mid-range phones (A73 / Pixel 8)++## Answer++Exports: g8h's exact architecture and config (continuous head, bounded scales, CLIFF focal), backbone swapped, network only (no FK/mesh), 224 px, opset 17, onnxsim. AI Hub, ORT runtime compiled for the S23. Raw rows: `benchmark/results/family_latency_aihub.json`; scripts: `tools/profiling/`.++| model | params (backbone) | S23 NPU | S23 CPU | A73 CPU | Pixel 8 CPU | A53 CPU | A14 CPU |+|---|---|---|---|---|---|---|---|+| g8h trained (B4) | 32.2 M (13.6 M) | 2.44 ms | 61.7 | 84.9 | 83.2 | 187.9 | 195.2 |+| random B4 | 32.2 M (13.6 M) | 2.47 | 60.0 | 81.7 | 78.7 | 194.2 | 199.4 |+| random B1 | 20.3 M (2.2 M) | 1.88 | 26.9 | 36.9 | 33.2 | 77.6 | 76.8 |+| random B0 | 20.0 M (1.9 M) | 1.60 | 23.9 | 32.7 | 29.9 | 79.7 | 68.6 |++- Random weights are a valid latency proxy: random B4 is within 2-6% of the trained g8h on every phone.+- All four place 100% of layers on the S23 NPU.+- **Recommendation: HGNetV2-B0 is the CPU-tier member** (under 33 ms on mid-range: A73 32.7, Pixel 8 29.9). B1 is 3.3 ms slower and just over budget on the A73 (36.9); worth training as the middle point of the accuracy-vs-latency curve.+- **Budget phones (A53, A14) stay at 70-80 ms with either backbone.** B0 and B1 differ by <2 ms there: below B4 the backbone is no longer the cost. ~18 M of the 20 M parameters are the decoder and heads, unchanged by the backbone swap. Reaching 30 FPS on budget phones needs a slimmer decoder (width / depth), which is a new work item, not a backbone choice.diff --git a/.scratch/v3-framing-robustness/issues/05-input-size-setting.md b/.scratch/v3-framing-robustness/issues/05-input-size-setting.mdindex 8996d7c..845d46c 100644--- a/.scratch/v3-framing-robustness/issues/05-input-size-setting.md+++ b/.scratch/v3-framing-robustness/issues/05-input-size-setting.md@@ -4,9 +4,22 @@**Blocked by:** None (can start immediately).-**Status:** ready-for-agent+**Status:** done (2026-09-25), uncommitted-- [ ] baseline and v2 augmented samples are bit-identical against `git show HEAD`, compared on alternating calls-- [ ] g8h_s0 evaluated through the harness gives the same 3DPW test J14 and EMDB-1 numbers as recorded, to the decimal-- [ ] A random-weight model configured for 288 runs end to end through the 3DPW and EMDB harnesses, and its ONNX metadata reads 288-- [ ] `ddp_smoke` passes at 224 and at 288+- [x] baseline and v2 augmented samples are bit-identical against `git show HEAD`, compared on alternating calls+- [x] g8h_s0 evaluated through the harness gives the same 3DPW test J14 and EMDB-1 numbers as recorded, to the decimal+- [x] A random-weight model configured for 288 runs end to end through the 3DPW and EMDB harnesses, and its ONNX metadata reads 288+- [x] `ddp_smoke` passes at 224 and at 288++## Answer++`--image-size` on the DDP trainer (multiple of 32), recorded in run_config.json. `config_from_checkpoint` reads the size from the weights (`mem_pos_embed` grid, 7x7 = 224, 9x9 = 288) and refuses a run_config.json that contradicts it. The dataset (both trainers and the self/overfit tests), `val3dpw`, the 3DPW / EMDB / SMPL-fit harnesses, `eval_all.py` grouping, `pth_to_onnx.py` metadata (`image_size`), `instanthmr.inference` and `ddp_smoke.py` all follow it.++Evidence: `tools/verify_input_size.py` PASS (baseline and v2 0/24 samples differ from the pre-change trainer; labels and CLIFF identical at 224 and 288; 288 checkpoint round-trips through config, export metadata, inference on a real frame and both harnesses). g8h_s0 full regression identical to 4 decimals (3DPW test J14+adapter PA 40.7983, EMDB SMPL24+adapter 51.9492). `ddp_smoke` passes at 224 and 288; `--self-test` passes.++Open, outside this repo (story 21 is NOT satisfied end to end yet):+- neurocomp-app's own exporter (`tools/modelexport/pose.py`) hard-codes `INPUT_SHAPES` 224 and its metadata stamp clears all props and never writes `image_size`.+- the Android engine hard-codes `POSE_INPUT_SIZE = 224` in `Preprocess.kt`.+Both must read `image_size` before a non-224 model can ship. File this in the phone-app tracker.++Note for ticket 10: `RandomPixelate` uses absolute resolutions (32-112 px), so at 288 it degrades relatively harder than at 224, which slightly confounds a 224-vs-288 comparison.diff --git a/benchmark/eval_3dpw_ckpt.py b/benchmark/eval_3dpw_ckpt.pyindex 125fc79..307d4fc 100644--- a/benchmark/eval_3dpw_ckpt.py+++ b/benchmark/eval_3dpw_ckpt.py@@ -54,6 +54,16 @@ def parse_args():p.add_argument("--fit-adapter", default=None,help="fit that regressor on THIS split and write it here ""(run on train, never on test)")+ p.add_argument("--landmarks", default="fitted", choices=["fitted", "exact"],+ help="how the 70 MHR keypoints are read off the prediction. "+ "fitted (default): the (70, 127) skeleton matrix, which "+ "is what the exported graph's consumer uses and what "+ "every recorded number in benchmark/results was scored "+ "with. exact: the teacher's mesh+joint mapping, the "+ "operator the stored annotations -- and so the adapter "+ "-- came out of. Worth 0.13 mm of J14+adapter PA-MPJPE "+ "on 3DPW test, with or without --exact-landmarks "+ "training.")p.add_argument("--backbone", default=None,help="override the backbone; normally read from the ""run_config.json beside the checkpoint")@@ -71,7 +81,8 @@ def parse_args():@torch.no_grad()-def predict(model, loader, mhr_module, device, use_amp: bool = True):+def predict(model, loader, mhr_module, device, use_amp: bool = True,+ exact_landmarks: bool = False):"""(N, 70, 3) predicted keypoints in mm, plus the indices they came from.The indices let the same forward pass be scored against more than one GT@@ -86,7 +97,8 @@ def predict(model, loader, mhr_module, device, use_amp: bool = True):with torch.amp.autocast(device_type="cuda", dtype=torch.float16, enabled=use_amp):out = model(img, cc)kp = mhr_module.get_native_keypoints(out["mhr_params"].float(),- out["shape_params"].float())+ out["shape_params"].float(),+ exact=exact_landmarks)# Same guard as val3dpw.evaluate: a non-finite prediction makes the# Procrustes SVD raise rather than return a large error.finite = torch.isfinite(kp).flatten(1).all(dim=1).cpu()@@ -129,10 +141,21 @@ def main():"conditioning is built once for all of them.\n""Evaluate the two groups in separate invocations.")cliff_focal = focals.pop()+ sizes = {c.image_size for c in cfgs.values()}+ if len(sizes) > 1:+ raise SystemExit(+ f"the checkpoints disagree on the input size ({sorted(sizes)}), and the "+ "crops are built once for all of them.\n"+ "Evaluate the groups in separate invocations.")+ input_size = sizes.pop()cfg = next(iter(cfgs.values()))- mhr = T.MHRForwardPass(cfg.mhr_model_path, device,- kp_regressor=np.load(cfg.kp_regressor_path))+ exact_lm = args.landmarks == "exact"+ mhr = T.MHRForwardPass(+ cfg.mhr_model_path, device, kp_regressor=np.load(cfg.kp_regressor_path),+ landmark_assets=cfg.landmark_assets_path if exact_lm else None)+ print(f"[readout] 70 keypoints via the "+ f"{'teacher exact mesh+joint mapping' if exact_lm else 'fitted (70, 127) skeleton matrix'}")W_adapt = W_adapt_gt = Noneif args.adapter:ad = np.load(args.adapter)@@ -145,7 +168,8 @@ def main():ds = val3dpw.ThreeDPWValSet(args.sequence_dir, args.image_root,split=args.split, stride=args.stride,gt=gt, gt_dir=args.gt_dir,- cliff_focal=cliff_focal)+ cliff_focal=cliff_focal,+ input_size=input_size)sets[gt] = dsprint(f"[3DPW] {args.split} / GT={gt}: {len(ds):,} person-frames "f"(stride {args.stride}), CLIFF conditioning "@@ -169,7 +193,8 @@ def main():# Strict: an unexpected key means the config restored above is wrong# and the model being scored is not the model that was trained.model.load_state_dict(ck["model_state_dict"])- pred, idx, n_dropped = predict(model, loader, mhr, device, cfg.use_amp)+ pred, idx, n_dropped = predict(model, loader, mhr, device, cfg.use_amp,+ exact_landmarks=exact_lm)del modeltorch.cuda.empty_cache()@@ -183,6 +208,7 @@ def main():report[f"{ck_path}|{gt}"] = dict(ckpt=ck_path, gt=gt, split=args.split, stride=args.stride,epoch=ck.get("epoch"), source=ck.get("source"),+ landmarks=args.landmarks,n=int(pred.shape[0]), n_dropped=n_dropped,results=list(rows.values()))diff --git a/benchmark/eval_all.py b/benchmark/eval_all.pyindex db036a5..c33ffd4 100644--- a/benchmark/eval_all.py+++ b/benchmark/eval_all.py@@ -65,11 +65,12 @@ def group_by_conditioning(ckpts: list[str]) -> list[list[str]]:import torchsys.path.insert(0, str(ROOT / "instanthmr_distill_train"))import train_distill_mhr_only as T- groups: dict[bool, list[str]] = {}+ groups: dict[tuple[bool, int], list[str]] = {}for c in ckpts:st = torch.load(c, map_location="cpu", weights_only=False)cfg, _ = T.config_from_checkpoint(st.get("model_state_dict", st), c)- groups.setdefault(bool(cfg.cliff_focal), []).append(c)+ # the input size too: the harnesses build one crop set per invocation+ groups.setdefault((bool(cfg.cliff_focal), int(cfg.image_size)), []).append(c)del streturn list(groups.values())diff --git a/benchmark/eval_emdb_ckpt.py b/benchmark/eval_emdb_ckpt.pyindex b696aee..2f46010 100644--- a/benchmark/eval_emdb_ckpt.py+++ b/benchmark/eval_emdb_ckpt.py@@ -68,6 +68,10 @@ def parse_args():p.add_argument("--gt-dir", default=None)p.add_argument("--adapter", default="benchmark/results/adapter_smpl24_teacher.npz",help="npz with the MHR70 -> SMPL24 regressor")+ p.add_argument("--landmarks", default="fitted", choices=["fitted", "exact"],+ help="how the 70 MHR keypoints are read off the prediction; "+ "see eval_3dpw_ckpt.py. fitted is the deployed readout "+ "and what every recorded number was scored with.")p.add_argument("--backbone", default=None)p.add_argument("--cliff-focal", dest="cliff_focal", action="store_true",default=None)@@ -89,12 +93,13 @@ class EMDBValSet(torch.utils.data.Dataset):"""def __init__(self, emdb_root, stride=1, bbox="gt-joints", bbox_scale=1.2,- gt_dir=None, cliff_focal=False):+ gt_dir=None, cliff_focal=False, input_size=val3dpw.INPUT_SIZE):self.samples, gt = D.build_samples(emdb_root, stride=stride, bbox=bbox, bbox_scale=bbox_scale,gt_dir=gt_dir)self.gt = np.asarray(gt, dtype=np.float32)self.cliff_focal = cliff_focal+ self.input_size = input_sizedef __len__(self):return len(self.samples)@@ -103,12 +108,13 @@ class EMDBValSet(torch.utils.data.Dataset):s = self.samples[i]bgr = cv2.imread(s["image_path"], cv2.IMREAD_COLOR)if bgr is None:- return {"image": torch.zeros(3, val3dpw.INPUT_SIZE, val3dpw.INPUT_SIZE),+ return {"image": torch.zeros(3, self.input_size, self.input_size),"cliff_cond": torch.zeros(3), "ok": torch.tensor(0.0)}rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)h, w = rgb.shape[:2]crop, cliff = val3dpw._preprocess(- rgb, s["bbox"], h, w, s["focal"] if self.cliff_focal else None)+ rgb, s["bbox"], h, w, s["focal"] if self.cliff_focal else None,+ size=self.input_size)return {"image": torch.from_numpy(crop),"cliff_cond": torch.from_numpy(cliff),"ok": torch.tensor(1.0)}@@ -168,10 +174,21 @@ def main():"conditioning is built once for all of them.\n""Evaluate the two groups in separate invocations.")cliff_focal = focals.pop()+ sizes = {c.image_size for c in cfgs.values()}+ if len(sizes) > 1:+ raise SystemExit(+ f"the checkpoints disagree on the input size ({sorted(sizes)}), and the "+ "crops are built once for all of them.\n"+ "Evaluate the groups in separate invocations.")+ input_size = sizes.pop()cfg = next(iter(cfgs.values()))- mhr = T.MHRForwardPass(cfg.mhr_model_path, device,- kp_regressor=np.load(cfg.kp_regressor_path))+ exact_lm = args.landmarks == "exact"+ mhr = T.MHRForwardPass(+ cfg.mhr_model_path, device, kp_regressor=np.load(cfg.kp_regressor_path),+ landmark_assets=cfg.landmark_assets_path if exact_lm else None)+ print(f"[readout] 70 keypoints via the "+ f"{'teacher exact mesh+joint mapping' if exact_lm else 'fitted (70, 127) skeleton matrix'}")ad = np.load(args.adapter)if str(ad["gt"]) != "smpl24":raise SystemExit(f"{args.adapter} targets '{ad['gt']}', not smpl24; "@@ -180,7 +197,7 @@ def main():ds = EMDBValSet(args.emdb_root, stride=args.stride, bbox=args.bbox,bbox_scale=args.bbox_scale, gt_dir=args.gt_dir,- cliff_focal=cliff_focal)+ cliff_focal=cliff_focal, input_size=input_size)seqs = sorted({s["sequence"] for s in ds.samples})print(f"[EMDB1] {len(ds):,} frames across {len(seqs)} sequences "f"(stride {args.stride}, boxes: {args.bbox}), CLIFF conditioning "@@ -199,7 +216,8 @@ def main():model = T.InstantHMRStudent(cfgs[ck_path], pretrained=False).to(device)ck = torch.load(ck_path, map_location="cpu", weights_only=False)model.load_state_dict(ck["model_state_dict"])- pred, idx, n_dropped = predict(model, loader, mhr, device, cfg.use_amp)+ pred, idx, n_dropped = predict(model, loader, mhr, device, cfg.use_amp,+ exact_landmarks=exact_lm)del modeltorch.cuda.empty_cache()@@ -219,7 +237,7 @@ def main():report[ck_path] = dict(ckpt=ck_path, dataset="EMDB1", split="emdb1", stride=args.stride,- bbox=args.bbox, adapter=str(args.adapter),+ bbox=args.bbox, adapter=str(args.adapter), landmarks=args.landmarks,epoch=ck.get("epoch"), source=ck.get("source"),n=int(pred.shape[0]), n_dropped=n_dropped,num_sequences=len(seqs), results=rows,diff --git a/benchmark/eval_smpl_fit.py b/benchmark/eval_smpl_fit.pyindex f10c46e..3e621d5 100644--- a/benchmark/eval_smpl_fit.py+++ b/benchmark/eval_smpl_fit.py@@ -92,10 +92,17 @@ def parse_args():p.add_argument("--bbox", default="gt-joints", choices=["gt-joints", "annotated"],help="EMDB only; 3DPW always uses projected-GT-joint boxes")p.add_argument("--fit-iters", type=int, default=400,- help="USE >= 1500. 400 converges on the round trip but NOT "- "on real predictions: 16.1 mm residual on 3DPW vs 8.0 "- "at 4000, worth 3.6 mm of J14 PA-MPJPE, and the sign "- "differs per dataset. See benchmark/README.md.")+ help="0 skips the SMPL parameter fit entirely and reports "+ "the fit-free rows only (PVE, PA-PVE, J14/H36M) -- the "+ "settled protocol, no optimiser and no free parameter, "+ "and the only way to reach PVE without a "+ "hyperparameter that has a 3-4 mm blast radius. "+ "Otherwise USE >= 1500: 400 converges on the round trip "+ "but NOT on real predictions -- 16.1 mm residual on "+ "3DPW vs 8.0 at 4000, worth 3.6 mm of J14 PA-MPJPE, "+ "and the sign differs per dataset. The fit buys the 24 "+ "SMPL KINEMATIC joints and nothing else. See "+ "benchmark/README.md.")p.add_argument("--fit-batch", type=int, default=64)p.add_argument("--batch-size", type=int, default=64)p.add_argument("--num-workers", type=int, default=10)@@ -241,12 +248,14 @@ def threedpw_gt_smpl(sequence_dir, split, samples, smpl_dir, device):class ThreeDPWSet(torch.utils.data.Dataset):"""3DPW crops, built by the same ``val3dpw._preprocess`` EMDB uses."""- def __init__(self, sequence_dir, image_root, split, stride, cliff_focal):+ def __init__(self, sequence_dir, image_root, split, stride, cliff_focal,+ input_size=val3dpw.INPUT_SIZE):from benchlib import threedpw as P3self.samples, _ = P3.build_samples(sequence_dir, image_root, split=split, stride=stride,gt="jointpositions")self.cliff_focal = cliff_focal+ self.input_size = input_sizedef __len__(self):return len(self.samples)@@ -255,12 +264,13 @@ class ThreeDPWSet(torch.utils.data.Dataset):s = self.samples[i]bgr = cv2.imread(s["image_path"], cv2.IMREAD_COLOR)if bgr is None:- return {"image": torch.zeros(3, val3dpw.INPUT_SIZE, val3dpw.INPUT_SIZE),+ return {"image": torch.zeros(3, self.input_size, self.input_size),"cliff_cond": torch.zeros(3), "ok": torch.tensor(0.0)}rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)h, w = rgb.shape[:2]crop, cliff = val3dpw._preprocess(- rgb, s["bbox"], h, w, s["focal"] if self.cliff_focal else None)+ rgb, s["bbox"], h, w, s["focal"] if self.cliff_focal else None,+ size=self.input_size)return {"image": torch.from_numpy(crop),"cliff_cond": torch.from_numpy(cliff),"ok": torch.tensor(1.0)}@@ -281,10 +291,10 @@ def main():if args.dataset == "emdb":ds = EMDBValSet(args.emdb_root, stride=args.stride, bbox=args.bbox,- cliff_focal=cfg.cliff_focal)+ cliff_focal=cfg.cliff_focal, input_size=cfg.image_size)else:ds = ThreeDPWSet(args.sequence_dir, args.image_root, args.split,- args.stride, cfg.cliff_focal)+ args.stride, cfg.cliff_focal, cfg.image_size)loader = torch.utils.data.DataLoader(ds, batch_size=args.batch_size, shuffle=False,num_workers=args.num_workers, pin_memory=True)@@ -306,11 +316,25 @@ def main():Jgt, Vgt, genders = threedpw_gt_smpl(args.sequence_dir, args.split,kept, args.smpl_dir, device)- # Fit per gender group, so one SMPL model serves each batch.+ # `barycentric_transfer` already returns the prediction in SMPL+ # topology, vertex i <-> SMPL vertex i, so this surface IS the answer+ # for every vertex-space metric. Keeping it costs one more (N, 6890, 3)+ # and removes the optimiser from PVE entirely.+ Vd = torch.zeros_like(Vgt)+ fitting = args.fit_iters > 0+ # Fit per gender group, so one SMPL model serves each batch. The fit+ # exists only to recover the 24 SMPL KINEMATIC joints, which come off+ # the chain and not off the surface.Jp = torch.zeros_like(Jgt)- Vp = torch.zeros_like(Vgt)+ # 2.9 GB at 35,463 frames, so it is allocated only when a fit fills it.+ Vp = torch.zeros_like(Vgt) if args.fit_iters > 0 else Noneresid = torch.zeros(len(kept))- for g in ("male", "female"):+ if not fitting:+ for k in range(0, len(kept), args.fit_batch):+ sel = np.arange(k, min(k + args.fit_batch, len(kept)))+ verts = rig.vertices(mp[sel].to(device), sh[sel].to(device))+ Vd[sel] = barycentric_transfer(verts, rig.faces, tri, bary).cpu()+ for g in ("male", "female") if fitting else ():gi = np.array([i for i, x in enumerate(genders) if x == g])if not len(gi):continue@@ -319,6 +343,7 @@ def main():sel = gi[k:k + args.fit_batch]verts = rig.vertices(mp[sel].to(device), sh[sel].to(device))tgt = barycentric_transfer(verts, rig.faces, tri, bary)+ Vd[sel] = tgt.cpu()th, be, tr, r = fit_smpl_to_targets(sm, tgt, iters=args.fit_iters)v, j = sm.forward(th, be, tr)Jp[sel] = j.cpu()@@ -344,30 +369,55 @@ def main():sel = J.J14_FROM_H36M17CH = 256pve_sum = pve_n = pa_pve_sum = 0.0- hp_parts, hg_parts = [], []- for i in range(0, Vp.shape[0], CH):- a = Vp[i:i + CH].numpy() * MM+ pve_d_sum = pa_pve_d_sum = 0.0+ hp_parts, hg_parts, hpd_parts = [], [], []+ for i in range(0, Vd.shape[0], CH):b = Vgt[i:i + CH].numpy() * MM- n = a.shape[0]- e = M.mpjpe(a, b, root=(rp[i:i + n], rg[i:i + n]))- pve_sum += float(e.sum()); pve_n += e.size- pa_pve_sum += float(M.pa_mpjpe(a, b).sum())- hp_parts.append(np.einsum("jv,nvc->njc", Jreg, a)[:, sel])- hg_parts.append(np.einsum("jv,nvc->njc", Jreg, b)[:, sel])- h_p = np.concatenate(hp_parts) # already mm+ n = b.shape[0]+ # Fit-free: the prediction's own surface against the GT surface,+ # rooted at the H36M pelvis regressed from EACH side by the same+ # operator, so the regressor's bias cancels instead of being+ # compared against a kinematic joint it does not equal.+ a_d = Vd[i:i + CH].numpy() * MM+ h_pd = np.einsum("jv,nvc->njc", Jreg, a_d)[:, sel]+ h_gd = np.einsum("jv,nvc->njc", Jreg, b)[:, sel]+ pve_d_sum += float(M.mpjpe(a_d, b,+ root=(J.pelvis(h_pd), J.pelvis(h_gd))).sum())+ pa_pve_d_sum += float(M.pa_mpjpe(a_d, b).sum())+ pve_n += a_d.shape[0] * a_d.shape[1]+ hpd_parts.append(h_pd)+ hg_parts.append(h_gd)+ if fitting:+ a = Vp[i:i + CH].numpy() * MM+ pve_sum += float(M.mpjpe(a, b, root=(rp[i:i + n], rg[i:i + n])).sum())+ pa_pve_sum += float(M.pa_mpjpe(a, b).sum())+ hp_parts.append(np.einsum("jv,nvc->njc", Jreg, a)[:, sel])+ h_pd = np.concatenate(hpd_parts) # already mmh_g = np.concatenate(hg_parts)- del hp_parts, hg_parts- hp_r, hg_r = J.pelvis(h_p), J.pelvis(h_g)+ h_p = np.concatenate(hp_parts) if fitting else None+ del hp_parts, hg_parts, hpd_parts+ hpd_r, hg_r = J.pelvis(h_pd), J.pelvis(h_g)+ hp_r = J.pelvis(h_p) if fitting else Noneres = dict(ckpt=ck_path, dataset=tag, bbox=args.bbox, stride=args.stride,n=int(len(kept)), fit_iters=args.fit_iters,epoch=st.get("epoch"), source=st.get("source"),- fit_residual_mm=float(resid.mean() * MM),- MPJPE_mm=float(M.mpjpe(jp, jg, root=(rp, rg)).mean()),- PA_MPJPE_mm=float(M.pa_mpjpe(jp, jg).mean()),- PVE_mm=pve_sum / pve_n,- PA_PVE_mm=pa_pve_sum / pve_n,+ fit_residual_mm=float(resid.mean() * MM) if fitting else None,+ # The fit-free rows: the prediction's own surface, in SMPL+ # topology, against the benchmark's own GT surface. No optimiser,+ # no free parameter, so these are the PVE numbers to quote. The+ # *_mm keys below keep their old meaning -- the FITTED mesh -- so+ # rows recorded before 2026-09-22 stay comparable.+ PVE_direct_mm=pve_d_sum / pve_n,+ PA_PVE_direct_mm=pa_pve_d_sum / pve_n,+ J14_h36m_direct_MPJPE_mm=float(+ M.mpjpe(h_pd, h_g, root=(hpd_r, hg_r)).mean()),+ J14_h36m_direct_PA_MPJPE_mm=float(M.pa_mpjpe(h_pd, h_g).mean()),+ MPJPE_mm=float(M.mpjpe(jp, jg, root=(rp, rg)).mean()) if fitting else None,+ PA_MPJPE_mm=float(M.pa_mpjpe(jp, jg).mean()) if fitting else None,+ PVE_mm=pve_sum / pve_n if fitting else None,+ PA_PVE_mm=pa_pve_sum / pve_n if fitting else None,# The published 3DPW protocol is 14 joints in the H36M convention,# reached here THROUGH the mesh conversion: MHR mesh -> barycentric# map -> fitted SMPL -> J_regressor_h36m, against the same@@ -380,21 +430,29 @@ def main():# space. Both are legitimate, and note that J_regressor_h36m is# itself a fitted linear map (SMPL vertices -> H36M joints) -- SMPL-# native methods use it for exactly the same reason we need ours.- J14_h36m_viamesh_MPJPE_mm=float(M.mpjpe(h_p, h_g, root=(hp_r, hg_r)).mean()),- J14_h36m_viamesh_PA_MPJPE_mm=float(M.pa_mpjpe(h_p, h_g).mean()),+ J14_h36m_viamesh_MPJPE_mm=float(+ M.mpjpe(h_p, h_g, root=(hp_r, hg_r)).mean()) if fitting else None,+ J14_h36m_viamesh_PA_MPJPE_mm=float(+ M.pa_mpjpe(h_p, h_g).mean()) if fitting else None,)- e = M.pa_mpjpe(jp, jg).mean(1)+ e = (M.pa_mpjpe(jp, jg) if fitting else M.pa_mpjpe(h_pd, h_g)).mean(1)res["per_sequence_PA_MPJPE_mm"] = {n: float(e[[i for i, s in enumerate(kept) if s["sequence"] == n]].mean())for n in sorted({s["sequence"] for s in kept})}report[ck_path] = res- print(f"\n {Path(ck_path).parts[-3]:<10s} SMPL24: MPJPE {res['MPJPE_mm']:6.2f} "- f"PA {res['PA_MPJPE_mm']:6.2f} PVE {res['PVE_mm']:6.2f} "- f"PA-PVE {res['PA_PVE_mm']:6.2f} | J14/H36M via mesh: MPJPE "- f"{res['J14_h36m_viamesh_MPJPE_mm']:6.2f} "- f"PA {res['J14_h36m_viamesh_PA_MPJPE_mm']:6.2f}"- f" (fit residual {res['fit_residual_mm']:.2f} mm)\n", flush=True)+ print(f"\n {Path(ck_path).parts[-3]:<10s} fit-free: PVE "+ f"{res['PVE_direct_mm']:6.2f} PA-PVE {res['PA_PVE_direct_mm']:6.2f}"+ f" | J14/H36M: MPJPE {res['J14_h36m_direct_MPJPE_mm']:6.2f} "+ f"PA {res['J14_h36m_direct_PA_MPJPE_mm']:6.2f}", flush=True)+ if fitting:+ print(f" {Path(ck_path).parts[-3]:<10s} fitted: SMPL24 MPJPE "+ f"{res['MPJPE_mm']:6.2f} PA {res['PA_MPJPE_mm']:6.2f} PVE "+ f"{res['PVE_mm']:6.2f} PA-PVE {res['PA_PVE_mm']:6.2f} | "+ f"J14/H36M via mesh: MPJPE {res['J14_h36m_viamesh_MPJPE_mm']:6.2f}"+ f" PA {res['J14_h36m_viamesh_PA_MPJPE_mm']:6.2f}"+ f" (fit residual {res['fit_residual_mm']:.2f} mm)", flush=True)+ print(flush=True)# Written after every checkpoint, not at the end: a full sweep is ~20# minutes per checkpoint and losing all of it to a crash in the last# one has already happened once.diff --git a/benchmark/results/family_latency_aihub.json b/benchmark/results/family_latency_aihub.jsonnew file mode 100644index 0000000..4100378--- /dev/null+++ b/benchmark/results/family_latency_aihub.json@@ -0,0 +1,266 @@+[+ {+ "model": "g8h_trained_b4",+ "unit": "npu",+ "device": "Samsung Galaxy S23",+ "job": "jp0moor2g",+ "ms": 2.44,+ "peak_mb": 160.2,+ "layers": {+ "NPU": 918+ }+ },+ {+ "model": "g8h_trained_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy S23",+ "job": "jp8ejj7zp",+ "ms": 61.74,+ "peak_mb": 199.1,+ "layers": {+ "CPU": 791+ }+ },+ {+ "model": "g8h_trained_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy A73 5G",+ "job": "jgk266yyg",+ "ms": 84.89,+ "peak_mb": 169.4,+ "layers": {+ "CPU": 791+ }+ },+ {+ "model": "g8h_trained_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy A53 5G",+ "job": "j5ql4427p",+ "ms": 187.85,+ "peak_mb": 177.8,+ "layers": {+ "CPU": 791+ }+ },+ {+ "model": "g8h_trained_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy A14 5G",+ "job": "jglywwke5",+ "ms": 195.18,+ "peak_mb": 176.4,+ "layers": {+ "CPU": 791+ }+ },+ {+ "model": "g8h_trained_b4",+ "unit": "cpu",+ "device": "Google Pixel 8",+ "job": "j568oo1vg",+ "ms": 83.2,+ "peak_mb": 233.4,+ "layers": {+ "CPU": 791+ }+ },+ {+ "model": "random_b4",+ "unit": "npu",+ "device": "Samsung Galaxy S23",+ "job": "jp3zoomx5",+ "ms": 2.47,+ "peak_mb": 80.9,+ "layers": {+ "NPU": 890+ }+ },+ {+ "model": "random_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy S23",+ "job": "jgolddv4g",+ "ms": 60.04,+ "peak_mb": 149.3,+ "layers": {+ "CPU": 762+ }+ },+ {+ "model": "random_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy A73 5G",+ "job": "jpvl22w75",+ "ms": 81.72,+ "peak_mb": 131.9,+ "layers": {+ "CPU": 762+ }+ },+ {+ "model": "random_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy A53 5G",+ "job": "jgjr33l7p",+ "ms": 194.15,+ "peak_mb": 134.0,+ "layers": {+ "CPU": 762+ }+ },+ {+ "model": "random_b4",+ "unit": "cpu",+ "device": "Samsung Galaxy A14 5G",+ "job": "jpe766v75",+ "ms": 199.35,+ "peak_mb": 134.5,+ "layers": {+ "CPU": 762+ }+ },+ {+ "model": "random_b4",+ "unit": "cpu",+ "device": "Google Pixel 8",+ "job": "jgzlzz7z5",+ "ms": 78.69,+ "peak_mb": 187.4,+ "layers": {+ "CPU": 762+ }+ },+ {+ "model": "random_b1",+ "unit": "npu",+ "device": "Samsung Galaxy S23",+ "job": "j5wlyy9zp",+ "ms": 1.88,+ "peak_mb": 97.8,+ "layers": {+ "NPU": 827+ }+ },+ {+ "model": "random_b1",+ "unit": "cpu",+ "device": "Samsung Galaxy S23",+ "job": "jg9zoo4qp",+ "ms": 26.94,+ "peak_mb": 136.7,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b1",+ "unit": "cpu",+ "device": "Samsung Galaxy A73 5G",+ "job": "jp1noo8kg",+ "ms": 36.88,+ "peak_mb": 123.1,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b1",+ "unit": "cpu",+ "device": "Samsung Galaxy A53 5G",+ "job": "jgdd66vkg",+ "ms": 77.58,+ "peak_mb": 129.8,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b1",+ "unit": "cpu",+ "device": "Samsung Galaxy A14 5G",+ "job": "j57eoodqp",+ "ms": 76.84,+ "peak_mb": 122.1,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b1",+ "unit": "cpu",+ "device": "Google Pixel 8",+ "job": "jp4yeewqp",+ "ms": 33.22,+ "peak_mb": 171.4,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b0",+ "unit": "npu",+ "device": "Samsung Galaxy S23",+ "job": "jpxl001jp",+ "ms": 1.6,+ "peak_mb": 81.8,+ "layers": {+ "NPU": 827+ }+ },+ {+ "model": "random_b0",+ "unit": "cpu",+ "device": "Samsung Galaxy S23",+ "job": "j5m099zyg",+ "ms": 23.86,+ "peak_mb": 134.3,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b0",+ "unit": "cpu",+ "device": "Samsung Galaxy A73 5G",+ "job": "jgnz119vg",+ "ms": 32.68,+ "peak_mb": 120.4,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b0",+ "unit": "cpu",+ "device": "Samsung Galaxy A53 5G",+ "job": "jprlxx4vp",+ "ms": 79.72,+ "peak_mb": 126.1,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b0",+ "unit": "cpu",+ "device": "Samsung Galaxy A14 5G",+ "job": "jp2roo7xg",+ "ms": 68.58,+ "peak_mb": 120.6,+ "layers": {+ "CPU": 722+ }+ },+ {+ "model": "random_b0",+ "unit": "cpu",+ "device": "Google Pixel 8",+ "job": "jpyo884r5",+ "ms": 29.89,+ "peak_mb": 166.4,+ "layers": {+ "CPU": 722+ }+ }+]\ No newline at end of filediff --git a/benchmark/results/gen8_emdb1_annotated.json b/benchmark/results/gen8_emdb1_annotated.jsonnew file mode 100644index 0000000..42b0f14--- /dev/null+++ b/benchmark/results/gen8_emdb1_annotated.json@@ -0,0 +1,346 @@+{+ "instanthmr_distill_train/runs/g8h_s0/g8h_s0/best_student_model_v3.pth": {+ "ckpt": "instanthmr_distill_train/runs/g8h_s0/g8h_s0/best_student_model_v3.pth",+ "dataset": "EMDB1",+ "split": "emdb1",+ "stride": 1,+ "bbox": "annotated",+ "adapter": "benchmark/results/adapter_smpl24_teacher.npz",+ "landmarks": "fitted",+ "epoch": 295,+ "source": "raw",+ "n": 24103,+ "n_dropped": 0,+ "num_sequences": 17,+ "results": [+ {+ "joint_set": "SMPL24+adapter",+ "num_joints": 24,+ "num_samples": 24103,+ "MPJPE_mm": 88.76262010632146,+ "PA_MPJPE_mm": 64.32690564747114,+ "PCK3D_50mm": 0.5623694837433791,+ "PCK3D_100mm": 0.8426821004300986,+ "AUC_0_150mm": 0.6134404029481004,+ "per_joint_PA_MPJPE_mm": {+ "pelvis": 42.72739616523092,+ "left_hip": 43.53664928490318,+ "right_hip": 42.47946649645917,+ "spine1": 51.80011363644271,+ "left_knee": 58.04927382682903,+ "right_knee": 56.29593421755156,+ "spine2": 51.50166121350724,+ "left_ankle": 70.10281185148675,+ "right_ankle": 70.07250431760954,+ "spine3": 50.67987416584745,+ "left_foot": 77.79060701035993,+ "right_foot": 78.21471337079649,+ "neck": 49.27518692448127,+ "left_collar": 43.24291462481559,+ "right_collar": 42.80192669461389,+ "head": 55.14169371785151,+ "left_shoulder": 47.904915709508444,+ "right_shoulder": 47.244650493662036,+ "left_elbow": 61.798039706379,+ "right_elbow": 57.81949942904837,+ "left_wrist": 99.87869570494013,+ "right_wrist": 101.27008016526156,+ "left_hand": 119.02842811021682,+ "right_hand": 125.18869870150422+ }+ },+ {+ "joint_set": "J14",+ "num_joints": 14,+ "num_samples": 24103,+ "MPJPE_mm": 100.20960661675163,+ "PA_MPJPE_mm": 67.70112248806943,+ "PCK3D_50mm": 0.48429359712187575,+ "PCK3D_100mm": 0.8266783625037785,+ "AUC_0_150mm": 0.5808690468383478,+ "per_joint_PA_MPJPE_mm": {+ "right_ankle": 70.23891257552113,+ "right_knee": 53.370854089752505,+ "right_hip": 51.9298041156822,+ "left_hip": 49.25334293941451,+ "left_knee": 56.7825543131599,+ "left_ankle": 72.06322892772938,+ "right_wrist": 102.62766164388755,+ "right_elbow": 53.70033232898691,+ "right_shoulder": 58.22250209407417,+ "left_shoulder": 49.761324857441736,+ "left_elbow": 58.86439891553489,+ "left_wrist": 103.59978537156461,+ "neck": 59.81198585359963,+ "head": 107.58902680662273+ }+ },+ {+ "joint_set": "J12",+ "num_joints": 12,+ "num_samples": 24103,+ "MPJPE_mm": 95.63595481944421,+ "PA_MPJPE_mm": 64.06419658736174,+ "PCK3D_50mm": 0.53339141738926,+ "PCK3D_100mm": 0.8486426309311428,+ "AUC_0_150mm": 0.6047484831005286,+ "per_joint_PA_MPJPE_mm": {+ "right_ankle": 69.03874380342945,+ "right_knee": 54.265726018154375,+ "right_hip": 51.31842966016909,+ "left_hip": 49.38816719105823,+ "left_knee": 57.51416548085523,+ "left_ankle": 70.59377879986292,+ "right_wrist": 98.00290890921097,+ "right_elbow": 50.425163158690815,+ "right_shoulder": 57.98955549907401,+ "left_shoulder": 54.372273287172675,+ "left_elbow": 56.59655723988543,+ "left_wrist": 99.26489000077778+ }+ }+ ],+ "per_sequence_SMPL24_adapter": {+ "P1_14_outdoor_climb": {+ "n": 1270,+ "PA_MPJPE_mm": 75.1569086897259+ },+ "P2_23_outdoor_hug_tree": {+ "n": 1086,+ "PA_MPJPE_mm": 70.37429717601877+ },+ "P3_31_outdoor_workout": {+ "n": 1216,+ "PA_MPJPE_mm": 58.15779649961338+ },+ "P3_32_outdoor_soccer_warmup_a": {+ "n": 1084,+ "PA_MPJPE_mm": 55.763399060412645+ },+ "P3_33_outdoor_soccer_warmup_b": {+ "n": 1433,+ "PA_MPJPE_mm": 64.57149990299354+ },+ "P5_42_indoor_dancing": {+ "n": 1291,+ "PA_MPJPE_mm": 83.09306734173849+ },+ "P5_44_indoor_rom": {+ "n": 1381,+ "PA_MPJPE_mm": 55.185263572409305+ },+ "P6_49_outdoor_big_stairs_down": {+ "n": 1559,+ "PA_MPJPE_mm": 77.3131934192975+ },+ "P6_50_outdoor_workout": {+ "n": 1532,+ "PA_MPJPE_mm": 55.07497626438046+ },+ "P6_51_outdoor_dancing": {+ "n": 1427,+ "PA_MPJPE_mm": 63.936823334609144+ },+ "P7_57_outdoor_rock_chair": {+ "n": 1558,+ "PA_MPJPE_mm": 56.42255440924773+ },+ "P7_59_outdoor_rom": {+ "n": 1839,+ "PA_MPJPE_mm": 59.816166687025365+ },+ "P7_60_outdoor_workout": {+ "n": 1693,+ "PA_MPJPE_mm": 55.52638191914596+ },+ "P8_64_outdoor_skateboard": {+ "n": 1704,+ "PA_MPJPE_mm": 62.05609838052144+ },+ "P8_68_outdoor_handstand": {+ "n": 1606,+ "PA_MPJPE_mm": 83.5236655338729+ },+ "P8_69_outdoor_cartwheel": {+ "n": 656,+ "PA_MPJPE_mm": 58.87959907752564+ },+ "P9_76_outdoor_sitting": {+ "n": 1768,+ "PA_MPJPE_mm": 59.30274501084284+ }+ }+ },+ "instanthmr_distill_train/runs/g8h_s1/g8h_s1/best_student_model_v3.pth": {+ "ckpt": "instanthmr_distill_train/runs/g8h_s1/g8h_s1/best_student_model_v3.pth",+ "dataset": "EMDB1",+ "split": "emdb1",+ "stride": 1,+ "bbox": "annotated",+ "adapter": "benchmark/results/adapter_smpl24_teacher.npz",+ "landmarks": "fitted",+ "epoch": 247,+ "source": "raw",+ "n": 24103,+ "n_dropped": 0,+ "num_sequences": 17,+ "results": [+ {+ "joint_set": "SMPL24+adapter",+ "num_joints": 24,+ "num_samples": 24103,+ "MPJPE_mm": 89.24478111621829,+ "PA_MPJPE_mm": 66.89051058318199,+ "PCK3D_50mm": 0.5427246262567592,+ "PCK3D_100mm": 0.8293072093377035,+ "AUC_0_150mm": 0.6015779501860072,+ "per_joint_PA_MPJPE_mm": {+ "pelvis": 43.810182621115835,+ "left_hip": 44.13638723000304,+ "right_hip": 42.90311175608087,+ "spine1": 53.19471820477527,+ "left_knee": 58.01026433731803,+ "right_knee": 56.6788951694521,+ "spine2": 53.05014683883006,+ "left_ankle": 70.70555056990528,+ "right_ankle": 71.9338142391788,+ "spine3": 52.454466492137556,+ "left_foot": 78.779893834194,+ "right_foot": 80.9591141900794,+ "neck": 52.30428934621626,+ "left_collar": 46.13822355677994,+ "right_collar": 45.72129105536187,+ "head": 59.05837199161462,+ "left_shoulder": 49.52196280870598,+ "right_shoulder": 49.190675699586116,+ "left_elbow": 63.988331123564954,+ "right_elbow": 60.81275012273367,+ "left_wrist": 106.9344451042545,+ "right_wrist": 106.81214625580598,+ "left_hand": 127.6084289102973,+ "right_hand": 130.6647925383788+ }+ },+ {+ "joint_set": "J14",+ "num_joints": 14,+ "num_samples": 24103,+ "MPJPE_mm": 100.28093707341523,+ "PA_MPJPE_mm": 69.73662325205572,+ "PCK3D_50mm": 0.46777816632191604,+ "PCK3D_100mm": 0.814531089787282,+ "AUC_0_150mm": 0.5708741153318393,+ "per_joint_PA_MPJPE_mm": {+ "right_ankle": 72.38978289942779,+ "right_knee": 54.03149026043775,+ "right_hip": 51.356108506491154,+ "left_hip": 49.811121151005466,+ "left_knee": 56.808665864142974,+ "left_ankle": 72.92868528434977,+ "right_wrist": 108.19024936320407,+ "right_elbow": 56.45063342066514,+ "right_shoulder": 60.36568877055438,+ "left_shoulder": 50.575689764376556,+ "left_elbow": 60.60541881006109,+ "left_wrist": 110.6946994511135,+ "neck": 62.07527690734759,+ "head": 110.02921507560288+ }+ },+ {+ "joint_set": "J12",+ "num_joints": 12,+ "num_samples": 24103,+ "MPJPE_mm": 96.16395031415158,+ "PA_MPJPE_mm": 66.1919422644989,+ "PCK3D_50mm": 0.5132832704089394,+ "PCK3D_100mm": 0.837381930326792,+ "AUC_0_150mm": 0.5940371720113368,+ "per_joint_PA_MPJPE_mm": {+ "right_ankle": 71.03626620506343,+ "right_knee": 54.86410905938645,+ "right_hip": 51.47818801727088,+ "left_hip": 50.6134082786491,+ "left_knee": 57.6991870489433,+ "left_ankle": 71.41209076510692,+ "right_wrist": 103.01994285775166,+ "right_elbow": 53.49245806856022,+ "right_shoulder": 60.90576162520488,+ "left_shoulder": 56.61177561619242,+ "left_elbow": 57.93463650794004,+ "left_wrist": 105.23548312391765+ }+ }+ ],+ "per_sequence_SMPL24_adapter": {+ "P1_14_outdoor_climb": {+ "n": 1270,+ "PA_MPJPE_mm": 83.788047040943+ },+ "P2_23_outdoor_hug_tree": {+ "n": 1086,+ "PA_MPJPE_mm": 74.10063803076984+ },+ "P3_31_outdoor_workout": {+ "n": 1216,+ "PA_MPJPE_mm": 61.92140648476573+ },+ "P3_32_outdoor_soccer_warmup_a": {+ "n": 1084,+ "PA_MPJPE_mm": 60.40650615697361+ },+ "P3_33_outdoor_soccer_warmup_b": {+ "n": 1433,+ "PA_MPJPE_mm": 70.6191221994521+ },+ "P5_42_indoor_dancing": {+ "n": 1291,+ "PA_MPJPE_mm": 81.43654095908283+ },+ "P5_44_indoor_rom": {+ "n": 1381,+ "PA_MPJPE_mm": 53.06881540316626+ },+ "P6_49_outdoor_big_stairs_down": {+ "n": 1559,+ "PA_MPJPE_mm": 81.41758018488571+ },+ "P6_50_outdoor_workout": {+ "n": 1532,+ "PA_MPJPE_mm": 57.0616796141908+ },+ "P6_51_outdoor_dancing": {+ "n": 1427,+ "PA_MPJPE_mm": 65.66184766547114+ },+ "P7_57_outdoor_rock_chair": {+ "n": 1558,+ "PA_MPJPE_mm": 59.24821940321826+ },+ "P7_59_outdoor_rom": {+ "n": 1839,+ "PA_MPJPE_mm": 62.59178945483681+ },+ "P7_60_outdoor_workout": {+ "n": 1693,+ "PA_MPJPE_mm": 64.53617334097041+ },+ "P8_64_outdoor_skateboard": {+ "n": 1704,+ "PA_MPJPE_mm": 64.85601161182977+ },+ "P8_68_outdoor_handstand": {+ "n": 1606,+ "PA_MPJPE_mm": 78.86868543325276+ },+ "P8_69_outdoor_cartwheel": {+ "n": 656,+ "PA_MPJPE_mm": 58.5489513986187+ },+ "P9_76_outdoor_sitting": {+ "n": 1768,+ "PA_MPJPE_mm": 59.203835018123364+ }+ }+ }+}\ No newline at end of filediff --git a/instanthmr/inference.py b/instanthmr/inference.pyindex 27b20ed..d08c109 100644--- a/instanthmr/inference.py+++ b/instanthmr/inference.py@@ -1,6 +1,7 @@"""ONNX inference for InstantHMR.-InstantHMR takes a 224x224 person crop and a 3-vector CLIFF condition+InstantHMR takes a square person crop (224 px unless the ONNX metadata+says otherwise) and a 3-vector CLIFF condition(bbox center / scale in full-frame coords) and returns mhr_params, shape,camera translation, and 70 joints in 2D (crop space) and 3D (camera coords,rig-local, metres, Y-down).@@ -200,6 +201,11 @@ class InstantHMR:# Passing `focal=` alone still implies the angular form, which is how# this argument behaved before `cliff_focal` existed.meta = self.session.get_modelmeta().custom_metadata_map or {}+ # The crop side the graph was trained at: stamped by tools/pth_to_onnx.py,+ # else the graph's static input height, else the historical 224.+ dim = self.session.get_inputs()[0].shape[2]+ self.input_size = int(meta.get("image_size")+ or (dim if isinstance(dim, int) else INPUT_SIZE))if cliff_focal is None:cliff_focal = (str(meta.get("cliff_focal", "")).lower() == "true"or focal is not None)@@ -281,7 +287,7 @@ class InstantHMR:outs = self.session.run(None,{- self._in_image: crop[np.newaxis], # (1, 3, 224, 224)+ self._in_image: crop[np.newaxis], # (1, 3, S, S), S = self.input_sizeself._in_cliff: cliff_cond[np.newaxis], # (1, 3)},)@@ -296,8 +302,8 @@ class InstantHMR:joints_3d_local = self._joints_3d(mhr_params, shape_params)# Re-project the 2D head from normalised crop space → full-frame pixels.- crop_px = (joints_2d_norm + 1.0) * 0.5 * INPUT_SIZE- scale = sq_size / INPUT_SIZE+ crop_px = (joints_2d_norm + 1.0) * 0.5 * self.input_size+ scale = sq_size / self.input_sizejoints_2d = np.stack([crop_px[:, 0] * scale + sq_x1, crop_px[:, 1] * scale + sq_y1],axis=-1,@@ -356,7 +362,7 @@ class InstantHMR:batch = max(padded_to or n, n) # always >= n; zero-pad the rest# Use zeros so padded slots produce deterministic (discarded) outputs.- crops = np.zeros((batch, 3, INPUT_SIZE, INPUT_SIZE), dtype=np.float32)+ crops = np.zeros((batch, 3, self.input_size, self.input_size), dtype=np.float32)cliffs = np.zeros((batch, 3), dtype=np.float32)sq_meta = [] # per-person (sq_x1, sq_y1, sq_size, bbox)for i, det in enumerate(detections):@@ -394,8 +400,8 @@ class InstantHMR:joints_3d_local = joints_3d_local_b[i]cam_trans = cam_trans_b[i]- crop_px = (joints_2d_norm + 1.0) * 0.5 * INPUT_SIZE- scale = sq_size / INPUT_SIZE+ crop_px = (joints_2d_norm + 1.0) * 0.5 * self.input_size+ scale = sq_size / self.input_sizejoints_2d = np.stack([crop_px[:, 0] * scale + sq_x1, crop_px[:, 1] * scale + sq_y1],axis=-1,@@ -425,7 +431,7 @@ class InstantHMR:upfront so no mid-demo stall occurs when persons first appear."""dummy_image = np.zeros(- (max_batch_size, 3, INPUT_SIZE, INPUT_SIZE), dtype=np.float32+ (max_batch_size, 3, self.input_size, self.input_size), dtype=np.float32)dummy_cliff = np.zeros((max_batch_size, 3), dtype=np.float32)# Large batch first — allocates the most memory; batch=1 reuses the plan.@@ -500,10 +506,10 @@ class InstantHMR:cv2.BORDER_CONSTANT, value=(0, 0, 0),)- crop_224 = cv2.resize(patch, (INPUT_SIZE, INPUT_SIZE), interpolation=cv2.INTER_LINEAR)- crop = crop_224.astype(np.float32) / 255.0+ crop_sq = cv2.resize(patch, (self.input_size, self.input_size), interpolation=cv2.INTER_LINEAR)+ crop = crop_sq.astype(np.float32) / 255.0crop = (crop - IMAGENET_MEAN) / IMAGENET_STD- crop = np.transpose(crop, (2, 0, 1)).astype(np.float32) # (3, 224, 224)+ crop = np.transpose(crop, (2, 0, 1)).astype(np.float32) # (3, S, S)return crop, float(sq_x1), float(sq_y1), float(sq_size), cliff_conddiff --git a/instanthmr_distill_train/README.md b/instanthmr_distill_train/README.mdindex f239791..c522325 100644--- a/instanthmr_distill_train/README.md+++ b/instanthmr_distill_train/README.md@@ -570,21 +570,69 @@ resampled identity; gradients agree to 2e-07 relative in `model_params` andso the readout is translation-equivariant and commutes with `to_vision` — whichis why the mapping is applied after the unit/axis conversion.-**Scoring is unchanged, on purpose.** `val3dpw.py` and-`benchmark/eval_3dpw_ckpt.py` both go through `get_native_keypoints`, i.e. the-fitted readout, and neither was touched. The flag changes what the model is-trained to match, not what it is measured with, so the reported J14 PA-MPJPE-stays comparable with earlier generations and keeps measuring the path that-actually ships. **Deployment is unchanged**: the exported graph never emitted-these 3D keypoints — the consumer derives them — so no operator is added to the-phone runtime. Making the deployed readout exact is a separate decision with its-own latency measurement.+**Scoring stays on the fitted readout by default, and it is now a switch.**+`val3dpw.evaluate(..., exact_landmarks=)`, `--val-landmarks {fitted,exact}` on+the trainer, and `--landmarks {fitted,exact}` on `benchmark/eval_3dpw_ckpt.py`+and `benchmark/eval_emdb_ckpt.py` all default to `fitted`, which reproduces+every recorded number to the decimal.++Measured 2026-09-22, 3DPW test stride 1, J14+adapter PA-MPJPE:++| run | trained with `--exact-landmarks` | fitted | exact |+|---|---|---|---|+| g8h_s0 | yes | 40.80 | **40.66** |+| g8h_s1 | yes | 40.89 | **40.77** |+| g8r_s0 | yes | 41.27 | **41.13** |+| g7h_s0 | **no** | 42.90 | **42.77** |++0.13 mm, the same on the run that never saw the flag — the switch is a property+of the operator, not of the training. EMDB-1 SMPL24+adapter moves 0.09-0.11 mm+the same way. It is an offset, not a re-ranking, and it is far inside the+34-49 mm seed spread.++What it removes is nonetheless a genuine mismatch: the MHR70 -> J14 and+MHR70 -> SMPL24 adapters are fitted by `benchmark/fit_adapter_mhr.py` from the+stored annotation `joints_3d`, which is the teacher's exact mesh+joint readout,+so `fitted` hands the adapter keypoints from a different operator than the one+it was fitted on. The GT side needs nothing: 3DPW's H36M joints and EMDB's SMPL+joints are SMPL-derived and independent of any MHR readout.++`fitted` remains the default because it is what the exported graph's consumer+computes, so the published number keeps describing the path that ships, and+because flipping it would shift every recorded row by 0.13 mm. **Deployment is+unchanged**: the exported graph never emitted these 3D keypoints — the consumer+derives them — so no operator is added to the phone runtime. Making the deployed+readout exact is a separate decision with its own latency measurement.`instanthmr_distill_train/assets/mhr_landmarks70.npz` (167 KB) is **a new fileand must be rsynced**, or the job dies at startup. Default off; with it offevery loss term is bit-identical, verified across `legacy`, `rebalanced`,`w_verts` on/off and `cont_head` on/off.+## `--image-size` — the network input side (added 2026-09-25)++Default 224, i.e. unchanged. Must be a multiple of 32, because the decoder's+positional grid is `image_size // 32` (7x7 at 224, 9x9 at 288). One setting+drives every place that builds the input: the training dataset, `val3dpw`+(and so the 3DPW / EMDB / SMPL-fit harnesses), `tools/pth_to_onnx.py` and+`instanthmr.inference`.++**It is recoverable from the weights.** `mem_pos_embed` is a saved buffer with+one row per patch, so `config_from_checkpoint` reads the grid off it; the exact+value comes from `run_config.json`, and a `run_config.json` that contradicts+the weights raises instead of silently resizing. The export stamps+`image_size` into the ONNX metadata, and `instanthmr.inference` reads it back+(falling back to the graph's static input shape, then 224). The harnesses+refuse to score checkpoints of different sizes in one invocation, like+`--cliff-focal`.++On today's 224 corpus a larger size only upsamples the stored crop; the real+test of resolution needs the context-crop corpus (`build_split.py --context`).++Verified by `tools/verify_input_size.py`: baseline and v2 samples bit-identical+at 224; a 288 checkpoint round-trips through the config, the export metadata,+the inference package and both harnesses.+## Tried and rejected: detaching the 2D head (2026-09-06 / 09-07)**Do not retry this without reading the failure mode.** The flag is gone fromdiff --git a/instanthmr_distill_train/train_distill_jz.py b/instanthmr_distill_train/train_distill_jz.pyindex 412e904..4e38c99 100755--- a/instanthmr_distill_train/train_distill_jz.py+++ b/instanthmr_distill_train/train_distill_jz.py@@ -389,7 +389,8 @@ def build_jz_loaders(cfg, args, rank: int, world: int):crop_centre_fix=cfg.crop_centre_fix,cliff_focal=cfg.cliff_focal,occl_p=cfg.occl_p, occl_scale=cfg.occl_scale,- jpeg_p=cfg.jpeg_p, jpeg_quality=cfg.jpeg_quality)+ jpeg_p=cfg.jpeg_p, jpeg_quality=cfg.jpeg_quality,+ image_size=cfg.image_size)# Rank 0 builds the index (or reads the cache) while the others wait, then# everyone loads the same cache. Building on all ranks at once would put@@ -538,7 +539,9 @@ def train(args, cfg, rank, local_rank, world):mhr_module = T.MHRForwardPass(cfg.mhr_model_path, device, kp_regressor=W,n_verts=cfg.n_verts if cfg.w_verts > 0 else 0,- landmark_assets=cfg.landmark_assets_path if cfg.exact_landmarks else None)+ landmark_assets=(cfg.landmark_assets_path+ if cfg.exact_landmarks or args.val_landmarks == "exact"+ else None))criterion = T.DistillationLoss(cfg, mhr_module)train_loader, val_loader, sampler, _ = build_jz_loaders(cfg, args, rank, world)@@ -563,7 +566,8 @@ def train(args, cfg, rank, local_rank, world):f"-> J14 numbers match benchmark/eval_3dpw_ckpt.py")dpw = ThreeDPWValSet(args.val_3dpw, args.val_3dpw_images,split=args.val_3dpw_split, stride=args.val_3dpw_stride,- gt=args.val_3dpw_gt, cliff_focal=cfg.cliff_focal)+ gt=args.val_3dpw_gt, cliff_focal=cfg.cliff_focal,+ input_size=cfg.image_size)# Stride-shard across ranks; the metric is all-reduced afterwards.dpw_loader = torch.utils.data.DataLoader(torch.utils.data.Subset(dpw, list(range(rank, len(dpw), world))),@@ -571,7 +575,8 @@ def train(args, cfg, rank, local_rank, world):num_workers=max(1, cfg.num_workers // 2), pin_memory=True)log(f"3DPW {args.val_3dpw_split}: {len(dpw):,} person-frames "f"(stride {args.val_3dpw_stride}, GT {args.val_3dpw_gt}) "- f"-> selection metric is J14 PA-MPJPE (adapter-applied)")+ f"-> selection metric is J14 PA-MPJPE (adapter-applied, "+ f"{args.val_landmarks} 70-keypoint readout)")# Optional read-only monitor on the split the paper reports. Selection# NEVER touches it: picking the best of ~100 epochs on the test split@@ -581,7 +586,8 @@ def train(args, cfg, rank, local_rank, world):if args.val_3dpw_test:dpw_t = ThreeDPWValSet(args.val_3dpw, args.val_3dpw_images,split="test", stride=args.val_3dpw_test_stride,- gt=args.val_3dpw_gt, cliff_focal=cfg.cliff_focal)+ gt=args.val_3dpw_gt, cliff_focal=cfg.cliff_focal,+ input_size=cfg.image_size)dpw_test_loader = torch.utils.data.DataLoader(torch.utils.data.Subset(dpw_t, list(range(rank, len(dpw_t), world))),batch_size=cfg.batch_size, shuffle=False, drop_last=False,@@ -695,6 +701,7 @@ def train(args, cfg, rank, local_rank, world):"lr": cfg.lr, "lr_arg": args.lr, "lr_scaling": args.lr_scaling,"mix": args.mix, "steps_per_epoch": steps_per_epoch,"backbone": cfg.backbone, "w_reproj": cfg.w_reproj,+ "image_size": cfg.image_size,"cont_head": cfg.cont_head, "root_rot_loss": cfg.root_rot_loss,"exact_landmarks": cfg.exact_landmarks,"geom_scale_max": cfg.geom_scale_max, "occl_p": cfg.occl_p,@@ -704,6 +711,7 @@ def train(args, cfg, rank, local_rank, world):"pose_split": cfg.pose_split, "pose_beta": cfg.pose_beta,"finger_weight": cfg.finger_weight,"val_3dpw_gt": args.val_3dpw_gt if args.val_3dpw else None,+ "val_landmarks": args.val_landmarks if args.val_3dpw else None,"cliff_focal": cfg.cliff_focal,"w_verts": cfg.w_verts, "n_verts": cfg.n_verts,"crop_centre_fix": cfg.crop_centre_fix,@@ -884,20 +892,22 @@ def train(args, cfg, rank, local_rank, world):dpw_raw = dpw_em = dpw_test_raw = dpw_test_em = Noneif dpw_loader is not None:import val3dpw+ exact_lm = args.val_landmarks == "exact"dpw_raw = all_reduce_mean(val3dpw.evaluate(ddp_model, dpw_loader, mhr_module, device,cfg.use_amp, gt=args.val_3dpw_gt,- adapter=dpw_adapter),+ adapter=dpw_adapter, exact_landmarks=exact_lm),device)dpw_em = all_reduce_mean(val3dpw.evaluate(ema, dpw_loader, mhr_module, device,cfg.use_amp, gt=args.val_3dpw_gt,- adapter=dpw_adapter),+ adapter=dpw_adapter, exact_landmarks=exact_lm),device)if dpw_test_loader is not None:dpw_test_raw = all_reduce_mean(val3dpw.evaluate(ddp_model, dpw_test_loader, mhr_module, device, cfg.use_amp,- gt=args.val_3dpw_gt, adapter=dpw_adapter), device)+ gt=args.val_3dpw_gt, adapter=dpw_adapter,+ exact_landmarks=exact_lm), device)dpw_test_em = all_reduce_mean(val3dpw.evaluate(ema, dpw_test_loader, mhr_module, device, cfg.use_amp,gt=args.val_3dpw_gt, adapter=dpw_adapter), device)@@ -1103,6 +1113,19 @@ def parse_args():"benchmark/make_3dpw_gt.py). jointpositions is the raw ""pickle field the runs before this flag selected on — ""several mm apart, so a resumed run must keep its own.")+ p.add_argument("--val-landmarks", dest="val_landmarks", default="fitted",+ choices=("fitted", "exact"),+ help="Which readout the 3DPW selection metric scores. "+ "fitted (default) is the (70, 127) skeleton matrix the "+ "exported graph's consumer uses, and what every "+ "recorded run was selected on. exact is the teacher's "+ "mesh+joint mapping -- what --exact-landmarks trains "+ "against, and the operator the adapter was fitted "+ "from. Measured 0.13 mm of J14+adapter PA-MPJPE on "+ "3DPW test and 0.05 mm on validation, with or without "+ "--exact-landmarks training, so it does not move "+ "selection; it is an offset, and mixing the two in one "+ "table is what to avoid.")p.add_argument("--mix", default="sqrt",help="Dataset mixture: sqrt | uniform | equal | ""'sa1b=0.4,aic=0.35,coco=0.15,mpii=0.1'.")@@ -1113,6 +1136,11 @@ def parse_args():p.add_argument("--w_simcc", type=float, default=None)p.add_argument("--weight_decay", type=float, default=None)p.add_argument("--backbone", default=None)+ p.add_argument("--image-size", dest="image_size", type=int, default=None,+ help="Network input side in px (default 224). A multiple of "+ "32: the decoder's positional grid is image_size // 32. "+ "Recorded in run_config.json and read back by "+ "config_from_checkpoint, the harnesses and the export.")p.add_argument("--ema-decay", type=float, default=None)p.add_argument("--sync-bn", action="store_true")@@ -1151,6 +1179,15 @@ def parse_args():"farthest-point subset of the rig's own 18,439-vertex ""mesh (default 595, lod6's count). Costs 0.6 ms per ""batch-64 step against 16.7 ms for the full mesh.")+ p.add_argument("--early-stop-patience", dest="early_stop_patience", type=int, default=None,+ help="Stop after this many epochs with no improvement in the "+ "selection metric (config default 150). The counter is "+ "carried in the checkpoint, so it accumulates across "+ "chained links. Raise it to disable: with OneCycleLR, "+ "which anneals to ~0 only in the last 10%% of the planned "+ "epochs, a plateau at high LR can expire the patience "+ "just before the refinement phase that would have ended "+ "it -- g8r_s0 stopped at 243 of 300 that way.")p.add_argument("--exact-landmarks", dest="exact_landmarks", action="store_true",help="Supervise the 70 keypoints with the teacher's exact ""mesh+joint mapping over the 468 vertices it references, "@@ -1208,11 +1245,17 @@ def build_cfg(args):cfg.cont_head = Truecfg.root_rot_loss = Trueif args.exact_landmarks: cfg.exact_landmarks = True+ if args.early_stop_patience is not None:+ cfg.early_stop_patience = args.early_stop_patienceif args.w_verts is not None: cfg.w_verts = args.w_vertsif args.n_verts is not None: cfg.n_verts = args.n_vertsif args.w_simcc is not None: cfg.w_simcc = args.w_simccif args.weight_decay is not None: cfg.weight_decay = args.weight_decayif args.ema_decay is not None: cfg.ema_decay = args.ema_decay+ if args.image_size is not None:+ if args.image_size % 32:+ raise SystemExit(f"--image-size {args.image_size} is not a multiple of 32")+ cfg.image_size = args.image_sizeif args.preset == "v2":apply_v2_preset(cfg)diff --git a/instanthmr_distill_train/train_distill_mhr_only.py b/instanthmr_distill_train/train_distill_mhr_only.pyindex 8f2dc3a..75395d1 100644--- a/instanthmr_distill_train/train_distill_mhr_only.py+++ b/instanthmr_distill_train/train_distill_mhr_only.py@@ -434,7 +434,9 @@ def apply_rebalanced_losses(cfg) -> None:def config_from_checkpoint(state, ckpt_path=None, **overrides):"""Rebuild the DistillConfig a checkpoint was actually trained with.- Two sources, because the checkpoint dict carries neither.+ Two sources, because the checkpoint dict carries neither. (``image_size``+ sits in both: the positional-embedding grid in the weights and the value+ in run_config.json, which must agree.)* The ARCHITECTURE flags are readable from the weights. ``--bound-scales``registers ``scale_lo`` / ``scale_hi`` buffers, so their presence in the@@ -484,6 +486,22 @@ def config_from_checkpoint(state, ckpt_path=None, **overrides):setattr(cfg, k, run_cfg[k])prov[k] = "run_config.json"+ # The input size IS in the weights: the decoder's positional embedding is a+ # saved buffer with one row per 32-px patch, so its grid fixes the size up+ # to the stride. run_config.json carries the exact value; a contradiction+ # between the two means the wrong run_config.json sits beside the weights.+ if "mem_pos_embed" in state:+ grid = math.isqrt(state["mem_pos_embed"].shape[1])+ cfg.image_size = grid * 32+ prov["image_size"] = "checkpoint weights"+ if "image_size" in run_cfg and int(run_cfg["image_size"]) != grid * 32:+ raise ValueError(+ f"run_config.json says image_size={run_cfg['image_size']} but "+ f"the weights have a {grid}x{grid} patch grid ({grid * 32} px)")+ elif "image_size" in run_cfg:+ cfg.image_size = int(run_cfg["image_size"])+ prov["image_size"] = "run_config.json"+for k, v in overrides.items():if v is not None:setattr(cfg, k, v)@@ -866,7 +884,8 @@ def build_dataloaders(cfg):crop_centre_fix=cfg.crop_centre_fix,cliff_focal=cfg.cliff_focal,occl_p=cfg.occl_p, occl_scale=cfg.occl_scale,- jpeg_p=cfg.jpeg_p, jpeg_quality=cfg.jpeg_quality)+ jpeg_p=cfg.jpeg_p, jpeg_quality=cfg.jpeg_quality,+ image_size=cfg.image_size)full_dataset = SAM3DStudentDataset(cfg.data_root,augment=cfg.augment,@@ -1421,10 +1440,21 @@ class MHRForwardPass:return (torch.einsum('kj,bjc->bkc', self.lm_W_joint, joints_vision)+ torch.einsum('kv,bvc->bkc', self.lm_W_vert, v))- def get_native_keypoints(self, model_params, shape_params):- """(B, 70, 3) native keypoints in the annotation frame."""- j = self.get_joints(model_params, shape_params)[..., :3]- return self.regress_keypoints(self.to_vision(j))+ def get_native_keypoints(self, model_params, shape_params, exact: bool = False):+ """(B, 70, 3) native keypoints in the annotation frame.++ `exact=True` switches from the fitted (70, 127) skeleton matrix to the+ teacher's own `Wj @ J + Wv @ V` readout -- the operator the stored+ annotations came out of, and therefore the one the MHR70 -> J14 / SMPL24+ adapters were fitted against. It needs a module built with+ `landmark_assets=`. Default off: the fitted readout is what the exported+ graph's consumer uses, so it is what the published metric measures.+ """+ if not exact:+ j = self.get_joints(model_params, shape_params)[..., :3]+ return self.regress_keypoints(self.to_vision(j))+ j, v = self.get_joints_and_vertices(model_params, shape_params)+ return self.regress_keypoints_exact(self.to_vision(j), self.to_vision(v))class DistillationLoss(nn.Module):def __init__(self, cfg, mhr_module):@@ -2477,7 +2507,7 @@ def _mock_preds_from(batch):def run_self_tests():print("--- Architecture Output Shapes ---")test_model = InstantHMRStudent(cfg, pretrained=False)- out = test_model(torch.randn(2, 3, 224, 224), torch.randn(2, 3))+ out = test_model(torch.randn(2, 3, cfg.image_size, cfg.image_size), torch.randn(2, 3))for k, v in out.items():print(f" {k}: {tuple(v.shape)}")n_par = sum(p.numel() for p in test_model.parameters())@@ -2496,7 +2526,8 @@ def run_self_tests():print("\n--- Perfect Student (identity batch) ---")clean_ds = SAM3DStudentDataset(cfg.data_root, augment=False,- max_images=256, per_dataset_caps=cfg.per_dataset_caps)+ max_images=256, per_dataset_caps=cfg.per_dataset_caps,+ image_size=cfg.image_size)clean_loader = DataLoader(clean_ds, batch_size=32, shuffle=False, num_workers=0)cb = {k: v.to(device) for k, v in next(iter(clean_loader)).items()}l_clean = criterion(_mock_preds_from(cb), cb)@@ -2514,7 +2545,8 @@ def run_self_tests():max_images=256, per_dataset_caps=cfg.per_dataset_caps,geom_p=1.0, geom_rot_deg=cfg.geom_rot_deg,geom_scale_range=cfg.geom_scale_range,- geom_trans=cfg.geom_trans, geom_flip_p=0.0)+ geom_trans=cfg.geom_trans, geom_flip_p=0.0,+ image_size=cfg.image_size)aug_loader = DataLoader(aug_ds, batch_size=32, shuffle=False, num_workers=0)ab = {k: v.to(device) for k, v in next(iter(aug_loader)).items()}@@ -2549,7 +2581,8 @@ def run_self_tests():flip_ds = SAM3DStudentDataset(cfg.data_root, augment=True,max_images=256, per_dataset_caps=cfg.per_dataset_caps,geom_p=1.0, geom_rot_deg=0.0, geom_scale_range=0.0,- geom_trans=0.0, geom_flip_p=1.0)+ geom_trans=0.0, geom_flip_p=1.0,+ image_size=cfg.image_size)flip_loader = DataLoader(flip_ds, batch_size=32, shuffle=False, num_workers=0)fb = {k: v.to(device) for k, v in next(iter(flip_loader)).items()}with torch.no_grad():@@ -2581,7 +2614,8 @@ def run_self_tests():max_images=256, per_dataset_caps=cfg.per_dataset_caps,geom_p=1.0, geom_rot_deg=0.0,geom_scale_range=cfg.geom_scale_range,- geom_trans=cfg.geom_trans, geom_flip_p=0.0)+ geom_trans=cfg.geom_trans, geom_flip_p=0.0,+ image_size=cfg.image_size)st_loader = DataLoader(st_ds, batch_size=32, shuffle=False, num_workers=0)sb = {k: v.to(device) for k, v in next(iter(st_loader)).items()}l_aug = criterion(_mock_preds_from(sb), sb)@@ -2632,7 +2666,8 @@ def run_overfit_test(steps=3000, subset=8, lr=5e-4):max_images=subset,per_dataset_caps=cfg.per_dataset_caps,geom_p=0.0, # STRICTLY DISABLE SPATIAL AUGS- geom_flip_p=0.0 # STRICTLY DISABLE FLIPS+ geom_flip_p=0.0, # STRICTLY DISABLE FLIPS+ image_size=cfg.image_size)overfit_loader = DataLoader(overfit_dataset, batch_size=subset, shuffle=False, num_workers=0)batch = next(iter(overfit_loader))diff --git a/instanthmr_distill_train/val3dpw.py b/instanthmr_distill_train/val3dpw.pyindex 8700837..222305d 100644--- a/instanthmr_distill_train/val3dpw.py+++ b/instanthmr_distill_train/val3dpw.py@@ -62,7 +62,7 @@ MM = 1000.0def _preprocess(image_rgb: np.ndarray, bbox: np.ndarray, h: int, w: int,- focal: float | None = None):+ focal: float | None = None, size: int = INPUT_SIZE):"""Square 1.2x crop + ImageNet normalise + CLIFF conditioning.A transcription of ``instanthmr.inference.InstantHMR._preprocess``. It is@@ -98,7 +98,7 @@ def _preprocess(image_rgb: np.ndarray, bbox: np.ndarray, h: int, w: int,if pl or pt or pr or pb:patch = cv2.copyMakeBorder(patch, pt, pb, pl, pr,cv2.BORDER_CONSTANT, value=(0, 0, 0))- crop = cv2.resize(patch, (INPUT_SIZE, INPUT_SIZE), interpolation=cv2.INTER_LINEAR)+ crop = cv2.resize(patch, (size, size), interpolation=cv2.INTER_LINEAR)crop = (crop.astype(np.float32) / 255.0 - IMAGENET_MEAN) / IMAGENET_STDreturn np.ascontiguousarray(crop.transpose(2, 0, 1)), cliff@@ -115,13 +115,15 @@ class ThreeDPWValSet(Dataset):def __init__(self, sequence_dir: str, image_root: str,split: str = "validation", stride: int = 5,bbox_scale: float = 1.2, gt: str = "h36m",- gt_dir: str | None = None, cliff_focal: bool = False):+ gt_dir: str | None = None, cliff_focal: bool = False,+ input_size: int = INPUT_SIZE):self.samples, self.gt = P.build_samples(sequence_dir, image_root, split=split,stride=stride, bbox_scale=bbox_scale, gt=gt, gt_dir=gt_dir)self.gt = np.asarray(self.gt, dtype=np.float32)self.gt_source = gtself.cliff_focal = cliff_focal+ self.input_size = input_sizedef __len__(self) -> int:return len(self.samples)@@ -132,14 +134,15 @@ class ThreeDPWValSet(Dataset):if bgr is None:# A missing frame must not kill a 20-hour job; return a zero crop# and flag it so the metric can drop it.- return {"image": torch.zeros(3, INPUT_SIZE, INPUT_SIZE),+ return {"image": torch.zeros(3, self.input_size, self.input_size),"cliff_cond": torch.zeros(3),"gt": torch.from_numpy(self.gt[i]),"ok": torch.tensor(0.0)}rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)h, w = rgb.shape[:2]crop, cliff = _preprocess(rgb, s["bbox"], h, w,- s["focal"] if self.cliff_focal else None)+ s["focal"] if self.cliff_focal else None,+ size=self.input_size)return {"image": torch.from_numpy(crop),"cliff_cond": torch.from_numpy(cliff),"gt": torch.from_numpy(self.gt[i]),@@ -148,12 +151,20 @@ class ThreeDPWValSet(Dataset):@torch.no_grad()def evaluate(model, loader, mhr_module, device, use_amp: bool = True,- gt: str = "h36m", adapter: np.ndarray | None = None) -> dict:+ gt: str = "h36m", adapter: np.ndarray | None = None,+ exact_landmarks: bool = False) -> dict:"""J14 PA-MPJPE and MPJPE in mm, adapter-applied — the paper number.- Predictions come from ``mhr_params`` through the same FK + (70, 127)- regressor the training loss uses, so this measures exactly the geometry the- model is being trained to produce.+ Predictions come from ``mhr_params`` through FK and the fitted (70, 127)+ regressor, which is the readout the exported graph's consumer uses.++ ``exact_landmarks=True`` uses the teacher's mesh+joint mapping instead —+ the readout ``--exact-landmarks`` trains against and the one the adapter+ was fitted from. Measured on 3DPW test stride 5 it is worth 0.13 mm of+ J14+adapter PA-MPJPE, the same amount on a run trained WITHOUT+ ``--exact-landmarks``, so it is a property of the operator and not of the+ training flag. Off by default: flipping it shifts every recorded number by+ that much and stops the metric describing the deployed path."""model.eval()preds, gts = [], []@@ -166,7 +177,8 @@ def evaluate(model, loader, mhr_module, device, use_amp: bool = True,with torch.amp.autocast(device_type="cuda", dtype=torch.float16, enabled=use_amp):out = model(img, cc)kp = mhr_module.get_native_keypoints(out["mhr_params"].float(),- out["shape_params"].float())+ out["shape_params"].float(),+ exact=exact_landmarks)# Drop non-finite predictions BEFORE they reach the metric. The forward# runs under fp16 autocast, so an early-training head output above# 65504 is already inf by the time .float() sees it, and the MHR forwarddiff --git a/tools/ddp_smoke.py b/tools/ddp_smoke.pyindex 7906ea2..5e4c52d 100644--- a/tools/ddp_smoke.py+++ b/tools/ddp_smoke.py@@ -3,6 +3,7 @@python tools/ddp_smoke.py --backbone hgnetv2_b4 --w-verts 0.35python tools/ddp_smoke.py --backbone repvit_m2_3 # the control+ python tools/ddp_smoke.py --image-size 288 # a non-224 inputWHY THIS EXISTS. A plain single-process smoke run -- `train_distill_jz.py` on alaptop, `--self-test`, `--overfit-test` -- builds no DDP Reducer at all, so it@@ -51,6 +52,8 @@ def main() -> int:p = argparse.ArgumentParser(description=__doc__,formatter_class=argparse.RawDescriptionHelpFormatter)p.add_argument("--backbone", default=None, help="timm name; default is the config's")+ p.add_argument("--image-size", dest="image_size", type=int, default=None,+ help="network input side; the positional grid and every crop follow it")p.add_argument("--data_root", default="data")p.add_argument("--batch_size", type=int, default=2,help="Small on purpose. Which parameters receive a gradient "@@ -98,10 +101,15 @@ def main() -> int:cfg.exact_landmarks = Trueif args.backbone:cfg.backbone = args.backbone+ if args.image_size is not None:+ if args.image_size % 32:+ raise SystemExit(f"--image-size {args.image_size} is not a multiple of 32")+ cfg.image_size = args.image_sizeds = T.SAM3DStudentDataset(cfg.data_root, augment=True, max_images=args.batch_size,per_dataset_caps={}, cliff_focal=cfg.cliff_focal,- crop_centre_fix=cfg.crop_centre_fix)+ crop_centre_fix=cfg.crop_centre_fix,+ image_size=cfg.image_size)loader = torch.utils.data.DataLoader(ds, batch_size=args.batch_size, num_workers=0)batch = {k: (v.to(T.device) if torch.is_tensor(v) else v)for k, v in next(iter(loader)).items()}diff --git a/tools/profiling/export_family.py b/tools/profiling/export_family.pynew file mode 100644index 0000000..04f0bf7--- /dev/null+++ b/tools/profiling/export_family.py@@ -0,0 +1,47 @@+"""Usage: python tools/profiling/export_family.py <out_dir> then python tools/profiling/profile_family.py <out_dir>+Ticket 03: ONNX exports of the g8h architecture with the backbone swapped.+Random weights for B0/B1/B4, plus the trained g8h, all through one path."""+import io, sys, copy, json+from pathlib import Path+import torch, onnx, onnxsim+REPO = Path(__file__).resolve().parents[2]+sys.path.insert(0, str(REPO / "instanthmr_distill_train"))+import train_distill_mhr_only as T++CKPT = REPO / "instanthmr_distill_train/runs/g8h_s0/g8h_s0/best_student_model_ema.pth"+OUT = Path(sys.argv[1])+blob = torch.load(CKPT, map_location="cpu", weights_only=False)+state = {k.replace("_orig_mod.", ""): v for k, v in blob["model_state_dict"].items()}+cfg, _ = T.config_from_checkpoint(state, CKPT)+cfg.kp2d_bins = state["head_2d_logits.bias"].shape[0] // 2+cfg.kp2d_range = float(state["kp2d_bin_centers"][-1])++class Deploy(torch.nn.Module):+ def __init__(self, m): super().__init__(); self.m = m+ def forward(self, image, cliff_cond):+ o = self.m(image, cliff_cond)+ return o["mhr_params"], o["shape_params"], o["cam_trans"], o["joints_2d"]++def export(model, name):+ model.eval()+ buf = io.BytesIO()+ torch.onnx.export(Deploy(model), (torch.zeros(1, 3, 224, 224), torch.zeros(1, 3)), buf,+ input_names=["image", "cliff_cond"],+ output_names=["mhr_params", "shape_params", "cam_trans", "joints_2d"],+ opset_version=17)+ m, ok = onnxsim.simplify(onnx.load_from_string(buf.getvalue()))+ assert ok+ p = OUT / f"{name}.onnx"; onnx.save(m, p)+ n = sum(x.numel() for x in model.parameters())+ nb = sum(x.numel() for x in model.backbone.parameters())+ print(json.dumps(dict(name=name, params_M=round(n/1e6, 2), backbone_M=round(nb/1e6, 2),+ mb=round(p.stat().st_size/2**20, 1))), flush=True)++trained = T.InstantHMRStudent(copy.deepcopy(cfg), pretrained=False)+missing, unexpected = trained.load_state_dict(state, strict=False)+assert not missing and not unexpected, (missing[:5], unexpected[:5])+export(trained, "g8h_trained_b4")+for bb in ("hgnetv2_b4", "hgnetv2_b1", "hgnetv2_b0"):+ c = copy.deepcopy(cfg); c.backbone = bb+ torch.manual_seed(0)+ export(T.InstantHMRStudent(c, pretrained=False), f"random_{bb.split('_')[1]}")diff --git a/tools/profiling/profile_family.py b/tools/profiling/profile_family.pynew file mode 100644index 0000000..85bca8c--- /dev/null+++ b/tools/profiling/profile_family.py@@ -0,0 +1,35 @@+import json, sys+from pathlib import Path+import qai_hub as hub+D = Path(sys.argv[1])+CPU = ["Samsung Galaxy S23", "Samsung Galaxy A73 5G", "Samsung Galaxy A53 5G",+ "Samsung Galaxy A14 5G", "Google Pixel 8"]+specs = dict(image=(1, 3, 224, 224), cliff_cond=(1, 3))+names = ["g8h_trained_b4", "random_b4", "random_b1", "random_b0"]+comp = {n: hub.submit_compile_job(model=str(D / f"{n}.onnx"), device=hub.Device("Samsung Galaxy S23"),+ input_specs=specs, options="--target_runtime onnx", name=f"t03-{n}")+ for n in names}+jobs = []+for n, cj in comp.items():+ m = cj.get_target_model()+ if m is None:+ print(json.dumps(dict(model=n, compile_failed=cj.get_status().message)), flush=True); continue+ jobs.append((n, "npu", "Samsung Galaxy S23",+ hub.submit_profile_job(model=m, device=hub.Device("Samsung Galaxy S23"), name=f"t03-{n}-npu")))+ for d in CPU:+ jobs.append((n, "cpu", d, hub.submit_profile_job(model=m, device=hub.Device(d),+ name=f"t03-{n}-cpu", options="--compute_unit cpu")))+out = []+for n, u, d, j in jobs:+ row = dict(model=n, unit=u, device=d, job=j.job_id)+ if j.wait().success:+ p = j.download_profile(); s = p["execution_summary"]+ units = {}+ for l in p.get("execution_detail", []):+ units[l.get("compute_unit")] = units.get(l.get("compute_unit"), 0) + 1+ row.update(ms=round(s["estimated_inference_time"] / 1000, 2),+ peak_mb=round(s.get("estimated_inference_peak_memory", 0) / 2**20, 1), layers=units)+ else:+ row["error"] = j.get_status().message+ print(json.dumps(row), flush=True); out.append(row)+json.dump(out, open(D / "profile.json", "w"), indent=1)diff --git a/tools/pth_to_onnx.py b/tools/pth_to_onnx.pyindex 104dea0..9479237 100644--- a/tools/pth_to_onnx.py+++ b/tools/pth_to_onnx.py@@ -7,7 +7,8 @@ It reuses the exact model architecture and deploy wrapper from the trainingscript that produced the checkpoint, plus the FP32 export recipe from notebookCell 15, so the graph I/O matches ``instanthmr/inference.py``:- inputs : image (N, 3, 224, 224), cliff_cond (N, 3)+ inputs : image (N, 3, S, S), cliff_cond (N, 3); S = the checkpoint's input+ size (224 unless trained with --image-size), stamped as image_sizeoutputs: mhr_params, shape_params, cam_trans, joints_2d[, joints_3d]Architecture selection@@ -35,7 +36,7 @@ Usage# or point at a specific checkpoint / outputpython ./tools/pth_to_onnx.py \- --ckpt ./instanthmr_distill_train/runs/g7h_s0/g7h_s0/best_student_model_v3.pth \+ --ckpt ./instanthmr_distill_train/runs/g8h_s1/g8h_s1/best_student_model_v3.pth \--output ./models/instanthmr.onnx"""import argparse@@ -208,12 +209,14 @@ def main() -> None:m = onnx.load(str(args.output))for k, v in (("cliff_focal", str(bool(getattr(cfg, "cliff_focal", False))).lower()),("bound_scales", str(bool(getattr(cfg, "bound_scales", False))).lower()),- ("arch", arch)):+ ("arch", arch),+ ("image_size", str(int(cfg.image_size)))):e = m.metadata_props.add()e.key, e.value = k, vonnx.save(m, str(args.output))print(f" metadata: cliff_focal={getattr(cfg, 'cliff_focal', False)} "- f"bound_scales={getattr(cfg, 'bound_scales', False)} arch={arch}")+ f"bound_scales={getattr(cfg, 'bound_scales', False)} arch={arch} "+ f"image_size={cfg.image_size}")except Exception as e: # noqa: BLE001print(f" [warning] could not stamp ONNX metadata: {e}")diff --git a/tools/verify_input_size.py b/tools/verify_input_size.pynew file mode 100644index 0000000..88fed6b--- /dev/null+++ b/tools/verify_input_size.py@@ -0,0 +1,234 @@+#!/usr/bin/env python3+"""Verify that the network input size is one checkpoint setting, end to end.++ python tools/verify_input_size.py --ref <reference train_distill_mhr_only.py> \+ --data_root data/sam3d_gt_coco --emdb-root ~/Downloads/EMDB_root \+ --dpw-seq ~/Downloads/sequenceFiles/sequenceFiles --dpw-img ~/Downloads/imageFiles++--ref is the trainer as it was before the change (default: HEAD). Checks:++1. baseline and v2 augmented samples are bit-identical to --ref at 224,+ compared on alternating calls (the rig's first call differs by ~5e-10).+2. A 288 checkpoint round-trips: config_from_checkpoint reads 288 from the+ weights (the positional-embedding grid) with or without run_config.json,+ and refuses a run_config.json that contradicts the weights.+3. The 3DPW / EMDB crop builder emits the configured size, and at 224 is+ byte-identical to --ref's.+4. The ONNX export of a 288 checkpoint carries image_size=288 in its metadata,+ and the inference package builds 288 crops from it on a real frame.+5. The EMDB and 3DPW harnesses run a 288 checkpoint end to end.+"""++from __future__ import annotations++import argparse+import importlib.util+import json+import random+import subprocess+import sys+import tempfile+from pathlib import Path++import cv2+import numpy as np+import torch++REPO = Path(__file__).resolve().parent.parent+TRAIN = REPO / "instanthmr_distill_train"+sys.path.insert(0, str(TRAIN))+sys.path.insert(0, str(REPO))+import train_distill_mhr_only as T # noqa: E402+import val3dpw # noqa: E402++G8H = TRAIN / "runs/g8h_s0/g8h_s0/best_student_model_v3.pth"+fails: list[str] = []+++def check(ok: bool, msg: str) -> None:+ print(f" {'OK ' if ok else 'BAD'} {msg}")+ if not ok:+ fails.append(msg)+++def load_module(path: Path, name: str):+ spec = importlib.util.spec_from_file_location(name, path)+ mod = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(mod)+ return mod+++def dataset(mod, root: str, v2: bool, augment: bool = True, **size):+ cfg = mod.DistillConfig()+ cfg.cliff_focal = True+ cfg.crop_centre_fix = True+ if v2:+ jz = load_module(TRAIN / "train_distill_jz.py", "jz_for_preset")+ jz.apply_v2_preset(cfg)+ return mod.SAM3DStudentDataset(+ root, augment=augment, max_images=64, geom_p=cfg.geom_p,+ geom_rot_deg=cfg.geom_rot_deg, geom_scale_range=cfg.geom_scale_range,+ geom_trans=cfg.geom_trans, geom_flip_p=cfg.geom_flip_p,+ geom_scale_max=cfg.geom_scale_max, cliff_follows_aug=cfg.cliff_follows_aug,+ crop_centre_fix=cfg.crop_centre_fix, cliff_focal=cfg.cliff_focal,+ occl_p=cfg.occl_p, occl_scale=cfg.occl_scale, jpeg_p=cfg.jpeg_p,+ jpeg_quality=cfg.jpeg_quality, **size)+++def sample(ds, i: int, seed: int) -> dict:+ random.seed(seed)+ np.random.seed(seed)+ torch.manual_seed(seed)+ return ds[i]+++def same(a, b) -> bool:+ if isinstance(a, torch.Tensor):+ return torch.equal(a, b)+ if isinstance(a, np.ndarray):+ return np.array_equal(a, b, equal_nan=a.dtype.kind == "f")+ if isinstance(a, dict):+ return a.keys() == b.keys() and all(same(a[k], b[k]) for k in a)+ return a == b+++def g8h_cfg():+ st = torch.load(G8H, map_location="cpu", weights_only=False)["model_state_dict"]+ cfg, _ = T.config_from_checkpoint(st, G8H)+ return cfg+++def main() -> None:+ p = argparse.ArgumentParser(description=__doc__,+ formatter_class=argparse.RawDescriptionHelpFormatter)+ p.add_argument("--ref", type=Path, default=None)+ p.add_argument("--data_root", default=str(REPO / "data/sam3d_gt_coco"))+ p.add_argument("--emdb-root", type=Path, default=Path.home() / "Downloads/EMDB_root")+ p.add_argument("--dpw-seq", type=Path,+ default=Path.home() / "Downloads/sequenceFiles/sequenceFiles")+ p.add_argument("--dpw-img", type=Path, default=Path.home() / "Downloads/imageFiles")+ p.add_argument("--n", type=int, default=24)+ args = p.parse_args()+ tmp = Path(tempfile.mkdtemp(prefix="verify_input_size_"))++ ref_path = args.ref+ if ref_path is None:+ ref_path = tmp / "train_distill_mhr_only_ref.py"+ ref_path.write_text(subprocess.run(+ ["git", "-C", str(REPO), "show",+ "HEAD:instanthmr_distill_train/train_distill_mhr_only.py"],+ check=True, capture_output=True, text=True).stdout)+ R = load_module(ref_path, "train_distill_mhr_only_ref")++ print("[1] augmented samples vs reference, 224")+ for v2 in (False, True):+ # the new code as both trainers now call it (image_size passed), the+ # reference as it was called before (no such argument)+ new = dataset(T, args.data_root, v2, image_size=224)+ old = dataset(R, args.data_root, v2)+ diff = 0+ for k in range(args.n):+ i = k % len(new)+ a = sample(new, i, 1000 + k) # alternate: new, old, new, old ...+ b = sample(old, i, 1000 + k)+ diff += not same(a, b)+ check(diff == 0, f"{'v2' if v2 else 'baseline'}: {diff} of {args.n} samples differ")++ print("[1b] labels do not depend on the input size (no augmentation)")+ d224 = dataset(T, args.data_root, False, augment=False, image_size=224)+ d288 = dataset(T, args.data_root, False, augment=False, image_size=288)+ bad = 0+ for i in range(min(args.n, len(d224))):+ a, b = d224[i], d288[i]+ bad += not (a["image"].shape[-1] == 224 and b["image"].shape[-1] == 288+ and torch.allclose(a["joints_2d"], b["joints_2d"], atol=1e-6)+ and torch.equal(a["cliff_cond"], b["cliff_cond"]))+ check(bad == 0, f"224 vs 288: {bad} samples with a different label or CLIFF vector")++ print("[2] a 288 checkpoint round-trips through config_from_checkpoint")+ cfg288 = g8h_cfg()+ cfg288.image_size = 288+ torch.manual_seed(0)+ m288 = T.InstantHMRStudent(cfg288, pretrained=False).eval()+ ck_dir = tmp / "r288"+ ck_dir.mkdir()+ ck = ck_dir / "best_student_model_v3.pth"+ torch.save({"model_state_dict": m288.state_dict(), "epoch": 0}, ck)+ st = m288.state_dict()+ c, prov = T.config_from_checkpoint(st, ck)+ check(c.image_size == 288, f"no run_config.json: image_size={c.image_size} "+ f"[{prov.get('image_size')}]")+ rc = {"image_size": 288, "cliff_focal": True, "backbone": cfg288.backbone}+ (ck_dir / "run_config.json").write_text(json.dumps(rc))+ c, prov = T.config_from_checkpoint(st, ck)+ check(c.image_size == 288, f"with run_config.json: image_size={c.image_size}")+ (ck_dir / "bad").mkdir()+ (ck_dir / "bad" / "run_config.json").write_text(json.dumps({**rc, "image_size": 224}))+ try:+ T.config_from_checkpoint(st, ck_dir / "bad" / "x.pth")+ check(False, "contradicting run_config.json was accepted")+ except ValueError as e:+ check(True, f"contradicting run_config.json refused: {e}")+ c224 = g8h_cfg()+ check(c224.image_size == 224, f"g8h_s0 reads back as {c224.image_size}")++ print("[3] crop builder (3DPW / EMDB validation)")+ frame = next(iter(sorted(args.dpw_img.rglob("*.jpg"))))+ rgb = cv2.cvtColor(cv2.imread(str(frame)), cv2.COLOR_BGR2RGB)+ h, w = rgb.shape[:2]+ box = np.array([w * 0.3, h * 0.2, w * 0.6, h * 0.9])+ Rv = load_module(args.ref.parent / "val3dpw.py", "val3dpw_ref") if args.ref else None+ a, ca = val3dpw._preprocess(rgb, box, h, w, 1500.0)+ if Rv is not None:+ b, cb = Rv._preprocess(rgb, box, h, w, 1500.0)+ check(np.array_equal(a, b) and np.array_equal(ca, cb), "224 crop byte-identical to reference")+ else:+ print(" SKIP 224 crop vs reference (pass --ref <dir>/train_distill_mhr_only.py with val3dpw.py beside it)")+ check(a.shape == (3, 224, 224), f"default crop {a.shape}")+ a288, _ = val3dpw._preprocess(rgb, box, h, w, 1500.0, size=288)+ check(a288.shape == (3, 288, 288), f"size=288 crop {a288.shape}")++ print("[4] ONNX export and the inference package")+ onnx_path = tmp / "r288.onnx"+ r = subprocess.run([sys.executable, str(REPO / "tools/pth_to_onnx.py"), "--ckpt", str(ck),+ "--output", str(onnx_path)], capture_output=True, text=True)+ check(r.returncode == 0, f"pth_to_onnx exit {r.returncode}"+ + ("" if r.returncode == 0 else f": {r.stderr[-400:]}"))+ if onnx_path.exists():+ import onnx+ meta = {q.key: q.value for q in onnx.load(str(onnx_path)).metadata_props}+ check(meta.get("image_size") == "288", f"ONNX metadata image_size={meta.get('image_size')}")+ from instanthmr.inference import InstantHMR+ from instanthmr.mhr_renderer import build_mhr+ net = InstantHMR(onnx_path, device="cpu",+ mhr=build_mhr(script_path=str(REPO / "checkpoints/mhr_model.pt")))+ check(net.input_size == 288, f"InstantHMR input_size={net.input_size}")+ out = net.predict(rgb, box)+ j2d = np.asarray(out.joints_2d)+ check(np.isfinite(j2d).all() and j2d.shape[-1] == 2, f"predict on a real frame: joints_2d {j2d.shape}")++ print("[5] harnesses run a 288 checkpoint")+ for script, extra in (+ ("benchmark/eval_emdb_ckpt.py",+ ["--emdb-root", str(args.emdb_root), "--stride", "400",+ "--adapter", str(REPO / "benchmark/results/adapter_smpl24_teacher.npz")]),+ ("benchmark/eval_3dpw_ckpt.py",+ ["--sequence-dir", str(args.dpw_seq), "--image-root", str(args.dpw_img),+ "--split", "test", "--stride", "400"])):+ out_json = tmp / (Path(script).stem + ".json")+ r = subprocess.run([sys.executable, str(REPO / script), "--ckpt", str(ck),+ *extra, "--num-workers", "2", "--out", str(out_json)],+ capture_output=True, text=True, cwd=REPO)+ check(r.returncode == 0 and out_json.exists(),+ f"{Path(script).name}: exit {r.returncode}"+ + ("" if r.returncode == 0 else f": {r.stderr[-400:]}"))++ print(f"scratch: {tmp}")+ if fails:+ print(f"FAIL ({len(fails)})")+ sys.exit(1)+ print("PASS")+++if __name__ == "__main__":+ main()