Python
feat: new install with multi hardware support. Added detector strid to reduce rf-detr bottleneck.
mohamdeva4e21835d03d
diff --git a/README.md b/README.mdindex eebe5b9..d4334a7 100644--- a/README.md+++ b/README.md@@ -20,18 +20,15 @@ you always know where the latency is going.RTX 4070 with the fp16 ONNX (CUDA EP); CPU works too (~25 FPS, dependingon hardware). On Apple Silicon, pass `--device coreml` to use`CoreMLExecutionProvider`.-- **Speed (full demo, end-to-end):** the demo also runs **RF-DETR** for- person detection on every frame, and on most hardware the detector —- not InstantHMR — is the bottleneck. Expect roughly 30–60 FPS on a- recent NVIDIA GPU with `--detector-variant medium`, and notably less- on CPU / Apple Silicon. If you're CPU-bound or on a Mac, use- `--detector-variant nano` for a large speed-up at a small accuracy- cost.-- **Note for RTX 50-series (Blackwell, sm_120):** the CUDA wheels pinned- in `requirements.txt` (CUDA 12 / cuDNN 9) and a stock Torch install- may not ship sm_120 kernels and will fall back to slower paths,- particularly for RF-DETR. Install Torch built for CUDA 12.8+ and a- matching `onnxruntime-gpu` to get the expected throughput.+- **Speed (full demo, end-to-end):** the demo also runs **RF-DETR** every+ frame, and on most hardware the detector — not InstantHMR — is the+ bottleneck. Use `--detector-stride N` to run RF-DETR only every Nth+ frame and reuse the previous bbox in between (see *Performance tuning*+ below).+- **Note for RTX 50-series (Blackwell, sm_120):** stock Torch with+ CUDA 12.4 wheels does not ship sm_120 kernels and falls back to slow+ paths. Use `python install.py` which automatically pulls Torch+ cu128 (>=2.7) on Blackwell GPUs.> **Note on the body mesh.** InstantHMR does **not** run the MHR mesh> decoder — rendering the full body mesh is expensive and we want the@@ -50,9 +47,19 @@ cd instanthmrconda create -n instanthmr python=3.11 -yconda activate instanthmr-pip install -r requirements.txt+# Picks the right torch + onnxruntime wheels for your machine:+# - Linux + NVIDIA GPU → cu128 (Blackwell, RTX 50-series) or cu124 (Ada/Ampere/Hopper)+# + onnxruntime-gpu + bundled CUDA / cuDNN runtime libs+# - macOS → stock torch + onnxruntime (CoreML EP included)+# - Linux without GPU → CPU torch + onnxruntime+python install.py```+`python install.py --dry-run` prints the pip commands without running+them. `python install.py --force-cpu` skips GPU detection on Linux. If+you'd rather manage wheels yourself, `pip install -r requirements.txt`+still works — it gets the CPU torch + onnxruntime path.+The first run downloads the RF-DETR (medium) checkpoint into the rfdetrcache directory automatically. The InstantHMR ONNX weights are releasedseparately on HuggingFace — see [Model weights](#model-weights) below.@@ -77,6 +84,38 @@ The demo opens a Rerun viewer with the source image + 2D skeleton, a livedetector / HMR / total-latency plot, and the 3D scene with the predictedcamera frustum.+## Performance tuning++On every machine we've measured, RF-DETR (the detector) costs ~5–10× more+per frame than InstantHMR. The two flags below target that bottleneck.++| Flag | Effect |+| --- | --- |+| `--detector-stride N` | Run RF-DETR every Nth frame; reuse the previous bbox (slightly expanded) on the in-between frames. Stride 2–3 is the single biggest knob — typically 2–3× end-to-end FPS for slow movement, with negligible quality loss. |+| `--detector-variant nano` | Use the smallest RF-DETR; biggest win on CPU / Apple Silicon where the detector is doing all the work. |+| `--device coreml` | On macOS, route the InstantHMR ONNX through `CoreMLExecutionProvider`. |+| `--no-batch-persons` | Disable batched multi-person HMR (one ONNX call per person). The default is batched. |++Measured on RTX 4070 + torch 2.5 cu121 + ORT 1.25 (1080p video, 1 person, 150 frames after warm-up):++| `--detector-stride` | RF-DETR ms | HMR ms | total ms | FPS |+| ---: | ---: | ---: | ---: | ---: |+| 1 | 27.1 | 5.3 | 32.4 | **30.9** |+| 2 | 13.3 | 5.3 | 18.6 | **53.8** |+| 3 | 8.7 | 5.5 | 14.2 | **70.4** |+| 4 | 6.6 | 5.3 | 11.9 | **83.9** |++CPU path on the same box (Linux + ORT CPU EP): 7.9 FPS at stride 1 →+13.3 FPS with `--detector-variant nano --detector-stride 3`. Apple+Silicon and RTX 5070 numbers are not measured by us — please report+back.++You can re-run the benchmark on your own hardware with:++```bash+python tools/bench.py --video vid1.mp4 --max-frames 150 --detector-stride 3+```+## Use it from Python```pythondiff --git a/demo.py b/demo.pyindex 47dad9e..2021a7b 100755--- a/demo.py+++ b/demo.py@@ -74,6 +74,16 @@ def parse_args() -> argparse.Namespace:"--frame-skip", type=int, default=1,help="Process every Nth video frame (default: 1).",)+ p.add_argument(+ "--detector-stride", type=int, default=1,+ help="Run RF-DETR every Nth processed frame; reuse the previous bbox in "+ "between (default: 1). The detector is the dominant cost — stride "+ "2-3 typically doubles end-to-end FPS with negligible quality loss.",+ )+ p.add_argument(+ "--no-batch-persons", action="store_true",+ help="Disable batched multi-person HMR (one ONNX call per person).",+ )p.add_argument("--save-rrd", type=str, default=None,help="Optional path to save the Rerun recording as a .rrd file.",@@ -102,6 +112,8 @@ def build_pipeline(args: argparse.Namespace) -> PosePipeline:detector_variant=args.detector_variant,det_confidence=args.det_confidence,max_persons=args.max_persons,+ detector_stride=args.detector_stride,+ batch_persons=not args.no_batch_persons,)diff --git a/install.py b/install.pynew file mode 100644index 0000000..01f8e38--- /dev/null+++ b/install.py@@ -0,0 +1,159 @@+#!/usr/bin/env python3+"""Platform-aware installer for InstantHMR.++Picks the right Torch + ONNX Runtime wheels for the current machine:++ - macOS (Apple Silicon / Intel) — stock ``torch`` + ``onnxruntime``+ (which bundles the CoreML execution provider).+ - Linux + NVIDIA GPU — Torch built for the matching CUDA toolkit+ (cu128 for Blackwell sm_120+, cu124 for Ada / Ampere / Turing) plus+ ``onnxruntime-gpu`` and the ``nvidia-*-cu12`` runtime wheels so a+ system CUDA install isn't required.+ - Linux without an NVIDIA GPU — Torch CPU wheel + ``onnxruntime``.++Run *after* you've created and activated a fresh Python environment::++ python install.py++For a dry run that just prints the commands::++ python install.py --dry-run+"""++from __future__ import annotations++import argparse+import platform+import subprocess+import sys+from typing import Optional+++def run_pip(args: list[str], dry_run: bool = False) -> None:+ cmd = [sys.executable, "-m", "pip", "install"] + args+ print(">>", " ".join(cmd))+ if not dry_run:+ subprocess.check_call(cmd)+++def detect_cuda_compute_cap() -> Optional[float]:+ """Return the highest CUDA compute capability across visible GPUs.++ Returns ``None`` if ``nvidia-smi`` is missing, fails, or reports no+ GPU.+ """+ try:+ out = subprocess.check_output(+ ["nvidia-smi", "--query-gpu=compute_cap", "--format=csv,noheader"],+ text=True, stderr=subprocess.DEVNULL,+ )+ except (FileNotFoundError, subprocess.CalledProcessError):+ return None+ caps: list[float] = []+ for line in out.splitlines():+ line = line.strip()+ if not line:+ continue+ try:+ caps.append(float(line))+ except ValueError:+ pass+ return max(caps) if caps else None+++def install_mac(dry_run: bool) -> None:+ print("[install] platform: macOS — using stock torch + onnxruntime (CoreML EP)")+ run_pip(["torch>=2.2", "torchvision"], dry_run)+ run_pip(["onnxruntime>=1.19"], dry_run)+++def install_linux_cpu(dry_run: bool) -> None:+ print("[install] platform: Linux without NVIDIA GPU — using CPU wheels")+ run_pip([+ "--index-url", "https://download.pytorch.org/whl/cpu",+ "torch>=2.2", "torchvision",+ ], dry_run)+ run_pip(["onnxruntime>=1.19"], dry_run)+++def install_linux_cuda(compute_cap: float, dry_run: bool) -> None:+ print(f"[install] platform: Linux + NVIDIA (compute capability {compute_cap})")+ if compute_cap >= 12.0:+ # Blackwell (RTX 50-series, B100, B200): needs sm_120 kernels which+ # only land natively in the cu128 wheels (Torch 2.7+).+ print("[install] → Blackwell detected, installing torch cu128 (>=2.7)")+ run_pip([+ "--index-url", "https://download.pytorch.org/whl/cu128",+ "torch>=2.7", "torchvision",+ ], dry_run)+ elif compute_cap >= 8.0:+ # Ampere (sm_80/86), Ada (sm_89), Hopper (sm_90). cu124 wheels cover+ # all of these with native kernels.+ print("[install] → Ampere/Ada/Hopper, installing torch cu124 (>=2.4)")+ run_pip([+ "--index-url", "https://download.pytorch.org/whl/cu124",+ "torch>=2.4", "torchvision",+ ], dry_run)+ else:+ # Turing (sm_75) and older — cu121 still has stable wheels here.+ print("[install] → pre-Ampere, installing torch cu121 (>=2.2)")+ run_pip([+ "--index-url", "https://download.pytorch.org/whl/cu121",+ "torch>=2.2", "torchvision",+ ], dry_run)++ run_pip(["onnxruntime-gpu>=1.19"], dry_run)+ # Bundled CUDA runtime libs so users without a system CUDA toolkit+ # can still load CUDA EP. ORT's preload_dlls() picks them up.+ run_pip([+ "nvidia-cudnn-cu12>=9.0,<10",+ "nvidia-cuda-runtime-cu12",+ "nvidia-cublas-cu12",+ ], dry_run)+++def install_common(dry_run: bool) -> None:+ print("[install] common deps")+ run_pip([+ "numpy>=1.24",+ "opencv-python>=4.8",+ "pillow>=10",+ "rfdetr>=1.0",+ "rerun-sdk>=0.21",+ ], dry_run)+++def main() -> None:+ parser = argparse.ArgumentParser(description=__doc__,+ formatter_class=argparse.RawDescriptionHelpFormatter)+ parser.add_argument("--dry-run", action="store_true",+ help="Print pip commands but don't run them.")+ parser.add_argument("--force-cpu", action="store_true",+ help="Force the CPU-only path on Linux even if an NVIDIA GPU is detected.")+ args = parser.parse_args()++ sysname = platform.system()+ machine = platform.machine()+ print(f"[install] python={platform.python_version()} system={sysname} machine={machine}")++ if sysname == "Darwin":+ install_mac(args.dry_run)+ elif sysname == "Linux":+ if args.force_cpu:+ install_linux_cpu(args.dry_run)+ else:+ cap = detect_cuda_compute_cap()+ if cap is None:+ install_linux_cpu(args.dry_run)+ else:+ install_linux_cuda(cap, args.dry_run)+ else:+ sys.exit(f"[install] unsupported platform: {sysname}. "+ f"Install torch + onnxruntime manually, then `pip install -r requirements.txt`.")++ install_common(args.dry_run)+ print("\n[install] done. verify with: python -c 'from instanthmr import PosePipeline; print(\"ok\")'")+++if __name__ == "__main__":+ main()diff --git a/instanthmr/inference.py b/instanthmr/inference.pyindex 8459e7d..5bbd62e 100644--- a/instanthmr/inference.py+++ b/instanthmr/inference.py@@ -103,7 +103,15 @@ class InstantHMR:if providers is None:providers = self._default_providers(device)- self.session = ort.InferenceSession(str(onnx_path), providers=providers)+ sess_options = ort.SessionOptions()+ sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL+ sess_options.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL+ # 0 == let ORT pick. We don't override threading on CUDA (compute+ # is on-device); on CPU/CoreML, ORT's defaults are saner than ours.++ self.session = ort.InferenceSession(+ str(onnx_path), sess_options=sess_options, providers=providers,+ )self.active_provider = self.session.get_providers()[0]in_names = [i.name for i in self.session.get_inputs()]@@ -184,6 +192,82 @@ class InstantHMR:shape_params=shape_params,)+ def predict_batch(+ self,+ image_rgb: np.ndarray,+ detections: list[dict],+ ) -> list[HMRPrediction]:+ """Run InstantHMR on multiple persons in a single ONNX call.++ Args:+ image_rgb: (H, W, 3) uint8 RGB full-frame image.+ detections: list of ``{"bbox": [x1,y1,x2,y2], "confidence": f}``.++ Returns:+ One ``HMRPrediction`` per input detection, in the same order.+ """+ if not detections:+ return []++ image_rgb = np.ascontiguousarray(image_rgb)+ h, w = image_rgb.shape[:2]+ n = len(detections)++ crops = np.empty((n, 3, INPUT_SIZE, INPUT_SIZE), dtype=np.float32)+ cliffs = np.empty((n, 3), dtype=np.float32)+ sq_meta = [] # per-person (sq_x1, sq_y1, sq_size, bbox)+ for i, det in enumerate(detections):+ bbox_arr = np.asarray(det["bbox"], dtype=np.float32).reshape(4)+ crop, sq_x1, sq_y1, sq_size, cliff = self._preprocess(+ image_rgb, bbox_arr, h, w,+ )+ crops[i] = crop+ cliffs[i] = cliff+ sq_meta.append((sq_x1, sq_y1, sq_size, bbox_arr))++ outs = self.session.run(+ None,+ {self._in_image: crops, self._in_cliff: cliffs},+ )+ mhr_params_b = outs[0].astype(np.float32, copy=False)+ shape_params_b = outs[1].astype(np.float32, copy=False)+ cam_trans_b = outs[2].astype(np.float32, copy=False)+ joints_2d_norm_b = outs[3].astype(np.float32, copy=False)+ joints_3d_local_b = outs[4].astype(np.float32, copy=False)++ f = math.sqrt(h * h + w * w)+ focal_length = np.array([f, f], dtype=np.float32)+ principal_point = np.array([w / 2.0, h / 2.0], dtype=np.float32)++ results: list[HMRPrediction] = []+ for i, det in enumerate(detections):+ sq_x1, sq_y1, sq_size, bbox_arr = sq_meta[i]+ joints_2d_norm = joints_2d_norm_b[i]+ joints_3d_local = joints_3d_local_b[i]+ cam_trans = cam_trans_b[i]++ crop_px = (joints_2d_norm + 1.0) * 0.5 * INPUT_SIZE+ scale = sq_size / INPUT_SIZE+ joints_2d = np.stack(+ [crop_px[:, 0] * scale + sq_x1, crop_px[:, 1] * scale + sq_y1],+ axis=-1,+ ).astype(np.float32)++ results.append(HMRPrediction(+ bbox=bbox_arr,+ confidence=float(det.get("confidence", 1.0)),+ joints_3d_local=joints_3d_local,+ joints_3d_cam=joints_3d_local + cam_trans,+ joints_2d=joints_2d,+ cam_trans=cam_trans,+ focal_length=focal_length,+ principal_point=principal_point,+ image_shape=(h, w),+ mhr_params=mhr_params_b[i],+ shape_params=shape_params_b[i],+ ))+ return results+# ------------------------------------------------------------------# Preprocessing# ------------------------------------------------------------------diff --git a/instanthmr/pipeline.py b/instanthmr/pipeline.pyindex 8da5eb5..a4c23a7 100644--- a/instanthmr/pipeline.py+++ b/instanthmr/pipeline.py@@ -24,10 +24,12 @@ class FrameResult:Attributes:persons: Per-person ``HMRPrediction``s (possibly empty).- detector_ms: Wall-clock time for the detector forward pass.+ detector_ms: Wall-clock time for the detector forward pass (0 on+ frames where the detector was skipped via ``detector_stride``).hmr_ms: Wall-clock time for InstantHMR across all persons in the- frame (sum, not average).- total_ms: ``detector_ms + hmr_ms`` plus any pipeline overhead.+ frame (sum, not average — or one batched call when+ ``batch_persons=True``).+ total_ms: ``detector_ms + hmr_ms`` plus pipeline overhead."""persons: list[HMRPrediction]@@ -36,21 +38,41 @@ class FrameResult:total_ms: float+def _expand_bbox(bbox: np.ndarray, w: int, h: int, expand: float) -> np.ndarray:+ bw = bbox[2] - bbox[0]+ bh = bbox[3] - bbox[1]+ return np.array([+ max(0.0, bbox[0] - bw * expand),+ max(0.0, bbox[1] - bh * expand),+ min(float(w), bbox[2] + bw * expand),+ min(float(h), bbox[3] + bh * expand),+ ], dtype=np.float32)++class PosePipeline:"""Full RF-DETR → InstantHMR pipeline.Args:onnx_path: Path to ``instanthmr.onnx``.- device: ``"cuda"`` or ``"cpu"`` for InstantHMR (RF-DETR auto-selects).+ device: ``"cuda"``, ``"coreml"`` or ``"cpu"`` for InstantHMR.detector_variant: ``"nano" | "small" | "medium" | "base" | "large"``.det_confidence: RF-DETR confidence threshold.max_persons: Maximum number of persons returned per frame.+ detector_stride: Run the detector every ``N`` frames; on the+ in-between frames, reuse the previous frame's detections+ (slightly expanded) to drive InstantHMR. ``1`` reproduces the+ original "detect every frame" behaviour. The detector is by+ far the dominant cost on most hardware, so stride 2–3 is the+ single biggest knob for end-to-end FPS.+ batch_persons: When ``True`` (default), all detected persons in a+ frame are batched into a single InstantHMR ONNX call instead+ of looping. This is a meaningful win for multi-person frames+ and free for single-person frames.Example-------- pipeline = PosePipeline("models/instanthmr.onnx")+ pipeline = PosePipeline("models/instanthmr.onnx", detector_stride=2)result = pipeline.predict(image_rgb)- print(f"det {result.detector_ms:.1f} ms / hmr {result.hmr_ms:.1f} ms")for p in result.persons:print(p.joints_3d_cam.shape) # (70, 3)"""@@ -62,54 +84,72 @@ class PosePipeline:detector_variant: str = "medium",det_confidence: float = 0.5,max_persons: int = 5,+ detector_stride: int = 1,+ batch_persons: bool = True,):+ if detector_stride < 1:+ raise ValueError("detector_stride must be >= 1")+self.hmr = InstantHMR(onnx_path, device=device)self.detector = RFDETRDetector(variant=detector_variant,confidence=det_confidence,max_persons=max_persons,)++ self._detector_stride = detector_stride+ self._batch_persons = batch_persons+ self._frame_idx = 0+ self._last_detections: list[dict] = []self._last_bbox: Optional[np.ndarray] = Nonedef predict(self, image_rgb: np.ndarray) -> FrameResult:- """Detect all persons in *image_rgb* and run InstantHMR on each.-- Returns a :class:`FrameResult` with per-stage timings so callers- can display detector vs HMR cost separately.- """+ """Detect all persons in *image_rgb* and run InstantHMR on each."""t_total_start = time.perf_counter()-- # ---- Detector ----- t0 = time.perf_counter()- detections = self.detector.detect(image_rgb)- detector_ms = (time.perf_counter() - t0) * 1000.0-- # Single-frame fallback: reuse the last bbox if the detector drops- # a frame. Useful for live camera mode where a transient miss is- # common.- if not detections and self._last_bbox is not None:- h, w = image_rgb.shape[:2]- lb = self._last_bbox- bw, bh = lb[2] - lb[0], lb[3] - lb[1]- expand = 0.1- fb = np.array([- max(0, lb[0] - bw * expand),- max(0, lb[1] - bh * expand),- min(w, lb[2] + bw * expand),- min(h, lb[3] + bh * expand),- ], dtype=np.float32)+ h, w = image_rgb.shape[:2]++ # ---- Detector (with optional stride) ----+ run_detector = (self._frame_idx % self._detector_stride) == 0+ if run_detector:+ t0 = time.perf_counter()+ detections = self.detector.detect(image_rgb)+ detector_ms = (time.perf_counter() - t0) * 1000.0+ if detections:+ self._last_detections = detections+ else:+ detections = []+ detector_ms = 0.0++ # If the detector skipped or missed this frame, reuse the previous+ # detections (slightly expanded to absorb small motion). This is a+ # cheap form of tracking that keeps HMR fed with a plausible bbox.+ if not detections and self._last_detections:+ detections = [+ {+ "bbox": _expand_bbox(d["bbox"], w, h, expand=0.1),+ "confidence": float(d["confidence"]) * 0.9,+ }+ for d in self._last_detections+ ]+ elif not detections and self._last_bbox is not None:+ # Backwards-compatible single-bbox fallback (used when no prior+ # multi-person detections exist).+ fb = _expand_bbox(self._last_bbox, w, h, expand=0.1)detections = [{"bbox": fb, "confidence": 0.0}]- # ---- InstantHMR (sum across all persons) ----+ # ---- InstantHMR ----outputs: list[HMRPrediction] = []t1 = time.perf_counter()- for det in detections:- out = self.hmr.predict(- image_rgb,- bbox=det["bbox"],- confidence=det["confidence"],- )- outputs.append(out)+ if detections:+ if self._batch_persons and len(detections) > 1:+ outputs = self.hmr.predict_batch(image_rgb, detections)+ else:+ for det in detections:+ outputs.append(self.hmr.predict(+ image_rgb,+ bbox=det["bbox"],+ confidence=det["confidence"],+ ))hmr_ms = (time.perf_counter() - t1) * 1000.0if outputs:@@ -117,6 +157,7 @@ class PosePipeline:self._last_bbox = best.bbox.copy()total_ms = (time.perf_counter() - t_total_start) * 1000.0+ self._frame_idx += 1return FrameResult(persons=outputs,diff --git a/requirements.txt b/requirements.txtindex 5bb1471..3c3f2ba 100644--- a/requirements.txt+++ b/requirements.txt@@ -1,23 +1,27 @@+# Recommended install path is `python install.py`, which picks the right+# torch + onnxruntime wheels for your hardware (Linux + CUDA, Mac, or CPU).+# This file is a portable fallback that uses the CPU wheel of torch and the+# stock CPU build of onnxruntime — it works everywhere but leaves a lot of+# performance on the table on NVIDIA GPUs and Apple Silicon.+# ----- inference + viz (required by demo.py) -----numpy>=1.24opencv-python>=4.8pillow>=10-onnxruntime-gpu>=1.19 ; platform_system != "Darwin"-onnxruntime>=1.19 ; platform_system == "Darwin"rfdetr>=1.0rerun-sdk>=0.21-# ----- CUDA runtime libs for onnxruntime-gpu ------# onnxruntime-gpu wheels are built against CUDA 12 + cuDNN 9 but do NOT-# bundle them. These pip packages provide the .so files so you don't need-# a system CUDA install. Instaharm calls onnxruntime.preload_dlls() at-# session creation, which finds them automatically (ORT >= 1.19).-nvidia-cudnn-cu12>=9.0,<10 ; platform_system != "Darwin"-nvidia-cuda-runtime-cu12 ; platform_system != "Darwin"-nvidia-cublas-cu12 ; platform_system != "Darwin"+# Pulled in transitively by rfdetr; on Linux + CUDA you almost certainly+# want a CUDA wheel from the pytorch index instead — see install.py.+torch>=2.2+torchvision++# Mac ships CoreML EP inside plain onnxruntime; Linux GPU wants+# onnxruntime-gpu (which install.py picks). The CPU wheel below is the+# safe fallback.+onnxruntime>=1.19# ----- training / annotation (optional, for reproduction) ------# torch>=2.2# timm>=1.0# tqdm# logurudiff --git a/tools/bench.py b/tools/bench.pynew file mode 100644index 0000000..115ee0b--- /dev/null+++ b/tools/bench.py@@ -0,0 +1,138 @@+#!/usr/bin/env python3+"""Headless benchmark for the InstantHMR pipeline.++Runs ``PosePipeline`` on a video and prints per-stage latency stats with no+Rerun, no GUI, no .rrd writing — so the numbers reflect inference cost and+not visualisation overhead.++Usage:+ python tools/bench.py --video vid1.mp4 [--device cuda] [--detector-variant medium]+ [--max-frames 200] [--warmup 10]+"""++from __future__ import annotations++import argparse+import statistics+import sys+import time+from pathlib import Path++import cv2+import numpy as np++# Make the package importable when run from anywhere.+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))++from instanthmr import PosePipeline+++def parse_args() -> argparse.Namespace:+ p = argparse.ArgumentParser()+ p.add_argument("--video", type=str, required=True)+ p.add_argument("--model", type=str, default="models/instanthmr.onnx")+ p.add_argument("--device", type=str, default="cuda")+ p.add_argument("--detector-variant", type=str, default="medium",+ choices=["nano", "small", "medium", "base", "large"])+ p.add_argument("--det-confidence", type=float, default=0.5)+ p.add_argument("--max-persons", type=int, default=5)+ p.add_argument("--detector-stride", type=int, default=1,+ help="Run RF-DETR every N frames (default 1 = every frame).")+ p.add_argument("--no-batch-persons", action="store_true",+ help="Disable batched multi-person HMR (one ORT call per person).")+ p.add_argument("--max-frames", type=int, default=200,+ help="Cap measured frames (after warm-up).")+ p.add_argument("--warmup", type=int, default=10,+ help="Frames discarded before timing starts.")+ return p.parse_args()+++def _summary(name: str, samples: list[float]) -> str:+ if not samples:+ return f"{name}: no samples"+ arr = np.asarray(samples, dtype=np.float64)+ return (+ f"{name:>14s} "+ f"mean={arr.mean():6.2f} ms "+ f"median={np.median(arr):6.2f} ms "+ f"p95={np.percentile(arr, 95):6.2f} ms "+ f"min={arr.min():6.2f} ms "+ f"max={arr.max():6.2f} ms"+ )+++def main() -> None:+ args = parse_args()++ cap = cv2.VideoCapture(args.video)+ if not cap.isOpened():+ sys.exit(f"[error] cannot open video: {args.video}")++ src_fps = cap.get(cv2.CAP_PROP_FPS) or 30.0+ total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))+ width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))+ height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))+ print(f"video: {args.video} {width}x{height} {total} frames @ {src_fps:.1f} fps")++ pipeline = PosePipeline(+ onnx_path=args.model,+ device=args.device,+ detector_variant=args.detector_variant,+ det_confidence=args.det_confidence,+ max_persons=args.max_persons,+ detector_stride=args.detector_stride,+ batch_persons=not args.no_batch_persons,+ )+ print(f"InstantHMR EP: {pipeline.hmr.active_provider}")+ print(f"detector variant: {args.detector_variant}")++ det: list[float] = []+ hmr: list[float] = []+ tot: list[float] = []+ wall: list[float] = []+ n_persons: list[int] = []++ idx = 0+ measured = 0+ last_wall = None+ while True:+ ret, bgr = cap.read()+ if not ret:+ break++ rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)+ t_wall = time.perf_counter()+ result = pipeline.predict(rgb)++ if idx >= args.warmup:+ det.append(result.detector_ms)+ hmr.append(result.hmr_ms)+ tot.append(result.total_ms)+ n_persons.append(len(result.persons))+ if last_wall is not None:+ wall.append((t_wall - last_wall) * 1000.0)+ measured += 1+ last_wall = t_wall+ idx += 1+ if measured >= args.max_frames:+ break++ cap.release()++ if not tot:+ sys.exit("[error] no measured frames — increase --max-frames or shorten --warmup")++ print()+ print(f"measured frames: {len(tot)} (warmup={args.warmup})")+ print(f"avg persons / frame: {statistics.mean(n_persons):.2f}")+ print(_summary("detector_ms", det))+ print(_summary("hmr_ms", hmr))+ print(_summary("total_ms", tot))+ if wall:+ print(_summary("wall_ms", wall))+ mean_total = float(np.mean(tot))+ print(f"\nthroughput (1 / mean total): {1000.0 / mean_total:5.1f} FPS")+++if __name__ == "__main__":+ main()