Skip to content
Python

feat: new install with multi hardware support. Added detector strid to reduce rf-detr bottleneck.

mohamdeva4e21835d03d
diff --git a/README.md b/README.md
index eebe5b9..d4334a7 100644
--- a/README.md
+++ b/README.md
@@ -20,18 +20,15 @@ you always know where the latency is going.
RTX 4070 with the fp16 ONNX (CUDA EP); CPU works too (~25 FPS, depending
on hardware). On Apple Silicon, pass `--device coreml` to use
`CoreMLExecutionProvider`.
-- **Speed (full demo, end-to-end):** the demo also runs **RF-DETR** for
- person detection on every frame, and on most hardware the detector —
- not InstantHMR — is the bottleneck. Expect roughly 30–60 FPS on a
- recent NVIDIA GPU with `--detector-variant medium`, and notably less
- on CPU / Apple Silicon. If you're CPU-bound or on a Mac, use
- `--detector-variant nano` for a large speed-up at a small accuracy
- cost.
-- **Note for RTX 50-series (Blackwell, sm_120):** the CUDA wheels pinned
- in `requirements.txt` (CUDA 12 / cuDNN 9) and a stock Torch install
- may not ship sm_120 kernels and will fall back to slower paths,
- particularly for RF-DETR. Install Torch built for CUDA 12.8+ and a
- matching `onnxruntime-gpu` to get the expected throughput.
+- **Speed (full demo, end-to-end):** the demo also runs **RF-DETR** every
+ frame, and on most hardware the detector — not InstantHMR — is the
+ bottleneck. Use `--detector-stride N` to run RF-DETR only every Nth
+ frame and reuse the previous bbox in between (see *Performance tuning*
+ below).
+- **Note for RTX 50-series (Blackwell, sm_120):** stock Torch with
+ CUDA 12.4 wheels does not ship sm_120 kernels and falls back to slow
+ paths. Use `python install.py` which automatically pulls Torch
+ cu128 (>=2.7) on Blackwell GPUs.
> **Note on the body mesh.** InstantHMR does **not** run the MHR mesh
> decoder — rendering the full body mesh is expensive and we want the
@@ -50,9 +47,19 @@ cd instanthmr
conda create -n instanthmr python=3.11 -y
conda activate instanthmr
-pip install -r requirements.txt
+# Picks the right torch + onnxruntime wheels for your machine:
+# - Linux + NVIDIA GPU → cu128 (Blackwell, RTX 50-series) or cu124 (Ada/Ampere/Hopper)
+# + onnxruntime-gpu + bundled CUDA / cuDNN runtime libs
+# - macOS → stock torch + onnxruntime (CoreML EP included)
+# - Linux without GPU → CPU torch + onnxruntime
+python install.py
```
+`python install.py --dry-run` prints the pip commands without running
+them. `python install.py --force-cpu` skips GPU detection on Linux. If
+you'd rather manage wheels yourself, `pip install -r requirements.txt`
+still works — it gets the CPU torch + onnxruntime path.
+
The first run downloads the RF-DETR (medium) checkpoint into the rfdetr
cache directory automatically. The InstantHMR ONNX weights are released
separately on HuggingFace — see [Model weights](#model-weights) below.
@@ -77,6 +84,38 @@ The demo opens a Rerun viewer with the source image + 2D skeleton, a live
detector / HMR / total-latency plot, and the 3D scene with the predicted
camera frustum.
+## Performance tuning
+
+On every machine we've measured, RF-DETR (the detector) costs ~5–10× more
+per frame than InstantHMR. The two flags below target that bottleneck.
+
+| Flag | Effect |
+| --- | --- |
+| `--detector-stride N` | Run RF-DETR every Nth frame; reuse the previous bbox (slightly expanded) on the in-between frames. Stride 2–3 is the single biggest knob — typically 2–3× end-to-end FPS for slow movement, with negligible quality loss. |
+| `--detector-variant nano` | Use the smallest RF-DETR; biggest win on CPU / Apple Silicon where the detector is doing all the work. |
+| `--device coreml` | On macOS, route the InstantHMR ONNX through `CoreMLExecutionProvider`. |
+| `--no-batch-persons` | Disable batched multi-person HMR (one ONNX call per person). The default is batched. |
+
+Measured on RTX 4070 + torch 2.5 cu121 + ORT 1.25 (1080p video, 1 person, 150 frames after warm-up):
+
+| `--detector-stride` | RF-DETR ms | HMR ms | total ms | FPS |
+| ---: | ---: | ---: | ---: | ---: |
+| 1 | 27.1 | 5.3 | 32.4 | **30.9** |
+| 2 | 13.3 | 5.3 | 18.6 | **53.8** |
+| 3 | 8.7 | 5.5 | 14.2 | **70.4** |
+| 4 | 6.6 | 5.3 | 11.9 | **83.9** |
+
+CPU path on the same box (Linux + ORT CPU EP): 7.9 FPS at stride 1 →
+13.3 FPS with `--detector-variant nano --detector-stride 3`. Apple
+Silicon and RTX 5070 numbers are not measured by us — please report
+back.
+
+You can re-run the benchmark on your own hardware with:
+
+```bash
+python tools/bench.py --video vid1.mp4 --max-frames 150 --detector-stride 3
+```
+
## Use it from Python
```python
diff --git a/demo.py b/demo.py
index 47dad9e..2021a7b 100755
--- a/demo.py
+++ b/demo.py
@@ -74,6 +74,16 @@ def parse_args() -> argparse.Namespace:
"--frame-skip", type=int, default=1,
help="Process every Nth video frame (default: 1).",
)
+ p.add_argument(
+ "--detector-stride", type=int, default=1,
+ help="Run RF-DETR every Nth processed frame; reuse the previous bbox in "
+ "between (default: 1). The detector is the dominant cost — stride "
+ "2-3 typically doubles end-to-end FPS with negligible quality loss.",
+ )
+ p.add_argument(
+ "--no-batch-persons", action="store_true",
+ help="Disable batched multi-person HMR (one ONNX call per person).",
+ )
p.add_argument(
"--save-rrd", type=str, default=None,
help="Optional path to save the Rerun recording as a .rrd file.",
@@ -102,6 +112,8 @@ def build_pipeline(args: argparse.Namespace) -> PosePipeline:
detector_variant=args.detector_variant,
det_confidence=args.det_confidence,
max_persons=args.max_persons,
+ detector_stride=args.detector_stride,
+ batch_persons=not args.no_batch_persons,
)
diff --git a/install.py b/install.py
new file mode 100644
index 0000000..01f8e38
--- /dev/null
+++ b/install.py
@@ -0,0 +1,159 @@
+#!/usr/bin/env python3
+"""Platform-aware installer for InstantHMR.
+
+Picks the right Torch + ONNX Runtime wheels for the current machine:
+
+ - macOS (Apple Silicon / Intel) — stock ``torch`` + ``onnxruntime``
+ (which bundles the CoreML execution provider).
+ - Linux + NVIDIA GPU — Torch built for the matching CUDA toolkit
+ (cu128 for Blackwell sm_120+, cu124 for Ada / Ampere / Turing) plus
+ ``onnxruntime-gpu`` and the ``nvidia-*-cu12`` runtime wheels so a
+ system CUDA install isn't required.
+ - Linux without an NVIDIA GPU — Torch CPU wheel + ``onnxruntime``.
+
+Run *after* you've created and activated a fresh Python environment::
+
+ python install.py
+
+For a dry run that just prints the commands::
+
+ python install.py --dry-run
+"""
+
+from __future__ import annotations
+
+import argparse
+import platform
+import subprocess
+import sys
+from typing import Optional
+
+
+def run_pip(args: list[str], dry_run: bool = False) -> None:
+ cmd = [sys.executable, "-m", "pip", "install"] + args
+ print(">>", " ".join(cmd))
+ if not dry_run:
+ subprocess.check_call(cmd)
+
+
+def detect_cuda_compute_cap() -> Optional[float]:
+ """Return the highest CUDA compute capability across visible GPUs.
+
+ Returns ``None`` if ``nvidia-smi`` is missing, fails, or reports no
+ GPU.
+ """
+ try:
+ out = subprocess.check_output(
+ ["nvidia-smi", "--query-gpu=compute_cap", "--format=csv,noheader"],
+ text=True, stderr=subprocess.DEVNULL,
+ )
+ except (FileNotFoundError, subprocess.CalledProcessError):
+ return None
+ caps: list[float] = []
+ for line in out.splitlines():
+ line = line.strip()
+ if not line:
+ continue
+ try:
+ caps.append(float(line))
+ except ValueError:
+ pass
+ return max(caps) if caps else None
+
+
+def install_mac(dry_run: bool) -> None:
+ print("[install] platform: macOS — using stock torch + onnxruntime (CoreML EP)")
+ run_pip(["torch>=2.2", "torchvision"], dry_run)
+ run_pip(["onnxruntime>=1.19"], dry_run)
+
+
+def install_linux_cpu(dry_run: bool) -> None:
+ print("[install] platform: Linux without NVIDIA GPU — using CPU wheels")
+ run_pip([
+ "--index-url", "https://download.pytorch.org/whl/cpu",
+ "torch>=2.2", "torchvision",
+ ], dry_run)
+ run_pip(["onnxruntime>=1.19"], dry_run)
+
+
+def install_linux_cuda(compute_cap: float, dry_run: bool) -> None:
+ print(f"[install] platform: Linux + NVIDIA (compute capability {compute_cap})")
+ if compute_cap >= 12.0:
+ # Blackwell (RTX 50-series, B100, B200): needs sm_120 kernels which
+ # only land natively in the cu128 wheels (Torch 2.7+).
+ print("[install] → Blackwell detected, installing torch cu128 (>=2.7)")
+ run_pip([
+ "--index-url", "https://download.pytorch.org/whl/cu128",
+ "torch>=2.7", "torchvision",
+ ], dry_run)
+ elif compute_cap >= 8.0:
+ # Ampere (sm_80/86), Ada (sm_89), Hopper (sm_90). cu124 wheels cover
+ # all of these with native kernels.
+ print("[install] → Ampere/Ada/Hopper, installing torch cu124 (>=2.4)")
+ run_pip([
+ "--index-url", "https://download.pytorch.org/whl/cu124",
+ "torch>=2.4", "torchvision",
+ ], dry_run)
+ else:
+ # Turing (sm_75) and older — cu121 still has stable wheels here.
+ print("[install] → pre-Ampere, installing torch cu121 (>=2.2)")
+ run_pip([
+ "--index-url", "https://download.pytorch.org/whl/cu121",
+ "torch>=2.2", "torchvision",
+ ], dry_run)
+
+ run_pip(["onnxruntime-gpu>=1.19"], dry_run)
+ # Bundled CUDA runtime libs so users without a system CUDA toolkit
+ # can still load CUDA EP. ORT's preload_dlls() picks them up.
+ run_pip([
+ "nvidia-cudnn-cu12>=9.0,<10",
+ "nvidia-cuda-runtime-cu12",
+ "nvidia-cublas-cu12",
+ ], dry_run)
+
+
+def install_common(dry_run: bool) -> None:
+ print("[install] common deps")
+ run_pip([
+ "numpy>=1.24",
+ "opencv-python>=4.8",
+ "pillow>=10",
+ "rfdetr>=1.0",
+ "rerun-sdk>=0.21",
+ ], dry_run)
+
+
+def main() -> None:
+ parser = argparse.ArgumentParser(description=__doc__,
+ formatter_class=argparse.RawDescriptionHelpFormatter)
+ parser.add_argument("--dry-run", action="store_true",
+ help="Print pip commands but don't run them.")
+ parser.add_argument("--force-cpu", action="store_true",
+ help="Force the CPU-only path on Linux even if an NVIDIA GPU is detected.")
+ args = parser.parse_args()
+
+ sysname = platform.system()
+ machine = platform.machine()
+ print(f"[install] python={platform.python_version()} system={sysname} machine={machine}")
+
+ if sysname == "Darwin":
+ install_mac(args.dry_run)
+ elif sysname == "Linux":
+ if args.force_cpu:
+ install_linux_cpu(args.dry_run)
+ else:
+ cap = detect_cuda_compute_cap()
+ if cap is None:
+ install_linux_cpu(args.dry_run)
+ else:
+ install_linux_cuda(cap, args.dry_run)
+ else:
+ sys.exit(f"[install] unsupported platform: {sysname}. "
+ f"Install torch + onnxruntime manually, then `pip install -r requirements.txt`.")
+
+ install_common(args.dry_run)
+ print("\n[install] done. verify with: python -c 'from instanthmr import PosePipeline; print(\"ok\")'")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/instanthmr/inference.py b/instanthmr/inference.py
index 8459e7d..5bbd62e 100644
--- a/instanthmr/inference.py
+++ b/instanthmr/inference.py
@@ -103,7 +103,15 @@ class InstantHMR:
if providers is None:
providers = self._default_providers(device)
- self.session = ort.InferenceSession(str(onnx_path), providers=providers)
+ sess_options = ort.SessionOptions()
+ sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+ sess_options.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL
+ # 0 == let ORT pick. We don't override threading on CUDA (compute
+ # is on-device); on CPU/CoreML, ORT's defaults are saner than ours.
+
+ self.session = ort.InferenceSession(
+ str(onnx_path), sess_options=sess_options, providers=providers,
+ )
self.active_provider = self.session.get_providers()[0]
in_names = [i.name for i in self.session.get_inputs()]
@@ -184,6 +192,82 @@ class InstantHMR:
shape_params=shape_params,
)
+ def predict_batch(
+ self,
+ image_rgb: np.ndarray,
+ detections: list[dict],
+ ) -> list[HMRPrediction]:
+ """Run InstantHMR on multiple persons in a single ONNX call.
+
+ Args:
+ image_rgb: (H, W, 3) uint8 RGB full-frame image.
+ detections: list of ``{"bbox": [x1,y1,x2,y2], "confidence": f}``.
+
+ Returns:
+ One ``HMRPrediction`` per input detection, in the same order.
+ """
+ if not detections:
+ return []
+
+ image_rgb = np.ascontiguousarray(image_rgb)
+ h, w = image_rgb.shape[:2]
+ n = len(detections)
+
+ crops = np.empty((n, 3, INPUT_SIZE, INPUT_SIZE), dtype=np.float32)
+ cliffs = np.empty((n, 3), dtype=np.float32)
+ sq_meta = [] # per-person (sq_x1, sq_y1, sq_size, bbox)
+ for i, det in enumerate(detections):
+ bbox_arr = np.asarray(det["bbox"], dtype=np.float32).reshape(4)
+ crop, sq_x1, sq_y1, sq_size, cliff = self._preprocess(
+ image_rgb, bbox_arr, h, w,
+ )
+ crops[i] = crop
+ cliffs[i] = cliff
+ sq_meta.append((sq_x1, sq_y1, sq_size, bbox_arr))
+
+ outs = self.session.run(
+ None,
+ {self._in_image: crops, self._in_cliff: cliffs},
+ )
+ mhr_params_b = outs[0].astype(np.float32, copy=False)
+ shape_params_b = outs[1].astype(np.float32, copy=False)
+ cam_trans_b = outs[2].astype(np.float32, copy=False)
+ joints_2d_norm_b = outs[3].astype(np.float32, copy=False)
+ joints_3d_local_b = outs[4].astype(np.float32, copy=False)
+
+ f = math.sqrt(h * h + w * w)
+ focal_length = np.array([f, f], dtype=np.float32)
+ principal_point = np.array([w / 2.0, h / 2.0], dtype=np.float32)
+
+ results: list[HMRPrediction] = []
+ for i, det in enumerate(detections):
+ sq_x1, sq_y1, sq_size, bbox_arr = sq_meta[i]
+ joints_2d_norm = joints_2d_norm_b[i]
+ joints_3d_local = joints_3d_local_b[i]
+ cam_trans = cam_trans_b[i]
+
+ crop_px = (joints_2d_norm + 1.0) * 0.5 * INPUT_SIZE
+ scale = sq_size / INPUT_SIZE
+ joints_2d = np.stack(
+ [crop_px[:, 0] * scale + sq_x1, crop_px[:, 1] * scale + sq_y1],
+ axis=-1,
+ ).astype(np.float32)
+
+ results.append(HMRPrediction(
+ bbox=bbox_arr,
+ confidence=float(det.get("confidence", 1.0)),
+ joints_3d_local=joints_3d_local,
+ joints_3d_cam=joints_3d_local + cam_trans,
+ joints_2d=joints_2d,
+ cam_trans=cam_trans,
+ focal_length=focal_length,
+ principal_point=principal_point,
+ image_shape=(h, w),
+ mhr_params=mhr_params_b[i],
+ shape_params=shape_params_b[i],
+ ))
+ return results
+
# ------------------------------------------------------------------
# Preprocessing
# ------------------------------------------------------------------
diff --git a/instanthmr/pipeline.py b/instanthmr/pipeline.py
index 8da5eb5..a4c23a7 100644
--- a/instanthmr/pipeline.py
+++ b/instanthmr/pipeline.py
@@ -24,10 +24,12 @@ class FrameResult:
Attributes:
persons: Per-person ``HMRPrediction``s (possibly empty).
- detector_ms: Wall-clock time for the detector forward pass.
+ detector_ms: Wall-clock time for the detector forward pass (0 on
+ frames where the detector was skipped via ``detector_stride``).
hmr_ms: Wall-clock time for InstantHMR across all persons in the
- frame (sum, not average).
- total_ms: ``detector_ms + hmr_ms`` plus any pipeline overhead.
+ frame (sum, not average — or one batched call when
+ ``batch_persons=True``).
+ total_ms: ``detector_ms + hmr_ms`` plus pipeline overhead.
"""
persons: list[HMRPrediction]
@@ -36,21 +38,41 @@ class FrameResult:
total_ms: float
+def _expand_bbox(bbox: np.ndarray, w: int, h: int, expand: float) -> np.ndarray:
+ bw = bbox[2] - bbox[0]
+ bh = bbox[3] - bbox[1]
+ return np.array([
+ max(0.0, bbox[0] - bw * expand),
+ max(0.0, bbox[1] - bh * expand),
+ min(float(w), bbox[2] + bw * expand),
+ min(float(h), bbox[3] + bh * expand),
+ ], dtype=np.float32)
+
+
class PosePipeline:
"""Full RF-DETR → InstantHMR pipeline.
Args:
onnx_path: Path to ``instanthmr.onnx``.
- device: ``"cuda"`` or ``"cpu"`` for InstantHMR (RF-DETR auto-selects).
+ device: ``"cuda"``, ``"coreml"`` or ``"cpu"`` for InstantHMR.
detector_variant: ``"nano" | "small" | "medium" | "base" | "large"``.
det_confidence: RF-DETR confidence threshold.
max_persons: Maximum number of persons returned per frame.
+ detector_stride: Run the detector every ``N`` frames; on the
+ in-between frames, reuse the previous frame's detections
+ (slightly expanded) to drive InstantHMR. ``1`` reproduces the
+ original "detect every frame" behaviour. The detector is by
+ far the dominant cost on most hardware, so stride 2–3 is the
+ single biggest knob for end-to-end FPS.
+ batch_persons: When ``True`` (default), all detected persons in a
+ frame are batched into a single InstantHMR ONNX call instead
+ of looping. This is a meaningful win for multi-person frames
+ and free for single-person frames.
Example
-------
- pipeline = PosePipeline("models/instanthmr.onnx")
+ pipeline = PosePipeline("models/instanthmr.onnx", detector_stride=2)
result = pipeline.predict(image_rgb)
- print(f"det {result.detector_ms:.1f} ms / hmr {result.hmr_ms:.1f} ms")
for p in result.persons:
print(p.joints_3d_cam.shape) # (70, 3)
"""
@@ -62,54 +84,72 @@ class PosePipeline:
detector_variant: str = "medium",
det_confidence: float = 0.5,
max_persons: int = 5,
+ detector_stride: int = 1,
+ batch_persons: bool = True,
):
+ if detector_stride < 1:
+ raise ValueError("detector_stride must be >= 1")
+
self.hmr = InstantHMR(onnx_path, device=device)
self.detector = RFDETRDetector(
variant=detector_variant,
confidence=det_confidence,
max_persons=max_persons,
)
+
+ self._detector_stride = detector_stride
+ self._batch_persons = batch_persons
+ self._frame_idx = 0
+ self._last_detections: list[dict] = []
self._last_bbox: Optional[np.ndarray] = None
def predict(self, image_rgb: np.ndarray) -> FrameResult:
- """Detect all persons in *image_rgb* and run InstantHMR on each.
-
- Returns a :class:`FrameResult` with per-stage timings so callers
- can display detector vs HMR cost separately.
- """
+ """Detect all persons in *image_rgb* and run InstantHMR on each."""
t_total_start = time.perf_counter()
-
- # ---- Detector ----
- t0 = time.perf_counter()
- detections = self.detector.detect(image_rgb)
- detector_ms = (time.perf_counter() - t0) * 1000.0
-
- # Single-frame fallback: reuse the last bbox if the detector drops
- # a frame. Useful for live camera mode where a transient miss is
- # common.
- if not detections and self._last_bbox is not None:
- h, w = image_rgb.shape[:2]
- lb = self._last_bbox
- bw, bh = lb[2] - lb[0], lb[3] - lb[1]
- expand = 0.1
- fb = np.array([
- max(0, lb[0] - bw * expand),
- max(0, lb[1] - bh * expand),
- min(w, lb[2] + bw * expand),
- min(h, lb[3] + bh * expand),
- ], dtype=np.float32)
+ h, w = image_rgb.shape[:2]
+
+ # ---- Detector (with optional stride) ----
+ run_detector = (self._frame_idx % self._detector_stride) == 0
+ if run_detector:
+ t0 = time.perf_counter()
+ detections = self.detector.detect(image_rgb)
+ detector_ms = (time.perf_counter() - t0) * 1000.0
+ if detections:
+ self._last_detections = detections
+ else:
+ detections = []
+ detector_ms = 0.0
+
+ # If the detector skipped or missed this frame, reuse the previous
+ # detections (slightly expanded to absorb small motion). This is a
+ # cheap form of tracking that keeps HMR fed with a plausible bbox.
+ if not detections and self._last_detections:
+ detections = [
+ {
+ "bbox": _expand_bbox(d["bbox"], w, h, expand=0.1),
+ "confidence": float(d["confidence"]) * 0.9,
+ }
+ for d in self._last_detections
+ ]
+ elif not detections and self._last_bbox is not None:
+ # Backwards-compatible single-bbox fallback (used when no prior
+ # multi-person detections exist).
+ fb = _expand_bbox(self._last_bbox, w, h, expand=0.1)
detections = [{"bbox": fb, "confidence": 0.0}]
- # ---- InstantHMR (sum across all persons) ----
+ # ---- InstantHMR ----
outputs: list[HMRPrediction] = []
t1 = time.perf_counter()
- for det in detections:
- out = self.hmr.predict(
- image_rgb,
- bbox=det["bbox"],
- confidence=det["confidence"],
- )
- outputs.append(out)
+ if detections:
+ if self._batch_persons and len(detections) > 1:
+ outputs = self.hmr.predict_batch(image_rgb, detections)
+ else:
+ for det in detections:
+ outputs.append(self.hmr.predict(
+ image_rgb,
+ bbox=det["bbox"],
+ confidence=det["confidence"],
+ ))
hmr_ms = (time.perf_counter() - t1) * 1000.0
if outputs:
@@ -117,6 +157,7 @@ class PosePipeline:
self._last_bbox = best.bbox.copy()
total_ms = (time.perf_counter() - t_total_start) * 1000.0
+ self._frame_idx += 1
return FrameResult(
persons=outputs,
diff --git a/requirements.txt b/requirements.txt
index 5bb1471..3c3f2ba 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,23 +1,27 @@
+# Recommended install path is `python install.py`, which picks the right
+# torch + onnxruntime wheels for your hardware (Linux + CUDA, Mac, or CPU).
+# This file is a portable fallback that uses the CPU wheel of torch and the
+# stock CPU build of onnxruntime — it works everywhere but leaves a lot of
+# performance on the table on NVIDIA GPUs and Apple Silicon.
+
# ----- inference + viz (required by demo.py) -----
numpy>=1.24
opencv-python>=4.8
pillow>=10
-onnxruntime-gpu>=1.19 ; platform_system != "Darwin"
-onnxruntime>=1.19 ; platform_system == "Darwin"
rfdetr>=1.0
rerun-sdk>=0.21
-# ----- CUDA runtime libs for onnxruntime-gpu -----
-# onnxruntime-gpu wheels are built against CUDA 12 + cuDNN 9 but do NOT
-# bundle them. These pip packages provide the .so files so you don't need
-# a system CUDA install. Instaharm calls onnxruntime.preload_dlls() at
-# session creation, which finds them automatically (ORT >= 1.19).
-nvidia-cudnn-cu12>=9.0,<10 ; platform_system != "Darwin"
-nvidia-cuda-runtime-cu12 ; platform_system != "Darwin"
-nvidia-cublas-cu12 ; platform_system != "Darwin"
+# Pulled in transitively by rfdetr; on Linux + CUDA you almost certainly
+# want a CUDA wheel from the pytorch index instead — see install.py.
+torch>=2.2
+torchvision
+
+# Mac ships CoreML EP inside plain onnxruntime; Linux GPU wants
+# onnxruntime-gpu (which install.py picks). The CPU wheel below is the
+# safe fallback.
+onnxruntime>=1.19
# ----- training / annotation (optional, for reproduction) -----
-# torch>=2.2
# timm>=1.0
# tqdm
# loguru
diff --git a/tools/bench.py b/tools/bench.py
new file mode 100644
index 0000000..115ee0b
--- /dev/null
+++ b/tools/bench.py
@@ -0,0 +1,138 @@
+#!/usr/bin/env python3
+"""Headless benchmark for the InstantHMR pipeline.
+
+Runs ``PosePipeline`` on a video and prints per-stage latency stats with no
+Rerun, no GUI, no .rrd writing — so the numbers reflect inference cost and
+not visualisation overhead.
+
+Usage:
+ python tools/bench.py --video vid1.mp4 [--device cuda] [--detector-variant medium]
+ [--max-frames 200] [--warmup 10]
+"""
+
+from __future__ import annotations
+
+import argparse
+import statistics
+import sys
+import time
+from pathlib import Path
+
+import cv2
+import numpy as np
+
+# Make the package importable when run from anywhere.
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from instanthmr import PosePipeline
+
+
+def parse_args() -> argparse.Namespace:
+ p = argparse.ArgumentParser()
+ p.add_argument("--video", type=str, required=True)
+ p.add_argument("--model", type=str, default="models/instanthmr.onnx")
+ p.add_argument("--device", type=str, default="cuda")
+ p.add_argument("--detector-variant", type=str, default="medium",
+ choices=["nano", "small", "medium", "base", "large"])
+ p.add_argument("--det-confidence", type=float, default=0.5)
+ p.add_argument("--max-persons", type=int, default=5)
+ p.add_argument("--detector-stride", type=int, default=1,
+ help="Run RF-DETR every N frames (default 1 = every frame).")
+ p.add_argument("--no-batch-persons", action="store_true",
+ help="Disable batched multi-person HMR (one ORT call per person).")
+ p.add_argument("--max-frames", type=int, default=200,
+ help="Cap measured frames (after warm-up).")
+ p.add_argument("--warmup", type=int, default=10,
+ help="Frames discarded before timing starts.")
+ return p.parse_args()
+
+
+def _summary(name: str, samples: list[float]) -> str:
+ if not samples:
+ return f"{name}: no samples"
+ arr = np.asarray(samples, dtype=np.float64)
+ return (
+ f"{name:>14s} "
+ f"mean={arr.mean():6.2f} ms "
+ f"median={np.median(arr):6.2f} ms "
+ f"p95={np.percentile(arr, 95):6.2f} ms "
+ f"min={arr.min():6.2f} ms "
+ f"max={arr.max():6.2f} ms"
+ )
+
+
+def main() -> None:
+ args = parse_args()
+
+ cap = cv2.VideoCapture(args.video)
+ if not cap.isOpened():
+ sys.exit(f"[error] cannot open video: {args.video}")
+
+ src_fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
+ total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+ width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+ height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+ print(f"video: {args.video} {width}x{height} {total} frames @ {src_fps:.1f} fps")
+
+ pipeline = PosePipeline(
+ onnx_path=args.model,
+ device=args.device,
+ detector_variant=args.detector_variant,
+ det_confidence=args.det_confidence,
+ max_persons=args.max_persons,
+ detector_stride=args.detector_stride,
+ batch_persons=not args.no_batch_persons,
+ )
+ print(f"InstantHMR EP: {pipeline.hmr.active_provider}")
+ print(f"detector variant: {args.detector_variant}")
+
+ det: list[float] = []
+ hmr: list[float] = []
+ tot: list[float] = []
+ wall: list[float] = []
+ n_persons: list[int] = []
+
+ idx = 0
+ measured = 0
+ last_wall = None
+ while True:
+ ret, bgr = cap.read()
+ if not ret:
+ break
+
+ rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
+ t_wall = time.perf_counter()
+ result = pipeline.predict(rgb)
+
+ if idx >= args.warmup:
+ det.append(result.detector_ms)
+ hmr.append(result.hmr_ms)
+ tot.append(result.total_ms)
+ n_persons.append(len(result.persons))
+ if last_wall is not None:
+ wall.append((t_wall - last_wall) * 1000.0)
+ measured += 1
+ last_wall = t_wall
+ idx += 1
+ if measured >= args.max_frames:
+ break
+
+ cap.release()
+
+ if not tot:
+ sys.exit("[error] no measured frames — increase --max-frames or shorten --warmup")
+
+ print()
+ print(f"measured frames: {len(tot)} (warmup={args.warmup})")
+ print(f"avg persons / frame: {statistics.mean(n_persons):.2f}")
+ print(_summary("detector_ms", det))
+ print(_summary("hmr_ms", hmr))
+ print(_summary("total_ms", tot))
+ if wall:
+ print(_summary("wall_ms", wall))
+ mean_total = float(np.mean(tot))
+ print(f"\nthroughput (1 / mean total): {1000.0 / mean_total:5.1f} FPS")
+
+
+if __name__ == "__main__":
+ main()