Skip to content

armgit / uzu

Public Imported from GitHub
0

A high-performance inference engine for AI models

update benchmarks readme (#348)

eugene7ba6613d8bdc
diff --git a/crates/uzu/tests/BENCHMARKS.md b/crates/uzu/tests/BENCHMARKS.md
new file mode 100644
index 000000000..72d338ff4
--- /dev/null
+++ b/crates/uzu/tests/BENCHMARKS.md
@@ -0,0 +1,96 @@
+# Kernel Benchmarks
+
+Criterion-based microbenchmarks for Metal kernels. Runs on macOS (native)
+and on iPhone (via `cargo-dinghy`). Results from both are consolidated
+under a single `target/criterion/<label>/` tree so you can compare
+baselines side by side.
+
+## Prerequisites
+
+- Rust nightly: `rustup toolchain install nightly`
+- For iOS runs:
+ - Apple target: `rustup target add aarch64-apple-ios`
+ - `cargo-dinghy` built from the [trymirai/dinghy](https://github.com/trymirai/dinghy/tree/mirai):
+
+ ```bash
+ cargo install \
+ --git https://github.com/trymirai/dinghy \
+ --branch mirai \
+ cargo-dinghy
+ ```
+
+ - A physical device connected via USB with a device ID from
+ `xcrun devicectl list devices`.
+
+## Available benchmark groups
+
+| Group id | Filter |
+| ------------------------------ | ------------------------------ |
+| `Metal/Kernel/Matmul/GEMM_MPP` | `Metal/Kernel/Matmul/GEMM_MPP` |
+| `Metal/Kernel/RMSNorm` | `Metal/Kernel/RMSNorm` |
+| `Metal/Kernel/Sampling/Argmax` | `Metal/Kernel/Sampling/Argmax` |
+
+## Output layout
+
+Every run writes into `target/criterion/<label>/…`, where `<label>` is a
+free-form name you choose (e.g. `m2_max`, `a18`). The Criterion baseline
+you saved lives at `target/criterion/<label>/<benchmark-path>/<baseline-name>/`.
+
+## Running on macOS
+
+From the repo root. Use an **absolute** `CRITERION_HOME` so it doesn't
+resolve relative to the package dir:
+
+```bash
+CRITERION_HOME="$PWD/target/criterion/m2_max" cargo bench \
+ -p uzu \
+ --bench kernel -- "Metal/Kernel/RMSNorm" \
+ --save-baseline rms_norm_baseline_m2_max
+```
+
+## Running on iPhone (via `cargo-dinghy`)
+
+Run one benchmark group at a time to avoid the iOS watchdog killing the
+app.
+
+Key flags:
+
+- `-e CRITERION_HOME=target/criterion/a18` — on-device env var. Path is
+ relative to the app's cwd (`Documents/`), so this becomes
+ `Documents/target/criterion/a18/` on device.
+- `--copy-back "Documents/target=$(pwd)/target"` — after the run,
+ `cargo-dinghy` pulls `Documents/target` from the device into your
+ repo's `target/`. `$(pwd)` is required (absolute DST) because the
+ cargo runner is launched with cwd set to the package dir, not the
+ workspace root.
+
+```bash
+DEVICE=<DEVICE_ID>
+
+cargo dinghy \
+ -d "$DEVICE" \
+ -e CRITERION_HOME=target/criterion/a18 \
+ --copy-back "Documents/target=$(pwd)/target" \
+ bench -p uzu --bench kernel -- \
+ "Metal/Kernel/Matmul/GEMM_MPP" \
+ --save-baseline matmul_gemm_mpp_baseline_a18
+```
+
+After the run completes you'll have
+`target/criterion/a18/Metal/Kernel/Matmul/GEMM_MPP/…/matmul_gemm_mpp_baseline_a18/`
+on the host, next to any `m2_max/` baselines.
+
+## Viewing reports
+
+Open the Criterion HTML report:
+
+```bash
+open target/criterion/report/index.html
+```
+
+To inspect a specific label only:
+
+```bash
+open target/criterion/m2_max/report/index.html
+open target/criterion/a18/report/index.html
+```
diff --git a/crates/uzu/tests/BENCHMARKS_IOS.md b/crates/uzu/tests/BENCHMARKS_IOS.md
deleted file mode 100644
index 9c7ea5541..000000000
--- a/crates/uzu/tests/BENCHMARKS_IOS.md
+++ /dev/null
@@ -1,49 +0,0 @@
-# Running Kernel Benchmarks on iOS
-
-## Prerequisites
-
-- iPhone connected via USB (iOS 26+)
-- Rust nightly toolchain: `rustup toolchain install nightly`
-- iOS target: `rustup target add aarch64-apple-ios`
-- cargo-dinghy: `cargo install cargo-dinghy`
-- Device ID: find with `xcrun devicectl list devices`
-
-## Available benchmarks
-
-| Benchmark group | Filter pattern | MXU required |
-| ------------------------------------- | ------------------ | ------------ |
-| `Metal/Kernel/Matmul/GEMM_MPP` | `GEMM_MPP/` | yes |
-| `Metal/Kernel/Matmul/GEMM_MPP_DIRECT` | `GEMM_MPP_DIRECT/` | yes |
-| `Metal/Kernel/RMSNorm` | `RMSNorm/` | no |
-| `Metal/Kernel/Sampling/Argmax` | `Argmax/` | no |
-
-## Running benchmarks
-
-Run one benchmark group at a time to avoid iOS watchdog killing the app:
-
-```bash
-DEVICE=<DEVICE_ID>
-
-# RMSNorm
-cargo +nightly dinghy -d "$DEVICE" bench -p uzu --bench kernel -- "RMSNorm/"
-```
-
-## Retrieving Criterion reports
-
-Criterion saves HTML reports inside the Dinghy app's sandbox on the device.
-To pull them to your local machine:
-
-```bash
-xcrun devicectl device copy from \
- --device <DEVICE_ID> \
- --domain-type appDataContainer \
- --domain-identifier org.zoy.kali.Dinghy \
- --source Documents/target/criterion \
- --destination ./criterion-ios
-```
-
-Open the report:
-
-```bash
-open criterion-ios/report/index.html
-```