A high-performance inference engine for AI models
update benchmarks readme (#348)
eugene7ba6613d8bdc
diff --git a/crates/uzu/tests/BENCHMARKS.md b/crates/uzu/tests/BENCHMARKS.mdnew file mode 100644index 000000000..72d338ff4--- /dev/null+++ b/crates/uzu/tests/BENCHMARKS.md@@ -0,0 +1,96 @@+# Kernel Benchmarks++Criterion-based microbenchmarks for Metal kernels. Runs on macOS (native)+and on iPhone (via `cargo-dinghy`). Results from both are consolidated+under a single `target/criterion/<label>/` tree so you can compare+baselines side by side.++## Prerequisites++- Rust nightly: `rustup toolchain install nightly`+- For iOS runs:+ - Apple target: `rustup target add aarch64-apple-ios`+ - `cargo-dinghy` built from the [trymirai/dinghy](https://github.com/trymirai/dinghy/tree/mirai):++ ```bash+ cargo install \+ --git https://github.com/trymirai/dinghy \+ --branch mirai \+ cargo-dinghy+ ```++ - A physical device connected via USB with a device ID from+ `xcrun devicectl list devices`.++## Available benchmark groups++| Group id | Filter |+| ------------------------------ | ------------------------------ |+| `Metal/Kernel/Matmul/GEMM_MPP` | `Metal/Kernel/Matmul/GEMM_MPP` |+| `Metal/Kernel/RMSNorm` | `Metal/Kernel/RMSNorm` |+| `Metal/Kernel/Sampling/Argmax` | `Metal/Kernel/Sampling/Argmax` |++## Output layout++Every run writes into `target/criterion/<label>/…`, where `<label>` is a+free-form name you choose (e.g. `m2_max`, `a18`). The Criterion baseline+you saved lives at `target/criterion/<label>/<benchmark-path>/<baseline-name>/`.++## Running on macOS++From the repo root. Use an **absolute** `CRITERION_HOME` so it doesn't+resolve relative to the package dir:++```bash+CRITERION_HOME="$PWD/target/criterion/m2_max" cargo bench \+ -p uzu \+ --bench kernel -- "Metal/Kernel/RMSNorm" \+ --save-baseline rms_norm_baseline_m2_max+```++## Running on iPhone (via `cargo-dinghy`)++Run one benchmark group at a time to avoid the iOS watchdog killing the+app.++Key flags:++- `-e CRITERION_HOME=target/criterion/a18` — on-device env var. Path is+ relative to the app's cwd (`Documents/`), so this becomes+ `Documents/target/criterion/a18/` on device.+- `--copy-back "Documents/target=$(pwd)/target"` — after the run,+ `cargo-dinghy` pulls `Documents/target` from the device into your+ repo's `target/`. `$(pwd)` is required (absolute DST) because the+ cargo runner is launched with cwd set to the package dir, not the+ workspace root.++```bash+DEVICE=<DEVICE_ID>++cargo dinghy \+ -d "$DEVICE" \+ -e CRITERION_HOME=target/criterion/a18 \+ --copy-back "Documents/target=$(pwd)/target" \+ bench -p uzu --bench kernel -- \+ "Metal/Kernel/Matmul/GEMM_MPP" \+ --save-baseline matmul_gemm_mpp_baseline_a18+```++After the run completes you'll have+`target/criterion/a18/Metal/Kernel/Matmul/GEMM_MPP/…/matmul_gemm_mpp_baseline_a18/`+on the host, next to any `m2_max/` baselines.++## Viewing reports++Open the Criterion HTML report:++```bash+open target/criterion/report/index.html+```++To inspect a specific label only:++```bash+open target/criterion/m2_max/report/index.html+open target/criterion/a18/report/index.html+```diff --git a/crates/uzu/tests/BENCHMARKS_IOS.md b/crates/uzu/tests/BENCHMARKS_IOS.mddeleted file mode 100644index 9c7ea5541..000000000--- a/crates/uzu/tests/BENCHMARKS_IOS.md+++ /dev/null@@ -1,49 +0,0 @@-# Running Kernel Benchmarks on iOS--## Prerequisites--- iPhone connected via USB (iOS 26+)-- Rust nightly toolchain: `rustup toolchain install nightly`-- iOS target: `rustup target add aarch64-apple-ios`-- cargo-dinghy: `cargo install cargo-dinghy`-- Device ID: find with `xcrun devicectl list devices`--## Available benchmarks--| Benchmark group | Filter pattern | MXU required |-| ------------------------------------- | ------------------ | ------------ |-| `Metal/Kernel/Matmul/GEMM_MPP` | `GEMM_MPP/` | yes |-| `Metal/Kernel/Matmul/GEMM_MPP_DIRECT` | `GEMM_MPP_DIRECT/` | yes |-| `Metal/Kernel/RMSNorm` | `RMSNorm/` | no |-| `Metal/Kernel/Sampling/Argmax` | `Argmax/` | no |--## Running benchmarks--Run one benchmark group at a time to avoid iOS watchdog killing the app:--```bash-DEVICE=<DEVICE_ID>--# RMSNorm-cargo +nightly dinghy -d "$DEVICE" bench -p uzu --bench kernel -- "RMSNorm/"-```--## Retrieving Criterion reports--Criterion saves HTML reports inside the Dinghy app's sandbox on the device.-To pull them to your local machine:--```bash-xcrun devicectl device copy from \- --device <DEVICE_ID> \- --domain-type appDataContainer \- --domain-identifier org.zoy.kali.Dinghy \- --source Documents/target/criterion \- --destination ./criterion-ios-```--Open the report:--```bash-open criterion-ios/report/index.html-```