Skip to content

armgit / uzu

Public Imported from GitHub
0

A high-performance inference engine for AI models

Remove ai instructions (#477)

uuuvnfea494280357
diff --git a/.claude/skills b/.claude/skills
deleted file mode 120000
index a769d4bd9..000000000
--- a/.claude/skills
+++ /dev/null
@@ -1 +0,0 @@
-../agents/skills
\ No newline at end of file
diff --git a/.codex/skills b/.codex/skills
deleted file mode 120000
index a769d4bd9..000000000
--- a/.codex/skills
+++ /dev/null
@@ -1 +0,0 @@
-../agents/skills
\ No newline at end of file
diff --git a/.cursorrules b/.cursorrules
deleted file mode 120000
index 95e38a6a9..000000000
--- a/.cursorrules
+++ /dev/null
@@ -1 +0,0 @@
-agents/AGENTS.md
\ No newline at end of file
diff --git a/.gemini/skills b/.gemini/skills
deleted file mode 120000
index a769d4bd9..000000000
--- a/.gemini/skills
+++ /dev/null
@@ -1 +0,0 @@
-../agents/skills
\ No newline at end of file
diff --git a/.github/skills b/.github/skills
deleted file mode 120000
index a769d4bd9..000000000
--- a/.github/skills
+++ /dev/null
@@ -1 +0,0 @@
-../agents/skills
\ No newline at end of file
diff --git a/AGENTS.md b/AGENTS.md
deleted file mode 120000
index 95e38a6a9..000000000
--- a/AGENTS.md
+++ /dev/null
@@ -1 +0,0 @@
-agents/AGENTS.md
\ No newline at end of file
diff --git a/agents/AGENTS.md b/agents/AGENTS.md
deleted file mode 100644
index 672c2160d..000000000
--- a/agents/AGENTS.md
+++ /dev/null
@@ -1,77 +0,0 @@
-# Rules
-
-The rules below are CRITICAL. Each rule should be respected. Rules can only be overridden by the user.
-
-## Workflow rules
-
-- Agent should use provided skills, especially the architecture description.
-- When given a coding task, agent carefully and deeply inspects the codebase to understand the context.
-- Agent doesn't ask for permission to read the contents of files and directories.
-- When given a task, agent first thinks about multiple high-level strategies, and then asks the user for feedback on every option.
-- Agent splits tasks into smallest possible subtasks, and solves them one by one while asking the user for detailed feedback during every step.
-- Agent never edits code before proposing a short draft to the user and receiving a feedback.
-- Agent is not allowed to make even the tiniest architectural decisions, it can only propose multiple options to the user.
-- Agent behaves like an interviewer and asks the user questions all the time.
-- Instead of asking multiple questions in one message, agent asks them one by one.
-- Agent always asks for permission before launching CLI tools which may take lots of time or produce side effects.
-- Before using any function from an external library, agent thoroughly inspects the code or documentation for the function to ensure that it uses it correctly.
-- When encountering an error, agent never rushes to quickly fix it. Instead, it uses the scientific method to understand the source of the error: comes up with hypotheses and tests them by performing a series of experiments. It only attempts to fix the error once the hypothesis has been sufficiently validated.
-- Agent avoids creating .md files with explanations unless explicitly requested by the user.
-- When receiving general recommendations from the user, immediately add them to agents/AGENTS.md under the appropriate section.
-
-## General coding guidelines
-
-- Agent never resorts to quick hacks or stubs to make something work. Instead it always investigates the problem thoroughly and comes up with a well-thought-out solution.
-- By default, agent does not write documentation or comments unless the code contains complex logic which is difficult to understand without them.
-- Agent strives for simplicity and maintainability. Agent avoids writing boilerplate code, instead it tries to come up with the right abstractions.
-- Agent prefers functional programming style, namely:
- - Avoids mutable state.
- - Avoids modifying objects in-place.
- - Avoids implicit side effects.
- - Prefers immutable data structures.
- - Strives to make invalid state unrepresentable.
-- Agent tries to write code in the way that makes bugs hard to make and easy to spot.
-- Agent writes the code so that assumptions about invariants are as explicit as possible.
-- Agent leverages strong typing as much as possible, for example, it always uses enums instead of literals.
-- Agent uses good descriptive variable names that make it easy to understand the purpose of the variable without looking at the code that uses it.
-- Agent does not manually parse things such as JSON configs. Instead it first models a schema using types, and then uses a serialisation library.
-
-## Testing guidelines
-
-- Agent strives for simplicity and minimalism when writing tests.
-- Agent avoids extensive mocking and stubbing.
-- Agent focuses on testing core functionality, and does not test minor details such as property accessors.
-- Agent avoids boilerplate in the testing code, and tries to come up with the right generic abstractions.
-
-## Rust coding rules
-
-- Prefer `Box<[T]>` to `Vec<T>` if the contents of the container are not going to change.
-- Avoid mutable variables as much as possible. Use iterator expressions to collect data into containers.
-- For calling API that was never used before, read the sources of the installed dependencies, otherwise, search in docs.rs.
-- Avoid inline imports in trait bounds or function signatures; always import with `use` at the top of the file
-- In sub-modules use `mod ...;` declarations with explicit `pub use module::{Item1, Item2}` — never global `pub use module::*`
-- Format with `cargo +nightly fmt` to enable unstable rustfmt features
-- Use full names, never abbreviate (e.g. `GroupDefinition` not `GroupDef`, `definition` not `def`)
-- Avoid excessive comments; code should be self-explanatory
-- Always use descriptive variable names; never single-letter names like `a`, `b`, `i`, `s`
-- Do not extract helper methods that are only called from one place; inline the logic instead
-- Avoid `super::` imports; always use `crate::` level imports
-- Derive `Serialize`, `Deserialize`, `Clone`, `Debug`, `PartialEq`, `Eq`, `Hash` on all data types where applicable; all configs must be `Serialize`/`Deserialize`
-- In recursive structures, prefer the base type name for child collections (e.g., `groups: Vec<Group>` not `children: Vec<Group>`)
-- Never panic or force unwrap in library code; use typed errors with `thiserror`; panics/unwraps are only acceptable in tests
-- All tests in `tests/` folder, named `test_<module>.rs`; test functions named `test_<module>_<behavior>`
-- Shared test helpers in `tests/helpers/`; avoid duplicating helpers between test files
-- Move dependencies to `[dev-dependencies]` when only used in tests
-- Always check for unused dependencies and imports
-
-## Python coding rules
-
-- Never use pip to manage dependencies. Instead use uv commands, such as `uv add`.
-- Assume Python 3.12. Use `T | None` instead of `Optional[T]` and `dict[K, V]` instead of `typing.Dict[K, V]`.
-- Every function should have full type annotations.
-- Prefer comprehensions to map and filter expressions.
-- Prefer dataclasses over vanilla python classes.
-- All path-like arguments should have type `Path | str`.
-- Prefer frozen dataclasses or named tuples as primary data structures.
-- Never use dicts in place of dataclasses.
-- __init__.py files should only contain reexports.
\ No newline at end of file
diff --git a/agents/skills/architecture/SKILL.md b/agents/skills/architecture/SKILL.md
deleted file mode 100644
index c3623d9b3..000000000
--- a/agents/skills/architecture/SKILL.md
+++ /dev/null
@@ -1,26 +0,0 @@
----
-name: architecture
-description: Description of the main concepts of the inference engine
----
-
-Read this carefully before doing any changes:
-
-## Architecture
-
-1. uzu runs models converted via https://github.com/trymirai/lalamo toolkit in unified format.
-
-2. Session. High-level object used to perform requests to the model. It supports multiple modes (ignore previous context, use static context to perform requests, and dynamically aggregate context). It creates a generator object which implements the decoding loop, manually calls prefill and generate steps, and collects performance stats. Decoding can be a fully GPU-driven pipeline or can use CPU sync after each token when needed (speculative and constrained decoding).
-
-3. Generator. Object that implements forward pass execution for prefill and decode stages. It also implements async decoding loop logic. It creates all necessary buffers, loads weights into memory, prepares masks, speculative suffixes, and grammar masks if needed. For sync loop it also implements preencode logic.
-
-4. Encodables. High-level objects that implement kernel selection. Each encodable has an encode method and depends on the corresponding config object from the model's config.json.
-
-5. Kernels. Backend-specific shaders that use DSL from ./crates/uzu/build to automatically generate Rust interfaces. Each kernel should have a corresponding unit test that validates correctness.
-
-## What is important?
-
-1. Performance is everything. All changes should be as efficient as possible in terms of prefill/decode speed and memory usage.
-
-2. Keep in mind that this is an SDK that will be used by a wide range of developers, so excellent developer experience is crucial.
-
-3. You can explore various docs here: https://docs.trymirai.com
\ No newline at end of file
diff --git a/agents/skills/benchmark-model/SKILL.md b/agents/skills/benchmark-model/SKILL.md
deleted file mode 100644
index aef2f13bd..000000000
--- a/agents/skills/benchmark-model/SKILL.md
+++ /dev/null
@@ -1,19 +0,0 @@
----
-name: benchmark-model
-description: Benchmark inference performance for a specific model
----
-
-Using CLI to perform benchmark.
-
-## Instructions
-
-1. Choose model you want to use from ./models/{ENGINE_VERSION}/{MODEL_NAME} or download it
-
-2. Ensure benchmark_task.json exists in the model folder (it is automatically generated after model download)
-
-3. Run
- ```
- cargo run --release -p cli -- bench ./models/{ENGINE_VERSION}/{MODEL_NAME} ./models/{ENGINE_VERSION}/{MODEL_NAME}/benchmark_task.json {RESULT_PATH}
- ```
-
-4. Check result json to check memory_used, time_to_first_token, prompt_tokens_per_second and generate_tokens_per_second. Optionally check text is readable.
diff --git a/agents/skills/download-model/SKILL.md b/agents/skills/download-model/SKILL.md
deleted file mode 100644
index 83ee3d91a..000000000
--- a/agents/skills/download-model/SKILL.md
+++ /dev/null
@@ -1,20 +0,0 @@
----
-name: download-model
-description: Download model to test inference engine
----
-
-Using ./tools/helpers Python package to download model
-
-## Instructions
-
-1. Go to helpers folder:
- ```
- cd ./tools/helpers
- ```
-
-2. Run:
- ```
- uv run main.py download-model {REPO_ID}
- ```
-
-3. Downloaded model will be in ./models/{ENGINE_VERSION}/{MODEL_NAME} folder
diff --git a/agents/skills/list-downloaded-models/SKILL.md b/agents/skills/list-downloaded-models/SKILL.md
deleted file mode 100644
index 81d0c8ba7..000000000
--- a/agents/skills/list-downloaded-models/SKILL.md
+++ /dev/null
@@ -1,8 +0,0 @@
----
-name: list-downloaded-models
-description: Get list of currently downloaded models
----
-
-## Instructions
-
-1. List folders inside ./models/{ENGINE_VERSION}, where each folder name is a model name
\ No newline at end of file
diff --git a/agents/skills/list-models/SKILL.md b/agents/skills/list-models/SKILL.md
deleted file mode 100644
index 163fc9740..000000000
--- a/agents/skills/list-models/SKILL.md
+++ /dev/null
@@ -1,20 +0,0 @@
----
-name: list-models
-description: Get list of models supported by inference engine
----
-
-Using ./tools/helpers Python package to list supported models
-
-## Instructions
-
-1. Go to helpers folder:
- ```
- cd ./tools/helpers
- ```
-
-2. Run:
- ```
- uv run main.py list-models
- ```
-
-3. Choose repo you want to use
diff --git a/agents/skills/metal-gpu-debug/SKILL.md b/agents/skills/metal-gpu-debug/SKILL.md
deleted file mode 100644
index ae37e147b..000000000
--- a/agents/skills/metal-gpu-debug/SKILL.md
+++ /dev/null
@@ -1,1010 +0,0 @@
----
-name: metal-gpu-debug
-description: >
- Metal GPU debugging and profiling via xctrace and Apple's Metal toolchain. Use this skill when the user mentions:
- Metal debugging, .gputrace files, Metal shaders, Metal pipeline state, Metal performance,
- Metal System Trace, Apple GPU profiling, Metal validation, shader compilation errors,
- MSL (Metal Shading Language), .metallib files, .air files, Metal render pass,
- Metal command buffer, Metal encoder, MTLDevice, MTLTexture, Metal Performance HUD,
- GPU counters on Apple Silicon, Metal frame capture, Metal API validation,
- Metal shader validation, xcrun metal, xctrace, Instruments GPU trace,
- MoltenVK debugging, Vulkan on macOS, Apple Silicon GPU, Metal 3, Metal 4,
- Metal compute kernel, MetalFX, Metal Performance Shaders, Metal ray tracing,
- "profile my Metal app", "why is my shader slow", "capture a Metal frame",
- "Metal validation error", "debug Metal on Mac", "GPU trace on macOS".
- DO NOT use for: RenderDoc, Vulkan on Windows/Linux, D3D debugging, PIX,
- Nsight, CSS rendering, React rendering, server-side rendering, HTML layout,
- browser DevTools, web performance.
----
-
-# Metal GPU Debugging & Profiling Skill
-
-## Overview
-
-This skill enables Metal GPU profiling, debugging, and shader analysis on macOS using Apple's command-line toolchain. The primary tools are:
-
-- **`gpu_trace`** — Unified CLI for recording, analyzing, and comparing Metal GPU traces (in `tools/gpu_trace/`, run via `uv run gpu_trace`). Subcommands:
- - `gpu_trace run` — Record a Metal System Trace while running a command
- - `gpu_trace analyze` — Analyze an existing `.trace` bundle with per-kernel timing & HW counters
- - `gpu_trace compare` — Compare two traces side-by-side
- - `gpu_trace capture` — Inspect `.gputrace` captures (list resources, read buffers/textures)
- - `gpu_trace view` — View a previously exported JSON trace
-- **`xctrace`** — CLI for Instruments (used internally by `gpu_trace`; can also be used directly)
-- **`xcrun metal`** — Metal shader compiler toolchain (compile, archive, link)
-- **Metal environment variables** — validation layers, Performance HUD, programmatic capture
-- **`log`** — macOS unified logging for Metal HUD and validation output
-
-### Prerequisites
-
-Before any Metal debugging, verify the environment:
-
-```bash
-# Check Xcode is installed (full Xcode, not just Command Line Tools)
-xcode-select -p
-# Expected: /Applications/Xcode.app/Contents/Developer
-
-# Check xctrace is available
-xcrun xctrace version
-
-# Check Metal compiler
-xcrun -sdk macosx metal --version
-
-# List available GPU devices (requires a Swift/ObjC helper or Python)
-system_profiler SPDisplaysDataType | grep -A5 "Chipset\|Metal"
-```
-
-**IMPORTANT**: `xctrace` and Metal tools require **full Xcode**, not just Command Line Tools.
-
-## 1. Automated Session Lifecycle
-
-Every Metal debugging session follows a strict pipeline: **Doctor → Record → Export → Parse → Analyze → Report → Cleanup**. Claude should execute this pipeline autonomously, not just suggest commands.
-
-### Step 0: Doctor — Verify environment
-
-**Run this FIRST before any debugging session.** If any check fails, stop and tell the user what's missing.
-
-```bash
-# Create working directories
-mkdir -p ./traces/analysis
-
-# Verify toolchain
-xcode-select -p # Must show Xcode.app path, NOT CommandLineTools
-xcrun xctrace version # Must succeed
-xcrun -sdk macosx metal --version # Must succeed
-system_profiler SPDisplaysDataType | grep -i "metal\|chipset" # GPU info
-
-# For iOS: check connected devices
-xcrun xctrace list devices 2>/dev/null # List all available targets
-
-# Check gpu_trace is available
-cd tools/gpu_trace && uv run gpu_trace --help >/dev/null 2>&1 && echo "gpu_trace OK"
-```
-
-**If `xcode-select -p` returns `/Library/Developer/CommandLineTools`**, tell the user:
-```
-Xcode Command Line Tools is installed but full Xcode is required.
-Install from: https://developer.apple.com/xcode/
-Then run: sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
-```
-
-### Step 1: Record — Capture a Metal System Trace
-
-**Preferred: Use `gpu_trace run`** (handles recording + export + analysis automatically):
-
-```bash
-# MODE A: Launch and profile (most common)
-cd tools/gpu_trace && uv run gpu_trace run -- /path/to/app [args...]
-
-# MODE A with hardware counters (ALU, memory, cache)
-cd tools/gpu_trace && uv run gpu_trace run --gpu-counters -- /path/to/app
-
-# MODE A with time limit
-cd tools/gpu_trace && uv run gpu_trace run --time-limit 10s -- /path/to/app
-
-# MODE B: Attach to running process
-cd tools/gpu_trace && uv run gpu_trace run --attach <PID> --time-limit 10s
-
-# MODE A with verbose output (all HW counters per kernel)
-cd tools/gpu_trace && uv run gpu_trace run -v --gpu-counters -- /path/to/app
-```
-
-**Fallback: Use `xctrace` directly** (for advanced options like validation env vars, iOS):
-
-```bash
-# MODE C: With validation enabled (for debugging errors)
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --env MTL_DEBUG_LAYER=1 \
- --env MTL_SHADER_VALIDATION=1 \
- --time-limit 10s \
- --no-prompt \
- --output ./traces/capture.trace \
- --launch -- /path/to/app
-
-# MODE D: iOS device (over USB)
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --device '<DEVICE_NAME_OR_UDID>' \
- --attach <APP_NAME> \
- --time-limit 10s \
- --no-prompt \
- --output ./traces/capture.trace
-```
-
-**Decision guide:**
-- User says "profile my app" → `gpu_trace run`
-- User says "it's already running" → `gpu_trace run --attach <PID>`
-- User says "I'm getting errors" or "something is wrong" → `xctrace` with validation env vars
-- User mentions iPhone/iPad → `xctrace` with `--device`
-
-### Steps 2–3: Export & Parse — Analyze the trace
-
-**Preferred: Use `gpu_trace analyze`** (handles export, parsing, and analysis in one step):
-
-```bash
-# Analyze an existing trace (recorded with gpu_trace run or xctrace)
-cd tools/gpu_trace && uv run gpu_trace analyze /path/to/capture.trace
-
-# Verbose output (all HW counters per kernel)
-cd tools/gpu_trace && uv run gpu_trace analyze /path/to/capture.trace -v
-
-# Export to JSON for further processing
-cd tools/gpu_trace && uv run gpu_trace analyze /path/to/capture.trace --json output.json
-
-# JSON to stdout (for piping)
-cd tools/gpu_trace && uv run gpu_trace analyze /path/to/capture.trace --json -
-```
-
-`gpu_trace analyze` automatically:
-- Exports all relevant schemas in parallel (GPU state, encoders, intervals, counters, shaders)
-- Resolves xctrace's XML reference/ID structure
-- Computes GPU utilization with overlapping interval union
-- Aggregates per-kernel timing statistics
-- Correlates hardware counters with kernel dispatches (O(D log S) bisect)
-- Detects bottlenecks (ALU, buffer, cache, MMU limiters)
-
-**Fallback: Use `xctrace export` directly** (for schemas not covered by `gpu_trace`):
-
-```bash
-# Get table of contents
-xcrun xctrace export --input ./traces/capture.trace --toc
-
-# Export a specific schema
-xcrun xctrace export --input ./traces/capture.trace \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-```
-
-**Schema availability varies** by Xcode version, template, and GPU. `gpu_trace analyze` handles missing schemas gracefully.
-
-### Step 4: Analyze — Interpret the data
-
-After parsing, Claude should analyze the results and look for:
-
-**Performance profiling:**
-- GPU busy time per frame (>16ms = below 60fps, >8ms = below 120fps)
-- Large wire memory events (excessive per-frame resource allocation)
-- Gaps between GPU submissions (CPU-bound)
-- Long shader execution intervals (complex shaders)
-- Imbalanced encoder durations (one pass dominating)
-
-**Validation errors:**
-- Pattern-match stderr and log output for Metal error codes
-- Classify errors: API misuse vs shader bug vs resource issue
-- Map errors to specific command encoders via labels
-
-**Shader issues:**
-- Compile-time warnings/errors from `xcrun metal`
-- Runtime validation errors from `MTL_SHADER_VALIDATION`
-
-### Step 5: Report — Present findings to user
-
-Structure the report as:
-1. **Environment**: GPU model, macOS version, device (Mac/iOS)
-2. **Summary**: Total events, recording duration, overall health
-3. **Key findings**: Sorted by severity (errors → warnings → info)
-4. **Specific data**: Relevant numbers, event counts, durations
-5. **Recommendations**: Concrete next steps
-
-### Step 6: Cleanup
-
-```bash
-# Remove large trace files when analysis is complete
-# (only if user doesn't need the raw trace)
-rm -rf ./traces/capture.trace
-
-# Keep analysis outputs for reference
-ls -la ./traces/analysis/
-```
-
-### Complete automated workflow example
-
-This is what Claude should execute end-to-end when a user says "profile my Metal app":
-
-```bash
-# Single command: records, exports (parallel), analyzes, and prints results
-cd tools/gpu_trace && uv run gpu_trace run --gpu-counters -- /path/to/app
-
-# Or if the trace was already recorded separately:
-cd tools/gpu_trace && uv run gpu_trace analyze /path/to/capture.trace -v
-```
-
-`gpu_trace run` handles the entire pipeline: Doctor → Record → Export (parallel) → Parse → Analyze → Report. JSON is automatically saved alongside the trace.
-
-For **before/after comparison**:
-
-```bash
-cd tools/gpu_trace
-uv run gpu_trace run -o before.trace -- /path/to/app_before
-# ... make code changes ...
-uv run gpu_trace run -o after.trace -- /path/to/app_after
-uv run gpu_trace compare before.trace after.trace --label1 before --label2 after
-```
-
-### Parallel workflows: Validation + Profiling + Logs
-
-Claude can run multiple debugging streams simultaneously for maximum signal:
-
-```bash
-# Record trace with all validation layers AND HUD logging in one shot
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --env MTL_DEBUG_LAYER=1 \
- --env MTL_SHADER_VALIDATION=1 \
- --env MTL_HUD_ENABLED=1 \
- --env MTL_HUD_LOGGING_ENABLED=1 \
- --time-limit 10s \
- --no-prompt \
- --output ./traces/full_debug.trace \
- --launch -- /path/to/app 2> ./traces/analysis/stderr.log &
-
-TRACE_PID=$!
-
-# Simultaneously capture Metal logs from unified log
-log stream --predicate 'subsystem == "com.apple.Metal"' \
- --timeout 15 > ./traces/analysis/metal_log.txt 2>/dev/null &
-
-LOG_PID=$!
-
-# Wait for trace to finish
-wait $TRACE_PID
-
-# Give log stream a moment then stop it
-sleep 2
-kill $LOG_PID 2>/dev/null
-
-# Now analyze ALL data sources:
-echo "=== Validation Errors (stderr) ==="
-grep -i "error\|warning\|invalid\|fault" ./traces/analysis/stderr.log || echo "None"
-
-echo "=== Metal Log Entries ==="
-wc -l < ./traces/analysis/metal_log.txt
-grep -i "error" ./traces/analysis/metal_log.txt || echo "No errors in log"
-
-echo "=== Trace Data ==="
-xcrun xctrace export --input ./traces/full_debug.trace --toc
-```
-
-### Session state awareness
-
-Unlike RenderDoc's daemon model (open/close), xctrace sessions are **fire-and-forget**: each `record` command runs to completion, and the resulting `.trace` file is a self-contained snapshot. This means:
-
-- **No session to manage** — no open/close, no daemon, no leaked processes
-- **Multiple traces can coexist** — name them meaningfully (before.trace, after.trace)
-- **Traces are immutable** — once recorded, they don't change
-- **Export is idempotent** — re-export the same trace as many times as needed
-- **Cleanup is just file deletion** — `rm` the .trace bundle when done
-
-## 2. iOS Device Support
-
-`xctrace` can profile Metal apps on physical iOS/iPadOS devices over USB or Wi-Fi.
-
-### List connected devices
-
-```bash
-xcrun xctrace list devices
-# Shows: Mac, connected iPhones/iPads, simulators
-```
-
-### Record on iOS device
-
-```bash
-# By device name
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --device 'iPhone 15 Pro' \
- --attach MyApp \
- --time-limit 10s \
- --output ios_trace.trace
-
-# By UDID (more reliable)
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --device 00008110-XXXXXXXXXXXX \
- --attach MyApp \
- --time-limit 10s \
- --output ios_trace.trace
-
-# Launch app on device (must be installed)
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --device 'iPhone 15 Pro' \
- --time-limit 10s \
- --output ios_trace.trace \
- --launch -- com.yourcompany.yourapp
-```
-
-### iOS-specific notes
-
-- Device must be **unlocked** and **trusted** by the Mac
-- App must be installed on device (use Xcode or `ios-deploy`)
-- `--launch` uses **bundle identifier** (not path) on iOS
-- `--attach` uses **process name** or PID
-- Export/parse works identically — the `.trace` format is the same
-- Shader compilation for iOS uses `-sdk iphoneos`:
- ```bash
- xcrun -sdk iphoneos metal -c Shader.metal -o Shader.air
- ```
-- `.gputrace` capture on iOS requires Xcode attached to the device
-
-## 3. Metal System Trace — Record & Export
-
-Metal System Trace is the primary tool for GPU profiling. It captures CPU/GPU timeline, driver events, shader execution, and hardware counters.
-
-### Record a trace
-
-```bash
-# Profile a running app by name
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --attach <PID_OR_NAME> \
- --time-limit 5s \
- --output trace.trace
-
-# Launch and profile
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --time-limit 5s \
- --output trace.trace \
- --launch -- /path/to/your/app [args...]
-
-# With environment variables (e.g., enable validation)
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --env MTL_DEBUG_LAYER=1 \
- --env MTL_SHADER_VALIDATION=1 \
- --time-limit 5s \
- --output trace.trace \
- --launch -- /path/to/your/app
-```
-
-Key options:
-- `--time-limit 5s` — auto-stop after duration (supports ms, s, m, h)
-- `--attach PID` — attach to running process
-- `--all-processes` — trace all Metal apps system-wide
-- `--env VAR=value` — set environment variables for launched process
-- `--target-stdout -` — redirect app stdout to terminal
-- `--no-prompt` — skip prompts (useful in scripts)
-
-### Other useful templates
-
-```bash
-# List all available templates
-xcrun xctrace list templates
-# Key Metal-relevant templates:
-# - Metal System Trace (GPU timeline, driver events, counters)
-# - Game Performance (Metal + display + thermal)
-# - Counters (hardware performance counters)
-# - GPU (GPU-focused template, if available)
-```
-
-### Export trace data as XML
-
-```bash
-# See what's in the trace (table of contents)
-xcrun xctrace export --input trace.trace --toc
-
-# Export Metal driver events (GPU work intervals, wire memory, etc.)
-xcrun xctrace export --input trace.trace \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-
-# Export to file instead of stdout
-xcrun xctrace export --input trace.trace \
- --output metal_events.xml \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-
-# Export GPU counter data
-xcrun xctrace export --input trace.trace \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="gpu-counter-intervals"]'
-```
-
-### Common Metal table schemas
-
-| Schema | Contains |
-|--------|----------|
-| `metal-driver-event-intervals` | Metal driver events (GPU work, wire memory, resource events) |
-| `gpu-counter-intervals` | Hardware GPU performance counters |
-| `metal-gpu-intervals` | GPU execution intervals per encoder |
-| `time-profile` | CPU time profiling samples |
-
-**TIP**: Always run `--toc` first to see available schemas — they vary by template, Xcode version, and GPU.
-
-### Parse exported XML
-
-The XML uses a reference system to avoid duplication. Nodes with `id` attributes are originals; nodes with `ref` attributes point back to them.
-
-```bash
-# Use gpu_trace to parse all schemas automatically
-cd tools/gpu_trace && uv run gpu_trace analyze trace.trace -v
-
-# Or export to JSON for programmatic access
-cd tools/gpu_trace && uv run gpu_trace analyze trace.trace --json output.json
-```
-
-## 4. Metal Validation Layers
-
-Runtime error detection for Metal API misuse and shader bugs.
-
-### Enable via environment variables
-
-```bash
-# API Validation — catches Metal API misuse
-export MTL_DEBUG_LAYER=1
-
-# Shader Validation — instruments shaders to detect GPU-side errors
-export MTL_SHADER_VALIDATION=1
-
-# Combined: launch app with both
-MTL_DEBUG_LAYER=1 MTL_SHADER_VALIDATION=1 /path/to/your/app
-```
-
-### Enable via xctrace
-
-```bash
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --env MTL_DEBUG_LAYER=1 \
- --env MTL_SHADER_VALIDATION=1 \
- --time-limit 10s \
- --output validated.trace \
- --launch -- /path/to/your/app
-```
-
-### Read validation errors
-
-Validation errors appear in stderr and in macOS unified log:
-
-```bash
-# Stream Metal validation errors live
-log stream --predicate 'subsystem == "com.apple.Metal"' --level error
-
-# Search recent logs
-log show --predicate 'subsystem == "com.apple.Metal"' --last 5m
-```
-
-### Programmatic validation log access
-
-If you're writing Metal code, `commandBuffer.logs` provides structured error info after completion — encoder label, debug location (file + line), and error classification.
-
-## 5. Metal Performance HUD
-
-Real-time overlay showing FPS, frame time, GPU time, memory usage.
-
-### Enable
-
-```bash
-# Per-process via environment variable
-MTL_HUD_ENABLED=1 /path/to/your/app
-
-# System-wide (all Metal apps)
-/bin/launchctl setenv MTL_HUD_ENABLED 1
-# Disable:
-/bin/launchctl unsetenv MTL_HUD_ENABLED
-
-# Enable HUD data logging to syslog
-MTL_HUD_ENABLED=1 MTL_HUD_LOGGING_ENABLED=1 /path/to/your/app
-```
-
-### Parse HUD log data
-
-When `MTL_HUD_LOGGING_ENABLED=1`, metrics are logged to the system log:
-
-```bash
-# Stream HUD metrics
-log stream --predicate 'subsystem == "com.apple.Metal" AND category == "HUD"'
-
-# Export recent HUD data
-log show --predicate 'subsystem == "com.apple.Metal" AND category == "HUD"' --last 1m
-```
-
-HUD metrics include: FPS, present interval (frame time), GPU time, process memory, GPU memory, display refresh rate, direct vs composited rendering path.
-
-## 6. Programmatic Frame Capture (.gputrace)
-
-Capture Metal frames to `.gputrace` files without Xcode attached, then inspect buffer/texture data from CLI.
-
-### Via environment variables (MoltenVK / any Metal app)
-
-```bash
-# For Vulkan apps via MoltenVK:
-export METAL_CAPTURE_ENABLED=1
-export MVK_CONFIG_AUTO_GPU_CAPTURE_SCOPE=2 # 1=device lifecycle, 2=first frame
-export MVK_CONFIG_AUTO_GPU_CAPTURE_OUTPUT_FILE=/tmp/capture.gputrace
-/path/to/vulkan/app
-
-# For native Metal apps (requires Info.plist MetalCaptureEnabled=true
-# or METAL_CAPTURE_ENABLED=1 environment variable):
-METAL_CAPTURE_ENABLED=1 /path/to/metal/app
-```
-
-### Via MTLCaptureManager (in-app)
-
-For apps you control, add capture support using MTLCaptureManager. See `capture_frame.swift` for a complete example. The key pattern for adding capture to an existing app:
-
-```swift
-let captureManager = MTLCaptureManager.shared()
-if captureManager.supportsDestination(.gpuTraceDocument) {
- let descriptor = MTLCaptureDescriptor()
- descriptor.captureObject = device
- descriptor.destination = .gpuTraceDocument
- descriptor.outputURL = URL(fileURLWithPath: "./capture.gputrace")
- try captureManager.startCapture(with: descriptor)
-
- // ... encode and submit Metal work ...
-
- captureManager.stopCapture()
-}
-```
-
-Run with: `METAL_CAPTURE_ENABLED=1 ./your_app`
-
-### Open .gputrace in Xcode
-
-```bash
-open /tmp/capture.gputrace
-# Opens in Xcode's Metal Debugger with full inspection capabilities:
-# - Draw call list and stepping
-# - Pipeline state at each draw
-# - Texture/buffer inspection
-# - Shader debugging
-# - Performance profiling
-# - Dependency viewer
-```
-
-### Autonomous Metal Debugging Workflow
-
-When debugging a Metal rendering issue, Claude executes this workflow autonomously. It maximizes signal by gathering data from multiple sources in parallel, then branches based on what data is actually available.
-
-#### Step 1: Gather all signal sources (run in parallel)
-
-Launch these data-gathering steps simultaneously — they are independent:
-
-```bash
-# A. Screenshot the app's rendered output
-./build_and_run.sh --screenshot # preferred: in-app framebuffer readback
-# OR: screencapture -w -x output.png # fallback: macOS window capture
-
-# B. Capture .gputrace (programmatic)
-METAL_CAPTURE_ENABLED=1 ./your_app # produces capture.gputrace
-
-# C. Compile shaders with maximum warnings
-xcrun -sdk macosx metal -c -Weverything Shaders.metal -o /dev/null 2>&1
-
-# D. Read source code — Claude reads .metal files and Swift renderer code directly
-```
-
-#### Step 2: Analyze the capture
-
-```bash
-# 2a. List all resources and shader functions
-cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace
-
-# 2b. Check what data files exist in the capture
-ls capture.gputrace/MTLBuffer-* 2>/dev/null && echo "BUFFER DATA AVAILABLE" || echo "NO BUFFER FILES"
-ls capture.gputrace/MTLTexture-* 2>/dev/null && echo "TEXTURE DATA AVAILABLE" || echo "NO TEXTURE FILES"
-```
-
-**Branch on buffer availability:**
-
-- **MTLBuffer files exist** (Xcode-initiated captures): Parse buffer data directly
- ```bash
- cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace --buffer "Vertex" --layout float4 --index 0-10
- cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace --dump-all
- ```
-- **No MTLBuffer files** (typical for programmatic captures): Fall back to source code analysis
- - Read the Swift/ObjC code that creates and fills buffers
- - Read the `.metal` shader code that consumes the buffers
- - The capture metadata still tells you what resources exist and what shaders run — use this as an inventory to guide source code reading
-- **MTLTexture files exist**: Read raw texture data for render target analysis
-
-#### Step 3: Diagnose from all available sources
-
-Combine signal from every source gathered in Steps 1–2. Each source catches different bug categories:
-
-| Source | What it reveals | Bug categories |
-|--------|----------------|----------------|
-| **Screenshot** | Visual output | Wrong colors, flipped geometry, missing faces, transparency issues, blank screen |
-| **Source code** (.metal + Swift) | Logic and data flow | Wrong vertex data, shader math errors, incorrect pipeline config, buffer layout mismatches |
-| **Capture metadata** | Resource inventory | Confirms what buffers/textures exist, shader function names, resource labels |
-| **Shader compilation warnings** | Static analysis | Unused variables, implicit conversions, potential precision issues |
-| **Validation errors** (if enabled) | API correctness | Mismatched formats, missing bindings, out-of-bounds access, shader faults |
-
-**Decision logic when buffer data is unavailable:**
-- If no `MTLBuffer` files → Claude reads source code to understand vertex data, uniform values, and buffer layouts
-- If no `MTLTexture` files → Claude uses screenshot for visual inspection of rendered output
-- Programmatic captures always provide: resource labels, shader function names, texture snapshots (render targets)
-
-#### Step 4: Fix and verify
-
-```bash
-# 1. Apply fixes to source code (.metal shaders, Swift renderer)
-
-# 2. Rebuild and screenshot
-./build_and_run.sh --screenshot
-
-# 3. Compare before/after visually (Claude reads output.png)
-
-# 4. If still wrong, repeat from Step 1
-```
-
-The loop continues until the screenshot shows correct output or the user is satisfied.
-
-### Label Injection
-
-Claude should ensure every Metal object has a `.label` set in source code. Labels are preserved in `.gputrace` captures and enable resource identification from CLI.
-
-```swift
-// Add .label to every MTLBuffer, MTLTexture, MTLCommandBuffer, MTLCommandEncoder
-vertexBuffer.label = "Vertex Buffer"
-colorBuffer.label = "Color Buffer"
-uniformBuffer.label = "Uniform Buffer"
-commandBuffer.label = "Frame \(frameNumber)"
-encoder.label = "Main Render Pass"
-```
-
-**Rules:**
-- Use descriptive names with spaces (e.g., "Particle Buffer" not "particleBuf")
-- Labels with spaces are reliably distinguishable from binary data in the `.gputrace`
-- Never remove existing labels — only add missing ones
-- Add labels to every object that lacks one
-
-### Reference: gpu_trace capture usage
-
-```bash
-# List all captured resources with their labels
-cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace
-
-# Read specific buffer by label (partial match works)
-cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace --buffer "Color Output" --layout float4 --index 100
-
-# Read compound struct (e.g., Particle = position + velocity + color)
-cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace --buffer "Particle" --layout "float4,float4,float4" --index 0-10
-
-# Dump summary statistics for all buffers
-cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace --dump-all
-
-# Output as JSON
-cd tools/gpu_trace && uv run gpu_trace capture capture.gputrace --buffer "Color Output" --layout float4 --index 100 --json
-```
-
-#### Supported layout types
-
-| Layout | Bytes | Example |
-|--------|-------|---------|
-| `float` | 4 | Scalar energy, distance, time |
-| `float2` | 8 | UV coordinates |
-| `float3` | 12 | Position, normal (without padding) |
-| `float4` | 16 | RGBA color, position+mass, SIMD4 |
-| `uint32` | 4 | Index, count |
-| `int32` | 4 | Signed integer |
-| `half4` | 8 | Half-precision RGBA |
-| `float4,float4,float4` | 48 | Compound struct (e.g., Particle) |
-
-### Reference: .gputrace internal structure
-
-| File | Format | Contents |
-|------|--------|----------|
-| `MTLBuffer-{id}-{snap}` | Raw binary (IEEE 754 LE) | GPU buffer memory snapshot |
-| `MTLTexture-{id}-{snap}` | Raw binary | GPU texture memory snapshot |
-| `metadata` | Binary plist | Session UUID, API, device info |
-| `device-resources-*` | MTSP binary | Resource registry with labels, shader names |
-| `store0` | zlib-compressed MTSP | Encoded command buffer data |
-| `capture` / `unsorted-capture` | MTSP binary | Metal API call sequences |
-| `index` | Custom (`xdic` magic) | File table / hash table |
-
-### Reference: CLI capabilities by capture type
-
-| Capability | CLI (programmatic capture) | CLI (Xcode-initiated capture) | Xcode GUI |
-|-----------|---------------------------|------------------------------|-----------|
-| List resources with labels | ✅ | ✅ | ✅ |
-| List shader function names | ✅ | ✅ | ✅ |
-| Read texture/render target data | ✅ (MTLTexture files present) | ✅ | ✅ |
-| Read buffer contents at index | ❌ (no MTLBuffer files) | ✅ | ✅ |
-| Buffer statistics (min/max/mean) | ❌ | ✅ | ❌ (manual) |
-| Draw call stepping | ❌ | ❌ | ✅ |
-| Pixel history | ❌ | ❌ | ✅ |
-| Shader step-through | ❌ | ❌ | ✅ |
-| Pipeline state inspection | ❌ | ❌ | ✅ |
-| Dependency viewer | ❌ | ❌ | ✅ |
-| Command buffer replay | ❌ | ❌ | ✅ |
-
-### Reference: Screenshot capture techniques
-
-#### Method 1: In-app auto-screenshot (Preferred)
-
-The most reliable approach is to add a `--screenshot` mode to the app itself. This renders a few frames, reads back the framebuffer texture, and saves it as a PNG — no external tools required.
-
-**Swift code to add to your Metal app:**
-
-```swift
-// Add to your renderer class:
-static func saveTexture(_ texture: MTLTexture, to path: String) {
- let w = texture.width, h = texture.height
- let bytesPerRow = w * 4
- var pixels = [UInt8](repeating: 0, count: h * bytesPerRow)
- texture.getBytes(&pixels, bytesPerRow: bytesPerRow,
- from: MTLRegion(origin: MTLOrigin(), size: MTLSize(width: w, height: h, depth: 1)),
- mipmapLevel: 0)
- // BGRA → RGBA, force alpha to 255 for screenshot visibility
- for i in stride(from: 0, to: pixels.count, by: 4) {
- let tmp = pixels[i]
- pixels[i] = pixels[i + 2]
- pixels[i + 2] = tmp
- pixels[i + 3] = 255
- }
- let rep = NSBitmapImageRep(bitmapDataPlanes: nil, pixelsWide: w, pixelsHigh: h,
- bitsPerSample: 8, samplesPerPixel: 4, hasAlpha: true,
- isPlanar: false, colorSpaceName: .deviceRGB,
- bytesPerRow: bytesPerRow, bitsPerPixel: 32)!
- memcpy(rep.bitmapData!, &pixels, pixels.count)
- let data = rep.representation(using: .png, properties: [:])!
- try! data.write(to: URL(fileURLWithPath: path))
-}
-```
-
-**Usage in the draw loop** (save after a few frames, then exit):
-
-```swift
-// In draw(in:) after cb.present(drawable) and cb.commit():
-if frameCount == 3 && screenshotMode {
- cb.waitUntilCompleted()
- Self.saveTexture(drawable.texture, to: "./output.png")
- print("Screenshot saved: ./output.png")
- DispatchQueue.main.async { NSApp.terminate(nil) }
-}
-```
-
-**Important**: Set `metalView.framebufferOnly = false` before rendering to allow texture readback.
-
-#### Method 2: macOS screencapture (Fallback)
-
-If the app doesn't support `--screenshot`, use macOS `screencapture` to capture the window:
-
-```bash
-# Launch the app in background
-./your_app &
-APP_PID=$!
-sleep 1
-
-# Capture the app's window (-w = frontmost window, -x = no sound)
-screencapture -w -x /tmp/screenshot.png
-
-# Stop the app
-kill $APP_PID 2>/dev/null
-```
-
-## 7. Metal Shader Compilation
-
-Compile, validate, and inspect Metal shaders from the command line.
-
-### Compile shaders
-
-```bash
-# Compile .metal to intermediate representation (.air)
-xcrun -sdk macosx metal -c MyShader.metal -o MyShader.air
-
-# With warnings and debug info
-xcrun -sdk macosx metal -c -gline-tables-only -Weverything MyShader.metal -o MyShader.air
-
-# Archive
-xcrun -sdk macosx metal-ar rcs MyShader.metalar MyShader.air
-
-# Link into a Metal library (.metallib)
-xcrun -sdk macosx metallib MyShader.metalar -o MyShader.metallib
-
-# One-step compile + link
-xcrun -sdk macosx metal MyShader.metal -o MyShader.metallib
-```
-
-### Cross-compile for specific targets
-
-```bash
-# List available GPU architectures
-xcrun metal-arch
-
-# Target specific platform
-xcrun -sdk iphoneos metal -c Shader.metal -o Shader.air
-xcrun -sdk macosx metal -c -target air64-apple-macos14 Shader.metal -o Shader.air
-```
-
-### HLSL to Metal (via shader converter)
-
-```bash
-# Convert HLSL → DXIL → Metal IR
-dxc shaders.hlsl -T vs_6_0 -E MainVS -Fo vertex.dxil
-metal-shaderconverter vertex.dxil -o vertex.metallib
-```
-
-### Validate shaders without running
-
-```bash
-# Compile with all warnings — catches issues at build time
-xcrun -sdk macosx metal -c -Weverything -Werror MyShader.metal -o /dev/null
-
-# Check Metal version compatibility
-xcrun -sdk macosx metal -c -std=metal3.0 MyShader.metal -o /dev/null
-```
-
-## 8. GPU Device Information
-
-```bash
-# System-level GPU info
-system_profiler SPDisplaysDataType
-
-# Metal feature set / GPU family (grep for relevant info)
-system_profiler SPDisplaysDataType | grep -E "Chipset|Metal|VRAM|GPU"
-```
-
-## 9. Debugging Recipes
-
-### Recipe: Profile GPU performance
-
-```bash
-# 1. Record a Metal System Trace
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --time-limit 10s \
- --output perf.trace \
- --launch -- /path/to/app
-
-# 2. See what data is available
-xcrun xctrace export --input perf.trace --toc
-
-# 3. Export GPU driver events
-xcrun xctrace export --input perf.trace \
- --output gpu_events.xml \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-
-# 4. Export GPU counters (if available)
-xcrun xctrace export --input perf.trace \
- --output gpu_counters.xml \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="gpu-counter-intervals"]'
-
-# 5. Open in Instruments for visual analysis
-open perf.trace
-```
-
-### Recipe: Find shader compilation errors
-
-```bash
-# 1. Compile with maximum warnings
-xcrun -sdk macosx metal -c -Weverything -Werror MyShader.metal -o /dev/null 2>&1
-
-# 2. If it compiles, check at runtime with validation
-MTL_SHADER_VALIDATION=1 /path/to/app 2>&1 | tee shader_errors.log
-
-# 3. Check logs for shader validation errors
-log show --predicate 'subsystem == "com.apple.Metal"' --last 5m --level error
-```
-
-### Recipe: Detect Metal API misuse
-
-```bash
-# 1. Run with API validation
-MTL_DEBUG_LAYER=1 /path/to/app 2>&1 | tee api_errors.log
-
-# 2. Record with validation for post-mortem analysis
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --env MTL_DEBUG_LAYER=1 \
- --time-limit 10s \
- --output validated.trace \
- --launch -- /path/to/app
-
-# 3. Check for errors
-grep -i "error\|warning\|invalid\|violation" api_errors.log
-```
-
-### Recipe: Monitor real-time performance
-
-```bash
-# 1. Enable HUD with logging
-MTL_HUD_ENABLED=1 MTL_HUD_LOGGING_ENABLED=1 /path/to/app &
-APP_PID=$!
-
-# 2. Stream performance data
-log stream --predicate 'subsystem == "com.apple.Metal" AND category == "HUD"' &
-
-# 3. When done, stop
-kill $APP_PID
-```
-
-### Recipe: Capture a frame for Xcode debugging
-
-```bash
-# For Vulkan apps (via MoltenVK)
-METAL_CAPTURE_ENABLED=1 \
-MVK_CONFIG_AUTO_GPU_CAPTURE_SCOPE=2 \
-MVK_CONFIG_AUTO_GPU_CAPTURE_OUTPUT_FILE=/tmp/frame.gputrace \
-/path/to/vulkan/app
-
-# Open in Xcode Metal Debugger
-open /tmp/frame.gputrace
-```
-
-### Recipe: Debug Metal rendering issues (visual + data)
-
-Use the **Autonomous Metal Debugging Workflow** in Section 6. It covers screenshot capture, .gputrace analysis, source code reading, shader compilation, and the fix/verify loop — with fallbacks when buffer data is unavailable.
-
-### Recipe: Compare performance before/after a change
-
-```bash
-# 1. Record baseline
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --time-limit 10s \
- --output before.trace \
- --launch -- /path/to/app
-
-# 2. Make your change, rebuild
-
-# 3. Record after
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --time-limit 10s \
- --output after.trace \
- --launch -- /path/to/app
-
-# 4. Export both and compare
-xcrun xctrace export --input before.trace --output before.xml \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-xcrun xctrace export --input after.trace --output after.xml \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-
-# 5. Diff the XML or parse with Python for numerical comparison
-```
-
-## 10. Output Size Management
-
-Metal System Traces can be very large. Follow these rules:
-
-1. **Use `--time-limit`**: Always limit recording duration (5-10s is usually enough)
-2. **Export specific schemas**: Use `--xpath` to extract only the data you need
-3. **Use `--toc` first**: Understand what's in the trace before bulk-exporting
-4. **Pipe through filters**: Use `grep`, `xmllint`, or Python to extract specific data
-5. **Clean up traces**: `.trace` bundles can be 100MB+; delete when done
-
-## 11. Limitations vs RenderDoc
-
-| Capability | Metal Skill | RenderDoc Skill |
-|-----------|-------------|-----------------|
-| Frame capture | ✅ .gputrace | ✅ .rdc (full CLI) |
-| GPU profiling | ✅ xctrace + export | ⚠️ GPU counters if available |
-| Shader compilation | ✅ xcrun metal | N/A (different workflow) |
-| Validation errors | ✅ env vars + log stream | ✅ API validation layer |
-| Performance HUD | ✅ MTL_HUD_ENABLED | N/A |
-| Buffer inspection (CLI) | ⚠️ Xcode-initiated captures only | ✅ rdc buffer |
-| Buffer statistics | ⚠️ Xcode-initiated captures only | ❌ |
-| Draw call inspection | ❌ Xcode only | ✅ rdc draws/pipeline |
-| Pixel history | ❌ Xcode only | ✅ rdc pixel |
-| Shader step-through | ❌ Xcode only | ✅ rdc debug pixel |
-| Render target export | ❌ Xcode only | ✅ rdc rt → PNG |
-| Texture readback (CLI) | ⚠️ MTLTexture files (if present) | ✅ Full rdc-cli API |
-| Visual output inspection | ✅ in-app saveTexture or screencapture | ❌ (headless only) |
-
-**Key difference**: Metal debugging is split between CLI (profiling/validation/buffer inspection) and GUI (Xcode for draw calls, pixel history, shader debugging). The `gpu_trace capture` command partially bridges this gap by enabling CLI buffer/texture data inspection from `.gputrace` captures. RenderDoc still provides more complete CLI access.
-
-## Command Reference
-
-For the complete xctrace command reference, see [references/xctrace-quick-ref.md](references/xctrace-quick-ref.md).
-
-For extended debugging recipes, see [references/debugging-recipes.md](references/debugging-recipes.md).
diff --git a/agents/skills/metal-gpu-debug/references/debugging-recipes.md b/agents/skills/metal-gpu-debug/references/debugging-recipes.md
deleted file mode 100644
index ab5d234f9..000000000
--- a/agents/skills/metal-gpu-debug/references/debugging-recipes.md
+++ /dev/null
@@ -1,361 +0,0 @@
-# Metal GPU Debugging Recipes
-
-Extended debugging workflows with expected outputs and step-by-step procedures.
-
-## Recipe 1: GPU Performance Profiling
-
-**Symptoms**: Low FPS, high frame time, stuttering, thermal throttling.
-
-### Step-by-step
-
-```bash
-# 1. Record and analyze in one step (with hardware counters)
-cd tools/gpu_trace && uv run gpu_trace run --gpu-counters --time-limit 10s -- /path/to/app
-
-# Or analyze a pre-recorded trace:
-cd tools/gpu_trace && uv run gpu_trace analyze perf.trace -v
-
-# Export to JSON for custom analysis:
-cd tools/gpu_trace && uv run gpu_trace analyze perf.trace --json perf.json
-
-# Open in Instruments for visual timeline
-open perf.trace
-```
-
-`gpu_trace` automatically exports all schemas in parallel, parses the XML, computes per-kernel timing, correlates hardware counters, and detects bottlenecks.
-
-**Performance red flags:**
-- GPU busy time > 16ms per frame (60fps target) or > 8ms (120fps target)
-- Wire memory events with large sizes (indicates heavy resource allocation per frame)
-- Shader execution intervals dominating GPU timeline (complex shaders)
-- Gaps between GPU submissions (CPU bottleneck)
-
----
-
-## Recipe 2: Shader Compilation Errors
-
-**Symptoms**: App crashes on launch, black screen, missing effects.
-
-### Step-by-step
-
-```bash
-# 1. Compile shader with maximum diagnostics
-xcrun -sdk macosx metal -c \
- -gline-tables-only \
- -Weverything \
- -Werror \
- MyShader.metal -o /dev/null 2>&1
-```
-
-**Expected error output:**
-```
-MyShader.metal:42:15: error: use of undeclared identifier 'baseColor'
- float4 c = baseColor * albedo;
- ^
-MyShader.metal:18:3: warning: unused variable 'temp' [-Wunused-variable]
- float temp = 0.0;
- ^
-```
-
-```bash
-# 2. If compile succeeds but runtime fails, check Metal version
-xcrun -sdk macosx metal -c -std=metal3.0 MyShader.metal -o /dev/null 2>&1
-
-# 3. Check for runtime shader errors
-MTL_SHADER_VALIDATION=1 /path/to/app 2>&1 | head -100
-
-# 4. Stream shader validation errors
-log stream --predicate 'subsystem == "com.apple.Metal"' --level error &
-MTL_SHADER_VALIDATION=1 /path/to/app
-
-# 5. Build the full pipeline: .metal → .air → .metalar → .metallib
-xcrun -sdk macosx metal -c -gline-tables-only MyShader.metal -o MyShader.air
-xcrun -sdk macosx metal-ar rcs MyShader.metalar MyShader.air
-xcrun -sdk macosx metallib MyShader.metalar -o MyShader.metallib
-echo "Shader library built successfully: MyShader.metallib"
-```
-
-**Common shader errors:**
-| Error | Cause | Fix |
-|-------|-------|-----|
-| undeclared identifier | Missing variable/function | Check spelling, includes, scope |
-| no matching function | Wrong argument types | Verify types match function signature |
-| cannot convert | Type mismatch | Use explicit casts (float4, half4, etc.) |
-| address space mismatch | Wrong buffer qualifier | Check device/constant/threadgroup qualifiers |
-| exceeds max total threads | Threadgroup size too large | Reduce threadgroup dimensions |
-
----
-
-## Recipe 3: Metal API Validation Errors
-
-**Symptoms**: Crashes, undefined behavior, incorrect rendering.
-
-### Step-by-step
-
-```bash
-# 1. Run with full API validation
-MTL_DEBUG_LAYER=1 /path/to/app 2>&1 | tee validation.log
-
-# 2. Check for errors
-grep -i "error\|warning\|invalid\|violation\|failed" validation.log
-
-# 3. For deeper validation, add shader validation too
-MTL_DEBUG_LAYER=1 MTL_SHADER_VALIDATION=1 /path/to/app 2>&1 | tee full_validation.log
-
-# 4. Stream structured Metal log data
-log stream --predicate 'subsystem == "com.apple.Metal"' --level error
-```
-
-**Expected validation error format:**
-```
--[MTLDebugRenderCommandEncoder setVertexBuffer:offset:atIndex:]:
- Execution of the command buffer was aborted due to an error during execution.
- Caused GPU Address Fault (0x00000001)
-```
-
-**Common validation issues:**
-| Error Pattern | Meaning | Fix |
-|--------------|---------|-----|
-| GPU Address Fault | Out-of-bounds buffer access | Check buffer sizes, offsets |
-| Shader Validation Error | GPU-side out-of-bounds or race | Check array indices, threadgroup sync |
-| Invalid Resource | Using deleted/nil resource | Check resource lifetimes |
-| Incompatible Pixel Format | Format mismatch | Verify texture formats match pipeline |
-| Exceeded max | Resource limit hit | Reduce resource usage |
-
----
-
-## Recipe 4: Real-Time Performance Monitoring
-
-**Symptoms**: Need ongoing performance data during development.
-
-### Step-by-step
-
-```bash
-# 1. Launch app with HUD and logging
-MTL_HUD_ENABLED=1 MTL_HUD_LOGGING_ENABLED=1 /path/to/app &
-APP_PID=$!
-
-# 2. Collect HUD data for analysis
-log show --predicate 'subsystem == "com.apple.Metal" AND category == "HUD"' \
- --last 30s > hud_data.log
-
-# 3. Parse HUD log for key metrics
-python3 << 'EOF'
-import re
-
-with open('hud_data.log') as f:
- for line in f:
- # Extract FPS, GPU time, memory from HUD log entries
- if 'FPS' in line or 'GPU' in line or 'Memory' in line:
- print(line.strip())
-EOF
-
-# 4. When done
-kill $APP_PID
-```
-
-**HUD overlay shows:**
-- FPS (frames per second)
-- Present interval / frame time (ms)
-- GPU time (ms)
-- Process memory (MB)
-- GPU memory (MB)
-- Display refresh rate
-- Rendering path (direct vs composited)
-
----
-
-## Recipe 5: Frame Capture for Xcode Debugging
-
-**Symptoms**: Visual bugs that need draw-call-level inspection.
-
-### For native Metal apps
-
-```bash
-# 1. Ensure capture is enabled (one of):
-# a) Set METAL_CAPTURE_ENABLED=1 environment variable
-# b) Add MetalCaptureEnabled=true to Info.plist
-# c) Use MTLCaptureManager in code
-
-# 2. For MoltenVK / Vulkan apps:
-METAL_CAPTURE_ENABLED=1 \
-MVK_CONFIG_AUTO_GPU_CAPTURE_SCOPE=2 \
-MVK_CONFIG_AUTO_GPU_CAPTURE_OUTPUT_FILE=/tmp/capture.gputrace \
-/path/to/vulkan/app
-
-# 3. Wait for capture to complete (app will likely freeze briefly)
-echo "Capture saved to /tmp/capture.gputrace"
-
-# 4. Open in Xcode
-open /tmp/capture.gputrace
-```
-
-**In Xcode Metal Debugger you can:**
-- Step through draw calls
-- Inspect pipeline state at any draw
-- View bound textures and buffers
-- Debug shaders line-by-line
-- Check pixel history
-- View GPU timeline
-- Inspect acceleration structures (ray tracing)
-- Analyze memory usage
-
----
-
-## Recipe 6: Compare Performance Across Changes
-
-**Symptoms**: Need to verify a change improved or didn't regress performance.
-
-### Step-by-step
-
-```bash
-# 1. Record baseline
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --time-limit 10s \
- --no-prompt \
- --output baseline.trace \
- --launch -- /path/to/app_before
-
-# 2. Record after changes
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --time-limit 10s \
- --no-prompt \
- --output changed.trace \
- --launch -- /path/to/app_after
-
-# 3. Export both
-for trace in baseline changed; do
- xcrun xctrace export \
- --input ${trace}.trace \
- --output ${trace}_events.xml \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-done
-
-# 4. Compare with Python
-python3 << 'EOF'
-import xml.etree.ElementTree as ET
-
-def count_events(xmlfile):
- tree = ET.parse(xmlfile)
- rows = tree.getroot().findall('.//row')
- return len(rows)
-
-baseline = count_events('baseline_events.xml')
-changed = count_events('changed_events.xml')
-
-print(f"Baseline: {baseline} events")
-print(f"Changed: {changed} events")
-print(f"Delta: {changed - baseline:+d} events ({(changed/baseline - 1)*100:+.1f}%)")
-EOF
-```
-
----
-
-## Recipe 7: Debug Vulkan App on macOS via MoltenVK
-
-**Symptoms**: Vulkan app has issues only on macOS.
-
-### Step-by-step
-
-```bash
-# 1. Enable MoltenVK logging
-export MVK_CONFIG_LOG_LEVEL=3 # 0=off, 1=error, 2=warning, 3=info, 4=debug
-
-# 2. Run with Metal validation
-MTL_DEBUG_LAYER=1 \
-MTL_SHADER_VALIDATION=1 \
-MVK_CONFIG_LOG_LEVEL=3 \
-/path/to/vulkan/app 2>&1 | tee moltenvk.log
-
-# 3. Capture a frame for Xcode debugging
-METAL_CAPTURE_ENABLED=1 \
-MVK_CONFIG_AUTO_GPU_CAPTURE_SCOPE=2 \
-MVK_CONFIG_AUTO_GPU_CAPTURE_OUTPUT_FILE=/tmp/vk_capture.gputrace \
-/path/to/vulkan/app
-
-# 4. Profile with xctrace
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --env MVK_CONFIG_LOG_LEVEL=3 \
- --time-limit 10s \
- --output vk_trace.trace \
- --launch -- /path/to/vulkan/app
-
-# 5. Open trace or capture
-open /tmp/vk_capture.gputrace # For state debugging
-open vk_trace.trace # For profiling
-```
-
----
-
-## Recipe 8: Profile an iOS Device
-
-**Symptoms**: Need to profile Metal performance on a physical iPhone/iPad.
-
-### Step-by-step
-
-```bash
-# 1. List connected devices
-xcrun xctrace list devices
-# Look for your device name and UDID
-
-# 2. Record (app must be running or installed on device)
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --device 'iPhone 15 Pro' \
- --attach MyApp \
- --time-limit 10s \
- --output ios_profile.trace
-
-# 3. Export and analyze — same as macOS
-xcrun xctrace export --input ios_profile.trace --toc
-
-xcrun xctrace export --input ios_profile.trace \
- --output ios_gpu_events.xml \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-
-# 4. Parse with gpu_trace
-cd tools/gpu_trace && uv run gpu_trace analyze ios_profile.trace -v
-
-# 5. Or open in Instruments
-open ios_profile.trace
-```
-
-**iOS-specific tips:**
-- Device must be unlocked and trusted
-- Use `--attach` with process name for already-running apps
-- Use `--launch com.bundle.id` to launch and trace
-- Apple Silicon iPhones (A14+) have rich GPU counters
-- Thermal throttling is more common on iOS — check for performance drops over time
-
----
-
-## Recipe 9: System-Wide GPU Monitoring
-
-**Symptoms**: Need to identify which process is consuming GPU resources.
-
-### Step-by-step
-
-```bash
-# 1. Trace all processes
-xcrun xctrace record \
- --template 'Metal System Trace' \
- --all-processes \
- --time-limit 10s \
- --output system.trace
-
-# 2. Check which processes are in the trace
-xcrun xctrace export --input system.trace \
- --xpath '/trace-toc/run[@number="1"]/processes'
-
-# 3. Export GPU events for a specific process
-# (after identifying process name from step 2)
-xcrun xctrace export --input system.trace \
- --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]' \
- --output all_gpu.xml
-
-# 4. Open in Instruments for per-process breakdown
-open system.trace
-```
diff --git a/agents/skills/metal-gpu-debug/references/xctrace-quick-ref.md b/agents/skills/metal-gpu-debug/references/xctrace-quick-ref.md
deleted file mode 100644
index add29df94..000000000
--- a/agents/skills/metal-gpu-debug/references/xctrace-quick-ref.md
+++ /dev/null
@@ -1,213 +0,0 @@
-# xctrace Command Quick Reference
-
-## `xctrace record`
-
-Record an Instruments trace.
-
-**Options:**
-
-| Flag | Help | Type | Default |
-|------|------|------|---------|
-| `--template` | Template name or path | text | (required) |
-| `--output` | Output .trace path | path | auto-generated |
-| `--device` | Device name or UDID | text | local |
-| `--time-limit` | Max recording time (e.g., 5s, 1m) | duration | unlimited |
-| `--all-processes` | Trace all processes | flag | |
-| `--attach` | Attach to PID or process name | text | |
-| `--launch` | Launch and trace command (must be last) | command | |
-| `--target-stdin` | Redirect stdin (- or file) | text | |
-| `--target-stdout` | Redirect stdout (- or file) | text | |
-| `--env` | Set env var for launched process (VAR=value) | text | |
-| `--package` | Install Instruments Package for duration | file | |
-| `--no-prompt` | Skip confirmation prompts | flag | |
-| `--notify-tracing-started` | Send Darwin notification when started | text | |
-| `--append-run` | Append run to existing trace | flag | |
-| `--window` | Windowed (ring buffer) mode | duration | |
-
-## `xctrace export`
-
-Export data from a .trace file to XML.
-
-**Options:**
-
-| Flag | Help | Type | Default |
-|------|------|------|---------|
-| `--input` | Input .trace file | file | (required) |
-| `--output` | Output file (default: stdout) | path | stdout |
-| `--toc` | Export table of contents | flag | |
-| `--xpath` | XPath query to select export entities | expression | |
-| `--har` | Export as HTTP Archive (if HTTP Traffic present) | flag | |
-
-### XPath patterns for Metal data
-
-```bash
-# Table of contents (always start here)
---toc
-
-# Metal driver events (GPU work intervals, wire memory)
---xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-driver-event-intervals"]'
-
-# GPU hardware counters
---xpath '/trace-toc/run[@number="1"]/data/table[@schema="gpu-counter-intervals"]'
-
-# Metal GPU execution intervals
---xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-gpu-intervals"]'
-
-# CPU time profile
---xpath '/trace-toc/run[@number="1"]/data/table[@schema="time-profile"]'
-
-# All tables in first run
---xpath '/trace-toc/run[@number="1"]/data/table'
-
-# Process info
---xpath '/trace-toc/run[@number="1"]/processes'
-
-# Specific process
---xpath '/trace-toc/run[@number="1"]/processes/process[@name="MyApp"]'
-```
-
-**NOTE**: Available schemas vary by template, Xcode version, and GPU. Always check `--toc` first.
-
-## `xctrace import`
-
-Import supported file formats into .trace.
-
-**Options:**
-
-| Flag | Help | Type | Default |
-|------|------|------|---------|
-| `--input` | Input file to import | file | (required) |
-| `--output` | Output .trace path | path | auto-generated |
-| `--template` | Template for import | text | |
-| `--package` | Instruments Package for import | file | |
-
-## `xctrace remodel`
-
-Remodel trace using installed packages.
-
-**Options:**
-
-| Flag | Help | Type | Default |
-|------|------|------|---------|
-| `--input` | Input .trace file | file | (required) |
-| `--output` | Output .trace path | path | |
-| `--package` | Instruments Package | file | |
-
-## `xctrace symbolicate`
-
-Symbolicate a trace file.
-
-**Options:**
-
-| Flag | Help | Type | Default |
-|------|------|------|---------|
-| `--input` | Input .trace file | file | (required) |
-| `--output` | Output .trace path | path | |
-| `--dsym` | Debug symbols file | file | |
-
-## `xctrace list`
-
-List available resources.
-
-**Subcommands:**
-
-| Subcommand | Description |
-|-----------|-------------|
-| `devices` | List available devices |
-| `templates` | List available recording templates |
-| `instruments` | List available instruments |
-
-## Metal Shader Toolchain Commands
-
-### `xcrun metal`
-
-Metal shader compiler.
-
-| Flag | Help |
-|------|------|
-| `-c` | Compile to .air (intermediate representation) |
-| `-o` | Output file path |
-| `-sdk macosx` | Target macOS SDK (also: iphoneos, appletvos) |
-| `-target air64-apple-macos14` | Target specific platform/version |
-| `-std=metal3.0` | Metal language standard version |
-| `-gline-tables-only` | Generate line-level debug info |
-| `-Weverything` | Enable all warnings |
-| `-Werror` | Treat warnings as errors |
-| `-I <path>` | Add include search path |
-| `-D <macro>` | Define preprocessor macro |
-
-### `xcrun metal-ar`
-
-Metal archive tool (create static libraries of .air files).
-
-```bash
-xcrun -sdk macosx metal-ar rcs output.metalar input1.air input2.air
-```
-
-### `xcrun metallib`
-
-Metal linker (create .metallib from .air or .metalar).
-
-```bash
-xcrun -sdk macosx metallib input.metalar -o output.metallib
-```
-
-### `xcrun metal-arch`
-
-List available Metal GPU architecture targets.
-
-### `metal-shaderconverter`
-
-Convert HLSL (via DXIL) to Metal IR.
-
-```bash
-metal-shaderconverter input.dxil -o output.metallib
-```
-
-## Metal Environment Variables
-
-### Debugging
-
-| Variable | Values | Effect |
-|----------|--------|--------|
-| `MTL_DEBUG_LAYER` | 1 | Enable API validation layer |
-| `MTL_SHADER_VALIDATION` | 1 | Enable GPU shader validation |
-| `METAL_CAPTURE_ENABLED` | 1 | Allow programmatic .gputrace capture |
-| `METAL_DEVICE_WRAPPER_TYPE` | 1 | Enable Metal device wrapper (extended validation) |
-
-### Performance HUD
-
-| Variable | Values | Effect |
-|----------|--------|--------|
-| `MTL_HUD_ENABLED` | 1 | Show performance overlay |
-| `MTL_HUD_LOGGING_ENABLED` | 1 | Log HUD metrics to system log |
-
-### MoltenVK (Vulkan apps)
-
-| Variable | Values | Effect |
-|----------|--------|--------|
-| `MVK_CONFIG_AUTO_GPU_CAPTURE_SCOPE` | 1=device, 2=first frame | Auto-capture scope |
-| `MVK_CONFIG_AUTO_GPU_CAPTURE_OUTPUT_FILE` | path | .gputrace output path |
-| `MVK_CONFIG_LOG_LEVEL` | 0-4 | MoltenVK logging verbosity |
-
-## macOS Log Commands for Metal
-
-```bash
-# Stream Metal errors live
-log stream --predicate 'subsystem == "com.apple.Metal"' --level error
-
-# Stream all Metal messages
-log stream --predicate 'subsystem == "com.apple.Metal"'
-
-# Stream HUD performance data
-log stream --predicate 'subsystem == "com.apple.Metal" AND category == "HUD"'
-
-# Show recent Metal errors
-log show --predicate 'subsystem == "com.apple.Metal"' --last 5m --level error
-
-# Show recent Metal messages (all levels)
-log show --predicate 'subsystem == "com.apple.Metal"' --last 5m
-
-# Export to file
-log show --predicate 'subsystem == "com.apple.Metal"' --last 10m > metal_log.txt
-```
diff --git a/agents/skills/trace-model/SKILL.md b/agents/skills/trace-model/SKILL.md
deleted file mode 100644
index fa94e63e0..000000000
--- a/agents/skills/trace-model/SKILL.md
+++ /dev/null
@@ -1,20 +0,0 @@
----
-name: trace-model
-description: Validate all intermediate calculations during inference are correct using source of truth values
----
-
-Using traces.safetensors to validate all intermediate calculations after each layer are correct.
-
-## Instructions
-
-1. Setup model you want to test in ./crates/uzu/tests/tracer_test.rs or ./crates/uzu/tests/mod.rs
-
-2. Check the model folder to ensure traces.safetensors exists (it should be automatically downloaded alongside the model)
-
-3. Run
-
- ```
- cargo test --release --package uzu --test tracer_test -- test_tracer --exact --nocapture
- ```
-
-4. Check results to ensure number of violations are less than the limit
\ No newline at end of file
diff --git a/agents/skills/validate-inference/SKILL.md b/agents/skills/validate-inference/SKILL.md
deleted file mode 100644
index 322deaa14..000000000
--- a/agents/skills/validate-inference/SKILL.md
+++ /dev/null
@@ -1,20 +0,0 @@
----
-name: validate-inference
-description: Validate inference generates correct text
----
-
-Using tests to validate inference is correct and performance has not degraded.
-
-## Instructions
-
-1. Setup model you want to test in ./crates/uzu/tests/text_session_test.rs or ./crates/uzu/tests/mod.rs
-
-2. Remember that some models can be too large to fit in device RAM. Do not use these models and do not attempt to download them.
-
-3. Run
-
- ```
- cargo test --release --package uzu --test text_session_test -- test_text_session_base --exact --nocapture
- ```
-
-4. Check generated text is correct and readable and stats (processed_tokens_per_second for prefill and tokens_per_second for generate)
\ No newline at end of file