Skip to content

The only tool you need to know what is happening and how to fix it.

feat(skills): add performance skill evals (Closes #253)

Liyu3c3ed3e92aa8
diff --git a/skills/traceway/evals/README.md b/skills/traceway/evals/README.md
new file mode 100644
index 00000000..18676638
--- /dev/null
+++ b/skills/traceway/evals/README.md
@@ -0,0 +1,48 @@
+# Performance Skill Evals
+
+Scenario-based evals for the `traceway` skill's **Performance** flow
+(`/traceway perf <endpoint or symptom>`). Each eval hands an agent a
+synthetic-but-realistic Traceway telemetry snapshot and asks it to diagnose
+the bottleneck using the skill; the expected diagnosis is pinned to one row
+of the bottleneck checklist in `../performance.md`.
+
+## How to run an eval
+
+1. Give the agent the `traceway` skill (or the `traceway` CLI against a
+ staging instance seeded with the snapshot data).
+2. Paste the **Task** prompt from the scenario file verbatim.
+3. Grade the answer against **Expected diagnosis** and **Pass criteria**.
+ A pass requires naming the right checklist suspect *and* citing the
+ telemetry that confirms it — a correct guess with no evidence is a fail.
+
+The snapshots are static text, so evals also work offline: the agent should
+reason from the snapshot exactly as it would from live `traceway` output.
+
+## Coverage
+
+| ID | Area | Framework | Checklist suspect |
+|---|---|---|---|
+| `backend-go-select-n-plus-1` | backend | Go (Gin) | Select N+1 |
+| `backend-python-pool-exhaustion` | backend | Python (FastAPI) | Connection-pool exhaustion |
+| `backend-node-sequential-calls` | backend | Node.js (NestJS) | Sequential calls that could run in parallel |
+| `backend-missing-index` | backend | any (PostgreSQL) | Missing or unused index |
+| `backend-sync-background-task` | backend | Go (Gin) | Synchronous work that belongs in a background task |
+| `frontend-js-large-payloads` | frontend | browser JS | Large request/response payloads |
+| `frontend-js-no-cache` | frontend | browser JS | No cache / low hit ratio |
+| `mobile-ios-cold-start` | mobile | iOS | Cold start (synchronous init on launch) |
+| `mobile-android-gc-pauses` | mobile | Android | Memory pressure / GC pauses |
+| `mobile-flutter-shader-jank` | mobile | Flutter | Cold start / JIT / shader warmup |
+| `infra-cpu-saturation` | infrastructure | n/a | CPU saturation |
+| `infra-cache-stampede` | infrastructure | n/a | Cache stampede / thundering herd |
+
+## Adding a new eval
+
+1. Copy the file format below: header table, `## Telemetry snapshot`,
+ `## Task`, `## Expected diagnosis`, `## Pass criteria`.
+2. Build the snapshot from a real incident when possible; keep numbers
+ internally consistent (percentiles, span durations, and metric values
+ must agree with each other).
+3. Pin the expected diagnosis to exactly one `performance.md` checklist
+ row, and make the snapshot contain the confirming signal that row
+ asks for — plus one plausible distractor the agent must rule out.
+4. Add a row to the coverage table above.
diff --git a/skills/traceway/evals/backend-go-select-n-plus-1.md b/skills/traceway/evals/backend-go-select-n-plus-1.md
new file mode 100644
index 00000000..afbb1892
--- /dev/null
+++ b/skills/traceway/evals/backend-go-select-n-plus-1.md
@@ -0,0 +1,55 @@
+# Eval: backend-go-select-n-plus-1
+
+- **ID**: `backend-go-select-n-plus-1`
+- **Area**: backend
+- **Framework**: Go (Gin + PostgreSQL)
+- **Checklist item**: Select N+1 (`performance.md` → Database and data access)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "GET /api/orders" --since 24h --output json`:
+
+```json
+{ "endpoint": "GET /api/orders", "count": 48210, "p50": 412, "p95": 2380, "p99": 3110, "errorCount": 0 }
+```
+
+`traceway endpoints show <endpointId> --recorded-at <t>` — span waterfall of one
+p95 request (total 2.31s):
+
+```
+GET /api/orders ............ 2310ms
+└── db.query ............... 3ms SELECT * FROM orders WHERE user_id = $1 LIMIT 50
+└── db.query ............... 2ms SELECT * FROM order_items WHERE order_id = $1
+└── db.query ............... 2ms SELECT * FROM order_items WHERE order_id = $1
+... (347 more near-identical order_items spans, 1–3ms each)
+└── json.encode ............ 41ms
+```
+
+`traceway metrics query --name system.cpu.utilization --aggregation max --since 24h`:
+flat at 22–31% all day. p50/p95 have crept up ~15% per week for a month with
+no deploy in that window (`git log` clean).
+
+## Task
+
+> `GET /api/orders` p95 is 2.4s and climbing ~15% a week with no recent
+> deploy. Diagnose the bottleneck to a specific checklist cause and propose
+> the fix.
+
+## Expected diagnosis
+
+- **Root cause**: Select N+1 — the handler loads 50 orders, then issues one
+ `order_items` query per order (≈348 near-identical short spans); total DB
+ time dominates while each call is tiny.
+- **Key evidence**: repeated near-identical query spans in the waterfall;
+ flat CPU (rules out saturation); latency creeping with data growth and no
+ deploy (rules out a regression).
+- **Fix**: single JOIN / batched `WHERE order_id IN (...)` load, or a
+ dataloader.
+
+## Pass criteria
+
+- Names **Select N+1** (not "slow query" or "missing index" — the individual
+ queries are 1–3ms, so neither fits).
+- Cites the repeated spans *and* rules out saturation/regression from the
+ flat CPU and clean deploy history.
+- Proposes batching the item load.
diff --git a/skills/traceway/evals/backend-missing-index.md b/skills/traceway/evals/backend-missing-index.md
new file mode 100644
index 00000000..731f217f
--- /dev/null
+++ b/skills/traceway/evals/backend-missing-index.md
@@ -0,0 +1,53 @@
+# Eval: backend-missing-index
+
+- **ID**: `backend-missing-index`
+- **Area**: backend
+- **Framework**: any (PostgreSQL)
+- **Checklist item**: Missing or unused index (`performance.md` → Database and data access)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "GET /api/users/lookup" --since 24h --output json`:
+
+```json
+{ "endpoint": "GET /api/users/lookup", "count": 88100, "p50": 940, "p95": 1210, "p99": 1480, "errorCount": 0 }
+```
+
+`traceway endpoints show <endpointId> --recorded-at <t>` — one request
+(total 1.02s):
+
+```
+GET /api/users/lookup ..... 1020ms
+└── db.query .............. 980ms SELECT id, name, email FROM users WHERE email = $1
+└── json.encode ........... 12ms
+```
+
+The same query span dominates in *every* sampled trace, at all hours, at all
+traffic levels. `endpoints chart --metric-type p50 --since 30d` shows p50
+rising smoothly from 210ms to 940ms over two months; `git log` shows no
+deploy touching this path; the `users` table grew from 2M to 9M rows in the
+same window. `EXPLAIN` (from a DBA paste in the linked issue) shows
+`Seq Scan on users`.
+
+## Task
+
+> `GET /api/users/lookup` p50 crept from 210ms to 940ms over two months with
+> no deploy on that path. Diagnose the bottleneck to a specific checklist
+> cause.
+
+## Expected diagnosis
+
+- **Root cause**: Missing index on `users(email)` — one query span is the
+ long pole in every trace, its duration grows with table size, and the
+ latency curve tracks data growth, not deploys.
+- **Key evidence**: single dominating query span across all traces; smooth
+ multi-week climb with no deploy; table growth 2M→9M rows; `Seq Scan`.
+- **Fix**: `CREATE INDEX CONCURRENTLY ON users (email)`; verify p50 drops
+ back.
+
+## Pass criteria
+
+- Names **missing index** (not N+1 — there is exactly one query; not
+ saturation — CPU flat, all-hours slowness).
+- Connects the multi-week climb + table growth + no deploy to data-driven
+ latency, and cites the `Seq Scan`.
diff --git a/skills/traceway/evals/backend-node-sequential-calls.md b/skills/traceway/evals/backend-node-sequential-calls.md
new file mode 100644
index 00000000..9da87559
--- /dev/null
+++ b/skills/traceway/evals/backend-node-sequential-calls.md
@@ -0,0 +1,54 @@
+# Eval: backend-node-sequential-calls
+
+- **ID**: `backend-node-sequential-calls`
+- **Area**: backend
+- **Framework**: Node.js (NestJS, microservice fan-out)
+- **Checklist item**: Sequential calls that could run in parallel
+ (`performance.md` → Network, IO, and external dependencies)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "GET /api/dashboard" --since 24h --output json`:
+
+```json
+{ "endpoint": "GET /api/dashboard", "count": 31200, "p50": 3890, "p95": 4520, "p99": 5100, "errorCount": 0 }
+```
+
+`traceway traces show <traceId> --recorded-at <t>` — cross-service timeline
+(total 4.41s):
+
+```
+GET /api/dashboard (edge) .............. 4410ms
+└── svc-users GET /profile ............. 890ms ─┐
+└── svc-billing GET /balance ........... 920ms ├ staircase: each starts
+└── svc-notify GET /unread ............. 870ms │ only when the previous
+└── svc-reco GET /suggestions .......... 940ms ─┘ ends
+└── json.encode ........................ 120ms
+```
+
+Each downstream service is healthy on its own (their p95s are 120–200ms in
+their own dashboards); the 900ms here is mostly network round-trip per hop.
+`system.cpu.utilization` flat at ~30%.
+
+## Task
+
+> `GET /api/dashboard` p50 is 3.9s. Every downstream service looks healthy
+> on its own. Diagnose the bottleneck to a specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: Sequential downstream calls that could run in parallel —
+ the staircase waterfall shows four independent service hops serialized;
+ each hop's own latency is fine, the sum is the problem.
+- **Key evidence**: staircase shape in `traces show`; downstream services
+ healthy in isolation; flat CPU rules out saturation.
+- **Fix**: `Promise.all` the four independent calls (they share no data
+ dependencies); expected p50 ≈ 1.1s.
+
+## Pass criteria
+
+- Names **sequential calls that could run in parallel** (not "slow
+ upstream" — upstreams are healthy; not "chatty fan-out" — four calls is
+ not chatty, the serialization is the issue).
+- Reads the staircase waterfall correctly and notes the calls are
+ independent.
diff --git a/skills/traceway/evals/backend-python-pool-exhaustion.md b/skills/traceway/evals/backend-python-pool-exhaustion.md
new file mode 100644
index 00000000..47c5aa92
--- /dev/null
+++ b/skills/traceway/evals/backend-python-pool-exhaustion.md
@@ -0,0 +1,55 @@
+# Eval: backend-python-pool-exhaustion
+
+- **ID**: `backend-python-pool-exhaustion`
+- **Area**: backend
+- **Framework**: Python (FastAPI + PostgreSQL, SQLAlchemy pool size 20)
+- **Checklist item**: Connection-pool exhaustion (`performance.md` → Database and data access)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "POST /api/checkout" --since 24h --output json`:
+
+```json
+{ "endpoint": "POST /api/checkout", "count": 96400, "p50": 118, "p95": 940, "p99": 8210, "errorCount": 12 }
+```
+
+p50 flat all day; p99 climbs steeply during the 12:00–14:00 and 19:00–21:00
+traffic peaks and collapses between them.
+
+`traceway endpoints show <endpointId> --recorded-at <t>` — one p99 request
+(total 8.04s):
+
+```
+POST /api/checkout ....... 8040ms
+├── <gap: no spans> ...... 3120ms
+└── db.query ............. 44ms SELECT ... FROM inventory WHERE sku = $1
+└── db.query ............. 61ms UPDATE orders SET ...
+└── http.post ............ 280ms POST https://payments.example.com/charge
+```
+
+`traceway metrics query --name system.cpu.utilization --aggregation max --since 24h`:
+35–48%, no spike aligned with the p99 humps. `traceway logs query --search "pool"`:
+`sqlalchemy.pool.impl.QueuePool: "QueuePool limit of size 20 overflow 10 reached, connection timed out"` WARN lines during both peaks.
+
+## Task
+
+> `POST /api/checkout` p50 is fine at 118ms but p99 hits 8.2s during traffic
+> peaks. Diagnose the bottleneck to a specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: Connection-pool exhaustion — the 3.1s gap *before* the
+ first DB span is time waiting to acquire a connection, not work; p99 rises
+ with concurrency while p50 stays flat; pool-timeout warnings in the logs
+ confirm it.
+- **Key evidence**: the pre-DB gap in the waterfall; p99-vs-p50 divergence
+ tracking traffic; `QueuePool limit reached` log lines.
+- **Fix**: raise pool size / add overflow, shorten checkout transaction scope,
+ or shed load (queue) at the peak.
+
+## Pass criteria
+
+- Names **connection-pool exhaustion** (not "slow DB" — the queries are
+ 44–61ms; not "slow upstream" — the payment call is 280ms).
+- Points at the gap *before* the first DB span as the finding, and cites the
+ p99/p50 divergence plus the pool log lines.
diff --git a/skills/traceway/evals/backend-sync-background-task.md b/skills/traceway/evals/backend-sync-background-task.md
new file mode 100644
index 00000000..18c38873
--- /dev/null
+++ b/skills/traceway/evals/backend-sync-background-task.md
@@ -0,0 +1,56 @@
+# Eval: backend-sync-background-task
+
+- **ID**: `backend-sync-background-task`
+- **Area**: backend
+- **Framework**: Go (Gin)
+- **Checklist item**: Synchronous work that belongs in a background task
+ (`performance.md` → Application and compute)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "POST /api/reports" --since 24h --output json`:
+
+```json
+{ "endpoint": "POST /api/reports", "count": 1400, "p50": 9400, "p95": 12100, "p99": 14800, "errorCount": 0 }
+```
+
+`traceway endpoints show <endpointId> --recorded-at <t>` — one request
+(total 11.9s):
+
+```
+POST /api/reports ........ 11900ms
+└── db.query ............. 320ms INSERT INTO reports ...
+└── pdf.render ............ 9100ms (in-process)
+└── s3.put ................ 410ms PUT reports/2026-10-16/...
+└── json.encode ........... 18ms {"report_id": "...", "status": "ready"}
+```
+
+The response returns only after the PDF is fully rendered and uploaded; the
+user does not need the bytes synchronously (the UI polls `GET
+/api/reports/:id`). `system.cpu.utilization` spikes to 91% on the single
+instance during each render. No queue depth anywhere — there is no task
+system on this path.
+
+## Task
+
+> `POST /api/reports` p50 is 9.4s. Diagnose the bottleneck to a specific
+> checklist cause and propose the fix.
+
+## Expected diagnosis
+
+- **Root cause**: Synchronous work that belongs in a background task — the
+ 9.1s `pdf.render` span runs before the response returns, for work the user
+ does not wait on.
+- **Key evidence**: the in-process render span dominating the waterfall;
+ response only returned after upload; UI already polls for completion, so
+ async is safe.
+- **Fix**: return `202 {"status": "processing"}` immediately and move render
+ + upload into a `CaptureTask` background task.
+
+## Pass criteria
+
+- Names **synchronous work that belongs in a background task** (not "CPU
+ saturation" — CPU spikes because of this render, it is the effect, not the
+ cause; not "slow upstream" — S3 put is 410ms).
+- Notes the user doesn't need the result synchronously (polling UI) and
+ proposes the background-task fix.
diff --git a/skills/traceway/evals/frontend-js-large-payloads.md b/skills/traceway/evals/frontend-js-large-payloads.md
new file mode 100644
index 00000000..45717ccf
--- /dev/null
+++ b/skills/traceway/evals/frontend-js-large-payloads.md
@@ -0,0 +1,51 @@
+# Eval: frontend-js-large-payloads
+
+- **ID**: `frontend-js-large-payloads`
+- **Area**: frontend
+- **Framework**: browser JS (React SPA)
+- **Checklist item**: Large request/response payloads (`performance.md` →
+ Network, IO, and external dependencies)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "GET /api/feed" --since 24h --output json`:
+
+```json
+{ "endpoint": "GET /api/feed", "count": 120400, "p50": 1810, "p95": 3400, "p99": 5200, "errorCount": 0 }
+```
+
+`traceway endpoints show <endpointId> --recorded-at <t>` — one request
+(total 3.28s):
+
+```
+GET /api/feed ............ 3280ms
+└── db.query ............. 210ms SELECT ... LIMIT 50
+└── json.encode .......... 180ms
+└── transfer ............. 2840ms response 2.3 MB
+```
+
+The response carries 50 feed cards with full-resolution image URLs inlined
+as base64 (`data:` URIs, ~45KB each). p50/p95 correlate with the `?page`
+depth, not with server load; `system.cpu.utilization` flat at ~25%. Mobile
+web p95 is 2× desktop — transfer-dominated.
+
+## Task
+
+> `GET /api/feed` p50 is 1.8s, worse on mobile web. Server-side spans are
+> fast. Diagnose the bottleneck to a specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: Large response payloads — 2.3MB per response, dominated by
+ inlined base64 images; transfer time is the long pole, not server work.
+- **Key evidence**: `transfer` span 2.84s vs 210ms DB; latency tracks page
+ depth and is worse on mobile (bandwidth-bound signature); flat CPU.
+- **Fix**: thumbnail URLs + lazy-load full images, paginate or compress,
+ strip base64 inlining.
+
+## Pass criteria
+
+- Names **large payloads** (not "slow DB" — DB is 210ms; not "no cache" —
+ there is no evidence about caching here).
+- Identifies transfer as the long pole and ties the mobile-web skew to
+ bandwidth.
diff --git a/skills/traceway/evals/frontend-js-no-cache.md b/skills/traceway/evals/frontend-js-no-cache.md
new file mode 100644
index 00000000..65c5353f
--- /dev/null
+++ b/skills/traceway/evals/frontend-js-no-cache.md
@@ -0,0 +1,44 @@
+# Eval: frontend-js-no-cache
+
+- **ID**: `frontend-js-no-cache`
+- **Area**: frontend
+- **Framework**: browser JS (React SPA)
+- **Checklist item**: No cache / low hit ratio (`performance.md` → Caching and delivery)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "GET /api/products" --since 24h --output json`:
+
+```json
+{ "endpoint": "GET /api/products", "count": 388000, "p50": 640, "p95": 980, "p99": 1400, "errorCount": 0 }
+```
+
+The product catalog changes at most twice a day, yet the endpoint is hit on
+*every* page navigation — 388k hits/day for ~40k daily users. The
+`cache.hit_ratio` metric (reported by the CDN edge) sits at 4%. Waterfall of
+one request is unremarkable: 380ms DB + 120ms serialize, no single slow span.
+
+`traceway endpoints chart --metric-type total_time --since 7d` shows total
+server time for this endpoint is 3× the next endpoint — it is the fleet's
+biggest cost center by volume, not by per-request latency.
+
+## Task
+
+> `GET /api/products` per-request latency looks fine, but it is the fleet's
+> biggest total-time cost center. Diagnose to a specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: No effective cache — a twice-daily-changing catalog is
+ fetched on every navigation with a 4% edge hit ratio; the DB is hit on
+ every request.
+- **Key evidence**: 388k hits vs 40k users; 4% hit ratio; total_time
+ dominance without per-request slownness.
+- **Fix**: edge-cache the catalog response with a 6h TTL + surrogate-key
+ purge on catalog publish; or `stale-while-revalidate`.
+
+## Pass criteria
+
+- Names **no cache / low hit ratio** (not "slow query" — per-request spans
+ are fine; the problem is volume × repetition).
+- Uses the hit-ratio metric and the hits-per-user ratio, not just latency.
diff --git a/skills/traceway/evals/infra-cache-stampede.md b/skills/traceway/evals/infra-cache-stampede.md
new file mode 100644
index 00000000..1cbfb4e1
--- /dev/null
+++ b/skills/traceway/evals/infra-cache-stampede.md
@@ -0,0 +1,46 @@
+# Eval: infra-cache-stampede
+
+- **ID**: `infra-cache-stampede`
+- **Area**: infrastructure
+- **Framework**: n/a (Redis-backed API)
+- **Checklist item**: Cache stampede / thundering herd (`performance.md` →
+ Caching and delivery)
+
+## Telemetry snapshot
+
+`traceway endpoints list --search "GET /api/homepage" --since 24h --output json`:
+
+```json
+{ "endpoint": "GET /api/homepage", "count": 210000, "p50": 95, "p95": 210, "p99": 4800, "errorCount": 0 }
+```
+
+`traceway endpoints chart --metric-type p99 --since 24h --interval-minutes 5`
+for this endpoint: p99 is ~200ms all day except sharp 4–5s spikes at
+**:00, :10, :20, :30, :40, :50** — exactly the 10-minute cache TTL boundary.
+
+During a spike window, `traces show` reveals hundreds of near-identical
+traces all recomputing the homepage payload (12 DB queries each) within the
+same second; between spikes, the cache-hit metric reads 99%. The stampede
+traces show no lock contention — every worker independently missed and
+recomputed.
+
+## Task
+
+> `GET /api/homepage` p99 spikes to 5s exactly every 10 minutes, aligned to
+> the cache TTL. Diagnose to a specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: Cache stampede / thundering herd — at each TTL expiry a
+ burst of workers simultaneously miss and recompute the same payload.
+- **Key evidence**: spikes phase-locked to the TTL boundary; burst of
+ identical recompute traces; 99% hit ratio between spikes (rules out "no
+ cache").
+- **Fix**: probabilistic early refresh / request coalescing (singleflight)
+ / staggered TTLs.
+
+## Pass criteria
+
+- Names **cache stampede** (not "no cache" — hit ratio is 99% between
+ spikes; not "slow DB" — queries are fine, the burst is the problem).
+- Cites the TTL-aligned periodicity as the decisive signal.
diff --git a/skills/traceway/evals/infra-cpu-saturation.md b/skills/traceway/evals/infra-cpu-saturation.md
new file mode 100644
index 00000000..9727c05b
--- /dev/null
+++ b/skills/traceway/evals/infra-cpu-saturation.md
@@ -0,0 +1,49 @@
+# Eval: infra-cpu-saturation
+
+- **ID**: `infra-cpu-saturation`
+- **Area**: infrastructure
+- **Framework**: n/a (fleet-wide)
+- **Checklist item**: CPU saturation (`performance.md` → Infrastructure and saturation)
+
+## Telemetry snapshot
+
+`traceway endpoints list --since 24h --order-by p95 --page-size 10`: the top
+8 endpoints *all* show p95 2–4× their 7-day baseline, starting at 15:40
+yesterday. No single span dominates — waterfalls show every span type
+(DB, serialize, upstream) uniformly slower, with no long pole.
+
+`traceway metrics query --name system.cpu.utilization --aggregation max --since 24h --interval-minutes 60`:
+
+```
+14:00 38% 15:00 44% 16:00 97% 17:00 98% 18:00 96% ...
+```
+
+The CPU step at ~15:40 lines up exactly with the latency rise across all
+endpoints. `git log` shows the only deploy in the window was a config-only
+change at 15:35 that doubled the log verbosity (see also
+`backend-excessive-logging`, not this eval). `mem.used` flat.
+
+## Task
+
+> Eight unrelated endpoints all went 2–4× slower at 15:40 yesterday, with no
+> single slow span. Diagnose to a specific checklist cause — and say what it
+> is *not*.
+
+## Expected diagnosis
+
+- **Root cause**: CPU saturation — fleet-wide latency rise with no long
+ pole, aligned exactly with the `system.cpu.utilization` step to 97%+.
+- **Key evidence**: broad rise across endpoints (rules out a code bug in one
+ path); no dominating span (rules out a slow query/upstream); metric
+ inflection at the same timestamp.
+- **What it is not**: not a code regression (config-only deploy, no single
+ culprit span); the doubled log verbosity is a *contributor* to CPU, worth
+ fixing, but the finding is saturation.
+- **Fix**: scale out / move the verbose logging off the hot path, then
+ re-check headroom.
+
+## Pass criteria
+
+- Names **CPU saturation** and explicitly rules out a single code/query
+ cause using the breadth of the rise and the metric alignment.
+- Does not blame the config deploy as a code regression without evidence.
diff --git a/skills/traceway/evals/mobile-android-gc-pauses.md b/skills/traceway/evals/mobile-android-gc-pauses.md
new file mode 100644
index 00000000..40718057
--- /dev/null
+++ b/skills/traceway/evals/mobile-android-gc-pauses.md
@@ -0,0 +1,52 @@
+# Eval: mobile-android-gc-pauses
+
+- **ID**: `mobile-android-gc-pauses`
+- **Area**: mobile
+- **Framework**: Android (native, ART)
+- **Checklist item**: Memory pressure / GC pauses (`performance.md` →
+ Infrastructure and saturation)
+
+## Telemetry snapshot
+
+`scroll.jank` trace group (feed scroll): p50 frame time 22ms, p99 190ms.
+The p99 frames cluster in sessions with >3 minutes of continuous scrolling.
+
+`traceway traces show <traceId> --recorded-at <t>` — one janky scroll
+window:
+
+```
+scroll.window ............ 2400ms
+├── frame ................ 24ms
+├── frame ................ 28ms
+├── <gap: no spans> ...... 165ms ← ART GC pause (concurrent → blocking)
+├── frame ................ 188ms
+└── image.decode ......... 96ms (full-res bitmap, 4096×4096)
+```
+
+`mem.used` climbs monotonically across the session; the image cache has no
+size cap (in-memory LRU unbounded). GC pause frequency tracks scroll depth,
+not network or server state. Server spans for the same feed are fast
+(p95 300ms).
+
+## Task
+
+> Android feed scroll p99 frame time is 190ms, worsening the longer the
+> session runs. Server side is fine. Diagnose to a specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: Memory pressure / GC pauses — unbounded in-memory image
+ cache grows with scroll depth; ART GC pauses surface as gaps in the frame
+ timeline.
+- **Key evidence**: the span-less gap (GC, not work); `mem.used` climbing
+ with session length; full-res decodes feeding an uncapped cache; server
+ side clean.
+- **Fix**: cap the image cache (e.g. 128MB LRU), decode at display size,
+ recycle bitmaps.
+
+## Pass criteria
+
+- Names **memory pressure / GC pauses** and reads the gap as GC rather than
+ missing work.
+- Connects the uncapped cache + `mem.used` climb to the pauses (not "slow
+ images" alone — decode is 96ms, the 165ms gap is the killer).
diff --git a/skills/traceway/evals/mobile-flutter-shader-jank.md b/skills/traceway/evals/mobile-flutter-shader-jank.md
new file mode 100644
index 00000000..bddfac4f
--- /dev/null
+++ b/skills/traceway/evals/mobile-flutter-shader-jank.md
@@ -0,0 +1,47 @@
+# Eval: mobile-flutter-shader-jank
+
+- **ID**: `mobile-flutter-shader-jank`
+- **Area**: mobile
+- **Framework**: Flutter
+- **Checklist item**: Cold start / JIT / connection warmup (`performance.md`
+ → Caching and delivery; "latency normalizes after warmup")
+
+## Telemetry snapshot
+
+`route.transition` trace group (product detail page): p50 320ms, p99 2.1s.
+The p99s are *first visits per install* to any page with the animated hero
+carousel — `firstSeen`-correlated per route; second visits render at ~300ms.
+
+`traceway traces show <traceId> --recorded-at <t>` — one first visit (2.0s):
+
+```
+route.transition ......... 2000ms
+├── build/layout ......... 180ms
+├── <gap: raster thread> . 1450ms ← shader compilation (SkSL warmup)
+└── first-frame .......... 370ms
+```
+
+`system.cpu.utilization` on device spikes during the gap (raster thread
+compiling shaders); no network spans in the window — the page data was
+already cached. After the first visit, subsequent transitions are smooth.
+
+## Task
+
+> Flutter product-page transitions hit 2.1s p99, but only the first visit
+> per install; data is cached and the network is idle. Diagnose to a
+> specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: Shader/JIT warmup jank — first-frame SkSL compilation on
+ the raster thread; classic warmup pattern (bad once, then fine).
+- **Key evidence**: first-visit-only outliers; raster-thread gap with no
+ spans and no network; CPU spike on device during the gap.
+- **Fix**: ship SkSL warmup (`flutter build --bundle-sksl-path`), or
+ pre-warm the carousel offscreen.
+
+## Pass criteria
+
+- Names **shader/JIT warmup** (not "slow build" — build/layout is 180ms;
+ not network — idle).
+- Recognizes the first-visit-only + normalizes-after-warmup signature.
diff --git a/skills/traceway/evals/mobile-ios-cold-start.md b/skills/traceway/evals/mobile-ios-cold-start.md
new file mode 100644
index 00000000..1f09f00b
--- /dev/null
+++ b/skills/traceway/evals/mobile-ios-cold-start.md
@@ -0,0 +1,49 @@
+# Eval: mobile-ios-cold-start
+
+- **ID**: `mobile-ios-cold-start`
+- **Area**: mobile
+- **Framework**: iOS (native)
+- **Checklist item**: Cold start (`performance.md` → Caching and delivery;
+ "first-request-after-deploy outliers … latency normalizes after warmup")
+
+## Telemetry snapshot
+
+Mobile `app.start` trace group, last 24h: p50 1.1s, p95 3.8s. The p95 is
+entirely first launches after install/update — `firstSeen`-correlated; warm
+starts sit at 0.4s.
+
+`traceway traces show <traceId> --recorded-at <t>` — one cold start (3.6s):
+
+```
+app.start ................ 3600ms
+└── didFinishLaunching ... 2200ms
+│ ├── sdk.init ......... 900ms analytics + crash + attribution SDKs, sequential
+│ ├── db.migrate ....... 700ms CoreData lightweight migration on main thread
+│ └── config.fetch ..... 600ms synchronous remote-config fetch, blocks return
+└── first-frame .......... 1400ms
+```
+
+No network saturation (config endpoint p95 120ms); the 600ms is one
+blocking call plus main-thread contention. Subsequent launches skip the
+migration and use cached config.
+
+## Task
+
+> iOS cold start p95 is 3.8s, but only on first launch after install/update.
+> Diagnose to a specific checklist cause.
+
+## Expected diagnosis
+
+- **Root cause**: Cold start with synchronous init on the main thread —
+ sequential SDK init, main-thread CoreData migration, and a blocking
+ remote-config fetch inside `didFinishLaunching`.
+- **Key evidence**: first-launch-only outliers (`firstSeen`-correlated);
+ warm starts fine; the three init spans serialized on the main thread.
+- **Fix**: defer non-critical SDK init off the critical path, move migration
+ off main thread, make remote config async with cached fallback.
+
+## Pass criteria
+
+- Names **cold start / synchronous launch init** and ties it to
+ first-launch-only data.
+- Identifies all three serialized init spans (not just "slow SDK").