A lightweight WebAssembly runtime that is fast, secure, and standards-compliant
Add a skill for reducing test cases (#13999)
Nick Fitzgerald1c97397a45c2
diff --git a/.agents/skills/reduce-test-cases/SKILL.md b/.agents/skills/reduce-test-cases/SKILL.mdnew file mode 100644index 0000000000..bc65383f64--- /dev/null+++ b/.agents/skills/reduce-test-cases/SKILL.md@@ -0,0 +1,399 @@+---+name: reduce-test-cases+description: >+ Reduce, shrink, or minimize a failing Wasmtime/Cranelift test case+ (.wasm/.wat, .wast, or .clif) down to the smallest input that still+ reproduces a bug, crash, trap, panic, or miscompilation. Gives you a+ predicate/command skeleton per reducer (wasm-tools shrink, Binaryen+ wasm-reduce, creduce, clif-util bugpoint), a wasm fallback chain for when a+ reducer can't parse your module, and a manual reduction playbook.+---++# Reduce Test Cases++A minimal reproducer is the most valuable artifact in a bug report. Reduce a+test case in two phases:++1. **Automated reducers** — the approach from+ `docs/contributing-reducing-test-cases.md`, extended to `.wast` and `.clif`+ and to a fallback chain for wasm the primary reducer can't parse.+2. **Manual reasoning** — when a reducer stalls, doesn't apply (e.g. a `.clif`+ miscompile, which `bugpoint` can't touch), or over-shrinks past your bug.++**The predicate is the whole game.** Every reducer does the same dumb loop:+mutate the input, ask a yes/no script "does this still reproduce the bug?", keep+the mutation if yes. The reducer is generic; the *predicate* is where all the+per-bug intelligence lives. This skill does **not** ship a do-everything driver+— your bug is not knowable in advance. Instead `templates/` holds a **skeleton+per reducer** that you **copy and edit** to pin your exact symptom:++| Reducer | How it hands you the candidate | "Interesting" means | Skeleton |+|---|---|---|---|+| `wasm-tools shrink` | as `$1` | script **exits 0** | `templates/predicate-shrink.sh` |+| Binaryen `wasm-reduce` | writes a **fixed `--test` file** | script's **stdout _and_ exit code both match the original** | `templates/command-wasm-reduce.sh` |+| `creduce` | candidate by **basename**, fresh temp dir | script **exits 0** | `templates/predicate-creduce-{wat,wast,clif}.sh` |+| `clif-util bugpoint` | *(no predicate)* | compile **panics** | *(built in)* |++For a **miscompile** — no trap or panic to grep, just a wrong result — the+predicate diffs *behavior* across two configs instead. That shape is orthogonal+to the reducer, so it has its own skeleton, `templates/predicate-differential.sh`,+that you drop into the shrink or creduce contract above. In practice this was the+most common real-world case, so reach for it first when the bug is a wrong answer.++All paths below are relative to the repo root.++## Prerequisites++Install the reducers (`wasm-tools` ships with the toolchain; the rest are+external):++```bash+cargo install wasm-tools # shrink + print (often already present)+brew install creduce # generic text reducer for .clif / .wast / .wat+brew install binaryen # wasm-reduce, the alternate wasm reducer+```++Build the engines your predicates will run. A **debug** build is required to+reproduce `debug_assert!` panics; a `--release` build iterates much faster for+everything else, which matters when a reducer runs the predicate thousands of+times:++```bash+cargo build --bin wasmtime # debug: target/debug/wasmtime+cargo build --release --bin wasmtime # fast: target/release/wasmtime+cargo build --release --manifest-path=./cranelift/Cargo.toml --bin clif-util # target/release/clif-util+```++Every skeleton has a `WASMTIME=` / `CLIF_UTIL=` line you must set to an+**absolute** path — reducers run from other directories, so a relative path+breaks. Print the absolute path and paste it in:++```bash+echo "$PWD/target/release/wasmtime"+echo "$PWD/target/release/clif-util"+```++## Workflow++Work in a scratch directory holding your failing input. The loop never changes:++1. **Reproduce the bug once, by hand,** to pin the *exact* symptom — the specific+ trap string, panic message, ISLE error, or diverging output. This is the most+ important step: a vague predicate reduces to a *different* bug. For a+ miscompile, also **confirm it's deterministic** — run each config a few times+ and check every run of a config agrees and the two configs disagree; a flaky+ config makes a differential predicate unsound.+2. **Copy the right skeleton** from `templates/` next to your input.+3. **Edit it:** set the absolute engine path, and replace the example symptom+ with yours. Keep it tight — a loose predicate lets the reducer converge on+ some other, spec-legal failure.+4. **Test the predicate on the ORIGINAL.** If it doesn't fire here, it never+ will on a smaller file. (For `shrink`/`creduce`: `./pred.sh input` should exit+ 0. For `wasm-reduce`: the command should print your marker.)+5. **Run the reducer** (per format, below).+6. **Eyeball the result** and re-confirm it's still *your* bug — reducers are+ happy to find a smaller program that fails for an unrelated reason.+7. **Re-verify the final case independently** against the predicate — especially+ anything a subagent/fork produced, or a hand-reduction. Convert every+ candidate to the same form before comparing sizes (wasm bytes *and* canonical+ `wasm-tools print` line count); raw creduce/hand whitespace isn't comparable.+8. **Re-author the winner to be legible**, not just small. Give locals/functions+ readable names, add a one-paragraph comment explaining the mechanism, and for+ a `.wast` make it self-checking with `assert_trap` / `assert_return` so the+ file *proves* the bug on its own. The best artifact teaches the fix.++## wasm / wat — a three-tool fallback chain++Each wasm reducer only understands the proposals its own parser implements, and+those subsets differ. Work down the chain until one bites:++### 1. `wasm-tools shrink` — try this first++Smartest, most structural, produces the smallest output. Uses+`templates/predicate-shrink.sh` (candidate is `$1`, exit 0 == reproduces):++```bash+cp .agents/skills/reduce-test-cases/templates/predicate-shrink.sh pred.sh+# edit pred.sh: set WASMTIME=<abs path>, set the symptom grep+chmod +x pred.sh+./pred.sh input.wasm && echo "reproduces on original" # sanity check first+wasm-tools shrink ./pred.sh input.wasm -o reduced.wasm+wasm-tools print reduced.wasm+```++Verified end-to-end: a 273-byte module (noise functions, globals, a table, data+segments) reduced to **42 bytes** — exactly the trapping core:++```+(module+ (type (;0;) (func))+ (export "_start" (func 0))+ (func (;0;) (type 0)+ i32.const 42+ i32.const 0+ i32.div_s+ drop))+```++### 2. Binaryen `wasm-reduce` — if shrink can't parse your module++Different proposal coverage from shrink — and the gap is real, not theoretical:+Binaryen can't parse wide-arithmetic (`i64.mul_wide_s`, `i64.add128`) or some+SIMD, while `wasm-tools shrink` handles them. So for a module using those, skip+straight from shrink to creduce; `wasm-reduce` is a non-starter. It also has a+**different contract**: it writes each candidate to a fixed `--test` file (not+`$1`), and keeps a candidate only if your command's **stdout and exit code both+match the original run** — so print one canonical marker when the bug+reproduces. Uses `templates/command-wasm-reduce.sh`:++```bash+cp .agents/skills/reduce-test-cases/templates/command-wasm-reduce.sh cmd.sh+# edit cmd.sh: set WASMTIME=<abs path>, set the symptom marker+cp input.wasm t.wasm && bash cmd.sh # sanity check: prints your marker+wasm-reduce input.wasm --command 'bash ./cmd.sh' --test t.wasm --working reduced.wasm -f+```++The `-f` matters: before reducing, `wasm-reduce` round-trips your module through+Binaryen's own parser/writer, and if it can't (an unsupported proposal) it prints+`failed to read and write the binary` and stops. `-f` forces past that. Verified:+with `-f`, the same 273-byte module reduced to **91 bytes** and still trapped —+larger than shrink's 42 bytes (it kept the table/memory/globals), which is+exactly why this is a fallback, not the first choice.++### 3. Disassemble + `creduce` — if neither reducer can parse it++`creduce` reduces **text** and never re-encodes the wasm, so **only the engine+has to understand the proposal** — the reducer stays out of the way. This always+*runs*, at the cost of being purely textual: because it doesn't understand `.wat`+structure it tends to plateau early on parenthesis-balance constraints (in one+real case it stalled at ~43% of the original and no further). Treat its output as+a starting point for the manual playbook, not the finish line. `wasmtime` runs a+`.wat` directly, so no reassembly is needed. Uses `templates/predicate-creduce-wat.sh`:++```bash+wasm-tools print input.wasm -o case.wat # disassemble+cp .agents/skills/reduce-test-cases/templates/predicate-creduce-wat.sh pred.sh+# edit pred.sh: set WASMTIME=<abs path>, set the symptom; CAND=case.wat+chmod +x pred.sh+mkdir -p /tmp/reduce && cp pred.sh case.wat /tmp/reduce && cd /tmp/reduce+creduce ./pred.sh case.wat # reduces case.wat in place+```++Verified: a 52-line disassembly reduced to two lines that still trap:++```+(module(export "_start"(func $a))(+ func $a i32.const 2 i32.const 0 i32.div_s drop))+```++## wast — `creduce`++`wasm-tools shrink` reduces **one module**; a `.wast` is a whole *script*+(multiple modules + assertions), so reduce the text with `creduce`. Run the+script with `wasmtime wast`. Uses `templates/predicate-creduce-wast.sh`:++```bash+target/release/wasmtime wast input.wast # reproduce first+cp .agents/skills/reduce-test-cases/templates/predicate-creduce-wast.sh pred.sh+# edit pred.sh: set WASMTIME=<abs path>, set the symptom; CAND=input.wast+chmod +x pred.sh+mkdir -p /tmp/reduce && cp pred.sh input.wast /tmp/reduce && cd /tmp/reduce+creduce ./pred.sh input.wast+```++Verified: a 30-line, 3-module script (with passing-assertion noise) reduced to a+single line that still traps with `integer divide by zero`:++```+(module(func $a i32.const 2 i32.const 0 i32.div_s drop)(start $a))+```++Feature flags come from `-W` on the CLI (e.g. `-W gc,function-references`), **not**+from the `;;! feature = true` header — that header is only read by+`cargo test --test wast`. `creduce` will delete the `;;!` header (the CLI ignores+it), so re-add it if the reduced case is destined for `tests/`.++## clif — `bugpoint` (panics only) or `creduce` (everything else)++`clif-util bugpoint <FILE> <TARGET>` reduces **compile panics only**. It has no+predicate hook — it catches panics internally and treats a clean compile *or a+verifier error* as "not interesting". On a `.clif` that fails *gracefully* (the+common `test compile expect-fail` case) it bails:++```bash+target/release/clif-util bugpoint be.clif riscv64+# Warning: Given function compiled successfully or gave a verifier error. (exit 0)+```++For anything that isn't a hard panic — miscompiles, wrong output, graceful+codegen errors, `test`/`run` failures — use `creduce` with a `clif-util`+predicate. Uses `templates/predicate-creduce-clif.sh`:++```bash+cp .agents/skills/reduce-test-cases/templates/predicate-creduce-clif.sh pred.sh+# edit pred.sh: set CLIF_UTIL=<abs path>, pick the sub-command, set the symptom+chmod +x pred.sh+mkdir -p /tmp/reduce && cp pred.sh be.clif /tmp/reduce && cd /tmp/reduce+creduce ./pred.sh be.clif+```++Verified: a 287-line, 46-function `big-endian.clif` reduced to **one function**+that still hits the target codegen error:++```+function % (i64)->i128 { block0(v0 : i64) : v1 = load.i128 big v0 return v1 }+```++```bash+target/release/clif-util compile --target riscv64 be.reduced.clif+# Error: Unsupported feature: should be implemented in ISLE: inst = `v1 = load.i128 big v0`, type = `Some(types::I128)`+```++The predicate's sub-command is your choice: `clif-util compile` (codegen; target+on the CLI), `clif-util test` (honors in-file `; run:` / `; check:` directives),+`clif-util run` (host JIT), or `clif-util interpret` (backend-independent — good+for miscompile differentials). `creduce` deletes the `test`/`set`/`target` header+lines, so re-add them if you want the reduced file to run under `clif-util test`.++## Miscompiles — differential predicates++A miscompile has no trap or panic to grep; "interesting" means *the answer is+wrong*, which you detect by running two configs and checking they **disagree**.+This was the most common real-world case. The reducer doesn't care — it's just a+predicate shape — so `templates/predicate-differential.sh` drops into either the+shrink (`$1`) or creduce (basename) contract. The load-bearing detail is the+guard:++```bash+a=$("$WASMTIME" -O opt-level=0 --invoke f "$CAND" 2>/dev/null); ra=$?+b=$("$WASMTIME" --invoke f "$CAND" 2>/dev/null); rb=$? # default == opt-level=2+[ $ra -eq 0 ] && [ $rb -eq 0 ] && [ -n "$a" ] && [ -n "$b" ] && [ "$a" != "$b" ]+```++Both configs must run cleanly *and* produce output *and* differ. Drop the+exit-0/non-empty guards and a candidate that merely became invalid or started+trapping satisfies `a != b` for free, so the reducer converges on a broken+program instead of a miscompile. Real fuzzer repros invoke a **named export with+arguments** and need the right **`-W` feature flags** (`-Wwide-arithmetic+--invoke f "$CAND" 1 -1`) — the defaults in the trap skeletons won't cut it.+Confirm determinism first (Workflow step 1). For a `.wast`, prefer the+self-checking form: encode the right answer as `assert_return` / `assert_trap`+and make "interesting" == "opt-level=0 passes but default fails".++## Manual reduction playbook++When the reducer stalls, over-shrinks, or the bug is a miscompile no predicate+cleanly captures, reduce by hand. Reason from the bug's *mechanism* back to the+minimal trigger, and re-verify after every cut:++- **Bisect coarse first.** Delete whole functions / modules / assertions, half at+ a time; keep the half that still reproduces. `clif-util bugpoint`'s own strategy+ is a good mental model: remove instruction → replace with const → replace with+ trap → remove block → drop unused entities → merge blocks.+- **Then bisect instructions** inside the surviving function the same way.+- **Neutralize data.** Shrink memory/table sizes to the minimum; truncate or zero+ data/element segments; drop unused globals, types, imports, and exports.+- **Simplify operands.** Replace computed values with constants; collapse control+ flow (drop unused blocks/params); inline single-use locals.+- **Re-verify each step** with `wasm-tools print` / `clif-util cat` and by+ re-running the engine. If a cut kills the repro, revert it and try another.+- **Keep a copy of the last-good reduction** so a bad cut is always recoverable.++For a **miscompile**, the manual pass is where you *win* — automated reducers+have no idea which bytes carry the bug, so they grind on structure while the+real reduction is semantic. Hunt the diverging value directly:++- **Neutralize inputs one at a time, don't delete them.** Replace each+ `local.get` / `global.get` (or `.clif` value) feeding the wrong result with a+ constant, one at a time; the one whose neutralization makes the divergence+ *vanish* is carrying the corruption. Prefer this over deleting/truncating:+ cutting code changes *what the optimizer does*, so a cut that "fixes" the bug+ may have perturbed the pass rather than removed the cause — neutralization+ keeps the shape and isolates the value.+- **Return the suspect directly.** Once you've found the diverging value,+ `return (local.get $culprit)` (or make it the sole result) collapses the+ entire downstream computation in one edit — often dropping the bulk of the+ program at a stroke.+- **Constant-fold the setup.** Hash/mix stages, loop trip counts, and data+ segments that only *produce* the suspect value can be replaced by the constant+ they compute. Collapse a loop to the smallest trip count that still diverges.+- **Then rebuild from scratch.** Once the mechanism is clear, hand-writing a+ fresh minimal module from the understood cause beats continuing to chip at the+ reduced-but-messy one — and yields the legible artifact you actually want. In+ one real fuzzer case this took a 649-line module to ~22 lines (4%), where+ creduce plateaued at 43% and shrink barely moved it.++**Race the reducers; don't serialize them.** shrink, creduce, and a manual pass+are cheap to run at once — start each in its **own** temp dir (creduce reduces+in place and will collide otherwise), let creduce run in the background (it far+exceeds a single tool-call timeout), and spin up a **fork/subagent for the+semantic reduction** with full context on the bug's mechanism. Then take the+smallest legible result. The manual fork usually wins the hard cases.++The goal is the smallest input where the bug's root cause is still obvious — not+just small, but *legible*.++## Gotchas++Battle scars from actually running these:++- **`wasm-reduce` preserves behavior, not "exit 0".** It keeps a candidate only+ if stdout *and* exit code match the original run — unlike shrink/creduce, which+ just want exit 0. Collapse "reproduces" to one canonical stdout marker and keep+ the exit code stable. And it reads the fixed `--test` file, **not `$1`**.+- **`wasm-reduce` needs `-f` for unsupported proposals.** It round-trips the+ module through Binaryen first and prints `failed to read and write the binary`+ if it can't. `-f` forces past it; if even that gets nowhere, drop to+ disassemble + creduce. Some proposals it simply can't represent (e.g.+ wide-arithmetic's `i64.mul_wide_s`) — for those, `-f` won't save it; skip to+ creduce.+- **Environment variables don't reliably reach `wasm-reduce`'s `--command`.**+ Hardcode the absolute engine path in the command script (the skeleton does).+- **creduce strips header directives.** It deletes anything that doesn't change+ interestingness — the `.clif` `test`/`set`/`target` lines and the `.wast`+ `;;! feature = true` header included. Fine for a repro whose predicate bakes the+ target/features into the command, but the reduced file won't run under+ `cargo test --test wast` or `clif-util test` unmodified. Re-add the header (or+ the matching `-W` / `--target` / `--set` flags) afterward.+- **creduce runs in a fresh temp dir** containing only the candidate, referenced+ by *basename*. Predicates must reference the candidate by basename and every+ tool by an **absolute** path. If you hand-roll a wrapper that `cd`s, resolve the+ predicate to an absolute path *before* changing directory.+- **Don't put `set -e` in a shrink/creduce predicate.** A trapping module makes+ the engine exit non-zero; under `set -e` the `out=$(engine …)` line aborts the+ script before your grep runs, so the predicate never reports "interesting". Let+ the final grep be the exit status.+- **`bugpoint` is panic-only, and only on the plain compile flow.** It ignores+ verifier and graceful codegen errors ("compiled successfully or gave a verifier+ error"), and it only drives `clif-util compile` — a panic that fires only under+ a `clif-util test <flavor>` pass (e.g. `test alias-analysis`) is invisible to+ it. It cannot reduce a miscompile either. For any of these, use `creduce`.+- **shrink plateaus on already-minimized or fragile fuzzer input.** If the bug+ report says it was minimized with `wasm-tools shrink`, re-running shrink buys+ almost nothing (one real case moved <2%) — you're at a shrink local minimum.+ Likewise a fragile miscompile where removing almost *anything* makes the+ divergence vanish resists every structural reducer. Recognize the plateau early+ and switch to creduce and the manual playbook rather than grinding.+- **Fuzzy predicates drift.** A reduced module can fail the same way for a+ *different*, spec-legal reason. Pin the exact message and eyeball the result.+- **Debug vs release — check the panic site.** A plain `panic!` / `.unwrap()` /+ `.expect()` fires in a `--release` engine too, so release (which iterates far+ faster) is enough; only `debug_assert!` needs a debug build. Look at the assert+ that fires before rebuilding. Point the skeleton's `WASMTIME=` / `CLIF_UTIL=`+ at whichever you actually need.++## Troubleshooting++- **Predicate fails on the original** — it doesn't reproduce yet. Wrong trap+ string, wrong `--invoke` export, missing `-W` feature flag, or wrong engine+ build. Fix the predicate before reducing anything.+- **`wasm-reduce` says "failed to read and write the binary" / "very unlikely+ reduction can succeed"** — Binaryen can't parse a proposal in your module. Add+ `-f`; if that stalls too, disassemble + creduce.+- **`bugpoint` prints "compiled successfully or gave a verifier error" (exit+ 0)** — the input fails *gracefully*, it doesn't panic. Expected for+ `expect-fail` files. Switch to `creduce`.+- **`creduce` finishes but the file is unchanged** — the predicate never passed+ on any mutation. Usual causes: a relative tool path that broke in creduce's temp+ dir, a predicate that isn't `chmod +x`, or `set -e` swallowing a trapping exit.+ Re-test the predicate on the original and confirm absolute tool paths.diff --git a/.agents/skills/reduce-test-cases/templates/command-wasm-reduce.sh b/.agents/skills/reduce-test-cases/templates/command-wasm-reduce.shnew file mode 100755index 0000000000..aac8e790b1--- /dev/null+++ b/.agents/skills/reduce-test-cases/templates/command-wasm-reduce.sh@@ -0,0 +1,40 @@+#!/usr/bin/env bash+#+# COMMAND SKELETON for Binaryen's `wasm-reduce` (wasm).+#+# wasm-reduce input.wasm --command 'bash ./this-script.sh' \+# --test t.wasm --working reduced.wasm+# # add -f if wasm-reduce refuses to start (see note 3 below)+#+# wasm-reduce works DIFFERENTLY from creduce / wasm-tools shrink:+#+# 1. It writes each candidate to the FIXED --test file (t.wasm here), then+# runs this command. This command must read that fixed path, NOT $1.+# 2. A candidate is kept iff this command's STDOUT *and* EXIT CODE match what+# they were on the original. It preserves behavior, it does not look for+# "exit 0". So collapse "my bug reproduces" down to ONE canonical line of+# stdout (below), and let everything else print something different.+# 3. Before reducing, wasm-reduce round-trips the module through Binaryen's+# own parser/writer as a sanity check. If Binaryen can't handle a wasm+# proposal in your module it prints "failed to read and write the binary"+# and stops — pass -f to force past it (reduction still runs; it just skips+# Binaryen's structural passes and relies on generic shrinking). If even -f+# gets nowhere, fall back to disassemble + creduce (see predicate-creduce-wat.sh).++# Must match the --test filename above.+TEST=t.wasm++# EDIT: absolute path to the engine you built.+# echo "$PWD/target/release/wasmtime"+WASMTIME=/ABS/PATH/TO/target/release/wasmtime++out=$("$WASMTIME" run --invoke _start "$TEST" 2>&1)++# EDIT: the symptom. Print the SAME canonical marker for every candidate that+# reproduces, and anything else otherwise. (Exit code is also preserved, so+# keep it stable too — here we always exit 0 and distinguish via stdout.)+if echo "$out" | grep -q 'integer divide by zero'; then+ echo REPRODUCES+else+ echo no+fidiff --git a/.agents/skills/reduce-test-cases/templates/predicate-creduce-clif.sh b/.agents/skills/reduce-test-cases/templates/predicate-creduce-clif.shnew file mode 100755index 0000000000..30a68cecd5--- /dev/null+++ b/.agents/skills/reduce-test-cases/templates/predicate-creduce-clif.sh@@ -0,0 +1,39 @@+#!/usr/bin/env bash+#+# PREDICATE SKELETON for `creduce` on a Cranelift `.clif` file.+#+# Use creduce for ANY clif bug that isn't a hard compile panic: miscompiles,+# wrong results, verifier errors, graceful codegen errors. (For an actual+# compile PANIC, prefer `clif-util bugpoint FILE TARGET` — it needs no+# predicate. See SKILL.md.)+#+# cp input.clif predicate-creduce-clif.sh /tmp/reduce && cd /tmp/reduce+# creduce ./predicate-creduce-clif.sh input.clif+#+# creduce runs this from a FRESH TEMP DIR with the candidate by BASENAME:+# reference the candidate by basename, tools by ABSOLUTE path, exit 0 == "still+# reproduces".++CAND=input.clif++# EDIT: absolute path to the tool you built.+# echo "$PWD/target/release/clif-util"+CLIF_UTIL=/ABS/PATH/TO/target/release/clif-util++# EDIT: pick the sub-command that exposes YOUR bug, and pin the symptom.+#+# codegen error / crash text (target on the CLI, so header lines aren't needed):+out=$("$CLIF_UTIL" compile --target riscv64 "$CAND" 2>&1)+echo "$out" | grep -q 'should be implemented in ISLE'+#+# in-file `; run:` / `; check:` expectations (honors the file's directives):+# "$CLIF_UTIL" test "$CAND" >/dev/null 2>&1; [ $? -ne 0 ] # interesting == FAILS+#+# miscompile via differential execution — first edit the file's expected value+# so a CORRECT engine passes and only the buggy path fails, then:+# out=$("$CLIF_UTIL" interpret "$CAND" 2>&1); echo "$out" | grep -q 'mismatch'+#+# NOTE: creduce deletes the `test ... / set ... / target ...` header lines (your+# predicate passes --target on the CLI, so they're not needed to reproduce). The+# reduced file therefore won't run under `clif-util test` unmodified — re-add the+# headers if you want a self-contained filetest. Eyeball with `clif-util cat`.diff --git a/.agents/skills/reduce-test-cases/templates/predicate-creduce-wast.sh b/.agents/skills/reduce-test-cases/templates/predicate-creduce-wast.shnew file mode 100755index 0000000000..13a35736f4--- /dev/null+++ b/.agents/skills/reduce-test-cases/templates/predicate-creduce-wast.sh@@ -0,0 +1,35 @@+#!/usr/bin/env bash+#+# PREDICATE SKELETON for `creduce` on a `.wast` script.+#+# `wasm-tools shrink` reduces ONE module; a `.wast` is a whole script (many+# modules + assertions), so use creduce on the text.+#+# cp input.wast predicate-creduce-wast.sh /tmp/reduce && cd /tmp/reduce+# creduce ./predicate-creduce-wast.sh input.wast+#+# creduce runs this from a FRESH TEMP DIR with the candidate by BASENAME:+# reference the candidate by basename, tools by ABSOLUTE path, exit 0 == "still+# reproduces".++CAND=input.wast++# EDIT: absolute path to the engine you built.+# echo "$PWD/target/release/wasmtime"+WASMTIME=/ABS/PATH/TO/target/release/wasmtime++# `wasmtime wast` runs the script and its assertions. Feature flags come from -W+# on the CLI, NOT from the `;;! feature = true` header (that header is only read+# by `cargo test --test wast`). Add any your case needs, e.g. -W gc,function-references.+out=$("$WASMTIME" wast "$CAND" 2>&1)+st=$?++# EDIT: pin YOUR exact symptom. Examples:+# unexpected trap / failure: echo "$out" | grep -q 'integer divide by zero'+# any failure at all: [ $st -ne 0 ]+echo "$out" | grep -q 'integer divide by zero'++# NOTE: creduce will happily DELETE the `;;! ...` header (the CLI ignores it, so+# removing it doesn't change this predicate). If the reduced case is destined for+# tests/ (run by `cargo test --test wast`), re-add the header afterward, or the+# harness won't enable the right features.diff --git a/.agents/skills/reduce-test-cases/templates/predicate-creduce-wat.sh b/.agents/skills/reduce-test-cases/templates/predicate-creduce-wat.shnew file mode 100755index 0000000000..10fbe2fc4a--- /dev/null+++ b/.agents/skills/reduce-test-cases/templates/predicate-creduce-wat.sh@@ -0,0 +1,32 @@+#!/usr/bin/env bash+#+# PREDICATE SKELETON for `creduce` on a disassembled `.wat` (wasm fallback).+#+# Use this when neither `wasm-tools shrink` nor `wasm-reduce` can handle your+# module (e.g. a wasm proposal neither tool fully supports). creduce reduces the+# TEXT and never re-encodes the wasm, so only the ENGINE has to understand the+# proposal — the reducer stays out of the way.+#+# wasm-tools print input.wasm -o case.wat # disassemble+# cp case.wat predicate-creduce-wat.sh /tmp/reduce && cd /tmp/reduce+# creduce ./predicate-creduce-wat.sh case.wat # reduces case.wat in place+#+# creduce copies the candidate into a FRESH TEMP DIR and runs this test there,+# referring to the file by BASENAME. So: reference the candidate by basename+# (no path), reference every tool by an ABSOLUTE path, and exit 0 == "still+# reproduces". `wasmtime` runs a `.wat` directly, so no reassembly is needed.++# Candidate basename (matches the filename you pass to creduce). No directory.+CAND=case.wat++# EDIT: absolute path to the engine you built.+# echo "$PWD/target/release/wasmtime"+WASMTIME=/ABS/PATH/TO/target/release/wasmtime++out=$("$WASMTIME" run --invoke _start "$CAND" 2>&1)++# EDIT: pin YOUR exact symptom. A parse error on a broken intermediate .wat will+# NOT contain this string, so creduce correctly rejects invalid candidates.+echo "$out" | grep -q 'integer divide by zero'++# After creduce finishes, eyeball case.wat and re-confirm it's still your bug.diff --git a/.agents/skills/reduce-test-cases/templates/predicate-differential.sh b/.agents/skills/reduce-test-cases/templates/predicate-differential.shnew file mode 100755index 0000000000..ce03e9b07c--- /dev/null+++ b/.agents/skills/reduce-test-cases/templates/predicate-differential.sh@@ -0,0 +1,53 @@+#!/usr/bin/env bash+#+# PREDICATE SKELETON for a MISCOMPILE — the input computes DIFFERENT results+# under two engine configs (two opt-levels, or two engines). "Interesting" ==+# they DISAGREE. This is the dominant shape for real miscompile reductions:+# there's no trap string or panic to grep, so you diff behavior instead.+#+# Written as a `wasm-tools shrink` predicate (candidate is $1, exit 0 ==+# reproduces). For the creduce variant see the note at the bottom.+#+# wasm-tools shrink ./this-script.sh input.wasm -o reduced.wasm+#+# BEFORE you trust this, confirm the bug is DETERMINISTIC. Run each config on+# the ORIGINAL a few times: every run of a given config must give the SAME+# answer, and the two configs must DISAGREE. If a single config is flaky+# (uninitialized memory, NaN payload bits, a nondeterministic host call) a+# differential predicate is UNSOUND — it "reproduces" on noise and the reducer+# converges on garbage. Pin the flakiness down (or seed it) first.++CAND="${1:?shrink passes the candidate as \$1}"++# EDIT: absolute path. echo "$PWD/target/release/wasmtime"+WASMTIME=/ABS/PATH/TO/target/release/wasmtime++# EDIT: the entry point, feature flags, and args your repro needs. Fuzzer repros+# almost always invoke a NAMED export with ARGUMENTS and enable a proposal with+# a -W flag (the fuzzers run the full proposal set), e.g.:+# -Wwide-arithmetic --invoke f "$CAND" 1 -1+# opt-level=0 vs default (default == opt-level=2) is the usual disagreeing pair;+# swap in two different engines if that's your differential instead.+a=$("$WASMTIME" -O opt-level=0 --invoke f "$CAND" 2>/dev/null); ra=$?+b=$("$WASMTIME" --invoke f "$CAND" 2>/dev/null); rb=$?++# Interesting == BOTH configs ran cleanly (exit 0) AND both produced output AND+# the outputs differ. The exit-0 and non-empty guards are ESSENTIAL, not+# decoration: without them a candidate that merely became INVALID (a parse+# error, empty output) or started TRAPPING would satisfy `a != b` trivially, and+# the reducer would happily converge on a broken program instead of a+# miscompile. This guarded form is exactly what the real reductions used.+[ $ra -eq 0 ] && [ $rb -eq 0 ] && [ -n "$a" ] && [ -n "$b" ] && [ "$a" != "$b" ]++# CREDUCE VARIANT: identical body, but reference the candidate by BASENAME+# (e.g. CAND=case.wat) instead of $1, run creduce from a fresh temp dir, and+# hand the .wat straight to wasmtime (no reassembly). The two runs, the four+# guards, and the determinism caveat are all the same.+#+# WAST VARIANT: make the reduced case SELF-CHECKING instead of diffing stdout —+# encode the correct answer as (assert_return ...) / (assert_trap ...) and let+# `wasmtime wast` at each opt-level pass or fail. Then "interesting" is+# "opt-level=0 passes but default fails":+# "$WASMTIME" wast -O opt-level=0 "$CAND" >/dev/null 2>&1; r0=$?+# "$WASMTIME" wast "$CAND" >/dev/null 2>&1; rd=$?+# [ $r0 -eq 0 ] && [ $rd -ne 0 ]diff --git a/.agents/skills/reduce-test-cases/templates/predicate-shrink.sh b/.agents/skills/reduce-test-cases/templates/predicate-shrink.shnew file mode 100755index 0000000000..05cf3998fd--- /dev/null+++ b/.agents/skills/reduce-test-cases/templates/predicate-shrink.sh@@ -0,0 +1,43 @@+#!/usr/bin/env bash+#+# PREDICATE SKELETON for `wasm-tools shrink` (wasm / wat).+#+# wasm-tools shrink ./this-script.sh input.wasm -o reduced.wasm+#+# shrink calls this script with the candidate module as $1 and keeps shrinking+# as long as the script exits 0. So: exit 0 == "still reproduces my bug".+#+# COPY this file, then edit the marked spots for YOUR bug. Test it on the+# ORIGINAL module first — if it doesn't exit 0 there, shrink has nothing to do.+#+# Do NOT add `set -e`: a trapping module makes wasmtime exit non-zero, and under+# `set -e` the `out=$(...)` line would abort the script before your grep runs.++CAND="${1:?shrink passes the candidate as \$1}"++# EDIT: absolute path to the engine you built. Absolute because reducers run+# from other directories. Get it with: echo "$PWD/target/release/wasmtime"+WASMTIME=/ABS/PATH/TO/target/release/wasmtime++# OPTIONAL — keep the reduction honest: require the instruction/feature that+# triggers your bug to still be present, so shrink can't "fix" it by deleting+# it. Uncomment and adapt:+# wasm-tools print "$CAND" | grep -q 'i32.div_s' || exit 1++# EDIT: the symptom. Pin it as tightly as you can — a loose predicate lets+# shrink converge on a DIFFERENT, spec-legal module that fails some other way.+# Pick/adapt ONE shape:+#+# trap message (this example):+out=$("$WASMTIME" run --invoke _start "$CAND" 2>&1)+echo "$out" | grep -q 'integer divide by zero'+#+# hard crash (segfault/abort => status >= 128; a clean wasm trap is lower):+# "$WASMTIME" run "$CAND" >/dev/null 2>&1; [ $? -ge 128 ]+#+# miscompile (two opt levels disagree): use templates/predicate-differential.sh+# instead — a differential needs exit-0 + non-empty guards or the reducer+# converges on a merely-broken program. The naive `[ "$a" != "$b" ]` is unsound.+#+# AFTER shrinking, eyeball the result — `wasm-tools print reduced.wasm` — and+# re-confirm it's still YOUR bug, not a lookalike.