Claude Code. Any Model. The most powerful AI coding agent now speaks every language.
test(bench): remove the CODEX_CODE_MODE_KEEP_LOGS log-copy stand-in
Jack Rudenkoca7a657e362e
diff --git a/ai-docs/benches/codex-code-mode/README.md b/ai-docs/benches/codex-code-mode/README.mdindex 26c201f6..559de029 100644--- a/ai-docs/benches/codex-code-mode/README.md+++ b/ai-docs/benches/codex-code-mode/README.md@@ -100,8 +100,7 @@ madbench madbench.yaml --harness mock # offline dry runThe real run, in a visible terminal (not piped, not backgrounded):```bash-CODEX_CODE_MODE_KEEP_LOGS=<dir> \- madbench madbench.yaml --concurrency 4 --trials 5 --report-json <out.json>+madbench madbench.yaml --concurrency 4 --trials 5 --report-json <out.json>```- **Concurrency needs a claudish build with `1066489b`** (see "Concurrency" below). An older@@ -176,9 +175,6 @@ the `relayed= resumed= lost=` tally), so most failures can be read straight from## Keeping the logs-An `exec` Check keeps its output only on failure, and madbench deletes the workspace after the run,-so a passing Check keeps none of the log lines it passed on. Until madbench keeps that output in the-report (MadAppGang/madbench#142), the `provenance` Check carries a TEMPORARY first-line: with `CODEX_CODE_MODE_KEEP_LOGS=<dir>` set it copies `logs/claudish_*.log` and-`claudish-provenance.txt` to `<dir>/<workspace name>/`. Match a folder to its Scenario with the-report's `.results[].session.work_dir`. Delete that line when madbench ships the feature.+madbench deletes the workspace after the run, but an `exec` Check keeps its command's output in the+report's Evidence whether it passes or fails (MadAppGang/madbench#142). The log lines a Check passed+on are therefore in the report itself; read them there.diff --git a/ai-docs/benches/codex-code-mode/madbench.yaml b/ai-docs/benches/codex-code-mode/madbench.yamlindex 2c035341..ed575b9e 100644--- a/ai-docs/benches/codex-code-mode/madbench.yaml+++ b/ai-docs/benches/codex-code-mode/madbench.yaml@@ -88,17 +88,7 @@ scenarios:type: execmetric: provenance# The shim ran, so this worktree's dist/index.js is what claudish was.- #- # TEMPORARY — the first line of this cmd stands in for- # MadAppGang/madbench#142 (exec: keep the command's output in Evidence- # when the Check passes; README.md, "Keeping the logs"). Delete it when- # madbench ships it. An exec Check records its output only on failure and the- # workspace is deleted after the run, so a PASS keeps none of the log- # lines that proved it. With CODEX_CODE_MODE_KEEP_LOGS=<dir> set, the- # logs are copied to <dir>/<workspace name>/ (map it to a Scenario by- # the report's session.work_dir). Unset, it does nothing.cmd: |- if [ -n "${CODEX_CODE_MODE_KEEP_LOGS:-}" ]; then d="$CODEX_CODE_MODE_KEEP_LOGS/$(basename "$PWD")"; mkdir -p "$d" && cp claudish-provenance.txt logs/claudish_*.log "$d/" 2>/dev/null; ficat claudish-provenance.txt && grep -q "^dist=/.*/packages/cli/dist/index.js$" claudish-provenance.txt && grep -q "^version=claudish version " claudish-provenance.txt# ── auto: code mode by discovery, the D1 regression ─────────────────────────