Skip to content

Claude Code. Any Model. The most powerful AI coding agent now speaks every language.

test(bench): remove the CODEX_CODE_MODE_KEEP_LOGS log-copy stand-in

Jack Rudenkoca7a657e362e
diff --git a/ai-docs/benches/codex-code-mode/README.md b/ai-docs/benches/codex-code-mode/README.md
index 26c201f6..559de029 100644
--- a/ai-docs/benches/codex-code-mode/README.md
+++ b/ai-docs/benches/codex-code-mode/README.md
@@ -100,8 +100,7 @@ madbench madbench.yaml --harness mock # offline dry run
The real run, in a visible terminal (not piped, not backgrounded):
```bash
-CODEX_CODE_MODE_KEEP_LOGS=<dir> \
- madbench madbench.yaml --concurrency 4 --trials 5 --report-json <out.json>
+madbench madbench.yaml --concurrency 4 --trials 5 --report-json <out.json>
```
- **Concurrency needs a claudish build with `1066489b`** (see "Concurrency" below). An older
@@ -176,9 +175,6 @@ the `relayed= resumed= lost=` tally), so most failures can be read straight from
## Keeping the logs
-An `exec` Check keeps its output only on failure, and madbench deletes the workspace after the run,
-so a passing Check keeps none of the log lines it passed on. Until madbench keeps that output in the
-report (MadAppGang/madbench#142), the `provenance` Check carries a TEMPORARY first
-line: with `CODEX_CODE_MODE_KEEP_LOGS=<dir>` set it copies `logs/claudish_*.log` and
-`claudish-provenance.txt` to `<dir>/<workspace name>/`. Match a folder to its Scenario with the
-report's `.results[].session.work_dir`. Delete that line when madbench ships the feature.
+madbench deletes the workspace after the run, but an `exec` Check keeps its command's output in the
+report's Evidence whether it passes or fails (MadAppGang/madbench#142). The log lines a Check passed
+on are therefore in the report itself; read them there.
diff --git a/ai-docs/benches/codex-code-mode/madbench.yaml b/ai-docs/benches/codex-code-mode/madbench.yaml
index 2c035341..ed575b9e 100644
--- a/ai-docs/benches/codex-code-mode/madbench.yaml
+++ b/ai-docs/benches/codex-code-mode/madbench.yaml
@@ -88,17 +88,7 @@ scenarios:
type: exec
metric: provenance
# The shim ran, so this worktree's dist/index.js is what claudish was.
- #
- # TEMPORARY — the first line of this cmd stands in for
- # MadAppGang/madbench#142 (exec: keep the command's output in Evidence
- # when the Check passes; README.md, "Keeping the logs"). Delete it when
- # madbench ships it. An exec Check records its output only on failure and the
- # workspace is deleted after the run, so a PASS keeps none of the log
- # lines that proved it. With CODEX_CODE_MODE_KEEP_LOGS=<dir> set, the
- # logs are copied to <dir>/<workspace name>/ (map it to a Scenario by
- # the report's session.work_dir). Unset, it does nothing.
cmd: |
- if [ -n "${CODEX_CODE_MODE_KEEP_LOGS:-}" ]; then d="$CODEX_CODE_MODE_KEEP_LOGS/$(basename "$PWD")"; mkdir -p "$d" && cp claudish-provenance.txt logs/claudish_*.log "$d/" 2>/dev/null; fi
cat claudish-provenance.txt && grep -q "^dist=/.*/packages/cli/dist/index.js$" claudish-provenance.txt && grep -q "^version=claudish version " claudish-provenance.txt
# ── auto: code mode by discovery, the D1 regression ─────────────────────────