Skip to content

Official Codebase of 3DCodeVerse

Python

docs: prompts on stdin, only the routed skills, usage and provider wait from each CLI's record, typed round failures

gaoypeng12a6d6ea4ccc
diff --git a/harness/docs/ARCHITECTURE.md b/harness/docs/ARCHITECTURE.md
index 385a4a51..521672c6 100644
--- a/harness/docs/ARCHITECTURE.md
+++ b/harness/docs/ARCHITECTURE.md
@@ -115,9 +115,11 @@ codeverse3d/
probe: no retries, no backoff), schema_utils.py (strict schema), registry.py
agents/ registry.py (the CodingAgent protocol + dispatch — every backend is a
vendor CLI; the in-process api-agent died 2026-08-28), backends.py
- (gemini-cli / claude-code / codex / antigravity), cli_common.py (sessions,
- the watchdog clocks, the transcript, retry trajectory naming,
- files_changed attribution), materialize.py
+ (gemini-cli / claude-code / codex / antigravity: argv, env, each CLI's own
+ record read back — envelope, chat record / stream, retry reports), cli_common.py
+ (sessions, the watchdog clocks, the prompt on stdin, the transcript, retry
+ trajectory naming, files_changed attribution, THE failure vocabulary,
+ provider_wait), materialize.py
languages/ LanguageRuntime + RuntimeLayout (base.py: every runtime states its file layout —
expected_files / files_for — and the tracks ask it); one merged module per language since 2026-08-28 —
blender/ cadquery/ threejs/ urdf/ scene_threejs/ glsl_shader/ opengl_python/ are each
@@ -318,7 +320,8 @@ runs/<slug>/
judge/rNN.json (+ rNN_cli.json from `3dcode judge`; rAA_vs_rBB_pairwise.json from `3dcode pick --by pairwise`)
tool_renders/rNN_<hash>/
trajectories/<label>_rNN/ prompt.md transcript.jsonl stdout.json stderr.log result.json
- (a retried label lands in <label>.a2_rNN — first attempt preserved)
+ (+ gemini_settings.json · mcp.json (claude) · agy.log (agy's own CLI log));
+ a retried label lands in <label>.a2_rNN — first attempt preserved
AGENTS.md GEMINI.md CLAUDE.md .gemini/settings.json .3dcode/cookbook.md .geminiignore .aiexclude
```
@@ -485,6 +488,10 @@ plateau, regression or diminishing-returns stop, no rewrite or surface-detail ro
a round left without a verdict (judge outage / degraded) is re-judged once before the next is planned
every round emits cost.round {stage → $, judge $, agent turns, wasted flag}; a round that raises mid-way
still reports what it burned (rounds/aborted_rNN.json + record.extra["aborted_rounds"])
+ a round whose every task failed raises RoundFailed, TYPED (2026-09-22 — the loop never reads its message):
+ .quota (a backend saw the vendor's usage limit) → agent_quota; .transient (a session died of a provider
+ failure — AgentResult.transient, or a ModelError outage) → the SAME round re-runs once, baseline included;
+ neither → no_change (a refine round) / the run fails (the baseline)
stop reasons: max_rounds | budget (the wall clock — BudgetGuard / max_minutes) | agent_quota | no_change (a refine
round's sessions changed nothing) | no_refine_tasks | judge_unavailable | failed
(a record from before 2026-09-22 may say passed / plateau: it loads as `stopped`)
@@ -571,7 +578,13 @@ $0.02–0.03 (flash) / ~$0.2 (pro); api-agent generation 3–6 min per object ro
* Scenes: fps is a relative cost; camera-in-geometry can miss open-back enclosures.
* threejs: textures are stripped on GLB export (the texture pass re-adds them as a
derived pack); `userData.tick` cannot survive export.
-* agy exposes no per-workspace MCP, cost or served model.
+* agy exposes no per-workspace MCP, cost or served model; a killed agy session books no
+ usage (its tokens are in the envelope only). A killed codex session neither (usage is
+ per turn and `--ephemeral` leaves no rollout); a killed claude-code session books its
+ per-message usage, whose output side is a floor. A killed or given-up gemini-cli
+ session books its chat record (exact).
+* Vendor sessions see only the routed skill bundles, except agy (5 built-ins; no
+ per-session switch) and any user-level skill root codex or gemini-cli might grow.
* Anthropic / OpenAI backends are mock-tested only (no keys on this box).
* Budget checks run between steps: a round the clock cuts mid-way is rolled back (its
spend stays in `aborted_rounds`) — give scenes `--max-minutes 60`.
diff --git a/harness/docs/INTERFACES.md b/harness/docs/INTERFACES.md
index 96fd0e1b..ee36631e 100644
--- a/harness/docs/INTERFACES.md
+++ b/harness/docs/INTERFACES.md
@@ -181,6 +181,19 @@ res = agent.run(AgentJob(workspace=..., prompt=..., label="baseline", timeout_s=
res.ok, res.exit_reason # completed | timeout | error | budget | model_substituted
res.turns # claude-code num_turns · codex turn.completed count · agy num_turns · gemini-cli 0 (not on the wire)
res.files_changed (git-derived, attributed per session), res.usage, res.transcript_path, res.tool_calls, res.errors
+res.transient # a provider failure a retry may get through (5xx / overloaded / 429 / dropped connection)
+res.quota # Δ 2026-09-22 the vendor's usage limit is spent — never also transient
+res.provider_wait_s # Δ seconds of duration_s lost to provider errors (cli_common.provider_wait; codex: 0.0)
+# Δ 2026-09-22 the prompt reaches EVERY CLI on stdin (cli_common.invoke; no -p / --print / positional text,
+# codex "-"), byte for byte; the >100 kB task_prompt.md stub is gone. The transcript's invoke row keeps
+# argv + stdin_bytes. res.usage is the CLI's envelope; with none (killed / given up) gemini-cli's comes
+# from its chat record (read_gemini_chats) and claude-code's from its stream's per-message usage
+# (output a floor); result.json "usage_from" says which. A session sees only the routed skill bundles:
+# claude --setting-sources project + CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1 + --settings skillOverrides;
+# codex -c skills.bundled.enabled=false; gemini-cli skills.disabled (its two built-ins); agy: no switch.
+from codeverse3d.agents.cli_common import is_transient_failure, is_quota_failure, is_rate_limited, provider_wait
+# THE failure vocabulary (one place; the backends apply it to what the CLI said about the call that ended
+# its session) and provider_wait(failures=[(t, backoff_s)], end, progress=[t] | None) -> seconds
```
`AgentJob` carries typed job context: `round`, `kind`, `language`, `track`,
`mcp_command` (override), `files_hint` (workspace-relative files/dirs the task is
@@ -504,6 +517,12 @@ GenerationTask.phase: int = 0 # tasks run in parallel WITHIN a phase, phases i
# entry file and the placement gates. Every other caller is phase 0, i.e. unchanged.
generate(ws, *, agent_id, task, ..., budget=BudgetGuard) -> GenerationResult
# GenerationResult.storm: every session died on a transient streak (AgentResult.transient) and wrote nothing
+ # Δ 2026-09-22 GenerationResult.transient / .quota carry the backend's typed class; a call that RAISED is
+ # GenerationResult.from_error(label, e) — transient = tracks.generation.is_model_outage(e) (ModelError
+ # retryable or 429/5xx; moved from scene_assets, the one exception classifier)
+from codeverse3d.tracks.steps import RoundFailed # RoundFailed(message, *, transient=False, quota=False);
+ # RoundFailed.of(results, default) — the round loop reads .quota (→ agent_quota) and .transient (→ re-run
+ # the round once), never the message (looks_transport / looks_quota deleted)
tracks.common.generate_for(ctx: RunContext, task: GenerationTask) -> GenerationResult
# THE call every stage makes (env, zones, rounds, repairs, asset ladder, judged fix): generate()
# with everything ctx knows, and a storm-dead task retried through single_shot_ctx(ctx) (D68)
diff --git a/harness/docs/RUNBOOK.md b/harness/docs/RUNBOOK.md
index 0c62ad64..fbc1c33a 100644
--- a/harness/docs/RUNBOOK.md
+++ b/harness/docs/RUNBOOK.md
@@ -77,11 +77,11 @@ round, then ≈ $0.36 / ~7 min per refine (give scenes `--max-minutes 60`).
| id | what runs | notes |
|---|---|---|
-| `gemini-cli:gemini-3.7-flash` (default) | `gemini -p … --approval-mode yolo --skip-trust --output-format json`; every spatial tool over MCP | cheapest agentic path; transcripts feed repair-pair mining; see gotchas below |
+| `gemini-cli:gemini-3.7-flash` (default) | `gemini -m … --approval-mode yolo --skip-trust --output-format json`, prompt on stdin; every spatial tool over MCP | cheapest agentic path; transcripts feed repair-pair mining; see gotchas below |
| `single-shot:gemini:gemini-3.7-flash` | one structured-output call → multi-file envelope, no tools | fastest/cheapest; baseline for "raw model" deltas |
-| `claude-code:<model>` | `claude -p … --dangerously-skip-permissions --mcp-config trajectories/<label>_rNN/mcp.json --strict-mcp-config …` | local subscription — test lightly |
-| `codex:<model>[@<effort>]` | `codex exec --json -C ws --sandbox workspace-write -c model_reasoning_effort=high … -c mcp_servers.3dcode.…` | subscription; MCP tools need `default_tools_approval_mode="approve"` (harness passes it); reasoning effort is always stated (`Settings.agents.codex_reasoning_effort`, default `high`; `codex:gpt-5.6-sol@medium` per id, `""` to defer to `~/.codex/config.toml`) |
-| `agy:<model>` | `agy --print … --add-dir ws` | no per-workspace MCP: tools via `3dcode tools <name> --json … --workspace .`; no served-model or cost reporting |
+| `claude-code:<model>` | `claude -p --dangerously-skip-permissions --setting-sources project --settings '{"skillOverrides":…}' --mcp-config trajectories/<label>_rNN/mcp.json --strict-mcp-config …`, prompt on stdin, `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1` | local subscription — test lightly; no user setting source, so `~/.claude/settings.json` (effortLevel, hooks, env) does not reach a session |
+| `codex:<model>[@<effort>]` | `codex exec --json -C ws --sandbox workspace-write -c model_reasoning_effort=high -c skills.bundled.enabled=false -c mcp_servers.3dcode.… -`, prompt on stdin | subscription; MCP tools need `default_tools_approval_mode="approve"` (harness passes it); reasoning effort is always stated (`Settings.agents.codex_reasoning_effort`, default `high`; `codex:gpt-5.6-sol@medium` per id, `""` to defer to `~/.codex/config.toml`) |
+| `agy:<model>` | `agy --add-dir ws --log-file trajectories/<label>_rNN/agy.log …`, prompt on stdin (no `--print`: `--print -` sends "-") | no per-workspace MCP: tools via `3dcode tools <name> --json … --workspace .`; no served-model or cost reporting; truncates a prompt past ~175 kB itself |
| `gemini:* / anthropic:* / openai:*` | ChatModel for planner / judge / captioner / single-shot | Anthropic/OpenAI untested live here |
### gemini-cli gotchas (handled by `agents/backends.py`; do not undo)
@@ -91,8 +91,13 @@ round, then ≈ $0.36 / ~7 min per refine (give scenes `--max-minutes 60`).
(else workspace MCP servers are silently ignored even with `--skip-trust`).
* Workspace `.gemini/settings.json` sets `context.fileFiltering.respectGitIgnore=false`
so the fine-grained `.geminiignore` (not the git ignore) decides what the agent can
- read: build/census/measurement JSON and its own `task_prompt.md` stay readable,
- renders/judge output/transcripts stay hidden.
+ read: build/census/measurement JSON stay readable, renders/judge output and the whole
+ `trajectories/` stay hidden (the prompt comes on stdin; there is no prompt file).
+* The system settings also disable gemini-cli's two built-in skills (`skills.disabled`),
+ so a session lists only the routed bundles.
+* Usage when the CLI printed no envelope (watchdog kill; a give-up after its own 503
+ retries, which writes an error object to STDERR and exits 247): the attempt's chat
+ record under `~/.gemini/tmp/<project>/chats/` — `result.json` `usage_from` says which.
* One pool key injected as `GEMINI_API_KEY`; other credential env stripped; retries
prefer a different key (never raise KeyPoolExhausted out of a session).
* Cost accounting: `tokens.prompt` (total, incl. cached) is the input count; each
diff --git a/harness/docs/SKILLS.md b/harness/docs/SKILLS.md
index fd85a64d..cd7ca9e2 100644
--- a/harness/docs/SKILLS.md
+++ b/harness/docs/SKILLS.md
@@ -53,6 +53,18 @@ Bundles reach a session only through its vendor's native skill loader: the harne
hands the agent no second index (the loader's own would list the same skills twice) and no tool;
the body file gets one MANDATORY sentence.
+**And a session sees nothing but the routed bundles** (2026-09-22). Every CLI also lists
+skills of its own; each is switched off with the CLI's documented knob where it has one
+(`agents/backends.py`), checked at zero cost against the real binaries with a local fake API
+(the claude `init` event, the codex and gemini request bodies) and `codex debug prompt-input`:
+
+| CLI | listed besides the routed bundles | switch | left |
+|---|---|---|---|
+| `claude-code` 2.1.280 | 26: bundled (`update-config`, `code-review`, `loop`, …), account-synced (`deep-research`, `anthropic-skills:*`, …) | `--setting-sources project` (synced skills and plugins come with the "user" source) + `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1` + `--settings {"skillOverrides": {"design": "off", "doctor": "off"}}` (the two that survive the bundled switch by design) | nothing — and the owner's `~/.claude/settings.json` (effortLevel, hooks, env) no longer reaches a session |
+| `codex` 0.155.1 | 5 `.system` (imagegen, openai-docs, plugin-creator, skill-creator, skill-installer) | `-c skills.bundled.enabled=false` | any user root (`~/.codex/skills`, `~/.agents/skills`): no stable switch; none here |
+| `gemini-cli` 0.53.0 | 2 built-ins (skill-creator, antigravity-support) — `gemini skills list` hides them, the model does not | `skills.disabled` in the per-session system settings (by name) | the user roots (`~/.gemini/skills`, `~/.agents/skills`): no switch; none here |
+| `agy` 1.2.8 | 5 built-ins (agy-customizations, antigravity-guide, generative_ui, migrate-workflows, permissioned-github; `agy -p /skills` lists them without a turn) | none per session: built-ins survive even an agent's `inherit_user: false` | all 5 |
+
**Adding a backend is one row in that table.** A test (`tests/skills/test_delivery.py`)
fails if per-backend knowledge leaks back out into the other modules, and another checks
that `prompting.py` agrees with the policy rather than re-deriving it.
@@ -233,7 +245,7 @@ failure verdict as `failed`), then one `tool_trace` row that closes the session
| backend | source | skill activation looks like |
|---|---|---|
| claude-code | `--output-format stream-json --verbose` (was `json`, which carried no tool stream); the `init` event's `skills` list is kept as `skills_index` | `Skill {"skill": name}` |
-| gemini-cli | its chat record, `~/.gemini/tmp/<project>/chats/session-*.jsonl`, found by the project's `.project_root` + mtime — so a session the watchdog killed before it printed JSON still has one. `--output-format json` stays: it has per-tool COUNTS only, and `stream-json` drops the `thoughts` tokens pricing needs | `activate_skill {"name": name}` |
+| gemini-cli | its chat record, `~/.gemini/tmp/<project>/chats/session-*.jsonl`, found by the project's `.project_root` + mtime — so a session the watchdog killed before it printed JSON still has one (and, since 2026-09-22, its usage: `read_gemini_chats` reads calls, tokens and times in one pass). `--output-format json` stays: it has per-tool COUNTS only, and `stream-json` drops the `thoughts` tokens pricing needs | `activate_skill {"name": name}` |
| codex | the `exec --json` events it already streamed (`command_execution`, `mcp_tool_call`, `file_change`) | a `sed`/`cat` of `…/<name>/SKILL.md` |
| agy | its conversation database, `~/.gemini/antigravity-cli/conversations/<conversation_id>.db` — a schema-free walk of each step's protobuf for (call id, tool, JSON args) | `view_file` of `…/<name>/SKILL.md` |
diff --git a/harness/docs/SKILLS_LEDGER.md b/harness/docs/SKILLS_LEDGER.md
index 27f6c06f..97865a7c 100644
--- a/harness/docs/SKILLS_LEDGER.md
+++ b/harness/docs/SKILLS_LEDGER.md
@@ -115,7 +115,8 @@ system settings listed the four + the control, all `[Enabled]`, from
* The two gemini sessions were killed by a provider storm, not by skills; both still left
their tool trace (found by `.project_root` + mtime, not `session_id`, for exactly this
case). They also expose a cost gap outside this lane: a killed gemini-cli session is
- recorded at $0 although its chat record holds ~$0.34 of tokens.
+ recorded at $0 although its chat record holds ~$0.34 of tokens (closed the same day:
+ the chat record is such a session's usage — `agents/backends.read_gemini_chats`).
* codex's MCP server did not start in this rig (codex hands MCP servers a scrubbed env; the
lane worktree is on `PYTHONPATH`, not installed), so it spent its session probing the
server by hand — a rig artefact, not a skills result.