Official Codebase of 3DCodeVerse
Python
docs: fixed rounds, every round kept, the pick after the run
gaoypeng00352bd316aa
diff --git a/eval/docs/EVAL.md b/eval/docs/EVAL.mdindex 7152e85f..4b9b4b60 100644--- a/eval/docs/EVAL.md+++ b/eval/docs/EVAL.md@@ -63,17 +63,22 @@ deterministic contract check.3dcode bench run bench/prompts/static_objects_v1.yaml --generator gemini-cli:gemini-3.7-flash --judge gemini:gemini-3.1-pro-preview --out bench/out/static_v1_gemcli3dcode bench report bench/out/static_v1_apiagent```-Each item = one full `3dcode make` run (its own workspace under `--out`); `results.jsonl`-rows carry `score_baseline`, `score_final`, `passed`, `rounds`, `cost_usd`, `minutes`,-`status`, `generator`, `judge`. `report.md/html` tabulates per tier and category:-mean baseline → final (the harness delta), pass rate, cost, time. Keep `--judge` fixed-across arms; vary only `--generator` (and `--planner` if that is the variable).-Resumable: re-running the same `--out` skips finished ids.--Reading the numbers: *baseline* (round 0) is the raw-model-with-contract result;-*final − baseline* is what the refine loop buys; the pass rate at the rubric threshold-(0.72 for `static_object_v1`) is the headline. Observed so far (flash judge,-n=1, single runs): chair 0.67 → 0.74, bench 0.64 → 0.89, cabinet 0.68 → 0.93,+Each item = one full `3dcode make` run (its own workspace under `--out`) followed by the+hand-over `make` does — `addons/select.pick` by score, then `package` (deliverable/ ++selection.json); `results.jsonl` rows carry `score_baseline`, `score_picked` (rows written+before 2026-09-22 say `score_final`), `picked_round`, `rounds`, `cost_usd`, `minutes`,+`status` (the stop reason), `generator`, `judge`. No `passed` column: since 2026-09-22 a run+is not passed or failed. `report.md/html` tabulates per tier and category: mean baseline →+picked (the harness delta), cost, time. Keep `--judge` fixed across arms; vary only+`--generator` (and `--planner` if that is the variable). Resumable: re-running the same+`--out` skips finished ids.++Reading the numbers: *baseline* (round 0) is the raw-model-with-contract result; *picked −+baseline* is what the refine loop buys — but the picked round is the best-scored of the+baseline + `--rounds` refine rounds, so that delta is ≥ 0 by construction and carries the+judge's noise as a maximum over n draws: compare arms on the picked score itself, at the+same `--rounds`. Observed before 2026-09-22 (flash judge, n=1, single runs, runs that+stopped on a pass): chair 0.67 → 0.74, bench 0.64 → 0.89, cabinet 0.68 → 0.93,scene 0.56 → 0.58 after one refine.## 4. Harness vs raw one-shot (`bench/compare_backends.py`)@@ -159,8 +164,8 @@ Pending: rerun the parked sol battery, then terra, then luna, then the claude-co## 5. Comparing backends inside the harnessSame battery, same `--judge`, different `--generator` (`api-agent:*`, `single-shot:*`,-`gemini-cli:*`, `claude-code:*`, `codex:*`, `agy:*`). Report baseline, final, delta,-pass rate, cost, time per tier; optionally feed the best rounds of two arms through+`gemini-cli:*`, `claude-code:*`, `codex:*`, `agy:*`). Report baseline, picked, delta,+cost, time per tier; optionally feed the picked rounds of two arms through`PairwiseJudge.compare(spec, renders_a, renders_b)` (`flywheel pairs` already emitscross-backend candidate pairs keyed by prompt hash).@@ -638,7 +643,8 @@ Until this date a profile that injected a `judge_samples>1` policy (economy, quasilently zeroed the static track's surface-detail round while balanced kept it; allthree profiles now get it, and `C3D_DETAIL_ROUNDS` remains the A/B switch. Anyeconomy/quality-vs-balanced comparison straddling this commit compares different-round counts.+round counts. (2026-09-22: the detail round is gone with the other judgement-driven rounds;+`C3D_DETAIL_ROUNDS` is a dead switch, and every run is the baseline + `--rounds`.)**2026-08-29 — per-language system prompts: NULL, three independent A/Bs.** Theone-line system prompts were replaced with evidence-grounded ones mined from eachdiff --git a/eval/docs/PAPER_WRITING.md b/eval/docs/PAPER_WRITING.mdindex cc193e4a..5578fa62 100644--- a/eval/docs/PAPER_WRITING.md+++ b/eval/docs/PAPER_WRITING.md@@ -25,8 +25,8 @@ format).## 0. The system, in one paragraphA request becomes an iterated loop: **plan → generate code → build → deterministic gates →-render → VLM judge → refine**, for N rounds under a wall-clock and dollar budget, delivering-the best round and recording every round. The generator is a coding agent writing real+render → VLM judge → refine**, for a fixed N rounds under a wall-clock budget, recording every+round and handing over the best-scored one after the run. The generator is a coding agent writing realsource (Blender `bpy`, CadQuery, three.js, GLSL, or a URDF robot plus a `bpy` link builder);the gates are program analysis, not model calls (lint, connectivity, dimensional contract,and — on the articulated track — a joint sweep that collides every link pair in every sampleddiff --git a/harness/CLAUDE.md b/harness/CLAUDE.mdindex 966639bc..ed127460 100644--- a/harness/CLAUDE.md+++ b/harness/CLAUDE.md@@ -89,6 +89,8 @@ cd /home/yipeng/3dcodeverse/harness3dcodeverse make "..." --track scene --language scene_threejs --rounds 2 --max-minutes 603dcodeverse make "neon rain on a window" --track graphics --language glsl_shader3dcodeverse make "..." --image ref.png --candidates 3 --rounds 2 --dim height=0.45 --must "three legs" --texture --no-run+3dcodeverse make "..." --no-pick # a run = baseline + --rounds refine rounds (fixed); make then picks a round → deliverable/ unless --no-pick+3dcode pick <slug> [--by score|pairwise] [--round N] [--texture] [--judge MODEL] # hand over a round (codeverse3d/addons/select.py)3dcode resume <slug> · 3dcode status <slug> · 3dcode render <slug> [--mode wire] · 3dcode judge <slug> [--model ... --n 3]3dcode texture pass <slug> [--no-judge] · 3dcode texture scene-pack <slug> · 3dcode texture show <slug>3dcode tools list · 3dcode tools measure --workspace runs/<slug> · 3dcode mcp --workspace runs/<slug>diff --git a/harness/docs/ARCHITECTURE.md b/harness/docs/ARCHITECTURE.mdindex b8512b53..ff7adb96 100644--- a/harness/docs/ARCHITECTURE.md+++ b/harness/docs/ARCHITECTURE.md@@ -31,15 +31,18 @@ layered on top of the language — for four **tracks**:of files, no tools) handled inside `tracks/generation.py`.Every run is: **spec → plan → skeleton → [scene stages] → baseline round-(optionally best-of-N candidates) → refine rounds → finalise → record**, with an-optional post-hoc **texture pass**. The record (plus git history of `src/`) is-the flywheel unit.+(optionally best-of-N candidates) → `max_rounds` refine rounds → finalise → record**.+Which round to hand over is decided AFTER the run, by the `addons/select` reader+(`3dcode pick`; `3dcode make` calls it by score): it writes `deliverable/` for that+round and runs the optional **texture pass** on it. The record (plus git history of+`src/` and every round's kept build) is the flywheel unit.## 1. Design laws (from the reference post-mortems)1. **Code is truth; artifacts are derived.** Raw code in `src/` is the- deliverable; GLB/renders are regenerated from it (finalise restores the best- commit and rebuilds). Textures are a *derived asset pack*+ deliverable; GLB/renders are regenerated from it. Every round is a commit and+ keeps its own built files under `artifacts/rNN/`, so any round can be handed over+ without a rebuild; the workspace ends at the last round. Textures are a *derived asset pack*(`object_textured.glb` + `artifacts/textures/`); `object.glb` is never touched.2. **Concrete beats abstract.** Contracts are delivered as skeleton code, exactnumbers and copyable snippets (cookbooks), never as clauses.@@ -167,19 +170,19 @@ codeverse3d/(world-metre unwrap, PBR map set, application + material normalisation),materials.py (named material library), run.py (texture_pass); __init__.py isdocstring-only — import from the submodules- orchestrator.py the round loop's LIBRARY, not the loop: round POLICIES (RoundPolicy /- StopPolicy / pick_best_round), refine-task compilation + grouping,+ orchestrator.py the round loop's LIBRARY, not the loop: the round knobs (RoundPolicy —+ no stop knob besides max_rounds), refine-task compilation + grouping,StageRunner + RunState (resume), BudgetGuard. The loop itself istracks/lifecycle.py:_round_loop → tracks/steps.py:run_round. A best-of-Ncandidate IS steps._run_round(kind='candidate') in a _cand/c<k> sub-workspacewith two knobs (render=candidates.quick_render, geometry_views=False), itsown _cand/c<k>/events.jsonl and a one-sample judgetracks/ __init__.py (get_track(track, **options) → a lifecycle.BaseTrack),- lifecycle.py, steps.py, candidates.py (best-of-N + the pure candidate/pairwise- decision logic), generation.py (agent + single-shot strategies + the file+ lifecycle.py, steps.py, candidates.py (best-of-N baseline candidates and+ their ranking), generation.py (agent + single-shot strategies + the fileenvelope), repair.py, planner.py, prompting.py (prompt helpers, split from common),- common.py (RunContext, Services), static_object.py (+ the detail round and the- reference-image gates), articulated_object.py (+ the planned-motion gate),+ common.py (RunContext, Services), static_object.py (+ the reference-image+ gates), articulated_object.py (+ the planned-motion gate),scene.py, scene_assets.py (+ cheap single-shot asset generation),zone_layout.py (L2 zone director: per-zone structured layout calls + deterministic validator),graphics.py (the whole graphics track: planner hooks, prompt context, frame@@ -195,7 +198,9 @@ codeverse3d/skills_hook.py (the round's view of codeverse3d/skills: attach before generating,probe reads after — a no-op under C3D_SKILLS=0; on by default since 2026-09-22)record/ what every run WRITES: record.py (finalize_record, load_record, iter_runs,- best_round_record), deliverable.py, telemetry.py, _git.py (files at a round's commit)+ effective_judgment), deliverable.py (every round's kept build under+ artifacts/rNN/, and deliverable/ for ONE round — built by addons/select),+ telemetry.py, _git.py (files at a round's commit)addons/ optional tools that READ finished runs; outside cli/ nothing imports them(tests/core/test_addons_boundary.py). dataset/ = export.py, pack.py, pairs.py,refine.py, captions.py, index.py, sample.py, quality.py (tiers + code fingerprints +@@ -217,7 +222,7 @@ codeverse3d/prompts/ EVERY piece of prompt material the harness writes, and the only place itlives: <lang>/{system,contract,cookbook}.md (incl. glsl_shader/,opengl_python/), system/* (harness contract, single-shot envelope,- role_{scope,detail,repair}.j2), texturing/*.md, tracks/*.j2.+ role_{scope,asset,repair}.j2), texturing/*.md, tracks/*.j2.catalog.py answers "what exists and how does each piece reach the model"— including the ONE language-id → prompts/<dir> mapping, which used to becopied four times and missing in a fifth. sections.py splits that@@ -268,6 +273,8 @@ tests/ agents blender_cadquery core cost flywheel_cli gallery gra```runs/<slug>/spec.json plan.json run_state.json record.json events.jsonl+ selection.json (after a pick) the round deliverable/ holds, how it was chosen, every round's score+ deliverable/ (after a pick) that round's code + artifacts + sheet + manifest.json (RUN_LAYOUT.md)src/ agent-authored RAW code (git repo; commits: spec, skeleton, pre:/agent:<label>, rNN <kind>)public/ (scene) compiled assets public/assets/<snake>.glb; (textured scenes) public/textures/*.png + manifest.json_assets/<snake>/ (scene) sub-workspaces for blender_glb assets (gitignored)@@ -281,11 +288,14 @@ runs/<slug>/alias; runs before 2026-08-23 have the root file only)run_state.json stages + rounds done; extra carries budget_snapshot and spec_fingerprint onlyartifacts/ object.glb object.stl|step robot.urdf meshes/ articulation.json build.json census.json- measurement.json … ; graphics: frames/fNN_tT.png frames_sheet.png preview.gif metrics.json- texturing: object_textured.glb textures/{<id>.png, texture_plan.json, texturing.json, gate/}+ measurement.json … — the LAST round's build; graphics: frames/fNN_tT.png frames_sheet.png+ preview.gif metrics.json; texturing (a pick's --texture): object_textured.glb+ textures/{<id>.png, texture_plan.json, texturing.json, gate/}+ rNN/ the round's own build, copied when it built: object.glb object.stl|step robot.urdf+ meshes/*.glb preview.gif frames_sheet.png (only what a hand-over needs)renders/rNN/ view_<name>.png sheet.png views.json (judge flags) (+ poses/ articulated; <cam>_t<t>.png metrics.json scenes)gates/rNN/ lint_<lang>.json connectivity.json contract.json joint_sweep.json motion_direction.json … (+ *_tool.json)- judge/rNN.json (+ rNN_cli.json from `3dcode judge`)+ judge/rNN.json (+ rNN_cli.json from `3dcode judge`; rAA_vs_rBB_pairwise.json from `3dcode pick --by pairwise`)tool_renders/rNN_<hash>/trajectories/<label>_rNN/ prompt.md transcript.jsonl stdout.json stderr.log result.json(a retried label lands in <label>.a2_rNN — first attempt preserved)@@ -408,7 +418,7 @@ citation auditable. Floors,deterministic caps from gate findings (`data["kind"]`), console errors, missingmust-acceptance and`missing_views` rules apply on top; degraded verdicts are glitches, not scores.-`PairwiseJudge` (position-swapped, tie on disagreement); `ReferenceJudge` for+`PairwiseJudge` (position-swapped, tie on disagreement; only `3dcode pick --by pairwise` asks it); `ReferenceJudge` forimage-conditioned specs.Rubrics: `static_object_v1` (0.72), `articulated_v1` (requires pose sheet),`scene_v1` (frame-gate caps), `asset_v1`, `reference_v1`, `shader_v1` (0.70).@@ -435,14 +445,11 @@ round 0 "baseline": generate → build_with_repair → measure → gates → rensession that owns the entry file, placement and the connectivity/contract gates. $C3D_SCOPED_PARTS=offrestores the single whole-object session; single-shot always uses it.)(--candidates N: N parallel baselines in <ws>/_cand/c<k>, quick 4-view judge, crashed candidate retried once,- selection build_ok → quick score → fewer gate errors with pairwise tie-break; winner copied back, normal r00 follows)-repeat while StopPolicy says continue (≤ max_rounds refine rounds, plateau_window=2 / min_delta=0.02,- target = rubric threshold, budget ok):- money stops, sized by the judge's MEASURED noise σ (cost/routing.JUDGE_NOISE: pro 0.030, flash 0.083):- regression — last round scored < best − 1σ ⇒ never another round of the same shape: the first one- switches strategy (ONE whole-artifact rewrite, kind "rewrite"), a second stops the run- diminishing_returns — from r03 on, only start when the last gain > 1.5σ AND best < target- refine tasks = gate ERRORS (fix hints, authoring-frame numbers) ∪ failed must-acceptance ∪ judge improvement plan+ selection build_ok → quick score → fewer gate errors → the earlier candidate; winner copied back, normal r00 follows)+then max_rounds refine rounds, EACH built on the round before it — FIXED rounds (owner, 2026-09-22): no pass,+plateau, regression or diminishing-returns stop, no rewrite or surface-detail round, no refine-from-best:+ refine tasks = the previous round's gate ERRORS (fix hints, authoring-frame numbers) ∪ failed must-acceptance+ ∪ its judge improvement plan(≤ 6 tasks, ≤ 6 compacted instruction lines each; reference runs add an IoU task when silhouette IoU < 0.6)fan out when ≥ 2 file-disjoint groups AND every task maps to files (threejs/blender parts, scene zones/assets/env)generate (no HARNESS turn cap by default — claude-code runs under AgentJob.max_turns=60 (+6-turn wrap-up),@@ -450,19 +457,18 @@ repeat while StopPolicy says continue (≤ max_rounds refine rounds, plateau_wina cap the machine sets (C3D_AGENT_MAX_TURNS / limits.agent_max_turns; no profile sets one) still buys a wrap-up sessionthat lands a final build + summary instead of being killed) → build+repair (error-focused,escalates on identical signatures) → gates → … → judge (SKIPPED only where the verdict is never- bought at all: no judge/renders or budget already exceeded)- pick_best_round: highest score, tie → fewer gate errors; |Δ| < pairwise_margin (0.03) → position-swapped- PairwiseJudge decides (replace only at confidence ≥ 0.6; note persisted in rNN.json)+ bought at all: no judge or no renders — a round that finishes past the clock is still judged)+ → commit src/ (the round's commit) and copy its build to artifacts/rNN/+ a round left without a verdict (judge outage / degraded) is re-judged once before the next is plannedevery round emits cost.round {stage → $, judge $, agent turns, wasted flag}; a round that raises mid-waystill reports what it burned (rounds/aborted_rNN.json + record.extra["aborted_rounds"])- a plateau / diminishing_returns stop on a CLEAN artifact (built, judged, no gate ERROR, score ≥ 0.45 and- within σ of the best) is converted into ONE round of kind "detail" instead: surface detail only — bevels,- panel lines, fasteners, wear, material variation — with the silhouette, placement and part list frozen and- a deterministic `detail_drift` gate that ERRORs if any part box moved > 5 mm. Measured on 88 refine-round- pairs: the part count never changed once and mean Δgeometry_detail was +0.003, so detail needed its own- round; the rounds that added geometry DURING repair lost 0.075 assembly_fit. $C3D_DETAIL_ROUNDS=0 is off.-stop reasons: pass | plateau | budget | max_rounds | no_change | no_refine_tasks | regression | diminishing_returns-finalise: restore best commit, rebuild so artifacts match delivered code, finalize_record → record.json+stop reasons: max_rounds | budget (the wall clock — BudgetGuard / max_minutes) | agent_quota | no_change (a refine+ round's sessions changed nothing) | no_refine_tasks | judge_unavailable | failed+ (a record from before 2026-09-22 may say passed / plateau: it loads as `stopped`)+finalise: the workspace ENDS AT THE LAST ROUND — src/ a cut round left dirty is put back and that round rebuilt;+ finalize_record → record.json. No best round, no deliverable/, no texture pass in the run.+after the run (`3dcode make`, unless --no-pick): addons/select.pick(by="score") — highest effective score, ties →+ fewer gate errors → the earlier round — then package(): deliverable/ + selection.json (+ --texture)```Track-specific gates: static `connectivity` + `contract` (+ `reference_silhouette`),articulated + `joint_sweep` + `motion_direction` (URDF axis vs plan motion text),@@ -484,7 +490,10 @@ a discarded load does not), graphics `gl_frames` (NaN/black/blown/static/flicker## 8. Texturing (derived asset pack)-`texturing.run.texture_pass(ws, spec, plan, model_id=…)` — after (or outside) a run:+`texturing.run.texture_pass(ws, spec, plan, model_id=…)` — never inside a run: `3dcode pick+--texture` (and `make --texture`, which picks after the run) texture the PICKED round's kept+`artifacts/rNN/object.glb`; a standalone `3dcode texture pass` textures the canonical one, i.e.+the last round:one VLM **material plan** (parts → shared texture ids, family, projection,`tile_size_m`; cached) → text-to-image tiles (`GeminiImageModel`,gemini-3.1-flash-image; mirror cross-fade makes them tileable, `seam_score ≤ 0.08`@@ -526,7 +535,7 @@ $0.02–0.03 (flash) / ~$0.2 (pro); api-agent generation 3–6 min per object ro* Scene judging is deliberately narrow: the judge sees the scene's OWN cameras (first,and the detail crop is taken from one of them) plus at most three overview-rig tiles;the harness's eye-level rig is a diagnostic only. A `must` acceptance item the harness- cannot verify caps a run at 0.60 AND fails it, so on the scene track only the spec's+ cannot verify caps a round at 0.60 AND fails its verdict, so on the scene track only the spec's`must_have` list keeps that priority (the plan's own checklist is `should`).* Articulated: candidate selection uses the quick 4-view sheet (not pose views);mimic joints are honoured (the sweep drives independent joints only and resolves@@ -536,8 +545,8 @@ $0.02–0.03 (flash) / ~$0.2 (pro); api-agent generation 3–6 min per object roderived pack); `userData.tick` cannot survive export.* agy exposes no per-workspace MCP, cost or served model.* Anthropic / OpenAI backends are mock-tested only (no keys on this box).-* Budget checks run between steps: a refine round that finishes its judge and then- trips the budget is not promoted to best — give scenes `--max-minutes 60`.+* Budget checks run between steps: a round the clock cuts mid-way is rolled back (its+ spend stays in `aborted_rounds`) — give scenes `--max-minutes 60`.* Gemini flash 503 storms happen; dead keys and 429s rotate freely now, but asustained outage can still fail a round (`3dcode resume` re-uses cached stages).* A few single-file wrappers were once over the old ~400-line guideline; the rule@@ -547,8 +556,9 @@ $0.02–0.03 (flash) / ~$0.2 (pro); api-agent generation 3–6 min per object ro`record.json` per run + git history; `3dcode flywheel export` writes sample folders(`<out>/<track>/<language>/<slug>/{code.<ext>, src/**, robot.urdf, meta.json,-captions.json, renders/}`) with **quality tiers** (A passed & 0 gate errors, B-passed, C best ≥ 0.6, D else), acceptance checklists, gate summaries and+captions.json, renders/}`) of the PICKED round (`addons/select`; the last built round+when none was judged) with **quality tiers** from that round's verdict (A passed & 0 gate+errors, B passed, C score ≥ 0.6, D else), acceptance checklists, gate summaries and`(code fingerprint, prompt)` dedupe (`--drop-duplicates`; side-car captions via`--captions-dir`); `metadata.parquet`/`.jsonl` + sqlite index carry tier, gateerrors, cost, fingerprints, `duplicate_of`; `--pack` tars with byte-range locators.diff --git a/harness/docs/COST.md b/harness/docs/COST.mdindex 69c89638..23dc8825 100644--- a/harness/docs/COST.md+++ b/harness/docs/COST.md@@ -45,6 +45,10 @@ superseded compare battery) is excluded.| recorded vs measured | `record.total_usage` says $85.08; the ledger finds **$86.30** (§6) || re-priced with the corrected table | **$89.10** (+3.2%, §7) |+(The pass count and the $ per passing artifact are this audit's. Since 2026-09-22 a run is+not passed or failed — it runs its fixed rounds and a pick hands one over — so `3dcode cost`+reports $ per run and no longer counts passes.)+**The money is generation, not judging.** Nine dollars in ten are the codingagent writing and rewriting code; the judge is a rounding error next to it(7.2%), the planner is noise (0.8%). Any cost programme that starts with "use a@@ -189,6 +193,15 @@ model — **88% of the clock is waiting for a model**, 12% is the harness| repair loop that never converged (run did not pass) | 4 | $1.31 | 1.5% || **total** | 59 | **$27.35** | **31.7%** |+> **2026-09-22 — three of these kinds are gone.** "Regression", "zero-delta round" and+> "unpromoted judge" measured money against the IN-RUN best round, and the owner removed+> the in-run best with the judgement stops: a run is now the baseline + `--rounds` refine+> rounds, each built on the round before it, every round is kept, and which one to hand+> over is picked after the run (`addons/select`). A round that scored lower is a candidate+> for that pick, not waste, so `3dcode cost` no longer reports those kinds (nor the+> `cost.round` `regression` / `zero_delta` flags). "Repair loop that never converged" is+> now repair money spent in a round that still did not build; post-budget stays.+Concrete examples:* `tool_med_hand_drill` r01: **$2.48** to go from 0.576 to **0.560**.@@ -349,7 +362,7 @@ Savings are estimated **on this data set** (61 runs, $86.30) unless stated.| # | change | est. saving | confidence | where ||---|---|---|---|---|-| 1 | **Stop refining after r01 unless the last delta ≥ 0.05.** r02+r03 cost $12.16 and bought +1.44 score points across 8 of 20 rounds. | **$9–12 (10–13%)** | high (measured) | `RoundPolicy` (min_delta 0.02 → 0.05 from r02, or plateau_window 1 after r01) |+| 1 | ~~**Stop refining after r01 unless the last delta ≥ 0.05.** r02+r03 cost $12.16 and bought +1.44 score points across 8 of 20 rounds.~~ **Removed 2026-09-22**: the round count is fixed (owner) — spend less with a lower `--rounds` or a profile, not with a judgement stop. | $9–12 (10–13%) | high (measured) | `--rounds` / `cost/profiles.py` || 2 | **Check the budget *before* starting a round, against the estimated round cost**, not only between steps. | **$8.76 (9.7%)** | high (measured waste) | `orchestrator.py` (`BudgetGuard`) + `tracks/steps.py`; use `codeverse3d.cost.estimate_call` / median round cost || 3 | ~~**Cap agent turns at ~25 and compact old tool results.** 39.3% of the agent bill is turn ≥20; 19 of the 33 sessions that ran ≥50 turns were cut off by their own budget.~~ **Tested, rejected: +$0.02 and −0.21 score** (A/B, n=3 per arm, §17) — the cap is off by default. | est. $8–15, **measured $0** | high (A/B) | `DEFAULT_AGENT_MAX_TURNS=0`; `$C3D_AGENT_MAX_TURNS` / `Settings.limits.agent_max_turns` still set one || 4 | **Fold the off-record spend into the budget** (cut rounds, post-hoc texture passes). | $0 saved, **$1.22 of blindness removed** | high | §6; the ledger (`codeverse3d.cost.record_call`) makes it automatic |@@ -698,7 +711,8 @@ tool is registered for every object track, so the coding agent could buy a passany run. That is why the quality run below shows a ledger 9.5 % above its record:the pass ran twice, once from inside a round-2 agent session.`codeverse3d.texturing.run.texture_requested(spec)` is now the single owner;-`tracks.lifecycle.finalise` and the tool both ask it, and the tool refuses with a+the CLI hand-over after the run (`make`/`resume` → `addons.select.package`, which textures the+PICKED round since 2026-09-22 — finalise no longer does) and the tool both ask it, and the tool refuses with ausage error (costing $0) in a run whose spec says no. `3dcode texture pass <slug>`is an explicit user instruction and is unaffected.@@ -805,7 +819,7 @@ out of tree (no such test ships here):|---|---|---|| `no file change` | **removed** | unreachable: a generation result is `ok` only when a file changed, so `run_generation_tasks` raises `RoundFailed` (→ the loop's `no_change` plateau) before any judge question is asked || `build not repaired within the repair budget` | **removed** | a broken build never reaches the judge at all (`run_round` judges only when `build.ok`), so it only ever fired on a round that BUILT but still lint-failed — and the loop's `rejudge_round` then bought the same verdict one iteration later. Measured on the reproduction: 3 rounds "skipped", 2 verdicts re-bought as `judge.retry`, and the last round left **without a score — it was the best of the three (0.7)** and could not be promoted |-| `budget already exceeded` | **kept** | the loop's next `budget_ok` check ends the run, so the verdict is never bought later. Guard, not a measured saving: 0 of the 108 recorded rounds bought a verdict after the budget ended, so this branch has saved $0 so far — it is kept because the state is knowable in advance and the run is over either way (§5) |+| `budget already exceeded` | kept, then **removed 2026-09-22** | the loop's next `budget_ok` check ends the run, so the verdict was never bought later. Guard, not a measured saving: 0 of the 108 recorded rounds bought a verdict after the budget ended. Removed with the in-run best: a round's verdict is now what lets a pick choose it after the run, and the generation it scores is already paid for || `no judge` / `no renders` | **kept** | pre-existing guards (nothing to buy). The third pre-existing reason, `gate errors` behind `judge_on_gate_errors=False`, went 2026-09-21: no caller ever set it |Two changes were needed to make the kept branches real:@@ -813,16 +827,16 @@ Two changes were needed to make the kept branches real:1. `rejudge_round` re-buys a verdict only after the judge was *tried* and failedor came back degraded — never one that policy deliberately skipped. Otherwiseevery skip is a `judge.retry` with the same price tag.-2. `pick_best_round` never promotes a round that did not build, and when no round has- a score at all it falls back to the built round with the **fewest gate errors**- (later on a tie) instead of simply the last one — an unjudged round that broke- the gates can no longer displace the clean artifact that came before it.--The two controls that DID survive are unchanged and re-verified after this-correction: the regression "switch then stop" rule and the r03+ marginal stop-still cut 2 runs / **$1.14** with **0 best rounds lost** over the 107 recorded-rounds (`waste-and-accounting/replay_stops.py`), and every round still emits its-`cost.round` event.+2. `pick_best_round` never promoted a round that did not build. (Removed 2026-09-22 with+ the in-run best; `addons/select.pick` picks among JUDGED rounds only, so an unbuilt or+ unjudged round is never handed over by score.)++The two controls that DID survive this correction — the regression "switch then stop"+rule and the r03+ marginal stop, which cut 2 runs / **$1.14** with **0 best rounds lost**+over the 107 recorded rounds (`waste-and-accounting/replay_stops.py`) — were **removed on+2026-09-22** with every other judgement stop: the owner fixed the round count (baseline ++`--rounds`, cut short only by the clock or a hard failure) and moved the choice of round to+after the run. Every round still emits its `cost.round` event.---diff --git a/harness/docs/INTERFACES.md b/harness/docs/INTERFACES.mdindex 200f3bee..ccc8bebf 100644--- a/harness/docs/INTERFACES.md+++ b/harness/docs/INTERFACES.md@@ -389,24 +389,14 @@ rec = get_track(spec.track, **options).run(spec, ws, resume=False) -> RunRecord# services=, judge=, agent=, model=, runtime=, policy=RoundPolicy, settings=, planner_model=, n_candidates=# (CLI --candidates > spec.options.candidates > settings.default_candidates); StaticObject | Articulated | Scene | GraphicsBaseTrack.run(spec, ws, *, resume=False, force=False) -> RunRecord # get_track returns a tracks.lifecycle.BaseTrack-from codeverse3d.orchestrator import RoundPolicy, StopPolicy, StopDecision, pick_best_round, judge_sigma, \- best_score, last_gain, REWRITE_KIND, build_refine_instructions, compact_instructions-RoundPolicy(max_rounds=4, plateau_window=2, min_delta=0.02, target=0.8, max_refine_tasks=6,- max_instructions_per_task=6, parallel_min_tasks=2, n_candidates=1, pairwise_margin=0.03,- pairwise_min_confidence=0.6, judge_samples=1,- judge_model="", regression_sigma=1.0, # money stops (docs/COST.md §5)- marginal_sigma=1.5, marginal_from_round=3, # r03+ must beat 1.5σ- detail_rounds=None, detail_min_score=0.45, detail_bbox_tol_m=0.005 # surface-detail round, tri-state:- # None = the track default (lifecycle.DEFAULT_DETAIL_ROUNDS=1 when supports_detail_round, else 0),- # 0 = off, N = N; $C3D_DETAIL_ROUNDS overrides all (the A/B switch). Injected policies keep it.- ) # lifecycle.build_context binds n_candidates + judge_model with ONE dataclasses.replace;- # RoundPolicy.target is ALWAYS overridden by the rubric pass threshold in BaseTrack.after_plan-policy.sigma / .regression_delta / .marginal_delta # judge_sigma() reads cost.routing.JUDGE_NOISE — THE σ table-StopPolicy(policy).evaluate(history, budget_ok=True) -> StopDecision(reason, strategy="same"|"switch"|"detail", detail)- # (.reason is the StopReason); strategy "switch" = ONE whole-artifact rewrite round (kind REWRITE_KIND),- # "detail" = ONE surface-detail round (kind DETAIL_KIND) — a plateau/diminishing stop is converted into it- # only when detail_blocked(history, policy) == "" (clean gates, built, judged, within σ of best, budget left)-from codeverse3d.orchestrator import DETAIL_KIND, KIND_FOR_STRATEGY, detail_blocked # THE strategy → kind dict+from codeverse3d.orchestrator import RoundPolicy, build_refine_instructions, compact_instructions, gate_error_count+RoundPolicy(max_rounds=4, max_refine_tasks=6, max_instructions_per_task=6, parallel_min_tasks=2,+ n_candidates=1, judge_samples=1) # lifecycle.build_context binds n_candidates with ONE dataclasses.replace+ # Δ 2026-09-22 FIXED rounds: a run is the baseline + max_rounds refine rounds, EACH built on the round before+ # it; only the clock (BudgetGuard) or a hard failure ends it early. Gone with the judgement stops: StopPolicy /+ # StopDecision, pick_best_round, judge_sigma, best_score, last_gain, REWRITE_KIND / DETAIL_KIND /+ # KIND_FOR_STRATEGY / detail_blocked, RunState.best_* / update_best, and the RoundPolicy fields plateau_window,+ # min_delta, target, pairwise_*, judge_model, regression_sigma, marginal_*, detail_*.from codeverse3d.orchestrator import BudgetGuard, usage_deltaBudgetGuard(budget, start_time=None, *, soft_fraction=1.0) # no run= / ledger=.charge(usage, *, stage="other", enforce=True) # (Δ 2026-08-30: role/label/round_index/outcome gone)@@ -418,27 +408,25 @@ BudgetGuard(budget, start_time=None, *, soft_fraction=1.0) # no run=# add(...) = charge(enforce=False) — "not enforced" never means "not seen"..by_stage / .stage_summary() / .mark() # what a round burned, live (by_round[i] deleted — no reader;# per-round money is CallCost.round in telemetry/cost.jsonl)-from codeverse3d.tracks.candidates import CandidateRecord, rank_candidates, decide_best # pure decision logic-from codeverse3d.tracks.candidates import run_best_of_n, choose_best_round, quick_render # N parallel baselines in <ws>/_cand/c<k>+from codeverse3d.tracks.candidates import CandidateRecord, rank_candidates # the one in-loop choice left+from codeverse3d.tracks.candidates import run_best_of_n, quick_render # N parallel baselines in <ws>/_cand/c<k># each candidate IS steps._run_round(kind="candidate") in its sub-workspace: render=quick_render(ctx, round_index, build,# measurement, *, pipeline) (4 views + the articulated pose views), geometry_views=False, its own events.jsonl and a# one-sample judge → _cand/c<k>/judge/r00.json (a degraded verdict stays score None); crashed candidate retried once;-# selection by build_ok → quick score → fewer gate errors, pairwise within margin (booked stage=Stage.PAIRWISE);+# selection by build_ok → quick score → fewer gate errors → the earlier candidate (no pairwise since 2026-09-22);# winner copied back, normal r00 pipeline follows; rounds/candidates.json IS record.extra["candidates"] (n, selected,-# candidates[], pairwise)+# candidates[])from codeverse3d.tracks.articulated_object import default_motion_checks, expected_direction # gate "motion_direction"from codeverse3d.tracks.static_object import silhouette_gate, reference_refine_tasks # gate "reference_silhouette" (IoU<0.6 → WARN + refine task)from codeverse3d.tracks.depth import depth_budget, DepthBudget, scope_groups, PartScope, interfaces_text, \scoped_generation_enabled # complexity-aware budgets + per-part scoped generationdepth_budget(plan, *, build_timeout_s=300) -> DepthBudget # min/target/max triangles + max_build_s sized from# the plan's LEAF count (parts × instances × children); .as_prompt() is the DETAIL BUDGET block every- # generate/refine/detail template shows in place of a flat "≤ 300k tris"+ # generate/refine template shows in place of a flat "≤ 300k tris"scope_groups(plan, *, files_for, max_groups=6, parts_per_scope=3, min_parts=8) -> [PartScope]# [] = one session owns the object (small plan, no per-part file ownership, or $C3D_SCOPED_PARTS=off);# otherwise attachment-subtree groups whose files are disjoint, so the sessions run in parallelinterfaces_text(plan, scope) -> str # the planned boxes of the neighbours this scope must weld to-from codeverse3d.tracks.static_object import drift_gate, detail_instructions, DRIFT_GATE # gate "detail_drift":- # ERROR when a detail round moved/resized/removed a part or changed the overall extents (tol from policy)from codeverse3d.tracks.prompting import base_prompt_context, reference_images, file_for_target_factory, \scope_context, budget_for, detail_budget_text # Δ split out offrom codeverse3d.tracks.prompting import select_cookbook_chapters, is_always_chapter@@ -509,7 +497,8 @@ generation prompts.```pythonfrom codeverse3d.texturing.run import texture_pass, texture_requested, load_reporttexture_requested(spec) -> bool # THE owner of "does this run texture?" (Spec.options.texture, or- # the legacy "texture" tag). Asked by tracks.lifecycle.finalise AND by the texture_pass spatial+ # the legacy "texture" tag). Asked by the CLI hand-over after the run (make/resume → addons.select.package(+ # texture=…), which textures the PICKED round — Δ 2026-09-22, finalise no longer does) AND by the texture_pass spatial# tool, which refuses in a run that did not ask — the tool is registered for every object track,# so `texture: false` used to be bypassable from inside an agent session.texture_pass(ws, spec, plan, *, model_id, image_model=None, judge=True, judge_model_id=None, rubric=None,@@ -526,15 +515,38 @@ from codeverse3d.texturing.plan import scene_texture_pack, texture_pack_prompt## record/ + addons/ + cli/```python-from codeverse3d.record.record import finalize_record, load_record, iter_runs, best_round_index, best_round_record-from codeverse3d.addons.gallery.index import hero_view # (ws, rec) -> (rel, label, n_views): the card image, rebased via ws-from codeverse3d.record.record import complexity_block, round_complexity # objective complexity of what shipped- # finalize_record fills record.extra["complexity"] = the BEST round's vector + plan_parts /- # parts_per_plan_part / by_round; every rounds_summary row gains "complexity" (the index or None)+from codeverse3d.record.record import finalize_record, load_record, iter_runs, effective_judgment, effective_score+ # Δ 2026-09-22: best_round_index / best_round_record are gone, and fill_derived no longer derives a best round,+ # baseline or final score — which round counts is addons.select's question+from codeverse3d.record.deliverable import keep_round_artifacts, round_outputs, build_deliverable, load_deliverable+ # keep_round_artifacts(ws, round_index) -> [names]: steps._run_round copies a BUILT round's hand-over files+ # (object.glb object.stl object.step robot.urdf preview.gif frames_sheet.png meshes/*.glb) to artifacts/rNN/+ # round_outputs(ws, rnd) -> Path | None: artifacts/rNN/, else artifacts/ for the best round a pre-2026-09-22+ # record.json names, else None+ # build_deliverable(ws, record, round_index, *, clean=True) -> RunDeliverable: deliverable/ for ONE round —+ # its commit's code + its kept files + its sheet (+ graphics: its judged frames; + a shipped texture pack+ # whose report names this round's GLB); never rebuilds. Only addons.select.package calls it.+from codeverse3d.addons import select # which round of a FINISHED run to hand over (cli/ + addons only)+select.round_rows(run_dir, *, record=None) -> [RoundRow{index, kind, score (effective), passed (the judge's own+ # verdict for THAT round), gate_errors, build_ok, commit, cost_usd, minutes}]+select.pick(run_dir, *, by="score"|"pairwise", pairwise_model=None, record=None, judge=None) -> int | None+ # highest effective score → fewer gate errors → the earlier round; None when no round was judged. "pairwise":+ # the top two within PAIRWISE_MARGIN=0.03 go to judges.pairwise.PairwiseJudge (position-swapped); the runner-up+ # wins at confidence ≥ 0.6; one paid verdict per (pair, model), cached at artifacts/judge/rAA_vs_rBB_pairwise.json+select.summarise(run_dir, *, record=None) -> RunSummary{rounds, stop_reason, baseline_score (r00's effective+ # score), picked_round, picked_score, delta, method}: selection.json's round when the run was packaged, else pick()+select.package(run_dir, round_index, *, texture=False, method="round", image_model=None) -> Path (deliverable/)+ # + selection.json {round, method, scores, textured, selected_at}; texture=True runs the pass on that round's+ # kept GLB unless a report already covers those bytes; a failed pass never fails the hand-over+from codeverse3d.addons.gallery.index import hero_view # (ws, rec, picked) -> (rel, label, n_views): the card image+from codeverse3d.record.record import complexity_block, round_complexity # objective complexity of what was built+ # finalize_record fills record.extra["complexity"] = the LAST measured round's vector + plan_parts /+ # parts_per_plan_part / by_round; every rounds_summary row gains "complexity" (the index or None); readers+ # that show one round (gallery, dataset) read that round's own round_complexityfrom codeverse3d.addons.dataset.export import export_samples # (runs_dir, out_dir, *, min_score=None, only_passed=False,# include_unbuilt=False, captions_dir=None, drop_duplicates=False) -> ExportReport{…, n_duplicates, duplicates, tiers}-from codeverse3d.addons.dataset.quality import quality_tier, prompt_hash, find_duplicates # tiers: A passed & 0 gate errors, B passed,- # C best ≥ 0.6, D else; dedupe = (code fingerprint, prompt)+from codeverse3d.addons.dataset.quality import quality_tier, prompt_hash, find_duplicates # tiers (of the exported round's verdict):+ # A passed & 0 gate errors, B passed, C score ≥ 0.6, D else; dedupe = (code fingerprint, prompt)from codeverse3d.addons.dataset.pairs import build_pairs # (runs_dir, out_jsonl, *, min_delta=0.05) -> nfrom codeverse3d.addons.dataset.refine import build_refine, transitions, RefineTransition, REFINE_KINDS, outcome_of# build_refine(runs_dir, out_jsonl, **kw) -> (rows written, Counter of drop reasons); writes via a .part file@@ -573,7 +585,7 @@ from codeverse3d.addons.gallery.urls import safe_join # (root, rel) ->from codeverse3d.addons.gallery.urls import content_type # .glb→model/gltf-binary, .py/.js/.frag→text/plain; charset=utf-8from codeverse3d.addons.dataset.index import build_index, query, summary # sqlite + parquet: adds quality_tier, gate_errors, cost_usd,# rounds, status, code_fingerprint, prompt_hash, duplicate_of, has_captions-3dcodeverse make [--profile economy|balanced|quality]|resume|status|show|render|judge|tools|mcp+3dcodeverse make [--profile economy|balanced|quality] [--no-pick]|resume [--no-pick]|pick|status|show|render|judge|tools|mcp|texture {pass,scene-pack,show}|cost {<slug>,show,cache,prices,profiles,estimate}|flywheel {export,pairs,refine,caption,index}|gallery {serve,build}|bench {run,report}|doctor # alias: 3dcode@@ -583,24 +595,27 @@ from codeverse3d.addons.dataset.index import build_index, query, summary # sql`EventLog.emit(event, **data)` writes `{"t", "event", ...}` (**Δ** key is `event`).Event names: `run.start`, `stage.start/done`, `plan.done`, `workspace.materialized`,`round.start`, `generate.done`, `build.done`, `gates.done`, `judge.done`,-`round.done`, `best.updated`, `refine.planned`, `recipes.seeded` (graphics: names written this call, present on disk, chapters), `asset.judged`, `assets.done`,+`round.done`, `refine.planned`, `recipes.seeded` (graphics: names written this call, present on disk, chapters), `asset.judged`, `assets.done`,`zones.done`, `assemble.done`, `round.no_change`, `candidates.start`,-`candidate.start/done/failed/retry/selected`, `pairwise.done`,+`candidate.start/done/failed/retry/selected`, `resume.reconciled` (…, `restored_last_round`),`texture.start/plan/generated/applied/gate/done`, `budget.exceeded`,-`finalise.rebuild`, `stop`, `run.done` / `run.failed`.+`finalise.rebuild`, `stop` (reason, rounds), `run.done` (status, rounds, last_score) / `run.failed`;+after the run: `pick.pairwise`, `pick.packaged`, `texture.skipped` / `texture.failed` (addons.select).Cost events: **`cost.round`** (per round: `stages{stage → $}`, `judge_usd`, `total_usd`,-`agent_turns` (= AgentResult.turns), `score`, `previous_best`, `wasted`, `waste_reason` ∈ aborted | build_failed |-unjudged | regression | zero_delta, `run_usd`) — emitted for aborted rounds too;-`judge.skipped` (reason), `generate.turn_cap` (label, max_turns, turns, cost_usd),-`strategy.switch` (regression → whole-artifact rewrite), `budget.overrun` (a post-loop-texture pass crossed the ceiling; the loop is already finished, so not `budget.exceeded`).-`RoundRecord` gained `pairwise: PairwiseNote | None` (Δ 2026-08-30) — the paid tie-break-verdict, stored so `reconcile_resume` REPLAYS it (`candidates.replay_best_round`) instead-of re-ranking on score and reversing a comparison the run bought. `PairwiseNote` moved-from `codeverse3d.tracks.candidates` to `codeverse3d.contracts.run` (it is pure data).+`agent_turns` (= AgentResult.turns), `score`, `wasted`, `waste_reason` ∈ aborted | build_failed |+unjudged, `run_usd`) — emitted for aborted rounds too (Δ 2026-09-22: `previous_best` and the+regression / zero_delta reasons went with the in-run best: a lower-scored round is kept, not wasted);+`judge.skipped` (reason), `generate.turn_cap` (label, max_turns, turns, cost_usd).+Δ 2026-09-22: `best.updated`, `strategy.switch`, `round.refine_from_best`, `pairwise.done` and+`budget.overrun` are gone, and so is `RoundRecord.pairwise` (the in-run tie-break note; an old+rNN.json that carries one still loads, the key ignored). `PairwiseNote` lives in `addons.select`.`RunRecord` (record.json): spec, plan, workspace, status, rounds[RoundRecord],-best_round, baseline_score, final_score, total_usage, environment,+total_usage, environment,prompt_hashes{contract, cookbook, generate, refine}, error, extra{stop_reason,rubric, budget, cost_by_stage, aborted_rounds?, rounds_summary, candidates? (therounds/candidates.json payload), texturing?, captions?}. `total_usage` is the BudgetGuard total whenever it-exceeds the sum of the rounds (aborted rounds, retried sessions, texture pass).+exceeds the sum of the rounds (aborted rounds, retried sessions); a texture pass bought by a pick is in+telemetry/cost.jsonl, not in `total_usage`. Δ 2026-09-22: no `best_round` / `baseline_score` /+`final_score` (old records carry them: ignored — `addons.select.summarise` answers), and `status` is only+the stop reason: max_rounds | budget | agent_quota | no_change | no_refine_tasks | judge_unavailable |+failed (an old `passed` / `plateau` loads as `stopped`).diff --git a/harness/docs/RUNBOOK.md b/harness/docs/RUNBOOK.mdindex abe36350..40fb30f2 100644--- a/harness/docs/RUNBOOK.md+++ b/harness/docs/RUNBOOK.md@@ -56,11 +56,14 @@ python -m pytest tests -q -m live # OPT-IN: real API calls```Useful flags (`3dcode make --help`): `--generator`, `--planner`, `--judge`, `--captioner`(backend ids, §3), `--image <png>` (repeatable; reference images → `ReferenceJudge` +-silhouette gate + IoU refine tasks), `--rounds N` (refine rounds after the baseline),+silhouette gate + IoU refine tasks), `--rounds N` (refine rounds after the baseline — a+run does exactly N, each built on the one before it, unless the clock or a hard failure+stops it; there is no pass / plateau / regression stop since 2026-09-22),`--candidates N` (best-of-N baseline: N parallel candidates in `<ws>/_cand/`,-quick-judged, pairwise tie-break, winner kept; multiplies baseline cost ≈ N;-default from `settings.default_candidates`), `--texture` (run the texture pass after-finalise; see §6), `--max-minutes`, `--dim height=0.45`, `--must`,+quick-judged, the highest quick score kept (fewer gate errors on a tie); multiplies baseline+cost ≈ N; default from `settings.default_candidates`), `--texture` (texture the PICKED round+after the run; see §6), `--no-pick` (package nothing: no `deliverable/`, no `selection.json` —+`3dcode pick` later), `--max-minutes`, `--dim height=0.45`, `--must`,`--must-not`, `--style`, `--tag`, `--seed`, `--slug`, `--runs-dir`, `--force`,`--no-run` (workspace + spec.json only — except that `--reference` still runs itspaid grounding pass first, since the grounded spec is what it writes).@@ -108,9 +111,12 @@ artifacts in `artifacts/` (`object.glb`, `robot.urdf` + `meshes/`, scene`metrics.json`, texturing `object_textured.glb` + `textures/`), per-round renders in`artifacts/renders/rNN/`, gate JSON in `artifacts/gates/rNN/`, verdicts in`artifacts/judge/rNN.json`, transcripts in `trajectories/<label>_rNN/` (retries in-`<label>.a2_rNN`), events in `events.jsonl`, the flywheel record in `record.json`.-`3dcode status <slug>` prints the rounds table (best round starred), cost and the last-events.+`<label>.a2_rNN`), events in `events.jsonl`, the flywheel record in `record.json`. The tree+and `artifacts/` end at the LAST round; every round's own built files stay in+`artifacts/rNN/`, and after the run `deliverable/` + `selection.json` hold the round a pick+handed over (RUN_LAYOUT.md). `3dcode status <slug>` prints the stop reason, baseline → picked+score, the rounds table (the picked round starred, each round's own judge verdict), cost and+the last events.### Looking at results locally@@ -123,12 +129,13 @@ events.**directories** too, so every link works: contact sheet, full-size renders, `record.json`,`plan.json`/`spec.json`, the `src/` tree (browsable, line-numbered, `raw` = text/plain),`object.glb` (orbit viewer on the vendored three.js — no network), articulation sheets,-`preview.gif`, frames, textures. The page has a live summary strip (n, pass rate,-mean/median score, total $, $ per passing artifact, wall clock — **recomputed per filter**),+`preview.gif`, frames, textures. The page has a live summary strip (n = judged · unjudged ·+error, mean/median score, total $, wall clock — **recomputed per filter**; no pass rate: a run+is not passed or failed),filters (track / language / tier / backend / verdict / battery + text search overprompt+slug) and sort (score / cost / time / name) that never reload, a card ⇄ tabletoggle, light/dark, and a detail page per run (`/run/<battery>/<slug>`) with every round,-the best round's judge verdict + issues + improvement plan, the measurement table, the+the picked round's judge verdict + issues + improvement plan, the measurement table, thefull render set, the cost breakdown and the code.Filters live in the query string, so a view can be curled or bookmarked:@@ -142,9 +149,20 @@ unless you type `--host` yourself; it never serves a path outside the declared r## 5. Resume, re-render, re-judge, texture, export```bash-3dcode resume <slug> [--candidates N] # continues from run_state + stages/*.json (input-hash cached). The budget+3dcode resume <slug> [--candidates N] [--no-pick]+ # continues from run_state + stages/*.json (input-hash cached) and from the+ # LAST round (a run recorded before 2026-09-22 that restored its best round+ # is put back on its last). A max_rounds run resumes only with a raised+ # --rounds, a budget run with any raised cap; an agent_quota or failed run+ # resumes as is; the other stops are finished. The budget# SNAPSHOT is restored: money/calls/active-minutes already spent still count,# so a raised --max-minutes grants only the difference (downtime never counts)+3dcode pick <slug> [--by score|pairwise] [--round N] [--texture] [--judge MODEL]+ # hand over a round: deliverable/ + selection.json (addons/select). score =+ # highest effective score, ties → fewer gate errors → the earlier round;+ # pairwise = the pairwise judge between the top two when within 0.03 (one+ # paid verdict per pair, cached); --round N = that round; --texture = the+ # texture pass on it first. `make`/`resume` run `pick --by score` for you3dcode render <slug> [--round N] [--mode shaded|wire|normals|clay|silhouette] [--out dir]3dcode judge <slug> [--round N] [--rubric static_object_v1] [--model gemini:gemini-3.1-pro-preview] [--n 3]# re-judges a round's recorded renders → artifacts/judge/rNN_cli.json@@ -216,9 +234,10 @@ scoreless cells afterwards — delete the row from `results.jsonl` and the run dthen `bench run --id <prompt>`.A refine session that dies on the VENDOR's usage limit ("You've hit your usage limit … try again at-Sep 14th", `RESOURCE_EXHAUSTED`, `insufficient_quota`) stops the run at `agent_quota` with status-`budget`: the best round ships, and the record says the agent's budget ended, not the code's-improvement (cmp8, 2026-09-09, filed three such runs as `plateau` before this).+Sep 14th", `RESOURCE_EXHAUSTED`, `insufficient_quota`) stops the run at `agent_quota`: every round+so far is kept and picked from as usual, the record says the agent's budget ended, not the code's+improvement (cmp8, 2026-09-09, filed three such runs as `plateau` before this), and `3dcode resume`+continues it once the limit resets.**A cell can also lose its renders to the box rather than to itself.** When the machineruns out of memory Chrome reaps the render tab and the driver reports@@ -244,7 +263,7 @@ asserts that the specific fixes the arm exists to measure are present.**A new worktree needs `runtime_js/node_modules` before it can run a battery.** Without it`render_glb` dies on every round, the judge is skipped for want of renders, and the cells-come back `status=plateau` with `score=None` — an arm that reads as healthy and measures+came back `status=plateau` (today: `judge_unavailable`) with `score=None` — an arm that reads as healthy and measuresnothing (2026-09-04, the mimic-off arm). `npm ci` in `runtime_js/`, or COPY thedirectory from a worktree that has it (the `package.json` is the same file). The row now (a symlink gave two 404s on every render of one worktree on 2026-09-07 — `serve.cjs` refuses a real path outside its root — while another probe served through one; copy and be sure)says `no verdict in any of N round(s)` when this happens.@@ -255,7 +274,10 @@ stages are reused — this also recovers from Gemini 503 storms). Ctrl-C is saf## 6. Texture pass-`3dcode texture pass <slug>` (or `--texture` on `make`): one VLM material plan →+`3dcode pick <slug> --texture` (or `--texture` on `make`, which picks after the run) textures the+PICKED round's own `artifacts/rNN/object.glb` — once per round: a pass that already started from+those bytes is re-used, never re-bought; `3dcode texture pass <slug>` textures the canonical+`artifacts/object.glb`, i.e. the LAST round. Either way: one VLM material plan →tileable texture images (gemini-3.1-flash-image, ~$0.07/tile, cached by prompt) →world-metre UVs → `artifacts/object_textured.glb` → seam gate + before/after judgegate (ships only when the score does not drop and the materials criterion improves).diff --git a/harness/docs/RUN_LAYOUT.md b/harness/docs/RUN_LAYOUT.mdindex 92061b15..0fa3080e 100644--- a/harness/docs/RUN_LAYOUT.md+++ b/harness/docs/RUN_LAYOUT.md@@ -6,8 +6,8 @@ three separate buckets:| bucket | question | contents ||---|---|---|-| `deliverable/` | **what did I get?** | the code snapshot of the best round, the canonical artifact (`object.glb` / `robot.urdf` + `meshes/` / `frames/` + `preview.gif` / the scene bundle), the best contact sheet, `captions.json`, `manifest.json` |-| `evidence/` (= `artifacts/`) | **why should I believe it?** | `renders/rNN/`, `gates/rNN/`, `judge/rNN.json`, `measurement.json`, `articulation.json`, `census.json`, `textures/`, `tool_renders/` |+| `deliverable/` | **what did I get?** | ONE round, the one a pick chose (`3dcode pick`; `3dcode make` picks by score after the run): its code snapshot, its artifact (`object.glb` / `robot.urdf` + `meshes/` / `frames/` + `preview.gif` / the scene bundle), its contact sheet, `captions.json`, `manifest.json` |+| `evidence/` (= `artifacts/`) | **why should I believe it?** | every round's own build (`rNN/`), `renders/rNN/`, `gates/rNN/`, `judge/rNN.json`, `measurement.json`, `articulation.json`, `census.json`, `textures/`, `tool_renders/` || `telemetry/` | **what did it cost and how was it configured?** | `settings.json`, `cost.json`, `usage.jsonl`, plus `events.jsonl`, `run_state.json`, `stages/`, `trajectories/` |The run's **identity** stays at the root, where every tool has always looked for it:@@ -15,12 +15,13 @@ The run's **identity** stays at the root, where every tool has always looked for```runs/<slug>/spec.json plan.json record.json identity: what was asked, planned, produced+ selection.json which round deliverable/ holds and how it was chosen (after a pick)run_state.json events.jsonl (physical home; also linked from telemetry/)- src/ live working tree (git; every round is a commit)+ src/ live working tree (git; every round is a commit; ends at the LAST round)public/ (scene) compiled assets- deliverable/ (a) the hand-over+ deliverable/ (a) the hand-over — one round, written by a pickevidence/ -> artifacts/ (b) the proof- artifacts/ renders/ gates/ judge/ measurement.json …+ artifacts/ the last round's build, rNN/ (each round's own), renders/ gates/ judge/ measurement.json …telemetry/ (c) the accountingsettings.json cost.json usage.jsonlevents.jsonl -> ../events.jsonl@@ -49,10 +50,11 @@ symlink. Which side is physical was not a matter of taste:(tmp + rename) — a rename onto a *symlink* replaces the link, so the symlinkmust be on the alias side, never on the side that gets written.* **`deliverable/`, `telemetry/settings.json` and `telemetry/cost.json` are real- files.** They are *derived*: rebuilt from the run's own git history,- artifacts, trajectories and events every time the run is finalised, and on- the fly by `3dcode show` / the exporters for a run that predates them — old- runs are read as-is, never rewritten. `deliverable/` is self-contained so it can be zipped and handed to+ files.** They are *derived*: `telemetry/` is rebuilt from the run's own+ artifacts, trajectories and events every time the run is finalised,+ `deliverable/` from a round's commit and its `artifacts/rNN/` every time a round+ is picked, and both on the fly by `3dcode show` / the exporters for a run that+ predates them — old runs are read as-is, never rewritten. `deliverable/` is self-contained so it can be zipped and handed tosomeone (the flywheel exporter only falls back to it when git cannot answer).`telemetry/usage.jsonl` is the one file that can be either: a realreconstructed ledger, or a symlink to the run's live `telemetry/cost.jsonl`@@ -64,32 +66,58 @@ symlink. Which side is physical was not a matter of taste:(as well as `artifacts/`, `stages/`, `trajectories/`, …) so the derived bucketsnever enter the code snapshot.+## Every round, kept (`artifacts/rNN/`)++The run ends at its LAST round (`src/` and the canonical `artifacts/` files), and since+2026-09-22 no round is chosen during the run. So every round keeps what a hand-over of+it needs, the moment it builds (`record.deliverable.keep_round_artifacts`, called by+`tracks/steps._run_round` after the round's commit):++```+artifacts/rNN/ real copies, never hard links (the next build replaces the canonical files)+ object.glb object.stl object.step (object tracks; whichever the language exports)+ robot.urdf meshes/<link>.glb (articulated)+ frames_sheet.png preview.gif (graphics)+```++The rest of a round is already per round: its code is its git commit (a scene's hand-over+— `src/` + `public/`, GLB assets included — is nothing else), its renders and contact sheet+are `renders/rNN/` (a graphics round's judged frames among them), its gates `gates/rNN/`,+its verdict `judge/rNN.json` and `rounds/rNN.json`. The census, `build.json`, the `.blend`+and the measurement are evidence of the LAST build only. A round that did not build keeps+nothing. A run recorded before 2026-09-22 has no `rNN/`: its canonical files are the build+of the best round its `record.json` still names (`round_outputs` reads it that way).+## `deliverable/`-Built by `codeverse3d.record.deliverable.build_deliverable(ws, record)` from the-**best round** (`record.best_round`, else the highest judged score):+Built for ONE round by `codeverse3d.addons.select.package(run_dir, round)` →+`record.deliverable.build_deliverable(ws, record, round)` — from that round's commit and its+`artifacts/rNN/`, never a rebuild. `3dcode make` / `resume` package `select.pick(by="score")`+after the run (highest effective judged score, ties → fewer gate errors → the earlier round;+`--no-pick` skips it); `3dcode pick <slug> [--by pairwise] [--round N] [--texture]` re-packages+any round:```deliverable/- src/** raw code at the best round's commit (public/** too, for scenes)+ src/** raw code at the round's commit (public/** too, for scenes)object.glb object.stl object.step object_textured.glbrobot.urdf meshes/<link>.glb (articulated)- frames/fNN_tT.png frames_sheet.png preview.gif (graphics)- textures/<id>.png (when the texture pass shipped)- sheet.png the best round's contact sheet+ frames/*.png frames_sheet.png preview.gif (graphics: the round's judged frames)+ textures/<id>.png (when a texture pass of THIS round shipped)+ sheet.png the round's contact sheetcaptions.json {detailed, instruction, factory} when captionedmanifest.json every file with role, size, sha256```-Every file is hashed with sha256. Small files (all code) are real copies, so-editing the hand-over folder can never touch the evidence; binaries of 256 KB and-up (frame stacks, GLBs, GIFs) are **hard-linked** to their evidence copy when the-filesystem allows — same bytes, one block on disk, and `cp` / `tar` / `rsync`-still produce a standalone copy (the six graphics bench runs grow from 140 MB to-144 MB, not to 210 MB). A file over 128 MB (or a bucket over 512 MB) is left-out and named in `manifest.skipped`. Rebuilding is idempotent: the manifest-keeps its previous `generated_at` when the content is unchanged, so re-running-the packager produces no diff.+Beside `record.json`, `selection.json` says which round that is and why: `{round, method+(score | pairwise | round), scores (every round's effective score), textured, selected_at}`.+`addons.select.summarise` reads it, so every reader reports the round that was handed over.++Every file is hashed with sha256 and is a real copy (never a hard link), so editing the+hand-over folder can never touch the evidence. A file over 128 MB (or a bucket over+512 MB) is left out and named in `manifest.skipped`. Rebuilding is idempotent: the manifest+keeps its previous `generated_at` when the content is unchanged, so re-running the packager+produces no diff.## `telemetry/`@@ -120,19 +148,22 @@ the packager produces no diff.zones / baseline / refine / repair / judge / pairwise / texture / …`), perrole, per model, per round, plus `ledger_usd`, `unattributed_usd` (theledger's residual row) and `post_run_usd` (priced calls outside the run total,- e.g. a texture pass that ran after the loop). `record.total_usage` stays the- authority on what the run cost.+ e.g. the texture pass or pairwise verdict a pick bought after the run).+ `record.total_usage` stays the authority on what the run cost.## `record.json`-Two additive blocks, both `None` on older records:+One additive block, `None` on older records:-* `record.telemetry` — `RunTelemetry(settings, cost, environment, files)`;-* `record.deliverable` — `RunDeliverable(best_round, commit, code_source, entry,- files[path, role, bytes, sha256], total_bytes, skipped, generated_at)`.+* `record.telemetry` — `RunTelemetry(settings, cost, environment, files)`.-Everything else is unchanged, so a `record.json` written before this layout-still validates and every consumer keeps working.+The hand-over is described by `deliverable/manifest.json` (`RunDeliverable(round, commit,+code_source, entry, files[path, role, bytes, sha256], total_bytes, skipped, generated_at)`) and+`selection.json`, not by the record: since 2026-09-22 the record names no best round and+carries no `deliverable` block. A `record.json` written before that still validates — its+`best_round` / `baseline_score` / `final_score` / `deliverable` keys are ignored, a+`passed` / `plateau` status reads `stopped`, and a manifest that says `best_round` loads as+`round`.## Reading a run@@ -146,8 +177,8 @@ still validates and every consumer keeps working.and settings block is computed on the fly (read-only) from the trajectories,judge verdicts and events already on disk.-For the money itself across many runs — waste, $ per passing artifact, price-provenance — use the cost package's own command, `3dcode cost show <runs-dir>`.+For the money itself across many runs — waste, $ per run, price provenance — use the cost+package's own command, `3dcode cost show <runs-dir>`.`3dcode show` is the single-run view; both read the same ledger rows.## Back-compatibility contract@@ -162,6 +193,6 @@ provenance — use the cost package's own command, `3dcode cost show <runs-dir>`(`meta.code_source == "deliverable"`), and adds a compact `meta.telemetry`digest when the run has one (`{}` otherwise).* the gallery (`3dcode gallery build`) links the packaged- `deliverable/object.glb` and `cost.json` when they exist and falls back to- `artifacts/` otherwise; the per-stage cost line appears only for runs that- carry telemetry.+ `deliverable/object.glb` and `cost.json` when they exist, else the picked round's+ `artifacts/rNN/` copy, else `artifacts/`; the per-stage cost line appears only for+ runs that carry telemetry.