Official Codebase of 3DCodeVerse
Python
docs: the refine exporter, the git read path and the loss-event data, where they belong
ziyao12c23ada77b0
diff --git a/harness/docs/ARCHITECTURE.md b/harness/docs/ARCHITECTURE.mdindex a5e9eebf..76ecffd6 100644--- a/harness/docs/ARCHITECTURE.md+++ b/harness/docs/ARCHITECTURE.md@@ -552,7 +552,18 @@ passed, C best ≥ 0.6, D else), acceptance checklists, gate summaries anderrors, cost, fingerprints, `duplicate_of`; `--pack` tars with byte-range locators.`flywheel pairs` emits preference pairs (round i < j by judge Δ ≥ τ) and round-levelrepair pairs (in-session trajectory mining died with the api-agent, 2026-08-28 —-vendor CLIs log raw stdout, not structured tool turns); `flywheel caption` adds+vendor CLIs log raw stdout, not structured tool turns); `flywheel refine` emits the loop's OWN transitions — one row per (round i → round+i+1) where the harness asked for a change: the failing round's gate findings and+judge complaint, the refine instructions the harness wrote, both code snapshots (git+`before`/`after` commits, `--with-code` inlines the files), the score before and+after, and `outcome` (`improved` when the judge gained ≥ 0.05, `regressed` when it+lost as much, `flat` otherwise, `build_failed` when the after-round did not build).+Rows are dropped, with the reason recorded, when a predecessor has no commit, no+verdict or did not build. That is the supervision the harness produces that a+one-shot corpus cannot: what a failing artefact looked like, what was wrong with it+in the harness's own words, and what the fix changed. `toolkits/llamafactory/`+turns those rows into training files; the harness writes the measurement, not the+trainer's format. `flywheel caption` adds{detailed, instruction, factory} captions (image-grounded, brand-free, `--out` forside-car mode); `flywheel gallery` is an alias of `3dcv gallery build --embed`(the flywheel package has no renderer of its own).diff --git a/harness/docs/DECISIONS.md b/harness/docs/DECISIONS.mdindex cbfdfcfb..51e33fbe 100644--- a/harness/docs/DECISIONS.md+++ b/harness/docs/DECISIONS.md@@ -667,7 +667,7 @@ written) that were accepted because the code works that way and the tests pin it**0.225**, SE 0.065, **2 SE ±0.130**, sign 3 up / 8 down, p 0.227, separated fromnoise **NO**; the rig's own n_for_power is ~506 pairs for ±0.02. Every phase-4/5result sat inside that band, and the rig printed "revert" from identical arms.- * **The channel that works.** `local/scripts/plan_degeneracy.py` runs the plan stage+ * **The channel that works.** `bench/plan_stage_bench.py` runs the plan stageALONE (≈$0.03 and ≈85 s per call), so a 280-call arm is affordable where a 14-cellbattery is not. `CV3D_PLAN_RESTART` (a plan that names links it never lists isre-sampled from the original request instead of edited in context) measured on 560@@ -676,7 +676,9 @@ written) that were accepted because the code works that way and the tests pin it39 of 40; cost per call unchanged ($0.0345 vs $0.0337). Ships ON with a kill switch.The rule this sets: measure the loss event (a run that produced nothing, a retriedtool call, a pose the mechanism cannot reach), not the judge mean, unless the battery- is large enough for the judge mean.+ is large enough for the judge mean. The 560 rows live in+ `bench/data/plan_stage/*.jsonl` with `bench/plan_stage_report.py`: a p-value whose+ data is not in the tree is not reproducible (review, 2026-09-03).* **What the judge channel could not have found.** `is_error = not obs.ok` (§30 ofdocs/COST.md) made 62 % of 1 445 `joint_sweep` calls and 23 % of 2 144 `build` callsarrive as broken calls the model retried, and — because gemini-cli stringifies an@@ -697,6 +699,23 @@ written) that were accepted because the code works that way and the tests pin itan inconclusive lever with no cost saving is not worth a second prompt path throughthree templates, and the measurement above is the record of what it was worth. Tore-run it, restore the branch commit named in docs/PAPER_WRITING.md §9.3.+ * **Reading an agent's repo is a sandbox boundary, and `git archive` is not inside it.**+ Every flywheel read of a workspace already ran under `GIT_SAFE_FLAGS` (no repo config,+ no hooks, no user config) plus `--no-ext-diff --no-textconv` for diffs, because a+ `.gitattributes` the agent writes can name a `diff.<name>.textconv` command that git+ RUNS on our side. `read_tree_at` still used `git archive`, which renders every blob+ through `convert_to_working_tree` — so a planted `filter.<name>.smudge` executes, and+ unlike textconv there is no flag that turns it off. It reads through `ls-tree -r -z`+ + `cat-file --batch` now: same bytes, no filter path. Pinned by a test that plants a+ smudge filter and fails on the old implementation (`tests/flywheel_cli`).+ * **The refine transitions are the corpus this harness is for.** `3dcv flywheel refine`+ emits one row per round the loop asked to change: the gate findings and judge+ complaint that condemned round i, the instructions the harness wrote, both code+ snapshots, and whether the score moved. A one-shot corpus cannot contain that pair;+ the loop produces it as a by-product of running. Format conversion (LLaMA-Factory+ messages) lives in `toolkits/llamafactory/`, not in the harness — the harness writes+ the measurement, a toolkit writes whatever a trainer wants (owner's boundary,+ 2026-09-03).* **`<mimic>` is used and it changes the sweep, which is what it was for.** In the samebattery the planner and agent declared couplings in 8 of 14 prompts (umbrella 6 joints,workbench 4, step ladder 4, scissor mirror 3, rolltop desk 3), and the sampled posesdiff --git a/harness/docs/INTERFACES.md b/harness/docs/INTERFACES.mdindex 302ae319..9bc8a2b3 100644--- a/harness/docs/INTERFACES.md+++ b/harness/docs/INTERFACES.md@@ -523,6 +523,19 @@ from codeverse.flywheel.export import export_samples # (runs_dir, out_dir, *,from codeverse.flywheel.quality import quality_tier, prompt_hash, find_duplicates # tiers: A passed & 0 gate errors, B passed,# C best ≥ 0.6, D else; dedupe = (code fingerprint, prompt)from codeverse.flywheel.pairs import build_pairs # (runs_dir, out_jsonl, *, min_delta=0.05) -> n+from codeverse.flywheel.refine import build_refine, transitions, RefineTransition, REFINE_KINDS+ # build_refine(runs_dir, out_jsonl, *, with_code=False) -> n; transitions(rec, ws, *, with_code) -> list[RefineTransition]+ # one row per round i -> i+1 the harness asked to change; outcome improved|regressed|flat|build_failed+ # (threshold: pairs.MIN_PREFERENCE_DELTA); dropped rows carry the reason (no_predecessor / no_commit /+ # predecessor_build_failed / predecessor_unjudged / git_read_failed / duplicate_run)+from codeverse.flywheel._git import read_tree_at, diff_between, changed_files_between, commit_exists, GitReadError+ # read_tree_at(ws, commit) -> {path: bytes} via ls-tree + cat-file --batch — NEVER `git archive`, which+ # renders content through a planted filter.<name>.smudge and has no --no-filters (tests/flywheel_cli)+ # diff_between(ws, before, after, *, max_bytes=None) -> (text, truncated); changed_files_between -> [path]+ # both under GIT_SAFE_DIFF_FLAGS (--no-ext-diff --no-textconv) on top of workspace.GIT_SAFE_FLAGS+from codeverse.workspace import GIT_SAFE_FLAGS, GIT_SAFE_DIFF_FLAGS, git_safe_env+ # every read of an agent-written repo goes through these: no repo config, no external diff/textconv,+ # no hooks, no user config (git_safe_env strips GIT_CONFIG*, GIT_DIR, GIT_EXEC_PATH, ...)from codeverse.flywheel.captions import caption_sample # Δ (ws, record, model_id, *, model=None, out_dir=None) -> Captions;# out_dir → side-car <out_dir>/<slug>.json, run untouchedfrom codeverse.gallery import build_index, default_roots, build_static, serve, GalleryApp # THE local gallery@@ -537,7 +550,7 @@ from codeverse.flywheel.index import build_index, query, summary # sqlite + pa# rounds, status, code_fingerprint, prompt_hash, duplicate_of, has_captions3dcodeverse make [--profile economy|balanced|quality]|resume|status|show|render|judge|tools|mcp|texture {pass,scene-pack,show}|cost {<slug>,show,cache,prices,profiles,estimate}- |flywheel {export,pairs,caption,index,dedupe,gallery}|gallery {serve,build}+ |flywheel {export,pairs,refine,caption,index,dedupe,gallery}|gallery {serve,build}|bench {run,report}|doctor # alias: 3dcv```diff --git a/harness/docs/PAPER_WRITING.md b/harness/docs/PAPER_WRITING.mdindex 5c03c6d5..ddf77e7c 100644--- a/harness/docs/PAPER_WRITING.md+++ b/harness/docs/PAPER_WRITING.md@@ -10,8 +10,10 @@ The same facts live, in their own idiom, in `DECISIONS.md` (D36–D52, the decisnoise floor) and `ARCHITECTURE.md`. Tools referenced throughout:`bench/prompts/{compare_v4,articulated_v2}.yaml` (batteries), `bench/ab_plan.py` (pairedA/B with an A/A mode), `bench/compare_backends.py` (harness vs one-shot),-`bench/paired_compare.py` (paired statistics), `bench/refine_pairs.py` and-`bench/refine_sft.py` (corpus export).+`bench/paired_compare.py` (paired statistics), `bench/plan_stage_bench.py` ++`bench/plan_stage_report.py` (the loss-event channel, data in `bench/data/plan_stage/`),+`3dcv flywheel refine` (the transition corpus) and `toolkits/llamafactory/` (its training+format).## 0. The system, in one paragraph@@ -163,9 +165,11 @@ Two of these carry a second, non-score reading worth keeping:### 5.1 Planner mortality (the one significant single-switch result)-**Method.** `local/scripts/plan_degeneracy.py` runs the **plan stage alone** — no build, no+**Method.** `bench/plan_stage_bench.py` runs the **plan stage alone** — no build, nojudge — at ≈$0.03 and ≈85 s per call, so 280 calls per arm is affordable where 14 judged-cells is not. Both arms run in the same window against the same provider.+cells is not. Both arms run in the same window against the same provider. The 560 rows+are in the tree (`bench/data/plan_stage/*.jsonl`, `bench/plan_stage_report.py` prints the+table): a paper cites the p-value, so the data it comes from is committed.**Failure class, measured first.** 200 calls over two code trees showed one class: theplanner writes **one top-level part** and hangs its joints off links it never lists, and@@ -288,7 +292,7 @@ that moved the bill.## 7. The corpus the loop produces-`bench/refine_pairs.py` exports every refine round as the transition it was: the previous+`3dcv flywheel refine` exports every refine round as the transition it was: the previousround's judge issues and improvement plan, the gate ERROR findings with their fix hints, theinstruction lines the sessions were actually handed, the unified `src/` diff between the tworecorded commits, and the score delta with an improved/unchanged/regressed label. On the@@ -297,8 +301,8 @@ cross-battery symlink aliases, 3 non-batteries, 2 empty), median diff 26 kB / 31lines, **87 improved / 73 unchanged / 44 regressed**. Both sides come from the round's ownrecorded commit — only 54 of 239 runs have `HEAD` at their last round.-`bench/refine_sft.py` turns the improved transitions into the message shape the finetune-pipeline consumes: **87 samples** (71 articulated, 16 static; median answer 26.6 kB, median+`toolkits/llamafactory/build_refine_sft.py` turns the improved transitions into the message+shape the finetune pipeline consumes: **87 samples** (71 articulated, 16 static; median answer 26.6 kB, medianbrief 28.6 kB), answered with the files the round produced in the `=== FILE: path ===`envelope. This is the one thing the harness produces that a one-shot corpus cannot: *givena judged, gated object and a list of what is wrong with it, write the corrected files.*@@ -376,7 +380,7 @@ instructions nor a diff.| base64 amplification | 261 k tokens vs 516 for one 275 kB sheet | measured on a recorded blob + vendor bundle | arithmetic + matching recorded prompt jumps || token growth in turns | input(t) = 12 364 + 1 822·t; total ∝ n^1.60; dollars ∝ n^1.19–1.34 | 1 063 in-process sessions, 30 401 turns | the in-process agent, since gemini-cli emits no per-turn usage || coupled mechanisms declared | 8 of 14 prompts; poses drive 1–2 joints (was 5–6) | `wave2_lean` URDFs and round records | — |-| refine corpus | 205 transitions, 87 SFT samples | `bench/refine_pairs.py`, `bench/refine_sft.py` over `bench/out` | corpus grows with every battery |+| refine corpus | 205 transitions, 87 SFT samples | `3dcv flywheel refine`, `toolkits/llamafactory/build_refine_sft.py` over `bench/out` | corpus grows with every battery |## 9.2 Open questions@@ -429,10 +433,13 @@ python bench/ab_plan.py --prompts bench/prompts/articulated_v2.yaml \python bench/compare_backends.py --prompts bench/prompts/articulated_v2.yaml \--arms harness:gemini-cli:gemini-3.7-flash,oneshot+repair:gemini:gemini-3.7-flash \--judge gemini:gemini-3.1-pro-preview --judge-samples 3 --out bench/out/art-# the loss-event channel: the plan stage alone (see local/scripts/plan_degeneracy.py)-# the corpus-python bench/refine_pairs.py bench/out --summary-python bench/refine_sft.py bench/out --out refine_sft.jsonl --summary+# the loss-event channel: the plan stage alone, one row per call+python bench/plan_stage_bench.py --tree . --label restart_on --reps 20 \+ --out bench/data/plan_stage/restart_on.jsonl --env CV3D_PLAN_RESTART=1+python bench/plan_stage_report.py bench/data/plan_stage/*.jsonl+# the corpus the loop produces+3dcv flywheel refine bench/out refine.jsonl --with-code+python toolkits/llamafactory/build_refine_sft.py refine.jsonl --out refine_sft.jsonl```Recorded runs referenced above (not committed): `bench/out/{compare_v4_calm, compare_art_v2,diff --git a/harness/docs/RUNBOOK.md b/harness/docs/RUNBOOK.mdindex 43f61ff3..1efeb6b7 100644--- a/harness/docs/RUNBOOK.md+++ b/harness/docs/RUNBOOK.md@@ -158,6 +158,14 @@ unless you type `--host` yourself; it never serves a path outside the declared r3dcv flywheel export runs/ dataset/ [--min-score 0.7] [--only-passed] [--pack] [--include-unbuilt][--captions-dir caps/] [--drop-duplicates]3dcv flywheel pairs runs/ pairs.jsonl [--min-delta 0.05]+3dcv flywheel refine runs/ refine.jsonl [--with-code]+ # one row per round i -> i+1 the harness asked to change:+ # gate findings + judge complaint + the refine instructions,+ # before/after commits (--with-code inlines the files),+ # score_before/score_after, outcome improved|regressed|flat|+ # build_failed, and dropped rows with the reason+ # (no_predecessor / no_commit / predecessor_unjudged / ...).+ # Training formats live in toolkits/llamafactory/, not here.3dcv flywheel caption <slug> [--model …] [--out caps/] # --out = side-car mode, run untouched3dcv flywheel gallery runs/ gallery.html [--title …] # alias of `3dcv gallery build --embed` (§4)3dcv flywheel index runs/ runs_index.sqlite · 3dcv flywheel dedupe dataset/