Skip to content

Official Codebase of 3DCodeVerse

Python

Merge origin/main (scene/articulated/finetune batch) into rig-v3

gaoypeng86a32e7d767c
diff --cc harness/docs/EVAL.md
index 55d02aa2,dc40ab44..c7c7494b
--- a/harness/docs/EVAL.md
+++ b/harness/docs/EVAL.md
@@@ -636,25 -636,50 +636,73 @@@ day), not in judge prose. Flash stays
Two calibration-tool defects found the same day (colliding `run` labels overwriting judgment
files; old records whose stored overall contradicts their own criterion scores) are fixed in
`judges/calibration.py` and flagged in its report.
+
+ ### 2026-08-30 — scene stack iteration: five levers, four batteries, one honest ledger
+
+ **Cross-project baseline (renders-only, our `scene_v1`, one judge):** scene_multifile_graphics
+ mean **0.433** (n=29, max 0.936, six runs ≥ 0.7) vs our pre-iteration scenes **0.272**
+ (n=24, max 0.516, zero ≥ 0.7). Every number below is the same six ToD-explicit briefs at a
+ 110-minute window unless noted; per-brief deltas at n=1 carry judge noise σ≈0.4 — only arm
+ means and mechanism evidence are read.
+
+ **Levers landed** (each commit message carries the measured motivation):
+ env-skeleton lint ERROR (251d099); boot-time settle (35168da) + slope-conformal guard
+ (e71221a); assets ∥ env (1ddea7f); plan-aware contract gates (1bdfd32); L2 zone layouts
+ (beb6605) + camera clearance (58da6b3) + `CV3D_ZONE_LAYOUTS` switch (020316d); opt-in camera
+ repair (0365fef, `CV3D_CAMERA_REPAIR=1`); opt-in auto-exposure (78d397c,
+ `CV3D_AUTO_EXPOSURE=1`).
+
+ **Longitudinal arms:**
+
+ | arm | config | vs prior arm |
+ |---|---|---|
+ | t36 | 3.6-flash, new ToD prompt | mean 0.483 (its own 75-min baseline was 0.259) |
+ | s37 | 3.7-flash + settle | Δ+0.029 vs t36 (n=6, 4W2L; nyc_dusk 0.718 = first `passed`) |
+ | fv | + camera-BLIND layouts + gates + ∥ | Δ−0.161 vs s37 (n=4, 0W3L1T) |
+ | fv2 | camera-AWARE layouts | Δ−0.036 vs s37 (n=5, 1W4L; izakaya **0.758 passed, best scene ever**) |
+
+ **Settle A/B** (re-render six finished workspaces, only variable = settle): 3W2L, mean
+ +0.063 — inconclusive at n=1, but the mechanism evidence is decisive: the slope guard cut
+ santorini's moves 17 → 3 (six hillside stairways, once lifted +1.3..+3.3 m and judged 0.0,
+ now refused), and small uniform reseats (+0.06..0.26 m) are what the wins are made of.
+
+ **Layout-layer anatomy** (why fv regressed, three distinct modes): camera-blindness — the
+ director placed BarCounter 0.7 m from a lens, three rounds of camera_in_geometry, FIXED by
+ putting cameras in the layout prompt + validator; internal overlap — RetainingWall ×
+ PrayerBench interpenetration, OPEN (a naive pairwise-distance rule false-positives on
+ legitimate adjacency like stools against a counter); richness variance with no caps at all.
+ Contract-gate lifecycle verified end to end on fv_nyc_dusk: r0 `missing_content: 2` → the
+ refine round fixed both → final round clean, settle moved nothing, 0.724 passed.
+
+ **Standing verdict:** the layout layer is net ≈ null after the camera fix and stays ON
+ (one-var off-switch exists); camera repair + auto-exposure ride in `scene_px_v1`
+ (vs fv2 same-brief = their isolated read, in flight). The honest gap to the baseline
+ project is no longer the mean — it is the ceiling (their 0.936 vs our 0.758) and the
+ floor (their 6 runs ≥ 0.7 vs our 2).
+
+ **Addendum (same day, later).** fv2 closed at n=6: the layout layer vs s37 is Δ−0.023 ≈ null (2W4L; izakaya +0.258 and alpine +0.047 are the wins). scene_px_v1 (camera repair + auto-exposure armed — verified in /proc of the live workers): isolated px−fv2 Δ+0.055 (3W1L, n=4, taj pending), arm mean 0.579 with 3/4 runs ≥ 0.6. The trigger sweep attributes that delta to VARIANCE, not to the levers: camera repair fired zero times across the battery (the layout layer's camera clearance already keeps lenses out of furniture upstream), and auto-exposure fired only on taj, where halving the exposure left the blown frame's luminance unchanged (0.8369 → 0.8369 — effectiveness investigation open; suspects: NoToneMapping renderers, toneMapped=false materials). Decisions: camera repair defaults ON (3f463ce — provably a no-op when healthy, and the class it insures against is fatal and unreachable by refine); auto-exposure stays opt-in; layouts stay ON behind the one-var switch. Next: scene_final_v1 — all 20 briefs on frozen defaults (3.7-flash + settle + layouts + camera repair, AE off) to meet the baseline's n=29 at comparable sample size.
+
+ **Addendum 2 (2026-08-31).** scene_final_v1 closed: all 20 briefs on the frozen defaults, n=19 scored (fishing_harbor died at r1 on budget with no judgeable views), stored mean 0.533, four runs `passed`, four ≥ 0.7 (wormhole 0.784, alpine 0.761, eiffel 0.739, souk 0.715). The same-scale verdict — renders only, one judge, our `scene_v1` on both corpora: **ours n=18 mean 0.488 med 0.467 max 0.802, ≥ 0.7: 3** vs the baseline project's n=29 mean 0.433 med 0.404 max 0.936, ≥ 0.7: 6. Mean and median are AHEAD of the baseline for the first time (from 0.272 at the start of this iteration); the ceiling (0.802 vs 0.936) and the ≥ 0.7 rate remain theirs. px closed with taj at 0.126 (budget), pulling the isolated P5+P6 read to null — consistent with the variance attribution above. The closing defect histogram (flat_ground 12, world_edge 11, undressed 11, thin_atmosphere 10, monotonous 10) chose the next lever: the density gate (layout budgets vs census instance counts) shipped the same day.
++
+**2026-08-31 — judge payload v3: the 14-view rig ADOPTED (D47).** 4-arm A/B on 42 items
+(18 corpus rounds + 24 h2h sides) × n=3 `gemini-3.1-pro-preview`, plus full 3.7-flash and
+3.6-flash replicas, ≈ $47. Arms: A = 8 views no clay (old baseline) · A2 =
+production-faithful 8 views + clay · B = 14 labelled 640 px single views · C = 14-view rig
++ clay through the montage machinery (5 montages + 2 crops). C mean overall 0.6835 vs A
+0.6427 / A2 0.6558 / B 0.6308; the only multiplicity survivor is same-cap C−A +0.038
+(n=29, SE 0.013, t≈2.98) — raw C−B p=.041 / C−A p=.024 do not survive Holm because cap flips make
+the deltas heavy-tailed. `untextured_flat` cap-rule fires: 11(A) / 6(A2) / 8(B) / 3(C).
+B rejected at $0.198 & 67.5k tok/verdict: its deficit is entirely cap flips (~half
+contradicting the pixels), and a defect-provenance pass over every PRESENT vote showed its
+defect-hunter halo was text-quoting — interpenetration pure-view TP is 0/14 in EVERY arm
+(the naive 14/14 measures reading comprehension of the shared gate text), and B's floating
+lead reduces to one genuinely visual item. Flash replicas: payload Δ ≈ 0 on both 3.7 and
+3.6 — the rig pays only at pro tier. C ships as D47 ($0.155/verdict, 40.9k input tok) — with the SHIPPED
+grouping re-measured (arm Cprod, $7.2): grouping alone moves the mean −0.044 vs C's
+accidental grouping (windmill 0.51 → 0.04; clean/dirty gap 0.134 → 0.155; ≈ A2 overall at
+−0.016) — a payload experiment must measure the exact grouping it ships. Underside
+full-res singles (arm Cplus, $6.8): recovers the espresso underside catch (0.795 +
+render_artifacts vs 0.965 blind) but posts the worst clean/dirty gap (0.112), the highest
+row-σ (0.040) and a windmill relapse to 0.62 — rejected; the replace-the-bottom-crop
+variant stays queued. Temperature A2 @ t=0 ($6.0): σ 0.037 → 0.022 (8/42 rows exactly
+deterministic) but mean −0.037 and pearson(gate errors) −0.195 → −0.126 — t = 0.2 stays.