Official Codebase of 3DCodeVerse
Python
docs: the fancy_v1 skills A/B final readout — n=12, mean −0.122, sign 4/12; skills stay OFF
gaoypeng18a5d6dd6029
diff --git a/harness/docs/SKILLS_LEDGER.md b/harness/docs/SKILLS_LEDGER.mdindex afd4480d..e2291863 100644--- a/harness/docs/SKILLS_LEDGER.md+++ b/harness/docs/SKILLS_LEDGER.md@@ -37,6 +37,18 @@ first-round gate rates (`--round first`) where the bundles act. Gemini spent th503 storm; pairs recorded `infra_failed` / `error` / `budget`-with-no-judged-round are outages(EVAL §7) and are re-run with `--redo-status`, never scored.+**Final readout, 12:50 (wave finished; 13 pairs, plan-pinned, ON − OFF, fixed pro judge):**+microscope +0.087, carousel −0.109, chamber_organ 0.000, harp +0.012, lamp −0.362 (−0.242+re-aggregated), orbital_shaker −0.211, lever_espresso −0.575, ships_wheel −0.256, gate_valve+−0.058, turbocharger +0.331 (control 0.000 was the evaluator's cadquery build defect, a303086 —+not a result; drop it), jukebox −0.326, marimba +0.077, smock_windmill +0.260. With+turbocharger dropped: n=12, mean **−0.122** (re-aggregated −0.112), sd 0.24, sign 4/12 —+inside the 0.202 floor on the mean, with the large negatives being real defects in the ON runs+(a lever espresso that fell apart, a lamp and a wheel that lost must-items). The whole+library, measured as shipped, does not help flash on hard objects and may hurt; **skills stay+OFF**. Storm cost of this wave: 16 pairs took 12 h of wall clock across four drivers for 13+real pairs; the redo machinery (RUNBOOK 7.x/7.w) is what produced the last seven.+**07:00 readout (5 real pairs, plan-pinned, ON − OFF):** harp +0.012, microscope +0.087,lamp −0.362, ships_wheel −0.256, bubbler_jukebox −0.326; mean −0.169, sd 0.204, sign 2/5.Three ON cells landed on exactly 0.600 — what a cap looks like from the outside. Readingdiff --git a/harness/docs/report.html b/harness/docs/report.htmlindex a6fcddee..df5657ae 100644--- a/harness/docs/report.html+++ b/harness/docs/report.html@@ -401,7 +401,7 @@ tr.hl td.arm{font-weight:600}<p>Also added: an import-direction test (<code>judges/</code>, <code>contracts/</code>, <code>proc.py</code>, <code>workspace.py</code> never import upward — it fired once the same night and was right), the <code>proc.py</code> rule, and the last wave-marker shim retired. Not done, on purpose: the 15 files over the 400-line law (0 lines saved, all pinned; one split per wave from <code>rounds.py</code>).</p><h4>The wave, at 07:10 — and the cap it exposed</h4>- <p>16 hard, design-realistic object prompts (blender 10, cadquery 3, threejs 3), skills-OFF vs skills-ON, plan-pinned, on all 22 keys, through a night of Gemini 503 storms (outage rows are redone from a fresh driver, never scored). Six real pairs so far, ON − OFF: harp +0.012, microscope +0.087, gate_valve −0.058, ships_wheel −0.256, jukebox −0.326, street lamp −0.362 — mean <strong>−0.150</strong>, sd 0.188, sign 2/6, against a paired noise floor of 0.202. Nothing here says the library helps; skills stay OFF.</p>+ <p>16 hard, design-realistic object prompts (blender 10, cadquery 3, threejs 3), skills-OFF vs skills-ON, plan-pinned, on all 22 keys, through a night of Gemini 503 storms (outage rows are redone from a fresh driver, never scored). Final, 13 pairs (one dropped as an evaluator defect): microscope +0.087, marimba +0.077, windmill +0.260, harp +0.012, organ 0.000, gate_valve −0.058, carousel −0.109, shaker −0.211, ships_wheel −0.256, jukebox −0.326, street lamp −0.362, lever espresso −0.575 — mean <strong>−0.122</strong>, sd 0.24, sign 4/12, against a paired noise floor of 0.202. Nothing here says the library helps; the large negatives are real defects in the ON runs. Skills stay OFF.</p><p>Three ON cells landed on <em>exactly</em> 0.600, which is what a cap looks like from the outside. The street lamp's was false: the connectivity gate had measured "all 9 parts connected, gap ≤ 2 mm" and said so in the judge's input; both pro samples read the dark seam under the pedestal as "floating in mid-air, a clear daylight gap", ticked <code>floating_part</code>, and the checklist cap held the run at 0.600 (uncapped 0.720) against 0.962 for a sibling the eye cannot tell apart. Fixed in code, not in the prompt: a checklist defect whose subject a passed gate measured absent no longer penalises or caps (<code>judges/caps.measured_absent</code>, named in the verdict tail). Re-aggregating all eleven judged cells from their stored samples moved exactly that one score (0.600 → 0.720; pair −0.242); the other 0.600s are must-item and interpenetration caps the gates agree with. With it, the wave reads mean −0.130, sd 0.167 — the same answer.</p><h4>The graphics judge, measured against an eye — and the loop that fixes it</h4><p>Seventeen judged graphics runs were scored by a person from the same contact sheets. Against the loop-time <code>shader_v1</code> verdicts: Spearman <strong>0.16</strong>, judge mean 0.773 vs eye 0.550 — three aurora versions nobody would take for an aurora at 0.78–0.94 with empty issue lists, hard pastel discs for bokeh at 0.92, a lifted purple wash for a nebula at 0.82, a crisp ukiyo-e wave at 0.59 under planner must items. The rubric scored the nouns of the brief being present. <code>shader_v2</code> scores likeness (would someone who has seen the thing accept it), tonal range, and a checklist of what flash actually draws (comb artefact, opaque shapes for light, white wash, candy saturation, no dark range). Re-judged in the same replay (pro n=2, plan digest + frame metrics): <strong>v1 0.34 → v2 0.71</strong>, mean 0.562 against the eye's 0.550. Reference photos now ride with the brief (<code>bench/refs/<id>/</code>): the agent sees them on its first message and a <code>LikenessJudge</code> puts them beside the frames — the first aurora run with photos was judged 0.49 (“rigid evenly spaced bars, not folding curtains”) where the same picture scored 0.94 the day before, and its refine round reached 0.70 with folded curtains. <code>docs/GRAPHICS_LOOP.md</code> is the recipe and the ledger.</p>