Official Codebase of 3DCodeVerse
Python
docs: the gate-veto of checklist defects (EVAL §6) and the fancy_v1 07:00 readout (SKILLS_LEDGER §0a)
gaoypeng65d2dc97756a
diff --git a/harness/docs/EVAL.md b/harness/docs/EVAL.mdindex b46b0502..b96594e3 100644--- a/harness/docs/EVAL.md+++ b/harness/docs/EVAL.md@@ -182,6 +182,23 @@ Notes: the earlier criteria-first schema compressed flash to 0.6–0.7 (std 0.01part should cap via `connectivity` findings with `data["kind"]="floating"`.* A better recurring smoke than re-judging e2e rounds: a separation set withdeliberately broken variants (exploded / floating / primitive-only).+* **A checklist defect a passed gate measured absent does not cap (7a9b6d3).** The+ judge's binary checklist feeds a per-item penalty and, for `floating_part` /+ `interpenetration`-class items, a hard cap (`defect:<id>` in `caps_applied`). Those are+ also what the connectivity gate *measures*. Measured 2026-08-26 on a plan-pinned pair+ (fancy_v1 `gas_street_lamp`, two lamps the eye cannot tell apart): the gate reported+ "all 9 parts connected, gap ≤ 2 mm" and was in the judge's input; both pro samples read+ the dark seam under the pedestal as "floating in mid-air, a clear daylight gap", and+ `defect:floating_part` capped the run at 0.600 (uncapped 0.720) against 0.962 for its+ sibling. `judges/caps.measured_absent`: when a `when=gate` cap rule with the same id+ has all its watched gates passed with no ERROR finding of its `kinds`, the checklist+ claim is switched off before the penalty and before `apply_caps`, and named in the verdict+ tail ("checklist claims contradicted by a passed gate"). A failed gate, a gate that did+ not run, or a defect nothing measures (`wrong_object`, `missing_named_part`) are+ untouched. Re-aggregating the eleven judged fancy_v1 cells from their stored samples+ changed exactly one score (that lamp, 0.600 → 0.720); the other 0.600s are+ `missing_must_acceptance` / interpenetration caps the gates agree with. 0b6f52b tells the+ judge the same thing in its prompt; this holds when the judge does not listen.* Never tune rubric text against the battery you report on; bump the rubric version(`*_v2`) instead and re-run.diff --git a/harness/docs/SKILLS_LEDGER.md b/harness/docs/SKILLS_LEDGER.mdindex 0a2a97f8..afd4480d 100644--- a/harness/docs/SKILLS_LEDGER.md+++ b/harness/docs/SKILLS_LEDGER.md@@ -37,6 +37,20 @@ first-round gate rates (`--round first`) where the bundles act. Gemini spent th503 storm; pairs recorded `infra_failed` / `error` / `budget`-with-no-judged-round are outages(EVAL §7) and are re-run with `--redo-status`, never scored.+**07:00 readout (5 real pairs, plan-pinned, ON − OFF):** harp +0.012, microscope +0.087,+lamp −0.362, ships_wheel −0.256, bubbler_jukebox −0.326; mean −0.169, sd 0.204, sign 2/5.+Three ON cells landed on exactly 0.600 — what a cap looks like from the outside. Reading+the raw judgments: the lamp's cap was a false `floating_part` on a passed connectivity gate+(fixed in scoring, EVAL §6; re-aggregated 0.720 → pair −0.242), the ships_wheel's is a+`missing_must_acceptance` cap (a must-item the ON run really lacks), the jukebox's is+interpenetration (15–18 mm WARNs) plus flat materials. With the lamp re-aggregated: mean+−0.145, sd 0.182 — still inside the 0.202 floor, still 2/5 on sign; nothing here says the+library helps, and the two large negatives are real defects in the ON runs, not judge noise.+Skills stay OFF. Pairs still in flight (carousel_horse, lever_espresso, smock_windmill,+gate_valve, turbocharger, marimba, jacobs_chuck redo) land in `results.jsonl` per driver;+merge with `python bench/skill_targets.py` or the paired readout (scratch `fancy_readout.py`+— rows are only paired when both arms are `scored`).+## 0. What this wave decided (2026-08-25, curate)Three measurement waves ran: a description A/B read out on **read rate** (ground truth from