Official Codebase of 3DCodeVerse
Python
judges: the veto's depth line is 8 mm, set by the experiment that broke 5
gaoypeng9d3b280d644a
diff --git a/harness/codeverse/judges/rubrics.py b/harness/codeverse/judges/rubrics.pyindex b8f1a981..6ac6b0ea 100644--- a/harness/codeverse/judges/rubrics.py+++ b/harness/codeverse/judges/rubrics.py@@ -547,10 +547,13 @@ def measured_absent(rubric: Rubric, defect_id: str, gates: list[GateReport]) ->#: A penetration the gate rated WARN but measured at least this deep is NOT "measured#: absent": the rubric excuses "weld overlaps of a few mm", and the veto that trusted every-#: WARN was switching off 163 interpenetration claims of which 79 sat on a 5 mm+ overlap-#: (median 4.8 mm) — and, since 2026-08-30, a stile 17 mm through a seat is a WARN too.-#: From here up the judge's reading stands and the picture decides.-VETO_PENETRATION_DEPTH_M = 0.005+#: WARN was switching off 163 interpenetration claims — and, since 2026-08-30, a stile 17 mm+#: through a seat is a WARN too. From here up the judge's reading stands and the picture+#: decides. The line is 8 mm, not 5: the paired re-judge marked the pipe tee's 5.8 mm+#: branch socket — the canonical DESIGNED weld — as the defect the moment a 5 mm line let+#: the claim stand (−0.32 on that side), and the corpus holds only 9 standing claims in the+#: 5–8 mm band (sockets, vise rails) against 16 at 8–10 mm and 43 with an ERROR behind them.+VETO_PENETRATION_DEPTH_M = 0.008def _rule_gates_clean(rule: CapRule, gates: list[GateReport]) -> bool:diff --git a/harness/docs/DECISIONS.md b/harness/docs/DECISIONS.mdindex e00aa2c3..63eacaba 100644--- a/harness/docs/DECISIONS.md+++ b/harness/docs/DECISIONS.md@@ -358,17 +358,20 @@ written) that were accepted because the code works that way and the tests pin it(`bench/rejudge_offline.py`, 419 verdicts, $0):(a) **`CapRule.measures`** names the checklist defect a gate rule is the measurement of, and theobject rubrics' `floating_part` / `penetration_error` watch `connectivity` only. The veto is- depth-aware: a penetration WARN measured at `VETO_PENETRATION_DEPTH_M` (5 mm) or deeper is not- "measured absent" — of the 163 claims a WARN-blind veto switched off, 79 sat on a 5 mm+ overlap- (median 4.8 mm), and since (d) a stile 17 mm through a seat is a WARN too. Replayed with (b):- 213 verdicts move (0 down), mean +0.078, pass 15.0 % → 19.3 %, pearson(gate errors, overall)- −0.219 → −0.291, vetoed on replay: interpenetration 93, floating 72. articulated_v1 keeps+ depth-aware: a penetration WARN measured at `VETO_PENETRATION_DEPTH_M` (8 mm) or deeper is not+ "measured absent" — the WARN-blind veto switched off 163 claims, and since (d) a stile 17 mm+ through a seat is a WARN too. 8 mm, not 5: at 5 the paired re-judge marked the pipe tee's+ 5.8 mm branch socket — the canonical designed weld — as the defect (−0.32 on that side), and+ the corpus holds only 9 standing claims in the 5–8 mm band against 16 at 8–10 and 43 with an+ ERROR behind them. Replayed with (b):+ 230 verdicts move (0 down), mean +0.086, pass 15.0 % → 21.0 %, pearson(gate errors, overall)+ −0.219 → −0.301, vetoed on replay: interpenetration 130, floating 72. articulated_v1 keeps`gate: "*"` — no single gate owns either measurement there (joint_sweep + connectivity).`Rubric.content_hash` hashes what the YAML declares, not the model's defaults, so a schema field(`measures`, `graded`) never re-keys recorded verdicts of an unchanged rubric.(b) **The acceptance cap is graded**: `cap + (1 − cap) · verified/total` must items. Pass/failunchanged (`must_missing` still fails). Without it 98 of the rounds (a) frees are re-pinned to- 0.600 by the flat cap; with it the 0.600 spike falls 130 → 19.+ 0.600 by the flat cap; with it the 0.600 spike falls 130 → 20.(c) **A defect vote tie is absent** (D36 amended); acceptance ties unchanged. `SCORING_VERSION`= 2 is stamped on every breakdown; the replay tool holds identity only to same-version verdicts.(d) **The gate measures overlap where it is**: a dense pass on the AABB-overlap region, adiff --git a/harness/docs/EVAL.md b/harness/docs/EVAL.mdindex f42e0598..3c81d5f9 100644--- a/harness/docs/EVAL.md+++ b/harness/docs/EVAL.md@@ -210,18 +210,19 @@ Notes: the earlier criteria-first schema compressed flash to 0.6–0.7 (std 0.01never vetoed), and the object rubrics' rules watched `gate: "*"`, so a failing contract gate(53 of the 62 blocked cases) switched off a veto connectivity had earned; they now watch`connectivity` alone, the only gate that emits floating / penetration ERRORs on this track- (540 findings over 356 records). The veto is depth-aware (`VETO_PENETRATION_DEPTH_M`, 5 mm):- a penetration WARN measured that deep is not "absent" — the WARN-blind version switched off- 163 interpenetration claims, 79 of them on a 5 mm+ overlap. With the graded cap below the- final replay moves 213 of 419 stored verdicts (none down), pass rate 15.0 % → 19.3 %,- pearson(gate errors, overall) −0.219 → −0.291, vetoed: interpenetration 93, floating 72. 0b6f52b tells the judge the same thing in its prompt; this holds when the judge+ (540 findings over 356 records). The veto is depth-aware (`VETO_PENETRATION_DEPTH_M`, 8 mm —+ 5 mm let the judge's tick on the pipe tee's 5.8 mm designed branch socket stand, −0.32 on that+ side of the paired re-judge): a penetration WARN measured that deep is not "absent" — the+ WARN-blind version switched off 163 interpenetration claims. With the graded cap below the+ final replay moves 230 of 419 stored verdicts (none down), pass rate 15.0 % → 21.0 %,+ pearson(gate errors, overall) −0.219 → −0.301, vetoed: interpenetration 130, floating 72. 0b6f52b tells the judge the same thing in its prompt; this holds when the judgedoes not listen.* **`missing_must_acceptance` is graded (2026-08-30).** The flat 0.6 was the decisive cap on121 of 424 static_object verdicts (28.6 %): one unverified must item out of ten scoredexactly like ten out of ten, and 130 of 419 stored scores sat on 0.600. The cap is now`0.6 + 0.4 · verified/total` over the must items (`CapRule.graded`; the ledger line says"k of n must items verified"). Pass/fail is unchanged — any unverified must item still- fails — only the score keeps its gradient: the 0.600 spike drops 130 → 19 on replay, σ+ fails — only the score keeps its gradient: the 0.600 spike drops 130 → 20 on replay, σ0.206 → 0.224. Every breakdown now carries `scoring_version` (`rubrics.SCORING_VERSION`,2 for this batch); `rejudge_offline --identity` holds only same-version verdicts to 1e-9.* **The judge reads the contact ledger, not WARN prose (2026-08-30).** Audited over 420@@ -233,10 +234,14 @@ Notes: the earlier criteria-first schema compressed flash to 0.6–0.7 (std 0.01joins; 171 have ≥ 1 OPEN one — the assembly_fit ground truth that did not exist), and thelowest point above the floor with its number. p50 307 / p90 484 tokens on the corpus.It rides on the v1 `judge_prompt_hash` (per-run text is not hashed), so its effect is NOT- in any replay: the measurement is a paired re-judge — 40 rounds, fixed order, arm A the- stored text vs arm B the ledger text over the same images (~$5.5), kill criteria: the- interpenetration claim rate on gate-clean rounds must fall from 39 %, and on open-join- rounds assembly_fit must separate by more than the fixed-order σ below.+ in any replay: the measurement is a paired re-judge. **Run 2026-08-30** (42 matched items,+ the 53-item σ battery as arm A vs the shipped bundle as arm B, fixed order, n=3, $6.12):+ on old-gate-clean items the interpenetration claim rate moved 21 % → 18 % (n=28 —+ underpowered against the corpus's 39 % criterion, which needs the 120 view-pruned rounds);+ Δ(B−A) overall +0.059 mean (corpus rounds +0.141, h2h ours −0.040, h2h theirs +0.026);+ within-arm σ unchanged (0.030 → 0.032). Two case reads: the new gate's 12.7 mm ERROR on+ clock_q4 is a real catch (0.912 → 0.700), and the pipe tee's −0.32 exposed the 5 mm veto+ line marking a designed 5.8 mm branch socket — which is why the line is 8 mm.* **The judge's own re-judge σ is 0.035 (2026-08-30, fixed montage order).** 53 items — the29 corpus rounds that still carry view PNGs + the 24 h2h object-sides re-rendered from theirGLBs — judged three times each with the identical prompt (`VlmJudge(fixed_order=True)`,diff --git a/harness/tests/judges/test_prompt_builder.py b/harness/tests/judges/test_prompt_builder.pyindex e4e5e871..363999d8 100644--- a/harness/tests/judges/test_prompt_builder.py+++ b/harness/tests/judges/test_prompt_builder.py@@ -243,9 +243,9 @@ def test_ledger_turns_penetration_warn_prose_into_one_measured_line():assert "- ground: lowest point 0.0 mm (BackPlate) — floor contact; parts within 10 mm of the floor: RibStructure 3.0 mm" in textassert ("CONNECTIVITY PASSED: all 16 parts are in measured contact (26 contacts, gap <= 2 mm)." in textand "Do NOT report any part as floating" in text)- # five overlaps reach 5 mm (the veto's line): no injunction against the interpenetration tick,+ # three overlaps reach 8 mm (the veto's line): no injunction against the interpenetration tick,# the measurement is named and the picture decides — the prompt and rubrics._rule_gates_clean agree- assert "All 23 overlaps measured; 5 reach 5 mm or more (" in text+ assert "All 23 overlaps measured; 3 reach 8 mm or more (" in textassert "mark interpenetration only if a part VISIBLY passes through another in a render" in textassert "do NOT mark the interpenetration defect" not in textassert text.index("errors (1)") < text.index("MEASURED STRUCTURE") < text.index("connectivity measured") < text.index("warnings (1)")