Skip to content

Official Codebase of 3DCodeVerse

Python

judges: the veto's depth line is 8 mm, set by the experiment that broke 5

gaoypeng9d3b280d644a
diff --git a/harness/codeverse/judges/rubrics.py b/harness/codeverse/judges/rubrics.py
index b8f1a981..6ac6b0ea 100644
--- a/harness/codeverse/judges/rubrics.py
+++ b/harness/codeverse/judges/rubrics.py
@@ -547,10 +547,13 @@ def measured_absent(rubric: Rubric, defect_id: str, gates: list[GateReport]) ->
#: A penetration the gate rated WARN but measured at least this deep is NOT "measured
#: absent": the rubric excuses "weld overlaps of a few mm", and the veto that trusted every
-#: WARN was switching off 163 interpenetration claims of which 79 sat on a 5 mm+ overlap
-#: (median 4.8 mm) — and, since 2026-08-30, a stile 17 mm through a seat is a WARN too.
-#: From here up the judge's reading stands and the picture decides.
-VETO_PENETRATION_DEPTH_M = 0.005
+#: WARN was switching off 163 interpenetration claims — and, since 2026-08-30, a stile 17 mm
+#: through a seat is a WARN too. From here up the judge's reading stands and the picture
+#: decides. The line is 8 mm, not 5: the paired re-judge marked the pipe tee's 5.8 mm
+#: branch socket — the canonical DESIGNED weld — as the defect the moment a 5 mm line let
+#: the claim stand (−0.32 on that side), and the corpus holds only 9 standing claims in the
+#: 5–8 mm band (sockets, vise rails) against 16 at 8–10 mm and 43 with an ERROR behind them.
+VETO_PENETRATION_DEPTH_M = 0.008
def _rule_gates_clean(rule: CapRule, gates: list[GateReport]) -> bool:
diff --git a/harness/docs/DECISIONS.md b/harness/docs/DECISIONS.md
index e00aa2c3..63eacaba 100644
--- a/harness/docs/DECISIONS.md
+++ b/harness/docs/DECISIONS.md
@@ -358,17 +358,20 @@ written) that were accepted because the code works that way and the tests pin it
(`bench/rejudge_offline.py`, 419 verdicts, $0):
(a) **`CapRule.measures`** names the checklist defect a gate rule is the measurement of, and the
object rubrics' `floating_part` / `penetration_error` watch `connectivity` only. The veto is
- depth-aware: a penetration WARN measured at `VETO_PENETRATION_DEPTH_M` (5 mm) or deeper is not
- "measured absent" — of the 163 claims a WARN-blind veto switched off, 79 sat on a 5 mm+ overlap
- (median 4.8 mm), and since (d) a stile 17 mm through a seat is a WARN too. Replayed with (b):
- 213 verdicts move (0 down), mean +0.078, pass 15.0 % → 19.3 %, pearson(gate errors, overall)
- −0.219 → −0.291, vetoed on replay: interpenetration 93, floating 72. articulated_v1 keeps
+ depth-aware: a penetration WARN measured at `VETO_PENETRATION_DEPTH_M` (8 mm) or deeper is not
+ "measured absent" — the WARN-blind veto switched off 163 claims, and since (d) a stile 17 mm
+ through a seat is a WARN too. 8 mm, not 5: at 5 the paired re-judge marked the pipe tee's
+ 5.8 mm branch socket — the canonical designed weld — as the defect (−0.32 on that side), and
+ the corpus holds only 9 standing claims in the 5–8 mm band against 16 at 8–10 and 43 with an
+ ERROR behind them. Replayed with (b):
+ 230 verdicts move (0 down), mean +0.086, pass 15.0 % → 21.0 %, pearson(gate errors, overall)
+ −0.219 → −0.301, vetoed on replay: interpenetration 130, floating 72. articulated_v1 keeps
`gate: "*"` — no single gate owns either measurement there (joint_sweep + connectivity).
`Rubric.content_hash` hashes what the YAML declares, not the model's defaults, so a schema field
(`measures`, `graded`) never re-keys recorded verdicts of an unchanged rubric.
(b) **The acceptance cap is graded**: `cap + (1 − cap) · verified/total` must items. Pass/fail
unchanged (`must_missing` still fails). Without it 98 of the rounds (a) frees are re-pinned to
- 0.600 by the flat cap; with it the 0.600 spike falls 130 → 19.
+ 0.600 by the flat cap; with it the 0.600 spike falls 130 → 20.
(c) **A defect vote tie is absent** (D36 amended); acceptance ties unchanged. `SCORING_VERSION`
= 2 is stamped on every breakdown; the replay tool holds identity only to same-version verdicts.
(d) **The gate measures overlap where it is**: a dense pass on the AABB-overlap region, a
diff --git a/harness/docs/EVAL.md b/harness/docs/EVAL.md
index f42e0598..3c81d5f9 100644
--- a/harness/docs/EVAL.md
+++ b/harness/docs/EVAL.md
@@ -210,18 +210,19 @@ Notes: the earlier criteria-first schema compressed flash to 0.6–0.7 (std 0.01
never vetoed), and the object rubrics' rules watched `gate: "*"`, so a failing contract gate
(53 of the 62 blocked cases) switched off a veto connectivity had earned; they now watch
`connectivity` alone, the only gate that emits floating / penetration ERRORs on this track
- (540 findings over 356 records). The veto is depth-aware (`VETO_PENETRATION_DEPTH_M`, 5 mm):
- a penetration WARN measured that deep is not "absent" — the WARN-blind version switched off
- 163 interpenetration claims, 79 of them on a 5 mm+ overlap. With the graded cap below the
- final replay moves 213 of 419 stored verdicts (none down), pass rate 15.0 % → 19.3 %,
- pearson(gate errors, overall) −0.219 → −0.291, vetoed: interpenetration 93, floating 72. 0b6f52b tells the judge the same thing in its prompt; this holds when the judge
+ (540 findings over 356 records). The veto is depth-aware (`VETO_PENETRATION_DEPTH_M`, 8 mm —
+ 5 mm let the judge's tick on the pipe tee's 5.8 mm designed branch socket stand, −0.32 on that
+ side of the paired re-judge): a penetration WARN measured that deep is not "absent" — the
+ WARN-blind version switched off 163 interpenetration claims. With the graded cap below the
+ final replay moves 230 of 419 stored verdicts (none down), pass rate 15.0 % → 21.0 %,
+ pearson(gate errors, overall) −0.219 → −0.301, vetoed: interpenetration 130, floating 72. 0b6f52b tells the judge the same thing in its prompt; this holds when the judge
does not listen.
* **`missing_must_acceptance` is graded (2026-08-30).** The flat 0.6 was the decisive cap on
121 of 424 static_object verdicts (28.6 %): one unverified must item out of ten scored
exactly like ten out of ten, and 130 of 419 stored scores sat on 0.600. The cap is now
`0.6 + 0.4 · verified/total` over the must items (`CapRule.graded`; the ledger line says
"k of n must items verified"). Pass/fail is unchanged — any unverified must item still
- fails — only the score keeps its gradient: the 0.600 spike drops 130 → 19 on replay, σ
+ fails — only the score keeps its gradient: the 0.600 spike drops 130 → 20 on replay, σ
0.206 → 0.224. Every breakdown now carries `scoring_version` (`rubrics.SCORING_VERSION`,
2 for this batch); `rejudge_offline --identity` holds only same-version verdicts to 1e-9.
* **The judge reads the contact ledger, not WARN prose (2026-08-30).** Audited over 420
@@ -233,10 +234,14 @@ Notes: the earlier criteria-first schema compressed flash to 0.6–0.7 (std 0.01
joins; 171 have ≥ 1 OPEN one — the assembly_fit ground truth that did not exist), and the
lowest point above the floor with its number. p50 307 / p90 484 tokens on the corpus.
It rides on the v1 `judge_prompt_hash` (per-run text is not hashed), so its effect is NOT
- in any replay: the measurement is a paired re-judge — 40 rounds, fixed order, arm A the
- stored text vs arm B the ledger text over the same images (~$5.5), kill criteria: the
- interpenetration claim rate on gate-clean rounds must fall from 39 %, and on open-join
- rounds assembly_fit must separate by more than the fixed-order σ below.
+ in any replay: the measurement is a paired re-judge. **Run 2026-08-30** (42 matched items,
+ the 53-item σ battery as arm A vs the shipped bundle as arm B, fixed order, n=3, $6.12):
+ on old-gate-clean items the interpenetration claim rate moved 21 % → 18 % (n=28 —
+ underpowered against the corpus's 39 % criterion, which needs the 120 view-pruned rounds);
+ Δ(B−A) overall +0.059 mean (corpus rounds +0.141, h2h ours −0.040, h2h theirs +0.026);
+ within-arm σ unchanged (0.030 → 0.032). Two case reads: the new gate's 12.7 mm ERROR on
+ clock_q4 is a real catch (0.912 → 0.700), and the pipe tee's −0.32 exposed the 5 mm veto
+ line marking a designed 5.8 mm branch socket — which is why the line is 8 mm.
* **The judge's own re-judge σ is 0.035 (2026-08-30, fixed montage order).** 53 items — the
29 corpus rounds that still carry view PNGs + the 24 h2h object-sides re-rendered from their
GLBs — judged three times each with the identical prompt (`VlmJudge(fixed_order=True)`,
diff --git a/harness/tests/judges/test_prompt_builder.py b/harness/tests/judges/test_prompt_builder.py
index e4e5e871..363999d8 100644
--- a/harness/tests/judges/test_prompt_builder.py
+++ b/harness/tests/judges/test_prompt_builder.py
@@ -243,9 +243,9 @@ def test_ledger_turns_penetration_warn_prose_into_one_measured_line():
assert "- ground: lowest point 0.0 mm (BackPlate) — floor contact; parts within 10 mm of the floor: RibStructure 3.0 mm" in text
assert ("CONNECTIVITY PASSED: all 16 parts are in measured contact (26 contacts, gap <= 2 mm)." in text
and "Do NOT report any part as floating" in text)
- # five overlaps reach 5 mm (the veto's line): no injunction against the interpenetration tick,
+ # three overlaps reach 8 mm (the veto's line): no injunction against the interpenetration tick,
# the measurement is named and the picture decides — the prompt and rubrics._rule_gates_clean agree
- assert "All 23 overlaps measured; 5 reach 5 mm or more (" in text
+ assert "All 23 overlaps measured; 3 reach 8 mm or more (" in text
assert "mark interpenetration only if a part VISIBLY passes through another in a render" in text
assert "do NOT mark the interpenetration defect" not in text
assert text.index("errors (1)") < text.index("MEASURED STRUCTURE") < text.index("connectivity measured") < text.index("warnings (1)")