Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(java): ConfidenceGate, and the state a boolean cannot express
Adeshina Lasisi3e683fda211d
diff --git a/.github/workflows/java.yml b/.github/workflows/java.ymlindex cc756551..b8a8ac0c 100644--- a/.github/workflows/java.yml+++ b/.github/workflows/java.yml@@ -160,7 +160,7 @@ jobs:if: ${{ !cancelled() }}run: >python laya-java/scripts/check_test_results.py- laya-java/laya-java/build/test-results/test --min-tests 625 --allow-aborted+ laya-java/laya-java/build/test-results/test --min-tests 680 --allow-aborted# The samples are single-file source programs, deliberately NOT a Gradle module: a# `samples` subproject would inherit the root build's MavenPublication and@@ -319,7 +319,7 @@ jobs:if: ${{ !cancelled() }}run: >python laya-java/scripts/check_test_results.py- laya-java/laya-java/build/test-results/test --min-tests 625 --allow-aborted+ laya-java/laya-java/build/test-results/test --min-tests 680 --allow-aborted--require-passed TestJvmVersionTest- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2if: ${{ failure() }}diff --git a/.github/workflows/release-java.yml b/.github/workflows/release-java.ymlindex b00971a2..bb51d071 100644--- a/.github/workflows/release-java.yml+++ b/.github/workflows/release-java.yml@@ -67,9 +67,10 @@ jobs:# that lane already covered.- name: Model-free testsrun: ./gradlew --no-daemon test- # 631 of the 640 registered tests run without a checkpoint. The floor was 440, which- # left 191 of slack -- the whole decode family could have vanished from this lane with- # the gate still green, and a floor that does not bind is not a gate.+ # 685 of the 699 registered tests run without a checkpoint. This floor has been raised+ # as the model-free suite grew -- 440 left 191 of slack, 625 left 60 -- because a floor+ # that does not bind is not a gate: under 625 the whole confidence-gate family of 26+ # tests could have vanished from this lane with the gate still green.## These lines sit ABOVE `run:`, not inside it. A `>` block folds to one line and `#` is# not a comment inside a YAML scalar, so a comment written in there reaches the shell and@@ -82,7 +83,7 @@ jobs:- name: The suite was not vacuousrun: >python3 scripts/check_test_results.py laya-java/build/test-results/test- --min-tests 625 --allow-aborted+ --min-tests 680 --allow-aborted- name: Build the jarsrun: ./gradlew --no-daemon build -x testdiff --git a/laya-java/CHANGELOG.md b/laya-java/CHANGELOG.mdindex 1722b9bb..ef9dba65 100644--- a/laya-java/CHANGELOG.md+++ b/laya-java/CHANGELOG.md@@ -20,6 +20,15 @@ see [Installing](README.md#installing).`confidence` alongside `answerConfidence`.- Python-compatible tokenization (BPE and Metaspace, added tokens, normalizers), sequencebuilding, collation, answer decoding and calibration.+- `ConfidenceGate`: the opt-in abstention gate. Reports three states — `PASSED`, `ABSTAINED`+ and `UNEVALUATED` — because an answer that carried no usable confidence is not a pass, and a+ boolean cannot say so. Reads `answerConfidence` (what temperature scaling fits) and falls back+ to `confidence` (normalised entropy, a different scale) only when the first is unusable.+ Per-bucket thresholds are supported, since one threshold does not transfer across option+ counts. Unlike the reference it does not mutate the answers — records cannot — so `apply`+ returns a report, and an **ungated** call returns an empty `Optional` rather than a report of+ nulls: the presence of the verdict is how a caller tells "no gate ran" from "everything+ passed".- `Agent.predictLong` and `LongPrediction`: questions answered over a state longer than thecontext window. `predict` truncates such a state to one window and drops the rest silently;this tokenizes it once, scans it in overlapping windows, scores them in shared graph calls anddiff --git a/laya-java/README.md b/laya-java/README.mdindex 2ce22fb0..3337f767 100644--- a/laya-java/README.md+++ b/laya-java/README.md@@ -301,6 +301,26 @@ String withSpecials = tok.decode(ids, false); // ...keptAgentConfig cfg = AgentConfig.fromModelDirectory(Path.of("./checkpoint"));```+## Abstaining on a low-confidence answer++```java+var gated = ConfidenceGate.apply(prediction.answers(), 0.8); // null = no gate at all+gated.ifPresent(report -> report.forEach((id, v) -> {+ if (v.abstention() == ConfidenceGate.Abstention.ABSTAINED) {+ escalate(id, v.answer(), v.threshold());+ }+}));+```++Three states, not a boolean: `UNEVALUATED` is an answer the gate could not read, and calling+that a pass is the same mistake as calling it a flag. An **ungated** call returns an empty+`Optional`, so the presence of a report is what tells you a gate ran.++A threshold is not a claim that the number is calibrated — "about c of the answers returned at+c are correct" holds only after temperatures have been fitted and validated for that checkpoint+and question shape. Pass a `Map` instead of a scalar to gate each option-count bucket+(`choice:2`, `score:6-10`, …) at the level its calibration actually earns.+## A state longer than the context window```javadiff --git a/laya-java/fixtures/confidence_gate.json b/laya-java/fixtures/confidence_gate.jsonnew file mode 100644index 00000000..89f0be8f--- /dev/null+++ b/laya-java/fixtures/confidence_gate.json@@ -0,0 +1,829 @@+{+ "states": [+ "passed",+ "abstained",+ "unevaluated"+ ],+ "cases": [+ {+ "case": "ungated-writes-nothing",+ "min_confidence": null,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ }+ }+ },+ {+ "case": "ungated-with-several",+ "min_confidence": null,+ "before": {+ "a": {+ "type": "choice",+ "probabilities": {+ "0": 0.333333,+ "1": 0.333333,+ "2": 0.333333+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.8+ },+ "b": {+ "type": "noul",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "noul": 0.6,+ "answer_confidence": 0.4,+ "confidence": 0.4+ }+ },+ "after": {+ "a": {+ "type": "choice",+ "probabilities": {+ "0": 0.333333,+ "1": 0.333333,+ "2": 0.333333+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.8+ },+ "b": {+ "type": "noul",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "noul": 0.6,+ "answer_confidence": 0.4,+ "confidence": 0.4+ }+ }+ },+ {+ "case": "scalar-abstains",+ "min_confidence": 0.8,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5,+ "low_confidence": true,+ "abstention": "abstained",+ "abstention_threshold": 0.8+ }+ }+ },+ {+ "case": "scalar-passes",+ "min_confidence": 0.8,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.5+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.5,+ "abstention": "passed",+ "abstention_threshold": 0.8+ }+ }+ },+ {+ "case": "scalar-exactly-at-threshold",+ "min_confidence": 0.8,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.8,+ "confidence": 0.5+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.8,+ "confidence": 0.5,+ "abstention": "passed",+ "abstention_threshold": 0.8+ }+ }+ },+ {+ "case": "scalar-zero-reports-passed",+ "min_confidence": 0.0,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5,+ "abstention": "passed",+ "abstention_threshold": 0.0+ }+ }+ },+ {+ "case": "scalar-one-abstains-everything",+ "min_confidence": 1.0,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.99,+ "confidence": 0.9+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.99,+ "confidence": 0.9,+ "low_confidence": true,+ "abstention": "abstained",+ "abstention_threshold": 1.0+ }+ }+ },+ {+ "case": "falls-back-to-confidence",+ "min_confidence": 0.5,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "confidence": 0.3+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "confidence": 0.3,+ "low_confidence": true,+ "abstention": "abstained",+ "abstention_threshold": 0.5+ }+ }+ },+ {+ "case": "prefers-answer-confidence",+ "min_confidence": 0.5,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.1+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.1,+ "abstention": "passed",+ "abstention_threshold": 0.5+ }+ }+ },+ {+ "case": "no-usable-confidence",+ "min_confidence": 0.5,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0"+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "abstention": "unevaluated",+ "abstention_threshold": 0.5+ }+ }+ },+ {+ "case": "nan-is-not-a-confidence",+ "min_confidence": 0.5,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": "__nan__",+ "confidence": "__nan__"+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": "__nan__",+ "confidence": "__nan__",+ "abstention": "unevaluated",+ "abstention_threshold": 0.5+ }+ }+ },+ {+ "case": "bool-is-not-a-confidence",+ "min_confidence": 0.5,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": true+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": true,+ "abstention": "unevaluated",+ "abstention_threshold": 0.5+ }+ }+ },+ {+ "case": "map-by-bucket",+ "min_confidence": {+ "choice:2": 0.9,+ "choice:3-5": 0.1+ },+ "before": {+ "two": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ },+ "five": {+ "type": "choice",+ "probabilities": {+ "0": 0.2,+ "1": 0.2,+ "2": 0.2,+ "3": 0.2,+ "4": 0.2+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ }+ },+ "after": {+ "two": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5,+ "low_confidence": true,+ "abstention": "abstained",+ "abstention_threshold": 0.9+ },+ "five": {+ "type": "choice",+ "probabilities": {+ "0": 0.2,+ "1": 0.2,+ "2": 0.2,+ "3": 0.2,+ "4": 0.2+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5,+ "abstention": "passed",+ "abstention_threshold": 0.1+ }+ }+ },+ {+ "case": "map-default-for-unnamed-bucket",+ "min_confidence": {+ "choice:2": 0.9,+ "default": 0.95+ },+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.083333,+ "1": 0.083333,+ "2": 0.083333,+ "3": 0.083333,+ "4": 0.083333,+ "5": 0.083333,+ "6": 0.083333,+ "7": 0.083333,+ "8": 0.083333,+ "9": 0.083333,+ "10": 0.083333,+ "11": 0.083333+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.083333,+ "1": 0.083333,+ "2": 0.083333,+ "3": 0.083333,+ "4": 0.083333,+ "5": 0.083333,+ "6": 0.083333,+ "7": 0.083333,+ "8": 0.083333,+ "9": 0.083333,+ "10": 0.083333,+ "11": 0.083333+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5,+ "low_confidence": true,+ "abstention": "abstained",+ "abstention_threshold": 0.95+ }+ }+ },+ {+ "case": "map-unnamed-bucket-no-default",+ "min_confidence": {+ "choice:2": 0.9+ },+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.083333,+ "1": 0.083333,+ "2": 0.083333,+ "3": 0.083333,+ "4": 0.083333,+ "5": 0.083333,+ "6": 0.083333,+ "7": 0.083333,+ "8": 0.083333,+ "9": 0.083333,+ "10": 0.083333,+ "11": 0.083333+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.083333,+ "1": 0.083333,+ "2": 0.083333,+ "3": 0.083333,+ "4": 0.083333,+ "5": 0.083333,+ "6": 0.083333,+ "7": 0.083333,+ "8": 0.083333,+ "9": 0.083333,+ "10": 0.083333,+ "11": 0.083333+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.7,+ "confidence": 0.5,+ "abstention": "passed",+ "abstention_threshold": 0.0+ }+ }+ },+ {+ "case": "map-noul-bucket",+ "min_confidence": {+ "noul:2": 0.8+ },+ "before": {+ "q": {+ "type": "noul",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "noul": 0.6,+ "answer_confidence": 0.6,+ "confidence": 0.6+ }+ },+ "after": {+ "q": {+ "type": "noul",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "noul": 0.6,+ "answer_confidence": 0.6,+ "confidence": 0.6,+ "low_confidence": true,+ "abstention": "abstained",+ "abstention_threshold": 0.8+ }+ }+ },+ {+ "case": "map-score-bucket",+ "min_confidence": {+ "score:6-10": 0.8+ },+ "before": {+ "q": {+ "type": "score",+ "probabilities": {+ "0": 0.142857,+ "1": 0.142857,+ "2": 0.142857,+ "3": 0.142857,+ "4": 0.142857,+ "5": 0.142857,+ "6": 0.142857+ },+ "action": {+ "act_probability": 0.5+ },+ "score": 1.0,+ "legend": {+ "0": "level 0",+ "1": "level 1",+ "2": "level 2",+ "3": "level 3",+ "4": "level 4",+ "5": "level 5",+ "6": "level 6"+ },+ "answer_confidence": 0.6,+ "confidence": 0.6+ }+ },+ "after": {+ "q": {+ "type": "score",+ "probabilities": {+ "0": 0.142857,+ "1": 0.142857,+ "2": 0.142857,+ "3": 0.142857,+ "4": 0.142857,+ "5": 0.142857,+ "6": 0.142857+ },+ "action": {+ "act_probability": 0.5+ },+ "score": 1.0,+ "legend": {+ "0": "level 0",+ "1": "level 1",+ "2": "level 2",+ "3": "level 3",+ "4": "level 4",+ "5": "level 5",+ "6": "level 6"+ },+ "answer_confidence": 0.6,+ "confidence": 0.6,+ "low_confidence": true,+ "abstention": "abstained",+ "abstention_threshold": 0.8+ }+ }+ },+ {+ "case": "reflagging-clears-a-stale-flag",+ "min_confidence": 0.5,+ "before": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.5,+ "low_confidence": true+ }+ },+ "after": {+ "q": {+ "type": "choice",+ "probabilities": {+ "0": 0.5,+ "1": 0.5+ },+ "action": {+ "act_probability": 0.5+ },+ "choice": "0",+ "answer_confidence": 0.9,+ "confidence": 0.5,+ "abstention": "passed",+ "abstention_threshold": 0.5+ }+ }+ }+ ],+ "refusals": [+ {+ "input": "1.5",+ "error": "min_confidence must be a float in [0.0, 1.0], got 1.5"+ },+ {+ "input": "-0.1",+ "error": "min_confidence must be a float in [0.0, 1.0], got -0.1"+ },+ {+ "input": "True",+ "error": "min_confidence must be a float in [0.0, 1.0], got True"+ },+ {+ "input": "'0.5'",+ "error": "min_confidence must be a float in [0.0, 1.0], got '0.5'"+ },+ {+ "input": "nan",+ "error": "min_confidence must be a float in [0.0, 1.0], got nan"+ },+ {+ "input": "{}",+ "error": "a min_confidence map must be a non-empty dict of bucket -> float, got {}"+ },+ {+ "input": "{'choice:2': 2.0}",+ "error": "min_confidence must be a float in [0.0, 1.0], got 2.0"+ },+ {+ "input": "{1: 0.5}",+ "error": "min_confidence map keys must be strings like 'choice:3-5', got 1"+ }+ ]+}diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/ConfidenceGate.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/ConfidenceGate.javanew file mode 100644index 00000000..635657a3--- /dev/null+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/ConfidenceGate.java@@ -0,0 +1,235 @@+package com.convaiinnovations.laya;++import java.util.LinkedHashMap;+import java.util.Map;+import java.util.Optional;+import java.util.OptionalDouble;++/**+ * The opt-in abstention gate: which answers a caller should not act on without review.+ *+ * <p>A gate is a policy, and a policy whose application cannot be observed is not one. So this+ * reports three states rather than a boolean — an answer that carried no usable confidence is+ * {@link Abstention#UNEVALUATED}, which a boolean cannot express, and reporting it as a pass is+ * the same lie as reporting it as a flag.+ *+ * <p><b>This does not mutate.</b> The reference writes {@code abstention},+ * {@code abstention_threshold} and {@code low_confidence} into the answer dicts in place, which+ * a record cannot do. {@link #apply} returns a report keyed the same way as the answers instead,+ * and an UNGATED call returns {@link Optional#empty()} rather than a report full of nulls —+ * because the reference's contract is that an ungated call writes <em>nothing at all</em>, and+ * the presence of the field is what tells a caller the gate ran. A sentinel for the unconfigured+ * case would put that information back into every caller's payload, which is what the contract+ * avoids.+ *+ * <p>The gate reads {@code answerConfidence} — the probability mass on the answer being+ * reported, which is what temperature scaling fits — and falls back to {@code confidence} only+ * when the first is unusable. Those are different quantities on different scales, and+ * {@code confidence} is normalised entropy that does not transfer across option counts, so the+ * order matters and is pinned by the fixtures.+ *+ * <p>A threshold is <em>not</em> a claim that the number is calibrated. "About c of the answers+ * returned at c are correct" holds only after temperatures have been fitted and validated for+ * that checkpoint and question shape.+ */+public final class ConfidenceGate {++ private ConfidenceGate() {+ }++ /** What the gate concluded about one answer. The three states, and only these three. */+ public enum Abstention {+ /** A gate ran and this answer's confidence cleared it. */+ PASSED("passed"),+ /** A gate ran and this answer's confidence fell below it. */+ ABSTAINED("abstained"),+ /** A gate ran and this answer carried no usable confidence, so it could not decide. */+ UNEVALUATED("unevaluated");++ private final String wireName;++ Abstention(String wireName) {+ this.wireName = wireName;+ }++ /** The name the reference writes, which is what a cross-language caller compares. */+ public String wireName() {+ return wireName;+ }+ }++ /**+ * One answer's verdict.+ *+ * @param lowConfidence the reference's {@code low_confidence} flag. Redundant with+ * {@link Abstention#ABSTAINED} and kept because it is what the wire format carries and+ * what an existing caller filters on+ * @param threshold the threshold THIS answer was gated at, which differs per answer+ * under a per-bucket map. Echoed because the flag consumes the threshold and drops it,+ * so without this a batch gated per bucket cannot be re-split+ */+ public record Gated(Answer answer, boolean lowConfidence, Abstention abstention,+ double threshold) {+ }++ /**+ * The calibrated confidence, or empty when the answer did not report a usable one.+ *+ * <p>In the reference this also rejects a bool and a missing field. Neither is representable+ * here — {@link Answer#answerConfidence()} is a primitive {@code double} that every answer+ * has — so NaN and infinity are the only ways to be unusable on this side. The reference's+ * other two paths are unreachable rather than unimplemented.+ */+ public static OptionalDouble answerConfidence(Answer answer) {+ double value = answer.answerConfidence();+ return Double.isFinite(value) ? OptionalDouble.of(value) : OptionalDouble.empty();+ }++ /**+ * The number the gate compares against the threshold, or empty when there is none.+ *+ * <p>{@code answerConfidence} first, falling back to {@code confidence} so an answer+ * carrying only the older field is still gated rather than silently passed. Empty means "no+ * usable number", never an unusable one: a caller that reads the value must be able to tell+ * "nothing to gate on" from "a gate ran on a NaN", and {@link #apply} reports those+ * differently.+ */+ public static OptionalDouble gateConfidence(Answer answer) {+ OptionalDouble calibrated = answerConfidence(answer);+ if (calibrated.isPresent()) {+ return calibrated;+ }+ double fallback = answer.confidence();+ return Double.isFinite(fallback) ? OptionalDouble.of(fallback) : OptionalDouble.empty();+ }++ /**+ * This answer's option-count bucket, in the reference's spelling, or empty when it has none.+ *+ * <p>{@code "choice:2"}, {@code "choice:3-5"}, {@code "score:6-10"}, {@code "noul:2"} and so+ * on. One confidence threshold does not transfer across option counts, which is why a map+ * can gate each bucket at the level its calibration actually earns.+ */+ public static Optional<String> optionBucket(Answer answer) {+ int options;+ if (answer instanceof Answer.Choice choice) {+ options = choice.probabilities().size();+ } else if (answer instanceof Answer.Score score) {+ options = score.probabilities().size();+ } else if (answer instanceof Answer.Noul) {+ // A noul carries no probability map, and the reference falls back to 2 for it rather+ // than refusing to bucket it.+ options = 2;+ } else {+ return Optional.empty();+ }+ if (options == 0) {+ return Optional.empty();+ }+ String size = options <= 2 ? "2" : options <= 5 ? "3-5" : options <= 10 ? "6-10" : "11+";+ return Optional.of(answer.type() + ":" + size);+ }++ /**+ * The threshold this answer's bucket is gated at under a per-bucket map.+ *+ * <p>Falls back to the map's {@code "default"} entry, then to 0.0 — gate nothing — so a+ * bucket the map does not name never abstains by surprise.+ */+ public static double resolve(Answer answer, Map<String, Double> thresholds) {+ Optional<String> bucket = optionBucket(answer);+ if (bucket.isPresent()) {+ Double exact = thresholds.get(bucket.get());+ if (exact != null) {+ return exact;+ }+ }+ return thresholds.getOrDefault("default", 0.0);+ }++ /** Validates a scalar threshold, returning it. */+ public static double checkMinConfidence(double value) {+ if (!Double.isFinite(value) || value < 0.0 || value > 1.0) {+ throw new IllegalArgumentException(String.format(+ "min_confidence must be a float in [0.0, 1.0], got %s", value));+ }+ return value;+ }++ /** Validates a per-bucket threshold map, returning it. */+ public static Map<String, Double> checkMinConfidenceMap(Map<String, ? extends Number> map) {+ if (map == null || map.isEmpty()) {+ throw new IllegalArgumentException(+ "a min_confidence map must be a non-empty map of bucket -> float, got " + map);+ }+ Map<String, Double> out = new LinkedHashMap<>();+ for (Map.Entry<String, ? extends Number> entry : map.entrySet()) {+ if (entry.getKey() == null) {+ throw new IllegalArgumentException(+ "min_confidence map keys must be strings like 'choice:3-5', got null");+ }+ out.put(entry.getKey(), checkMinConfidence(entry.getValue().doubleValue()));+ }+ return Map.copyOf(out);+ }++ /**+ * Gates every answer at one threshold.+ *+ * @param minConfidence the threshold, or null for an UNGATED call+ * @return the report, or empty when no gate was configured — which is how a caller tells+ * "this run had no gate" from "every answer cleared the gate"+ */+ public static Optional<Map<String, Gated>> apply(Map<String, Answer> answers,+ Double minConfidence) {+ if (minConfidence == null) {+ return Optional.empty();+ }+ double threshold = checkMinConfidence(minConfidence);+ return Optional.of(gate(answers, answer -> threshold, threshold == 0.0));+ }++ /**+ * Gates each answer at its own bucket's threshold.+ *+ * @param minConfidence the map, or null for an UNGATED call+ */+ public static Optional<Map<String, Gated>> apply(Map<String, Answer> answers,+ Map<String, ? extends Number> minConfidence) {+ if (minConfidence == null) {+ return Optional.empty();+ }+ Map<String, Double> thresholds = checkMinConfidenceMap(minConfidence);+ // A map is never the "0.0 clears the flag" case: the reference applies that only to a+ // scalar 0.0, because under a map each answer has its own threshold and one of them+ // being zero says nothing about the others.+ return Optional.of(gate(answers, answer -> resolve(answer, thresholds), false));+ }++ private interface ThresholdFor {+ double of(Answer answer);+ }++ private static Map<String, Gated> gate(Map<String, Answer> answers, ThresholdFor thresholds,+ boolean scalarZero) {+ Map<String, Gated> out = new LinkedHashMap<>();+ for (Map.Entry<String, Answer> entry : answers.entrySet()) {+ Answer answer = entry.getValue();+ double threshold = thresholds.of(answer);+ OptionalDouble confidence = gateConfidence(answer);++ boolean low = confidence.isPresent() && confidence.getAsDouble() < threshold;+ if (scalarZero) {+ // A scalar 0.0 was set, so states ARE reported -- but nothing can fall below it,+ // and the reference clears any stale flag rather than leaving one from an+ // earlier, stricter run.+ low = false;+ }+ Abstention state = low ? Abstention.ABSTAINED+ : confidence.isEmpty() ? Abstention.UNEVALUATED+ : Abstention.PASSED;+ out.put(entry.getKey(), new Gated(answer, low, state, threshold));+ }+ return Map.copyOf(out);+ }+}diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/ConfidenceGateTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/ConfidenceGateTest.javanew file mode 100644index 00000000..e9020f48--- /dev/null+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/ConfidenceGateTest.java@@ -0,0 +1,265 @@+package com.convaiinnovations.laya;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.junit.jupiter.api.DynamicTest.dynamicTest;++import java.util.ArrayList;+import java.util.LinkedHashMap;+import java.util.List;+import java.util.Map;+import java.util.Optional;+import org.junit.jupiter.api.DisplayName;+import org.junit.jupiter.api.DynamicTest;+import org.junit.jupiter.api.Test;+import org.junit.jupiter.api.TestFactory;++/**+ * {@link ConfidenceGate} against {@code fixtures/confidence_gate.json}, recorded from+ * {@code laya.confidence}.+ *+ * <p>The gate's whole contract is what it writes, so the fixture records each answer before and+ * after and this asserts the difference. Three things it pins that a port gets wrong by default:+ *+ * <ul>+ * <li>WHICH NUMBER is read. {@code answerConfidence} first, {@code confidence} only as a+ * fallback — different quantities on different scales, so a port reading the wrong one+ * gates at a number that looks plausible and is not the one the calibration fitted.</li>+ * <li>That an UNGATED call writes NOTHING. The presence of the verdict is how a caller tells+ * "no gate ran" from "everything passed", so a port that reports a fourth "unconfigured"+ * state breaks that distinction for every caller.</li>+ * <li>That a scalar {@code 0.0} IS a configured gate. Nothing can fall below it, so every+ * answer with a usable number reads {@code passed} and any stale flag is cleared.</li>+ * </ul>+ *+ * <p>Two of the reference's "unusable confidence" paths — a missing field and a bool — are+ * unrepresentable here, because {@link Answer#answerConfidence()} is a primitive double. Those+ * cases are asserted to reach {@code UNEVALUATED} through the one route that remains, NaN,+ * rather than being skipped.+ */+final class ConfidenceGateTest {++ @SuppressWarnings("unchecked")+ private static Map<String, Object> family() {+ return (Map<String, Object>) (Map<?, ?>) Fixtures.load("confidence_gate.json");+ }++ /** One recorded answer, as the typed record. {@code "__nan__"} is how the fixture spells NaN. */+ @SuppressWarnings("unchecked")+ private static Answer answerOf(Map<String, Object> recorded) {+ double confidence = number(recorded.get("confidence"));+ double answerConfidence = number(recorded.get("answer_confidence"));+ double act = number(((Map<String, Object>) recorded.get("action")).get("act_probability"));+ Map<String, Double> probabilities = new LinkedHashMap<>();+ ((Map<String, Object>) recorded.get("probabilities"))+ .forEach((k, v) -> probabilities.put(k, number(v)));+ switch ((String) recorded.get("type")) {+ case "noul":+ return new Answer.Noul(number(recorded.get("noul")), confidence,+ answerConfidence, act);+ case "score": {+ Map<String, String> legend = new LinkedHashMap<>();+ ((Map<String, Object>) recorded.get("legend"))+ .forEach((k, v) -> legend.put(k, (String) v));+ return new Answer.Score(number(recorded.get("score")), legend, probabilities,+ confidence, answerConfidence, act);+ }+ default:+ return new Answer.Choice((String) recorded.get("choice"), probabilities,+ confidence, answerConfidence, act);+ }+ }++ /**+ * A recorded number.+ *+ * <p>An ABSENT field becomes NaN, not zero. The reference treats "no usable confidence" and+ * "a confidence of 0.0" as different outcomes — one is {@code unevaluated} and the other is+ * {@code abstained} — and a port that defaulted the absent case to 0.0 would turn every+ * answer the gate could not evaluate into one it rejected.+ */+ private static double number(Object value) {+ if (value == null || "__nan__".equals(value) || value instanceof Boolean) {+ // Absent, NaN and a BOOL all mean "no usable confidence". The reference rejects a+ // bool explicitly, because `isinstance(True, int)` is true in Python and `True`+ // would otherwise gate as 1.0 -- an answer that always clears every threshold. A+ // primitive double cannot hold one, so NaN is the nearest thing Java can express+ // and the outcome it must produce is the same: UNEVALUATED.+ //+ // Absent is NOT zero. "No usable confidence" and "a confidence of 0.0" are+ // different outcomes -- unevaluated against abstained -- and defaulting the first+ // to zero would turn every answer the gate could not read into one it rejected.+ return Double.NaN;+ }+ return ((Number) value).doubleValue();+ }++ @TestFactory+ @DisplayName("every recorded gate outcome matches, including the no-op")+ @SuppressWarnings("unchecked")+ List<DynamicTest> recordedOutcomesMatch() {+ List<DynamicTest> tests = new ArrayList<>();+ int ungated = 0;+ int unevaluated = 0;+ for (Object raw : (List<Object>) family().get("cases")) {+ Map<String, Object> one = (Map<String, Object>) raw;+ Object threshold = one.get("min_confidence");+ if (threshold == null) {+ ungated++;+ }+ Map<String, Object> after = (Map<String, Object>) one.get("after");+ for (Object a : after.values()) {+ if ("unevaluated".equals(((Map<String, Object>) a).get("abstention"))) {+ unevaluated++;+ }+ }+ tests.add(dynamicTest((String) one.get("case"), () -> {+ Map<String, Answer> answers = new LinkedHashMap<>();+ ((Map<String, Object>) one.get("before")).forEach(+ (qid, value) -> answers.put(qid, answerOf((Map<String, Object>) value)));++ Optional<Map<String, ConfidenceGate.Gated>> report;+ if (threshold == null) {+ report = ConfidenceGate.apply(answers, (Double) null);+ } else if (threshold instanceof Map) {+ report = ConfidenceGate.apply(answers,+ (Map<String, ? extends Number>) threshold);+ } else {+ report = ConfidenceGate.apply(answers, ((Number) threshold).doubleValue());+ }++ if (threshold == null) {+ // The whole no-op contract: not an empty report, NO report. The presence of+ // the verdict is what tells a caller the gate ran.+ assertTrue(report.isEmpty(),+ "an ungated call must produce no verdict at all");+ for (Object value : after.values()) {+ Map<String, Object> a = (Map<String, Object>) value;+ assertFalse(a.containsKey("abstention"),+ "the reference wrote nothing, so nothing may be reported");+ assertFalse(a.containsKey("abstention_threshold"), "nor a threshold");+ }+ return;+ }++ Map<String, ConfidenceGate.Gated> got = report.orElseThrow();+ assertEquals(after.keySet(), got.keySet(), "one verdict per answer");+ for (Map.Entry<String, Object> expected : after.entrySet()) {+ Map<String, Object> a = (Map<String, Object>) expected.getValue();+ ConfidenceGate.Gated verdict = got.get(expected.getKey());+ String id = one.get("case") + "/" + expected.getKey();++ assertEquals(a.get("abstention"), verdict.abstention().wireName(),+ id + " abstention state");+ assertEquals(number(a.get("abstention_threshold")), verdict.threshold(),+ 1e-12, id + " echoed threshold");+ // `low_confidence` is ABSENT in the reference when it does not apply, which+ // is false here -- and must not be true merely because the answer abstained+ // for want of a number.+ assertEquals(Boolean.TRUE.equals(a.get("low_confidence")),+ verdict.lowConfidence(), id + " low_confidence flag");+ }+ }));+ }+ final int noop = ungated;+ final int unknown = unevaluated;+ assertTrue(noop >= 2, () -> "the no-op contract needs cases and has " + noop);+ assertTrue(unknown >= 3,+ () -> "UNEVALUATED is the state a boolean cannot express and only " + unknown+ + " case(s) reach it");+ return tests;+ }++ @TestFactory+ @DisplayName("every threshold the reference refuses is refused here")+ @SuppressWarnings("unchecked")+ List<DynamicTest> refusalsMatch() {+ List<DynamicTest> tests = new ArrayList<>();+ for (Object raw : (List<Object>) family().get("refusals")) {+ Map<String, Object> one = (Map<String, Object>) raw;+ String input = (String) one.get("input");+ boolean refused = one.get("error") != null;+ // Only the cases Java can express: a string threshold and a non-string map key are+ // compile errors here, not runtime ones, which is a stronger guarantee than the+ // reference's and is why they are skipped rather than asserted.+ Double scalar = switch (input) {+ case "1.5" -> 1.5;+ case "-0.1" -> -0.1;+ case "nan" -> Double.NaN;+ default -> null;+ };+ if (scalar == null) {+ continue;+ }+ tests.add(dynamicTest("scalar " + input, () -> {+ assertTrue(refused, "the reference refuses " + input);+ assertThrows(IllegalArgumentException.class,+ () -> ConfidenceGate.checkMinConfidence(scalar),+ input + " must be refused");+ }));+ }+ tests.add(dynamicTest("an empty map", () ->+ assertThrows(IllegalArgumentException.class,+ () -> ConfidenceGate.checkMinConfidenceMap(Map.of()),+ "an empty map is not a gate")));+ tests.add(dynamicTest("a map with an out-of-range value", () ->+ assertThrows(IllegalArgumentException.class,+ () -> ConfidenceGate.checkMinConfidenceMap(Map.of("choice:2", 2.0)),+ "a value outside [0,1] is refused wherever it appears")));+ return tests;+ }++ @Test+ @DisplayName("the option bucket is the reference's, including the noul fallback")+ void buckets() {+ assertEquals(Optional.of("choice:2"), ConfidenceGate.optionBucket(+ choice(2)), "two options");+ assertEquals(Optional.of("choice:3-5"), ConfidenceGate.optionBucket(choice(4)));+ assertEquals(Optional.of("choice:3-5"), ConfidenceGate.optionBucket(choice(5)),+ "5 is the top of the 3-5 band, not the bottom of the next");+ assertEquals(Optional.of("choice:6-10"), ConfidenceGate.optionBucket(choice(6)));+ assertEquals(Optional.of("choice:11+"), ConfidenceGate.optionBucket(choice(12)));+ // A noul carries no probability map at all, and the reference buckets it as 2 rather+ // than refusing -- so a per-bucket map can name `noul:2` and have it apply.+ assertEquals(Optional.of("noul:2"),+ ConfidenceGate.optionBucket(new Answer.Noul(0.6, 0.6, 0.6, 0.5)));+ }++ @Test+ @DisplayName("an unnamed bucket with no default gates at zero, so it never abstains")+ void unnamedBucketNeverAbstainsBySurprise() {+ Answer twelve = choice(12);+ assertEquals(0.0, ConfidenceGate.resolve(twelve, Map.of("choice:2", 0.9)), 1e-12,+ "an unconfigured bucket must not inherit another bucket's threshold");+ assertEquals(0.95, ConfidenceGate.resolve(twelve,+ Map.of("choice:2", 0.9, "default", 0.95)), 1e-12, "...but a default applies");+ }++ @Test+ @DisplayName("gateConfidence prefers the calibrated number and falls back to entropy")+ void whichNumberIsRead() {+ Answer both = new Answer.Choice("a", Map.of("a", 0.9, "b", 0.1), 0.1, 0.9, 0.5);+ assertEquals(0.9, ConfidenceGate.gateConfidence(both).orElseThrow(), 1e-12,+ "answerConfidence wins; the two differ here so a port reading the wrong one "+ + "gates at 0.1 instead of 0.9");+ Answer onlyEntropy = new Answer.Choice("a", Map.of("a", 0.9, "b", 0.1),+ 0.3, Double.NaN, 0.5);+ assertEquals(0.3, ConfidenceGate.gateConfidence(onlyEntropy).orElseThrow(), 1e-12,+ "an answer carrying only the older field is still gated, not silently passed");+ Answer neither = new Answer.Choice("a", Map.of("a", 0.9, "b", 0.1),+ Double.NaN, Double.NaN, 0.5);+ assertTrue(ConfidenceGate.gateConfidence(neither).isEmpty(),+ "no usable number is empty, never zero -- zero would read as a real confidence "+ + "that fails every threshold");+ }++ private static Answer choice(int options) {+ Map<String, Double> probabilities = new LinkedHashMap<>();+ for (int i = 0; i < options; i++) {+ probabilities.put(Integer.toString(i), 1.0 / options);+ }+ return new Answer.Choice("0", probabilities, 0.5, 0.7, 0.5);+ }+}diff --git a/laya-java/scripts/gen_fixtures.py b/laya-java/scripts/gen_fixtures.pyindex 676cc668..1b335cb9 100644--- a/laya-java/scripts/gen_fixtures.py+++ b/laya-java/scripts/gen_fixtures.py@@ -2935,6 +2935,140 @@ def predict_long_scan():" batches; this family exists to pin those" % (name, multi, spans, capped))return out+#: (case id, answers, min_confidence). The gate's whole contract is observable from the payload+#: it writes, so each case records the answer dicts before and after.+CONFIDENCE_CASES = [+ # The no-op contract. An ungated call must return EXACTLY the payload it returned before --+ # no `abstention`, no `abstention_threshold`, no flag -- because the PRESENCE of the field is+ # what tells a caller the gate ran.+ ("ungated-writes-nothing", [("q", "choice", 0.7, 0.5, 2)], None),+ ("ungated-with-several", [("a", "choice", 0.9, 0.8, 3), ("b", "noul", 0.4, 0.4, 2)], None),+ # A scalar threshold.+ ("scalar-abstains", [("q", "choice", 0.7, 0.5, 2)], 0.8),+ ("scalar-passes", [("q", "choice", 0.9, 0.5, 2)], 0.8),+ ("scalar-exactly-at-threshold", [("q", "choice", 0.8, 0.5, 2)], 0.8),+ # 0.0 WAS set, so states are reported -- and nothing can fall below it, so every answer with+ # a usable confidence reads `passed`. The threshold echo is what distinguishes that from a+ # real pass at a real threshold.+ ("scalar-zero-reports-passed", [("q", "choice", 0.7, 0.5, 2)], 0.0),+ ("scalar-one-abstains-everything", [("q", "choice", 0.99, 0.9, 2)], 1.0),+ # `answer_confidence` first, `confidence` only as a fallback -- a DIFFERENT quantity on a+ # different scale, so a port that reads the wrong one gates at the wrong number.+ ("falls-back-to-confidence", [("q", "choice", None, 0.3, 2)], 0.5),+ ("prefers-answer-confidence", [("q", "choice", 0.9, 0.1, 2)], 0.5),+ # No usable number is NOT a pass and NOT a flag: the gate ran and could not decide.+ ("no-usable-confidence", [("q", "choice", None, None, 2)], 0.5),+ ("nan-is-not-a-confidence", [("q", "choice", "nan", "nan", 2)], 0.5),+ ("bool-is-not-a-confidence", [("q", "choice", True, None, 2)], 0.5),+ # Per-bucket thresholds: one number does not transfer across option counts.+ ("map-by-bucket", [("two", "choice", 0.7, 0.5, 2), ("five", "choice", 0.7, 0.5, 5)],+ {"choice:2": 0.9, "choice:3-5": 0.1}),+ ("map-default-for-unnamed-bucket", [("q", "choice", 0.7, 0.5, 12)],+ {"choice:2": 0.9, "default": 0.95}),+ ("map-unnamed-bucket-no-default", [("q", "choice", 0.7, 0.5, 12)], {"choice:2": 0.9}),+ ("map-noul-bucket", [("q", "noul", 0.6, 0.6, 2)], {"noul:2": 0.8}),+ ("map-score-bucket", [("q", "score", 0.6, 0.6, 7)], {"score:6-10": 0.8}),+ # A previously-flagged answer that now clears the bar must have the flag REMOVED, not left.+ ("reflagging-clears-a-stale-flag", [("q", "choice", 0.9, 0.5, 2)], 0.5),+]+++def _conf_answer(qtype, answer_confidence, confidence, options):+ """One answer dict in the shape `predict` returns."""+ def number(v):+ return float("nan") if v == "nan" else v+ probabilities = {str(i): round(1.0 / options, 6) for i in range(options)}+ answer = {"type": qtype, "probabilities": probabilities,+ "action": {"act_probability": 0.5}}+ if qtype == "choice":+ answer["choice"] = "0"+ elif qtype == "score":+ answer["score"] = 1.0+ answer["legend"] = {str(i): "level %d" % i for i in range(options)}+ else:+ answer["noul"] = 0.6+ if answer_confidence is not None:+ answer["answer_confidence"] = number(answer_confidence)+ if confidence is not None:+ answer["confidence"] = number(confidence)+ return answer+++def confidence_gate():+ """`laya.confidence`: which number the gate reads, and what it writes.++ The gate is a policy, and a policy whose application cannot be observed is not one. Its whole+ contract is the payload it writes, so each case records the answers BEFORE and AFTER, and the+ cases are chosen so a port that reads the wrong field or writes the wrong state differs+ somewhere:++ * `answer_confidence` is the quantity temperature scaling fits; `confidence` is normalized+ entropy, a different quantity on a different scale. The gate reads the first and falls+ back to the second, so `prefers-answer-confidence` and `falls-back-to-confidence` differ+ only in which field is present and must give different thresholds' worth of behaviour.+ * an UNSET threshold writes nothing at all -- the presence of `abstention` is how a caller+ tells "this run had no gate" from "this answer cleared the gate", so a port that writes+ a sentinel for the unconfigured case breaks that distinction for every caller.+ * `0.0` IS a threshold that was set. Nothing can fall below it, so every answer with a+ usable number reads `passed`, and the echoed threshold is what distinguishes that from a+ real pass at a real bar.+ * a bool and a NaN are not confidences. Both give "no usable number", which the gate+ reports as `unevaluated` -- not as a pass, which is the same lie as reporting it as a flag.+ """+ import copy+ import math++ from laya.confidence import GATE_STATES, apply_confidence_gate, check_min_confidence++ cases = []+ for cid, answers, threshold in CONFIDENCE_CASES:+ payload = {qid: _conf_answer(t, ac, c, n) for qid, t, ac, c, n in answers}+ if cid == "reflagging-clears-a-stale-flag":+ # A result dict reused from an earlier, stricter run.+ for a in payload.values():+ a["low_confidence"] = True+ before = copy.deepcopy(payload)+ validated = None if threshold is None else check_min_confidence(threshold)+ results = [{"model": "laya-rl-agent-onnx", "answers": payload, "usage": {}}]+ apply_confidence_gate(results, validated)++ def clean(d):+ # NaN does not survive JSON, and recording it as null would lose the distinction+ # between "absent" and "not a number" -- which is exactly what this family tests.+ out = {}+ for k, v in d.items():+ if isinstance(v, float) and math.isnan(v):+ out[k] = "__nan__"+ elif isinstance(v, dict):+ out[k] = clean(v)+ else:+ out[k] = v+ return out++ cases.append({+ "case": cid,+ "min_confidence": threshold,+ "before": {q: clean(a) for q, a in before.items()},+ "after": {q: clean(a) for q, a in results[0]["answers"].items()},+ })++ # Validation is part of the contract: a threshold outside [0,1], a bool, or an empty map is+ # refused, and the message is what a caller sees.+ refusals = []+ for bad in (1.5, -0.1, True, "0.5", float("nan"), {}, {"choice:2": 2.0}, {1: 0.5}):+ try:+ check_min_confidence(bad)+ refusals.append({"input": repr(bad), "error": None})+ except (ValueError, TypeError) as problem:+ refusals.append({"input": repr(bad), "error": str(problem)})++ gated = sum(1 for c in cases if any("abstention" in a for a in c["after"].values()))+ if gated < 12 or len(GATE_STATES) != 3:+ raise AssertionError(+ "only %d case(s) reach a gated answer across %d state(s); this family exists to pin"+ " the three states and the no-op" % (gated, len(GATE_STATES)))+ return {"states": list(GATE_STATES), "cases": cases, "refusals": refusals}+def _decoder_shape(tokenizer_json):"""The decoder section's type chain, flattened, so the fixture says what it was recorded on."""import io@@ -2956,6 +3090,7 @@ def _decoder_shape(tokenizer_json):FAMILIES = {+ "confidence_gate.json": confidence_gate,"predict_long.json": predict_long_scan,"window_plan.json": window_plan,"lang_tables.json": lang_tables,