Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(confidence): report the abstention gate's state, only where a gate ran
Bruce-Yii6bbdae05cbb5
diff --cc README.mdindex 97a7307f,328a6194..9274451c--- a/README.md+++ b/README.md@@@ -97,7 -97,7 +97,7 @@@ The shipped checkpoints work zero-shot## What's new in 0.3.21* **ONNX catches up with PyTorch.** `ONNXAgent` gains `predict_batch` (with `sort_by_length`), `predict_long` and `decide_batch`, `scripts/export_onnx.py --quantize` writes a per-channel INT8 copy for CPU, and `laya-evals run --onnx` scores an export with the same gates as the torch path.- * **Opt-in abstention.** `min_confidence=` on `predict`, `predict_batch`, `decide` and `decide_batch` flags answers below a threshold on `answer_confidence` with `low_confidence: True`, and `decide` returns `None` for them. Every answer also reports `abstention` — `not_configured`, `passed`, `abstained` or `unevaluated` — so a caller can tell an ungated run from a gated one that cleared.-* **Opt-in abstention.** `min_confidence=` on `predict`, `predict_batch`, `decide` and `decide_batch` flags answers below a threshold on `answer_confidence` with `low_confidence: True`, and `decide` returns `None` for them.++* **Opt-in abstention.** `min_confidence=` on `predict`, `predict_batch`, `decide` and `decide_batch` flags answers below a threshold on `answer_confidence` with `low_confidence: True`, and `decide` returns `None` for them. When a threshold is set, every answer also reports `abstention` — `passed`, `abstained` or `unevaluated` — so a caller can tell a gate that cleared from a gate that never ran; with no threshold, nothing is added at all.* **Batch everywhere.** `decide_batch`, `Router.predict_long`, `laya --batch FILE`, the MCP `laya_predict_batch` / `laya_route_batch` / `laya_decide` tools, and LangChain `batch()` / `abatch()` all run on shared forward passes. New `LayaDecision` (LangChain), LlamaIndex selectors (`laya[llamaindex]`) and CrewAI routing (`laya[crewai]`).* **Per-request token budget.** `max_len` / `head_max_len` now reach every surface: `laya-serve` (capped by `LAYA_MAX_TOKEN_BUDGET`), `Router.predict_batch` requests, the CLI (`--questions`, `--max-len`, `--head-max-len`), MCP tools and LangChain nodes.* **Operations.** `LAYA_MAX_LOADED`, `LAYA_REVISION` and per-checkpoint SHA-256 maps; `/health` reports the device a checkpoint really runs on and its CPU-fallback count; the 503 busy answer carries `Retry-After`; `compile=True` no longer recompiles for every request shape.@@@ -719,17 -815,6 +815,18 @@@ elseThe threshold reads `answer_confidence` (`max(p)`) — the calibrated quantity, invariant to the number of options — never the entropy `confidence`. With `decide(..., min_confidence=...)` a low-confidence field comes back as `None` in the schema output, while `return_details=True` keeps the answer and its confidence. [LangChain `LayaRouter`](docs/langchain.md)'s `confidence_threshold` reads the same value: `answer_confidence` when the answer carries it, `confidence` otherwise. Left unset, `min_confidence` changes nothing.- A gate is a policy, and a policy whose application you cannot observe is not one. `low_confidence` is written only when the gate fires, so its absence cannot tell you "you passed no `min_confidence`" apart from "a gate ran and this answer cleared it" — you could neither compute an abstention rate nor prove the gate was in effect. Every answer therefore also reports `abstention`, one of four values, present whether or not a gate was configured:++A gate is a policy, and a policy whose application you cannot observe is not one. `low_confidence` is written only when the gate fires, so its absence cannot tell you "a gate ran and this answer cleared it" apart from "no gate ran at all" — you could neither compute an abstention rate nor prove the gate was in effect. **When you pass `min_confidence`,** every answer therefore also reports `abstention`:++| `abstention` | meaning |+|---|---|- | `not_configured` | no `min_confidence` was passed, so there is no gate |+| `passed` | a gate ran and this answer's `answer_confidence` cleared it |+| `abstained` | a gate ran and this answer's `answer_confidence` fell below it |+| `unevaluated` | a gate ran and this answer carried no usable confidence, so the gate could not decide |++`abstention_threshold` echoes the threshold whenever a gate ran, so a log can be re-split by the gate that produced it instead of by whatever the caller happened to remember passing. `unevaluated` is the case a boolean cannot express — reporting it as a pass would be as wrong as reporting it as a flag, so a NaN, a missing confidence or a `bool` lands in its own state. The raw answer, probabilities and confidence are untouched either way.+++Left without `min_confidence`, the response is byte-for-byte what it was before any of this: no `abstention`, no `abstention_threshold`, no flag. That is deliberate — a field on every answer that only some calls populate is a schema change for callers who never asked to be gated. The presence of `abstention` is how you tell a gated run from an ungated one; a `min_confidence` of exactly `0.0` still counts as set, so it reports states, and `flag_low_confidence` treats `0.0` as a no-op because nothing can fall below it.++---## Prediction Hooksdiff --cc docs/reference/helpers.mdindex 56751064,7f53485e..eeda50a0--- a/docs/reference/helpers.md+++ b/docs/reference/helpers.md@@@ -44,12 -44,10 +44,14 @@@::: laya.confidence.flag_low_confidence+::: laya.confidence.apply_confidence_gate++::: laya.confidence.GATE_STATES+## Calibration and training+ ::: laya.common.answer_confidence+::: laya.common.confidence_from_probs::: laya.common.ece_scorediff --cc laya/agent.pyindex c7d914bc,c98e6859..866fae43--- a/laya/agent.py+++ b/laya/agent.py@@@ -28,11 -35,14 +35,14 @@@ from .common import_reuse_question_tokens,_disable_question_token_reuse,encode_text,+ render_criterion,render_options,+ resolve_lang_temperatures,serialize_state,temp_bucket,+ unpermute_probs,)-from .confidence import check_min_confidence, flag_low_confidence+from .confidence import apply_confidence_gate, check_min_confidencefrom .hooks import (HookRegistry, PredictContext, _as_sequence, aggregate_usage, compose_hooks, dispatch,normalise_hooks, validate_timeout,@@@ -1273,14 -1488,16 +1487,20 @@@ class Agent(HookRegistry)usage: Dict[str, Any] = {}for r in results:for key, val in r["usage"].items():- usage[key] = (usage.get(key, 0) + val) if isinstance(val, (int, float)) else val+ prev = usage.get(key)+ if isinstance(val, (int, float)):+ usage[key] = (prev if isinstance(prev, (int, float)) else 0) + val+ elif isinstance(val, dict) and isinstance(prev, dict):+ usage[key] = {**prev, **val}+ else:+ usage[key] = valusage["output_tokens"] = 0usage["windows"] = len(results)- return {"model": "laya-rl-agent", "answers": answers, "usage": usage}+ result = {"model": "laya-rl-agent", "answers": answers, "usage": usage}+ # `predict_long` accepts no `min_confidence` -- the window loop has no single confidence to+ # gate on -- so every answer reports that it ran ungated rather than reporting nothing.+ apply_confidence_gate([result], None)+ return result@torch.no_grad()def system_one(self, state: Union[str, dict, list], questions: Dict[str, Dict[str, Any]], lang: Optional[str] = None,diff --cc laya/confidence.pyindex 495d2037,5a20657a..43b67ee5--- a/laya/confidence.py+++ b/laya/confidence.py@@@ -7,23 -7,46 +7,54 @@@ import matfrom typing import Any, Dict, List, Optional+ def answer_confidence_value(answer: Dict[str, Any]) -> Optional[float]:+ """The `answer_confidence` of `answer`, or None if it did not report a usable one.++ `answer_confidence` is `max(p)` -- the probability mass on the answer being reported. It is+ the quantity temperature scaling fits, the quantity every calibration figure in this+ repository is computed on, and the quantity the `min_confidence` gate is defined against+ (#361). That makes it the right number for a threshold a caller compares against; it is+ not on its own a claim that the number is right. Reading it as "about c of the answers+ returned at c are correct" holds only after temperatures have been fitted and validated on+ held-out data for that checkpoint and question shape -- the shipped checkpoints are+ over-confident as shipped, and `laya-multilingual` ships with no fitted temperatures at all+ (README, Calibration). `common.answer_confidence` states the quantity; this states when it+ carries that property.++ `confidence` is a different quantity on a different scale -- normalized entropy, which+ `common.confidence_from_probs` calls "not calibrated" and #394 says does not transfer across+ option counts. So this deliberately does **not** fall back to it: reporting the entropy+ number under the `answer_confidence` name would be worse than reporting nothing, because the+ name is what a caller filters on.++ A `bool` is not a confidence and a NaN is not a decision, so both give None rather than 0.0.+ """+ conf = answer.get("answer_confidence")+ if isinstance(conf, (int, float)) and not isinstance(conf, bool) and math.isfinite(conf):+ return float(conf)+ return None++def _gate_confidence(answer: Dict[str, Any]) -> Optional[float]:- """The confidence the abstention gate reads for `answer`, or None if there is not a usable one.+ """The number the abstention gate compares against `min_confidence`, or None if there is none.- `answer_confidence` is the calibrated max(p) confidence and is what the gate must read;- `confidence` is the fallback, and for `choice` and `score` it is normalized entropy on a- different scale, which must not be compared against the same threshold. A bool, a NaN, an- infinity or a missing field is not a number the gate can decide on.+ `answer_confidence` first, falling back to the entropy `confidence` so an answer that carries+ only the older field is still gated rather than silently passed.+- Shared by :func:`flag_low_confidence` and :func:`apply_confidence_gate` so the flag and the- reported state cannot disagree about which quantity was read.++ None means *no usable number*, never an unusable one. An earlier version returned whatever++ `confidence` held when the validity test failed, so an answer with no `answer_confidence` and a++ NaN `confidence` returned NaN. `flag_low_confidence` was indifferent to that -- `NaN < x` is++ False and `None is not None` is also False, so it flagged either way -- but a caller that++ *reads* the number could not tell "nothing to gate on" from "a gate ran on a NaN", and++ :func:`apply_confidence_gate` reports those two differently on purpose."""- conf = answer.get("answer_confidence")- if conf is None:- conf = answer.get("confidence")- if isinstance(conf, (int, float)) and not isinstance(conf, bool) and math.isfinite(conf):+ conf = answer_confidence_value(answer)- if conf is None:- conf = answer.get("confidence")- if isinstance(conf, (int, float)) and not isinstance(conf, bool) and math.isfinite(conf):- return float(conf)- return conf++ if conf is not None:+ return conf++ fallback = answer.get("confidence")++ if isinstance(fallback, (int, float)) and not isinstance(fallback, bool) and math.isfinite(fallback):++ return float(fallback)+ return Nonedef check_min_confidence(v: Any) -> float:@@@ -55,60 -79,3 +87,67 @@@ def flag_low_confidence(results: List[Dconf = _gate_confidence(a)if conf is not None and conf < min_confidence:a["low_confidence"] = True++- #: The confidence gate's state, as reported on every answer by :func:`apply_confidence_gate`.- #:- #: `low_confidence` is written only when the gate fires, so its absence cannot tell a caller "no- #: ``min_confidence`` was passed" apart from "a gate ran and this answer cleared it". These four- #: values make the state a thing you read rather than a thing you infer from a missing field.- GATE_NOT_CONFIGURED = "not_configured" # no threshold was passed, so there is no gate to apply- GATE_PASSED = "passed" # a gate ran and this answer's confidence cleared it- GATE_ABSTAINED = "abstained" # a gate ran and this answer's confidence fell below it- GATE_UNEVALUATED = "unevaluated" # a gate ran and this answer carried no usable confidence++#: The states :func:`apply_confidence_gate` reports, and the only ones. There is deliberately no++#: "no gate was configured" member: when `min_confidence` is not set the function writes nothing at++#: all, so a caller tells "this run had no gate" from "this answer cleared the gate" by whether++#: `abstention` is present. A sentinel for the unconfigured case would put that same information++#: back into the payload for every caller, which is the thing the gate is meant to avoid.++GATE_PASSED = "passed" # a gate ran and this answer's confidence cleared it++GATE_ABSTAINED = "abstained" # a gate ran and this answer's confidence fell below it++GATE_UNEVALUATED = "unevaluated" # a gate ran and this answer carried no usable confidence+- GATE_STATES = (GATE_NOT_CONFIGURED, GATE_PASSED, GATE_ABSTAINED, GATE_UNEVALUATED)++GATE_STATES = (GATE_PASSED, GATE_ABSTAINED, GATE_UNEVALUATED)+++def apply_confidence_gate(results: List[Dict[str, Any]], min_confidence: Optional[float] = None) -> None:- """Report the confidence gate's state on every answer, whether or not a gate was configured.++ """Report the confidence gate's state, on the answers a gate was actually applied to.+- A gate is a policy, and a policy whose application cannot be observed is not one. This writes- ``abstention`` -- one of :data:`GATE_STATES` -- onto every answer, plus ``abstention_threshold``- whenever a gate actually ran. It is what lets a caller answer three questions it otherwise++ A gate is a policy, and a policy whose application cannot be observed is not one. With++ `min_confidence` set, this writes ``abstention`` -- one of :data:`GATE_STATES` -- onto every++ answer, plus ``abstention_threshold``, so a caller can answer three questions it otherwise+ cannot:+- * what fraction of decisions abstained, without guessing which of "no gate" and "gate- cleared it" produced a missing ``low_confidence``;- * whether the gate was in effect at all on this run;- * what threshold produced these results, which ``flag_low_confidence`` consumes and drops --- without it a batch that used different thresholds per request class cannot be re-split.++ * what fraction of decisions abstained, rather than inferring it from whether++ ``low_confidence`` happened to be set;++ * how many answers the gate could not decide on, which a boolean cannot express at all;++ * what threshold produced these results -- ``flag_low_confidence`` consumes the threshold and++ drops it, so without this a batch run with per-class thresholds cannot be re-split.++ ``GATE_UNEVALUATED`` is the case a boolean cannot express: the gate ran and the answer carried- no usable confidence, so the gate could not decide. Reporting that as a pass is the same lie as- reporting it as a flag.-- The flag itself stays :func:`flag_low_confidence`'s -- this delegates to it rather than- re-implementing the rule, so the boolean and the state cannot drift apart. Call this- unconditionally, once per call, in place of a `if min_confidence is not None:` guard: that- guard is what leaves a path reporting nothing at all.++ no usable confidence, so the gate could not decide. Reporting that as a pass is the same lie++ as reporting it as a flag.++++ **With `min_confidence` unset, this writes nothing.** No ``abstention``, no++ ``abstention_threshold``, no flag. That is the whole contract: an ungated call returns exactly++ the payload it returned before, and the presence of the field -- not a fourth value read out++ of it -- is what tells a caller the gate ran. Call it unconditionally, once per call, in place++ of an `if min_confidence is not None:` guard: that guard is what leaves a path reporting++ nothing at all, which is the state this function exists to distinguish.++++ The flag itself stays :func:`flag_low_confidence`'s -- this delegates rather than++ re-implementing the rule, so the boolean and the reported state cannot drift apart.++++ A `min_confidence` of exactly ``0.0`` *was* set, so states are reported, and++ :func:`flag_low_confidence` treats ``0.0`` as a no-op because nothing can fall below it. Every++ answer carrying a usable confidence therefore reads ``passed``, and the threshold echo is what++ distinguishes that from a real pass at a real threshold.+ """- if min_confidence is not None:- flag_low_confidence(results, min_confidence)++ if min_confidence is None:++ return++ flag_low_confidence(results, min_confidence)+ for res in results:+ answers = res.get("answers") if isinstance(res, dict) else None+ if not isinstance(answers, dict):+ continue+ for a in answers.values():+ if not isinstance(a, dict):+ continue- if min_confidence is None:- a["abstention"] = GATE_NOT_CONFIGURED- continue+ if a.get("low_confidence"):+ a["abstention"] = GATE_ABSTAINED+ elif _gate_confidence(a) is None:+ a["abstention"] = GATE_UNEVALUATED+ else:+ a["abstention"] = GATE_PASSED+ a["abstention_threshold"] = float(min_confidence)diff --cc laya/onnx_agent.pyindex 0095bb12,8a67f065..c5675121--- a/laya/onnx_agent.py+++ b/laya/onnx_agent.py@@@ -511,13 -545,16 +544,19 @@@ class ONNXAgent(HookRegistry)usage: Dict[str, Any] = {}for r in results:for key, val in r["usage"].items():- usage[key] = (usage.get(key, 0) + val) if isinstance(val, (int, float)) else val+ prev = usage.get(key)+ if isinstance(val, (int, float)):+ usage[key] = (prev if isinstance(prev, (int, float)) else 0) + val+ elif isinstance(val, dict) and isinstance(prev, dict):+ usage[key] = {**prev, **val}+ else:+ usage[key] = valusage["output_tokens"] = 0usage["windows"] = len(results)- return {"model": "laya-rl-agent-onnx", "answers": answers, "usage": usage}+ result = {"model": "laya-rl-agent-onnx", "answers": answers, "usage": usage}+ # As in `Agent.predict_long`: no `min_confidence` here, so say so on every answer.+ apply_confidence_gate([result], None)+ return resultdef _infer(self, state: Union[str, dict, list], questions: Dict[str, Dict[str, Any]],max_len: Optional[int] = None, head_max_len: Optional[int] = None,diff --cc laya/structured.pyindex fe0bd34e,9e75dcec..e4cd512a--- a/laya/structured.py+++ b/laya/structured.py@@@ -20,7 -20,7 +20,7 @@@ from collections.abc import Sequence afrom dataclasses import dataclass, fieldfrom typing import Any, Dict, List, Optional, Sequence, Tuple- from .confidence import apply_confidence_gate, check_min_confidence-from .confidence import answer_confidence_value, check_min_confidence, flag_low_confidence++from .confidence import answer_confidence_value, apply_confidence_gate, check_min_confidenceMAX_PROPERTIES = 32MAX_OPTIONS = 32diff --cc tests/test_confidence.pyindex ae86d594,02dc784e..d4c210bf--- a/tests/test_confidence.py+++ b/tests/test_confidence.py@@@ -15,6 -15,6 +15,7 @@@ untouched, so nothing a caller gates oNo weights are loaded: the confidence helpers are pure."""++import copyimport mathimport osimport sys@@@ -145,170 -145,6 +146,183 @@@ check_true("flag/q_exact unflagged", "lcheck("flag/answer_confidence intact", ans["q_low"]["answer_confidence"], 0.45)check("flag/raw choice intact", ans["q_high"]["choice"], "a")+# --------------------------------------------------------------- the gate's state is reported+# `low_confidence` is written only when the gate fires, so its absence cannot tell a caller- # "no min_confidence was passed" apart from "a gate ran and this answer cleared it". An operator- # with 10,000 logged decisions cannot compute an abstention rate, cannot prove the gate was in- # effect, and cannot re-split a batch that used different thresholds per request class -- the- # threshold is consumed and dropped. `docs/staged-adoption.md` asks a shadow record to carry- # "errors and any fallback or review decision"; there was no field to put that in.++# "a gate ran and this answer cleared it" from "no gate ran at all". An operator with 10,000++# logged decisions cannot compute an abstention rate, cannot tell whether a run was gated, and++# cannot re-split a batch that used different thresholds per request class -- the threshold is++# consumed and dropped. `docs/staged-adoption.md` asks a shadow record to carry "errors and any++# fallback or review decision"; there was no field to put that in.+#- # The contract: the gate's state is a value on every answer, so it is never inferred from a- # missing field.++# The contract: where a gate ran, its state is an explicit value, so it is never inferred from a++# missing field. Where no gate was configured, nothing is written at all -- an ungated call returns++# the payload it always returned, rather than growing a field every existing caller would then have++# to read. The two cases are told apart by the presence of `abstention`, which is why the vocabulary++# has no "not configured" member to read out of the field.+from laya.confidence import ( # noqa: E402+ GATE_ABSTAINED,- GATE_NOT_CONFIGURED,+ GATE_PASSED,+ GATE_STATES,+ GATE_UNEVALUATED,+ apply_confidence_gate,+)++check("gate/the vocabulary is closed",- list(GATE_STATES), ["not_configured", "passed", "abstained", "unevaluated"])++ list(GATE_STATES), ["passed", "abstained", "unevaluated"])++fresh = lambda: [{ # noqa: E731+ "answers": {+ "q_high": {"type": "choice", "choice": "a", "answer_confidence": 0.92},+ "q_low": {"type": "choice", "choice": "b", "answer_confidence": 0.45},+ }+}]+- # No gate: every answer says so, and nothing is flagged.++# No gate: the payload is untouched. Not a sentinel, not a flag -- byte-for-byte what the caller++# got before this helper existed, so no existing caller has to learn a field to keep working.+off = fresh()++before = copy.deepcopy(off)+apply_confidence_gate(off, None)++check("gate/an ungated call writes nothing at all", off, before)+for qid, a in off[0]["answers"].items():- check("gate/not_configured %s" % qid, a["abstention"], GATE_NOT_CONFIGURED)- check_true("gate/not_configured carries no threshold %s" % qid, "abstention_threshold" not in a)- check_true("gate/not_configured is not a flag %s" % qid, "low_confidence" not in a)++ check_true("gate/ungated %s reports no state" % qid, "abstention" not in a)++ check_true("gate/ungated %s carries no threshold" % qid, "abstention_threshold" not in a)++ check_true("gate/ungated %s is not a flag" % qid, "low_confidence" not in a)++# A gate that ran: the answer that cleared it says `passed`, which is the state the boolean+# could not express. This is the whole point -- before, this answer was byte-identical to the+# ungated one above.+on = fresh()+apply_confidence_gate(on, 0.80)+check("gate/cleared answer says passed", on[0]["answers"]["q_high"]["abstention"], GATE_PASSED)+check("gate/cleared answer is still unflagged", "low_confidence" in on[0]["answers"]["q_high"], False)+check("gate/cleared answer echoes the threshold", on[0]["answers"]["q_high"]["abstention_threshold"], 0.80)+check("gate/below-threshold answer says abstained", on[0]["answers"]["q_low"]["abstention"], GATE_ABSTAINED)+check("gate/below-threshold answer is flagged", on[0]["answers"]["q_low"]["low_confidence"], True)+check("gate/below-threshold echoes the threshold", on[0]["answers"]["q_low"]["abstention_threshold"], 0.80)+check_true("gate/the raw answer is untouched",+ on[0]["answers"]["q_high"]["choice"] == "a" and on[0]["answers"]["q_low"]["choice"] == "b")++# The two states a caller used to be unable to tell apart are now told apart by a value, and the+# distinction survives a round trip through JSON, which is how a shadow record reaches a log.+import json as _json # noqa: E402++round_tripped = _json.loads(_json.dumps(on[0]["answers"]))+check("gate/survives a JSON round trip", round_tripped["q_high"]["abstention"], GATE_PASSED)++# One rule, one implementation: the flag stays `flag_low_confidence`'s, not a second copy of it.+# `apply_confidence_gate(results, 0.0)` is a gate that ran and nothing can fail, which is why+# `flag_low_confidence`'s 0.0 no-op above is still correct and still asserted.+zero = fresh()+apply_confidence_gate(zero, 0.0)+check("gate/0.0 reports passed", zero[0]["answers"]["q_low"]["abstention"], GATE_PASSED)+check_true("gate/0.0 still flags nothing", "low_confidence" not in zero[0]["answers"]["q_low"])++# A non-finite or missing confidence is skipped by `flag_low_confidence`, so it is not an+# abstention. It is also not a pass: the gate ran and could not decide, which is the state Argo+# Rollouts ships as `Inconclusive` and the one collapsing it into success would be the same lie+# `low_confidence`'s absence already tells.+odd = [{"answers": {+ "q_nan": {"type": "choice", "answer_confidence": float("nan")},+ "q_none": {"type": "choice"},+ "q_inf": {"type": "choice", "answer_confidence": float("inf")},+ "q_bool": {"type": "choice", "answer_confidence": True},++ # The fallback path, which is where the number used to leak. With no usable++ # `answer_confidence` the gate reads the entropy `confidence`, and an entropy NaN has to come++ # back as "nothing to gate on" -- not as the NaN itself, which is not None and would++ # therefore read as a pass. `flag_low_confidence` never noticed (NaN < x is False, and++ # None is not None is also False), so nothing else pinned this down.++ "q_nan_entropy": {"type": "choice", "confidence": float("nan")},++ "q_bool_entropy": {"type": "choice", "confidence": True},+ "q_fine": {"type": "choice", "answer_confidence": 0.99},+}}]+apply_confidence_gate(odd, 0.80)+for qid, a in odd[0]["answers"].items():+ if qid == "q_fine":+ check("gate/a usable confidence is evaluated %s" % qid, a["abstention"], GATE_PASSED)+ else:+ check("gate/unusable confidence is unevaluated %s" % qid, a["abstention"], GATE_UNEVALUATED)+ check_true("gate/unusable confidence is not flagged %s" % qid, "low_confidence" not in a)++# `answer_confidence` is the quantity the gate reads, so a `noul`-style entropy `confidence`+# below threshold does not abstain while `answer_confidence` above it does. The state follows+# the same field the flag follows.+pref = [{"answers": {+ "q_a": {"type": "noul", "answer_confidence": 0.91, "confidence": 0.10},+ "q_b": {"type": "choice", "answer_confidence": 0.10, "confidence": 0.99},+}}]+apply_confidence_gate(pref, 0.80)+check("gate/gate reads answer_confidence (a)", pref[0]["answers"]["q_a"]["abstention"], GATE_PASSED)+check("gate/gate reads answer_confidence (b)", pref[0]["answers"]["q_b"]["abstention"], GATE_ABSTAINED)++# The guards the six call sites rely on: a result that is not a dict, answers that are not a+# dict, and answers that are not dicts. Every call site passes a list it has already partly+apply_confidence_gate([None, {"answers": None}, {"answers": [1, 2]}], 0.5)+apply_confidence_gate([], None)+PASS.append("gate/non-dict results are skipped without raising")++# --------------------------------------------------------------- the operator's three questions+# End to end through a real call site, weight-free. `decide` forwards to any runner with a+# `predict`, so this exercises `laya/structured.py`'s own gate call site rather than the helper+# in isolation -- the six call sites are where the state used to be skipped entirely.+from laya.structured import decide # noqa: E402++_GATE_Q = {"type": "choice", "instructions": "Pick one.",+ "criteria": {"a": "alpha", "b": "beta"}}+CONFIDENCES = {"high": 0.95, "low": 0.40}+QUESTIONS = {qid: dict(_GATE_Q) for qid in CONFIDENCES}+++class _StubRunner:+ def __init__(self, conf):+ self.conf = conf++ def predict(self, state, questions, **kwargs):+ return {"model": "stub", "answers": {+ qid: {"type": "choice", "choice": "a", "answer_confidence": self.conf[qid]}+ for qid in questions}}+++stub = _StubRunner(CONFIDENCES)++# 1. Was the gate in effect at all? Before, both answers were byte-identical to the gated pair- # below, so the answer was "no way to tell".++# below, so the answer was "no way to tell". Now the ungated run simply has no `abstention`++# key, and the gated one does -- which is how a caller tells them apart.+ungated = decide(stub, "state", questions=QUESTIONS)+for qid in CONFIDENCES:- check("workflow/ungated %s says so" % qid, ungated[qid]["abstention"], GATE_NOT_CONFIGURED)++ check_true("workflow/ungated %s reports no state" % qid, "abstention" not in ungated[qid])++gated = decide(stub, "state", questions=QUESTIONS, min_confidence=0.80)+check("workflow/the cleared answer reports passed", gated["high"]["abstention"], GATE_PASSED)+check("workflow/the low answer reports abstained", gated["low"]["abstention"], GATE_ABSTAINED)++# 2. What fraction abstained, over whatever decisions the operator kept? Computable from the+# artifact, with no re-run and no out-of-band record of which threshold was used.+log = [gated, decide(stub, "other", questions=QUESTIONS, min_confidence=0.80)]+answers = [a for d in log for a in d.values()]+abstained = sum(a["abstention"] == GATE_ABSTAINED for a in answers)+check("workflow/abstention rate over the log", abstained, 2)+check("workflow/…out of", len(answers), 4)+check_true("workflow/…and no answer is left unlabelled", all("abstention" in a for a in answers))++# 3. What threshold produced these? `flag_low_confidence` consumes it and drops it, so before+# this a batch that gated different request classes differently could not be re-split.+check("workflow/the threshold is recoverable", gated["low"]["abstention_threshold"], 0.80)+check_true("workflow/…and a per-class mix is re-splittable",+ {a["abstention_threshold"] for a in gated.values()} == {0.80})++# The schema projection is unchanged -- an abstained field is still `null` -- so the flat+# `values` mapping still cannot say why. `return_details=True` is where the state is readable,+# and it was unreadable before.+SCHEMA = {"type": "object",+ "properties": {"high": {"type": "string", "enum": ["a", "b"]},+ "low": {"type": "string", "enum": ["a", "b"]}}}+projected = decide(stub, "state", schema=SCHEMA, min_confidence=0.80)+check("workflow/the schema projection is unchanged", projected["low"], None)+check("workflow/…and still reports the cleared field", projected["high"], "a")+detailed = decide(stub, "state", schema=SCHEMA, min_confidence=0.80, return_details=True)+check("workflow/return_details carries the state",+ detailed.answers["low"]["abstention"], GATE_ABSTAINED)+# --------------------------------------------------------------- exportedimport laya # noqa: E402@@@ -321,21 -157,6 +335,20 @@@ check_true("export/check_min_confidenccheck_true("export/flag_low_confidence is importable from laya", hasattr(laya, "flag_low_confidence"))check_true("export/flag_low_confidence is in __all__", "flag_low_confidence" in laya.__all__)check_true("export/flag_low_confidence is in dir()", "flag_low_confidence" in dir(laya))+check_true("export/apply_confidence_gate is importable from laya", hasattr(laya, "apply_confidence_gate"))+check_true("export/apply_confidence_gate is in __all__", "apply_confidence_gate" in laya.__all__)+check_true("export/apply_confidence_gate is in dir()", "apply_confidence_gate" in dir(laya))+check_true("export/GATE_STATES is importable from laya", hasattr(laya, "GATE_STATES"))+check_true("export/GATE_STATES is in __all__", "GATE_STATES" in laya.__all__)+check_true("export/GATE_STATES is in dir()", "GATE_STATES" in dir(laya))- # The four values are reached through GATE_STATES rather than exported one by one: iterating the++# The three values are reached through GATE_STATES rather than exported one by one: iterating the+# vocabulary is the whole use, and every `laya.__all__` name has to earn a documented entry+# (`tests/test_packaging.py` enforces that).+import laya.confidence as _confidence # noqa: E402++check("export/GATE_STATES is the vocabulary a caller iterates", list(GATE_STATES),- [_confidence.GATE_NOT_CONFIGURED, _confidence.GATE_PASSED,- _confidence.GATE_ABSTAINED, _confidence.GATE_UNEVALUATED])++ [_confidence.GATE_PASSED, _confidence.GATE_ABSTAINED, _confidence.GATE_UNEVALUATED])+check("export/laya re-exports the same tuple", laya.GATE_STATES, _confidence.GATE_STATES)print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))for f in FAIL:diff --cc tests/test_predict_long.pyindex d9924639,19397de2..5666c4bb--- a/tests/test_predict_long.py+++ b/tests/test_predict_long.py@@@ -206,11 -257,7 +257,11 @@@ stub = _LongStub(out = _router(stub).predict_long({"body": "y" * 300}, Q, model="english",window=64, stride=32, batch_size=8, lang="de")check("router/routing key attached", out["routing"]["model"], "english")-check("router/answers come from the scan", out["answers"], {"scanned": {"noul": 0.5}})+# `predict_long` takes no `min_confidence` -- a window scan has no single confidence to gate on --- # so every scanned answer says it ran ungated instead of saying nothing. That is the point: an- # operator reading a long-document result can now see there was no gate on it.++# so the gate writes nothing and the scan's answers come back exactly as produced. A long-document++# result is not a gated decision, and now it does not claim to be one.+check("router/answers come from the scan", out["answers"],- {"scanned": {"noul": 0.5, "abstention": "not_configured"}})++ {"scanned": {"noul": 0.5}})check("router/window forwarded", stub.calls[0]["window"], 64)check("router/stride forwarded", stub.calls[0]["stride"], 32)check("router/batch_size forwarded", stub.calls[0]["batch_size"], 8)@@@ -259,8 -306,7 +310,8 @@@ class _Recorderrecorder_stub = _LongStub()_router(recorder_stub).predict_long({"body": "y" * 300}, Q, model="english",on_predict_end=_Recorder().on_predict_end)-check("router/per-call end hook sees the scan", ended["answers"], {"scanned": {"noul": 0.5}})+check("router/per-call end hook sees the scan", ended["answers"],- {"scanned": {"noul": 0.5, "abstention": "not_configured"}})++ {"scanned": {"noul": 0.5}})# 8. an agent with no predict_long is a named caller error, not a bare AttributeErrorclass _NoScan:diff --cc tests/test_structured.pyindex 5a80d331,e570a77e..ccda8890--- a/tests/test_structured.py+++ b/tests/test_structured.py@@@ -163,10 -172,8 +172,10 @@@ check("decide/details routing", detailsrunner = FakeRunner({"a": {"type": "noul", "noul": 0.9, "confidence": 0.9}})out = decide(runner, "s", questions={"a": {"type": "noul", "instructions": "?"}})- # The pass-through is the raw answer, and it now carries the gate's state: no `min_confidence`- # was passed, so every answer reports `not_configured` rather than reporting nothing.++# The pass-through is the raw answer, and no `min_confidence` was passed, so it comes back exactly++# as the runner produced it -- the gate adds nothing to a call it was not asked to gate.check("decide/questions pass-through returns answers", out,- {"a": {"type": "noul", "noul": 0.9, "confidence": 0.9, "abstention": "not_configured"}})+ {"a": {"type": "noul", "noul": 0.9, "confidence": 0.9}})check_raises("decide/requires schema or questions", ValueError, lambda: decide(runner, "s"))check_raises("decide/rejects both", ValueError, lambda: decide(runner, "s", schema=SCHEMA, questions={}))@@@ -282,8 -348,7 +350,8 @@@ check("batch/details routing", details[# questions= is the raw-answers pass-through, exactly like decide()raw = decide_batch(FakeBatchRunner({"a": {"x": {"type": "noul", "noul": 0.9, "confidence": 0.9}}}),["a"], questions={"x": {"type": "noul", "instructions": "?"}})-check("batch/questions pass-through", raw, [{"x": {"type": "noul", "noul": 0.9, "confidence": 0.9}}])+check("batch/questions pass-through", raw,- [{"x": {"type": "noul", "noul": 0.9, "confidence": 0.9, "abstention": "not_configured"}}])++ [{"x": {"type": "noul", "noul": 0.9, "confidence": 0.9}}])class FakeRouterLike: