Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

Merge pull request #725 from aashish254/feat-cli-min-confidence

Nandakishor234be2f69c03
diff --cc README.md
index 134de8ea,98785b20..8bdb6b7e
--- a/README.md
+++ b/README.md
@@@ -848,18 -816,24 +849,36 @@@ else
The threshold reads `answer_confidence` (`max(p)`) — the calibrated quantity, invariant to the number of options — never the entropy `confidence`. With `decide(..., min_confidence=...)` a low-confidence field comes back as `None` in the schema output, while `return_details=True` keeps the answer and its confidence. [LangChain `LayaRouter`](docs/langchain.md)'s `confidence_threshold` reads the same value: `answer_confidence` when the answer carries it, `confidence` otherwise. Left unset, `min_confidence` changes nothing.
+A gate is a policy, and a policy whose application you cannot observe is not one. `low_confidence` is written only when the gate fires, so its absence cannot tell you "a gate ran and this answer cleared it" apart from "no gate ran at all" — you could neither compute an abstention rate nor prove the gate was in effect. **When you pass `min_confidence`,** every answer therefore also reports `abstention`:
+
+| `abstention` | meaning |
+|---|---|
+| `passed` | a gate ran and this answer's `answer_confidence` cleared it |
+| `abstained` | a gate ran and this answer's `answer_confidence` fell below it |
+| `unevaluated` | a gate ran and this answer carried no usable confidence, so the gate could not decide |
+
+`abstention_threshold` echoes the threshold whenever a gate ran, so a log can be re-split by the gate that produced it instead of by whatever the caller happened to remember passing. `unevaluated` is the case a boolean cannot express — reporting it as a pass would be as wrong as reporting it as a flag, so a NaN, a missing confidence or a `bool` lands in its own state. The raw answer, probabilities and confidence are untouched either way.
+
+Left without `min_confidence`, the response is byte-for-byte what it was before any of this: no `abstention`, no `abstention_threshold`, no flag. That is deliberate — a field on every answer that only some calls populate is a schema change for callers who never asked to be gated. The presence of `abstention` is how you tell a gated run from an ungated one; a `min_confidence` of exactly `0.0` still counts as set, so it reports states, and `flag_low_confidence` treats `0.0` as a no-op because nothing can fall below it.
+
+ The same gate is a flag on the `laya` command:
+
+ ```bash
+ laya "Is it the blue one or the green one" --preset triage --min-confidence 0.9
+ ```
+
+ ```text
+ intent : other (p=0.992)
+ frustration : 1.06 [low-confidence]
+ churn_risk : 0.158 [low-confidence]
+ ```
+
+ Each marked answer gets `[low-confidence]` on its printed line and the answer itself is still
+ printed, so a run at a terminal reads the same way the dict does; `--json` carries the raw
-`low_confidence` key. `--batch FILE` applies one threshold to the whole file, which is the shape
++`low_confidence` and `abstention` keys. `--batch FILE` applies one threshold to the whole file, which is the shape
+ `predict_batch` takes. Routing has no answer to gate, so `laya --min-confidence 0.9 "..."` without
+ `--predict`, `--preset` or `--questions` is refused rather than ignored.
+
---
## Prediction Hooks
diff --cc laya/cli.py
index 1d94fe47,89315c37..4b46c774
--- a/laya/cli.py
+++ b/laya/cli.py
@@@ -221,8 -262,7 +267,8 @@@ def run(text, args, router=None)
state = {state_key: text}
result = router.predict(state, questions,
model=args.model, task=args.task, lang=args.lang,
+ lang_guess=args.lang_guess,
- **budget_overrides(args))
+ **budget_overrides(args), **abstention_override(args))
if args.json:
print(json.dumps(result, ensure_ascii=False, indent=2, default=str))
else: