Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(api): clarify independent probability rounding
monsterlag1478a56f1ec1
diff --git a/docs/http-api.md b/docs/http-api.mdindex 6e3a22f2..c9d805b8 100644--- a/docs/http-api.md+++ b/docs/http-api.md@@ -190,6 +190,11 @@ The sample is one answer this server gave, verbatim: the request above, the cachcheckpoint on CPU. `answers` and `usage` are the keys Jev clients decode; `model` is the constantname of the decision head, and the checkpoint that answered is in `routing`.+`probabilities` on `choice` and `score` answers are rounded independently to four decimal places+and need not sum exactly to 1, just as in the [in-process API](questions-and-answers.md#probability-precision).+For strict unit-sum validation, allow a rounding tolerance that accounts for the number of options,+or normalize a copy of the distribution.+| answer type | keys ||---|---|| `choice` | `choice` (the argmax option), `probabilities` per option, `x_jev_confidence` |@@ -312,7 +317,7 @@ The answer keeps the Jev shape, so the official SDK parses it:- `probabilities` names every option, in criteria order. The finalists carry the final round'sprobabilities and an option eliminated in an earlier round carries `0.0`, so `choice` is still- the argmax and the values still sum to 1;+ the argmax, but the rounded values need not sum exactly to 1;- `x_tournament` lists the `finalists` and the number of `rounds`, so a caller can tell aneliminated option from one the final round scored. Under `LAYA_JEV_STRICT` it is not sent;- `confidence` and `answer_confidence` are the final round's, over the finalists;diff --git a/docs/questions-and-answers.md b/docs/questions-and-answers.mdindex 90d0ce39..c3ccd583 100644--- a/docs/questions-and-answers.md+++ b/docs/questions-and-answers.md@@ -112,6 +112,14 @@ disagreement with the model. If you want the label, take the argmax of `probabil`answer_confidence`'s counterpart from the legend — do not round `score` and assume it is thelabel. `legend` exists so you never have to guess which index means what.+## Probability precision++On `choice` and `score` answers, `probabilities` are rounded independently to four decimal places.+They need not sum exactly to 1: three equally likely options return `0.3333` each, totalling+`0.9999`. The returned values are not renormalized after rounding. If your consumer requires a+unit sum, allow a rounding tolerance that accounts for the number of options, or normalize a copy+of the distribution before validating it.+## Reading confidenceEvery answer carries two confidence numbers, and they measure different things.diff --git a/tests/test_confidence.py b/tests/test_confidence.pyindex 949db982..c38730b5 100644--- a/tests/test_confidence.py+++ b/tests/test_confidence.py@@ -2566,6 +2566,38 @@ for _qid in ("q", "r", "s"):check_true("jev_public/served noul carries no field","x_jev_confidence" not in _served["n"], _served["n"].keys())+# --- probability precision: both decoders round independently, not to a unit sum (#555) ------+from laya.agent import Agent # noqa: E402+from laya.onnx_agent import ONNXAgent # noqa: E402++for _cls in (Agent, ONNXAgent):+ _agent = _cls.__new__(_cls) # no checkpoint, model or ONNX session+ _agent.temperature = [1.0, 1.0, 1.0]+ _agent.temperature_by_options = {}+ for _k, _value, _total in ((3, 0.3333, 0.9999), (6, 0.1667, 1.0002)):+ for _qtype in ("choice", "score"):+ _labels = ["o%d" % i for i in range(_k)]+ _crit = dict.fromkeys(_labels) if _qtype == "choice" else _labels+ _answer = _agent._decode_answers(+ np.zeros((1, _k)), np.array([[1.0, 0.0]]), [{"markers": list(range(_k))}],+ ["q"], {"q": {"t": _qtype, "crit": _crit}}, 0)["q"]+ _probs = list(_answer["probabilities"].values())+ _name = "rounding/%s/%s/k=%d" % (_cls.__name__, _qtype, _k)+ check(_name + "/four decimal places", _probs, [_value] * _k)+ check_true(_name + "/non-unit sum", close(sum(_probs), _total), repr(_probs))++for _page in ("questions-and-answers.md", "http-api.md"):+ _text = _prose(_read_page(os.path.join(DOC_ROOT, "docs", _page)))+ check_true("rounding/docs/%s explains precision" % _page,+ bool(_re.search(r"`probabilities`.*?rounded independently to four decimal places", _text)))+ check_true("rounding/docs/%s qualifies the unit sum" % _page,+ "need not sum exactly to 1" in _text)++_tournament = _prose(_read_page(os.path.join(DOC_ROOT, "docs", "http-api.md")).split(+ "### Large choices:", 1)[1].split("## Limits", 1)[0])+check_true("rounding/docs/tournament does not promise an exact unit sum",+ "need not sum exactly to 1" in _tournament)+print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))for f in FAIL:print(" FAIL", f)