Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(confidence): answer_confidence describes the clamped temperature, not the fitted one

Adeshina Lasisi5ea8d12de4c6
diff --git a/laya/common.py b/laya/common.py
index 8625fee9..ebf46666 100644
--- a/laya/common.py
+++ b/laya/common.py
@@ -785,8 +785,11 @@ def answer_confidence(p: np.ndarray, k: int) -> float:
of the answers returned at confidence c, about c of them are right. That property is
conditional, and the condition is not met by default -- it holds only after the temperatures
have been fitted and validated on held-out data for this checkpoint and this option count.
- The shipped checkpoints are over-confident: `choice:11+` is a ~10x sharpener that returns a
- point mass at 1.0, so a threshold applied to them selects below model accuracy (issue #394).
+ The shipped checkpoints are over-confident: `choice:11+` is fitted at 0.1006, a ~10x
+ sharpener that would publish a genuine 0.24 top probability as 0.9999. `clamp_temperature`
+ refuses it and applies `TEMP_MIN` (0.5) instead, which still sharpens 2x and publishes that
+ same 0.24 as 0.4993 -- so a threshold applied to these numbers still selects below model
+ accuracy (issue #394). The clamp bounds the damage; it does not calibrate anything.
`confidence_from_probs` below reports a different quantity on a different scale and carries
no such guarantee, so the two must not be compared against the same threshold.