Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(integrations): the shipped `answer_confidence` the wrappers gate on is not calibrated

Adeshina Lasisi2c6a2243ef70
diff --git a/docs/crewai.md b/docs/crewai.md
index a7fa74ad..48016240 100644
--- a/docs/crewai.md
+++ b/docs/crewai.md
@@ -101,13 +101,21 @@ except LayaTaskGuardError as e:
---
-## 3. Calibrated Confidence Gating
+## 3. Confidence Gating
-`LayaCrewRouter` gates on calibrated `answer_confidence` (`max(p)`):
+`LayaCrewRouter` gates on `answer_confidence` (`max(p)`):
- **Automatic Fallback:** Specify `fallback_agent_index` to route ambiguous tasks to a human supervisor or general lead agent.
- **Strict Guarding:** Set `raise_on_low_confidence=True` to raise `LayaLowConfidenceError` when a task cannot be matched to an agent role with sufficient confidence.
+`answer_confidence` is the quantity temperature scaling fits and the one every calibration figure
+in the repository is computed on, which is why the threshold reads it rather than the entropy
+`confidence`. It is not calibrated as shipped: `laya-multilingual` ships no fitted temperatures at
+all (`temperature: [1.0, 1.0, 1.0]`, empty `temperature_by_options`), and english's `choice:11+` is
+refused by the loader's own clamp with a warning that says to treat it as uncalibrated. Fit and
+validate any threshold on held-out data at the option counts your workload uses -- see the
+README's [Calibration](https://github.com/NandhaKishorM/laya#calibration) section.
+
---
## 4. Remote HTTP Server Deployments
diff --git a/docs/langchain.md b/docs/langchain.md
index a572ca37..5660c6ab 100644
--- a/docs/langchain.md
+++ b/docs/langchain.md
@@ -74,9 +74,13 @@ result = app.invoke({"input": "I was billed twice for last month's subscription.
print(result["response"]) # -> "Handling billing..."
```
-`confidence_threshold` reads `answer_confidence`, the calibrated `max(p)` confidence the
-calibration figures describe, when the answer carries it, and falls back to the entropy
-`confidence` otherwise.
+`confidence_threshold` reads `answer_confidence`, the `max(p)` confidence the calibration figures
+are computed on, when the answer carries it, and falls back to the entropy `confidence` otherwise.
+It is not calibrated as shipped: `laya-multilingual` ships `temperature: [1.0, 1.0, 1.0]` and no
+`temperature_by_options` at all, and the english checkpoint's `choice:11+` entry is refused by the
+loader's own clamp with a warning that says to treat it as uncalibrated. Fit and validate the
+threshold on held-out data at the option counts your workload uses -- see the README's
+[Calibration](https://github.com/NandhaKishorM/laya#calibration) section.
### Routing with the full conversation
diff --git a/docs/llamaindex.md b/docs/llamaindex.md
index 1c37c96c..63958a9b 100644
--- a/docs/llamaindex.md
+++ b/docs/llamaindex.md
@@ -117,11 +117,19 @@ Both synchronous `query()` and asynchronous `aquery()` are supported.
## 4. Confidence Threshold Gating
-Like Laya's LangChain integration, `LayaSingleSelector` and `LayaQueryRouter` read calibrated `answer_confidence` (`max(p)`):
+Like Laya's LangChain integration, `LayaSingleSelector` and `LayaQueryRouter` read `answer_confidence` (`max(p)`):
- **Automatic Fallback:** Specify `fallback_index` (or `fallback_key`) to seamlessly divert uncertain queries to a safe default engine.
- **Strict Guarding:** Set `raise_on_low_confidence=True` on `LayaSingleSelector` to raise `LayaLowConfidenceError` when input is ambiguous, allowing caller escalation.
+`answer_confidence` is the quantity temperature scaling fits and the one every calibration figure
+in the repository is computed on, which is why the threshold reads it rather than the entropy
+`confidence`. It is not calibrated as shipped: `laya-multilingual` ships no fitted temperatures at
+all (`temperature: [1.0, 1.0, 1.0]`, empty `temperature_by_options`), and english's `choice:11+` is
+refused by the loader's own clamp with a warning that says to treat it as uncalibrated. Fit and
+validate any threshold on held-out data at the option counts your workload uses -- see the
+README's [Calibration](https://github.com/NandhaKishorM/laya#calibration) section.
+
---
## 5. Remote HTTP Deployments