Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs: add staged adoption guide
Bruce-Yiief2a4e20713d
diff --git a/docs/index.md b/docs/index.mdindex 4c8228c4..6ae5a5a5 100644--- a/docs/index.md+++ b/docs/index.md@@ -20,6 +20,7 @@ is one copy to keep current.|---|---|| get typed values back from a JSON schema or a pydantic model | [Schema-driven decisions](structured.md) || log, redact, cache or gate every decision without forking Laya | [Prediction hooks](hooks/index.md) |+| adopt Laya incrementally without granting execution permission | [Staged adoption](staged-adoption.md) || route, screen or triage inside a LangChain or LangGraph app | [LangChain & LangGraph](langchain.md) || run the SDK or the `laya-serve` HTTP API in a container, on CPU or an NVIDIA GPU | [Docker quickstart](docker.md) || build for ARM64 hosts or DGX Spark | [ARM64 and DGX Spark containers](docker-platforms.md) |diff --git a/docs/staged-adoption.md b/docs/staged-adoption.mdnew file mode 100644index 00000000..860d27c0--- /dev/null+++ b/docs/staged-adoption.md@@ -0,0 +1,130 @@+# Staged adoption++Laya returns a typed decision, not permission to execute it. Adopt a decision engine in stages so+that the application keeps control of the real action while evidence accumulates.++This guide describes the application-side rollout around Laya. Laya supplies the decision,+probabilities, confidence and hook events; the application owns the incumbent action, rollout+policy, review boundary and rollback.++## 1. Shadow: observe without side effects++Run Laya on representative real traffic, but keep the incumbent action authoritative. A shadow+record should contain enough context to reproduce a comparison later:++- the request class and the Laya question schema;+- Laya's answer, per-option probabilities, confidence, checkpoint and `run_id`;+- the incumbent action and the eventual reviewed or ground-truth outcome;+- latency, errors and any fallback or review decision.++Use the existing prediction hooks for Laya-side evidence. The logging functions below are+application-owned placeholders; they are not Laya APIs:++```python+from laya import Router++class ShadowLog:+ def on_predict_end(self, ctx):+ result = ctx.results[0] if ctx.results else None+ write_shadow_record({+ "run_id": ctx.run_id,+ "model": ctx.model,+ "decision": ctx.decision,+ "result": result,+ "elapsed_ms": ctx.elapsed_ms,+ "error": None if ctx.error is None else repr(ctx.error),+ })++router = Router(hooks=[ShadowLog()])++def handle(request):+ laya_result = router.predict(request.state, request.questions)+ # The shadow result is recorded by the hook; do not execute it here.+ return run_incumbent_action(request)+```++Keep sensitive fields redacted according to the application's policy. If a Laya call fails before+or during inference, record the application-visible failure and continue or fall back according to+the incumbent policy; a shadow logger must not turn logging into a new user-facing action.++See [Prediction hooks](hooks/index.md), the [hook lifecycle](hooks/lifecycle.md), and+[Tracing](hooks/tracing.md) for the event order and `run_id` correlation.++## 2. Compare: disagreement is a signal, not a verdict++Compare Laya with the incumbent on the same request and question meaning. A disagreement is not+automatically an error: the incumbent may be wrong, the cases may be ambiguous, or the action may+require human judgment. Use reviewed labels or ground-truth outcomes where available, and keep an+explicit `unknown` or review bucket instead of forcing every disagreement into a binary score.++Review comparisons by checkpoint, language or route, question schema, action type and risk class.+Record coverage and disagreement alongside accuracy. A high agreement rate on an easy subset does+not justify promotion for a different language, action or question shape.++## 3. Choose a policy from held-out evidence++A confidence threshold is an application policy, not a property supplied by Laya. Fit or calibrate+the decision scores on representative held-out data, then choose a threshold from the measured+accuracy and error cost at the coverage your application can tolerate. There is no universal number+that transfers across checkpoints, question types, languages or action risks.++Record the checkpoint and question-schema version, calibration method, threshold, evaluation set and+owner with the policy. Re-evaluate it when those inputs change. Confidence orders decisions; it does+not establish that a decision is correct, and high confidence is never execution permission by+itself.++The README's [Automated Confidence Gating](https://github.com/NandhaKishorM/laya#automated-confidence-gating),+[Calibration](https://github.com/NandhaKishorM/laya#calibration), and [Honest limits](https://github.com/NandhaKishorM/laya#honest-limits) sections+give the existing calibration and confidence context. Keep irreversible or high-cost actions behind+an explicit review boundary even when their confidence is high.++## 4. Promote a bounded slice++Promotion should be a measured, reversible change rather than a global on/off switch. Define an+eligibility boundary before enabling automation, for example:++- the checkpoint, language/route and question schema are in the evaluated set;+- the action is reversible or has an explicit human review path;+- the request is not missing required context and has no Laya error;+- the slice has a size or traffic cap and a named rollback condition.++Start with a small canary. Keep review or fallback for ineligible, ambiguous and failed cases. Continue+sampling promoted decisions, compare them with the incumbent and reviewed outcomes, and monitor+disagreement, coverage, fallback rate, errors and latency. Roll back when the agreed guardrail is+breached; promotion is a bounded step, not a permanent declaration that the model is correct.++## Application/Laya boundary++Laya provides the decision evidence and exposes it through the existing API and hooks. The+application owns the incumbent result, action execution, eligibility rules, threshold, review,+fallback and rollback. Hooks can log or annotate evidence, but they do not make a high-impact action+safe to execute.++A practical rollout is therefore:++```text+real request+ ├─ incumbent action (authoritative)+ └─ Laya shadow decision ──> log, compare, evaluate+ └─ bounded eligible slice+ └─ review / fallback / rollback+```++## Rollout checklist++- [ ] Shadow logging is side-effect free and correlated by `run_id`.+- [ ] Comparison data includes the incumbent outcome and reviewed or ground-truth labels where+ available.+- [ ] Thresholds are fitted and validated on representative held-out data.+- [ ] Irreversible or high-cost actions have an explicit review boundary.+- [ ] Promotion is bounded, sampled and reversible, with a named fallback and rollback path.+- [ ] The policy owner and re-evaluation trigger are recorded.++## See also++- [Prediction hooks](hooks/index.md) — the extension seam for audit, metrics and gating.+- [Hook API reference](hooks/api.md) — `PredictContext` fields and lifecycle events.+- [Tracing](hooks/tracing.md) — `run_id` and span correlation.+- [README: Automated Confidence Gating](https://github.com/NandhaKishorM/laya#automated-confidence-gating) — confidence is a+ policy input, not a correctness guarantee.+- [README: Calibration](https://github.com/NandhaKishorM/laya#calibration) and [Honest limits](https://github.com/NandhaKishorM/laya#honest-limits).