Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(integrations): teach LayaDecision the controls it now forwards
Aashish8dab016f146a
diff --git a/docs/langchain.md b/docs/langchain.mdindex da3b9655..9db95835 100644--- a/docs/langchain.md+++ b/docs/langchain.md@@ -8,7 +8,7 @@ Laya provides fast, non-autoregressive decision components for **LangChain** and* **`LayaEvaluator`**: Rubric-based output grading and hallucination evaluation.* **`LayaDecision`**: Schema-driven decisions -- a JSON schema or pydantic model in, schema-shaped values out.-Every node also takes core's five per-call prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`).+Every node also takes core's per-call prediction controls: the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`).Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` without requiring PyTorch on edge clients.@@ -263,6 +263,38 @@ landed between -3.9% and +2.1%, so treat the overhead as unmeasurable rather thasilicon that loop can overlap forwards on a thread pool, and concurrent MPS forwards abort theprocess; pass `max_concurrency=1` there, or call `invoke()` in a loop.+### The per-call controls++The node plans the questions itself, but the call it makes is an ordinary one, so it takes the+same seven per-call arguments as the other four nodes -- the two token budgets and the five+prediction hooks:++```python+decide = LayaDecision(+ Ticket,+ max_len=8192, # the document is longer than the checkpoint's state window+ head_max_len=512, # the enum has more members than the default option budget fits+ hooks=[Memo()], # the cache pair from section 8, on a schema decision+ hooks_timeout=0.25,+)+```++`head_max_len` is the one worth knowing here, because a schema writes the option list for you:+an enum with many members is a wide option prompt, and the trimming an over-wide prompt gets is+silent -- several members can reach the model as the same text, which is a wrong answer rather+than an error. See [Widening the Token Budget](#7-widening-the-token-budget-for-many-options) for+the measured collapse and the recovery.++Leave an argument out and it is not sent at all, so the decision keeps whatever the runner was+built with. `head_max_len=0` and `hooks=[]` are decisions rather than absences and are forwarded+as given.++**Remote mode forwards the budgets and refuses the hooks.** A `LayaDecision` with a `base_url`+puts `max_len` / `head_max_len` in the request body like its siblings, under the same+`LAYA_MAX_TOKEN_BUDGET` ceiling. A hook is a Python callable and cannot cross HTTP, so passing one+to a remote node raises at the call site instead of quietly deciding without the cache or the+audit line.+---## 6. Batching Many Inputs@@ -343,7 +375,7 @@ the labels already fit the default budget and four answers move. The docs do notthe wider collation changes those four -- it is enough that it can. That is why the two argumentsare opt-in per node: set the knob to fix a question that does not fit, not to sharpen one that does.-The same override applies to `LayaGuardrail`, `LayaTriage` and `LayaEvaluator`.+The same override applies to `LayaGuardrail`, `LayaTriage`, `LayaEvaluator` and `LayaDecision`.It is per node, so a chain can give its wide routing step room while every other node keeps thecheckpoint's defaults, which is the point of not raising `agent.cfg["head_max_len"]` process-wide.