Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(integrations): teach LayaDecision the controls it now forwards

Aashish8dab016f146a
diff --git a/docs/langchain.md b/docs/langchain.md
index da3b9655..9db95835 100644
--- a/docs/langchain.md
+++ b/docs/langchain.md
@@ -8,7 +8,7 @@ Laya provides fast, non-autoregressive decision components for **LangChain** and
* **`LayaEvaluator`**: Rubric-based output grading and hallucination evaluation.
* **`LayaDecision`**: Schema-driven decisions -- a JSON schema or pydantic model in, schema-shaped values out.
-Every node also takes core's five per-call prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`).
+Every node also takes core's per-call prediction controls: the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`).
Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` without requiring PyTorch on edge clients.
@@ -263,6 +263,38 @@ landed between -3.9% and +2.1%, so treat the overhead as unmeasurable rather tha
silicon that loop can overlap forwards on a thread pool, and concurrent MPS forwards abort the
process; pass `max_concurrency=1` there, or call `invoke()` in a loop.
+### The per-call controls
+
+The node plans the questions itself, but the call it makes is an ordinary one, so it takes the
+same seven per-call arguments as the other four nodes -- the two token budgets and the five
+prediction hooks:
+
+```python
+decide = LayaDecision(
+ Ticket,
+ max_len=8192, # the document is longer than the checkpoint's state window
+ head_max_len=512, # the enum has more members than the default option budget fits
+ hooks=[Memo()], # the cache pair from section 8, on a schema decision
+ hooks_timeout=0.25,
+)
+```
+
+`head_max_len` is the one worth knowing here, because a schema writes the option list for you:
+an enum with many members is a wide option prompt, and the trimming an over-wide prompt gets is
+silent -- several members can reach the model as the same text, which is a wrong answer rather
+than an error. See [Widening the Token Budget](#7-widening-the-token-budget-for-many-options) for
+the measured collapse and the recovery.
+
+Leave an argument out and it is not sent at all, so the decision keeps whatever the runner was
+built with. `head_max_len=0` and `hooks=[]` are decisions rather than absences and are forwarded
+as given.
+
+**Remote mode forwards the budgets and refuses the hooks.** A `LayaDecision` with a `base_url`
+puts `max_len` / `head_max_len` in the request body like its siblings, under the same
+`LAYA_MAX_TOKEN_BUDGET` ceiling. A hook is a Python callable and cannot cross HTTP, so passing one
+to a remote node raises at the call site instead of quietly deciding without the cache or the
+audit line.
+
---
## 6. Batching Many Inputs
@@ -343,7 +375,7 @@ the labels already fit the default budget and four answers move. The docs do not
the wider collation changes those four -- it is enough that it can. That is why the two arguments
are opt-in per node: set the knob to fix a question that does not fit, not to sharpen one that does.
-The same override applies to `LayaGuardrail`, `LayaTriage` and `LayaEvaluator`.
+The same override applies to `LayaGuardrail`, `LayaTriage`, `LayaEvaluator` and `LayaDecision`.
It is per node, so a chain can give its wide routing step room while every other node keeps the
checkpoint's defaults, which is the point of not raising `agent.cfg["head_max_len"]` process-wide.