Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(structured): split decide_batch's keywords by who reads them

Aashish1b6c4c038bbb
diff --git a/docs/structured.md b/docs/structured.md
index f4a9ce14..61c10ac5 100644
--- a/docs/structured.md
+++ b/docs/structured.md
@@ -141,9 +141,16 @@ values = agent.decide_batch(ticket_texts, schema=Ticket) # values[i] ma
results = router.decide_batch(states, schema=Ticket, return_details=True, batch_size=64)
```
-On a `Router` each state is still routed on its own, so one call can span checkpoints. Keyword
-arguments reach `predict_batch`, so `batch_size=`, `model=` and hooks work as they do for `decide`.
-`Agent`, `ONNXAgent` and `Router` all have it; a runner without `predict_batch` raises
+On a `Router` each state is still routed on its own, so one call can span checkpoints. Which
+keywords reach `predict_batch` depends on who reads them. The call-level ones — `batch_size=`,
+`hooks=`, `sort_by_length=` — are passed as keywords to `predict_batch`. The routing pins and the
+token budget — `model=`, `task=`, `lang=`, `lang_guess=`, `max_len=`, `head_max_len=`, the tuple
+`laya/structured.py::_ROUTER_REQUEST_KEYS` names — are read off every *request* by a `Router`, whose
+`predict_batch` declares none of them, so `decide_batch` copies them onto each request instead; that
+is why `decide_batch(states, schema=..., model="multilingual")` pins a checkpoint the way `decide`
+does. On an `Agent`-like runner the split is visible: its `predict_batch` declares `batch_size=`,
+`hooks=` and `lang=` but no `model=` and swallows nothing, so `model=` there is a `TypeError`, not a
+silent extra keyword. `Agent`, `ONNXAgent` and `Router` all have it; a runner without `predict_batch` raises
`TypeError` rather than silently falling back to a loop — call `decide` per state there. Batching can shift borderline argmaxes the
same way `predict_batch` does; the README records the measured speedups for both devices.