Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(integrations): teach the wrappers the two hints that pick the checkpoint

Aashishf8551dc63a09
diff --git a/README.md b/README.md
index cdfe1c30..ff53b648 100644
--- a/README.md
+++ b/README.md
@@ -1159,7 +1159,7 @@ email = agent.predict({"body": "Your account is locked, verify now"}, laya.email
## LangChain and LangGraph Integration
-Fast System 1 routing and guardrails directly inside LangGraph workflows and LCEL chains. Every node also takes core's per-call prediction hooks (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`):
+Fast System 1 routing and guardrails directly inside LangGraph workflows and LCEL chains. Every node also takes core's per-call prediction hooks (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) and the two routing hints that pick the checkpoint answering (`task`, `lang_guess`) — see [Which checkpoint answers](docs/langchain.md#9-which-checkpoint-answers):
```python
from laya.integrations.langchain import LayaDecision, LayaGuardrail, LayaRouter
diff --git a/docs/crewai.md b/docs/crewai.md
index 48016240..bfc4f768 100644
--- a/docs/crewai.md
+++ b/docs/crewai.md
@@ -5,7 +5,7 @@ Laya provides sub-35ms, non-autoregressive decision components for **CrewAI** mu
* **`LayaCrewRouter`**: Sub-35ms task delegation router replacing LLM managers in hierarchical crews.
* **`LayaTaskGuard`**: Pre-execution task guardrail screening prompts and instructions for jailbreaks, injections, and policy violations.
-Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`, `min_confidence`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) -- see [Per-call decision controls](#5-per-call-decision-controls).
+Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`, `min_confidence`), the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) and the two routing hints (`task`, `lang_guess`) -- see [Per-call decision controls](#5-per-call-decision-controls).
Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.
@@ -140,10 +140,10 @@ The remote client uses Python's standard library `urllib` with zero heavy depend
`LayaCrewRouter` and `LayaTaskGuard` take the same per-call arguments the core API does: the two
token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`,
-`min_confidence`), and the five prediction-hook arguments (`hooks`,
-`on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are per instance, so a
-crew with a large roster can be given room while the rest of the pipeline keeps the checkpoint's
-defaults.
+`min_confidence`), the five prediction-hook arguments (`hooks`,
+`on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`), and the two routing hints
+(`task`, `lang_guess`). They are per instance, so a crew with a large roster can be given room while
+the rest of the pipeline keeps the checkpoint's defaults.
A delegation choice shares the checkpoint's *option* budget -- `head_max_len`, 192 tokens on `laya`
-- and every candidate contributes its role and goal, so past roughly 20 agents the later goals
@@ -184,8 +184,9 @@ integration](langchain.md#7-widening-the-token-budget-for-many-options) for that
**Hooks run on the local path only.** A router or guard with a `base_url` and `hooks=[...]` raises
`ValueError` rather than reporting a success whose hook never ran -- a hook is a Python callable that
executes inside `predict`, and no wire format carries it. Install hooks in the process that runs
-inference. The two budgets do travel to a remote node, in the request body, up to its
-`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.
+inference. The two budgets, the language and abstention controls and the two routing hints all
+travel to a remote node, in the request body, up to its `LAYA_MAX_TOKEN_BUDGET` ceiling; a larger
+value comes back as a 422.
### Language and abstention
@@ -209,3 +210,29 @@ router = LayaCrewRouter(
min_confidence=0.3, # report an abstention below 0.3; the delegation is still made
)
```
+
+### Which checkpoint answers
+
+**The two routing hints choose the checkpoint that decides.** `task="typed-decisions"` pins the
+workflow rather than waiting for auto task detection to recognise the question ids, and `lang_guess`
+takes a language code -- or a callable of the state being screened -- from something that already
+knows it, consulted after an explicit `lang` and before core's own script detection; any non-English
+code routes to the multilingual checkpoint. One naming trap, since both are on the same call:
+`route(task, agents)` delegates a CrewAI *task*, while `task=` is core's selector and takes a
+workflow name such as `"typed-decisions"` -- it does not say which CrewAI task is being delegated.
+
+```python
+guard = LayaTaskGuard(
+ questions=screening_questions,
+ lang_guess="de", # this crew's tickets arrive in German
+)
+```
+
+A node built on an `Agent` (`agent=Agent(...)`) reads neither, because an `Agent` has no routing step
+to read them in -- it answers on the checkpoint it was built with. Setting one there raises
+`ValueError` naming the argument instead of taking the call and dropping the hint, and a callable
+`lang_guess` on a `base_url` node raises before the request goes out rather than coming back as
+serve's 422 after it. The same rule lives in `laya.integrations._controls`, which the LangChain and
+LlamaIndex wrappers import; see the [LangChain
+integration](langchain.md#9-which-checkpoint-answers) for the precedence, the measured count of
+German queries detection leaves on the English checkpoint, and the recorded before.
diff --git a/docs/langchain.md b/docs/langchain.md
index 5660c6ab..d62d0870 100644
--- a/docs/langchain.md
+++ b/docs/langchain.md
@@ -470,3 +470,84 @@ through `on_predict_start=`/`on_predict_end=` instead of `hooks=` measured 359 m
`laya-serve` has no way to receive or run one, so a node with a `base_url` and any of the five set
raises `ValueError` naming the arguments rather than reporting success for a cache that never ran.
Install hooks on the process that runs inference.
+
+---
+
+## 9. Which Checkpoint Answers
+
+`Router.route` decides between the three checkpoints on this precedence -- explicit `model` >
+explicit `task` > detected workflow > explicit `lang` > `lang_guess` > detected script/language --
+and until now a chain step could state only `model` of those six. The two that were missing are the
+two that do not name a repository:
+
+- **`task`** forces a checkpoint by workflow name, so `task="typed-decisions"` answers on the
+ typed-decisions checkpoint without waiting for auto task detection to recognise the question ids.
+ An unknown name raises `ValueError` from core and lists what it does know.
+- **`lang_guess`** is a language code from something that already knows the answer -- a user id, a
+ locale header, an `Accept-Language` value. It is consulted after an explicit `lang` and before
+ detection, and it only answers "can the English checkpoint read this?", so any non-English code
+ routes to the multilingual one. It may also be a callable taking the state, and a blank one falls
+ through to detection, which is how an abstaining language identifier is supposed to behave.
+
+```python
+router = LayaRouter(
+ criteria=queue_criteria,
+ task="typed-decisions", # this workflow, on that checkpoint, always
+ lang_guess=lambda state: state["locale"][:2],
+)
+```
+
+Both are per node, like the budgets, and unset means unsent: leave one out and the deployment's own
+`Router(task=...)` / `Router(lang_guess=...)` stays in charge rather than being overwritten with
+`None`. `batch()` sends them with every request in the batch, `abatch()` on the same path.
+
+**What the hint moves, measured.** Sixteen German support queries through `Router._route` on
+`6d942c9` -- routing only, so no checkpoint is loaded and no answer is taken -- counted by which
+checkpoint the decision lands on:
+
+| German queries | no hint | `lang_guess="de"` |
+|---|---|---|
+| 8 written without umlauts or `ß` | 6 answered on `english` | 8 on `multilingual` |
+| 8 carrying umlauts or `ß` | 2 answered on `english` | 8 on `multilingual` |
+
+Detection is a good guesser when the text gives it letters to work with; `Zahlung fehlgeschlagen`
+brings it to its stated abstention (`"Latin script, language not identified and no non-English
+letters; using default (english)"`), and `Was kostet die MwSt?` to a wrong read (`"English Latin
+text"`). A caller that knows the language from a user id or an `Accept-Language` header knows better
+than that, which is the whole argument for the hint. What is *not* claimed here is what the two
+checkpoints then score on those queries -- that needs weights and a labelled set, and this table is
+about which one answers, not how well.
+
+**A recorded `before`.** On `6d942c9`, in real (installed `langchain-core`) mode, a node built with
+both hints and a budget, answering through a runner that prints what `predict` received:
+
+```
+node.task 'typed_decisions' # extra="allow" stores it
+model_extra {'task': 'typed_decisions', 'lang_guess': 'de'}
+predict() received {'max_len': 8192} # and the call sees neither hint
+```
+
+The node *looked* configured -- `router.task` read back what it was given -- and the decision above
+was still taken on the deployment's defaults. The forwarding itself is what the gate asserts, in
+`tests/test_langchain.py`, `tests/test_crewai.py` and `tests/test_llamaindex.py`: no latency claim is
+made for it, because a routing hint changes which weights answer, not how long the pass takes.
+
+**Remote mode sends them; one shape it cannot.** A node with a `base_url` puts both in the request
+body, where `laya-serve` treats them exactly as documented in the [HTTP
+API](http-api.md). The callable form of `lang_guess` does not cross HTTP, so a remote node called
+with one raises `ValueError` before the request goes out rather than coming back as serve's `422`
+after it; install the callable on the `Router` where serve runs, or pass a code.
+
+**`agent=` decides the rule.** A node handed `agent=Agent(...)` is answered by an `Agent`, and an
+`Agent` has no routing step to read a routing hint in -- `Agent.predict` takes neither name. Setting
+a hint on such a node raises instead of being dropped:
+
+```
+ValueError: task is a routing hint, and Agent.predict does not read it: this runner answers on
+the checkpoint it was built with. Pass a Router as `agent=`, or drop it.
+```
+
+The rule is read from the signature of the call about to be made, not from a list of classes, so a
+runner that declares `**kwargs` is forwarded to rather than refused -- the wrapper cannot tell what
+it reads, and refusing on a guess would block a call that works. `laya.integrations.crewai` and
+`laya.integrations.llamaindex` import the same rule from `laya.integrations._controls`.
diff --git a/docs/llamaindex.md b/docs/llamaindex.md
index 63958a9b..a8e1d28d 100644
--- a/docs/llamaindex.md
+++ b/docs/llamaindex.md
@@ -154,10 +154,10 @@ The remote client uses Python's standard library `urllib` with zero heavy depend
`LayaSingleSelector`, `LayaMultiSelector` and `LayaQueryRouter` take the same per-call arguments the
core API does: the two token budgets (`max_len`, `head_max_len`), the language and abstention
-controls (`lang`, `min_confidence`), and the five prediction-hook
-arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are
-per selector, so a wide routing step can be given room while the rest of the pipeline keeps the
-checkpoint's defaults.
+controls (`lang`, `min_confidence`), the five prediction-hook arguments (`hooks`, `on_predict_start`,
+`on_predict_end`, `hooks_raise`, `hooks_timeout`), and the two routing hints (`task`, `lang_guess`).
+They are per selector, so a wide routing step can be given room while the rest of the pipeline keeps
+the checkpoint's defaults.
A choice question's options share the checkpoint's *option* budget -- `head_max_len`, 192 tokens on
`laya` -- and every candidate contributes its name and description, so past roughly 20 tools the
@@ -192,8 +192,8 @@ integration](langchain.md#7-widening-the-token-budget-for-many-options) for that
**Hooks run on the local path only.** A selector with a `base_url` and `hooks=[...]` raises
`ValueError` rather than reporting a success whose cache never ran -- a hook is a Python callable
that runs inside `predict`, and no wire format carries it. Install hooks in the process that runs
-inference. The two budgets do travel to a remote node, in the request body, up to its
-`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.
+inference. The two budgets and the two routing hints do travel to a remote node, in the request body,
+up to its `LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.
### Language and abstention
@@ -210,10 +210,33 @@ Both are read by `Agent.predict` and `Router.predict` alike and accepted by
unset one is omitted, not sent as `None`, so it cannot shadow the deployment's own default;
`min_confidence=0.0` and `lang=""` are real values and are forwarded as given.
+### Which checkpoint answers
+
+**The two routing hints choose the checkpoint that selects.** `task="typed-decisions"` pins the
+workflow rather than waiting for auto task detection to recognise the question ids, and `lang_guess`
+takes a language code -- or a callable of the query being selected against -- from something that
+already knows it, consulted after an explicit `lang` and before core's own script detection; any
+non-English code routes to the multilingual checkpoint. A German-documented index is the case the
+hint is for: measured on sixteen German support queries, detection leaves 6 of the 8 written without
+umlauts and 2 of the 8 carrying them on the `english` checkpoint, while `lang_guess="de"` moves all
+16 to `multilingual`. See the [LangChain
+integration](langchain.md#9-which-checkpoint-answers) for that table and its
+reason strings.
+
```python
selector = LayaSingleSelector(
instructions="Which tool or query engine is best suited to answer this query?",
lang="de", # answer German queries in German
min_confidence=0.3, # report an abstention below 0.3; the selection is still made
+ lang_guess="de", # this index's documents are German
)
```
+
+A selector built on an `Agent` (`agent=Agent(...)`) reads neither routing hint, because an `Agent`
+has no routing step to read them in -- it answers on the checkpoint it was built with. Setting one
+there raises `ValueError` naming the argument instead of taking the call and dropping the hint, and a
+callable `lang_guess` on a `base_url` selector raises before the request goes out rather than coming
+back as serve's 422 after it. The same rule lives in `laya.integrations._controls`, which the LangChain
+and CrewAI wrappers import; see the [LangChain
+integration](langchain.md#9-which-checkpoint-answers) for the precedence and the
+recorded before.