Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(integrations): teach the wrappers the two hints that pick the checkpoint
Aashishf8551dc63a09
diff --git a/README.md b/README.mdindex cdfe1c30..ff53b648 100644--- a/README.md+++ b/README.md@@ -1159,7 +1159,7 @@ email = agent.predict({"body": "Your account is locked, verify now"}, laya.email## LangChain and LangGraph Integration-Fast System 1 routing and guardrails directly inside LangGraph workflows and LCEL chains. Every node also takes core's per-call prediction hooks (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`):+Fast System 1 routing and guardrails directly inside LangGraph workflows and LCEL chains. Every node also takes core's per-call prediction hooks (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) and the two routing hints that pick the checkpoint answering (`task`, `lang_guess`) — see [Which checkpoint answers](docs/langchain.md#9-which-checkpoint-answers):```pythonfrom laya.integrations.langchain import LayaDecision, LayaGuardrail, LayaRouterdiff --git a/docs/crewai.md b/docs/crewai.mdindex 48016240..bfc4f768 100644--- a/docs/crewai.md+++ b/docs/crewai.md@@ -5,7 +5,7 @@ Laya provides sub-35ms, non-autoregressive decision components for **CrewAI** mu* **`LayaCrewRouter`**: Sub-35ms task delegation router replacing LLM managers in hierarchical crews.* **`LayaTaskGuard`**: Pre-execution task guardrail screening prompts and instructions for jailbreaks, injections, and policy violations.-Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`, `min_confidence`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) -- see [Per-call decision controls](#5-per-call-decision-controls).+Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`, `min_confidence`), the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) and the two routing hints (`task`, `lang_guess`) -- see [Per-call decision controls](#5-per-call-decision-controls).Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.@@ -140,10 +140,10 @@ The remote client uses Python's standard library `urllib` with zero heavy depend`LayaCrewRouter` and `LayaTaskGuard` take the same per-call arguments the core API does: the twotoken budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`,-`min_confidence`), and the five prediction-hook arguments (`hooks`,-`on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are per instance, so a-crew with a large roster can be given room while the rest of the pipeline keeps the checkpoint's-defaults.+`min_confidence`), the five prediction-hook arguments (`hooks`,+`on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`), and the two routing hints+(`task`, `lang_guess`). They are per instance, so a crew with a large roster can be given room while+the rest of the pipeline keeps the checkpoint's defaults.A delegation choice shares the checkpoint's *option* budget -- `head_max_len`, 192 tokens on `laya`-- and every candidate contributes its role and goal, so past roughly 20 agents the later goals@@ -184,8 +184,9 @@ integration](langchain.md#7-widening-the-token-budget-for-many-options) for that**Hooks run on the local path only.** A router or guard with a `base_url` and `hooks=[...]` raises`ValueError` rather than reporting a success whose hook never ran -- a hook is a Python callable thatexecutes inside `predict`, and no wire format carries it. Install hooks in the process that runs-inference. The two budgets do travel to a remote node, in the request body, up to its-`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.+inference. The two budgets, the language and abstention controls and the two routing hints all+travel to a remote node, in the request body, up to its `LAYA_MAX_TOKEN_BUDGET` ceiling; a larger+value comes back as a 422.### Language and abstention@@ -209,3 +210,29 @@ router = LayaCrewRouter(min_confidence=0.3, # report an abstention below 0.3; the delegation is still made)```++### Which checkpoint answers++**The two routing hints choose the checkpoint that decides.** `task="typed-decisions"` pins the+workflow rather than waiting for auto task detection to recognise the question ids, and `lang_guess`+takes a language code -- or a callable of the state being screened -- from something that already+knows it, consulted after an explicit `lang` and before core's own script detection; any non-English+code routes to the multilingual checkpoint. One naming trap, since both are on the same call:+`route(task, agents)` delegates a CrewAI *task*, while `task=` is core's selector and takes a+workflow name such as `"typed-decisions"` -- it does not say which CrewAI task is being delegated.++```python+guard = LayaTaskGuard(+ questions=screening_questions,+ lang_guess="de", # this crew's tickets arrive in German+)+```++A node built on an `Agent` (`agent=Agent(...)`) reads neither, because an `Agent` has no routing step+to read them in -- it answers on the checkpoint it was built with. Setting one there raises+`ValueError` naming the argument instead of taking the call and dropping the hint, and a callable+`lang_guess` on a `base_url` node raises before the request goes out rather than coming back as+serve's 422 after it. The same rule lives in `laya.integrations._controls`, which the LangChain and+LlamaIndex wrappers import; see the [LangChain+integration](langchain.md#9-which-checkpoint-answers) for the precedence, the measured count of+German queries detection leaves on the English checkpoint, and the recorded before.diff --git a/docs/langchain.md b/docs/langchain.mdindex 5660c6ab..d62d0870 100644--- a/docs/langchain.md+++ b/docs/langchain.md@@ -470,3 +470,84 @@ through `on_predict_start=`/`on_predict_end=` instead of `hooks=` measured 359 m`laya-serve` has no way to receive or run one, so a node with a `base_url` and any of the five setraises `ValueError` naming the arguments rather than reporting success for a cache that never ran.Install hooks on the process that runs inference.++---++## 9. Which Checkpoint Answers++`Router.route` decides between the three checkpoints on this precedence -- explicit `model` >+explicit `task` > detected workflow > explicit `lang` > `lang_guess` > detected script/language --+and until now a chain step could state only `model` of those six. The two that were missing are the+two that do not name a repository:++- **`task`** forces a checkpoint by workflow name, so `task="typed-decisions"` answers on the+ typed-decisions checkpoint without waiting for auto task detection to recognise the question ids.+ An unknown name raises `ValueError` from core and lists what it does know.+- **`lang_guess`** is a language code from something that already knows the answer -- a user id, a+ locale header, an `Accept-Language` value. It is consulted after an explicit `lang` and before+ detection, and it only answers "can the English checkpoint read this?", so any non-English code+ routes to the multilingual one. It may also be a callable taking the state, and a blank one falls+ through to detection, which is how an abstaining language identifier is supposed to behave.++```python+router = LayaRouter(+ criteria=queue_criteria,+ task="typed-decisions", # this workflow, on that checkpoint, always+ lang_guess=lambda state: state["locale"][:2],+)+```++Both are per node, like the budgets, and unset means unsent: leave one out and the deployment's own+`Router(task=...)` / `Router(lang_guess=...)` stays in charge rather than being overwritten with+`None`. `batch()` sends them with every request in the batch, `abatch()` on the same path.++**What the hint moves, measured.** Sixteen German support queries through `Router._route` on+`6d942c9` -- routing only, so no checkpoint is loaded and no answer is taken -- counted by which+checkpoint the decision lands on:++| German queries | no hint | `lang_guess="de"` |+|---|---|---|+| 8 written without umlauts or `ß` | 6 answered on `english` | 8 on `multilingual` |+| 8 carrying umlauts or `ß` | 2 answered on `english` | 8 on `multilingual` |++Detection is a good guesser when the text gives it letters to work with; `Zahlung fehlgeschlagen`+brings it to its stated abstention (`"Latin script, language not identified and no non-English+letters; using default (english)"`), and `Was kostet die MwSt?` to a wrong read (`"English Latin+text"`). A caller that knows the language from a user id or an `Accept-Language` header knows better+than that, which is the whole argument for the hint. What is *not* claimed here is what the two+checkpoints then score on those queries -- that needs weights and a labelled set, and this table is+about which one answers, not how well.++**A recorded `before`.** On `6d942c9`, in real (installed `langchain-core`) mode, a node built with+both hints and a budget, answering through a runner that prints what `predict` received:++```+node.task 'typed_decisions' # extra="allow" stores it+model_extra {'task': 'typed_decisions', 'lang_guess': 'de'}+predict() received {'max_len': 8192} # and the call sees neither hint+```++The node *looked* configured -- `router.task` read back what it was given -- and the decision above+was still taken on the deployment's defaults. The forwarding itself is what the gate asserts, in+`tests/test_langchain.py`, `tests/test_crewai.py` and `tests/test_llamaindex.py`: no latency claim is+made for it, because a routing hint changes which weights answer, not how long the pass takes.++**Remote mode sends them; one shape it cannot.** A node with a `base_url` puts both in the request+body, where `laya-serve` treats them exactly as documented in the [HTTP+API](http-api.md). The callable form of `lang_guess` does not cross HTTP, so a remote node called+with one raises `ValueError` before the request goes out rather than coming back as serve's `422`+after it; install the callable on the `Router` where serve runs, or pass a code.++**`agent=` decides the rule.** A node handed `agent=Agent(...)` is answered by an `Agent`, and an+`Agent` has no routing step to read a routing hint in -- `Agent.predict` takes neither name. Setting+a hint on such a node raises instead of being dropped:++```+ValueError: task is a routing hint, and Agent.predict does not read it: this runner answers on+the checkpoint it was built with. Pass a Router as `agent=`, or drop it.+```++The rule is read from the signature of the call about to be made, not from a list of classes, so a+runner that declares `**kwargs` is forwarded to rather than refused -- the wrapper cannot tell what+it reads, and refusing on a guess would block a call that works. `laya.integrations.crewai` and+`laya.integrations.llamaindex` import the same rule from `laya.integrations._controls`.diff --git a/docs/llamaindex.md b/docs/llamaindex.mdindex 63958a9b..a8e1d28d 100644--- a/docs/llamaindex.md+++ b/docs/llamaindex.md@@ -154,10 +154,10 @@ The remote client uses Python's standard library `urllib` with zero heavy depend`LayaSingleSelector`, `LayaMultiSelector` and `LayaQueryRouter` take the same per-call arguments thecore API does: the two token budgets (`max_len`, `head_max_len`), the language and abstention-controls (`lang`, `min_confidence`), and the five prediction-hook-arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are-per selector, so a wide routing step can be given room while the rest of the pipeline keeps the-checkpoint's defaults.+controls (`lang`, `min_confidence`), the five prediction-hook arguments (`hooks`, `on_predict_start`,+`on_predict_end`, `hooks_raise`, `hooks_timeout`), and the two routing hints (`task`, `lang_guess`).+They are per selector, so a wide routing step can be given room while the rest of the pipeline keeps+the checkpoint's defaults.A choice question's options share the checkpoint's *option* budget -- `head_max_len`, 192 tokens on`laya` -- and every candidate contributes its name and description, so past roughly 20 tools the@@ -192,8 +192,8 @@ integration](langchain.md#7-widening-the-token-budget-for-many-options) for that**Hooks run on the local path only.** A selector with a `base_url` and `hooks=[...]` raises`ValueError` rather than reporting a success whose cache never ran -- a hook is a Python callablethat runs inside `predict`, and no wire format carries it. Install hooks in the process that runs-inference. The two budgets do travel to a remote node, in the request body, up to its-`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.+inference. The two budgets and the two routing hints do travel to a remote node, in the request body,+up to its `LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.### Language and abstention@@ -210,10 +210,33 @@ Both are read by `Agent.predict` and `Router.predict` alike and accepted byunset one is omitted, not sent as `None`, so it cannot shadow the deployment's own default;`min_confidence=0.0` and `lang=""` are real values and are forwarded as given.+### Which checkpoint answers++**The two routing hints choose the checkpoint that selects.** `task="typed-decisions"` pins the+workflow rather than waiting for auto task detection to recognise the question ids, and `lang_guess`+takes a language code -- or a callable of the query being selected against -- from something that+already knows it, consulted after an explicit `lang` and before core's own script detection; any+non-English code routes to the multilingual checkpoint. A German-documented index is the case the+hint is for: measured on sixteen German support queries, detection leaves 6 of the 8 written without+umlauts and 2 of the 8 carrying them on the `english` checkpoint, while `lang_guess="de"` moves all+16 to `multilingual`. See the [LangChain+integration](langchain.md#9-which-checkpoint-answers) for that table and its+reason strings.+```pythonselector = LayaSingleSelector(instructions="Which tool or query engine is best suited to answer this query?",lang="de", # answer German queries in Germanmin_confidence=0.3, # report an abstention below 0.3; the selection is still made+ lang_guess="de", # this index's documents are German)```++A selector built on an `Agent` (`agent=Agent(...)`) reads neither routing hint, because an `Agent`+has no routing step to read them in -- it answers on the checkpoint it was built with. Setting one+there raises `ValueError` naming the argument instead of taking the call and dropping the hint, and a+callable `lang_guess` on a `base_url` selector raises before the request goes out rather than coming+back as serve's 422 after it. The same rule lives in `laya.integrations._controls`, which the LangChain+and CrewAI wrappers import; see the [LangChain+integration](langchain.md#9-which-checkpoint-answers) for the precedence and the+recorded before.