Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(integrations): forward lang and min_confidence through the framework wrappers
Aashish7f214109fe9a
diff --git a/docs/crewai.md b/docs/crewai.mdindex 0c196b22..da66f11f 100644--- a/docs/crewai.md+++ b/docs/crewai.md@@ -5,7 +5,7 @@ Laya provides sub-35ms, non-autoregressive decision components for **CrewAI** mu* **`LayaCrewRouter`**: Sub-35ms task delegation router replacing LLM managers in hierarchical crews.* **`LayaTaskGuard`**: Pre-execution task guardrail screening prompts and instructions for jailbreaks, injections, and policy violations.-Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) -- see [Per-call decision controls](#5-per-call-decision-controls).+Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`, `min_confidence`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) -- see [Per-call decision controls](#5-per-call-decision-controls).Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.@@ -129,7 +129,8 @@ The remote client uses Python's standard library `urllib` with zero heavy depend## 5. Per-call decision controls`LayaCrewRouter` and `LayaTaskGuard` take the same per-call arguments the core API does: the two-token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`,+token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`,+`min_confidence`), and the five prediction-hook arguments (`hooks`,`on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are per instance, so acrew with a large roster can be given room while the rest of the pipeline keeps the checkpoint'sdefaults.@@ -175,3 +176,21 @@ integration](langchain.md#7-widening-the-token-budget-for-many-options) for thatexecutes inside `predict`, and no wire format carries it. Install hooks in the process that runsinference. The two budgets do travel to a remote node, in the request body, up to its`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.++### Language and abstention++`lang` pins the language the task is routed and answered in -- selecting the answering+checkpoint's per-language calibration instead of relying on built-in detection -- and+`min_confidence` is core's abstention gate: a decision under it comes back as an abstention rather+than a forced delegation. Both are read by `Agent.predict` and `Router.predict` alike and accepted by+`laya-serve` in the request body, so a router or guard forwards them on the local and the remote+path. An unset one is omitted, not sent as `None`, so it cannot shadow the deployment's own default;+`min_confidence=0.0` and `lang=""` are real values and are forwarded as given.++```python+router = LayaCrewRouter(+ confidence_threshold=0.80,+ lang="fr", # route a French-language crew in French+ min_confidence=0.3, # abstain on a delegation the model is not sure about+)+```diff --git a/docs/langchain.md b/docs/langchain.mdindex da3b9655..3ca5d902 100644--- a/docs/langchain.md+++ b/docs/langchain.md@@ -8,7 +8,7 @@ Laya provides fast, non-autoregressive decision components for **LangChain** and* **`LayaEvaluator`**: Rubric-based output grading and hallucination evaluation.* **`LayaDecision`**: Schema-driven decisions -- a JSON schema or pydantic model in, schema-shaped values out.-Every node also takes core's five per-call prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`).+Every node also takes core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`), the language and abstention controls (`lang`, `min_confidence`), and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`).Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` without requiring PyTorch on edge clients.@@ -351,6 +351,28 @@ checkpoint's defaults, which is the point of not raising `agent.cfg["head_max_lerequest body, and `laya-serve` applies them up to its `LAYA_MAX_TOKEN_BUDGET` ceiling (8192 bydefault); a larger value comes back as a 422.+### Language and abstention++Every runnable also takes `lang` and `min_confidence`, the two per-request controls `Agent.predict`+and `Router.predict` both read and `laya-serve` both accepts in the body. `lang` pins the language+the state is routed and answered in -- select the answering checkpoint's per-language calibration+rather than relying on built-in detection -- and `min_confidence` is core's abstention gate: a+decision under it comes back as an abstention instead of a forced choice. Both are forwarded on the+local and the remote path, and an unset one is omitted rather than sent as `None` so it cannot+shadow a checkpoint's own default. `min_confidence=0.0` and `lang=""` are real values, not absences,+and are forwarded as given.++```python+router = LayaRouter(+ criteria={"billing": "invoices", "tech": "bugs"},+ lang="es", # route and answer in Spanish+ min_confidence=0.3, # abstain below a 0.3 calibrated confidence+)+```++Unlike `task` and `lang_guess` -- Router-only routing keywords a direct `Agent.predict` rejects --+these two are safe on every runner and every deployment.+---## 8. Prediction Hooks on a Single Nodediff --git a/docs/llamaindex.md b/docs/llamaindex.mdindex f75eea4a..d543dd12 100644--- a/docs/llamaindex.md+++ b/docs/llamaindex.md@@ -145,7 +145,8 @@ The remote client uses Python's standard library `urllib` with zero heavy depend## 6. Per-call decision controls`LayaSingleSelector`, `LayaMultiSelector` and `LayaQueryRouter` take the same per-call arguments the-core API does: the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook+core API does: the two token budgets (`max_len`, `head_max_len`), the language and abstention+controls (`lang`, `min_confidence`), and the five prediction-hookarguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They areper selector, so a wide routing step can be given room while the rest of the pipeline keeps thecheckpoint's defaults.@@ -185,3 +186,21 @@ integration](langchain.md#7-widening-the-token-budget-for-many-options) for thatthat runs inside `predict`, and no wire format carries it. Install hooks in the process that runsinference. The two budgets do travel to a remote node, in the request body, up to its`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.++### Language and abstention++`lang` pins the language the query is routed and answered in -- selecting the answering+checkpoint's per-language calibration instead of relying on built-in detection -- and+`min_confidence` is core's abstention gate: a decision under it comes back as an abstention rather+than a forced selection. Both are read by `Agent.predict` and `Router.predict` alike and accepted by+`laya-serve` in the request body, so a selector forwards them on the local and the remote path. An+unset one is omitted, not sent as `None`, so it cannot shadow the deployment's own default;+`min_confidence=0.0` and `lang=""` are real values and are forwarded as given.++```python+selector = LayaSingleSelector(+ instructions="Which tool or query engine is best suited to answer this query?",+ lang="de", # answer German queries in German+ min_confidence=0.3, # abstain when no tool clears a 0.3 confidence+)+```diff --git a/laya/integrations/_controls.py b/laya/integrations/_controls.pyindex a8b1caaa..999f5ce4 100644--- a/laya/integrations/_controls.py+++ b/laya/integrations/_controls.py@@ -18,6 +18,14 @@ from typing import Any, Dict, Optional# laya-serve accepts in the request body up to its `LAYA_MAX_TOKEN_BUDGET` ceiling.PREDICT_CONTROLS = ("max_len", "head_max_len")+# The per-call language and abstention controls. Unlike `task` / `lang_guess` -- Router-only+# routing keywords a direct `Agent.predict` does not accept -- both `lang` and `min_confidence`+# are read by `Agent.predict`/`system_one` AND `Router.predict`, and laya-serve takes each in the+# request body, so the same kwargs are safe on the local and the remote path alike. `lang` routes+# non-English text and selects the answering checkpoint's per-language calibration;+# `min_confidence` is core's abstention gate (#361).+DECISION_CONTROLS = ("lang", "min_confidence")+# The per-call hook family. These are Python callables and flags that run inside `predict`, so# they exist only on the local path -- see `reject_remote_hooks`.HOOK_CONTROLS = ("hooks", "on_predict_start", "on_predict_end", "hooks_raise", "hooks_timeout")@@ -35,13 +43,29 @@ def budget_kwargs(max_len: Optional[int] = None,if value is not None}+def decision_kwargs(lang: Optional[str] = None,+ min_confidence: Optional[float] = None) -> Dict[str, Any]:+ """The language and abstention overrides, with the unset ones omitted.++ Absent rather than `None`: a `None` `lang` would shadow the checkpoint's own detection and a+ `None` `min_confidence` would override the runner's abstention default with nothing. Both use+ the `is not None` test rather than truthiness -- an empty `lang` is core's documented "fall+ through to detection", and a `0.0` abstention gate is a real decision (abstain over nothing),+ not an absence.+ """+ return {key: value for key, value in (("lang", lang), ("min_confidence", min_confidence))+ if value is not None}++def predict_kwargs(model: Optional[str] = None, max_len: Optional[int] = None,- head_max_len: Optional[int] = None) -> Dict[str, Any]:+ head_max_len: Optional[int] = None, lang: Optional[str] = None,+ min_confidence: Optional[float] = None) -> Dict[str, Any]:"""The per-request overrides a local runner accepts, with the unset ones omitted."""kwargs: Dict[str, Any] = {}if model:kwargs["model"] = modelkwargs.update(budget_kwargs(max_len, head_max_len))+ kwargs.update(decision_kwargs(lang, min_confidence))return kwargsdiff --git a/laya/integrations/crewai.py b/laya/integrations/crewai.pyindex 435d96c1..8f712b75 100644--- a/laya/integrations/crewai.py+++ b/laya/integrations/crewai.py@@ -55,6 +55,7 @@ class LayaTaskGuardError(ValueError):# `predict` call and the same laya-serve request body, and three copies of the rule is how two of# them came to forward only `model`.from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs # noqa: E402+from ._controls import decision_kwargs as _decision_kwargs # noqa: E402from ._controls import predict_kwargs as _predict_kwargs, reject_remote_hooks as _reject_remote_hooks # noqa: E402# One class for every integration, so `except LayaLowConfidenceError` catches all of them.@@ -146,11 +147,15 @@ def _call_remote(timeout: float = 10.0,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,) -> Dict[str, Any]:"""Send decision request to a remote laya-serve HTTP instance using standard library urllib.`max_len` / `head_max_len` travel in the body; laya-serve applies them up to its- `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422.+ `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422. `lang` / `min_confidence`+ ride in the same body (they are laya-serve `BODY_CONTROLS` too): the language codes the state+ is routed and answered in, and core's abstention gate."""url = base_url.rstrip("/")if not url.endswith("/v1/systemone"):@@ -163,6 +168,10 @@ def _call_remote(payload["max_len"] = max_lenif head_max_len is not None:payload["head_max_len"] = head_max_len+ if lang is not None:+ payload["lang"] = lang+ if min_confidence is not None:+ payload["min_confidence"] = min_confidencedata = json.dumps(payload).encode("utf-8")headers = {"Content-Type": "application/json"}@@ -204,6 +213,8 @@ def _execute_decision(model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -219,9 +230,11 @@ def _execute_decision(if base_url:_reject_remote_hooks(hook_kwargs, base_url)budget = _budget_kwargs(max_len, head_max_len)- return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)+ decision = _decision_kwargs(lang, min_confidence)+ return _call_remote(base_url, state, questions, api_key=api_key, model=model,+ **budget, **decision)runner = agent if agent is not None else _get_default_router()- kwargs = _predict_kwargs(model, max_len, head_max_len)+ kwargs = _predict_kwargs(model, max_len, head_max_len, lang, min_confidence)kwargs.update(hook_kwargs)return runner.predict(state, questions, **kwargs)@@ -245,6 +258,8 @@ class LayaCrewRouter:model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -261,6 +276,8 @@ class LayaCrewRouter:self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -297,6 +314,8 @@ class LayaCrewRouter:model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,@@ -414,6 +433,8 @@ class LayaTaskGuard:model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -430,6 +451,8 @@ class LayaTaskGuard:self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -455,6 +478,8 @@ class LayaTaskGuard:model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,diff --git a/laya/integrations/langchain.py b/laya/integrations/langchain.pyindex d6282877..868b19ca 100644--- a/laya/integrations/langchain.py+++ b/laya/integrations/langchain.py@@ -30,6 +30,7 @@ except ImportError:# at the same `predict` call and the same laya-serve body; keeping the rule in three places is how# two of them ended up forwarding only `model`.from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs+from ._controls import decision_kwargs as _decision_kwargsfrom ._controls import predict_kwargs as _predict_kwargsfrom ._controls import reject_remote_hooks as _reject_remote_hooks@@ -157,11 +158,15 @@ def _call_remote(timeout: float = 10.0,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,) -> Dict[str, Any]:"""Send decision request to a remote laya-serve HTTP instance using standard library urllib.`max_len` / `head_max_len` travel in the body; laya-serve applies them up to its- `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422.+ `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422. `lang` / `min_confidence`+ ride in the same body (they are laya-serve `BODY_CONTROLS` too): the language codes the state+ for routing and calibration, and the abstention gate flags a low-confidence answer."""url = base_url.rstrip("/")if not url.endswith("/v1/systemone"):@@ -174,6 +179,10 @@ def _call_remote(payload["max_len"] = max_lenif head_max_len is not None:payload["head_max_len"] = head_max_len+ if lang is not None:+ payload["lang"] = lang+ if min_confidence is not None:+ payload["min_confidence"] = min_confidencedata = json.dumps(payload).encode("utf-8")headers = {"Content-Type": "application/json"}@@ -215,6 +224,8 @@ def _execute_decision(model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -225,9 +236,11 @@ def _execute_decision(if base_url:_reject_remote_hooks(hook_kwargs, base_url)budget = _budget_kwargs(max_len, head_max_len)- return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)+ decision = _decision_kwargs(lang, min_confidence)+ return _call_remote(base_url, state, questions, api_key=api_key, model=model,+ **budget, **decision)runner = agent if agent is not None else _get_default_router()- kwargs = _predict_kwargs(model, max_len, head_max_len)+ kwargs = _predict_kwargs(model, max_len, head_max_len, lang, min_confidence)kwargs.update(hook_kwargs)return runner.predict(state, questions, **kwargs)@@ -381,6 +394,8 @@ class LayaRouter(_BatchedRunnable, RunnableSerializable):model: Optional[str] = Nonemax_len: Optional[int] = Nonehead_max_len: Optional[int] = None+ lang: Optional[str] = None+ min_confidence: Optional[float] = Nonehooks: Optional[Any] = Noneon_predict_start: Optional[Any] = Noneon_predict_end: Optional[Any] = None@@ -406,6 +421,8 @@ class LayaRouter(_BatchedRunnable, RunnableSerializable):model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -426,6 +443,8 @@ class LayaRouter(_BatchedRunnable, RunnableSerializable):model=model,max_len=max_len,head_max_len=head_max_len,+ lang=lang,+ min_confidence=min_confidence,hooks=hooks,on_predict_start=on_predict_start,on_predict_end=on_predict_end,@@ -445,6 +464,8 @@ class LayaRouter(_BatchedRunnable, RunnableSerializable):self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -488,6 +509,8 @@ class LayaRouter(_BatchedRunnable, RunnableSerializable):model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,@@ -527,6 +550,8 @@ class LayaGuardrail(_BatchedRunnable, RunnableSerializable):model: Optional[str] = Nonemax_len: Optional[int] = Nonehead_max_len: Optional[int] = None+ lang: Optional[str] = None+ min_confidence: Optional[float] = Nonehooks: Optional[Any] = Noneon_predict_start: Optional[Any] = Noneon_predict_end: Optional[Any] = None@@ -550,6 +575,8 @@ class LayaGuardrail(_BatchedRunnable, RunnableSerializable):model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -572,6 +599,8 @@ class LayaGuardrail(_BatchedRunnable, RunnableSerializable):model=model,max_len=max_len,head_max_len=head_max_len,+ lang=lang,+ min_confidence=min_confidence,hooks=hooks,on_predict_start=on_predict_start,on_predict_end=on_predict_end,@@ -591,6 +620,8 @@ class LayaGuardrail(_BatchedRunnable, RunnableSerializable):self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -684,6 +715,8 @@ class LayaGuardrail(_BatchedRunnable, RunnableSerializable):model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,@@ -711,6 +744,8 @@ class LayaTriage(_BatchedRunnable, RunnableSerializable):model: Optional[str] = Nonemax_len: Optional[int] = Nonehead_max_len: Optional[int] = None+ lang: Optional[str] = None+ min_confidence: Optional[float] = Nonehooks: Optional[Any] = Noneon_predict_start: Optional[Any] = Noneon_predict_end: Optional[Any] = None@@ -730,6 +765,8 @@ class LayaTriage(_BatchedRunnable, RunnableSerializable):model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -746,6 +783,8 @@ class LayaTriage(_BatchedRunnable, RunnableSerializable):model=model,max_len=max_len,head_max_len=head_max_len,+ lang=lang,+ min_confidence=min_confidence,hooks=hooks,on_predict_start=on_predict_start,on_predict_end=on_predict_end,@@ -761,6 +800,8 @@ class LayaTriage(_BatchedRunnable, RunnableSerializable):self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -804,6 +845,8 @@ class LayaTriage(_BatchedRunnable, RunnableSerializable):model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,@@ -831,6 +874,8 @@ class LayaEvaluator(_BatchedRunnable, RunnableSerializable):model: Optional[str] = Nonemax_len: Optional[int] = Nonehead_max_len: Optional[int] = None+ lang: Optional[str] = None+ min_confidence: Optional[float] = Nonehooks: Optional[Any] = Noneon_predict_start: Optional[Any] = Noneon_predict_end: Optional[Any] = None@@ -851,6 +896,8 @@ class LayaEvaluator(_BatchedRunnable, RunnableSerializable):model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -868,6 +915,8 @@ class LayaEvaluator(_BatchedRunnable, RunnableSerializable):model=model,max_len=max_len,head_max_len=head_max_len,+ lang=lang,+ min_confidence=min_confidence,hooks=hooks,on_predict_start=on_predict_start,on_predict_end=on_predict_end,@@ -884,6 +933,8 @@ class LayaEvaluator(_BatchedRunnable, RunnableSerializable):self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -902,6 +953,8 @@ class LayaEvaluator(_BatchedRunnable, RunnableSerializable):model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,@@ -927,6 +980,8 @@ class LayaEvaluator(_BatchedRunnable, RunnableSerializable):model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,diff --git a/laya/integrations/llamaindex.py b/laya/integrations/llamaindex.pyindex 260df8f3..057e1361 100644--- a/laya/integrations/llamaindex.py+++ b/laya/integrations/llamaindex.py@@ -64,6 +64,7 @@ except ImportError:# `predict` call and the same laya-serve request body, and three copies of the rule is how two of# them came to forward only `model`.from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs # noqa: E402+from ._controls import decision_kwargs as _decision_kwargs # noqa: E402from ._controls import predict_kwargs as _predict_kwargs, reject_remote_hooks as _reject_remote_hooks # noqa: E402# One class for every integration, so `except LayaLowConfidenceError` catches all of them.@@ -130,11 +131,15 @@ def _call_remote(timeout: float = 10.0,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,) -> Dict[str, Any]:"""Send decision request to a remote laya-serve HTTP instance using standard library urllib.`max_len` / `head_max_len` travel in the body; laya-serve applies them up to its- `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422.+ `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422. `lang` / `min_confidence`+ ride in the same body (they are laya-serve `BODY_CONTROLS` too): the language codes the state+ is routed and answered in, and core's abstention gate."""url = base_url.rstrip("/")if not url.endswith("/v1/systemone"):@@ -147,6 +152,10 @@ def _call_remote(payload["max_len"] = max_lenif head_max_len is not None:payload["head_max_len"] = head_max_len+ if lang is not None:+ payload["lang"] = lang+ if min_confidence is not None:+ payload["min_confidence"] = min_confidencedata = json.dumps(payload).encode("utf-8")headers = {"Content-Type": "application/json"}@@ -188,6 +197,8 @@ def _execute_decision(model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -203,9 +214,11 @@ def _execute_decision(if base_url:_reject_remote_hooks(hook_kwargs, base_url)budget = _budget_kwargs(max_len, head_max_len)- return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)+ decision = _decision_kwargs(lang, min_confidence)+ return _call_remote(base_url, state, questions, api_key=api_key, model=model,+ **budget, **decision)runner = agent if agent is not None else _get_default_router()- kwargs = _predict_kwargs(model, max_len, head_max_len)+ kwargs = _predict_kwargs(model, max_len, head_max_len, lang, min_confidence)kwargs.update(hook_kwargs)return runner.predict(state, questions, **kwargs)@@ -230,6 +243,8 @@ class LayaSingleSelector(BaseSelector):model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -246,6 +261,8 @@ class LayaSingleSelector(BaseSelector):self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -282,6 +299,8 @@ class LayaSingleSelector(BaseSelector):model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,@@ -370,6 +389,8 @@ class LayaMultiSelector(BaseSelector):model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -386,6 +407,8 @@ class LayaMultiSelector(BaseSelector):self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -421,6 +444,8 @@ class LayaMultiSelector(BaseSelector):model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,@@ -519,6 +544,8 @@ class LayaQueryRouter:model: Optional[str] = None,max_len: Optional[int] = None,head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,hooks: Optional[Any] = None,on_predict_start: Optional[Any] = None,on_predict_end: Optional[Any] = None,@@ -536,6 +563,8 @@ class LayaQueryRouter:self.model = modelself.max_len = max_lenself.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidenceself.hooks = hooksself.on_predict_start = on_predict_startself.on_predict_end = on_predict_end@@ -568,6 +597,8 @@ class LayaQueryRouter:model=self.model,max_len=self.max_len,head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,hooks=self.hooks,on_predict_start=self.on_predict_start,on_predict_end=self.on_predict_end,diff --git a/tests/test_crewai.py b/tests/test_crewai.pyindex bebbaa32..fb7a3e60 100644--- a/tests/test_crewai.py+++ b/tests/test_crewai.py@@ -298,7 +298,8 @@ from laya.integrations import langchain as langchain_modulefrom laya.integrations import llamaindex as llamaindex_modulefrom laya.router import Router-CONTROLS = tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.HOOK_CONTROLS)+CONTROLS = (tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.DECISION_CONTROLS)+ + tuple(_controls.HOOK_CONTROLS))def _params(fn):@@ -308,6 +309,8 @@ def _params(fn):# The shared tuples name the arguments the shared builders accept, in both directions.check("controls/budget tuple names budget_kwargs",set(_params(_controls.budget_kwargs)), set(_controls.PREDICT_CONTROLS))+check("controls/decision tuple names decision_kwargs",+ set(_params(_controls.decision_kwargs)), set(_controls.DECISION_CONTROLS))check("controls/hook tuple names hook_kwargs",set(_params(_controls.hook_kwargs)), set(_controls.HOOK_CONTROLS))@@ -354,7 +357,8 @@ class RecordingAgent:}-ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "hooks": ["H"], "on_predict_start": "S",+ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "lang": "fr", "min_confidence": 0.4,+ "hooks": ["H"], "on_predict_start": "S","on_predict_end": "E", "hooks_raise": True, "hooks_timeout": 0.5}@@ -384,13 +388,16 @@ def guard_call(**controls):for label, call in (("router", crew_call), ("guard", guard_call)):check("controls/%s with nothing set sends nothing" % label, call(), {})- check("controls/%s forwards all seven" % label, call(**ALL_CONTROLS), ALL_CONTROLS)+ check("controls/%s forwards every control" % label, call(**ALL_CONTROLS), ALL_CONTROLS)check("controls/%s forwards one budget alone" % label, call(head_max_len=256),{"head_max_len": 256})+ check("controls/%s forwards lang alone" % label, call(lang="fr"), {"lang": "fr"})+ check("controls/%s forwards min_confidence alone" % label, call(min_confidence=0.4),+ {"min_confidence": 0.4})# 0 and [] are decisions, not absences: truthiness tests here would drop them.check("controls/%s keeps falsy values" % label,- call(head_max_len=0, hooks=[], hooks_raise=False),- {"head_max_len": 0, "hooks": [], "hooks_raise": False})+ call(head_max_len=0, hooks=[], hooks_raise=False, min_confidence=0.0),+ {"head_max_len": 0, "hooks": [], "hooks_raise": False, "min_confidence": 0.0})check("controls/%s alongside model" % label,call(model="laya-multilingual", max_len=1024),{"model": "laya-multilingual", "max_len": 1024})@@ -426,6 +433,16 @@ check("controls/remote body omits unset budgets",[k for k in remote_body({}) if k in _controls.PREDICT_CONTROLS], [])check("controls/remote body keeps a zero",remote_body({"head_max_len": 0}).get("head_max_len"), 0)+# `lang` / `min_confidence` are laya-serve `BODY_CONTROLS` too, so the same override reaches the+# remote node -- and an unset one stays out of the body rather than shadowing the deployment.+check("controls/remote body carries the decision controls",+ {k: v for k, v in remote_body({"lang": "es", "min_confidence": 0.3}).items()+ if k in _controls.DECISION_CONTROLS},+ {"lang": "es", "min_confidence": 0.3})+check("controls/remote body omits unset decision controls",+ [k for k in remote_body({}) if k in _controls.DECISION_CONTROLS], [])+check("controls/remote body keeps min_confidence=0.0",+ remote_body({"min_confidence": 0.0}).get("min_confidence"), 0.0)# A hook is a Python callable that runs inside `predict`; a serve node cannot receive one. Saying# so beats reporting success after never calling it.diff --git a/tests/test_langchain.py b/tests/test_langchain.pyindex e56c09ad..1a3fd79d 100644--- a/tests/test_langchain.py+++ b/tests/test_langchain.py@@ -992,6 +992,120 @@ finally:langchain_module.urllib.request.build_opener = _real_build_opener+# --------------------------------------------------------------- 7b. Per-request lang / abstention+#+# `lang` and `min_confidence` are core's language routing and abstention gate (#361). Both are+# read by `Agent.predict`/`system_one` AND `Router.predict`, and both are laya-serve+# `BODY_CONTROLS`, so they are safe to forward on the local and the remote path alike -- unlike+# `task` / `lang_guess`, which are Router-only. A wrapper that drops them cannot route a+# non-English state or measure the abstention gate it is supposed to honor.+from laya.integrations import _controls # noqa: E402+import inspect # noqa: E402+from laya.agent import Agent # noqa: E402+from laya.router import Router # noqa: E402++DECISION_CRITERIA = {"billing": "invoices", "tech": "bugs"}+DECISION_NODES = (+ ("router", lambda a, **kw: LayaRouter(DECISION_CRITERIA, agent=a, **kw)),+ ("guardrail", lambda a, **kw: LayaGuardrail(+ questions={"jailbreak": {"type": "noul", "instructions": "jailbreak?"}}, agent=a, **kw)),+ ("triage", lambda a, **kw: LayaTriage(agent=a, **kw)),+ ("evaluator", lambda a, **kw: LayaEvaluator(+ questions={"faithful": {"type": "noul", "instructions": "faithful?"}}, agent=a, **kw)),+)++check("decision/_controls names the tuple", _controls.DECISION_CONTROLS, ("lang", "min_confidence"))+for _c in _controls.DECISION_CONTROLS:+ check_true("decision/%s accepted by Agent.system_one" % _c,+ _c in set(inspect.signature(Agent.system_one).parameters))+ check_true("decision/%s accepted by Router.predict" % _c,+ _c in set(inspect.signature(Router.predict).parameters))++for name, build in DECISION_NODES:+ plain = BudgetAgent()+ build(plain).invoke("some state")+ check("decision/%s default sends nothing" % name, plain.kwargs[0], {})++ both = BudgetAgent()+ build(both, lang="fr", min_confidence=0.4).invoke("some state")+ check("decision/%s forwards both" % name, both.kwargs[0],+ {"lang": "fr", "min_confidence": 0.4})++ # Each knob works alone; an override built from one must not carry the other.+ one = BudgetAgent()+ build(one, lang="es").invoke("some state")+ check("decision/%s forwards lang alone" % name, one.kwargs[0], {"lang": "es"})++ # A 0.0 abstention gate is a real decision ("abstain over nothing"), not an absence.+ zero = BudgetAgent()+ build(zero, min_confidence=0.0).invoke("some state")+ check("decision/%s keeps min_confidence=0.0" % name, zero.kwargs[0], {"min_confidence": 0.0})++ mixed = BudgetAgent()+ build(mixed, model="laya-multilingual", lang="de").invoke("some state")+ check("decision/%s with model" % name, mixed.kwargs[0],+ {"model": "laya-multilingual", "lang": "de"})+++# A remote node forwards them in the request body instead of dropping them.+decision_remote_calls = []+_decision_real_call_remote = langchain_module._call_remote+++def decision_spy_call_remote(base_url, state, questions, api_key=None, model=None, **extras):+ decision_remote_calls.append(dict({"model": model}, **extras))+ return {"answers": {"route": {"type": "choice", "choice": "billing", "confidence": 0.9}}}+++langchain_module._call_remote = decision_spy_call_remote+try:+ for field, value in (("lang", "it"), ("min_confidence", 0.25), ("min_confidence", 0.0)):+ LayaRouter(DECISION_CRITERIA, base_url="http://laya:8000", **{field: value}).invoke("x")+ check("decision/remote forwards %s=%r" % (field, value), decision_remote_calls[-1],+ {"model": None, field: value})+ # Nothing set sends nothing.+ LayaRouter(DECISION_CRITERIA, base_url="http://laya:8000").invoke("x")+ check_true("decision/remote omits unset decision controls",+ not any(k in decision_remote_calls[-1] for k in _controls.DECISION_CONTROLS),+ repr(decision_remote_calls[-1]))+finally:+ langchain_module._call_remote = _decision_real_call_remote++# ...and the real `_call_remote` writes them into the JSON body.+_decision_sent = {}+++class _DecisionFakeResponse:+ def __enter__(self):+ return self++ def __exit__(self, *exc):+ return False++ def read(self):+ return b'{"answers": {}}'+++class _DecisionFakeOpener:+ def open(self, req, timeout=None):+ _decision_sent["body"] = json.loads(req.data.decode("utf-8"))+ return _DecisionFakeResponse()+++_decision_real_build_opener = langchain_module.urllib.request.build_opener+langchain_module.urllib.request.build_opener = lambda *a, **k: _DecisionFakeOpener()+try:+ langchain_module._call_remote("http://laya:8000", "x", {}, lang="fr", min_confidence=0.0)+ check("decision/_call_remote sends lang", _decision_sent["body"].get("lang"), "fr")+ check("decision/_call_remote keeps min_confidence=0.0",+ _decision_sent["body"].get("min_confidence"), 0.0)+ langchain_module._call_remote("http://laya:8000", "x", {})+ check_true("decision/_call_remote omits unset decision controls",+ "lang" not in _decision_sent["body"] and "min_confidence" not in _decision_sent["body"])+finally:+ langchain_module.urllib.request.build_opener = _decision_real_build_opener++# --------------------------------------------------------------- 7. Prediction hooksfrom laya.integrations import langchain as langchain_module # noqa: E402from laya.hooks import PredictContext # noqa: E402diff --git a/tests/test_llamaindex.py b/tests/test_llamaindex.pyindex dcfbb4ce..09718f76 100644--- a/tests/test_llamaindex.py+++ b/tests/test_llamaindex.py@@ -409,7 +409,8 @@ from laya.integrations import langchain as langchain_modulefrom laya.integrations import llamaindex as llamaindex_modulefrom laya.router import Router-CONTROLS = tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.HOOK_CONTROLS)+CONTROLS = (tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.DECISION_CONTROLS)+ + tuple(_controls.HOOK_CONTROLS))def _params(fn):@@ -418,6 +419,8 @@ def _params(fn):check("controls/budget tuple names budget_kwargs",set(_params(_controls.budget_kwargs)), set(_controls.PREDICT_CONTROLS))+check("controls/decision tuple names decision_kwargs",+ set(_params(_controls.decision_kwargs)), set(_controls.DECISION_CONTROLS))check("controls/hook tuple names hook_kwargs",set(_params(_controls.hook_kwargs)), set(_controls.HOOK_CONTROLS))@@ -464,7 +467,8 @@ class RecordingAgent:}-ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "hooks": ["H"], "on_predict_start": "S",+ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "lang": "fr", "min_confidence": 0.4,+ "hooks": ["H"], "on_predict_start": "S","on_predict_end": "E", "hooks_raise": True, "hooks_timeout": 0.5}# One entry per class that can take a decision here: (label, class, constructor extras, one call).@@ -508,14 +512,19 @@ def remote_body(cls, extra, run, **controls):for _label, _cls, _extra, _run in SURFACES:check("controls/%s with nothing set sends nothing" % _label,local_kwargs(_cls, _extra, _run), {})- check("controls/%s forwards all seven" % _label,+ check("controls/%s forwards every control" % _label,local_kwargs(_cls, _extra, _run, **ALL_CONTROLS), ALL_CONTROLS)check("controls/%s forwards one budget alone" % _label,local_kwargs(_cls, _extra, _run, head_max_len=256), {"head_max_len": 256})+ check("controls/%s forwards lang alone" % _label,+ local_kwargs(_cls, _extra, _run, lang="fr"), {"lang": "fr"})+ check("controls/%s forwards min_confidence alone" % _label,+ local_kwargs(_cls, _extra, _run, min_confidence=0.4), {"min_confidence": 0.4})# 0 and [] are decisions, not absences: truthiness tests here would drop them.check("controls/%s keeps falsy values" % _label,- local_kwargs(_cls, _extra, _run, head_max_len=0, hooks=[], hooks_raise=False),- {"head_max_len": 0, "hooks": [], "hooks_raise": False})+ local_kwargs(_cls, _extra, _run, head_max_len=0, hooks=[], hooks_raise=False,+ min_confidence=0.0),+ {"head_max_len": 0, "hooks": [], "hooks_raise": False, "min_confidence": 0.0})check("controls/%s alongside model" % _label,local_kwargs(_cls, _extra, _run, model="laya-multilingual", max_len=1024),{"model": "laya-multilingual", "max_len": 1024})@@ -528,6 +537,16 @@ for _label, _cls, _extra, _run in SURFACES:[k for k in remote_body(_cls, _extra, _run) if k in _controls.PREDICT_CONTROLS], [])check("controls/%s remote body keeps a zero" % _label,remote_body(_cls, _extra, _run, head_max_len=0).get("head_max_len"), 0)+ # `lang` / `min_confidence` are laya-serve `BODY_CONTROLS`, so the same override reaches the+ # remote node -- and an unset one stays out of the body rather than shadowing the deployment.+ check("controls/%s remote body carries the decision controls" % _label,+ {k: v for k, v in remote_body(_cls, _extra, _run, lang="es", min_confidence=0.3).items()+ if k in _controls.DECISION_CONTROLS},+ {"lang": "es", "min_confidence": 0.3})+ check("controls/%s remote body omits unset decision controls" % _label,+ [k for k in remote_body(_cls, _extra, _run) if k in _controls.DECISION_CONTROLS], [])+ check("controls/%s remote body keeps min_confidence=0.0" % _label,+ remote_body(_cls, _extra, _run, min_confidence=0.0).get("min_confidence"), 0.0)# A hook is a Python callable that runs inside `predict`; a serve node cannot receive one.for _c, _sample in HOOK_SAMPLES:try: