Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(integrations): forward token budgets and hooks from CrewAI and LlamaIndex
Aashish0b3ee71bd539
diff --git a/docs/crewai.md b/docs/crewai.mdindex 497e412a..0c196b22 100644--- a/docs/crewai.md+++ b/docs/crewai.md@@ -5,6 +5,8 @@ Laya provides sub-35ms, non-autoregressive decision components for **CrewAI** mu* **`LayaCrewRouter`**: Sub-35ms task delegation router replacing LLM managers in hierarchical crews.* **`LayaTaskGuard`**: Pre-execution task guardrail screening prompts and instructions for jailbreaks, injections, and policy violations.+Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) -- see [Per-call decision controls](#5-per-call-decision-controls).+Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.---@@ -121,3 +123,55 @@ router = LayaCrewRouter(```The remote client uses Python's standard library `urllib` with zero heavy dependencies, preventing cross-origin credential forwarding and matching the `/v1/systemone` specification.++---++## 5. Per-call decision controls++`LayaCrewRouter` and `LayaTaskGuard` take the same per-call arguments the core API does: the two+token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`,+`on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are per instance, so a+crew with a large roster can be given room while the rest of the pipeline keeps the checkpoint's+defaults.++A delegation choice shares the checkpoint's *option* budget -- `head_max_len`, 192 tokens on `laya`+-- and every candidate contributes its role and goal, so past roughly 20 agents the later goals+start reaching the model as the same truncated text.++```python+router = LayaCrewRouter(+ confidence_threshold=0.80,+ max_len=1024, # total window+ head_max_len=512, # tokens shared by the roster+)++decision = router.route(task, agents) # agents: 59 candidates+```++Measured with `laya` on Apple silicon, one forward pass per task, scored on the delegated agent,+against a 59-agent roster built from the MASSIVE en intent labels (role only, goal left empty) with+one utterance per label, so ground truth is exact. Each cell is how many of the 59 tasks were given+to their own agent; both repeats gave the same count.++| 59-agent roster | Default budget | `max_len=1024, head_max_len=384` | `…, head_max_len=512` |+|---|---|---|---|+| Tasks on their own agent | 2/59 | 7/59 | 16/59 |+| Median ms per task | 146 | 190 | 265 |++Before this, the same run could not be asked at all: `LayaCrewRouter.__init__() got an unexpected+keyword argument 'max_len'`.++Absolute accuracy is not the claim here -- the checkpoint is not a MASSIVE classifier and 59 similar+labels are a stress shape. The claim is reachability and price: a roster the default budget collapses+to near-nothing is readable from a crew, and at this size the wider window costs little time. Note+the middle row is 7/59 where the identical criteria sent straight to `Router.predict` scored 8/59;+only the `instructions` sentence differed, which is the expected sensitivity of a choice question to+its own wording. With fewer than about 20 candidates the roles already fit and widening can move+answers the wrong way, which is why both budgets are opt-in per instance. See the [LangChain+integration](langchain.md#7-widening-the-token-budget-for-many-options) for that measured cliff.++**Hooks run on the local path only.** A router or guard with a `base_url` and `hooks=[...]` raises+`ValueError` rather than reporting a success whose hook never ran -- a hook is a Python callable that+executes inside `predict`, and no wire format carries it. Install hooks in the process that runs+inference. The two budgets do travel to a remote node, in the request body, up to its+`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.diff --git a/docs/llamaindex.md b/docs/llamaindex.mdindex 17ebdf59..f75eea4a 100644--- a/docs/llamaindex.md+++ b/docs/llamaindex.md@@ -139,3 +139,49 @@ selector = LayaSingleSelector(```The remote client uses Python's standard library `urllib` with zero heavy dependencies, preventing cross-origin credential forwarding and matching the `/v1/systemone` specification.++---++## 6. Per-call decision controls++`LayaSingleSelector`, `LayaMultiSelector` and `LayaQueryRouter` take the same per-call arguments the+core API does: the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook+arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are+per selector, so a wide routing step can be given room while the rest of the pipeline keeps the+checkpoint's defaults.++A choice question's options share the checkpoint's *option* budget -- `head_max_len`, 192 tokens on+`laya` -- and every candidate contributes its name and description, so past roughly 20 tools the+descriptions start reaching the model as the same text.++```python+selector = LayaSingleSelector(+ instructions="Which tool or query engine is best suited to answer this query?",+ max_len=1024, # total window+ head_max_len=512, # tokens shared by the option prompt+)+result = selector.select(tools, query) # tools: 59 descriptions+```++Measured on `laya` (Apple silicon, one forward pass per query, scored on the chosen tool) with a+59-tool roster built from the MASSIVE en intent labels and one utterance per label, so ground truth+is exact. Each cell is how many of the 59 queries reached their own tool; both repeats gave the+same count.++| 59-tool roster | Default budget | `max_len=1024, head_max_len=384` | `…, head_max_len=512` |+|---|---|---|---|+| Queries on their own tool | 2/59 | 8/59 | 15/59 |+| Median ms per query | 160 | 172 | 184 |++Absolute accuracy is not the claim here: the checkpoint is not a MASSIVE classifier, and 59 similar+labels are a stress shape. The claim is the direction and the price -- a roster the default budget+collapses to near-nothing is readable, and at this size the window costs little time. With fewer+than about 20 options the labels already fit and widening can move answers the wrong way, which is+why both arguments are opt-in per selector. See the [LangChain+integration](langchain.md#7-widening-the-token-budget-for-many-options) for that measured cliff.++**Hooks run on the local path only.** A selector with a `base_url` and `hooks=[...]` raises+`ValueError` rather than reporting a success whose cache never ran -- a hook is a Python callable+that runs inside `predict`, and no wire format carries it. Install hooks in the process that runs+inference. The two budgets do travel to a remote node, in the request body, up to its+`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.diff --git a/laya/integrations/_controls.py b/laya/integrations/_controls.pynew file mode 100644index 00000000..a8b1caaa--- /dev/null+++ b/laya/integrations/_controls.py@@ -0,0 +1,74 @@+"""Per-call decision controls shared by the framework integrations.++Every integration wrapper ends at one of two places: a local `runner.predict(state, questions,+**overrides)` call, or a POST to a laya-serve node. Which overrides those two accept is a single+rule, and three independent copies of it drift -- this module is the copy the LangChain, CrewAI+and LlamaIndex wrappers import.++The names are listed once, in `PREDICT_CONTROLS` and `HOOK_CONTROLS`, and the signatures below+are checked against them by `tests/test_crewai.py` and `tests/test_llamaindex.py`, which also+assert every wrapper's constructor accepts all of them. A control added here without reaching a+wrapper's `__init__` fails that wrapper's suite rather than being dropped silently.+"""+from __future__ import annotations++from typing import Any, Dict, Optional++# The token-budget overrides `Agent.predict` / `Router.predict` take per call, and that+# laya-serve accepts in the request body up to its `LAYA_MAX_TOKEN_BUDGET` ceiling.+PREDICT_CONTROLS = ("max_len", "head_max_len")++# The per-call hook family. These are Python callables and flags that run inside `predict`, so+# they exist only on the local path -- see `reject_remote_hooks`.+HOOK_CONTROLS = ("hooks", "on_predict_start", "on_predict_end", "hooks_raise", "hooks_timeout")+++def budget_kwargs(max_len: Optional[int] = None,+ head_max_len: Optional[int] = None) -> Dict[str, Any]:+ """The two token budgets, with the unset ones omitted.++ Absent rather than `None`: on the local path a `None` budget would override the checkpoint's+ own default with nothing, and on the remote path an older stand-in `_call_remote` -- including+ the ones these integrations' own tests install -- must keep accepting the call.+ """+ return {key: value for key, value in (("max_len", max_len), ("head_max_len", head_max_len))+ if value is not None}+++def predict_kwargs(model: Optional[str] = None, max_len: Optional[int] = None,+ head_max_len: Optional[int] = None) -> Dict[str, Any]:+ """The per-request overrides a local runner accepts, with the unset ones omitted."""+ kwargs: Dict[str, Any] = {}+ if model:+ kwargs["model"] = model+ kwargs.update(budget_kwargs(max_len, head_max_len))+ return kwargs+++def hook_kwargs(hooks: Optional[Any] = None, on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None, hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None) -> Dict[str, Any]:+ """The per-call hook overrides, with the unset ones omitted.++ Core reads `None` as "inherit whatever the runner was built with", so an unset hook has to be+ absent rather than passed as `None`. Note the `is not None` tests: `hooks=[]` means "no hooks+ for this call", and `hooks_raise=False` means "keep deciding after a hook fails" -- both are+ decisions a caller made, not absences.+ """+ given = {"hooks": hooks, "on_predict_start": on_predict_start, "on_predict_end": on_predict_end,+ "hooks_raise": hooks_raise, "hooks_timeout": hooks_timeout}+ return {k: v for k, v in given.items() if v is not None}+++def reject_remote_hooks(given: Dict[str, Any], base_url: Optional[str]) -> None:+ """Refuse hooks on a remote node rather than dropping them silently.++ A hook is a Python callable that runs inside `predict` -- it can cache a decision, gate one or+ rewrite its state. `laya-serve` has no way to receive or run one, so a node with a `base_url`+ and hooks configured would report success while never calling them.+ """+ if base_url and given:+ raise ValueError(+ "%s run in the local runner and cannot be sent to a laya-serve endpoint; "+ "install them where serve runs, or drop them" % ", ".join(sorted(given))+ )diff --git a/laya/integrations/crewai.py b/laya/integrations/crewai.pyindex 5c154c71..435d96c1 100644--- a/laya/integrations/crewai.py+++ b/laya/integrations/crewai.py@@ -51,6 +51,12 @@ class LayaTaskGuardError(ValueError):self.raw_decision = raw_decision+# The per-call control rules are shared with the other two integrations: they end at the same+# `predict` call and the same laya-serve request body, and three copies of the rule is how two of+# them came to forward only `model`.+from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs # noqa: E402+from ._controls import predict_kwargs as _predict_kwargs, reject_remote_hooks as _reject_remote_hooks # noqa: E402+# One class for every integration, so `except LayaLowConfidenceError` catches all of them.from ._errors import LayaLowConfidenceError # noqa: E402@@ -138,8 +144,14 @@ def _call_remote(api_key: Optional[str] = None,model: Optional[str] = None,timeout: float = 10.0,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,) -> Dict[str, Any]:- """Send decision request to a remote laya-serve HTTP instance using standard library urllib."""+ """Send decision request to a remote laya-serve HTTP instance using standard library urllib.++ `max_len` / `head_max_len` travel in the body; laya-serve applies them up to its+ `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422.+ """url = base_url.rstrip("/")if not url.endswith("/v1/systemone"):url = f"{url}/v1/systemone"@@ -147,6 +159,10 @@ def _call_remote(payload: Dict[str, Any] = {"state": state, "questions": questions}if model:payload["model"] = model+ if max_len is not None:+ payload["max_len"] = max_len+ if head_max_len is not None:+ payload["head_max_len"] = head_max_lendata = json.dumps(payload).encode("utf-8")headers = {"Content-Type": "application/json"}@@ -186,11 +202,27 @@ def _execute_decision(base_url: Optional[str] = None,api_key: Optional[str] = None,model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,) -> Dict[str, Any]:+ """One decision on the local runner or a laya-serve node, with the per-call controls kept.++ Which controls either path accepts is `._controls`' rule, shared with the LangChain and+ LlamaIndex wrappers -- they end at the same `predict` call and the same request body.+ """+ hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)if base_url:- return _call_remote(base_url, state, questions, api_key=api_key, model=model)+ _reject_remote_hooks(hook_kwargs, base_url)+ budget = _budget_kwargs(max_len, head_max_len)+ return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)runner = agent if agent is not None else _get_default_router()- kwargs = {"model": model} if model else {}+ kwargs = _predict_kwargs(model, max_len, head_max_len)+ kwargs.update(hook_kwargs)return runner.predict(state, questions, **kwargs)@@ -211,6 +243,13 @@ class LayaCrewRouter:base_url: Optional[str] = None,api_key: Optional[str] = None,model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,):self.instructions = instructionsself.confidence_threshold = confidence_threshold@@ -220,6 +259,13 @@ class LayaCrewRouter:self.base_url = base_urlself.api_key = api_keyself.model = model+ self.max_len = max_len+ self.head_max_len = head_max_len+ self.hooks = hooks+ self.on_predict_start = on_predict_start+ self.on_predict_end = on_predict_end+ self.hooks_raise = hooks_raise+ self.hooks_timeout = hooks_timeoutself.last_decision: Optional[Dict[str, Any]] = Nonedef route(@@ -249,6 +295,13 @@ class LayaCrewRouter:base_url=self.base_url,api_key=self.api_key,model=self.model,+ max_len=self.max_len,+ head_max_len=self.head_max_len,+ hooks=self.hooks,+ on_predict_start=self.on_predict_start,+ on_predict_end=self.on_predict_end,+ hooks_raise=self.hooks_raise,+ hooks_timeout=self.hooks_timeout,)self.last_decision = res@@ -359,6 +412,13 @@ class LayaTaskGuard:base_url: Optional[str] = None,api_key: Optional[str] = None,model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,):self.questions = questionsself.action = action@@ -368,6 +428,13 @@ class LayaTaskGuard:self.base_url = base_urlself.api_key = api_keyself.model = model+ self.max_len = max_len+ self.head_max_len = head_max_len+ self.hooks = hooks+ self.on_predict_start = on_predict_start+ self.on_predict_end = on_predict_end+ self.hooks_raise = hooks_raise+ self.hooks_timeout = hooks_timeoutself.last_decision: Optional[Dict[str, Any]] = Nonedef _default_questions(self) -> Dict[str, Any]:@@ -386,6 +453,13 @@ class LayaTaskGuard:base_url=self.base_url,api_key=self.api_key,model=self.model,+ max_len=self.max_len,+ head_max_len=self.head_max_len,+ hooks=self.hooks,+ on_predict_start=self.on_predict_start,+ on_predict_end=self.on_predict_end,+ hooks_raise=self.hooks_raise,+ hooks_timeout=self.hooks_timeout,)self.last_decision = resanswers = res.get("answers", {})diff --git a/laya/integrations/langchain.py b/laya/integrations/langchain.pyindex 9285f04d..c46d6847 100644--- a/laya/integrations/langchain.py+++ b/laya/integrations/langchain.py@@ -26,6 +26,13 @@ except ImportError:RunnableSerializable = object # type: ignoreRunnableConfig = Any # type: ignore+# The per-call control rules live in `._controls` because the CrewAI and LlamaIndex wrappers end+# at the same `predict` call and the same laya-serve body; keeping the rule in three places is how+# two of them ended up forwarding only `model`.+from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs+from ._controls import predict_kwargs as _predict_kwargs+from ._controls import reject_remote_hooks as _reject_remote_hooks+class LayaGuardrailError(ValueError):"""Raised when an input violates a Laya guardrail policy."""@@ -156,48 +163,6 @@ def _get_default_router():return _DEFAULT_ROUTER-def _predict_kwargs(model: Optional[str] = None, max_len: Optional[int] = None,- head_max_len: Optional[int] = None) -> Dict[str, Any]:- """The per-request overrides a local runner accepts, with the unset ones omitted."""- kwargs: Dict[str, Any] = {}- if model:- kwargs["model"] = model- if max_len is not None:- kwargs["max_len"] = max_len- if head_max_len is not None:- kwargs["head_max_len"] = head_max_len- return kwargs---def _reject_remote_hooks(hook_kwargs: Dict[str, Any], base_url: Optional[str]) -> None:- """Refuse hooks on a remote node rather than dropping them silently.-- A hook is a Python callable that runs inside `predict` -- it can cache a decision, gate one or- rewrite its state. `laya-serve` has no way to receive or run one, so a node with a `base_url`- and hooks configured would report success while never calling them.- """- if base_url and hook_kwargs:- raise ValueError(- "%s run in the local runner and cannot be sent to a laya-serve endpoint; "- "install them where serve runs, or drop them" % ", ".join(sorted(hook_kwargs))- )---def _hook_kwargs(hooks: Optional[Any] = None, on_predict_start: Optional[Any] = None,- on_predict_end: Optional[Any] = None, hooks_raise: Optional[bool] = None,- hooks_timeout: Optional[float] = None) -> Dict[str, Any]:- """The per-call hook overrides, with the unset ones omitted.-- Core reads `None` as "inherit whatever the runner was built with", so an unset hook has to be- absent rather than passed as `None`. Note the `is not None` tests: `hooks=[]` means "no hooks- for this call", and `hooks_raise=False` means "keep deciding after a hook fails" -- both are- decisions a caller made, not absences.- """- given = {"hooks": hooks, "on_predict_start": on_predict_start, "on_predict_end": on_predict_end,- "hooks_raise": hooks_raise, "hooks_timeout": hooks_timeout}- return {k: v for k, v in given.items() if v is not None}--def _execute_decision(state: Any,questions: Dict[str, Any],@@ -216,9 +181,7 @@ def _execute_decision(hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)if base_url:_reject_remote_hooks(hook_kwargs, base_url)- # Only what was set, so a stand-in `_call_remote` without the budget keywords still works.- budget = {k: v for k, v in (("max_len", max_len), ("head_max_len", head_max_len))- if v is not None}+ budget = _budget_kwargs(max_len, head_max_len)return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)runner = agent if agent is not None else _get_default_router()kwargs = _predict_kwargs(model, max_len, head_max_len)diff --git a/laya/integrations/llamaindex.py b/laya/integrations/llamaindex.pyindex 6e7615a2..260df8f3 100644--- a/laya/integrations/llamaindex.py+++ b/laya/integrations/llamaindex.py@@ -60,6 +60,12 @@ except ImportError:return self.query_str+# The per-call control rules are shared with the other two integrations: they end at the same+# `predict` call and the same laya-serve request body, and three copies of the rule is how two of+# them came to forward only `model`.+from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs # noqa: E402+from ._controls import predict_kwargs as _predict_kwargs, reject_remote_hooks as _reject_remote_hooks # noqa: E402+# One class for every integration, so `except LayaLowConfidenceError` catches all of them.from ._errors import LayaLowConfidenceError # noqa: E402@@ -122,8 +128,14 @@ def _call_remote(api_key: Optional[str] = None,model: Optional[str] = None,timeout: float = 10.0,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,) -> Dict[str, Any]:- """Send decision request to a remote laya-serve HTTP instance using standard library urllib."""+ """Send decision request to a remote laya-serve HTTP instance using standard library urllib.++ `max_len` / `head_max_len` travel in the body; laya-serve applies them up to its+ `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422.+ """url = base_url.rstrip("/")if not url.endswith("/v1/systemone"):url = f"{url}/v1/systemone"@@ -131,6 +143,10 @@ def _call_remote(payload: Dict[str, Any] = {"state": state, "questions": questions}if model:payload["model"] = model+ if max_len is not None:+ payload["max_len"] = max_len+ if head_max_len is not None:+ payload["head_max_len"] = head_max_lendata = json.dumps(payload).encode("utf-8")headers = {"Content-Type": "application/json"}@@ -170,11 +186,27 @@ def _execute_decision(base_url: Optional[str] = None,api_key: Optional[str] = None,model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,) -> Dict[str, Any]:+ """One decision on the local runner or a laya-serve node, with the per-call controls kept.++ Which controls either path accepts is `._controls`' rule, shared with the LangChain and+ CrewAI wrappers -- they end at the same `predict` call and the same request body.+ """+ hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)if base_url:- return _call_remote(base_url, state, questions, api_key=api_key, model=model)+ _reject_remote_hooks(hook_kwargs, base_url)+ budget = _budget_kwargs(max_len, head_max_len)+ return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)runner = agent if agent is not None else _get_default_router()- kwargs = {"model": model} if model else {}+ kwargs = _predict_kwargs(model, max_len, head_max_len)+ kwargs.update(hook_kwargs)return runner.predict(state, questions, **kwargs)@@ -196,6 +228,13 @@ class LayaSingleSelector(BaseSelector):base_url: Optional[str] = None,api_key: Optional[str] = None,model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,):self.instructions = instructionsself.confidence_threshold = confidence_threshold@@ -205,6 +244,13 @@ class LayaSingleSelector(BaseSelector):self.base_url = base_urlself.api_key = api_keyself.model = model+ self.max_len = max_len+ self.head_max_len = head_max_len+ self.hooks = hooks+ self.on_predict_start = on_predict_start+ self.on_predict_end = on_predict_end+ self.hooks_raise = hooks_raise+ self.hooks_timeout = hooks_timeoutself.last_decision: Optional[Dict[str, Any]] = Nonedef _select(@@ -234,6 +280,13 @@ class LayaSingleSelector(BaseSelector):base_url=self.base_url,api_key=self.api_key,model=self.model,+ max_len=self.max_len,+ head_max_len=self.head_max_len,+ hooks=self.hooks,+ on_predict_start=self.on_predict_start,+ on_predict_end=self.on_predict_end,+ hooks_raise=self.hooks_raise,+ hooks_timeout=self.hooks_timeout,)self.last_decision = res@@ -315,6 +368,13 @@ class LayaMultiSelector(BaseSelector):base_url: Optional[str] = None,api_key: Optional[str] = None,model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,):self.instructions = instructionsself.max_outputs = max_outputs@@ -324,6 +384,13 @@ class LayaMultiSelector(BaseSelector):self.base_url = base_urlself.api_key = api_keyself.model = model+ self.max_len = max_len+ self.head_max_len = head_max_len+ self.hooks = hooks+ self.on_predict_start = on_predict_start+ self.on_predict_end = on_predict_end+ self.hooks_raise = hooks_raise+ self.hooks_timeout = hooks_timeoutself.last_decision: Optional[Dict[str, Any]] = Nonedef _select(@@ -352,6 +419,13 @@ class LayaMultiSelector(BaseSelector):base_url=self.base_url,api_key=self.api_key,model=self.model,+ max_len=self.max_len,+ head_max_len=self.head_max_len,+ hooks=self.hooks,+ on_predict_start=self.on_predict_start,+ on_predict_end=self.on_predict_end,+ hooks_raise=self.hooks_raise,+ hooks_timeout=self.hooks_timeout,)self.last_decision = res@@ -443,6 +517,13 @@ class LayaQueryRouter:base_url: Optional[str] = None,api_key: Optional[str] = None,model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,):self.query_engines = query_enginesself.descriptions = descriptions or {}@@ -453,6 +534,13 @@ class LayaQueryRouter:self.base_url = base_urlself.api_key = api_keyself.model = model+ self.max_len = max_len+ self.head_max_len = head_max_len+ self.hooks = hooks+ self.on_predict_start = on_predict_start+ self.on_predict_end = on_predict_end+ self.hooks_raise = hooks_raise+ self.hooks_timeout = hooks_timeoutself.last_decision: Optional[Dict[str, Any]] = Nonedef route(self, query: Union[QueryBundle, str]) -> str:@@ -478,6 +566,13 @@ class LayaQueryRouter:base_url=self.base_url,api_key=self.api_key,model=self.model,+ max_len=self.max_len,+ head_max_len=self.head_max_len,+ hooks=self.hooks,+ on_predict_start=self.on_predict_start,+ on_predict_end=self.on_predict_end,+ hooks_raise=self.hooks_raise,+ hooks_timeout=self.hooks_timeout,)self.last_decision = resdiff --git a/tests/test_crewai.py b/tests/test_crewai.pyindex c2587763..bebbaa32 100644--- a/tests/test_crewai.py+++ b/tests/test_crewai.py@@ -283,6 +283,175 @@ with patch("urllib.request.build_opener") as mock_build_opener:check("remote/auth", req.headers.get("Authorization"), "Bearer sk-crew-key")+# --------------------------------------------------------------- 5. Per-call decision controls+#+# `laya.integrations.langchain` has forwarded `max_len` / `head_max_len` and the five hook+# arguments since #530 / #532; this surface forwarded only `model`, so a crew with more candidate+# agents than the default head budget fits could not widen its own window. The rule now lives in+# one module the three integrations import, and the lists below are read out of it rather than+# written out again here -- a control added to `._controls` without reaching this file fails here.+import inspect+from laya.agent import Agent+from laya.integrations import _controls+from laya.integrations import crewai as crewai_module+from laya.integrations import langchain as langchain_module+from laya.integrations import llamaindex as llamaindex_module+from laya.router import Router++CONTROLS = tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.HOOK_CONTROLS)+++def _params(fn):+ return set(inspect.signature(fn).parameters)+++# The shared tuples name the arguments the shared builders accept, in both directions.+check("controls/budget tuple names budget_kwargs",+ set(_params(_controls.budget_kwargs)), set(_controls.PREDICT_CONTROLS))+check("controls/hook tuple names hook_kwargs",+ set(_params(_controls.hook_kwargs)), set(_controls.HOOK_CONTROLS))++# Everything this file calls a control is an argument a real runner accepts, or a chain would fail+# with a TypeError deep inside core instead of at the call site.+_agent_params = _params(Agent.system_one)+_router_params = _params(Router.predict)+for _c in CONTROLS:+ check_true("controls/%s accepted by Agent" % _c, _c in _agent_params)+ check_true("controls/%s accepted by Router.predict" % _c, _c in _router_params)++# Both classes and the shared executor must accept all of them. Two classes, two call sites --+# a third of them missing is exactly the bug this section exists to keep shut.+for cls in (LayaCrewRouter, LayaTaskGuard):+ for _c in CONTROLS:+ check_true("controls/%s takes %s" % (cls.__name__, _c), _c in _params(cls.__init__))+check("controls/_execute_decision takes every control",+ set(_params(crewai_module._execute_decision)) - {"state", "questions", "agent",+ "base_url", "api_key", "model"},+ set(CONTROLS))++# The three wrappers end at the same runner call, so they must accept the same controls.+for _mod in (langchain_module, llamaindex_module):+ check("controls/%s agrees with crewai" % _mod.__name__.rsplit(".", 1)[-1],+ set(_params(_mod._execute_decision)), set(_params(crewai_module._execute_decision)))+++class RecordingAgent:+ """A runner that keeps the kwargs of every call, the way a chain would have to be debugged."""++ device = "cpu"++ def __init__(self):+ self.calls = []++ def predict(self, state, questions, **kwargs):+ self.calls.append(kwargs)+ return {+ "answers": {+ "delegation": {"choice": "agent_0", "confidence": 0.9, "answer_confidence": 0.9},+ "injection_risk": {"type": "noul", "noul": 0.1, "confidence": 0.9},+ },+ "routing": {"model": "english", "repo": None, "reason": "explicit model"},+ }+++ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "hooks": ["H"], "on_predict_start": "S",+ "on_predict_end": "E", "hooks_raise": True, "hooks_timeout": 0.5}+++def _kwargs(build, run):+ """Run one decision on a fresh surface, or report the surface's own refusal.++ The broad except is deliberate: a wrapper that cannot even be built or that drops an attribute+ has to show up as a named mismatch, not as a traceback that hides every later check.+ """+ agent = RecordingAgent()+ try:+ run(build(agent))+ except Exception as exc:+ return "%s: %s" % (type(exc).__name__, exc)+ return agent.calls[0]+++def crew_call(**controls):+ return _kwargs(lambda agent: LayaCrewRouter(agent=agent, **controls),+ lambda surface: surface.route("Analyze Q3 sales", agents))+++def guard_call(**controls):+ return _kwargs(lambda agent: LayaTaskGuard(agent=agent, **controls),+ lambda surface: surface.screen("Summarize this report"))+++for label, call in (("router", crew_call), ("guard", guard_call)):+ check("controls/%s with nothing set sends nothing" % label, call(), {})+ check("controls/%s forwards all seven" % label, call(**ALL_CONTROLS), ALL_CONTROLS)+ check("controls/%s forwards one budget alone" % label, call(head_max_len=256),+ {"head_max_len": 256})+ # 0 and [] are decisions, not absences: truthiness tests here would drop them.+ check("controls/%s keeps falsy values" % label,+ call(head_max_len=0, hooks=[], hooks_raise=False),+ {"head_max_len": 0, "hooks": [], "hooks_raise": False})+ check("controls/%s alongside model" % label,+ call(model="laya-multilingual", max_len=1024),+ {"model": "laya-multilingual", "max_len": 1024})++# The caller's own hook objects have to arrive, not a copy or a re-wrapped stand-in. Read with+# `.get()` so a surface that drops them reports a named failure instead of a KeyError that hides+# the rest of the run.+_sentinel_hooks = [RecordingAgent()]+_ident_agent = RecordingAgent()+LayaCrewRouter(agent=_ident_agent, hooks=_sentinel_hooks).route("Analyze Q3 sales", agents)+_seen_hooks = _ident_agent.calls[0].get("hooks")+check_true("controls/forwards the caller's objects",+ _seen_hooks is _sentinel_hooks and _seen_hooks[0] is _sentinel_hooks[0], repr(_seen_hooks))+++# --------------------------------------------------------------- 5b. Budgets on a remote node+def remote_body(controls):+ """POST through the real urllib path with the opener mocked, and return the JSON body sent."""+ with patch("urllib.request.build_opener") as mock_build_opener:+ mock_opener = MagicMock()+ mock_opener.open.return_value = DummyHTTPResponse(remote_response)+ mock_build_opener.return_value = mock_opener+ LayaCrewRouter(base_url="http://localhost:8000", **controls).route(+ "Analyze codebase", agents)+ return json.loads(mock_opener.open.call_args[0][0].data)+++check("controls/remote body carries both budgets",+ {k: v for k, v in remote_body({"max_len": 1024, "head_max_len": 384}).items()+ if k in _controls.PREDICT_CONTROLS},+ {"max_len": 1024, "head_max_len": 384})+check("controls/remote body omits unset budgets",+ [k for k in remote_body({}) if k in _controls.PREDICT_CONTROLS], [])+check("controls/remote body keeps a zero",+ remote_body({"head_max_len": 0}).get("head_max_len"), 0)++# A hook is a Python callable that runs inside `predict`; a serve node cannot receive one. Saying+# so beats reporting success after never calling it.+for _c, _sample in (("hooks", [object()]), ("on_predict_start", object()),+ ("on_predict_end", object()), ("hooks_raise", False),+ ("hooks_timeout", 0.5)):+ try:+ LayaCrewRouter(base_url="http://localhost:8000", **{_c: _sample}).route(+ "Analyze codebase", agents)+ check_true("controls/remote refuses %s" % _c, False, "no error raised")+ except ValueError as exc:+ check_true("controls/remote refuses %s" % _c, True)+ check_true("controls/remote %s names itself" % _c, _c in str(exc))+ check_true("controls/remote %s names the endpoint" % _c, "laya-serve" in str(exc))+ except Exception as exc:+ check_true("controls/remote refuses %s" % _c, False, type(exc).__name__)++try:+ LayaTaskGuard(base_url="http://localhost:8000", hooks=[object()]).screen("hello")+ check_true("controls/guard remote refuses hooks", False, "no error raised")+except ValueError as exc:+ check_true("controls/guard remote refuses hooks", "hooks" in str(exc))+except Exception as exc:+ check_true("controls/guard remote refuses hooks", False, type(exc).__name__)++# --------------------------------------------------------------- Results Summaryprint(f"PASS: {len(PASS)}")print(f"FAIL: {len(FAIL)}")diff --git a/tests/test_llamaindex.py b/tests/test_llamaindex.pyindex 7b3cbc23..dcfbb4ce 100644--- a/tests/test_llamaindex.py+++ b/tests/test_llamaindex.py@@ -393,6 +393,164 @@ with patch("urllib.request.build_opener") as mock_build_opener:check("remote/auth", req.headers.get("Authorization"), "Bearer sk-test-token")+# --------------------------------------------------------------- 6. Per-call decision controls+#+# `laya.integrations.langchain` has forwarded `max_len` / `head_max_len` and the five hook+# arguments since #530 / #532; this surface forwarded only `model`, so a RAG pipeline choosing+# between more tools than the default head budget fits could not widen its own window. The rule+# now lives in one module the three integrations import, and the lists below are read out of it+# rather than written out again here -- a control added to `._controls` without reaching this file+# fails here. Three classes, three `_execute_decision` call sites, so every surface is driven.+import inspect+from laya.agent import Agent+from laya.integrations import _controls+from laya.integrations import crewai as crewai_module+from laya.integrations import langchain as langchain_module+from laya.integrations import llamaindex as llamaindex_module+from laya.router import Router++CONTROLS = tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.HOOK_CONTROLS)+++def _params(fn):+ return set(inspect.signature(fn).parameters)+++check("controls/budget tuple names budget_kwargs",+ set(_params(_controls.budget_kwargs)), set(_controls.PREDICT_CONTROLS))+check("controls/hook tuple names hook_kwargs",+ set(_params(_controls.hook_kwargs)), set(_controls.HOOK_CONTROLS))++# Everything called a control here is an argument a real runner accepts, or a selector would fail+# with a TypeError deep inside core instead of at the call site.+_agent_params = _params(Agent.system_one)+_router_params = _params(Router.predict)+for _c in CONTROLS:+ check_true("controls/%s accepted by Agent" % _c, _c in _agent_params)+ check_true("controls/%s accepted by Router.predict" % _c, _c in _router_params)++for cls in (LayaSingleSelector, LayaMultiSelector, LayaQueryRouter):+ for _c in CONTROLS:+ check_true("controls/%s takes %s" % (cls.__name__, _c), _c in _params(cls.__init__))+check("controls/_execute_decision takes every control",+ set(_params(llamaindex_module._execute_decision)) - {"state", "questions", "agent",+ "base_url", "api_key", "model"},+ set(CONTROLS))++# The three wrappers end at the same runner call, so they must accept the same controls.+for _mod in (langchain_module, crewai_module):+ check("controls/%s agrees with llamaindex" % _mod.__name__.rsplit(".", 1)[-1],+ set(_params(_mod._execute_decision)),+ set(_params(llamaindex_module._execute_decision)))+++class RecordingAgent:+ """A runner that keeps the kwargs of every call, the way a pipeline would have to be debugged."""++ device = "cpu"++ def __init__(self):+ self.calls = []++ def predict(self, state, questions, **kwargs):+ self.calls.append(kwargs)+ return {+ "answers": {+ "selector": {"choice": "choice_0", "confidence": 0.9, "answer_confidence": 0.9,+ "probabilities": {"choice_0": 0.9, "choice_1": 0.4}},+ "route": {"choice": "sql", "confidence": 0.9, "answer_confidence": 0.9},+ },+ "routing": {"model": "english", "repo": None, "reason": "explicit model"},+ }+++ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "hooks": ["H"], "on_predict_start": "S",+ "on_predict_end": "E", "hooks_raise": True, "hooks_timeout": 0.5}++# One entry per class that can take a decision here: (label, class, constructor extras, one call).+SURFACES = [+ ("single", LayaSingleSelector, {},+ lambda s: s.select(tools, "Run a query on the SQL table")),+ ("multi", LayaMultiSelector, {},+ lambda s: s.select(tools, "Run a query on the SQL table")),+ ("query_router", LayaQueryRouter, {"query_engines": engines},+ lambda s: s.route("Run a query on the SQL table")),+]++HOOK_SAMPLES = (("hooks", [object()]), ("on_predict_start", object()),+ ("on_predict_end", object()), ("hooks_raise", False), ("hooks_timeout", 0.5))+++def local_kwargs(cls, extra, run, **controls):+ """Run one decision on a fresh surface, or report the surface's own refusal.++ The broad except is deliberate: a wrapper that cannot even be built or that drops an attribute+ has to show up as a named mismatch, not as a traceback that hides every later check.+ """+ agent = RecordingAgent()+ try:+ run(cls(agent=agent, **dict(extra, **controls)))+ except Exception as exc:+ return "%s: %s" % (type(exc).__name__, exc)+ return agent.calls[0]+++def remote_body(cls, extra, run, **controls):+ """POST through the real urllib path with the opener mocked, and return the JSON body sent."""+ with patch("urllib.request.build_opener") as mock_build_opener:+ mock_opener = MagicMock()+ mock_opener.open.return_value = DummyHTTPResponse(remote_response)+ mock_build_opener.return_value = mock_opener+ run(cls(base_url="http://localhost:8000", **dict(extra, **controls)))+ return json.loads(mock_opener.open.call_args[0][0].data)+++for _label, _cls, _extra, _run in SURFACES:+ check("controls/%s with nothing set sends nothing" % _label,+ local_kwargs(_cls, _extra, _run), {})+ check("controls/%s forwards all seven" % _label,+ local_kwargs(_cls, _extra, _run, **ALL_CONTROLS), ALL_CONTROLS)+ check("controls/%s forwards one budget alone" % _label,+ local_kwargs(_cls, _extra, _run, head_max_len=256), {"head_max_len": 256})+ # 0 and [] are decisions, not absences: truthiness tests here would drop them.+ check("controls/%s keeps falsy values" % _label,+ local_kwargs(_cls, _extra, _run, head_max_len=0, hooks=[], hooks_raise=False),+ {"head_max_len": 0, "hooks": [], "hooks_raise": False})+ check("controls/%s alongside model" % _label,+ local_kwargs(_cls, _extra, _run, model="laya-multilingual", max_len=1024),+ {"model": "laya-multilingual", "max_len": 1024})+ check("controls/%s remote body carries both budgets" % _label,+ {k: v for k, v in remote_body(_cls, _extra, _run, max_len=1024,+ head_max_len=384).items()+ if k in _controls.PREDICT_CONTROLS},+ {"max_len": 1024, "head_max_len": 384})+ check("controls/%s remote body omits unset budgets" % _label,+ [k for k in remote_body(_cls, _extra, _run) if k in _controls.PREDICT_CONTROLS], [])+ check("controls/%s remote body keeps a zero" % _label,+ remote_body(_cls, _extra, _run, head_max_len=0).get("head_max_len"), 0)+ # A hook is a Python callable that runs inside `predict`; a serve node cannot receive one.+ for _c, _sample in HOOK_SAMPLES:+ try:+ remote_body(_cls, _extra, _run, **{_c: _sample})+ check_true("controls/%s remote refuses %s" % (_label, _c), False, "no error raised")+ except ValueError as exc:+ check_true("controls/%s remote refuses %s" % (_label, _c),+ _c in str(exc) and "laya-serve" in str(exc))+ except Exception as exc:+ check_true("controls/%s remote refuses %s" % (_label, _c), False, type(exc).__name__)++# The caller's own hook objects have to arrive, not a copy or a re-wrapped stand-in. Read with+# `.get()` so a surface that drops them reports a named failure instead of a KeyError that hides+# the rest of the run.+_sentinel_hooks = [RecordingAgent()]+_ident_agent = RecordingAgent()+LayaSingleSelector(agent=_ident_agent, hooks=_sentinel_hooks).select(+ tools, "Run a query on the SQL table")+_seen_hooks = _ident_agent.calls[0].get("hooks")+check_true("controls/forwards the caller's objects",+ _seen_hooks is _sentinel_hooks and _seen_hooks[0] is _sentinel_hooks[0], repr(_seen_hooks))++# --------------------------------------------------------------- Results Summaryprint(f"PASS: {len(PASS)}")print(f"FAIL: {len(FAIL)}")