Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(integrations): forward token budgets and hooks from CrewAI and LlamaIndex

Aashish0b3ee71bd539
diff --git a/docs/crewai.md b/docs/crewai.md
index 497e412a..0c196b22 100644
--- a/docs/crewai.md
+++ b/docs/crewai.md
@@ -5,6 +5,8 @@ Laya provides sub-35ms, non-autoregressive decision components for **CrewAI** mu
* **`LayaCrewRouter`**: Sub-35ms task delegation router replacing LLM managers in hierarchical crews.
* **`LayaTaskGuard`**: Pre-execution task guardrail screening prompts and instructions for jailbreaks, injections, and policy violations.
+Both take core's per-call decision controls -- the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`) -- see [Per-call decision controls](#5-per-call-decision-controls).
+
Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.
---
@@ -121,3 +123,55 @@ router = LayaCrewRouter(
```
The remote client uses Python's standard library `urllib` with zero heavy dependencies, preventing cross-origin credential forwarding and matching the `/v1/systemone` specification.
+
+---
+
+## 5. Per-call decision controls
+
+`LayaCrewRouter` and `LayaTaskGuard` take the same per-call arguments the core API does: the two
+token budgets (`max_len`, `head_max_len`) and the five prediction-hook arguments (`hooks`,
+`on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are per instance, so a
+crew with a large roster can be given room while the rest of the pipeline keeps the checkpoint's
+defaults.
+
+A delegation choice shares the checkpoint's *option* budget -- `head_max_len`, 192 tokens on `laya`
+-- and every candidate contributes its role and goal, so past roughly 20 agents the later goals
+start reaching the model as the same truncated text.
+
+```python
+router = LayaCrewRouter(
+ confidence_threshold=0.80,
+ max_len=1024, # total window
+ head_max_len=512, # tokens shared by the roster
+)
+
+decision = router.route(task, agents) # agents: 59 candidates
+```
+
+Measured with `laya` on Apple silicon, one forward pass per task, scored on the delegated agent,
+against a 59-agent roster built from the MASSIVE en intent labels (role only, goal left empty) with
+one utterance per label, so ground truth is exact. Each cell is how many of the 59 tasks were given
+to their own agent; both repeats gave the same count.
+
+| 59-agent roster | Default budget | `max_len=1024, head_max_len=384` | `…, head_max_len=512` |
+|---|---|---|---|
+| Tasks on their own agent | 2/59 | 7/59 | 16/59 |
+| Median ms per task | 146 | 190 | 265 |
+
+Before this, the same run could not be asked at all: `LayaCrewRouter.__init__() got an unexpected
+keyword argument 'max_len'`.
+
+Absolute accuracy is not the claim here -- the checkpoint is not a MASSIVE classifier and 59 similar
+labels are a stress shape. The claim is reachability and price: a roster the default budget collapses
+to near-nothing is readable from a crew, and at this size the wider window costs little time. Note
+the middle row is 7/59 where the identical criteria sent straight to `Router.predict` scored 8/59;
+only the `instructions` sentence differed, which is the expected sensitivity of a choice question to
+its own wording. With fewer than about 20 candidates the roles already fit and widening can move
+answers the wrong way, which is why both budgets are opt-in per instance. See the [LangChain
+integration](langchain.md#7-widening-the-token-budget-for-many-options) for that measured cliff.
+
+**Hooks run on the local path only.** A router or guard with a `base_url` and `hooks=[...]` raises
+`ValueError` rather than reporting a success whose hook never ran -- a hook is a Python callable that
+executes inside `predict`, and no wire format carries it. Install hooks in the process that runs
+inference. The two budgets do travel to a remote node, in the request body, up to its
+`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.
diff --git a/docs/llamaindex.md b/docs/llamaindex.md
index 17ebdf59..f75eea4a 100644
--- a/docs/llamaindex.md
+++ b/docs/llamaindex.md
@@ -139,3 +139,49 @@ selector = LayaSingleSelector(
```
The remote client uses Python's standard library `urllib` with zero heavy dependencies, preventing cross-origin credential forwarding and matching the `/v1/systemone` specification.
+
+---
+
+## 6. Per-call decision controls
+
+`LayaSingleSelector`, `LayaMultiSelector` and `LayaQueryRouter` take the same per-call arguments the
+core API does: the two token budgets (`max_len`, `head_max_len`) and the five prediction-hook
+arguments (`hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise`, `hooks_timeout`). They are
+per selector, so a wide routing step can be given room while the rest of the pipeline keeps the
+checkpoint's defaults.
+
+A choice question's options share the checkpoint's *option* budget -- `head_max_len`, 192 tokens on
+`laya` -- and every candidate contributes its name and description, so past roughly 20 tools the
+descriptions start reaching the model as the same text.
+
+```python
+selector = LayaSingleSelector(
+ instructions="Which tool or query engine is best suited to answer this query?",
+ max_len=1024, # total window
+ head_max_len=512, # tokens shared by the option prompt
+)
+result = selector.select(tools, query) # tools: 59 descriptions
+```
+
+Measured on `laya` (Apple silicon, one forward pass per query, scored on the chosen tool) with a
+59-tool roster built from the MASSIVE en intent labels and one utterance per label, so ground truth
+is exact. Each cell is how many of the 59 queries reached their own tool; both repeats gave the
+same count.
+
+| 59-tool roster | Default budget | `max_len=1024, head_max_len=384` | `…, head_max_len=512` |
+|---|---|---|---|
+| Queries on their own tool | 2/59 | 8/59 | 15/59 |
+| Median ms per query | 160 | 172 | 184 |
+
+Absolute accuracy is not the claim here: the checkpoint is not a MASSIVE classifier, and 59 similar
+labels are a stress shape. The claim is the direction and the price -- a roster the default budget
+collapses to near-nothing is readable, and at this size the window costs little time. With fewer
+than about 20 options the labels already fit and widening can move answers the wrong way, which is
+why both arguments are opt-in per selector. See the [LangChain
+integration](langchain.md#7-widening-the-token-budget-for-many-options) for that measured cliff.
+
+**Hooks run on the local path only.** A selector with a `base_url` and `hooks=[...]` raises
+`ValueError` rather than reporting a success whose cache never ran -- a hook is a Python callable
+that runs inside `predict`, and no wire format carries it. Install hooks in the process that runs
+inference. The two budgets do travel to a remote node, in the request body, up to its
+`LAYA_MAX_TOKEN_BUDGET` ceiling; a larger value comes back as a 422.
diff --git a/laya/integrations/_controls.py b/laya/integrations/_controls.py
new file mode 100644
index 00000000..a8b1caaa
--- /dev/null
+++ b/laya/integrations/_controls.py
@@ -0,0 +1,74 @@
+"""Per-call decision controls shared by the framework integrations.
+
+Every integration wrapper ends at one of two places: a local `runner.predict(state, questions,
+**overrides)` call, or a POST to a laya-serve node. Which overrides those two accept is a single
+rule, and three independent copies of it drift -- this module is the copy the LangChain, CrewAI
+and LlamaIndex wrappers import.
+
+The names are listed once, in `PREDICT_CONTROLS` and `HOOK_CONTROLS`, and the signatures below
+are checked against them by `tests/test_crewai.py` and `tests/test_llamaindex.py`, which also
+assert every wrapper's constructor accepts all of them. A control added here without reaching a
+wrapper's `__init__` fails that wrapper's suite rather than being dropped silently.
+"""
+from __future__ import annotations
+
+from typing import Any, Dict, Optional
+
+# The token-budget overrides `Agent.predict` / `Router.predict` take per call, and that
+# laya-serve accepts in the request body up to its `LAYA_MAX_TOKEN_BUDGET` ceiling.
+PREDICT_CONTROLS = ("max_len", "head_max_len")
+
+# The per-call hook family. These are Python callables and flags that run inside `predict`, so
+# they exist only on the local path -- see `reject_remote_hooks`.
+HOOK_CONTROLS = ("hooks", "on_predict_start", "on_predict_end", "hooks_raise", "hooks_timeout")
+
+
+def budget_kwargs(max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None) -> Dict[str, Any]:
+ """The two token budgets, with the unset ones omitted.
+
+ Absent rather than `None`: on the local path a `None` budget would override the checkpoint's
+ own default with nothing, and on the remote path an older stand-in `_call_remote` -- including
+ the ones these integrations' own tests install -- must keep accepting the call.
+ """
+ return {key: value for key, value in (("max_len", max_len), ("head_max_len", head_max_len))
+ if value is not None}
+
+
+def predict_kwargs(model: Optional[str] = None, max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None) -> Dict[str, Any]:
+ """The per-request overrides a local runner accepts, with the unset ones omitted."""
+ kwargs: Dict[str, Any] = {}
+ if model:
+ kwargs["model"] = model
+ kwargs.update(budget_kwargs(max_len, head_max_len))
+ return kwargs
+
+
+def hook_kwargs(hooks: Optional[Any] = None, on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None, hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None) -> Dict[str, Any]:
+ """The per-call hook overrides, with the unset ones omitted.
+
+ Core reads `None` as "inherit whatever the runner was built with", so an unset hook has to be
+ absent rather than passed as `None`. Note the `is not None` tests: `hooks=[]` means "no hooks
+ for this call", and `hooks_raise=False` means "keep deciding after a hook fails" -- both are
+ decisions a caller made, not absences.
+ """
+ given = {"hooks": hooks, "on_predict_start": on_predict_start, "on_predict_end": on_predict_end,
+ "hooks_raise": hooks_raise, "hooks_timeout": hooks_timeout}
+ return {k: v for k, v in given.items() if v is not None}
+
+
+def reject_remote_hooks(given: Dict[str, Any], base_url: Optional[str]) -> None:
+ """Refuse hooks on a remote node rather than dropping them silently.
+
+ A hook is a Python callable that runs inside `predict` -- it can cache a decision, gate one or
+ rewrite its state. `laya-serve` has no way to receive or run one, so a node with a `base_url`
+ and hooks configured would report success while never calling them.
+ """
+ if base_url and given:
+ raise ValueError(
+ "%s run in the local runner and cannot be sent to a laya-serve endpoint; "
+ "install them where serve runs, or drop them" % ", ".join(sorted(given))
+ )
diff --git a/laya/integrations/crewai.py b/laya/integrations/crewai.py
index 5c154c71..435d96c1 100644
--- a/laya/integrations/crewai.py
+++ b/laya/integrations/crewai.py
@@ -51,6 +51,12 @@ class LayaTaskGuardError(ValueError):
self.raw_decision = raw_decision
+# The per-call control rules are shared with the other two integrations: they end at the same
+# `predict` call and the same laya-serve request body, and three copies of the rule is how two of
+# them came to forward only `model`.
+from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs # noqa: E402
+from ._controls import predict_kwargs as _predict_kwargs, reject_remote_hooks as _reject_remote_hooks # noqa: E402
+
# One class for every integration, so `except LayaLowConfidenceError` catches all of them.
from ._errors import LayaLowConfidenceError # noqa: E402
@@ -138,8 +144,14 @@ def _call_remote(
api_key: Optional[str] = None,
model: Optional[str] = None,
timeout: float = 10.0,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
) -> Dict[str, Any]:
- """Send decision request to a remote laya-serve HTTP instance using standard library urllib."""
+ """Send decision request to a remote laya-serve HTTP instance using standard library urllib.
+
+ `max_len` / `head_max_len` travel in the body; laya-serve applies them up to its
+ `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422.
+ """
url = base_url.rstrip("/")
if not url.endswith("/v1/systemone"):
url = f"{url}/v1/systemone"
@@ -147,6 +159,10 @@ def _call_remote(
payload: Dict[str, Any] = {"state": state, "questions": questions}
if model:
payload["model"] = model
+ if max_len is not None:
+ payload["max_len"] = max_len
+ if head_max_len is not None:
+ payload["head_max_len"] = head_max_len
data = json.dumps(payload).encode("utf-8")
headers = {"Content-Type": "application/json"}
@@ -186,11 +202,27 @@ def _execute_decision(
base_url: Optional[str] = None,
api_key: Optional[str] = None,
model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
) -> Dict[str, Any]:
+ """One decision on the local runner or a laya-serve node, with the per-call controls kept.
+
+ Which controls either path accepts is `._controls`' rule, shared with the LangChain and
+ LlamaIndex wrappers -- they end at the same `predict` call and the same request body.
+ """
+ hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)
if base_url:
- return _call_remote(base_url, state, questions, api_key=api_key, model=model)
+ _reject_remote_hooks(hook_kwargs, base_url)
+ budget = _budget_kwargs(max_len, head_max_len)
+ return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)
runner = agent if agent is not None else _get_default_router()
- kwargs = {"model": model} if model else {}
+ kwargs = _predict_kwargs(model, max_len, head_max_len)
+ kwargs.update(hook_kwargs)
return runner.predict(state, questions, **kwargs)
@@ -211,6 +243,13 @@ class LayaCrewRouter:
base_url: Optional[str] = None,
api_key: Optional[str] = None,
model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
):
self.instructions = instructions
self.confidence_threshold = confidence_threshold
@@ -220,6 +259,13 @@ class LayaCrewRouter:
self.base_url = base_url
self.api_key = api_key
self.model = model
+ self.max_len = max_len
+ self.head_max_len = head_max_len
+ self.hooks = hooks
+ self.on_predict_start = on_predict_start
+ self.on_predict_end = on_predict_end
+ self.hooks_raise = hooks_raise
+ self.hooks_timeout = hooks_timeout
self.last_decision: Optional[Dict[str, Any]] = None
def route(
@@ -249,6 +295,13 @@ class LayaCrewRouter:
base_url=self.base_url,
api_key=self.api_key,
model=self.model,
+ max_len=self.max_len,
+ head_max_len=self.head_max_len,
+ hooks=self.hooks,
+ on_predict_start=self.on_predict_start,
+ on_predict_end=self.on_predict_end,
+ hooks_raise=self.hooks_raise,
+ hooks_timeout=self.hooks_timeout,
)
self.last_decision = res
@@ -359,6 +412,13 @@ class LayaTaskGuard:
base_url: Optional[str] = None,
api_key: Optional[str] = None,
model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
):
self.questions = questions
self.action = action
@@ -368,6 +428,13 @@ class LayaTaskGuard:
self.base_url = base_url
self.api_key = api_key
self.model = model
+ self.max_len = max_len
+ self.head_max_len = head_max_len
+ self.hooks = hooks
+ self.on_predict_start = on_predict_start
+ self.on_predict_end = on_predict_end
+ self.hooks_raise = hooks_raise
+ self.hooks_timeout = hooks_timeout
self.last_decision: Optional[Dict[str, Any]] = None
def _default_questions(self) -> Dict[str, Any]:
@@ -386,6 +453,13 @@ class LayaTaskGuard:
base_url=self.base_url,
api_key=self.api_key,
model=self.model,
+ max_len=self.max_len,
+ head_max_len=self.head_max_len,
+ hooks=self.hooks,
+ on_predict_start=self.on_predict_start,
+ on_predict_end=self.on_predict_end,
+ hooks_raise=self.hooks_raise,
+ hooks_timeout=self.hooks_timeout,
)
self.last_decision = res
answers = res.get("answers", {})
diff --git a/laya/integrations/langchain.py b/laya/integrations/langchain.py
index 9285f04d..c46d6847 100644
--- a/laya/integrations/langchain.py
+++ b/laya/integrations/langchain.py
@@ -26,6 +26,13 @@ except ImportError:
RunnableSerializable = object # type: ignore
RunnableConfig = Any # type: ignore
+# The per-call control rules live in `._controls` because the CrewAI and LlamaIndex wrappers end
+# at the same `predict` call and the same laya-serve body; keeping the rule in three places is how
+# two of them ended up forwarding only `model`.
+from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs
+from ._controls import predict_kwargs as _predict_kwargs
+from ._controls import reject_remote_hooks as _reject_remote_hooks
+
class LayaGuardrailError(ValueError):
"""Raised when an input violates a Laya guardrail policy."""
@@ -156,48 +163,6 @@ def _get_default_router():
return _DEFAULT_ROUTER
-def _predict_kwargs(model: Optional[str] = None, max_len: Optional[int] = None,
- head_max_len: Optional[int] = None) -> Dict[str, Any]:
- """The per-request overrides a local runner accepts, with the unset ones omitted."""
- kwargs: Dict[str, Any] = {}
- if model:
- kwargs["model"] = model
- if max_len is not None:
- kwargs["max_len"] = max_len
- if head_max_len is not None:
- kwargs["head_max_len"] = head_max_len
- return kwargs
-
-
-def _reject_remote_hooks(hook_kwargs: Dict[str, Any], base_url: Optional[str]) -> None:
- """Refuse hooks on a remote node rather than dropping them silently.
-
- A hook is a Python callable that runs inside `predict` -- it can cache a decision, gate one or
- rewrite its state. `laya-serve` has no way to receive or run one, so a node with a `base_url`
- and hooks configured would report success while never calling them.
- """
- if base_url and hook_kwargs:
- raise ValueError(
- "%s run in the local runner and cannot be sent to a laya-serve endpoint; "
- "install them where serve runs, or drop them" % ", ".join(sorted(hook_kwargs))
- )
-
-
-def _hook_kwargs(hooks: Optional[Any] = None, on_predict_start: Optional[Any] = None,
- on_predict_end: Optional[Any] = None, hooks_raise: Optional[bool] = None,
- hooks_timeout: Optional[float] = None) -> Dict[str, Any]:
- """The per-call hook overrides, with the unset ones omitted.
-
- Core reads `None` as "inherit whatever the runner was built with", so an unset hook has to be
- absent rather than passed as `None`. Note the `is not None` tests: `hooks=[]` means "no hooks
- for this call", and `hooks_raise=False` means "keep deciding after a hook fails" -- both are
- decisions a caller made, not absences.
- """
- given = {"hooks": hooks, "on_predict_start": on_predict_start, "on_predict_end": on_predict_end,
- "hooks_raise": hooks_raise, "hooks_timeout": hooks_timeout}
- return {k: v for k, v in given.items() if v is not None}
-
-
def _execute_decision(
state: Any,
questions: Dict[str, Any],
@@ -216,9 +181,7 @@ def _execute_decision(
hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)
if base_url:
_reject_remote_hooks(hook_kwargs, base_url)
- # Only what was set, so a stand-in `_call_remote` without the budget keywords still works.
- budget = {k: v for k, v in (("max_len", max_len), ("head_max_len", head_max_len))
- if v is not None}
+ budget = _budget_kwargs(max_len, head_max_len)
return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)
runner = agent if agent is not None else _get_default_router()
kwargs = _predict_kwargs(model, max_len, head_max_len)
diff --git a/laya/integrations/llamaindex.py b/laya/integrations/llamaindex.py
index 6e7615a2..260df8f3 100644
--- a/laya/integrations/llamaindex.py
+++ b/laya/integrations/llamaindex.py
@@ -60,6 +60,12 @@ except ImportError:
return self.query_str
+# The per-call control rules are shared with the other two integrations: they end at the same
+# `predict` call and the same laya-serve request body, and three copies of the rule is how two of
+# them came to forward only `model`.
+from ._controls import budget_kwargs as _budget_kwargs, hook_kwargs as _hook_kwargs # noqa: E402
+from ._controls import predict_kwargs as _predict_kwargs, reject_remote_hooks as _reject_remote_hooks # noqa: E402
+
# One class for every integration, so `except LayaLowConfidenceError` catches all of them.
from ._errors import LayaLowConfidenceError # noqa: E402
@@ -122,8 +128,14 @@ def _call_remote(
api_key: Optional[str] = None,
model: Optional[str] = None,
timeout: float = 10.0,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
) -> Dict[str, Any]:
- """Send decision request to a remote laya-serve HTTP instance using standard library urllib."""
+ """Send decision request to a remote laya-serve HTTP instance using standard library urllib.
+
+ `max_len` / `head_max_len` travel in the body; laya-serve applies them up to its
+ `LAYA_MAX_TOKEN_BUDGET` ceiling and answers a larger value with 422.
+ """
url = base_url.rstrip("/")
if not url.endswith("/v1/systemone"):
url = f"{url}/v1/systemone"
@@ -131,6 +143,10 @@ def _call_remote(
payload: Dict[str, Any] = {"state": state, "questions": questions}
if model:
payload["model"] = model
+ if max_len is not None:
+ payload["max_len"] = max_len
+ if head_max_len is not None:
+ payload["head_max_len"] = head_max_len
data = json.dumps(payload).encode("utf-8")
headers = {"Content-Type": "application/json"}
@@ -170,11 +186,27 @@ def _execute_decision(
base_url: Optional[str] = None,
api_key: Optional[str] = None,
model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
) -> Dict[str, Any]:
+ """One decision on the local runner or a laya-serve node, with the per-call controls kept.
+
+ Which controls either path accepts is `._controls`' rule, shared with the LangChain and
+ CrewAI wrappers -- they end at the same `predict` call and the same request body.
+ """
+ hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)
if base_url:
- return _call_remote(base_url, state, questions, api_key=api_key, model=model)
+ _reject_remote_hooks(hook_kwargs, base_url)
+ budget = _budget_kwargs(max_len, head_max_len)
+ return _call_remote(base_url, state, questions, api_key=api_key, model=model, **budget)
runner = agent if agent is not None else _get_default_router()
- kwargs = {"model": model} if model else {}
+ kwargs = _predict_kwargs(model, max_len, head_max_len)
+ kwargs.update(hook_kwargs)
return runner.predict(state, questions, **kwargs)
@@ -196,6 +228,13 @@ class LayaSingleSelector(BaseSelector):
base_url: Optional[str] = None,
api_key: Optional[str] = None,
model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
):
self.instructions = instructions
self.confidence_threshold = confidence_threshold
@@ -205,6 +244,13 @@ class LayaSingleSelector(BaseSelector):
self.base_url = base_url
self.api_key = api_key
self.model = model
+ self.max_len = max_len
+ self.head_max_len = head_max_len
+ self.hooks = hooks
+ self.on_predict_start = on_predict_start
+ self.on_predict_end = on_predict_end
+ self.hooks_raise = hooks_raise
+ self.hooks_timeout = hooks_timeout
self.last_decision: Optional[Dict[str, Any]] = None
def _select(
@@ -234,6 +280,13 @@ class LayaSingleSelector(BaseSelector):
base_url=self.base_url,
api_key=self.api_key,
model=self.model,
+ max_len=self.max_len,
+ head_max_len=self.head_max_len,
+ hooks=self.hooks,
+ on_predict_start=self.on_predict_start,
+ on_predict_end=self.on_predict_end,
+ hooks_raise=self.hooks_raise,
+ hooks_timeout=self.hooks_timeout,
)
self.last_decision = res
@@ -315,6 +368,13 @@ class LayaMultiSelector(BaseSelector):
base_url: Optional[str] = None,
api_key: Optional[str] = None,
model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
):
self.instructions = instructions
self.max_outputs = max_outputs
@@ -324,6 +384,13 @@ class LayaMultiSelector(BaseSelector):
self.base_url = base_url
self.api_key = api_key
self.model = model
+ self.max_len = max_len
+ self.head_max_len = head_max_len
+ self.hooks = hooks
+ self.on_predict_start = on_predict_start
+ self.on_predict_end = on_predict_end
+ self.hooks_raise = hooks_raise
+ self.hooks_timeout = hooks_timeout
self.last_decision: Optional[Dict[str, Any]] = None
def _select(
@@ -352,6 +419,13 @@ class LayaMultiSelector(BaseSelector):
base_url=self.base_url,
api_key=self.api_key,
model=self.model,
+ max_len=self.max_len,
+ head_max_len=self.head_max_len,
+ hooks=self.hooks,
+ on_predict_start=self.on_predict_start,
+ on_predict_end=self.on_predict_end,
+ hooks_raise=self.hooks_raise,
+ hooks_timeout=self.hooks_timeout,
)
self.last_decision = res
@@ -443,6 +517,13 @@ class LayaQueryRouter:
base_url: Optional[str] = None,
api_key: Optional[str] = None,
model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
):
self.query_engines = query_engines
self.descriptions = descriptions or {}
@@ -453,6 +534,13 @@ class LayaQueryRouter:
self.base_url = base_url
self.api_key = api_key
self.model = model
+ self.max_len = max_len
+ self.head_max_len = head_max_len
+ self.hooks = hooks
+ self.on_predict_start = on_predict_start
+ self.on_predict_end = on_predict_end
+ self.hooks_raise = hooks_raise
+ self.hooks_timeout = hooks_timeout
self.last_decision: Optional[Dict[str, Any]] = None
def route(self, query: Union[QueryBundle, str]) -> str:
@@ -478,6 +566,13 @@ class LayaQueryRouter:
base_url=self.base_url,
api_key=self.api_key,
model=self.model,
+ max_len=self.max_len,
+ head_max_len=self.head_max_len,
+ hooks=self.hooks,
+ on_predict_start=self.on_predict_start,
+ on_predict_end=self.on_predict_end,
+ hooks_raise=self.hooks_raise,
+ hooks_timeout=self.hooks_timeout,
)
self.last_decision = res
diff --git a/tests/test_crewai.py b/tests/test_crewai.py
index c2587763..bebbaa32 100644
--- a/tests/test_crewai.py
+++ b/tests/test_crewai.py
@@ -283,6 +283,175 @@ with patch("urllib.request.build_opener") as mock_build_opener:
check("remote/auth", req.headers.get("Authorization"), "Bearer sk-crew-key")
+# --------------------------------------------------------------- 5. Per-call decision controls
+#
+# `laya.integrations.langchain` has forwarded `max_len` / `head_max_len` and the five hook
+# arguments since #530 / #532; this surface forwarded only `model`, so a crew with more candidate
+# agents than the default head budget fits could not widen its own window. The rule now lives in
+# one module the three integrations import, and the lists below are read out of it rather than
+# written out again here -- a control added to `._controls` without reaching this file fails here.
+import inspect
+from laya.agent import Agent
+from laya.integrations import _controls
+from laya.integrations import crewai as crewai_module
+from laya.integrations import langchain as langchain_module
+from laya.integrations import llamaindex as llamaindex_module
+from laya.router import Router
+
+CONTROLS = tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.HOOK_CONTROLS)
+
+
+def _params(fn):
+ return set(inspect.signature(fn).parameters)
+
+
+# The shared tuples name the arguments the shared builders accept, in both directions.
+check("controls/budget tuple names budget_kwargs",
+ set(_params(_controls.budget_kwargs)), set(_controls.PREDICT_CONTROLS))
+check("controls/hook tuple names hook_kwargs",
+ set(_params(_controls.hook_kwargs)), set(_controls.HOOK_CONTROLS))
+
+# Everything this file calls a control is an argument a real runner accepts, or a chain would fail
+# with a TypeError deep inside core instead of at the call site.
+_agent_params = _params(Agent.system_one)
+_router_params = _params(Router.predict)
+for _c in CONTROLS:
+ check_true("controls/%s accepted by Agent" % _c, _c in _agent_params)
+ check_true("controls/%s accepted by Router.predict" % _c, _c in _router_params)
+
+# Both classes and the shared executor must accept all of them. Two classes, two call sites --
+# a third of them missing is exactly the bug this section exists to keep shut.
+for cls in (LayaCrewRouter, LayaTaskGuard):
+ for _c in CONTROLS:
+ check_true("controls/%s takes %s" % (cls.__name__, _c), _c in _params(cls.__init__))
+check("controls/_execute_decision takes every control",
+ set(_params(crewai_module._execute_decision)) - {"state", "questions", "agent",
+ "base_url", "api_key", "model"},
+ set(CONTROLS))
+
+# The three wrappers end at the same runner call, so they must accept the same controls.
+for _mod in (langchain_module, llamaindex_module):
+ check("controls/%s agrees with crewai" % _mod.__name__.rsplit(".", 1)[-1],
+ set(_params(_mod._execute_decision)), set(_params(crewai_module._execute_decision)))
+
+
+class RecordingAgent:
+ """A runner that keeps the kwargs of every call, the way a chain would have to be debugged."""
+
+ device = "cpu"
+
+ def __init__(self):
+ self.calls = []
+
+ def predict(self, state, questions, **kwargs):
+ self.calls.append(kwargs)
+ return {
+ "answers": {
+ "delegation": {"choice": "agent_0", "confidence": 0.9, "answer_confidence": 0.9},
+ "injection_risk": {"type": "noul", "noul": 0.1, "confidence": 0.9},
+ },
+ "routing": {"model": "english", "repo": None, "reason": "explicit model"},
+ }
+
+
+ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "hooks": ["H"], "on_predict_start": "S",
+ "on_predict_end": "E", "hooks_raise": True, "hooks_timeout": 0.5}
+
+
+def _kwargs(build, run):
+ """Run one decision on a fresh surface, or report the surface's own refusal.
+
+ The broad except is deliberate: a wrapper that cannot even be built or that drops an attribute
+ has to show up as a named mismatch, not as a traceback that hides every later check.
+ """
+ agent = RecordingAgent()
+ try:
+ run(build(agent))
+ except Exception as exc:
+ return "%s: %s" % (type(exc).__name__, exc)
+ return agent.calls[0]
+
+
+def crew_call(**controls):
+ return _kwargs(lambda agent: LayaCrewRouter(agent=agent, **controls),
+ lambda surface: surface.route("Analyze Q3 sales", agents))
+
+
+def guard_call(**controls):
+ return _kwargs(lambda agent: LayaTaskGuard(agent=agent, **controls),
+ lambda surface: surface.screen("Summarize this report"))
+
+
+for label, call in (("router", crew_call), ("guard", guard_call)):
+ check("controls/%s with nothing set sends nothing" % label, call(), {})
+ check("controls/%s forwards all seven" % label, call(**ALL_CONTROLS), ALL_CONTROLS)
+ check("controls/%s forwards one budget alone" % label, call(head_max_len=256),
+ {"head_max_len": 256})
+ # 0 and [] are decisions, not absences: truthiness tests here would drop them.
+ check("controls/%s keeps falsy values" % label,
+ call(head_max_len=0, hooks=[], hooks_raise=False),
+ {"head_max_len": 0, "hooks": [], "hooks_raise": False})
+ check("controls/%s alongside model" % label,
+ call(model="laya-multilingual", max_len=1024),
+ {"model": "laya-multilingual", "max_len": 1024})
+
+# The caller's own hook objects have to arrive, not a copy or a re-wrapped stand-in. Read with
+# `.get()` so a surface that drops them reports a named failure instead of a KeyError that hides
+# the rest of the run.
+_sentinel_hooks = [RecordingAgent()]
+_ident_agent = RecordingAgent()
+LayaCrewRouter(agent=_ident_agent, hooks=_sentinel_hooks).route("Analyze Q3 sales", agents)
+_seen_hooks = _ident_agent.calls[0].get("hooks")
+check_true("controls/forwards the caller's objects",
+ _seen_hooks is _sentinel_hooks and _seen_hooks[0] is _sentinel_hooks[0], repr(_seen_hooks))
+
+
+# --------------------------------------------------------------- 5b. Budgets on a remote node
+def remote_body(controls):
+ """POST through the real urllib path with the opener mocked, and return the JSON body sent."""
+ with patch("urllib.request.build_opener") as mock_build_opener:
+ mock_opener = MagicMock()
+ mock_opener.open.return_value = DummyHTTPResponse(remote_response)
+ mock_build_opener.return_value = mock_opener
+ LayaCrewRouter(base_url="http://localhost:8000", **controls).route(
+ "Analyze codebase", agents)
+ return json.loads(mock_opener.open.call_args[0][0].data)
+
+
+check("controls/remote body carries both budgets",
+ {k: v for k, v in remote_body({"max_len": 1024, "head_max_len": 384}).items()
+ if k in _controls.PREDICT_CONTROLS},
+ {"max_len": 1024, "head_max_len": 384})
+check("controls/remote body omits unset budgets",
+ [k for k in remote_body({}) if k in _controls.PREDICT_CONTROLS], [])
+check("controls/remote body keeps a zero",
+ remote_body({"head_max_len": 0}).get("head_max_len"), 0)
+
+# A hook is a Python callable that runs inside `predict`; a serve node cannot receive one. Saying
+# so beats reporting success after never calling it.
+for _c, _sample in (("hooks", [object()]), ("on_predict_start", object()),
+ ("on_predict_end", object()), ("hooks_raise", False),
+ ("hooks_timeout", 0.5)):
+ try:
+ LayaCrewRouter(base_url="http://localhost:8000", **{_c: _sample}).route(
+ "Analyze codebase", agents)
+ check_true("controls/remote refuses %s" % _c, False, "no error raised")
+ except ValueError as exc:
+ check_true("controls/remote refuses %s" % _c, True)
+ check_true("controls/remote %s names itself" % _c, _c in str(exc))
+ check_true("controls/remote %s names the endpoint" % _c, "laya-serve" in str(exc))
+ except Exception as exc:
+ check_true("controls/remote refuses %s" % _c, False, type(exc).__name__)
+
+try:
+ LayaTaskGuard(base_url="http://localhost:8000", hooks=[object()]).screen("hello")
+ check_true("controls/guard remote refuses hooks", False, "no error raised")
+except ValueError as exc:
+ check_true("controls/guard remote refuses hooks", "hooks" in str(exc))
+except Exception as exc:
+ check_true("controls/guard remote refuses hooks", False, type(exc).__name__)
+
+
# --------------------------------------------------------------- Results Summary
print(f"PASS: {len(PASS)}")
print(f"FAIL: {len(FAIL)}")
diff --git a/tests/test_llamaindex.py b/tests/test_llamaindex.py
index 7b3cbc23..dcfbb4ce 100644
--- a/tests/test_llamaindex.py
+++ b/tests/test_llamaindex.py
@@ -393,6 +393,164 @@ with patch("urllib.request.build_opener") as mock_build_opener:
check("remote/auth", req.headers.get("Authorization"), "Bearer sk-test-token")
+# --------------------------------------------------------------- 6. Per-call decision controls
+#
+# `laya.integrations.langchain` has forwarded `max_len` / `head_max_len` and the five hook
+# arguments since #530 / #532; this surface forwarded only `model`, so a RAG pipeline choosing
+# between more tools than the default head budget fits could not widen its own window. The rule
+# now lives in one module the three integrations import, and the lists below are read out of it
+# rather than written out again here -- a control added to `._controls` without reaching this file
+# fails here. Three classes, three `_execute_decision` call sites, so every surface is driven.
+import inspect
+from laya.agent import Agent
+from laya.integrations import _controls
+from laya.integrations import crewai as crewai_module
+from laya.integrations import langchain as langchain_module
+from laya.integrations import llamaindex as llamaindex_module
+from laya.router import Router
+
+CONTROLS = tuple(_controls.PREDICT_CONTROLS) + tuple(_controls.HOOK_CONTROLS)
+
+
+def _params(fn):
+ return set(inspect.signature(fn).parameters)
+
+
+check("controls/budget tuple names budget_kwargs",
+ set(_params(_controls.budget_kwargs)), set(_controls.PREDICT_CONTROLS))
+check("controls/hook tuple names hook_kwargs",
+ set(_params(_controls.hook_kwargs)), set(_controls.HOOK_CONTROLS))
+
+# Everything called a control here is an argument a real runner accepts, or a selector would fail
+# with a TypeError deep inside core instead of at the call site.
+_agent_params = _params(Agent.system_one)
+_router_params = _params(Router.predict)
+for _c in CONTROLS:
+ check_true("controls/%s accepted by Agent" % _c, _c in _agent_params)
+ check_true("controls/%s accepted by Router.predict" % _c, _c in _router_params)
+
+for cls in (LayaSingleSelector, LayaMultiSelector, LayaQueryRouter):
+ for _c in CONTROLS:
+ check_true("controls/%s takes %s" % (cls.__name__, _c), _c in _params(cls.__init__))
+check("controls/_execute_decision takes every control",
+ set(_params(llamaindex_module._execute_decision)) - {"state", "questions", "agent",
+ "base_url", "api_key", "model"},
+ set(CONTROLS))
+
+# The three wrappers end at the same runner call, so they must accept the same controls.
+for _mod in (langchain_module, crewai_module):
+ check("controls/%s agrees with llamaindex" % _mod.__name__.rsplit(".", 1)[-1],
+ set(_params(_mod._execute_decision)),
+ set(_params(llamaindex_module._execute_decision)))
+
+
+class RecordingAgent:
+ """A runner that keeps the kwargs of every call, the way a pipeline would have to be debugged."""
+
+ device = "cpu"
+
+ def __init__(self):
+ self.calls = []
+
+ def predict(self, state, questions, **kwargs):
+ self.calls.append(kwargs)
+ return {
+ "answers": {
+ "selector": {"choice": "choice_0", "confidence": 0.9, "answer_confidence": 0.9,
+ "probabilities": {"choice_0": 0.9, "choice_1": 0.4}},
+ "route": {"choice": "sql", "confidence": 0.9, "answer_confidence": 0.9},
+ },
+ "routing": {"model": "english", "repo": None, "reason": "explicit model"},
+ }
+
+
+ALL_CONTROLS = {"max_len": 1024, "head_max_len": 512, "hooks": ["H"], "on_predict_start": "S",
+ "on_predict_end": "E", "hooks_raise": True, "hooks_timeout": 0.5}
+
+# One entry per class that can take a decision here: (label, class, constructor extras, one call).
+SURFACES = [
+ ("single", LayaSingleSelector, {},
+ lambda s: s.select(tools, "Run a query on the SQL table")),
+ ("multi", LayaMultiSelector, {},
+ lambda s: s.select(tools, "Run a query on the SQL table")),
+ ("query_router", LayaQueryRouter, {"query_engines": engines},
+ lambda s: s.route("Run a query on the SQL table")),
+]
+
+HOOK_SAMPLES = (("hooks", [object()]), ("on_predict_start", object()),
+ ("on_predict_end", object()), ("hooks_raise", False), ("hooks_timeout", 0.5))
+
+
+def local_kwargs(cls, extra, run, **controls):
+ """Run one decision on a fresh surface, or report the surface's own refusal.
+
+ The broad except is deliberate: a wrapper that cannot even be built or that drops an attribute
+ has to show up as a named mismatch, not as a traceback that hides every later check.
+ """
+ agent = RecordingAgent()
+ try:
+ run(cls(agent=agent, **dict(extra, **controls)))
+ except Exception as exc:
+ return "%s: %s" % (type(exc).__name__, exc)
+ return agent.calls[0]
+
+
+def remote_body(cls, extra, run, **controls):
+ """POST through the real urllib path with the opener mocked, and return the JSON body sent."""
+ with patch("urllib.request.build_opener") as mock_build_opener:
+ mock_opener = MagicMock()
+ mock_opener.open.return_value = DummyHTTPResponse(remote_response)
+ mock_build_opener.return_value = mock_opener
+ run(cls(base_url="http://localhost:8000", **dict(extra, **controls)))
+ return json.loads(mock_opener.open.call_args[0][0].data)
+
+
+for _label, _cls, _extra, _run in SURFACES:
+ check("controls/%s with nothing set sends nothing" % _label,
+ local_kwargs(_cls, _extra, _run), {})
+ check("controls/%s forwards all seven" % _label,
+ local_kwargs(_cls, _extra, _run, **ALL_CONTROLS), ALL_CONTROLS)
+ check("controls/%s forwards one budget alone" % _label,
+ local_kwargs(_cls, _extra, _run, head_max_len=256), {"head_max_len": 256})
+ # 0 and [] are decisions, not absences: truthiness tests here would drop them.
+ check("controls/%s keeps falsy values" % _label,
+ local_kwargs(_cls, _extra, _run, head_max_len=0, hooks=[], hooks_raise=False),
+ {"head_max_len": 0, "hooks": [], "hooks_raise": False})
+ check("controls/%s alongside model" % _label,
+ local_kwargs(_cls, _extra, _run, model="laya-multilingual", max_len=1024),
+ {"model": "laya-multilingual", "max_len": 1024})
+ check("controls/%s remote body carries both budgets" % _label,
+ {k: v for k, v in remote_body(_cls, _extra, _run, max_len=1024,
+ head_max_len=384).items()
+ if k in _controls.PREDICT_CONTROLS},
+ {"max_len": 1024, "head_max_len": 384})
+ check("controls/%s remote body omits unset budgets" % _label,
+ [k for k in remote_body(_cls, _extra, _run) if k in _controls.PREDICT_CONTROLS], [])
+ check("controls/%s remote body keeps a zero" % _label,
+ remote_body(_cls, _extra, _run, head_max_len=0).get("head_max_len"), 0)
+ # A hook is a Python callable that runs inside `predict`; a serve node cannot receive one.
+ for _c, _sample in HOOK_SAMPLES:
+ try:
+ remote_body(_cls, _extra, _run, **{_c: _sample})
+ check_true("controls/%s remote refuses %s" % (_label, _c), False, "no error raised")
+ except ValueError as exc:
+ check_true("controls/%s remote refuses %s" % (_label, _c),
+ _c in str(exc) and "laya-serve" in str(exc))
+ except Exception as exc:
+ check_true("controls/%s remote refuses %s" % (_label, _c), False, type(exc).__name__)
+
+# The caller's own hook objects have to arrive, not a copy or a re-wrapped stand-in. Read with
+# `.get()` so a surface that drops them reports a named failure instead of a KeyError that hides
+# the rest of the run.
+_sentinel_hooks = [RecordingAgent()]
+_ident_agent = RecordingAgent()
+LayaSingleSelector(agent=_ident_agent, hooks=_sentinel_hooks).select(
+ tools, "Run a query on the SQL table")
+_seen_hooks = _ident_agent.calls[0].get("hooks")
+check_true("controls/forwards the caller's objects",
+ _seen_hooks is _sentinel_hooks and _seen_hooks[0] is _sentinel_hooks[0], repr(_seen_hooks))
+
+
# --------------------------------------------------------------- Results Summary
print(f"PASS: {len(PASS)}")
print(f"FAIL: {len(FAIL)}")