Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(hooks): opt-in prediction hooks to observe or shape every decision (#270)
Emiliano G.O.b668a1b6bcd7
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 52ae7c16..3c996a08 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -59,6 +59,8 @@ jobs:python tests/test_router.pypython tests/test_criteria.pypython tests/test_batch.py+ python tests/test_hooks.py+ python tests/test_hooks_api.pypython tests/test_download.pypython tests/test_shortlist.pypython tests/test_decision_model.pydiff --git a/README.md b/README.mdindex 286815eb..a0869a4a 100644--- a/README.md+++ b/README.md@@ -444,6 +444,29 @@ else:---+## Prediction Hooks++Hooks observe or shape every decision without forking: audit logging, PII redaction before+inference, caching, metrics, confidence gating, routing overrides, and forwarding to an+external service. They are opt-in, and unset hooks are a no-op.++```python+import laya++def log(ctx):+ print(ctx.model, ctx.results[0]["answers"], ctx.elapsed_ms)++agent = laya.load("convaiinnovations/laya", on_predict_end=log)+agent.system_one("I was charged twice.", {"urgent": {"type": "noul", "instructions": "Urgent?"}})+```++A hook is a plain callable, or an object implementing any of `on_predict_start`,+`on_predict_end`, `on_route`, `on_load`, `on_evict`, `on_error`. A start hook can rewrite the+state/questions or `ctx.skip(...)` a cached answer; an end hook can rewrite the result. See+[**`docs/hooks/`**](docs/hooks/index.md) and [`examples/hooks/`](examples/hooks/).++---+## Built-in Workflow PresetsLaya provides pre-tuned question schemas for immediate production use:diff --git a/docs/hooks/api.md b/docs/hooks/api.mdnew file mode 100644index 00000000..ac5b3a84--- /dev/null+++ b/docs/hooks/api.md@@ -0,0 +1,283 @@+# API reference++Everything on this page is importable from `laya` (the common names) or `laya.hooks` (the+whole surface).++```python+from laya import PredictContext, PredictHook, Hook+from laya.hooks import HOOK_EVENTS, normalise_hooks, dispatch, aggregate_usage+```++- [PredictContext](#predictcontext)+- [Hook protocol](#hook-protocol)+- [Convenience types](#convenience-types)+- [Configuration surface](#configuration-surface)+- [Event payloads](#event-payloads)+- [Validation](#validation)+- [Advanced helpers](#advanced-helpers)++## PredictContext++A `PredictContext` is created once per public call and passed to every hook of that call. It is+**mutable**: hooks may rewrite `states`, `questions` and `results`, and `on_route` may rewrite+`decision`. It uses identity equality (`eq=False`), so a context is hashable and two contexts are+never equal.++```python+@dataclass(eq=False)+class PredictContext:+ states: List[Any]+ questions: Dict[str, Any]+ run_id: str = <uuid4 hex>+ results: Optional[List[Dict[str, Any]]] = None+ decision: Optional[Dict[str, Any]] = None+ model: Optional[str] = None+ agent: Any = None+ router: Any = None+ max_len: Optional[int] = None+ head_max_len: Optional[int] = None+ usage: Optional[Dict[str, int]] = None+ started_at: float = <perf_counter()>+ elapsed_ms: Optional[float] = None+ error: Optional[BaseException] = None+```++| field | type | set when | mutable | meaning |+|---|---|---|---|---|+| `states` | `list` | always | yes (start) | the states for this call. `system_one`/`Router.predict` pass one; `predict_batch` passes many. A start hook may replace the list. |+| `questions` | `dict` | always | yes (start) | the questions. A start hook may replace the dict. |+| `run_id` | `str` | always | no | a unique id shared by every hook of this call. Use it to correlate events and spans. |+| `results` | `list \| None` | end (and on a skip) | yes (end) | per-state result dicts, each shaped like `system_one`'s return. `None` until inference finishes. |+| `decision` | `dict \| None` | Router only | yes (route) | the `RouteDecision` (a `dict`) that selected the checkpoint. |+| `model` | `str \| None` | always | no | the checkpoint id: `Agent.model_id` for an Agent, the resolved alias (for example `"english"`) for a Router. |+| `agent` | `Agent \| ONNXAgent \| None` | predict events | no | the runtime answering the call. |+| `router` | `Router \| None` | Router events | no | the Router, when one is in play. |+| `max_len` | `int \| None` | always | yes (start) | per-call token budget for the encoder. `None` uses the agent config. |+| `head_max_len` | `int \| None` | always | yes (start) | per-call token budget for the question head. `None` uses the agent config. |+| `usage` | `dict \| None` | end | yes (end) | `{"input_tokens", "output_tokens"}`, summed over the states of the call. |+| `started_at` | `float` | always | no | `time.perf_counter()` when the call began. |+| `elapsed_ms` | `float \| None` | end | no | wall time for the whole call, milliseconds. |+| `error` | `BaseException \| None` | failure path | no | the exception, set before `on_error` and `on_predict_end`. |++### `PredictContext.skip(results)`++Short-circuits inference. Called from `on_predict_start`, it sets `ctx.results` so the forward+pass is skipped; `on_predict_end` still runs and the supplied results are returned.++```python+def cache_read(ctx):+ hit = CACHE.get(key(ctx.states[0], ctx.questions))+ if hit is not None:+ ctx.skip([hit]) # list of per-state results, same shape as predict_batch's return+```++On the `Router`, a skipped payload gets a `routing` key added (without overwriting one it+already has), so `Router.predict` keeps its documented return shape.++## Hook protocol++`Hook` is a `typing.Protocol`. Implement any subset of the methods; the rest are skipped.++```python+class Hook(Protocol):+ def on_predict_start(self, ctx: PredictContext) -> None: ...+ def on_predict_end(self, ctx: PredictContext) -> None: ...+ def on_route(self, ctx: PredictContext) -> None: ...+ def on_load(self, ctx: PredictContext) -> None: ...+ def on_evict(self, ctx: PredictContext) -> None: ...+ def on_error(self, ctx: PredictContext) -> None: ...+```++| event | where | runs | can change |+|---|---|---|---|+| `on_predict_start` | Agent, Router | before tokenization/forward | `states`, `questions`, or `skip()` |+| `on_predict_end` | Agent, Router | after results exist, success or failure | `results` |+| `on_route` | Router | after detection, before loading | `decision` |+| `on_load` | Router | after a checkpoint is built | nothing (observe) |+| `on_evict` | Router | after a checkpoint is freed | nothing (observe) |+| `on_error` | Agent, Router | when a predict call fails | nothing (observe) |++A hook is free to define extra attributes and methods; only the six event names are consulted.+If a hook defines one of the six as a non-callable, configuration fails fast (see+[Validation](#validation)).++## Convenience types++```python+PredictHook = Callable[[PredictContext], None]+```++`PredictHook` is the type of a plain callable used with `on_predict_start=` / `on_predict_end=`.+Pass a single callable or a sequence of them; each is wrapped into a minimal hook.++## Runtime registration++Every runtime mixes in `HookRegistry`, so hooks can be added, removed or scoped after+construction. Mutation is thread-safe; a call reads a snapshot of the list, so adding or+removing a hook never disturbs a call in flight.++```python+agent.add_hook(tracer) # one hook or a sequence; returns self for chaining+agent.remove_hook(tracer) # by identity; True if it was installed++with agent.hooks_installed(debug): # installed for the block, removed on exit+ agent.system_one(state, questions)+```++`add_hook` accepts the same objects as `hooks=` (not plain callables). `hooks_installed` takes+any number of hook objects or sequences and restores the previous list on exit, including when+the block raises.++## Configuration surface++Every entry point accepts the same five hook parameters. `hooks` takes an object or a sequence+of objects; `on_predict_start` / `on_predict_end` take a callable or a sequence.++| parameter | type | default | meaning |+|---|---|---|---|+| `hooks` | `Hook \| Sequence[Hook] \| None` | `None` | lifecycle hooks (any of the six events). |+| `on_predict_start` | `PredictHook \| Sequence[PredictHook] \| None` | `None` | convenience callables for one event. |+| `on_predict_end` | `PredictHook \| Sequence[PredictHook] \| None` | `None` | convenience callables for one event. |+| `hooks_raise` | `bool` | `True` | `True`: a hook exception propagates. `False`: warn and continue. |+| `hooks_concurrent` | `bool` | `True` | `False`: dispatch hooks under a lock, one at a time. |++### Agent++```python+Agent(+ model_id_or_path="convaiinnovations/laya",+ device=None, token=None, subfolder=None, fast=False, compile=False,+ hooks=None, on_predict_start=None, on_predict_end=None,+ hooks_raise=True, hooks_concurrent=True,+)++load(..., hooks=None, on_predict_start=None, on_predict_end=None,+ hooks_raise=True, hooks_concurrent=True)++agent.predict_batch(states, questions, batch_size=None,+ hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None,+ max_len=None, head_max_len=None)++agent.system_one(state, questions,+ hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None,+ max_len=None, head_max_len=None)++agent.predict(...) # alias of system_one+```++- `hooks_raise` on a per-call method defaults to `None`, meaning "use the instance value".+- `hooks_concurrent` is instance-level only.++### Router++```python+Router(+ models=None, device=None, token=None, max_loaded=2, default="english",+ auto_task_detection=False, standalone_repos=False, preload=False, lang_guess=None,+ hooks=None, on_predict_start=None, on_predict_end=None,+ hooks_raise=True, hooks_concurrent=True,+)++router.route(state, questions=None, model=None, task=None, lang=None, lang_guess=None,+ hooks=None, hooks_raise=None)++router.predict(state, questions, model=None, task=None, lang=None, lang_guess=None,+ hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None,+ max_len=None, head_max_len=None)++router.system_one(...) # alias of predict+router.load(name) # builds on first use; fires on_load+router.preload(names=None) # builds several; fires on_load per build+router.unload(name=None) # frees one or all; fires on_evict+router.attach(name, agent) # registers an existing agent; does not fire on_load+router.loaded # list of resident checkpoint names+```++- Per-call `hooks=` on `route` and `predict` apply to the whole call, including `on_route`.+- `route()` is public: calling it dispatches `on_route` with the installed hooks plus any+ per-call `hooks`.++### ONNXAgent++```python+ONNXAgent(model_id_or_path, onnx_path="laya.onnx", subfolder=None,+ hooks=None, on_predict_start=None, on_predict_end=None,+ hooks_raise=True, hooks_concurrent=True)++onnx_agent.system_one(state, questions,+ hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None,+ max_len=None, head_max_len=None)++onnx_agent.predict(...) # alias of system_one+```++`ONNXAgent` has no Router, so it exposes only the predict-level events.++## Event payloads++Which fields are populated, per event and runtime:++| event | runtime | `states` | `questions` | `decision` | `model` | `agent` | `router` | `results` | `usage` | `elapsed_ms` | `error` |+|---|---|---|---|---|---|---|---|---|---|---|---|+| `on_predict_start` | Agent | ✓ | ✓ | – | ✓ | ✓ | – | – | – | – | – |+| `on_predict_start` | Router | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | – | – | – | – |+| `on_predict_end` | Agent | ✓ | ✓ | – | ✓ | ✓ | – | ✓ | on success | ✓ | on failure |+| `on_predict_end` | Router | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | on success | ✓ | on failure |+| `on_error` | both | ✓ | ✓ | ✓ (Router) | ✓ | ✓ | ✓ (Router) | – | – | – | ✓ |+| `on_route` | Router | ✓ | ✓ | ✓ | – | – | ✓ | – | – | – | – |+| `on_load` | Router | `[]` | `{}` | – | ✓ | ✓ | ✓ | – | – | – | – |+| `on_evict` | Router | `[]` | `{}` | – | ✓ | – | ✓ | – | – | – | – |++Timing details:++- `on_predict_end` sees `results` on the success path. On the failure path `results` is `None`+ unless a start hook had set them via `skip()`, and `usage` is therefore `None` too (it is+ derived from `results`); `elapsed_ms` is always set.+- `on_error` runs before the `finally` block that computes `elapsed_ms` and `usage`, so both are+ `None` there. Read timing and usage from `on_predict_end` instead.+- `run_id` is always populated.++## Validation++Configuration is validated when hooks are normalised, which happens at construction for+installed hooks and at call time for per-call hooks. These raise `TypeError`:++| case | message |+|---|---|+| a class is passed instead of an instance | `hooks entries must be instances, not classes; ...` |+| an object implements none of the six events | `hooks entries must implement at least one of ...` |+| an event attribute is not callable | `hooks entry X.on_predict_start must be callable, got int` |+| `on_predict_start=` / `on_predict_end=` is not callable | `on_predict_start must be callable, got int` |++`hooks=` does not accept plain callables, because a bare callable does not say *which* event it+is for. Use `on_predict_start=` / `on_predict_end=` for those.++## Advanced helpers++These are used internally and are stable, but most users do not need them.++```python+HOOK_EVENTS # tuple of the six event names, in dispatch order+normalise_hooks(hooks=None, on_predict_start=None, on_predict_end=None) -> list+dispatch(hooks, event, ctx, *, raise_errors=True, lock=None) -> None+aggregate_usage(results) -> {"input_tokens": int, "output_tokens": int}+```++`normalise_hooks` flattens a `hooks` object/sequence and the two callables into one ordered list.+`dispatch` calls `event` on every hook that implements it, applying the raise policy and lock.+`aggregate_usage` sums per-state usage blocks.++```python+from laya.hooks import normalise_hooks, dispatch, PredictContext++hooks = normalise_hooks(on_predict_start=[log, redact])+ctx = PredictContext(states=["..."], questions={...})+dispatch(hooks, "on_predict_start", ctx)+```++## See also++- [Lifecycle](lifecycle.md): when each event runs, with flowcharts.+- [Errors](errors.md): the failure matrix and chaining rules.+- [Patterns and anti-patterns](patterns.md): how to structure hooks well.+- [Examples](examples.md): recipes for every use case.diff --git a/docs/hooks/errors.md b/docs/hooks/errors.mdnew file mode 100644index 00000000..90e40005--- /dev/null+++ b/docs/hooks/errors.md@@ -0,0 +1,142 @@+# Error handling++Hooks run inside the request they observe, so how their failures are handled matters. This page+is the exact policy.++- [The two policies](#the-two-policies)+- [What runs when something fails](#what-runs-when-something-fails)+- [Exception chaining](#exception-chaining)+- [BaseException](#baseexception)+- [Configuration errors](#configuration-errors)+- [Warnings](#warnings)+- [Choosing a policy](#choosing-a-policy)++## The two policies++`hooks_raise` controls what happens when a hook raises.++| `hooks_raise` | behaviour |+|---|---|+| `True` (default) | the hook exception propagates out of the call. |+| `False` | the hook is skipped with a `RuntimeWarning` and the call continues. |++It is set per instance and can be overridden per call (`hooks_raise=` on `predict_batch`,+`system_one`, `Router.route`, `Router.predict`). Per-call `None` means "use the instance value".++```python+# strict: a broken audit hook fails the request+laya.load("convaiinnovations/laya", on_predict_end=audit, hooks_raise=True)++# lenient: telemetry must never take down a served request+laya.load("convaiinnovations/laya", on_predict_end=metrics, hooks_raise=False)+```++`dispatch` catches `Exception`. Anything that is not an `Exception` (see+[BaseException](#baseexception)) is never swallowed, even with `hooks_raise=False`.++## What runs when something fails++The predict lifecycle is wrapped in `try / except / finally`, so cleanup hooks run on failure.++```+try:+ on_predict_start+ inference+except BaseException as exc:+ ctx.error = exc+ on_error (best effort; cannot mask exc)+ raise+finally:+ elapsed_ms, usage+ on_predict_end (best effort; cannot mask exc on the failure path)+```++Failure matrix, per event and runtime:++| event | runtime | if it raises |+|---|---|---|+| `on_predict_start` | Agent / Router | `hooks_raise=True`: `on_error` and `on_predict_end` still run, then the exception propagates. `False`: warn and continue (mutations made before the raise remain). |+| inference | Agent / Router | `ctx.error` set, `on_error` runs, `on_predict_end` runs, exception propagates. |+| `on_error` | Agent / Router | never masks the original exception; chained as `__context__`. |+| `on_predict_end` (success path) | Agent / Router | `hooks_raise=True`: propagates (the result is computed but the call fails). `False`: warn. |+| `on_predict_end` (failure path) | Agent / Router | never masks the original exception; chained as `__context__`. |+| `on_route` | Router | propagates directly; there is no predict context yet. |+| `on_load` | Router | propagates directly; the checkpoint stays built and resident. |+| `on_evict` | Router | propagates directly; the checkpoint is already freed. |++Consequences worth knowing:++- A failing `on_load` leaves the model cached, so the next `load` returns it without firing+ `on_load` again.+- A failing `on_predict_end` on the success path means the caller gets an exception instead of a+ result, even though inference succeeded. Use `hooks_raise=False` for end hooks that are pure+ side effects.++## Exception chaining++When a hook fails while another exception is already propagating, the original exception is+re-raised and the hook's exception is attached as `__context__`. The root cause is never lost.++```python+class BadTelemetry:+ def on_error(self, ctx):+ raise RuntimeError("telemetry down")++try:+ agent.system_one(state, questions, hooks=[BadTelemetry()])+except RuntimeError as exc:+ assert exc.__context__ is not None # the telemetry failure+```++The same rule applies to a failing `on_predict_end` on the failure path.++## BaseException++`dispatch` catches `Exception`, not `BaseException`, so `KeyboardInterrupt` and `SystemExit`+always propagate. They still trigger the `except BaseException` branch of the predict lifecycle,+which means `on_error` and `on_predict_end` run before the process unwinds. Keep those hooks+fast and non-blocking if you care about interrupt latency.++## Configuration errors++Bad configuration fails fast with `TypeError`, before any inference:++| case | raised at | example |+|---|---|---|+| class instead of instance | construction | `hooks=[MyHook]` |+| no lifecycle method | construction | `hooks=[object()]` |+| non-callable event | construction | `on_predict_start = 5` |+| non-callable convenience hook | construction | `on_predict_start=123` |+| plain callable in `hooks=` | construction | `hooks=[lambda ctx: None]` |++Per-call hooks are validated when the call is made, so a bad per-call hook raises `TypeError`+from `predict`/`system_one` rather than at construction.++## Warnings++With `hooks_raise=False`, each failing hook emits one `RuntimeWarning` naming the hook type and+event:++```+laya: hook Metrics.on_predict_end failed: connection reset+```++The warning is emitted once per failure, not once per hook definition, so a flaky hook under+load can be noisy. Aggregate or rate-limit inside the hook if that matters.++## Choosing a policy++| hook kind | recommended | why |+|---|---|---|+| policy / guardrail / redaction | `hooks_raise=True` | a policy that silently fails is a security hole. |+| audit / logging | `hooks_raise=True` in tests, often `False` in production | losing an audit record should be loud, but not necessarily fatal. |+| metrics / tracing | `hooks_raise=False` | observability must not fail the request. |+| cache read/write | `hooks_raise=True` | a broken cache should surface, not silently miss. |++You can mix: install a guardrail with the instance default and give a telemetry hook its own+try/except, or use a separate `hooks_raise` per call.++## See also++- [Lifecycle](lifecycle.md): the `try/except/finally` shape in context.+- [Patterns and anti-patterns](patterns.md): common mistakes with error handling.diff --git a/docs/hooks/examples.md b/docs/hooks/examples.mdnew file mode 100644index 00000000..242a44e5--- /dev/null+++ b/docs/hooks/examples.md@@ -0,0 +1,340 @@+# Examples++Copy-paste recipes. Every snippet is self-contained apart from the helpers it names+(`ship`, `CACHE`, and so on), which you supply.++- [Quick start](#quick-start)+- [Audit](#audit)+- [Redact PII](#redact-pii)+- [Cache](#cache)+- [Metrics](#metrics)+- [Guardrail](#guardrail)+- [Confidence gate](#confidence-gate)+- [Routing pin](#routing-pin)+- [Lifecycle](#lifecycle)+- [Composition](#composition)+- [Per-call hooks](#per-call-hooks)+- [Batch](#batch)+- [HTTP server](#http-server)+- [ONNXAgent](#onnxagent)+- [Runtime registration](#runtime-registration)+- [Token budget](#token-budget)+- [Testing hooks](#testing-hooks)++## Quick start++```python+import laya++def log(ctx):+ print(ctx.model, ctx.results[0]["answers"])++agent = laya.load("convaiinnovations/laya", on_predict_end=log)+agent.system_one("I was charged twice.", {"urgent": {"type": "noul", "instructions": "Urgent?"}})+```++## Audit++The browser-use use case: capture every decision and ship it to an external service.++```python+import json, sys+import laya++def audit(ctx):+ record = {+ "run_id": ctx.run_id,+ "model": ctx.model,+ "routing": ctx.results[0].get("routing") if ctx.results else None,+ "answers": ctx.results[0]["answers"] if ctx.results else None,+ "usage": ctx.usage,+ "elapsed_ms": round(ctx.elapsed_ms or 0.0, 3),+ }+ print(json.dumps(record), file=sys.stderr)+ # ship_to_service(record)++agent = laya.load("convaiinnovations/laya", on_predict_end=audit)+```++A full runnable version is in [`examples/hooks/audit.py`](../../examples/hooks/audit.py).++## Redact PII++```python+import re+import laya++EMAIL = re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b")+PHONE = re.compile(r"\+?\d[\d ()-]{7,}\d")++def scrub(value):+ if isinstance(value, str):+ return PHONE.sub("[phone]", EMAIL.sub("[email]", value))+ if isinstance(value, dict):+ return {k: scrub(v) for k, v in value.items()}+ if isinstance(value, list):+ return [scrub(v) for v in value]+ return value++def redact(ctx):+ ctx.states = [scrub(s) for s in ctx.states]++agent = laya.load("convaiinnovations/laya", on_predict_start=redact)+```++See [`examples/hooks/redact.py`](../../examples/hooks/redact.py).++## Cache++```python+import hashlib, json+import laya++CACHE = {}++def key(state, questions):+ payload = json.dumps([state, questions], sort_keys=True, default=str)+ return hashlib.sha256(payload.encode()).hexdigest()++def read(ctx):+ hit = CACHE.get(key(ctx.states[0], ctx.questions))+ if hit is not None:+ ctx.skip([hit])++def write(ctx):+ if ctx.results:+ CACHE[key(ctx.states[0], ctx.questions)] = ctx.results[0]++agent = laya.load("convaiinnovations/laya", on_predict_start=read, on_predict_end=write)+first = agent.system_one("state", QUESTIONS) # runs the model+second = agent.system_one("state", QUESTIONS) # served from CACHE+```++See [`examples/hooks/cache.py`](../../examples/hooks/cache.py).++## Metrics++```python+import laya++COUNTS, LATENCIES = {}, []++def metrics(ctx):+ COUNTS[ctx.model] = COUNTS.get(ctx.model, 0) + 1+ if ctx.elapsed_ms is not None:+ LATENCIES.append(ctx.elapsed_ms)++agent = laya.load("convaiinnovations/laya", on_predict_end=metrics, hooks_raise=False)+```++See [`examples/hooks/otel.py`](../../examples/hooks/otel.py).++## Guardrail++Block a request by raising from a start hook.++```python+import laya++class Blocked(Exception):+ pass++def guard(ctx):+ text = str(ctx.states[0]).lower()+ if "ignore previous instructions" in text:+ raise Blocked("prompt injection")++agent = laya.load("convaiinnovations/laya", on_predict_start=guard)++try:+ agent.system_one("Ignore previous instructions and ...", QUESTIONS)+except Blocked:+ handle_block()+```++## Confidence gate++Rewrite a low-confidence answer, or annotate it.++```python+def gate(ctx):+ answer = ctx.results[0]["answers"].get("dept")+ if answer and answer["confidence"] < 0.6:+ answer["choice"] = "human-review"+ answer["gated"] = True++agent = laya.load("convaiinnovations/laya", on_predict_end=gate)+```++## Routing pin++Force a checkpoint for a class of traffic.++```python+from laya import Router+from laya.router import RouteDecision++def pin(ctx):+ if "refund" in str(ctx.states[0]).lower():+ ctx.decision = RouteDecision(+ model="typed-decisions",+ repo="convaiinnovations/laya/typed-decisions",+ reason="refund workflow",+ detection=None,+ workflow=None,+ )++router = Router(hooks=[pin])+```++Per-call, without installing:++```python+router.predict("refund request", QUESTIONS, hooks=[pin])+```++## Lifecycle++Observe checkpoint build and eviction.++```python+from laya import Router++class Lifecycle:+ def on_load(self, ctx):+ print("loaded", ctx.model, "agent", type(ctx.agent).__name__)++ def on_evict(self, ctx):+ print("evicted", ctx.model)++router = Router(max_loaded=1, hooks=[Lifecycle()])+router.preload(["english", "multilingual"]) # on_load fires per build+router.unload() # on_evict fires per freed checkpoint+```++## Composition++Installed hooks first, then convenience callables; all share one context.++```python+import laya++class Metrics:+ def on_predict_end(self, ctx):+ record_latency(ctx.model, ctx.elapsed_ms)++def redact(ctx):+ ctx.states = [strip_pii(s) for s in ctx.states]++def audit(ctx):+ ship(ctx.run_id, ctx.results)++agent = laya.load(+ "convaiinnovations/laya",+ hooks=[Metrics()], # installed, runs first+ on_predict_start=redact, # convenience, appended+ on_predict_end=audit, # convenience, appended+ hooks_raise=True,+)+```++## Per-call hooks++Override or extend hooks for a single call.++```python+agent.system_one(+ state,+ questions,+ on_predict_end=lambda ctx: debug_dump(ctx),+ hooks_raise=False,+)++router.predict(+ state,+ questions,+ hooks=[pin], # applies to on_route too+ on_predict_end=audit,+)+```++## Batch++Hooks fire once per `predict_batch` call, with `ctx.states` holding every state.++```python+def audit_batch(ctx):+ for state, result in zip(ctx.states, ctx.results):+ ship_one(ctx.run_id, state, result)++results = agent.predict_batch([state_a, state_b, state_c], questions, on_predict_end=audit_batch)+```++## HTTP server++Router hooks fire for `laya.serve` automatically, because the server calls `Router.predict`.++```python+from laya import Router+from laya.serve import create_app++router = Router(hooks=[Metrics()], on_predict_end=audit, hooks_raise=False)+app = create_app(router=router)+```++## ONNXAgent++`ONNXAgent` exposes the predict-level events only.++```python+from laya.onnx_agent import ONNXAgent++agent = ONNXAgent("convaiinnovations/laya", onnx_path="laya.onnx", on_predict_end=audit)+agent.system_one(state, questions)+```++## Runtime registration++Attach, detach or scope hooks after construction.++```python+agent.add_hook(Metrics()) # attach at runtime+agent.remove_hook(Metrics()) # by identity++with agent.hooks_installed(DebugDump()):+ agent.system_one(state, questions) # DebugDump only here+```++## Token budget++Shape the token budget for one call, from a hook or a per-call argument.++```python+def widen(ctx):+ k = len(next(iter(ctx.questions.values())).get("criteria", {}) or {})+ if k >= 50:+ ctx.head_max_len = 16 + 4 * k++agent = laya.load("convaiinnovations/laya", on_predict_start=widen)++# or per call+agent.system_one(state, questions, head_max_len=324, max_len=1024)+```++## Testing hooks++Assert what a hook saw without a model: drive `predict_batch` with the encode/forward/decode+helpers stubbed, as [`tests/test_hooks.py`](../../tests/test_hooks.py) does.++```python+seen = []+agent.predict_batch(["s0"], questions, on_predict_end=lambda ctx: seen.append(ctx.results))+assert len(seen) == 1+```++The API surface is pinned by [`tests/test_hooks_api.py`](../../tests/test_hooks_api.py).++## See also++- [Tracing](tracing.md): `run_id`, spans, OpenTelemetry.+- [Patterns and anti-patterns](patterns.md): the reasoning behind these recipes.diff --git a/docs/hooks/index.md b/docs/hooks/index.mdnew file mode 100644index 00000000..1f5f75fe--- /dev/null+++ b/docs/hooks/index.md@@ -0,0 +1,127 @@+# Prediction Hooks++Hooks let you observe or shape every decision Laya makes, without forking it.++They are the extension seam for the things every real deployment needs: audit logging, PII+redaction before inference, caching, metrics, confidence gating, routing overrides, and+forwarding a decision to an external service. They are **opt-in**: with no hooks configured the+behaviour of `Agent`, `Router` and `ONNXAgent` is unchanged.++This folder is the full reference. Start here, then dive into the page you need:++| page | what is in it |+|---|---|+| [API reference](api.md) | every class, field, parameter and default |+| [Lifecycle](lifecycle.md) | exactly when each hook runs, with flowcharts |+| [Errors](errors.md) | `hooks_raise`, `on_error`, exception chaining, failure matrix |+| [Patterns and anti-patterns](patterns.md) | what to do, what to avoid, and why |+| [Examples](examples.md) | copy-paste recipes for every use case |+| [Tracing](tracing.md) | `run_id`, span correlation, OpenTelemetry |++## Quick start++```python+import laya++def log(ctx):+ print(ctx.model, ctx.results[0]["answers"], ctx.elapsed_ms)++agent = laya.load("convaiinnovations/laya", on_predict_end=log)+agent.system_one("I was charged twice.", {"urgent": {"type": "noul", "instructions": "Urgent?"}})+```++An object can implement any subset of the lifecycle events:++```python+class Audit:+ def on_predict_start(self, ctx):+ print("start", ctx.run_id)++ def on_predict_end(self, ctx):+ print("end", ctx.run_id, ctx.usage, ctx.elapsed_ms)++ def on_error(self, ctx):+ print("failed", ctx.run_id, ctx.error)++laya.load("convaiinnovations/laya", hooks=[Audit()])+```++Hooks can also be added later or scoped to a block:++```python+agent.add_hook(tracer) # attach at runtime+with agent.hooks_installed(debug): # installed for the block, removed on exit+ agent.system_one(state, questions)+```++See [runtime registration](api.md#runtime-registration).++## The mental model++There are three ideas.++1. **A hook is a callable or an object.** A plain function is convenient for one event; an+ object is convenient for several. Both are passed to `hooks=` / `on_predict_start=` /+ `on_predict_end=`.++2. **Every hook of one call shares one mutable `PredictContext`.** It carries the states,+ questions, results, routing decision, model name, usage, timing and any error. Because it is+ mutable, a hook can *shape* the call, not only watch it: redact the state, rewrite the+ questions, replace the result, or skip inference with a cached answer.++3. **There are two scopes.** `Agent` hooks wrap a forward pass; `Router` hooks wrap routing plus+ inference and can also see model lifecycle (`on_route`, `on_load`, `on_evict`). This mirrors+ the "run hooks" vs "agent hooks" split in other agent frameworks.++```+ Router.predict(state, questions)+ ┌──────────────────────────────────────────────────────────────────────────┐+ │ route() │+ │ ├─ detect language / workflow │+ │ └─ on_route ctx.decision (a hook may replace it) │+ │ │+ │ load(decision.model) │+ │ ├─ build checkpoint on first use ──► on_load ctx.model, ctx.agent │+ │ └─ evict LRU checkpoint ───────────► on_evict ctx.model │+ │ │+ │ on_predict_start ctx.states, ctx.questions, ctx.decision │+ │ │ │+ │ ├── ctx.skip(results)? ──► skip the forward pass │+ │ │ │+ │ └── Agent.system_one(...) ──► Agent-level hooks run here │+ │ on_predict_start ─► forward ─► on_predict_end │+ │ │+ │ result["routing"] = decision │+ │ on_predict_end ctx.results, ctx.usage, ctx.elapsed_ms │+ └──────────────────────────────────────────────────────────────────────────┘+ any failure on the way ──► on_error, then on_predict_end+```++## Scope at a glance++| | `Agent` / `ONNXAgent` | `Router` |+|---|---|---|+| `on_predict_start` | yes | yes |+| `on_predict_end` | yes | yes |+| `on_error` | yes | yes |+| `on_route` | no | yes |+| `on_load` | no | yes |+| `on_evict` | no | yes |++`laya.serve` and the MCP server call `Router.predict`, so Router hooks fire for them+automatically. `Agent` hooks fire whenever the Router runs an attached or built agent.++## Compatibility++- No hooks configured means no behavioural change. The unset path is regression-tested.+- All hook parameters are keyword arguments with defaults, so existing calls keep working.+- `laya/hooks.py` is pure Python: `import laya` does not pull in torch because of it.+- Hooks are synchronous. Keep them fast and non-blocking; see+ [errors](errors.md) and [patterns](patterns.md) for the consequences on `laya.serve`.++## See also++- [`examples/hooks/`](../../examples/hooks/): runnable audit, redact, cache and metrics hooks.+- [`tests/test_hooks.py`](../../tests/test_hooks.py): the behaviour spec.+- [`tests/test_hooks_api.py`](../../tests/test_hooks_api.py): the API-stability guard.+- [`laya/hooks.py`](../../laya/hooks.py): the implementation.diff --git a/docs/hooks/lifecycle.md b/docs/hooks/lifecycle.mdnew file mode 100644index 00000000..b1745f70--- /dev/null+++ b/docs/hooks/lifecycle.md@@ -0,0 +1,200 @@+# Lifecycle++This page is the precise order of events for each entry point. If you only read one diagram,+read the [Router](#routerpredict) one; it is the superset.++- [Agent.predict_batch](#agentpredict_batch)+- [Agent.system_one / predict](#agentsystem_one-predict)+- [Router.predict](#routerpredict)+- [Model lifecycle](#model-lifecycle)+- [Caching with skip](#caching-with-skip)+- [Empty inputs](#empty-inputs)+- [Ordering rules](#ordering-rules)+- [Concurrency](#concurrency)++## Agent.predict_batch++`predict_batch` is the single implementation; `system_one` and `predict` call it with one state.++```+predict_batch(states, questions, batch_size=..., hooks=..., ...)+ │+ ├─ active = installed hooks + per-call hooks (installed first)+ ├─ ctx = PredictContext(states, questions, model=self.model_id, agent=self)+ │+ ├─ try:+ │ │+ │ ├─ on_predict_start ─────────────────────────────┐+ │ │ a hook may: │+ │ │ • rewrite ctx.states / ctx.questions │+ │ │ • set ctx.max_len / ctx.head_max_len │+ │ │ • ctx.skip(results) ─────────────┐ │+ │ │ • raise (aborts; see errors) │ │+ │ │ │ │+ │ ├─ if ctx.results is not None: ◄─────────┘ │ cache hit+ │ │ skip tokenization and forward │+ │ ├─ else: │+ │ │ validate states is a list │+ │ │ for each batch chunk: │+ │ │ _encode_state ─► collate ─► _forward │+ │ │ _decode_answers │+ │ │ ctx.results = [...] │+ │ │ │+ │ └─ (any failure here) ──► except BaseException: │+ │ ctx.error = exc │+ │ on_error │+ │ re-raise │+ │ │+ │ finally: │+ │ ctx.elapsed_ms = now - ctx.started_at │+ │ if ctx.results: ctx.usage = aggregate_usage(...)│+ │ on_predict_end ─────────────────────────────────┘+ │+ └─ return ctx.results+```++The same `ctx` object flows through start, error and end, so `run_id` correlates them and an end+hook can read `ctx.error`.++## Agent.system_one / predict++```+system_one(state, questions, hooks=..., ...)+ └─ predict_batch([state], questions, hooks=..., ...)[0]+```++So `system_one` inherits every hook and the same lifecycle, with `ctx.states == [state]`.++## Router.predict++```+Router.predict(state, questions, model=..., hooks=..., on_predict_start=..., on_predict_end=...)+ │+ ├─ active = installed hooks + per-call hooks+ │+ ├─ route(state, questions, ..., hooks=per-call, hooks_raise=...)+ │ │+ │ ├─ _route(...) detect script / language / workflow+ │ ├─ on_route ──► ctx.decision a hook may replace the decision+ │ └─ return ctx.decision+ │+ ├─ load(decision["model"])+ │ │+ │ ├─ already resident? ──► return it+ │ ├─ else build Agent(...) ──► on_load (after the Router lock is released)+ │ └─ evict LRU checkpoints ──► on_evict (after the Router lock is released)+ │+ ├─ ctx = PredictContext(states=[state], questions, decision, model=decision.model,+ │ agent=agent, router=self)+ ├─ try:+ │ ├─ on_predict_start+ │ ├─ if ctx.results is None:+ │ │ result = agent.system_one(ctx.states[0], ctx.questions)+ │ │ └─ the Agent's own hooks run here (start / forward / end)+ │ │ result["routing"] = decision+ │ │ ctx.results = [result]+ │ └─ else:+ │ for each cached result: result.setdefault("routing", decision)+ │ └─ (any failure) ──► except: on_error, re-raise+ │ └─ finally: elapsed_ms, usage, on_predict_end+ │+ └─ return ctx.results[0]+```++Key points:++- `on_route` runs before the model is loaded, so a hook can pin a checkpoint and avoid loading+ another one.+- Router-level predict hooks wrap the whole call. They are **not** forwarded into the Agent;+ an attached Agent with its own hooks runs those too, which is expected.+- A Router-level `ctx.skip()` still adds `routing`, so the return shape is stable.++## Model lifecycle++`on_load` fires when a checkpoint is built; `on_evict` when one is freed. Both run **after** the+Router's internal lock is released, so a hook may safely call back into the Router.++```+load("multilingual")+ │+ ├─ [lock]+ │ build Agent(...) (seconds: download + weights)+ │ register in _agents / _order+ │ evict LRU if over max_loaded ──► evicted = ["english"]+ ├─ [unlock]+ ├─ on_evict("english")+ └─ on_load("multilingual")++unload("english")+ ├─ [lock] remove from _agents / _order+ ├─ [unlock]+ └─ on_evict("english")+```++`attach(name, agent)` registers an existing agent and does **not** fire `on_load`, because no+checkpoint was built.++## Caching with skip++```+on_predict_start+ ├─ cache hit? ctx.skip([cached_result])+ │ └─ forward pass skipped+ │ └─ on_predict_end still runs+ │ └─ Router adds `routing` if missing+ └─ cache miss? nothing+ └─ forward pass runs+ └─ on_predict_end can store the result+```++See [`examples/hooks/cache.py`](../../examples/hooks/cache.py) for a working cache.++## Empty inputs++Hooks still fire so audit sees every call:++| input | `ctx.results` at end |+|---|---|+| `predict_batch([])` | `[]` |+| `predict_batch(states, {})` (no questions) | one empty-answer payload per state |+| `system_one(state, {})` | a single empty-answer payload |++No tokenization or forward pass happens in these cases, but `on_predict_start` and+`on_predict_end` run.++## Ordering rules++1. Installed hooks run before per-call hooks, always.+2. Within a list, hooks run in list order.+3. For one event, every hook that implements it runs, in that order, before the next event.+4. `on_error` runs before `on_predict_end` on the failure path.+5. `on_evict` runs before `on_load` when a single `load` both evicts and builds.++```+installed: [A, B] per-call: [C]+on_predict_start: A, B, C+on_predict_end: A, B, C+```++## Concurrency++`Agent` and `Router` are safe to call from many threads. Each call creates its own+`PredictContext`, so contexts never leak across requests. The only shared state is the hook+objects themselves, so a hook that is not thread-safe must either guard its own state or be+installed with `hooks_concurrent=False`.++```+hooks_concurrent=True (default) hooks_concurrent=False+ thread 1 ─┐ thread 1 ─┐+ thread 2 ─┼─ hooks run in parallel thread 2 ─┼─ one hook at a time+ thread 3 ─┘ thread 3 ─┘ (RLock)+```++`hooks_concurrent=False` serialises each hook invocation, not whole calls: two calls can still+interleave between events. It uses a re-entrant lock, so a hook may call back into the same+`Agent`/`Router` without deadlocking.++## See also++- [Errors](errors.md): what happens when a hook raises, per event.+- [Patterns and anti-patterns](patterns.md): how to use the lifecycle well.diff --git a/docs/hooks/patterns.md b/docs/hooks/patterns.mdnew file mode 100644index 00000000..d3942852--- /dev/null+++ b/docs/hooks/patterns.md@@ -0,0 +1,398 @@+# Patterns and anti-patterns++Hooks are a small seam, and it is easy to use them well or badly. This page collects the shapes+that hold up in production and the ones that bite.++- [Patterns](#patterns)+ - [Audit log](#audit-log)+ - [PII redaction](#pii-redaction)+ - [Caching](#caching)+ - [Metrics](#metrics)+ - [Guardrails](#guardrails)+ - [Confidence gating](#confidence-gating)+ - [Routing override](#routing-override)+ - [Model lifecycle](#model-lifecycle)+ - [Multi-tenant context](#multi-tenant-context)+ - [Composition](#composition)+ - [Scoped instrumentation](#scoped-instrumentation)+ - [Token-budget shaping](#token-budget-shaping)+- [Anti-patterns](#anti-patterns)+ - [Blocking work](#blocking-work)+ - [Raising from end hooks for control flow](#raising-from-end-hooks-for-control-flow)+ - [Shared mutable state without a lock](#shared-mutable-state-without-a-lock)+ - [Silent failure](#silent-failure)+ - [Retaining contexts](#retaining-contexts)+ - [Redacting too late](#redacting-too-late)+ - [Per-question logic in a per-call hook](#per-question-logic-in-a-per-call-hook)+ - [Recursive predict](#recursive-predict)+ - [Plain callables in `hooks=`](#plain-callables-in-hooks)+ - [Assuming results exist in end hooks](#assuming-results-exist-in-end-hooks)+ - [Order-dependent hooks](#order-dependent-hooks)++## Patterns++### Audit log++Record every decision with enough to reconstruct it: the state, the questions, the answers, the+model, the routing decision, usage and latency.++```python+import json++def audit(ctx):+ json.dump({+ "run_id": ctx.run_id,+ "model": ctx.model,+ "routing": ctx.results[0].get("routing") if ctx.results else None,+ "answers": ctx.results[0]["answers"] if ctx.results else None,+ "usage": ctx.usage,+ "elapsed_ms": round(ctx.elapsed_ms or 0.0, 3),+ }, sys.stdout)+ sys.stdout.write("\n")++laya.load("convaiinnovations/laya", on_predict_end=audit)+```++Make it lenient if losing a log line must not fail a request: `hooks_raise=False`. Make it+strict if the audit trail is a compliance requirement.++### PII redaction++Redaction must happen in `on_predict_start`, before tokenization, or the model has already seen+the data.++```python+import re+EMAIL = re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b")++def redact(ctx):+ ctx.states = [+ EMAIL.sub("[email]", s) if isinstance(s, str) else s+ for s in ctx.states+ ]++laya.load("convaiinnovations/laya", on_predict_start=redact)+```++A redaction hook is a policy hook: keep `hooks_raise=True`, because a silently broken redactor+is a data leak.++### Caching++A start hook checks the cache and calls `ctx.skip(...)`; an end hook fills it. The forward pass+is skipped on a hit.++```python+import hashlib, json++CACHE = {}++def key(state, questions):+ return hashlib.sha256(json.dumps([state, questions], sort_keys=True, default=str).encode()).hexdigest()++def read(ctx):+ hit = CACHE.get(key(ctx.states[0], ctx.questions))+ if hit is not None:+ ctx.skip([hit])++def write(ctx):+ if ctx.results:+ CACHE[key(ctx.states[0], ctx.questions)] = ctx.results[0]++laya.load("convaiinnovations/laya", on_predict_start=read, on_predict_end=write)+```++Guard the cache with a lock when serving concurrently. On the Router the cached payload still+gets a `routing` key, so the return shape is unchanged.++### Metrics++Counters and histograms from `ctx.model`, `ctx.usage` and `ctx.elapsed_ms`. Keep it lenient.++```python+COUNTS, LATENCIES = {}, []++def metrics(ctx):+ COUNTS[ctx.model] = COUNTS.get(ctx.model, 0) + 1+ if ctx.elapsed_ms is not None:+ LATENCIES.append(ctx.elapsed_ms)++laya.load("convaiinnovations/laya", on_predict_end=metrics, hooks_raise=False)+```++### Guardrails++A policy hook raises to block a request. `hooks_raise=True` (the default) lets the block reach+the caller; `on_error` and `on_predict_end` still run, so the audit trail records it.++```python+class Blocked(Exception):+ pass++def guard(ctx):+ if "ssn" in str(ctx.states[0]).lower():+ raise Blocked("possible PII in state")++laya.load("convaiinnovations/laya", on_predict_start=guard)+```++### Confidence gating++An end hook rewrites a low-confidence answer to a safe fallback, or annotates it for downstream+logic. This is a result mutation, not a rejection.++```python+def gate(ctx):+ answer = ctx.results[0]["answers"].get("dept")+ if answer and answer["confidence"] < 0.6:+ answer["choice"] = "human-review"+ answer["gated"] = True++laya.load("convaiinnovations/laya", on_predict_end=gate)+```++### Routing override++`on_route` may replace `ctx.decision` to pin a checkpoint for a class of traffic.++```python+from laya.router import RouteDecision++def pin(ctx):+ if "refund" in str(ctx.states[0]).lower():+ ctx.decision = RouteDecision(+ model="typed-decisions",+ repo="convaiinnovations/laya/typed-decisions",+ reason="refund workflow",+ detection=None,+ workflow=None,+ )++Router(hooks=[pin])+```++### Model lifecycle++`on_load` and `on_evict` observe checkpoints. Use them for warmup logs, memory accounting, or+eviction alerts. They run outside the Router lock, so a hook may call back into the Router.++```python+class Lifecycle:+ def on_load(self, ctx):+ print("loaded", ctx.model)++ def on_evict(self, ctx):+ print("evicted", ctx.model)++Router(hooks=[Lifecycle()])+```++### Multi-tenant context++Thread a tenant id through by capturing it in the hook closure, or by reading it from a+context-local. Do not store per-request state on the hook object without a lock.++```python+def make_audit(tenant):+ def audit(ctx):+ ship(tenant, ctx.run_id, ctx.results)+ return audit++agent = laya.load("convaiinnovations/laya", on_predict_end=make_audit("acme"))+```++### Composition++Several hooks of different kinds compose naturally; installed hooks run first, in order.++```python+agent = laya.load(+ "convaiinnovations/laya",+ hooks=[Metrics(), Guardrail()], # metrics first, then policy+ on_predict_start=redact, # convenience callables appended after hooks+ hooks_raise=True, # policy failures are fatal+)+```++Keep the ordering deliberate and documented, because a later hook sees the mutations of an+earlier one.++### Scoped instrumentation++Attach a tracer or debug hook only for the code that needs it, instead of reconstructing the+agent. `hooks_installed` restores the previous list on exit, even if the block raises.++```python+with agent.hooks_installed(DebugDump()):+ agent.system_one(state, questions) # DebugDump only here+```++`add_hook`/`remove_hook` do the same without a block, for a tracer that lives as long as the+process.++### Token-budget shaping++A start hook can raise the token budget for one call, for example when a question has many+options and the default head budget would collapse the labels. This does not touch the shared+agent config, so concurrent calls are unaffected.++```python+def widen_for_high_cardinality(ctx):+ k = len(next(iter(ctx.questions.values())).get("criteria", {}) or {})+ if k >= 50:+ ctx.head_max_len = max(ctx.head_max_len or 192, 16 + 4 * k)++agent = laya.load("convaiinnovations/laya", on_predict_start=widen_for_high_cardinality)+```++The same knobs are available per call: `agent.system_one(state, questions, head_max_len=324)`.++## Anti-patterns++### Blocking work++Hooks run on the calling thread, and `laya.serve` uses a single inference worker. A hook that+sleeps, waits on a network round-trip, or calls `input()` stalls every other request behind it.++```python+# bad: blocks the whole server+def audit(ctx):+ requests.post("https://slow.example/decisions", json=..., timeout=30)++# better: enqueue, let a background worker ship it+def audit(ctx):+ QUEUE.put_nowait(record(ctx))+```++If you must do slow work, set `hooks_concurrent=False` to at least keep the hook itself from+overlapping, and run `laya.serve` behind a queue.++### Raising from end hooks for control flow++`on_predict_end` runs after inference. Raising there throws away a computed result and, on the+success path, surfaces to the caller. Use a start hook to block before paying for inference, or+rewrite `ctx.results` to change the answer.++### Shared mutable state without a lock++The same hook instance runs on many threads. `self.counter += 1` races.++```python+# bad+class Count:+ def __init__(self): self.n = 0+ def on_predict_end(self, ctx): self.n += 1++# good+import threading+class Count:+ def __init__(self):+ self.n = 0+ self._lock = threading.Lock()+ def on_predict_end(self, ctx):+ with self._lock:+ self.n += 1+```++### Silent failure++`hooks_raise=False` warns once per failure, but a hook that catches everything itself hides+real problems.++```python+# bad: no one will ever know the audit trail stopped+def audit(ctx):+ try:+ ship(record(ctx))+ except Exception:+ pass+```++If a hook is optional, let `hooks_raise=False` handle it and watch the warnings. If it is not,+let it raise.++### Retaining contexts++A hook that appends `ctx` to a list keeps the whole state, questions, results and agent alive.++```python+# bad: unbounded memory growth+SEEN = []+def audit(ctx):+ SEEN.append(ctx)++# good: keep only what you need+SEEN = []+def audit(ctx):+ SEEN.append((ctx.run_id, ctx.model, ctx.elapsed_ms))+```++### Redacting too late++By `on_predict_end` the model has already tokenized the state. Redact in `on_predict_start`.++### Per-question logic in a per-call hook++There is one `PredictContext` per call, and one forward pass answers every question. There are no+per-question events. Iterate the answers inside `on_predict_end`.++```python+def flag(ctx):+ for qid, answer in ctx.results[0]["answers"].items():+ if answer.get("confidence", 1.0) < 0.5:+ alert(qid, ctx.run_id)+```++### Recursive predict++A hook that calls `agent.predict`/`system_one` runs the hooks again. Without a depth guard this+recurses.++```python+# bad+def enrich(ctx):+ ctx.results = [agent.predict(ctx.states[0], EXTRA_QUESTIONS)]++# good: guard, or use a separate agent with no hooks+def enrich(ctx):+ if getattr(ctx, "_enriched", False):+ return+ ctx._enriched = True+ ctx.results = [enricher.predict(ctx.states[0], EXTRA_QUESTIONS)]+```++### Plain callables in `hooks=`++`hooks=` takes hook objects; a bare callable does not say which event it is for, so it is+rejected. Use `on_predict_start=` / `on_predict_end=`.++```python+# bad: TypeError+laya.load("convaiinnovations/laya", hooks=[lambda ctx: None])++# good+laya.load("convaiinnovations/laya", on_predict_end=lambda ctx: None)+```++### Assuming results exist in end hooks++On the failure path `ctx.results` is `None` unless a start hook set it. Always check.++```python+def audit(ctx):+ if ctx.results is None:+ log_failure(ctx.run_id, ctx.error)+ return+ log_success(ctx.run_id, ctx.results)+```++### Order-dependent hooks++A hook that reads a mutation from another hook is fragile unless the order is pinned. Installed+hooks run in list order, then convenience callables; document any coupling, or merge the coupled+hooks into one object.++## See also++- [Errors](errors.md): the failure matrix behind several of these anti-patterns.+- [Examples](examples.md): fuller versions of the patterns above.diff --git a/docs/hooks/tracing.md b/docs/hooks/tracing.mdnew file mode 100644index 00000000..d01c4434--- /dev/null+++ b/docs/hooks/tracing.md@@ -0,0 +1,173 @@+# Tracing++Every hook of one call receives the same `PredictContext.run_id`, so a tracer can correlate the+start, end and error events (and any spans it opens) without keeping its own bookkeeping.++- [run_id](#run_id)+- [A minimal tracer](#a-minimal-tracer)+- [Span lifetime](#span-lifetime)+- [Error spans](#error-spans)+- [OpenTelemetry](#opentelemetry)+- [Distributed propagation](#distributed-propagation)+- [Nested calls](#nested-calls)++## run_id++- A `uuid4().hex` string, created once per public call (`predict_batch`, `system_one`,+ `Router.predict`, `ONNXAgent.system_one`).+- Shared by every hook of that call, including `on_error` and `on_predict_end`.+- Not global and not persisted: it identifies a call within the process. Put it in your logs and+ outbound payloads to correlate across systems.+- Distinct per call, so two calls never collide.++```+call A: run_id=1a2b... start ──┐+ ├─ end+ error ─┘+call B: run_id=9f8e... start ─── end+```++## A minimal tracer++Keep a map from `run_id` to the span you opened at start, and close it at end or error.++```python+import time++class Tracer:+ def __init__(self):+ self.spans = {}++ def on_predict_start(self, ctx):+ self.spans[ctx.run_id] = {+ "model": ctx.model,+ "started_at": ctx.started_at,+ }++ def on_predict_end(self, ctx):+ span = self.spans.pop(ctx.run_id, None)+ if span is None:+ return+ emit_span(+ name="laya.predict",+ run_id=ctx.run_id,+ model=ctx.model,+ duration_ms=ctx.elapsed_ms,+ usage=ctx.usage,+ ok=ctx.error is None,+ )++ def on_error(self, ctx):+ # on_error runs before on_predict_end; leaving the span for on_predict_end is fine,+ # or close it here if you prefer.+ pass++agent = laya.load("convaiinnovations/laya", hooks=[Tracer()])+```++Because `on_predict_end` always runs, it is the natural place to close a span, and it can see+`ctx.error` on the failure path.++## Span lifetime++```+on_predict_start ──► open span (run_id, model, started_at)+ │+ ├─ inference+ │+on_error ──► record ctx.error on the span+ │+on_predict_end ──► close span (elapsed_ms, usage, ok)+```++## Error spans++`on_predict_end` runs on the failure path with `ctx.error` set, so one close site handles both:++```python+def on_predict_end(self, ctx):+ span = self.spans.pop(ctx.run_id, None)+ if span is None:+ return+ if ctx.error is not None:+ span["status"] = "error"+ span["error_type"] = type(ctx.error).__name__+ span["error_message"] = str(ctx.error)+ span["duration_ms"] = ctx.elapsed_ms+ emit(span)+```++If you only implement `on_error`, remember it fires before `on_predict_end`; do not close the+span in both or you will double-count.++## OpenTelemetry++The example [`examples/hooks/otel.py`](../../examples/hooks/otel.py) records counters and a+histogram. For real spans, drive the OTel API from the tracer. Hooks are synchronous, so use the+synchronous exporter (or enqueue and export from a worker).++```python+from opentelemetry import trace++tracer = trace.get_tracer("laya")++class OTelHooks:+ def __init__(self):+ self.spans = {}++ def on_predict_start(self, ctx):+ span = tracer.start_span("laya.predict", attributes={"laya.run_id": ctx.run_id, "laya.model": ctx.model})+ self.spans[ctx.run_id] = span++ def on_predict_end(self, ctx):+ span = self.spans.pop(ctx.run_id, None)+ if span is None:+ return+ if ctx.usage:+ span.set_attribute("laya.input_tokens", ctx.usage["input_tokens"])+ if ctx.error is not None:+ span.record_exception(ctx.error)+ span.set_status(trace.Status(trace.StatusCode.ERROR))+ span.end()++laya.load("convaiinnovations/laya", hooks=[OTelHooks()], hooks_raise=False)+```++Set `hooks_raise=False` so a tracer outage never fails a request.++## Distributed propagation++`run_id` is a plain string, so include it in whatever leaves the process: log lines, the payload+sent to an audit service, or an HTTP header if a decision triggers a downstream call.++```python+def audit(ctx):+ requests.post(+ "https://audit.example/decisions",+ json=record(ctx),+ headers={"X-Laya-Run-Id": ctx.run_id},+ timeout=2,+ )+```++Remember that a blocking call like this stalls the calling thread; enqueue it instead when+serving concurrently. See [patterns](patterns.md#blocking-work).++## Nested calls++A hook that calls `predict` again starts a new call with a new `run_id`. The parent and child are+independent unless you link them yourself. Capture the parent id and pass it along:++```python+def enrich(ctx):+ child = enricher.predict(ctx.states[0], EXTRA_QUESTIONS)+ record_child_span(parent_run_id=ctx.run_id, child_run_id=child.get("run_id"))+```++Guard against recursion (see [anti-patterns](patterns.md#recursive-predict)); the easiest guard+is a separate `enricher` agent with no hooks.++## See also++- [API reference](api.md): the full `PredictContext`.+- [Patterns and anti-patterns](patterns.md): non-blocking tracing.diff --git a/examples/hooks/README.md b/examples/hooks/README.mdnew file mode 100644index 00000000..d454bae7--- /dev/null+++ b/examples/hooks/README.md@@ -0,0 +1,10 @@+# Prediction hook examples++Small, runnable hooks. See [`docs/hooks/index.md`](../../docs/hooks/index.md) for the full reference.++- [`audit.py`](audit.py): log every decision and optionally ship it to an external service.+- [`redact.py`](redact.py): strip emails/phones from the state before inference.+- [`cache.py`](cache.py): cache decisions and skip the forward pass on a hit.+- [`otel.py`](otel.py): counters and a latency histogram per decision.++Each example loads a real checkpoint, so the first run downloads it.diff --git a/examples/hooks/audit.py b/examples/hooks/audit.pynew file mode 100644index 00000000..6b7b1e8b--- /dev/null+++ b/examples/hooks/audit.py@@ -0,0 +1,43 @@+"""Audit every decision: log it, and optionally ship it to an external service.++Run from the repository root:++ python examples/hooks/audit.py+"""+import json++import laya+++def on_predict_end(ctx):+ record = {+ "model": ctx.model,+ "answers": ctx.results[0]["answers"] if ctx.results else None,+ "routing": ctx.results[0].get("routing") if ctx.results else None,+ "usage": ctx.usage,+ "elapsed_ms": round(ctx.elapsed_ms or 0.0, 2),+ }+ print(json.dumps(record, indent=2))+ # Ship it to an external service if you want:+ # import requests+ # requests.post("https://example.invalid/decisions", json=record, timeout=2)+++QUESTIONS = {+ "dept": {+ "type": "choice",+ "instructions": "Which team should handle this?",+ "criteria": {"billing": "invoices and payments", "support": "product help"},+ }+}+++# Direct Agent use.+agent = laya.load("convaiinnovations/laya", on_predict_end=on_predict_end)+agent.system_one("I was charged twice for the same invoice.", QUESTIONS)++# Router use: the hook also sees ctx.decision (which checkpoint was chosen).+from laya import Router # noqa: E402++router = Router(on_predict_end=on_predict_end)+router.predict("I was charged twice for the same invoice.", QUESTIONS)diff --git a/examples/hooks/cache.py b/examples/hooks/cache.pynew file mode 100644index 00000000..3c3a0190--- /dev/null+++ b/examples/hooks/cache.py@@ -0,0 +1,41 @@+"""Cache decisions by state + questions, and skip inference on a hit.++A start hook calls `ctx.skip(results)`; the engine skips the forward pass and still runs+the end hooks.++ python examples/hooks/cache.py+"""+import hashlib+import json++import laya++CACHE = {}+++def cache_key(state, questions):+ payload = json.dumps({"state": state, "questions": questions}, sort_keys=True, default=str)+ return hashlib.sha256(payload.encode()).hexdigest()+++def cache_read(ctx):+ hit = CACHE.get(cache_key(ctx.states[0], ctx.questions))+ if hit is not None:+ ctx.skip([hit])+++def cache_write(ctx):+ if ctx.results:+ CACHE[cache_key(ctx.states[0], ctx.questions)] = ctx.results[0]+++agent = laya.load("convaiinnovations/laya",+ on_predict_start=cache_read, on_predict_end=cache_write)++STATE = "I was charged twice for the same invoice."+QUESTIONS = {"urgent": {"type": "noul", "instructions": "Is this urgent?"}}++first = agent.system_one(STATE, QUESTIONS) # runs the model, fills the cache+second = agent.system_one(STATE, QUESTIONS) # served from CACHE, no forward pass+print("cache entries:", len(CACHE))+print("same answer:", first["answers"] == second["answers"])diff --git a/examples/hooks/otel.py b/examples/hooks/otel.pynew file mode 100644index 00000000..cd383e56--- /dev/null+++ b/examples/hooks/otel.py@@ -0,0 +1,28 @@+"""Emit a counter and a latency histogram for every decision.++Works without any dependency; uncomment the OpenTelemetry lines if it is installed.++ python examples/hooks/otel.py+"""+import laya++COUNTS = {}+LATENCIES = []+++def on_predict_end(ctx):+ model = ctx.model or "auto"+ COUNTS[model] = COUNTS.get(model, 0) + 1+ if ctx.elapsed_ms is not None:+ LATENCIES.append(ctx.elapsed_ms)+ # With OpenTelemetry installed (`pip install opentelemetry-api`):+ # from opentelemetry import metrics+ # meter = metrics.get_meter("laya")+ # meter.create_counter("laya.decisions").add(1, {"model": model})+ # meter.create_histogram("laya.decision.latency_ms").record(ctx.elapsed_ms or 0.0)+++agent = laya.load("convaiinnovations/laya", on_predict_end=on_predict_end)+agent.system_one("I was charged twice.", {"urgent": {"type": "noul", "instructions": "Urgent?"}})+print("counts:", COUNTS)+print("latency samples:", len(LATENCIES))diff --git a/examples/hooks/redact.py b/examples/hooks/redact.pynew file mode 100644index 00000000..423dc1ee--- /dev/null+++ b/examples/hooks/redact.py@@ -0,0 +1,34 @@+"""Redact PII from the state before the model ever sees it.++A start hook can rewrite `ctx.states`; the rewrite is what gets tokenized.++ python examples/hooks/redact.py+"""+import re++import laya++EMAIL = re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b")+PHONE = re.compile(r"\+?\d[\d ()-]{7,}\d")+++def scrub(value):+ if isinstance(value, str):+ return PHONE.sub("[phone]", EMAIL.sub("[email]", value))+ if isinstance(value, dict):+ return {k: scrub(v) for k, v in value.items()}+ if isinstance(value, list):+ return [scrub(v) for v in value]+ return value+++def redact(ctx):+ ctx.states = [scrub(state) for state in ctx.states]+++agent = laya.load("convaiinnovations/laya", on_predict_start=redact)+result = agent.system_one(+ "Email [email protected] or call +1 555 010 9999 about invoice 42.",+ {"urgent": {"type": "noul", "instructions": "Is this urgent?"}},+)+print(result["answers"])diff --git a/laya/__init__.py b/laya/__init__.pyindex 3f99b89a..330016e8 100644--- a/laya/__init__.py+++ b/laya/__init__.py@@ -1,6 +1,7 @@"""Laya: Fast, non-autoregressive System 1 decision engine with calibrated probabilities."""from .email import clean_email_body, email_state+from .hooks import Hook, PredictContext, PredictHookfrom .lang import analyse as detect_languagefrom .lang import detect_script, is_englishfrom .presets import (@@ -87,5 +88,8 @@ __all__ = ["LayaGuardrailError","LayaTriage","LayaEvaluator",+ "PredictContext",+ "PredictHook",+ "Hook","__version__",]diff --git a/laya/agent.py b/laya/agent.pyindex 99df622f..f5a21227 100644--- a/laya/agent.py+++ b/laya/agent.py@@ -3,6 +3,7 @@ import jsonimport osimport tempfileimport threading+import timeimport warningsfrom contextlib import nullcontextfrom typing import Any, Dict, List, Optional, Union@@ -24,6 +25,7 @@ from .common import (render_options,temp_bucket,)+from .hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooksdef _fix_tokenizer_config(path: str):@@ -177,9 +179,16 @@ def _amp_context(device, dtype):return nullcontext()-class Agent:+class Agent(HookRegistry):"""System 1 decision model runtime: fast, non-autoregressive, calibrated decisions."""+ # Hooks are opt-in. `hooks`/`_hooks_mutex` defaults come from HookRegistry; the rest keep a+ # hand-built instance (`Agent.__new__` in tests) working and make an unset hook a no-op.+ hooks_raise = True+ hooks_concurrent = True+ _hooks_lock = None+ model_id = None+def __init__(self,model_id_or_path: str = "convaiinnovations/laya",@@ -188,6 +197,11 @@ class Agent:subfolder: Optional[str] = None,fast: bool = False,compile: bool = False,+ hooks=None,+ on_predict_start=None,+ on_predict_end=None,+ hooks_raise: bool = True,+ hooks_concurrent: bool = True,):"""Load a Laya checkpoint.@@ -197,7 +211,18 @@ class Agent:`subfolder` selects one checkpoint from a repo that bundles several, e.g.`Agent("convaiinnovations/laya", subfolder="multilingual")`. Only that subfolder isdownloaded, so bundling does not cost every user the whole family.++ `hooks` / `on_predict_start` / `on_predict_end` observe or shape every prediction; see+ `laya.hooks`. `hooks_raise=False` warns and continues when a hook fails, and+ `hooks_concurrent=False` serialises hooks that are not safe to run in parallel."""+ self.hooks = normalise_hooks(hooks, on_predict_start, on_predict_end)+ self.hooks_raise = bool(hooks_raise)+ self.hooks_concurrent = bool(hooks_concurrent)+ self._hooks_lock = threading.RLock() if not hooks_concurrent else None+ self._hooks_mutex = threading.Lock()+ self.model_id = model_id_or_path+from safetensors.torch import load_filetry:from transformers.initialization import no_init_weights@@ -478,10 +503,15 @@ class Agent:q["labels"] = qdef["labels"]return q- def _encode_state(self, state: Union[str, dict, list], ids: List[str], internal: Dict[str, Dict]) -> List[Dict]:- """Tokenize one state against every (already validated + normalized) question."""- max_len = self.cfg.get("max_len", 512)- head_max_len = self.cfg.get("head_max_len", 192)+ def _encode_state(self, state: Union[str, dict, list], ids: List[str], internal: Dict[str, Dict],+ max_len: Optional[int] = None, head_max_len: Optional[int] = None) -> List[Dict]:+ """Tokenize one state against every (already validated + normalized) question.++ `max_len` / `head_max_len` override the agent config for this call (a start hook may set+ `ctx.max_len` / `ctx.head_max_len`).+ """+ max_len = self.cfg.get("max_len", 512) if max_len is None else max_len+ head_max_len = self.cfg.get("head_max_len", 192) if head_max_len is None else head_max_len# A chronological conversation list is serialized newest-last, so the default# right-truncation (st[:room]) would silently drop the newest turn. Truncate# from the left for lists so the most recent intent is preserved.@@ -569,7 +599,11 @@ class Agent:@torch.no_grad()def predict_batch(self, states: List[Union[str, dict, list]], questions: Dict[str, Dict[str, Any]],- batch_size: Optional[int] = None) -> List[Dict[str, Any]]:+ batch_size: Optional[int] = None, hooks=None,+ on_predict_start=None, on_predict_end=None,+ hooks_raise: Optional[bool] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None) -> List[Dict[str, Any]]:"""Evaluate the same questions over many states, packing them into shared forward passes.This is the throughput path. `system_one`/`predict` handle one state per forward pass; on a@@ -583,56 +617,106 @@ class Agent:questions: Question definitions, exactly as accepted by `system_one`.batch_size: Optional cap on states per forward pass. `None` sends them all in one pass;set it to bound peak memory when batching many or long states.+ hooks, on_predict_start, on_predict_end: Per-call hooks, appended after any installed on+ the Agent. `on_predict_start` may rewrite the state/questions or call+ `ctx.skip(...)` to short-circuit inference; `on_predict_end` may rewrite the+ results. See `laya.hooks`.+ hooks_raise: Override the Agent's `hooks_raise` for this call.+ max_len, head_max_len: Override the agent config for this call. A start hook may also+ set `ctx.max_len` / `ctx.head_max_len` to shape the token budget.Returns:A list of per-state result dicts, each identical in shape to `system_one`'s output andaligned with `states` by index."""- if isinstance(states, (str, bytes, dict)):- raise TypeError(- "predict_batch expects a list of states; pass a single state to predict()/system_one()."- )- states = list(states)- if not states:- return []-- ids = list(questions.keys())- # Empty questions: empty answers, zero usage, no tokenization or forward (matches system_one).- if not ids:- return [{"model": "laya-rl-agent", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}- for _ in states]-- # Validate + normalize each question once (state-independent), preserving system_one's checks.- for qid in ids:- self._check_question(qid, questions[qid])- internal = {qid: self._to_internal(questions[qid]) for qid in ids}- chunk = batch_size if (batch_size and batch_size > 0) else len(states)-- results: List[Dict[str, Any]] = []- for start in range(0, len(states), chunk):- part = states[start:start + chunk]- per_state_items = [self._encode_state(st, ids, internal) for st in part]-- b = collate_items(per_state_items, self.tok.pad_token_id)- logits, act = self._forward(b)- att = b["attention_mask"]-- row = 0- for items in per_state_items:- nrows = len(items)- n_tokens = int(att[row:row + nrows].sum())- answers = self._decode_answers(logits, act, items, ids, internal, row)- results.append({- "model": "laya-rl-agent",- "answers": answers,- "usage": {"input_tokens": n_tokens, "output_tokens": 0},- })- row += nrows-- return results+ active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)+ raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)+ ctx = PredictContext(states=states, questions=questions, model=self.model_id, agent=self,+ max_len=max_len, head_max_len=head_max_len)+ try:+ dispatch(active, "on_predict_start", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ states, questions = ctx.states, ctx.questions+ if ctx.results is None:+ # A start hook may have normalised a bare string/dict into a list; only the value+ # that survives the hook is validated.+ if isinstance(states, (str, bytes, dict)):+ raise TypeError(+ "predict_batch expects a list of states; pass a single state to predict()/system_one()."+ )+ states = list(states)+ if not states:+ ctx.results = []+ else:+ ids = list(questions.keys())+ # Empty questions: empty answers, zero usage, no tokenization or forward.+ if not ids:+ ctx.results = [+ {"model": "laya-rl-agent", "answers": {},+ "usage": {"input_tokens": 0, "output_tokens": 0}} for _ in states+ ]+ else:+ # Validate + normalize each question once (state-independent).+ for qid in ids:+ self._check_question(qid, questions[qid])+ internal = {qid: self._to_internal(questions[qid]) for qid in ids}+ chunk = batch_size if (batch_size and batch_size > 0) else len(states)++ # Per-call token-budget overrides (a start hook may have set them).+ overrides: Dict[str, int] = {}+ if ctx.max_len is not None:+ overrides["max_len"] = ctx.max_len+ if ctx.head_max_len is not None:+ overrides["head_max_len"] = ctx.head_max_len++ results: List[Dict[str, Any]] = []+ for start in range(0, len(states), chunk):+ part = states[start:start + chunk]+ per_state_items = [self._encode_state(st, ids, internal, **overrides) for st in part]++ b = collate_items(per_state_items, self.tok.pad_token_id)+ logits, act = self._forward(b)+ att = b["attention_mask"]++ row = 0+ for items in per_state_items:+ nrows = len(items)+ n_tokens = int(att[row:row + nrows].sum())+ answers = self._decode_answers(logits, act, items, ids, internal, row)+ results.append({+ "model": "laya-rl-agent",+ "answers": answers,+ "usage": {"input_tokens": n_tokens, "output_tokens": 0},+ })+ row += nrows+ ctx.results = results+ except BaseException as exc:+ ctx.error = exc+ try:+ dispatch(active, "on_error", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ except BaseException as hook_exc:+ # A failing on_error hook must not hide the failure that triggered it.+ exc.__context__ = hook_exc+ raise+ finally:+ ctx.elapsed_ms = (time.perf_counter() - ctx.started_at) * 1000.0+ if ctx.results is not None:+ ctx.usage = aggregate_usage(ctx.results)+ try:+ dispatch(active, "on_predict_end", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ except BaseException as hook_exc:+ # End hooks run on the failure path too; do not let one mask the real error.+ if ctx.error is not None:+ ctx.error.__context__ = hook_exc+ else:+ raise+ return ctx.results@torch.no_grad()- def system_one(self, state: Union[str, dict, list], questions: Dict[str, Dict[str, Any]]) -> Dict[str, Any]:+ def system_one(self, state: Union[str, dict, list], questions: Dict[str, Dict[str, Any]],+ hooks=None, on_predict_start=None, on_predict_end=None,+ hooks_raise: Optional[bool] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None) -> Dict[str, Any]:"""Evaluate typed questions across state in a single, parallel forward pass.Args:@@ -655,7 +739,10 @@ class Agent:To score many states at once, see `predict_batch`, which shares forward passes across them."""- return self.predict_batch([state], questions)[0]+ return self.predict_batch([state], questions, hooks=hooks,+ on_predict_start=on_predict_start,+ on_predict_end=on_predict_end, hooks_raise=hooks_raise,+ max_len=max_len, head_max_len=head_max_len)[0]def __enter__(self):return self@@ -681,7 +768,9 @@ RLAgent = Agentdef load(model_id_or_path: str = "convaiinnovations/laya", device: Optional[str] = None,- token: Optional[str] = None, subfolder: Optional[str] = None, fast: bool = False) -> Agent:+ token: Optional[str] = None, subfolder: Optional[str] = None, fast: bool = False,+ hooks=None, on_predict_start=None, on_predict_end=None,+ hooks_raise: bool = True, hooks_concurrent: bool = True) -> Agent:"""Load a Laya agent.`subfolder` picks one checkpoint out of a repo that bundles several:@@ -689,5 +778,10 @@ def load(model_id_or_path: str = "convaiinnovations/laya", device: Optional[str]laya.load("convaiinnovations/laya") # English (repo root)laya.load("convaiinnovations/laya", subfolder="multilingual")laya.load("convaiinnovations/laya", fast=True) # TileLang GPU fast path++ `hooks` / `on_predict_start` / `on_predict_end` observe or shape every prediction; see+ `laya.hooks`."""- return Agent(model_id_or_path, device=device, token=token, subfolder=subfolder, fast=fast)+ return Agent(model_id_or_path, device=device, token=token, subfolder=subfolder, fast=fast,+ hooks=hooks, on_predict_start=on_predict_start, on_predict_end=on_predict_end,+ hooks_raise=hooks_raise, hooks_concurrent=hooks_concurrent)diff --git a/laya/hooks.py b/laya/hooks.pynew file mode 100644index 00000000..96e3e3f0--- /dev/null+++ b/laya/hooks.py@@ -0,0 +1,242 @@+"""Opt-in prediction hooks: observe or shape every decision without forking.++A hook is either a plain callable or an object implementing any subset of the lifecycle+methods on `Hook`. Hooks are configured on `Agent` / `Router` and can be overridden per call.+Everything here is pure Python: importing `laya` must not start pulling torch.+"""+from __future__ import annotations++import threading+import time+import uuid+import warnings+from contextlib import contextmanager+from dataclasses import dataclass, field+from typing import Any, Callable, Dict, List, Optional, Protocol, Sequence, Union+++@dataclass(eq=False)+class PredictContext:+ """Mutable state passed to every hook for one call.++ `states` / `questions` may be rewritten by `on_predict_start`; `results` may be rewritten+ by `on_predict_end`. A start hook can call `skip()` to short-circuit inference with a+ cached result.++ Identity semantics (`eq=False`): two contexts are never equal, and a context is hashable+ by identity, so a hook can keep one in a set without comparing agent/router internals.+ """++ states: List[Any]+ questions: Dict[str, Any]+ run_id: str = field(default_factory=lambda: uuid.uuid4().hex) # shared by every hook of one call+ results: Optional[List[Dict[str, Any]]] = None+ decision: Optional[Dict[str, Any]] = None # Router: the RouteDecision dict+ model: Optional[str] = None # resolved checkpoint name+ agent: Any = None+ router: Any = None+ max_len: Optional[int] = None # per-call overrides; None = agent config+ head_max_len: Optional[int] = None+ usage: Optional[Dict[str, int]] = None # aggregated input/output tokens+ started_at: float = field(default_factory=time.perf_counter)+ elapsed_ms: Optional[float] = None+ error: Optional[BaseException] = None++ def skip(self, results: List[Dict[str, Any]]) -> None:+ """Set cached results from a start hook; inference is skipped, end hooks still run."""+ self.results = results+++class Hook(Protocol):+ """Optional lifecycle methods. Implement any subset; missing methods are skipped."""++ def on_predict_start(self, ctx: PredictContext) -> None: ...+ def on_predict_end(self, ctx: PredictContext) -> None: ...+ def on_route(self, ctx: PredictContext) -> None: ...+ def on_load(self, ctx: PredictContext) -> None: ...+ def on_evict(self, ctx: PredictContext) -> None: ...+ def on_error(self, ctx: PredictContext) -> None: ...+++PredictHook = Callable[[PredictContext], None]+HookArg = Union[Hook, Sequence[Hook], None]+PredictHookArg = Union[PredictHook, Sequence[PredictHook], None]++HOOK_EVENTS = (+ "on_predict_start",+ "on_predict_end",+ "on_route",+ "on_load",+ "on_evict",+ "on_error",+)+++class _StartAdapter:+ """Wrap a plain `on_predict_start` callable as a `Hook`."""++ __slots__ = ("fn",)++ def __init__(self, fn: PredictHook):+ if not callable(fn):+ raise TypeError("on_predict_start must be callable, got %s" % type(fn).__name__)+ self.fn = fn++ def on_predict_start(self, ctx: PredictContext) -> None:+ self.fn(ctx)+++class _EndAdapter:+ """Wrap a plain `on_predict_end` callable as a `Hook`."""++ __slots__ = ("fn",)++ def __init__(self, fn: PredictHook):+ if not callable(fn):+ raise TypeError("on_predict_end must be callable, got %s" % type(fn).__name__)+ self.fn = fn++ def on_predict_end(self, ctx: PredictContext) -> None:+ self.fn(ctx)+++def _as_sequence(value: Any) -> tuple:+ if value is None:+ return ()+ if isinstance(value, (list, tuple)):+ return tuple(value)+ return (value,)+++def normalise_hooks(+ hooks: HookArg = None,+ on_predict_start: PredictHookArg = None,+ on_predict_end: PredictHookArg = None,+) -> List[Any]:+ """Flatten `hooks` and the two convenience callables into one ordered hook list.++ Installed hooks are normalised once at construction; per-call arguments are appended+ after them, so per-call hooks always run last.+ """+ result: List[Any] = []+ for hook in _as_sequence(hooks):+ if isinstance(hook, type):+ raise TypeError(+ "hooks entries must be instances, not classes; got %s. Instantiate it first."+ % hook.__name__+ )+ if not any(hasattr(hook, event) for event in HOOK_EVENTS):+ raise TypeError(+ "hooks entries must implement at least one of %s; got %s. Pass a plain "+ "callable as on_predict_start=/on_predict_end= instead."+ % (", ".join(HOOK_EVENTS), type(hook).__name__)+ )+ for event in HOOK_EVENTS:+ method = getattr(hook, event, None)+ if method is not None and not callable(method):+ raise TypeError(+ "hooks entry %s.%s must be callable, got %s"+ % (type(hook).__name__, event, type(method).__name__)+ )+ result.append(hook)+ for fn in _as_sequence(on_predict_start):+ result.append(_StartAdapter(fn))+ for fn in _as_sequence(on_predict_end):+ result.append(_EndAdapter(fn))+ return result+++class HookRegistry:+ """Mixin giving a runtime-mutable hook list.++ `Agent`, `Router` and `ONNXAgent` use it so hooks can be added, removed or scoped after+ construction. Mutation is guarded by `_hooks_mutex`; a call reads a snapshot of the list,+ so adding or removing a hook never disturbs a call in flight.+ """++ hooks = ()+ _hooks_mutex = None++ def add_hook(self, hook: HookArg) -> "HookRegistry":+ """Install one hook or a sequence of them. Returns self for chaining."""+ self._extend_hooks(normalise_hooks(hook))+ return self++ def remove_hook(self, hook: Any) -> bool:+ """Remove a hook by identity. Returns True if it was installed."""+ return self._remove_hooks(lambda installed: installed is hook) > 0++ @contextmanager+ def hooks_installed(self, *hooks: HookArg):+ """Install hooks for the duration of the `with` block, then remove them.++ with agent.hooks_installed(tracer):+ agent.system_one(state, questions)+ """+ added = normalise_hooks(list(hooks))+ self._extend_hooks(added)+ try:+ yield self+ finally:+ self._remove_hooks(lambda installed: any(installed is one for one in added))++ def _extend_hooks(self, hooks: Sequence[Any]) -> None:+ if not hooks:+ return+ with self._hooks_mutex_for_registry():+ self.hooks = list(self.hooks) + list(hooks)++ def _remove_hooks(self, predicate: Callable[[Any], bool]) -> int:+ with self._hooks_mutex_for_registry():+ current = list(self.hooks)+ remaining = [hook for hook in current if not predicate(hook)]+ self.hooks = remaining+ return len(current) - len(remaining)++ def _hooks_mutex_for_registry(self) -> threading.Lock:+ lock = self._hooks_mutex+ if lock is None:+ # Only reachable for an instance built without __init__ (tests); real instances+ # get their mutex in the constructor.+ lock = self._hooks_mutex = threading.Lock()+ return lock+++def aggregate_usage(results: Sequence[Dict[str, Any]]) -> Dict[str, int]:+ """Sum the per-state usage blocks so a hook sees one total for the call."""+ def total(key: str) -> int:+ return sum(int((r.get("usage") or {}).get(key, 0) or 0) for r in results)+ return {"input_tokens": total("input_tokens"), "output_tokens": total("output_tokens")}+++def dispatch(+ hooks: Sequence[Any],+ event: str,+ ctx: PredictContext,+ *,+ raise_errors: bool = True,+ lock: Any = None,+) -> None:+ """Call `event` on every hook that implements it.++ `raise_errors=False` warns and continues, for hooks (telemetry) that must not fail a+ request. `lock` serialises dispatch for hooks that are not safe to run concurrently.+ """+ for hook in hooks:+ method = getattr(hook, event, None)+ if method is None:+ continue+ try:+ if lock is not None:+ with lock:+ method(ctx)+ else:+ method(ctx)+ except Exception as exc: # noqa: BLE001 -- policy depends on raise_errors+ if raise_errors:+ raise+ warnings.warn(+ "laya: hook %s.%s failed: %s" % (type(hook).__name__, event, exc),+ RuntimeWarning,+ stacklevel=2,+ )diff --git a/laya/onnx_agent.py b/laya/onnx_agent.pyindex 273454f1..8664bd25 100644--- a/laya/onnx_agent.py+++ b/laya/onnx_agent.py@@ -1,10 +1,13 @@import jsonimport os+import threading+import timeimport warningsfrom typing import Any, Dict, Optional, Unionimport numpy as np+from laya.hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooksfrom laya.common import (QTYPES,build_sequence,@@ -18,14 +21,26 @@ from laya.common import ()-class ONNXAgent:+class ONNXAgent(HookRegistry):"""System 1 decision model runtime via ONNX: fast CPU-optimized decisions."""+ # Opt-in hooks; defaults keep a hand-built instance working and make an unset hook a no-op.+ # `hooks`/`_hooks_mutex` come from HookRegistry.+ hooks_raise = True+ hooks_concurrent = True+ _hooks_lock = None+ model_id = None+def __init__(self,model_id_or_path: str,onnx_path: str = "laya.onnx",subfolder: Optional[str] = None,+ hooks=None,+ on_predict_start=None,+ on_predict_end=None,+ hooks_raise: bool = True,+ hooks_concurrent: bool = True,):"""Load a Laya agent backed by ONNX Runtime.@@ -34,7 +49,17 @@ class ONNXAgent:(used to load the tokenizer and config).onnx_path: Path to the exported .onnx file.subfolder: Optional subfolder if downloading from a repo bundle.+ hooks, on_predict_start, on_predict_end: Opt-in prediction hooks; see `laya.hooks`.+ hooks_raise: When False, a failing hook warns and inference continues.+ hooks_concurrent: When False, hooks are serialised with a lock."""+ self.hooks = normalise_hooks(hooks, on_predict_start, on_predict_end)+ self.hooks_raise = bool(hooks_raise)+ self.hooks_concurrent = bool(hooks_concurrent)+ self._hooks_lock = threading.RLock() if not hooks_concurrent else None+ self._hooks_mutex = threading.Lock()+ self.model_id = model_id_or_path+import onnxruntime as ortfrom transformers import AutoTokenizer@@ -129,15 +154,55 @@ class ONNXAgent:q["labels"] = qdef["labels"]return q- def system_one(self, state: Union[str, dict, list], questions: Dict[str, Dict[str, Any]]) -> Dict[str, Any]:+ def system_one(self, state: Union[str, dict, list], questions: Dict[str, Dict[str, Any]],+ hooks=None, on_predict_start=None, on_predict_end=None,+ hooks_raise: Optional[bool] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None) -> Dict[str, Any]:+ """Evaluate typed questions, running any opt-in hooks around the inference."""+ active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)+ raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)+ ctx = PredictContext(states=[state], questions=questions, model=self.model_id, agent=self,+ max_len=max_len, head_max_len=head_max_len)+ try:+ dispatch(active, "on_predict_start", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ if ctx.results is None:+ overrides = {}+ if ctx.max_len is not None:+ overrides["max_len"] = ctx.max_len+ if ctx.head_max_len is not None:+ overrides["head_max_len"] = ctx.head_max_len+ ctx.results = [self._infer(ctx.states[0], ctx.questions, **overrides)]+ except BaseException as exc:+ ctx.error = exc+ try:+ dispatch(active, "on_error", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ except BaseException as hook_exc:+ exc.__context__ = hook_exc+ raise+ finally:+ ctx.elapsed_ms = (time.perf_counter() - ctx.started_at) * 1000.0+ if ctx.results is not None:+ ctx.usage = aggregate_usage(ctx.results)+ try:+ dispatch(active, "on_predict_end", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ except BaseException as hook_exc:+ if ctx.error is not None:+ ctx.error.__context__ = hook_exc+ else:+ raise+ return ctx.results[0]++ def _infer(self, state: Union[str, dict, list], questions: Dict[str, Dict[str, Any]],+ max_len: Optional[int] = None, head_max_len: Optional[int] = None) -> Dict[str, Any]:from .agent import Agent as _Agentids = list(questions.keys())for qid in ids:_Agent._check_question(qid, questions[qid])items = []- max_len = self.cfg.get("max_len", 512)- head_max_len = self.cfg.get("head_max_len", 192)+ max_len = self.cfg.get("max_len", 512) if max_len is None else max_len+ head_max_len = self.cfg.get("head_max_len", 192) if head_max_len is None else head_max_lenfor qid in ids:q = self._to_internal(questions[qid])diff --git a/laya/router.py b/laya/router.pyindex d756faf2..23cfc3a1 100644--- a/laya/router.py+++ b/laya/router.py@@ -30,8 +30,10 @@ synthetic workflows and should not be a silent default.import gcimport osimport threading+import timefrom typing import Any, Dict, List, Optional, Union+from .hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooksfrom .lang import analyse# The hub repo bundles all three checkpoints; only the requested subfolder is downloaded.@@ -146,7 +148,7 @@ def _english_from_code(value: Any) -> Optional[bool]:return primary in _ENGLISH_SUBTAGS-class Router:+class Router(HookRegistry):"""Lazily loads Laya checkpoints and sends each request to the right one.from laya import Router@@ -173,8 +175,18 @@ class Router:r = Router(preload=True) # all three resident, routing is freer = Router(preload=True, device="cuda")r.preload(["english", "multilingual"]) # or just the two you serve++ Hooks are opt-in and run at the Router level: `on_route` sees the routing decision,+ `on_load` / `on_evict` see model lifecycle, and `on_predict_start` / `on_predict_end`+ wrap the whole route+infer call. See `laya.hooks`."""+ # Opt-in defaults so a hand-built instance (`Router.__new__` in tests) works unset.+ # `hooks`/`_hooks_mutex` come from HookRegistry.+ hooks_raise = True+ hooks_concurrent = True+ _hooks_lock = None+def __init__(self,models: Optional[Dict[str, str]] = None,@@ -186,7 +198,17 @@ class Router:standalone_repos: bool = False,preload: bool = False,lang_guess: Optional[Any] = None,+ hooks=None,+ on_predict_start=None,+ on_predict_end=None,+ hooks_raise: bool = True,+ hooks_concurrent: bool = True,):+ self.hooks = normalise_hooks(hooks, on_predict_start, on_predict_end)+ self.hooks_raise = bool(hooks_raise)+ self.hooks_concurrent = bool(hooks_concurrent)+ self._hooks_lock = threading.RLock() if not hooks_concurrent else None+ self._hooks_mutex = threading.Lock()self.models = dict(STANDALONE_MODELS if standalone_repos else DEFAULT_MODELS)if models:self.models.update({normalise_name(k): v for k, v in models.items()})@@ -226,8 +248,13 @@ class Router:agent = Agent(repo, device=self.device, token=self.token, subfolder=sub)self._agents[key] = agentself._order.append(key)- self._evict()- return agent+ evicted = self._evict_locked()+ # Lifecycle hooks fire after the lock is released, so a hook can safely call the Router.+ self._dispatch_lifecycle("on_evict", evicted)+ dispatch(self.hooks, "on_load",+ PredictContext(states=[], questions={}, model=key, agent=agent, router=self),+ raise_errors=self.hooks_raise, lock=self._hooks_lock)+ return agentdef _touch(self, key: str):with self._lock:@@ -235,30 +262,41 @@ class Router:self._order.remove(key)self._order.append(key)- def _evict(self):+ def _evict_locked(self) -> List[str]:+ """Drop least-recently-used agents until `max_loaded` holds. Returns evicted names."""+ evicted: List[str] = []+ while len(self._order) > self.max_loaded:+ victim = self._order.pop(0)+ agent = self._agents.pop(victim, None)+ if agent is not None:+ evicted.append(victim)+ del agent+ if len(self._order) < len(self._agents): # keep the two views consistent+ for k in list(self._agents):+ if k not in self._order:+ agent = self._agents.pop(k, None)+ if agent is not None:+ evicted.append(k)+ del agent+ if evicted:+ gc.collect()+ try:+ import torch+ if torch.cuda.is_available():+ torch.cuda.empty_cache()+ except Exception:+ pass+ return evicted++ def _evict(self) -> List[str]:with self._lock:- evicted = False- while len(self._order) > self.max_loaded:- victim = self._order.pop(0)- agent = self._agents.pop(victim, None)- if agent is not None:- evicted = True- del agent- if len(self._order) < len(self._agents): # keep the two views consistent- for k in list(self._agents):- if k not in self._order:- agent = self._agents.pop(k, None)- if agent is not None:- evicted = True- del agent- if evicted:- gc.collect()- try:- import torch- if torch.cuda.is_available():- torch.cuda.empty_cache()- except Exception:- pass+ return self._evict_locked()++ def _dispatch_lifecycle(self, event: str, names: List[str]) -> None:+ for name in names:+ dispatch(self.hooks, event,+ PredictContext(states=[], questions={}, model=name, router=self),+ raise_errors=self.hooks_raise, lock=self._hooks_lock)def attach(self, name: str, agent: Any):"""Register an already-built Agent under `name` instead of loading a second copy.@@ -285,15 +323,19 @@ class Router:names = [normalise_name(n) for n in (list(self.models) if names is None else names)]with self._lock:self.max_loaded = max(self.max_loaded, len(set(names) | set(self._agents)))- for n in names:- if n not in self._agents: # an attached agent is already built- self.load(n)+ for n in names:+ with self._lock:+ already = n in self._agents # an attached agent is already built+ if not already:+ # load() dispatches on_load outside the lock; do not hold it across the call.+ self.load(n)return selfdef unload(self, name: Optional[str] = None):"""Free one model, or all of them."""with self._lock:if name is None:+ freed = list(self._order)self._agents.clear()self._order.clear()else:@@ -301,6 +343,7 @@ class Router:agent = self._agents.pop(key, None)if key in self._order:self._order.remove(key)+ freed = [key] if agent is not None else []del agentgc.collect()try:@@ -309,6 +352,7 @@ class Router:torch.cuda.empty_cache()except Exception:pass+ self._dispatch_lifecycle("on_evict", freed)@propertydef loaded(self) -> List[str]:@@ -337,6 +381,30 @@ class Router:task: Optional[str] = None,lang: Optional[str] = None,lang_guess: Optional[Any] = None,+ hooks=None,+ hooks_raise: Optional[bool] = None,+ ) -> RouteDecision:+ """Decide which checkpoint to use, then let `on_route` hooks observe or replace it.++ `ctx.decision` is the `RouteDecision`; a hook may replace it (for example to pin a+ checkpoint) and the replacement is what gets returned and used. `hooks` are per-call+ hooks, appended after any installed on the Router.+ """+ decision = self._route(state, questions, model=model, task=task, lang=lang, lang_guess=lang_guess)+ raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)+ active = list(self.hooks) + normalise_hooks(hooks)+ ctx = PredictContext(states=[state], questions=questions or {}, decision=decision, router=self)+ dispatch(active, "on_route", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ return ctx.decision++ def _route(+ self,+ state: Union[str, dict, list, None],+ questions: Optional[Dict[str, Any]] = None,+ model: Optional[str] = None,+ task: Optional[str] = None,+ lang: Optional[str] = None,+ lang_guess: Optional[Any] = None,) -> RouteDecision:"""Decide which checkpoint to use, without loading or running anything.@@ -423,16 +491,68 @@ class Router:task: Optional[str] = None,lang: Optional[str] = None,lang_guess: Optional[Any] = None,+ hooks=None,+ on_predict_start=None,+ on_predict_end=None,+ hooks_raise: Optional[bool] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,) -> Dict[str, Any]:"""Route, then answer every question in one forward pass on the chosen checkpoint.The result is the usual `system_one` payload plus a `routing` key recording the decision.+ Router-level `on_predict_start` / `on_predict_end` hooks wrap the whole route+infer call+ and see `ctx.decision`; see `laya.hooks`. `max_len` / `head_max_len` override the agent+ token budget for this call (a start hook may set `ctx.max_len` / `ctx.head_max_len`)."""- decision = self.route(state, questions, model=model, task=task, lang=lang, lang_guess=lang_guess)+ active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)+ raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)++ # Per-call hooks apply to the whole call, including on_route inside route().+ decision = self.route(state, questions, model=model, task=task, lang=lang,+ lang_guess=lang_guess, hooks=hooks, hooks_raise=hooks_raise)agent = self.load(decision["model"])- result = agent.system_one(state, questions)- result["routing"] = dict(decision)- return result+ ctx = PredictContext(states=[state], questions=questions, decision=dict(decision),+ model=decision["model"], agent=agent, router=self,+ max_len=max_len, head_max_len=head_max_len)+ try:+ dispatch(active, "on_predict_start", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ if ctx.results is None:+ # Pass token-budget overrides only when set, so any Agent-like object that does+ # not accept them still works on the default path.+ overrides = {}+ if ctx.max_len is not None:+ overrides["max_len"] = ctx.max_len+ if ctx.head_max_len is not None:+ overrides["head_max_len"] = ctx.head_max_len+ result = agent.system_one(ctx.states[0], ctx.questions, **overrides)+ result["routing"] = dict(decision)+ ctx.results = [result]+ else:+ # A cache hit short-circuits inference, but Router.predict still promises a+ # `routing` key. Add it without overwriting a routing the cached payload has.+ for result in ctx.results:+ if isinstance(result, dict):+ result.setdefault("routing", dict(decision))+ except BaseException as exc:+ ctx.error = exc+ try:+ dispatch(active, "on_error", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ except BaseException as hook_exc:+ exc.__context__ = hook_exc+ raise+ finally:+ ctx.elapsed_ms = (time.perf_counter() - ctx.started_at) * 1000.0+ if ctx.results is not None:+ ctx.usage = aggregate_usage(ctx.results)+ try:+ dispatch(active, "on_predict_end", ctx, raise_errors=raise_errors, lock=self._hooks_lock)+ except BaseException as hook_exc:+ if ctx.error is not None:+ ctx.error.__context__ = hook_exc+ else:+ raise+ return ctx.results[0]def __enter__(self):return selfdiff --git a/tests/test_hooks.py b/tests/test_hooks.pynew file mode 100644index 00000000..c1f57fe0--- /dev/null+++ b/tests/test_hooks.py@@ -0,0 +1,649 @@+"""Prediction hooks: observe or shape a decision without a model or a download.++The Agent path uses the same weight-free fake as `test_batch.py` (the real `predict_batch`+with `_encode_state` / `_forward` / `_decode_answers` stubbed). The Router path patches+`laya.agent.Agent` so a model build never touches the Hub.++Run: python tests/test_hooks.py+"""+import os+import sys+import threading+import time+import warnings++import numpy as np++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))++import laya.agent as _agent_mod # noqa: E402+from laya.agent import Agent # noqa: E402+from laya.hooks import normalise_hooks # noqa: E402+from laya.router import RouteDecision, Router # noqa: E402++PASS, FAIL = [], []+++def check(name, got, want):+ if got == want:+ PASS.append(name)+ else:+ FAIL.append("%s: got %r, want %r" % (name, got, want))+++def check_true(name, cond, detail=""):+ if cond:+ PASS.append(name)+ else:+ FAIL.append("%s%s" % (name, ": " + detail if detail else ""))+++def check_raises(name, exc, fn):+ try:+ fn()+ except exc:+ PASS.append(name)+ return+ except BaseException as other: # noqa: BLE001+ FAIL.append("%s: raised %r, want %r" % (name, other, exc))+ return+ FAIL.append("%s: did not raise %r" % (name, exc))+++NQ = 2+QUESTIONS = {"a": {"type": "noul", "instructions": "?"}, "b": {"type": "noul", "instructions": "?"}}+++def make_fake():+ """A real `predict_batch` with the three composed helpers stubbed out."""+ fake = Agent.__new__(Agent)+ fake.tok = type("Tok", (), {"pad_token_id": 0})()+ fake._to_internal = staticmethod(Agent._to_internal).__func__+ fake._encode_states = []+ fake._forward_calls = []++ def _encode_state(state, ids, internal):+ fake._encode_states.append(state)+ return [{"ids": [1, 2, 3], "markers": [0, 1], "qtype": 2} for _ in ids]++ def _forward(b):+ n = b["input_ids"].shape[0]+ fake._forward_calls.append(n)+ return np.zeros((n, 2), dtype=np.float32), np.full((n, 2), 0.5, dtype=np.float32)++ def _decode_answers(logits, act, items, ids, internal, offset):+ return {"_offset": offset}++ fake._encode_state = _encode_state+ fake._forward = _forward+ fake._decode_answers = _decode_answers+ return fake+++class FakeAgent:+ def system_one(self, state, questions):+ return {"model": "fake", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}+++# --------------------------------------------------------------- order and context+events = []+results_at_start = []+results_at_end = []+run_id_start = []+run_id_end = []+++def start(ctx):+ events.append(("start", ctx))+ results_at_start.append(ctx.results)+ run_id_start.append(ctx.run_id)+++def end(ctx):+ events.append(("end", ctx))+ results_at_end.append(ctx.results)+ run_id_end.append(ctx.run_id)+++f = make_fake()+res = f.predict_batch(["s0", "s1"], QUESTIONS, on_predict_start=start, on_predict_end=end)+check("order/start then end", [e[0] for e in events], ["start", "end"])+check("order/once each", len(events), 2)+check("context/start sees states", events[0][1].states, ["s0", "s1"])+check("context/start sees questions", events[0][1].questions, QUESTIONS)+check("context/start has no results", results_at_start, [None])+check_true("context/end results is the returned object", results_at_end[0] is res)+check("context/end result count", len(results_at_end[0]), 2)+check_true("context/elapsed_ms set", events[1][1].elapsed_ms is not None)+check("context/usage aggregated", events[1][1].usage, {"input_tokens": 12, "output_tokens": 0})+check_true("context/agent set", events[1][1].agent is f)+check_true("context/run_id is a non-empty string",+ isinstance(run_id_start[0], str) and bool(run_id_start[0]))+check("context/run_id is shared by start and end", run_id_end[0], run_id_start[0])++_rids = []+_f2 = make_fake()+_f2.predict_batch(["s0"], QUESTIONS, on_predict_end=lambda c: _rids.append(c.run_id))+_f2.predict_batch(["s0"], QUESTIONS, on_predict_end=lambda c: _rids.append(c.run_id))+check("context/run_id differs per call", len(set(_rids)), 2)+++# --------------------------------------------------------------- mutation+def rewrite(ctx):+ ctx.states = ["rewritten"]+ ctx.questions = {"a": {"type": "noul", "instructions": "?"}}+++f = make_fake()+f.predict_batch(["orig"], QUESTIONS, on_predict_start=rewrite)+check("mutation/start rewrite is what gets encoded", f._encode_states, ["rewritten"])+++def replace_results(ctx):+ ctx.results = [{"model": "replaced"}]+++f = make_fake()+res = f.predict_batch(["s0"], QUESTIONS, on_predict_end=replace_results)+check("mutation/end rewrite is returned", res, [{"model": "replaced"}])+++# --------------------------------------------------------------- caching short-circuit+end_calls = {"n": 0}+++def cache_hit(ctx):+ ctx.skip([{"model": "cached"}])+++def count_end(ctx):+ end_calls["n"] += 1+++f = make_fake()+res = f.predict_batch(["s0"], QUESTIONS, on_predict_start=cache_hit, on_predict_end=count_end)+check("skip/returns cached results", res, [{"model": "cached"}])+check("skip/no forward pass", f._forward_calls, [])+check("skip/end still runs", end_calls["n"], 1)+++# --------------------------------------------------------------- installed vs per-call+order = []+f = make_fake()+f.hooks = normalise_hooks(on_predict_start=lambda ctx: order.append("installed"))+f.predict_batch(["s0"], QUESTIONS, on_predict_start=lambda ctx: order.append("percall"))+check("installed/per-call additive and ordered", order, ["installed", "percall"])++seq = []+f = make_fake()+f.predict_batch(["s0"], QUESTIONS,+ on_predict_start=[lambda c: seq.append(1), lambda c: seq.append(2)])+check("sequence/runs in order", seq, [1, 2])+++# --------------------------------------------------------------- system_one inherits+seen = []+f = make_fake()+f.hooks = normalise_hooks(on_predict_start=lambda ctx: seen.append(ctx.states))+out = f.system_one("state", QUESTIONS)+check("system_one/hook fired", seen, [["state"]])+check_true("system_one/returns a dict", isinstance(out, dict))+++# --------------------------------------------------------------- error policy+def boom(ctx):+ raise ValueError("hook boom")+++f = make_fake()+check_raises("hooks_raise=True/propagates", ValueError,+ lambda: f.predict_batch(["s0"], QUESTIONS, on_predict_start=boom))++f = make_fake()+with warnings.catch_warnings(record=True) as caught:+ warnings.simplefilter("always")+ res = f.predict_batch(["s0"], QUESTIONS, on_predict_start=boom, hooks_raise=False)+check("hooks_raise=False/continues", len(res), 1)+check_true("hooks_raise=False/warns",+ any(issubclass(w.category, RuntimeWarning) for w in caught))+++class ErrHook:+ def __init__(self):+ self.errors = []++ def on_error(self, ctx):+ self.errors.append(ctx.error)+++fail_seen = {}+++def end_on_fail(ctx):+ fail_seen["error"] = ctx.error+++def bad_forward(b):+ raise RuntimeError("infer boom")+++rec = ErrHook()+f = make_fake()+f._forward = bad_forward+raised = None+try:+ f.predict_batch(["s0"], QUESTIONS, hooks=[rec], on_predict_end=end_on_fail)+except RuntimeError as exc:+ raised = exc+check_true("on_error/propagates original", isinstance(raised, RuntimeError))+check("on_error/fired once", len(rec.errors), 1)+check("on_error/saw the error", str(rec.errors[0]), "infer boom")+check("on_error/end still ran", str(fail_seen.get("error")), "infer boom")+++class BadErrHook:+ def on_error(self, ctx):+ raise ValueError("hook error")+++f = make_fake()+f._forward = bad_forward+raised = None+try:+ f.predict_batch(["s0"], QUESTIONS, hooks=[BadErrHook()])+except BaseException as exc: # noqa: BLE001+ raised = exc+check_true("on_error/failing hook does not mask the original", isinstance(raised, RuntimeError))+check_true("on_error/failing hook is chained",+ isinstance(getattr(raised, "__context__", None), ValueError))+++# a start hook that raises must still run on_error and the end hooks+class StartErrRec:+ def __init__(self):+ self.errors = []++ def on_error(self, ctx):+ self.errors.append(ctx.error)+++start_fail_seen = []+++def raising_start(ctx):+ raise ValueError("start boom")+++def end_after_start_fail(ctx):+ start_fail_seen.append(ctx.error)+++rec2 = StartErrRec()+f = make_fake()+raised = None+try:+ f.predict_batch(["s0"], QUESTIONS, hooks=[rec2],+ on_predict_start=raising_start, on_predict_end=end_after_start_fail)+except ValueError as exc:+ raised = exc+check_true("start failure/propagates", isinstance(raised, ValueError))+check("start failure/on_error fired", len(rec2.errors), 1)+check("start failure/end fired with the error", len(start_fail_seen), 1)+check("start failure/end saw the error", str(start_fail_seen[0]), "start boom")+++# ctx.model carries the agent's model id+f = make_fake()+f.model_id = "convaiinnovations/laya"+models = []+f.predict_batch(["s0"], QUESTIONS, on_predict_end=lambda c: models.append(c.model))+check("context/model is the agent model_id", models, ["convaiinnovations/laya"])+++# --------------------------------------------------------------- empty inputs+empty = []+f = make_fake()+res = f.predict_batch([], QUESTIONS,+ on_predict_start=lambda c: empty.append(("s", c.results)),+ on_predict_end=lambda c: empty.append(("e", c.results)))+check("empty states/returns []", res, [])+check("empty states/start results None", empty[0][1], None)+check("empty states/end results []", empty[1][1], [])++empty = []+f = make_fake()+f.predict_batch(["s0"], {}, on_predict_end=lambda c: empty.append(c.results))+check("empty questions/end sees empty results",+ empty[0],+ [{"model": "laya-rl-agent", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}])+++# --------------------------------------------------------------- input guards and rescue+f = make_fake()+check_raises("guard/bare string raises", TypeError, lambda: f.predict_batch("nope", QUESTIONS))+f = make_fake()+check_raises("guard/bare dict raises", TypeError, lambda: f.predict_batch({"body": "x"}, QUESTIONS))+++def normalise_state(ctx):+ ctx.states = [ctx.states]+++f = make_fake()+f.predict_batch("nope", QUESTIONS, on_predict_start=normalise_state)+check("guard/a hook can normalise a bare string", f._encode_states, ["nope"])+++# --------------------------------------------------------------- hand-built instance+f = make_fake()+res = f.predict_batch(["s0", "s1", "s2"], QUESTIONS)+check("defaults/unset hooks are a no-op", len(res), 3)+check("defaults/offsets unchanged", [r["answers"]["_offset"] for r in res], [0, NQ, 2 * NQ])+++# --------------------------------------------------------------- dynamic registration+class Tag:+ def __init__(self, log, tag):+ self.log = log+ self.tag = tag++ def on_predict_end(self, ctx):+ self.log.append(self.tag)+++log = []+f = make_fake()+f.add_hook(Tag(log, "a"))+f.predict_batch(["s0"], QUESTIONS)+check("add_hook/fires", log, ["a"])+check("remove_hook/returns True", f.remove_hook(f.hooks[-1]), True)+log.clear()+f.predict_batch(["s0"], QUESTIONS)+check("remove_hook/no longer fires", log, [])+check("remove_hook/unknown returns False", f.remove_hook(object()), False)++log = []+f = make_fake()+check_true("add_hook/returns self", f.add_hook([Tag(log, "x"), Tag(log, "y")]) is f)+f.predict_batch(["s0"], QUESTIONS)+check("add_hook/sequence order", log, ["x", "y"])++log = []+f = make_fake()+f.add_hook(Tag(log, "installed"))+with f.hooks_installed(Tag(log, "temp")):+ f.predict_batch(["s0"], QUESTIONS)+f.predict_batch(["s0"], QUESTIONS)+check("hooks_installed/only during the block", log, ["installed", "temp", "installed"])++log = []+r = Router()+r.add_hook(Tag(log, "router"))+r.attach("english", FakeAgent())+r.predict("hello", QUESTIONS)+check("router/add_hook fires", log, ["router"])+++# --------------------------------------------------------------- per-call token budget+def make_len_fake():+ fake = make_fake()+ fake._seen = []++ def _encode(state, ids, internal, max_len=None, head_max_len=None):+ fake._seen.append((max_len, head_max_len))+ return [{"ids": [1, 2, 3], "markers": [0, 1], "qtype": 2} for _ in ids]++ fake._encode_state = _encode+ return fake+++f = make_len_fake()+f.predict_batch(["s"], QUESTIONS, max_len=128, head_max_len=64)+check("budget/per-call kwargs reach _encode_state", f._seen, [(128, 64)])++f = make_len_fake()+f.predict_batch(["s"], QUESTIONS, on_predict_start=lambda c: setattr(c, "max_len", 200))+check("budget/hook-set ctx reaches _encode_state", f._seen, [(200, None)])++f = make_len_fake()+f.predict_batch(["s"], QUESTIONS)+check("budget/default passes no override", f._seen, [(None, None)])+++class LenFake:+ def __init__(self):+ self.seen = []++ def system_one(self, state, questions, max_len=None, head_max_len=None):+ self.seen.append((max_len, head_max_len))+ return {"model": "x", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}+++lf = LenFake()+r = Router()+r.attach("english", lf)+r.predict("hello", QUESTIONS, max_len=256, head_max_len=128)+check("budget/router per-call reaches the agent", lf.seen, [(256, 128)])++lf = LenFake()+r = Router()+r.attach("english", lf)+r.predict("hello", QUESTIONS, on_predict_start=lambda c: setattr(c, "head_max_len", 96))+check("budget/router hook-set reaches the agent", lf.seen, [(None, 96)])+++class StrictFake:+ """An agent-like object that does not accept the override kwargs."""++ def system_one(self, state, questions):+ return {"model": "x", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}+++r = Router()+r.attach("english", StrictFake())+r.predict("hello", QUESTIONS)+check("budget/default does not pass override kwargs", True, True)+++# --------------------------------------------------------------- context semantics+from laya.hooks import PredictContext # noqa: E402++_c1 = PredictContext(states=[], questions={})+_c2 = PredictContext(states=[], questions={})+check_true("context/hashable by identity", isinstance(hash(_c1), int))+check("context/two contexts are never equal", _c1 == _c2, False)+++# --------------------------------------------------------------- validation+class NotAHook:+ pass+++check_raises("validation/hooks= rejects a class", TypeError, lambda: normalise_hooks(hooks=[NotAHook]))+check_raises("validation/hooks= rejects a plain callable", TypeError,+ lambda: normalise_hooks(hooks=[lambda ctx: None]))+check_raises("validation/on_predict_start must be callable", TypeError,+ lambda: normalise_hooks(on_predict_start=123))+++class BadMethod:+ on_predict_start = 5+++check_raises("validation/hook method must be callable", TypeError,+ lambda: normalise_hooks(hooks=[BadMethod()]))+++# --------------------------------------------------------------- Router+class RouteHook:+ def __init__(self):+ self.decisions = []++ def on_route(self, ctx):+ self.decisions.append(dict(ctx.decision))+ ctx.decision = RouteDecision(model="multilingual", repo="convaiinnovations/laya/multilingual",+ reason="pinned by hook", detection=None, workflow=None)+++rh = RouteHook()+r = Router(hooks=[rh])+decision = r.route("hello", QUESTIONS)+check("router/on_route fired", len(rh.decisions), 1)+check("router/on_route saw the original", rh.decisions[0]["model"], "english")+check("router/on_route can replace the decision", decision["model"], "multilingual")+++class LoadHook:+ def __init__(self):+ self.loads = []+ self.evicts = []++ def on_load(self, ctx):+ self.loads.append(ctx.model)++ def on_evict(self, ctx):+ self.evicts.append(ctx.model)+++class BuiltAgent(FakeAgent):+ def __init__(self, *args, **kwargs):+ pass+++lh = LoadHook()+real_agent = _agent_mod.Agent+_agent_mod.Agent = BuiltAgent+try:+ r = Router(max_loaded=1, hooks=[lh])+ r.load("english")+ r.load("multilingual") # evicts english+finally:+ _agent_mod.Agent = real_agent+check("router/on_load fired per build", lh.loads, ["english", "multilingual"])+check("router/on_evict fired on eviction", lh.evicts, ["english"])+++predict_seen = {}+r = Router(on_predict_start=lambda ctx: predict_seen.update(decision=dict(ctx.decision)),+ on_predict_end=lambda ctx: predict_seen.update(results=ctx.results))+r.attach("english", FakeAgent())+out = r.predict("hello", QUESTIONS)+check("router/predict start sees the decision", predict_seen["decision"]["model"], "english")+check("router/predict end sees results", len(predict_seen["results"]), 1)+check("router/predict keeps routing", out["routing"]["model"], "english")++cached = [{"model": "cached", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}]+r = Router()+r.attach("english", FakeAgent())+skipped = r.predict("hello", QUESTIONS, on_predict_start=lambda c: c.skip(cached))+check("router/skip returns the cached payload", skipped, cached[0])+check("router/skip still adds routing", skipped.get("routing", {}).get("model"), "english")+++class PerCallRoute:+ def __init__(self):+ self.decisions = []++ def on_route(self, ctx):+ self.decisions.append(dict(ctx.decision))+++pcr = PerCallRoute()+r = Router()+r.attach("english", FakeAgent())+r.predict("hello", QUESTIONS, hooks=[pcr])+check("router/per-call hooks apply to on_route", len(pcr.decisions), 1)+++# --------------------------------------------------------------- hooks_concurrent storage+r = Router()+check("router/hooks_concurrent default True", r.hooks_concurrent, True)+check_true("router/hooks_concurrent default has no lock", r._hooks_lock is None)+r = Router(hooks_concurrent=False)+check("router/hooks_concurrent=False stored", r.hooks_concurrent, False)+check_true("router/hooks_concurrent=False installs a lock", r._hooks_lock is not None)+++# --------------------------------------------------------------- concurrency+contexts = []+++def slow_start(ctx):+ time.sleep(0.01)+ contexts.append(id(ctx))+++f = make_fake()+threads = [threading.Thread(target=lambda: f.predict_batch(["s0"], QUESTIONS,+ on_predict_start=slow_start))+ for _ in range(5)]+for t in threads:+ t.start()+for t in threads:+ t.join()+check("concurrency/one independent context per call", len(set(contexts)), 5)+++class ConcHook:+ def __init__(self):+ self.active = 0+ self.max_active = 0+ self._lock = threading.Lock()++ def on_predict_start(self, ctx):+ with self._lock:+ self.active += 1+ self.max_active = max(self.max_active, self.active)+ time.sleep(0.02)+ with self._lock:+ self.active -= 1+++conc = ConcHook()+f = make_fake()+f.hooks = [conc]+f._hooks_lock = threading.RLock() # what hooks_concurrent=False installs+threads = [threading.Thread(target=lambda: f.predict_batch(["s0"], QUESTIONS)) for _ in range(5)]+for t in threads:+ t.start()+for t in threads:+ t.join()+check("concurrency/hooks_concurrent=False serialises", conc.max_active, 1)+++# --------------------------------------------------------------- ONNXAgent parity+from laya.onnx_agent import ONNXAgent # noqa: E402++o = ONNXAgent.__new__(ONNXAgent)+o.model_id = "convaiinnovations/laya-onnx"+o._infer = lambda state, questions: {"model": "onnx", "answers": {},+ "usage": {"input_tokens": 0, "output_tokens": 0}}+onnx_seen = []+onnx_models = []+onnx_out = o.system_one("s", QUESTIONS,+ on_predict_start=lambda c: onnx_seen.append("start"),+ on_predict_end=lambda c: (onnx_seen.append("end"), onnx_models.append(c.model)))+check("onnx/hooks fire", onnx_seen, ["start", "end"])+check("onnx/returns the inference result", onnx_out["model"], "onnx")+check("onnx/context model is the agent model_id", onnx_models, ["convaiinnovations/laya-onnx"])++o = ONNXAgent.__new__(ONNXAgent)+++def _never(*args):+ raise AssertionError("inference should have been skipped")+++o._infer = _never+check("onnx/skip short-circuits",+ o.system_one("s", QUESTIONS, on_predict_start=lambda c: c.skip([{"model": "cached"}])),+ {"model": "cached"})+++# --------------------------------------------------------------- report+print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))+for f_ in FAIL:+ print(" FAIL " + f_)+if not FAIL:+ print("all hook tests passed")+sys.exit(1 if FAIL else 0)diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.pynew file mode 100644index 00000000..fd283504--- /dev/null+++ b/tests/test_hooks_api.py@@ -0,0 +1,148 @@+"""API-stability guard for prediction hooks.++These tests pin the public hook surface (parameter names, kinds, defaults, context fields,+lifecycle events, exports) so a change that would break callers fails here first. If a change+is intentional, update this file in the same commit.++Run: python tests/test_hooks_api.py+"""+import dataclasses+import inspect+import os+import sys++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))++import laya # noqa: E402+from laya import Agent, PredictContext, PredictHook, Router, load # noqa: E402+from laya.hooks import HOOK_EVENTS, Hook # noqa: E402+from laya.onnx_agent import ONNXAgent # noqa: E402++PASS, FAIL = [], []+++def check(name, got, want):+ if got == want:+ PASS.append(name)+ else:+ FAIL.append("%s: got %r, want %r" % (name, got, want))+++def check_true(name, cond, detail=""):+ if cond:+ PASS.append(name)+ else:+ FAIL.append("%s%s" % (name, ": " + detail if detail else ""))+++def sig(fn):+ return inspect.signature(fn).parameters+++def check_param(name, fn, param, default, kind=None):+ params = sig(fn)+ if param not in params:+ FAIL.append("%s/%s: missing parameter" % (name, param))+ return+ p = params[param]+ check("%s/%s default" % (name, param), p.default, default)+ if kind is not None:+ check("%s/%s kind" % (name, param), p.kind, kind)+++HOOK_KEYS = {+ "hooks": None,+ "on_predict_start": None,+ "on_predict_end": None,+ "hooks_raise": True,+ "hooks_concurrent": True,+}++# --------------------------------------------------------------- constructors+for label, fn in (("Agent.__init__", Agent.__init__), ("load", load),+ ("Router.__init__", Router.__init__), ("ONNXAgent.__init__", ONNXAgent.__init__)):+ for param, default in HOOK_KEYS.items():+ check_param(label, fn, param, default)++# Router keeps lang_guess too+check_param("Router.__init__", Router.__init__, "lang_guess", None)++# --------------------------------------------------------------- predict surfaces+for label, fn in (("Agent.predict_batch", Agent.predict_batch),+ ("Agent.system_one", Agent.system_one),+ ("Router.predict", Router.predict),+ ("ONNXAgent.system_one", ONNXAgent.system_one)):+ check_param(label, fn, "hooks", None)+ check_param(label, fn, "on_predict_start", None)+ check_param(label, fn, "on_predict_end", None)+ check_param(label, fn, "hooks_raise", None)++check_param("Agent.predict_batch", Agent.predict_batch, "batch_size", None)++# per-call token-budget overrides+for label, fn in (("Agent.predict_batch", Agent.predict_batch),+ ("Agent.system_one", Agent.system_one),+ ("Router.predict", Router.predict),+ ("ONNXAgent.system_one", ONNXAgent.system_one)):+ check_param(label, fn, "max_len", None)+ check_param(label, fn, "head_max_len", None)++# route() takes per-call hooks so a hook can pin a checkpoint for one call+check_param("Router.route", Router.route, "hooks", None)+check_param("Router.route", Router.route, "hooks_raise", None)++# --------------------------------------------------------------- aliases+check_true("Agent.predict is Agent.system_one", Agent.predict is Agent.system_one)+check_true("Router.system_one is Router.predict", Router.system_one is Router.predict)+check_true("ONNXAgent.predict is ONNXAgent.system_one", ONNXAgent.predict is ONNXAgent.system_one)++# --------------------------------------------------------------- context+FIELDS = ["states", "questions", "run_id", "results", "decision", "model", "agent", "router",+ "max_len", "head_max_len", "usage", "started_at", "elapsed_ms", "error"]+check("PredictContext fields", [f.name for f in dataclasses.fields(PredictContext)], FIELDS)+check("PredictContext/states required", PredictContext.__dataclass_fields__["states"].default,+ dataclasses.MISSING)+check("PredictContext/questions required", PredictContext.__dataclass_fields__["questions"].default,+ dataclasses.MISSING)+for optional in ("results", "decision", "model", "agent", "router", "usage", "elapsed_ms", "error"):+ check("PredictContext/%s default None" % optional,+ PredictContext.__dataclass_fields__[optional].default, None)+check_true("PredictContext/skip exists", callable(getattr(PredictContext, "skip", None)))+check_true("PredictContext/run_id has a factory",+ PredictContext.__dataclass_fields__["run_id"].default_factory is not dataclasses.MISSING)++# --------------------------------------------------------------- hook protocol+check("Hook lifecycle events", set(HOOK_EVENTS),+ {"on_predict_start", "on_predict_end", "on_route", "on_load", "on_evict", "on_error"})+for event in HOOK_EVENTS:+ check_true("Hook/%s declared" % event, hasattr(Hook, event))+check_true("PredictHook is callable-typed", callable(PredictHook))++# --------------------------------------------------------------- exports+for name in ("PredictContext", "PredictHook", "Hook"):+ check_true("__all__/%s" % name, name in laya.__all__)+ check_true("laya.%s exists" % name, hasattr(laya, name))++# --------------------------------------------------------------- class defaults+for label, cls in (("Agent", Agent), ("Router", Router), ("ONNXAgent", ONNXAgent)):+ check("%s/hooks default" % label, cls.hooks, ())+ check("%s/hooks_raise default" % label, cls.hooks_raise, True)+ check("%s/hooks_concurrent default" % label, cls.hooks_concurrent, True)+ check("%s/_hooks_lock default" % label, cls._hooks_lock, None)++# Agent and ONNXAgent carry the checkpoint id for ctx.model; Router has no single model.+for label, cls in (("Agent", Agent), ("ONNXAgent", ONNXAgent)):+ check("%s/model_id default" % label, cls.model_id, None)++# runtime registration surface+for label, cls in (("Agent", Agent), ("Router", Router), ("ONNXAgent", ONNXAgent)):+ for method in ("add_hook", "remove_hook", "hooks_installed"):+ check_true("%s/%s exists" % (label, method), callable(getattr(cls, method, None)))+++print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))+for f in FAIL:+ print(" FAIL", f)+if not FAIL:+ print("all hook API tests passed")+sys.exit(1 if FAIL else 0)