Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Prediction Hooks
Hooks let you observe or shape every decision Laya makes, without forking it.
They are the extension seam for the things every real deployment needs: audit logging, PII
redaction before inference, caching, metrics, confidence gating, routing overrides, and
forwarding a decision to an external service. They are opt-in: with no hooks configured the
behaviour of Agent, Router and ONNXAgent is unchanged.
They are not only for direct calls. Each LangChain and LangGraph runnable takes the same five per-call arguments, so a hook can be attached to one node in a graph rather than to the whole agent.
This folder is the full reference. Start here, then dive into the page you need:
| page | what is in it |
|---|---|
| API reference | every class, field, parameter and default |
| Lifecycle | exactly when each hook runs, with flowcharts |
| Errors | hooks_raise, on_error, exception chaining, failure matrix |
| Patterns and anti-patterns | what to do, what to avoid, and why |
| Examples | copy-paste recipes for every use case |
| Tracing | run_id, span correlation, OpenTelemetry |
Quick start
import laya
def log(ctx):
print(ctx.model, ctx.results[0]["answers"], ctx.elapsed_ms)
agent = laya.load("convaiinnovations/laya", on_predict_end=log)
agent.system_one("I was charged twice.", {"urgent": {"type": "noul", "instructions": "Urgent?"}})
An object can implement any subset of the lifecycle events:
class Audit:
def on_predict_start(self, ctx):
print("start", ctx.run_id)
def on_predict_end(self, ctx):
print("end", ctx.run_id, ctx.usage, ctx.elapsed_ms)
def on_error(self, ctx):
print("failed", ctx.run_id, ctx.error)
laya.load("convaiinnovations/laya", hooks=[Audit()])
Hooks can also be added later or scoped to a block:
agent.add_hook(tracer) # attach at runtime
with agent.hooks_installed(debug): # installed for the block, removed on exit
agent.system_one(state, questions)
See runtime registration. For a hook that should apply everywhere without threading it through every call, register it once with process-wide defaults:
from laya import hooks
hooks.set_default_hooks(hooks=[Tracer()])
The mental model
There are three ideas.
A hook is a callable or an object. A plain function is convenient for one event; an object is convenient for several. Only the object form goes to
hooks=-- a plain callable there is refused at construction, becausehooks=reads for the six lifecycle methods (on_predict_start,on_predict_end,on_route,on_load,on_evict,on_error) and a bare function has none of them. Pass a plain callable ason_predict_start=oron_predict_end=; those are the single-event parameters, so a callable is exactly what they take.Every hook of one call shares one mutable
PredictContext. It carries the states, questions, results, routing decision, model name, usage, timing and any error. A call can carry many states at once (predict_batch), so a hook that means to cover every decision has to iteratectx.statesandctx.results;ctx.usageandctx.elapsed_msare totals for the call. Because the context is mutable, a hook can shape the call, not only watch it: redact the state, rewrite the questions, replace the result, or skip inference with a cached answer.There are two scopes.
Agenthooks wrap a forward pass;Routerhooks wrap routing plus inference and can also see model lifecycle (on_route,on_load,on_evict). This mirrors the "run hooks" vs "agent hooks" split in other agent frameworks.
Router.predict(state, questions)
┌──────────────────────────────────────────────────────────────────────────┐
│ route() │
│ ├─ detect language / workflow │
│ └─ on_route ctx.decision (a hook may replace it) │
│ │
│ load(decision.model) │
│ ├─ build checkpoint on first use ──► on_load ctx.model, ctx.agent │
│ └─ evict LRU checkpoint ───────────► on_evict ctx.model │
│ │
│ on_predict_start ctx.states, ctx.questions, ctx.decision │
│ │ │
│ ├── ctx.skip(results)? ──► skip the forward pass │
│ │ │
│ └── Agent.system_one(...) ──► Agent-level hooks run here │
│ on_predict_start ─► forward ─► on_predict_end │
│ │
│ result["routing"] = decision │
│ on_predict_end ctx.results, ctx.usage, ctx.elapsed_ms │
└──────────────────────────────────────────────────────────────────────────┘
any failure on the way ──► on_error, then on_predict_end
Scope at a glance
Agent / ONNXAgent |
Router |
|
|---|---|---|
on_predict_start |
yes | yes |
on_predict_end |
yes | yes |
on_error |
yes | yes |
on_route |
no | yes |
on_load |
no | yes |
on_evict |
no | yes |
laya.serve and the MCP server call Router.predict, so Router hooks fire for them
automatically. Agent hooks fire whenever the Router runs an attached or built agent.
Compatibility
- No hooks configured means no behavioural change. The unset path is regression-tested.
- All hook parameters are keyword arguments with defaults, so existing calls keep working.
laya/hooks.pyis pure Python:import layadoes not pull in torch because of it.- Hooks are synchronous by default. An
async defevent can be wrapped inAsyncHook, or passed as a plain async callable, and it runs to completion for you. hooks_timeoutbounds a slow hook so it cannot hang a served request.- Keep hooks fast and non-blocking; see errors and patterns for the
consequences on
laya.serve.
See also
examples/hooks/: runnable audit, redact, cache and metrics hooks.tests/test_hooks.py: the behaviour spec.tests/test_hooks_api.py: the API-stability guard.laya/hooks.py: the implementation.