Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(hooks): BaseHook and process-wide default hooks (#276)

Emiliano G.O.27b6576abdce
diff --git a/docs/hooks/api.md b/docs/hooks/api.md
index 5072179b..519e5559 100644
--- a/docs/hooks/api.md
+++ b/docs/hooks/api.md
@@ -101,6 +101,22 @@ A hook is free to define extra attributes and methods; only the six event names
If a hook defines one of the six as a non-callable, configuration fails fast (see
[Validation](#validation)).
+### BaseHook
+
+`BaseHook` is the concrete counterpart to the protocol: a class with a no-op body for every event.
+Subclass it and override only the events you need.
+
+```python
+from laya import BaseHook
+
+class Audit(BaseHook):
+ def on_predict_end(self, ctx):
+ ship(ctx.run_id, ctx.results)
+```
+
+`Hook` is best when you want structural typing (any object with the right methods); `BaseHook` is
+best when you want an explicit base to subclass and call `super()` on.
+
## Convenience types
```python
@@ -128,6 +144,27 @@ with agent.hooks_installed(debug): # installed for the block, removed on exit
any number of hook objects or sequences and restores the previous list on exit, including when
the block raises.
+## Process-wide defaults
+
+`laya.hooks` keeps a small process-wide registry, so a tracer, metrics hook or tenant tagger does
+not have to be threaded through every `Agent` and `Router`. Defaults run **first**, then the
+hooks installed on the instance, then per-call hooks.
+
+```python
+from laya import hooks
+
+hooks.set_default_hooks(hooks=[Tracer()]) # replaces the set, accepts the hooks= arguments
+hooks.add_default_hook(Metrics()) # appends
+hooks.clear_default_hooks() # removes everything
+hooks.default_hooks() # a copy of the current list
+
+hooks.compose_hooks(agent.hooks) # defaults + installed (advanced)
+```
+
+Defaults apply to every event, including the Router lifecycle events `on_load` and `on_evict`.
+The registry is read at call time, so hooks set after an `Agent` or `Router` is built still apply.
+There is no per-instance opt-out; call `clear_default_hooks()` to turn the process-wide set off.
+
## Configuration surface
Every entry point accepts the same five hook parameters. `hooks` takes an object or a sequence
diff --git a/docs/hooks/examples.md b/docs/hooks/examples.md
index 242a44e5..ddfd73b6 100644
--- a/docs/hooks/examples.md
+++ b/docs/hooks/examples.md
@@ -18,6 +18,7 @@ Copy-paste recipes. Every snippet is self-contained apart from the helpers it na
- [HTTP server](#http-server)
- [ONNXAgent](#onnxagent)
- [Runtime registration](#runtime-registration)
+- [Base class and process-wide defaults](#base-class-and-process-wide-defaults)
- [Token budget](#token-budget)
- [Testing hooks](#testing-hooks)
@@ -305,6 +306,24 @@ with agent.hooks_installed(DebugDump()):
agent.system_one(state, questions) # DebugDump only here
```
+## Base class and process-wide defaults
+
+Subclass `BaseHook` to override only what you need, and register something once for the whole
+process instead of passing it to every `Agent` and `Router`.
+
+```python
+from laya import BaseHook, hooks
+
+class Audit(BaseHook):
+ def on_predict_end(self, ctx):
+ ship(ctx.run_id, ctx.results)
+
+hooks.set_default_hooks(hooks=[Audit()]) # runs for every call in the process
+
+# later, or in tests:
+hooks.clear_default_hooks()
+```
+
## Token budget
Shape the token budget for one call, from a hook or a per-call argument.
diff --git a/docs/hooks/index.md b/docs/hooks/index.md
index 1f5f75fe..9a96e3cc 100644
--- a/docs/hooks/index.md
+++ b/docs/hooks/index.md
@@ -54,7 +54,15 @@ with agent.hooks_installed(debug): # installed for the block, removed on exit
agent.system_one(state, questions)
```
-See [runtime registration](api.md#runtime-registration).
+See [runtime registration](api.md#runtime-registration). For a hook that should apply everywhere
+without threading it through every call, register it once with
+[process-wide defaults](api.md#process-wide-defaults):
+
+```python
+from laya import hooks
+
+hooks.set_default_hooks(hooks=[Tracer()])
+```
## The mental model
diff --git a/docs/hooks/patterns.md b/docs/hooks/patterns.md
index d3942852..61e8655d 100644
--- a/docs/hooks/patterns.md
+++ b/docs/hooks/patterns.md
@@ -15,6 +15,7 @@ that hold up in production and the ones that bite.
- [Multi-tenant context](#multi-tenant-context)
- [Composition](#composition)
- [Scoped instrumentation](#scoped-instrumentation)
+ - [Process-wide instrumentation](#process-wide-instrumentation)
- [Token-budget shaping](#token-budget-shaping)
- [Anti-patterns](#anti-patterns)
- [Blocking work](#blocking-work)
@@ -230,6 +231,24 @@ with agent.hooks_installed(DebugDump()):
`add_hook`/`remove_hook` do the same without a block, for a tracer that lives as long as the
process.
+### Process-wide instrumentation
+
+A tracer or metrics hook that every decision should see can be registered once, instead of being
+passed to each `Agent` and `Router`. Defaults run before the instance and per-call hooks.
+
+```python
+from laya import BaseHook, hooks
+
+class Metrics(BaseHook):
+ def on_predict_end(self, ctx):
+ record(ctx.model, ctx.elapsed_ms)
+
+hooks.set_default_hooks(hooks=[Metrics()])
+```
+
+This is global state, so scope it deliberately: set it once at startup, and `clear_default_hooks()`
+in tests so one test cannot leak a hook into the next.
+
### Token-budget shaping
A start hook can raise the token budget for one call, for example when a question has many
diff --git a/laya/__init__.py b/laya/__init__.py
index cb41e415..facc843d 100644
--- a/laya/__init__.py
+++ b/laya/__init__.py
@@ -1,7 +1,7 @@
"""Laya: Fast, non-autoregressive System 1 decision engine with calibrated probabilities."""
from .email import clean_email_body, email_state
-from .hooks import Hook, PredictContext, PredictHook
+from .hooks import BaseHook, Hook, PredictContext, PredictHook
from .lang import analyse as detect_language
from .lang import detect_script, is_english
from .presets import (
@@ -91,5 +91,6 @@ __all__ = [
"PredictContext",
"PredictHook",
"Hook",
+ "BaseHook",
"__version__",
]
diff --git a/laya/agent.py b/laya/agent.py
index 19520f5f..4182f3ed 100644
--- a/laya/agent.py
+++ b/laya/agent.py
@@ -25,7 +25,7 @@ from .common import (
render_options,
temp_bucket,
)
-from .hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooks
+from .hooks import HookRegistry, PredictContext, aggregate_usage, compose_hooks, dispatch, normalise_hooks
def _fix_tokenizer_config(path: str):
@@ -634,7 +634,7 @@ class Agent(HookRegistry):
A list of per-state result dicts, each identical in shape to `system_one`'s output and
aligned with `states` by index.
"""
- active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)
+ active = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end)
raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)
ctx = PredictContext(states=states, questions=questions, model=self.model_id, agent=self,
max_len=max_len, head_max_len=head_max_len)
diff --git a/laya/hooks.py b/laya/hooks.py
index 96e3e3f0..27f419fc 100644
--- a/laya/hooks.py
+++ b/laya/hooks.py
@@ -58,6 +58,32 @@ class Hook(Protocol):
def on_error(self, ctx: PredictContext) -> None: ...
+class BaseHook:
+ """No-op base class: subclass it and override only the events you need.
+
+ `Hook` is the structural protocol; `BaseHook` is the concrete convenience when you would
+ rather subclass than implement methods by shape. Every method does nothing by default.
+ """
+
+ def on_predict_start(self, ctx: PredictContext) -> None:
+ pass
+
+ def on_predict_end(self, ctx: PredictContext) -> None:
+ pass
+
+ def on_route(self, ctx: PredictContext) -> None:
+ pass
+
+ def on_load(self, ctx: PredictContext) -> None:
+ pass
+
+ def on_evict(self, ctx: PredictContext) -> None:
+ pass
+
+ def on_error(self, ctx: PredictContext) -> None:
+ pass
+
+
PredictHook = Callable[[PredictContext], None]
HookArg = Union[Hook, Sequence[Hook], None]
PredictHookArg = Union[PredictHook, Sequence[PredictHook], None]
@@ -146,6 +172,49 @@ def normalise_hooks(
return result
+_DEFAULT_HOOKS: List[Any] = []
+_DEFAULT_HOOKS_LOCK = threading.Lock()
+
+
+def default_hooks() -> List[Any]:
+ """The process-wide hooks, a copy, in order. Empty unless set via `set_default_hooks`."""
+ with _DEFAULT_HOOKS_LOCK:
+ return list(_DEFAULT_HOOKS)
+
+
+def set_default_hooks(hooks=None, on_predict_start=None, on_predict_end=None) -> None:
+ """Replace the process-wide default hooks.
+
+ Defaults run before installed and per-call hooks for every `Agent`, `Router` and
+ `ONNXAgent` in the process, so a tracer or metrics hook does not have to be threaded
+ through every construction. Accepts the same arguments as the `hooks=` parameter.
+ """
+ normalised = normalise_hooks(hooks, on_predict_start, on_predict_end)
+ with _DEFAULT_HOOKS_LOCK:
+ _DEFAULT_HOOKS[:] = normalised
+
+
+def add_default_hook(hook: HookArg) -> None:
+ """Append one hook or a sequence to the process-wide defaults."""
+ normalised = normalise_hooks(hook)
+ with _DEFAULT_HOOKS_LOCK:
+ _DEFAULT_HOOKS.extend(normalised)
+
+
+def clear_default_hooks() -> None:
+ """Remove every process-wide default hook."""
+ with _DEFAULT_HOOKS_LOCK:
+ _DEFAULT_HOOKS.clear()
+
+
+def compose_hooks(installed, hooks=None, on_predict_start=None, on_predict_end=None) -> List[Any]:
+ """Effective hook list for one call: defaults, then installed, then per-call hooks.
+
+ Reads the process-wide defaults at call time, so hooks set after construction still apply.
+ """
+ return default_hooks() + list(installed) + normalise_hooks(hooks, on_predict_start, on_predict_end)
+
+
class HookRegistry:
"""Mixin giving a runtime-mutable hook list.
diff --git a/laya/onnx_agent.py b/laya/onnx_agent.py
index 8664bd25..8d363df7 100644
--- a/laya/onnx_agent.py
+++ b/laya/onnx_agent.py
@@ -7,7 +7,7 @@ from typing import Any, Dict, Optional, Union
import numpy as np
-from laya.hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooks
+from laya.hooks import HookRegistry, PredictContext, aggregate_usage, compose_hooks, dispatch, normalise_hooks
from laya.common import (
QTYPES,
build_sequence,
@@ -160,7 +160,7 @@ class ONNXAgent(HookRegistry):
max_len: Optional[int] = None,
head_max_len: Optional[int] = None) -> Dict[str, Any]:
"""Evaluate typed questions, running any opt-in hooks around the inference."""
- active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)
+ active = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end)
raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)
ctx = PredictContext(states=[state], questions=questions, model=self.model_id, agent=self,
max_len=max_len, head_max_len=head_max_len)
diff --git a/laya/router.py b/laya/router.py
index 837d6fe0..10dcb97b 100644
--- a/laya/router.py
+++ b/laya/router.py
@@ -35,7 +35,7 @@ import time
from collections.abc import Sequence as SequenceABC
from typing import Any, Dict, List, Optional, Sequence, Union
-from .hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooks
+from .hooks import HookRegistry, PredictContext, aggregate_usage, compose_hooks, dispatch, normalise_hooks
from .lang import analyse
# The hub repo bundles all three checkpoints; only the requested subfolder is downloaded.
@@ -263,7 +263,7 @@ class Router(HookRegistry):
evicted = self._evict_locked()
# Lifecycle hooks fire after the lock is released, so a hook can safely call the Router.
self._dispatch_lifecycle("on_evict", evicted)
- dispatch(self.hooks, "on_load",
+ dispatch(compose_hooks(self.hooks), "on_load",
PredictContext(states=[], questions={}, model=key, agent=agent, router=self),
raise_errors=self.hooks_raise, lock=self._hooks_lock)
return agent
@@ -306,7 +306,7 @@ class Router(HookRegistry):
def _dispatch_lifecycle(self, event: str, names: List[str]) -> None:
for name in names:
- dispatch(self.hooks, event,
+ dispatch(compose_hooks(self.hooks), event,
PredictContext(states=[], questions={}, model=name, router=self),
raise_errors=self.hooks_raise, lock=self._hooks_lock)
@@ -404,7 +404,7 @@ class Router(HookRegistry):
"""
decision = self._route(state, questions, model=model, task=task, lang=lang, lang_guess=lang_guess)
raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)
- active = list(self.hooks) + normalise_hooks(hooks)
+ active = compose_hooks(self.hooks, hooks)
ctx = PredictContext(states=[state], questions=questions or {}, decision=decision, router=self)
dispatch(active, "on_route", ctx, raise_errors=raise_errors, lock=self._hooks_lock)
return ctx.decision
@@ -522,7 +522,7 @@ class Router(HookRegistry):
and see `ctx.decision`; see `laya.hooks`. `max_len` / `head_max_len` override the agent
token budget for this call (a start hook may set `ctx.max_len` / `ctx.head_max_len`).
"""
- active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)
+ active = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end)
raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)
# Per-call hooks apply to the whole call, including on_route inside route().
diff --git a/tests/test_hooks.py b/tests/test_hooks.py
index 5ce598cc..560354d3 100644
--- a/tests/test_hooks.py
+++ b/tests/test_hooks.py
@@ -890,6 +890,81 @@ check("onnx/skip short-circuits",
{"model": "cached"})
+# --------------------------------------------------------------- BaseHook
+from laya import BaseHook # noqa: E402
+
+
+class OnlyEnd(BaseHook):
+ def __init__(self, log, tag):
+ self.log = log
+ self.tag = tag
+
+ def on_predict_end(self, ctx):
+ self.log.append(self.tag)
+
+
+log = []
+f = make_fake()
+f.add_hook(OnlyEnd(log, "end"))
+f.predict_batch(["s0"], QUESTIONS)
+check("BaseHook/overridden event fires", log, ["end"])
+
+f = make_fake()
+f.add_hook(BaseHook()) # every method is a no-op
+f.predict_batch(["s0"], QUESTIONS)
+check("BaseHook/no-op instance is harmless", f._forward_calls, [2])
+
+
+# --------------------------------------------------------------- process-wide defaults
+from laya import hooks as _hooks # noqa: E402
+
+log = []
+_hooks.set_default_hooks(on_predict_end=lambda ctx: log.append("default"))
+try:
+ f = make_fake()
+ f.add_hook(Tag(log, "installed"))
+ f.predict_batch(["s0"], QUESTIONS, on_predict_end=lambda ctx: log.append("percall"))
+ check("defaults/run before installed and per-call", log, ["default", "installed", "percall"])
+finally:
+ _hooks.clear_default_hooks()
+check("defaults/clear empties the registry", _hooks.default_hooks(), [])
+
+log = []
+_hooks.add_default_hook(Tag(log, "a"))
+_hooks.add_default_hook(Tag(log, "b"))
+try:
+ f = make_fake()
+ f.predict_batch(["s0"], QUESTIONS)
+ check("defaults/add in order", log, ["a", "b"])
+finally:
+ _hooks.clear_default_hooks()
+
+
+class LifeDefaults(BaseHook):
+ def __init__(self):
+ self.events = []
+
+ def on_load(self, ctx):
+ self.events.append(("load", ctx.model))
+
+ def on_evict(self, ctx):
+ self.events.append(("evict", ctx.model))
+
+
+ld = LifeDefaults()
+_hooks.set_default_hooks(hooks=[ld])
+try:
+ _agent_mod.Agent = BuiltAgent
+ r = Router(max_loaded=1)
+ r.load("english")
+ r.load("multilingual") # evicts english
+finally:
+ _agent_mod.Agent = real_agent
+ _hooks.clear_default_hooks()
+check("defaults/cover router lifecycle", ld.events,
+ [("load", "english"), ("evict", "english"), ("load", "multilingual")])
+
+
# --------------------------------------------------------------- report
print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))
for f_ in FAIL:
diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.py
index fd283504..7b66d6ea 100644
--- a/tests/test_hooks_api.py
+++ b/tests/test_hooks_api.py
@@ -14,7 +14,7 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import laya # noqa: E402
-from laya import Agent, PredictContext, PredictHook, Router, load # noqa: E402
+from laya import Agent, BaseHook, PredictContext, PredictHook, Router, load # noqa: E402
from laya.hooks import HOOK_EVENTS, Hook # noqa: E402
from laya.onnx_agent import ONNXAgent # noqa: E402
@@ -119,10 +119,20 @@ for event in HOOK_EVENTS:
check_true("PredictHook is callable-typed", callable(PredictHook))
# --------------------------------------------------------------- exports
-for name in ("PredictContext", "PredictHook", "Hook"):
+for name in ("PredictContext", "PredictHook", "Hook", "BaseHook"):
check_true("__all__/%s" % name, name in laya.__all__)
check_true("laya.%s exists" % name, hasattr(laya, name))
+# BaseHook is the concrete no-op base class; all six events exist and are callable.
+for event in HOOK_EVENTS:
+ check_true("BaseHook/%s callable" % event, callable(getattr(BaseHook, event, None)))
+
+# process-wide default registry lives in laya.hooks (not the top level)
+for helper in ("default_hooks", "set_default_hooks", "add_default_hook", "clear_default_hooks",
+ "compose_hooks"):
+ check_true("laya.hooks/%s exists" % helper, callable(getattr(__import__("laya.hooks", fromlist=[helper]), helper, None)))
+check_true("defaults/not exported at top level", not hasattr(laya, "set_default_hooks"))
+
# --------------------------------------------------------------- class defaults
for label, cls in (("Agent", Agent), ("Router", Router), ("ONNXAgent", ONNXAgent)):
check("%s/hooks default" % label, cls.hooks, ())