Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(hooks): BaseHook and process-wide default hooks (#276)
Emiliano G.O.27b6576abdce
diff --git a/docs/hooks/api.md b/docs/hooks/api.mdindex 5072179b..519e5559 100644--- a/docs/hooks/api.md+++ b/docs/hooks/api.md@@ -101,6 +101,22 @@ A hook is free to define extra attributes and methods; only the six event namesIf a hook defines one of the six as a non-callable, configuration fails fast (see[Validation](#validation)).+### BaseHook++`BaseHook` is the concrete counterpart to the protocol: a class with a no-op body for every event.+Subclass it and override only the events you need.++```python+from laya import BaseHook++class Audit(BaseHook):+ def on_predict_end(self, ctx):+ ship(ctx.run_id, ctx.results)+```++`Hook` is best when you want structural typing (any object with the right methods); `BaseHook` is+best when you want an explicit base to subclass and call `super()` on.+## Convenience types```python@@ -128,6 +144,27 @@ with agent.hooks_installed(debug): # installed for the block, removed on exitany number of hook objects or sequences and restores the previous list on exit, including whenthe block raises.+## Process-wide defaults++`laya.hooks` keeps a small process-wide registry, so a tracer, metrics hook or tenant tagger does+not have to be threaded through every `Agent` and `Router`. Defaults run **first**, then the+hooks installed on the instance, then per-call hooks.++```python+from laya import hooks++hooks.set_default_hooks(hooks=[Tracer()]) # replaces the set, accepts the hooks= arguments+hooks.add_default_hook(Metrics()) # appends+hooks.clear_default_hooks() # removes everything+hooks.default_hooks() # a copy of the current list++hooks.compose_hooks(agent.hooks) # defaults + installed (advanced)+```++Defaults apply to every event, including the Router lifecycle events `on_load` and `on_evict`.+The registry is read at call time, so hooks set after an `Agent` or `Router` is built still apply.+There is no per-instance opt-out; call `clear_default_hooks()` to turn the process-wide set off.+## Configuration surfaceEvery entry point accepts the same five hook parameters. `hooks` takes an object or a sequencediff --git a/docs/hooks/examples.md b/docs/hooks/examples.mdindex 242a44e5..ddfd73b6 100644--- a/docs/hooks/examples.md+++ b/docs/hooks/examples.md@@ -18,6 +18,7 @@ Copy-paste recipes. Every snippet is self-contained apart from the helpers it na- [HTTP server](#http-server)- [ONNXAgent](#onnxagent)- [Runtime registration](#runtime-registration)+- [Base class and process-wide defaults](#base-class-and-process-wide-defaults)- [Token budget](#token-budget)- [Testing hooks](#testing-hooks)@@ -305,6 +306,24 @@ with agent.hooks_installed(DebugDump()):agent.system_one(state, questions) # DebugDump only here```+## Base class and process-wide defaults++Subclass `BaseHook` to override only what you need, and register something once for the whole+process instead of passing it to every `Agent` and `Router`.++```python+from laya import BaseHook, hooks++class Audit(BaseHook):+ def on_predict_end(self, ctx):+ ship(ctx.run_id, ctx.results)++hooks.set_default_hooks(hooks=[Audit()]) # runs for every call in the process++# later, or in tests:+hooks.clear_default_hooks()+```+## Token budgetShape the token budget for one call, from a hook or a per-call argument.diff --git a/docs/hooks/index.md b/docs/hooks/index.mdindex 1f5f75fe..9a96e3cc 100644--- a/docs/hooks/index.md+++ b/docs/hooks/index.md@@ -54,7 +54,15 @@ with agent.hooks_installed(debug): # installed for the block, removed on exitagent.system_one(state, questions)```-See [runtime registration](api.md#runtime-registration).+See [runtime registration](api.md#runtime-registration). For a hook that should apply everywhere+without threading it through every call, register it once with+[process-wide defaults](api.md#process-wide-defaults):++```python+from laya import hooks++hooks.set_default_hooks(hooks=[Tracer()])+```## The mental modeldiff --git a/docs/hooks/patterns.md b/docs/hooks/patterns.mdindex d3942852..61e8655d 100644--- a/docs/hooks/patterns.md+++ b/docs/hooks/patterns.md@@ -15,6 +15,7 @@ that hold up in production and the ones that bite.- [Multi-tenant context](#multi-tenant-context)- [Composition](#composition)- [Scoped instrumentation](#scoped-instrumentation)+ - [Process-wide instrumentation](#process-wide-instrumentation)- [Token-budget shaping](#token-budget-shaping)- [Anti-patterns](#anti-patterns)- [Blocking work](#blocking-work)@@ -230,6 +231,24 @@ with agent.hooks_installed(DebugDump()):`add_hook`/`remove_hook` do the same without a block, for a tracer that lives as long as theprocess.+### Process-wide instrumentation++A tracer or metrics hook that every decision should see can be registered once, instead of being+passed to each `Agent` and `Router`. Defaults run before the instance and per-call hooks.++```python+from laya import BaseHook, hooks++class Metrics(BaseHook):+ def on_predict_end(self, ctx):+ record(ctx.model, ctx.elapsed_ms)++hooks.set_default_hooks(hooks=[Metrics()])+```++This is global state, so scope it deliberately: set it once at startup, and `clear_default_hooks()`+in tests so one test cannot leak a hook into the next.+### Token-budget shapingA start hook can raise the token budget for one call, for example when a question has manydiff --git a/laya/__init__.py b/laya/__init__.pyindex cb41e415..facc843d 100644--- a/laya/__init__.py+++ b/laya/__init__.py@@ -1,7 +1,7 @@"""Laya: Fast, non-autoregressive System 1 decision engine with calibrated probabilities."""from .email import clean_email_body, email_state-from .hooks import Hook, PredictContext, PredictHook+from .hooks import BaseHook, Hook, PredictContext, PredictHookfrom .lang import analyse as detect_languagefrom .lang import detect_script, is_englishfrom .presets import (@@ -91,5 +91,6 @@ __all__ = ["PredictContext","PredictHook","Hook",+ "BaseHook","__version__",]diff --git a/laya/agent.py b/laya/agent.pyindex 19520f5f..4182f3ed 100644--- a/laya/agent.py+++ b/laya/agent.py@@ -25,7 +25,7 @@ from .common import (render_options,temp_bucket,)-from .hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooks+from .hooks import HookRegistry, PredictContext, aggregate_usage, compose_hooks, dispatch, normalise_hooksdef _fix_tokenizer_config(path: str):@@ -634,7 +634,7 @@ class Agent(HookRegistry):A list of per-state result dicts, each identical in shape to `system_one`'s output andaligned with `states` by index."""- active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)+ active = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end)raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)ctx = PredictContext(states=states, questions=questions, model=self.model_id, agent=self,max_len=max_len, head_max_len=head_max_len)diff --git a/laya/hooks.py b/laya/hooks.pyindex 96e3e3f0..27f419fc 100644--- a/laya/hooks.py+++ b/laya/hooks.py@@ -58,6 +58,32 @@ class Hook(Protocol):def on_error(self, ctx: PredictContext) -> None: ...+class BaseHook:+ """No-op base class: subclass it and override only the events you need.++ `Hook` is the structural protocol; `BaseHook` is the concrete convenience when you would+ rather subclass than implement methods by shape. Every method does nothing by default.+ """++ def on_predict_start(self, ctx: PredictContext) -> None:+ pass++ def on_predict_end(self, ctx: PredictContext) -> None:+ pass++ def on_route(self, ctx: PredictContext) -> None:+ pass++ def on_load(self, ctx: PredictContext) -> None:+ pass++ def on_evict(self, ctx: PredictContext) -> None:+ pass++ def on_error(self, ctx: PredictContext) -> None:+ pass++PredictHook = Callable[[PredictContext], None]HookArg = Union[Hook, Sequence[Hook], None]PredictHookArg = Union[PredictHook, Sequence[PredictHook], None]@@ -146,6 +172,49 @@ def normalise_hooks(return result+_DEFAULT_HOOKS: List[Any] = []+_DEFAULT_HOOKS_LOCK = threading.Lock()+++def default_hooks() -> List[Any]:+ """The process-wide hooks, a copy, in order. Empty unless set via `set_default_hooks`."""+ with _DEFAULT_HOOKS_LOCK:+ return list(_DEFAULT_HOOKS)+++def set_default_hooks(hooks=None, on_predict_start=None, on_predict_end=None) -> None:+ """Replace the process-wide default hooks.++ Defaults run before installed and per-call hooks for every `Agent`, `Router` and+ `ONNXAgent` in the process, so a tracer or metrics hook does not have to be threaded+ through every construction. Accepts the same arguments as the `hooks=` parameter.+ """+ normalised = normalise_hooks(hooks, on_predict_start, on_predict_end)+ with _DEFAULT_HOOKS_LOCK:+ _DEFAULT_HOOKS[:] = normalised+++def add_default_hook(hook: HookArg) -> None:+ """Append one hook or a sequence to the process-wide defaults."""+ normalised = normalise_hooks(hook)+ with _DEFAULT_HOOKS_LOCK:+ _DEFAULT_HOOKS.extend(normalised)+++def clear_default_hooks() -> None:+ """Remove every process-wide default hook."""+ with _DEFAULT_HOOKS_LOCK:+ _DEFAULT_HOOKS.clear()+++def compose_hooks(installed, hooks=None, on_predict_start=None, on_predict_end=None) -> List[Any]:+ """Effective hook list for one call: defaults, then installed, then per-call hooks.++ Reads the process-wide defaults at call time, so hooks set after construction still apply.+ """+ return default_hooks() + list(installed) + normalise_hooks(hooks, on_predict_start, on_predict_end)++class HookRegistry:"""Mixin giving a runtime-mutable hook list.diff --git a/laya/onnx_agent.py b/laya/onnx_agent.pyindex 8664bd25..8d363df7 100644--- a/laya/onnx_agent.py+++ b/laya/onnx_agent.py@@ -7,7 +7,7 @@ from typing import Any, Dict, Optional, Unionimport numpy as np-from laya.hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooks+from laya.hooks import HookRegistry, PredictContext, aggregate_usage, compose_hooks, dispatch, normalise_hooksfrom laya.common import (QTYPES,build_sequence,@@ -160,7 +160,7 @@ class ONNXAgent(HookRegistry):max_len: Optional[int] = None,head_max_len: Optional[int] = None) -> Dict[str, Any]:"""Evaluate typed questions, running any opt-in hooks around the inference."""- active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)+ active = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end)raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)ctx = PredictContext(states=[state], questions=questions, model=self.model_id, agent=self,max_len=max_len, head_max_len=head_max_len)diff --git a/laya/router.py b/laya/router.pyindex 837d6fe0..10dcb97b 100644--- a/laya/router.py+++ b/laya/router.py@@ -35,7 +35,7 @@ import timefrom collections.abc import Sequence as SequenceABCfrom typing import Any, Dict, List, Optional, Sequence, Union-from .hooks import HookRegistry, PredictContext, aggregate_usage, dispatch, normalise_hooks+from .hooks import HookRegistry, PredictContext, aggregate_usage, compose_hooks, dispatch, normalise_hooksfrom .lang import analyse# The hub repo bundles all three checkpoints; only the requested subfolder is downloaded.@@ -263,7 +263,7 @@ class Router(HookRegistry):evicted = self._evict_locked()# Lifecycle hooks fire after the lock is released, so a hook can safely call the Router.self._dispatch_lifecycle("on_evict", evicted)- dispatch(self.hooks, "on_load",+ dispatch(compose_hooks(self.hooks), "on_load",PredictContext(states=[], questions={}, model=key, agent=agent, router=self),raise_errors=self.hooks_raise, lock=self._hooks_lock)return agent@@ -306,7 +306,7 @@ class Router(HookRegistry):def _dispatch_lifecycle(self, event: str, names: List[str]) -> None:for name in names:- dispatch(self.hooks, event,+ dispatch(compose_hooks(self.hooks), event,PredictContext(states=[], questions={}, model=name, router=self),raise_errors=self.hooks_raise, lock=self._hooks_lock)@@ -404,7 +404,7 @@ class Router(HookRegistry):"""decision = self._route(state, questions, model=model, task=task, lang=lang, lang_guess=lang_guess)raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)- active = list(self.hooks) + normalise_hooks(hooks)+ active = compose_hooks(self.hooks, hooks)ctx = PredictContext(states=[state], questions=questions or {}, decision=decision, router=self)dispatch(active, "on_route", ctx, raise_errors=raise_errors, lock=self._hooks_lock)return ctx.decision@@ -522,7 +522,7 @@ class Router(HookRegistry):and see `ctx.decision`; see `laya.hooks`. `max_len` / `head_max_len` override the agenttoken budget for this call (a start hook may set `ctx.max_len` / `ctx.head_max_len`)."""- active = list(self.hooks) + normalise_hooks(hooks, on_predict_start, on_predict_end)+ active = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end)raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)# Per-call hooks apply to the whole call, including on_route inside route().diff --git a/tests/test_hooks.py b/tests/test_hooks.pyindex 5ce598cc..560354d3 100644--- a/tests/test_hooks.py+++ b/tests/test_hooks.py@@ -890,6 +890,81 @@ check("onnx/skip short-circuits",{"model": "cached"})+# --------------------------------------------------------------- BaseHook+from laya import BaseHook # noqa: E402+++class OnlyEnd(BaseHook):+ def __init__(self, log, tag):+ self.log = log+ self.tag = tag++ def on_predict_end(self, ctx):+ self.log.append(self.tag)+++log = []+f = make_fake()+f.add_hook(OnlyEnd(log, "end"))+f.predict_batch(["s0"], QUESTIONS)+check("BaseHook/overridden event fires", log, ["end"])++f = make_fake()+f.add_hook(BaseHook()) # every method is a no-op+f.predict_batch(["s0"], QUESTIONS)+check("BaseHook/no-op instance is harmless", f._forward_calls, [2])+++# --------------------------------------------------------------- process-wide defaults+from laya import hooks as _hooks # noqa: E402++log = []+_hooks.set_default_hooks(on_predict_end=lambda ctx: log.append("default"))+try:+ f = make_fake()+ f.add_hook(Tag(log, "installed"))+ f.predict_batch(["s0"], QUESTIONS, on_predict_end=lambda ctx: log.append("percall"))+ check("defaults/run before installed and per-call", log, ["default", "installed", "percall"])+finally:+ _hooks.clear_default_hooks()+check("defaults/clear empties the registry", _hooks.default_hooks(), [])++log = []+_hooks.add_default_hook(Tag(log, "a"))+_hooks.add_default_hook(Tag(log, "b"))+try:+ f = make_fake()+ f.predict_batch(["s0"], QUESTIONS)+ check("defaults/add in order", log, ["a", "b"])+finally:+ _hooks.clear_default_hooks()+++class LifeDefaults(BaseHook):+ def __init__(self):+ self.events = []++ def on_load(self, ctx):+ self.events.append(("load", ctx.model))++ def on_evict(self, ctx):+ self.events.append(("evict", ctx.model))+++ld = LifeDefaults()+_hooks.set_default_hooks(hooks=[ld])+try:+ _agent_mod.Agent = BuiltAgent+ r = Router(max_loaded=1)+ r.load("english")+ r.load("multilingual") # evicts english+finally:+ _agent_mod.Agent = real_agent+ _hooks.clear_default_hooks()+check("defaults/cover router lifecycle", ld.events,+ [("load", "english"), ("evict", "english"), ("load", "multilingual")])++# --------------------------------------------------------------- reportprint("\n%d passed, %d failed" % (len(PASS), len(FAIL)))for f_ in FAIL:diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.pyindex fd283504..7b66d6ea 100644--- a/tests/test_hooks_api.py+++ b/tests/test_hooks_api.py@@ -14,7 +14,7 @@ import syssys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))import laya # noqa: E402-from laya import Agent, PredictContext, PredictHook, Router, load # noqa: E402+from laya import Agent, BaseHook, PredictContext, PredictHook, Router, load # noqa: E402from laya.hooks import HOOK_EVENTS, Hook # noqa: E402from laya.onnx_agent import ONNXAgent # noqa: E402@@ -119,10 +119,20 @@ for event in HOOK_EVENTS:check_true("PredictHook is callable-typed", callable(PredictHook))# --------------------------------------------------------------- exports-for name in ("PredictContext", "PredictHook", "Hook"):+for name in ("PredictContext", "PredictHook", "Hook", "BaseHook"):check_true("__all__/%s" % name, name in laya.__all__)check_true("laya.%s exists" % name, hasattr(laya, name))+# BaseHook is the concrete no-op base class; all six events exist and are callable.+for event in HOOK_EVENTS:+ check_true("BaseHook/%s callable" % event, callable(getattr(BaseHook, event, None)))++# process-wide default registry lives in laya.hooks (not the top level)+for helper in ("default_hooks", "set_default_hooks", "add_default_hook", "clear_default_hooks",+ "compose_hooks"):+ check_true("laya.hooks/%s exists" % helper, callable(getattr(__import__("laya.hooks", fromlist=[helper]), helper, None)))+check_true("defaults/not exported at top level", not hasattr(laya, "set_default_hooks"))+# --------------------------------------------------------------- class defaultsfor label, cls in (("Agent", Agent), ("Router", Router), ("ONNXAgent", ONNXAgent)):check("%s/hooks default" % label, cls.hooks, ())