Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(hooks): the empty batch passes predict_batch's usage guard
Aashish185a6ef98c80
diff --git a/docs/hooks/lifecycle.md b/docs/hooks/lifecycle.mdindex 8d95a9e8..e2aff881 100644--- a/docs/hooks/lifecycle.md+++ b/docs/hooks/lifecycle.md@@ -48,7 +48,9 @@ predict_batch(states, questions, batch_size=..., hooks=..., ...)│ ││ finally: ││ ctx.elapsed_ms = now - ctx.started_at │- │ if ctx.results: ctx.usage = aggregate_usage(...)│+ │ if ctx.results is not None: │+ │ ctx.usage = aggregate_usage(...) │+ │ apply_confidence_gate(..., min_confidence) ││ on_predict_end ─────────────────────────────────┘│└─ return ctx.results@@ -57,6 +59,14 @@ predict_batch(states, questions, batch_size=..., hooks=..., ...)The same `ctx` object flows through start, error and end, so `run_id` correlates them and an endhook can read `ctx.error`.+The `finally` guard is `ctx.results is not None`, not truthiness, and the empty batch is the only+case where they differ: `predict_batch([])` sets `ctx.results = []` (`laya/agent.py::Agent.predict_batch`),+so the guard passes, `ctx.usage` becomes `{"input_tokens": 0, "output_tokens": 0}`, and the confidence+gate still runs over the (empty) result list. A call that raised before any result leaves+`ctx.results` and `ctx.usage` both `None`. So for an end hook `usage is None` means "this call+produced nothing", and it is not the same signal as "this call was empty" -- an audit hook that+treats the second as the first reports every empty batch as a failure.+## Agent.system_one / predict```@@ -216,14 +226,17 @@ See [`examples/hooks/cache.py`](../../examples/hooks/cache.py) for a working cacHooks still fire so audit sees every call:-| input | `ctx.results` at end |-|---|---|-| `predict_batch([])` | `[]` |-| `predict_batch(states, {})` (no questions) | one empty-answer payload per state |-| `system_one(state, {})` | a single empty-answer payload |+| input | `ctx.results` at end | `ctx.usage` at end |+|---|---|---|+| `predict_batch([])` | `[]` | `{"input_tokens": 0, "output_tokens": 0}` |+| `predict_batch(states, {})` (no questions) | one empty-answer payload per state | zeros, summed over those payloads |+| `system_one(state, {})` | a single empty-answer payload | `{"input_tokens": 0, "output_tokens": 0}` |No tokenization or forward pass happens in these cases, but `on_predict_start` and-`on_predict_end` run.+`on_predict_end` run. The third `usage` column is the point of the table: every one of these calls+leaves an end hook with a *real* usage dict, because the guard that fills it is+`ctx.results is not None` and `[]` passes it. Only a call that raised before producing results+leaves `ctx.usage` as `None`.## Ordering rulesdiff --git a/tests/test_batch.py b/tests/test_batch.pyindex b96f1b9d..5ce37075 100644--- a/tests/test_batch.py+++ b/tests/test_batch.py@@ -11,6 +11,7 @@ These tests need no model weights. They cover:"""import inspectimport os+import reimport sysfrom contextlib import contextmanagerfrom types import SimpleNamespace@@ -206,6 +207,96 @@ check("length/error hook receives failure once", [str(e) for e in errors], ["forf = make_fake()check("orchestration/empty states -> empty list", f.predict_batch([], QUESTIONS), [])+# What the end hook of that same call sees. `predict_batch`'s finally block guards the usage+# aggregation with `ctx.results is not None`, not with truthiness, and `[]` is the one value where+# the two disagree: truthiness would leave `ctx.usage` as None on an empty batch, which is exactly+# the value a call that raised before any result leaves behind. docs/hooks/lifecycle.md's+# "Empty inputs" table states the difference to readers who write audit hooks against `ctx.usage`,+# so it is pinned here against the real method.+end_seen = []+++class WatchEnd:+ def on_predict_end(self, ctx):+ end_seen.append((ctx.results, ctx.usage))+++f = make_fake()+f.predict_batch([], QUESTIONS, hooks=[WatchEnd()])+check("orchestration/empty batch: the end hook still runs, once", len(end_seen), 1)+check("orchestration/empty batch: results [] but usage summed, not None", end_seen[0],+ ([], {"input_tokens": 0, "output_tokens": 0}))++end_seen.clear()+f = make_fake()+f.predict_batch(["s0", "s1"], {}, hooks=[WatchEnd()])+check("orchestration/no questions: usage is the sum of the empty payloads", end_seen[0][1],+ {"input_tokens": 0, "output_tokens": 0})++end_seen.clear()+f = make_fake()+f._forward = fail_forward+check_raises("orchestration/forward error propagates on one state", RuntimeError,+ lambda: f.predict_batch(["s0"], QUESTIONS, hooks=[WatchEnd()]))+check("orchestration/a failed call is the only one that leaves usage None", end_seen,+ [(None, None)])++# The same guard read from both sides. lifecycle.md transcribes this `finally` block into its+# diagram, so the transcription has to track the code -- a guard that changes shape in one of the+# two is a page that now lies about when `ctx.usage` is None. Source inspection is already how this+# file's contract section pins `predict_batch` (it reads `Agent.predict is Agent.system_one` off the+# class); the doc side is a literal pull out of the fenced diagram. Box-drawing characters and the+# diagram's own arrows are why both sides go through one regex instead of a line compare.+_REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))+++def _finally_guard(text):+ """The condition the first `if` after `finally:` tests, as written."""+ tail = text.split("finally:", 1)+ if len(tail) < 2:+ return None+ found = re.search(r"if (ctx\.results[^:]*)", tail[1])+ return found.group(1).strip() if found else None+++_code_text = inspect.getsource(Agent.predict_batch)+with open(os.path.join(_REPO, "docs", "hooks", "lifecycle.md"), encoding="utf-8") as _life:+ _doc_text = _life.read()+check("guard/predict_batch has exactly one finally block to transcribe",+ _code_text.count("finally:"), 1)+check_true("guard/lifecycle.md's diagram has a finally block", "finally:" in _doc_text)+# The diagram is the text between the section heading and its closing fence.+_diagram = _doc_text.split("## Agent.predict_batch", 1)[1].split("```", 2)[1]+_GUARD_CODE = _finally_guard(_code_text)+_GUARD_DOC = _finally_guard(_diagram)+check("guard/the diagram tests what the code tests", _GUARD_DOC, _GUARD_CODE)+check_true("guard/and that guard is the identity test, not truthiness",+ _GUARD_CODE == "ctx.results is not None", repr(_GUARD_CODE))+++def _stmts_after_guard(text):+ """The call names the two statements that follow that guard, in source order.++ Scoped after `finally:` for the same reason `_finally_guard` is: the body has an earlier+ `if ctx.results is None:` on the cache-hit branch, and transcribing that one would be a+ different claim.+ """+ tail = text.split("finally:", 1)+ if len(tail) < 2:+ return []+ lines = tail[1].splitlines()+ for n, ln in enumerate(lines):+ if "if ctx.results" in ln:+ return re.findall(r"\b([a-z_]+)\(", "\n".join(lines[n + 1:n + 3]))+ return []+++_code_stmts = _stmts_after_guard(_code_text)+check("guard/the guard runs two statements worth transcribing", _code_stmts,+ ["aggregate_usage", "apply_confidence_gate"])+check("guard/and the diagram transcribes both of them",+ _stmts_after_guard(_diagram), _code_stmts)+f = make_fake()res = f.predict_batch(["s0", "s1", "s2"], QUESTIONS)check("orchestration/one result per state", len(res), 3)