Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(serve): review fixes -- exact container types, list coverage, and three wrong claims
Adeshina Lasisi90ba149e6959
diff --git a/examples/server.py b/examples/server.pyindex 23bc3ec4..27371bc0 100644--- a/examples/server.py+++ b/examples/server.py@@ -293,9 +293,12 @@ def _check_request_limits(state: Any, questions: Dict[str, Any]) -> None:Laya encodes the state once per question, so cost is questions x state size,collated into one tensor, and a choice or score question adds one sequence peroption against a shared head budget. The state length comes from laya.serve's own- `_state_length`, so the two surfaces cannot disagree about what a state measures,- and the option counts exactly as it counts them, over `choice` and `score`- criteria only.+ `_state_length` by `getattr`, so the two surfaces cannot disagree about what a state+ *measures*, and the option counts exactly as it counts them, over `choice` and+ `score` criteria only. They can still differ in which limit they report first: this+ handler checks state size before the option budgets and `laya.serve` checks it after,+ so a request violating both gets a different `detail` from each. That ordering is+ unchanged here.Checked here rather than declared as pydantic constraints on the request models,for two reasons: a `Field(max_length=...)` violation is reported as 422 wherediff --git a/laya/serve.py b/laya/serve.pyindex 1ec74655..0d4e01f9 100644--- a/laya/serve.py+++ b/laya/serve.py@@ -199,9 +199,11 @@ def _resolve_port() -> int:return port-#: Values examined by `_provably_over_state_limit` before it gives up. A constant, so the probe can-#: never cost more than one: it exists to catch an oversized value for free, not to replace the-#: encoder.+#: Values the walk in `_state_length_lower_bound_over` will examine before giving up. This bounds the+#: values *inspected*, not the total work: `stack.extend` pushes a container's elements without+#: charging them against the budget, so a 2 MiB body of many small values costs ~0.43 ms of walking+#: before falling through to the dump it was going to pay anyway (~6% on top). An ACCEPTED state is+#: capped at MAX_STATE_CHARS, where the walk is ~0.01 ms._STATE_PROBE_VALUES = 64@@ -217,6 +219,10 @@ def _state_length_lower_bound_over(state: Any, cap: int) -> int:60 012-character state. That is the same fabricated-count flaw this gate replaces, and anearlier revision of this branch reintroduced it.+ One consequence worth knowing: the number in a 413 is therefore the exact serialized length when+ the encoder ran, and this understated bound when it did not. Both are truthful and neither+ overstates, but a client trimming exactly the difference the message names can still be refused.+The bound: the raw lengths of the string values are a *lower* bound on the JSON length. Escapingmaps each character to one or more characters and so never shortens; keys, separators, bracketsand quotes only add; numbers, booleans and nulls contribute at least one character each and are@@ -240,12 +246,20 @@ def _state_length_lower_bound_over(state: Any, cap: int) -> int:total += len(item)if total > cap:return total- elif isinstance(item, dict):+ elif type(item) is dict:# `.values()` and `extend` stay at C level; the keys are ignored, which keeps this a# lower bound. Iterating items in Python here made a 2000-key state 1.66x slower.stack.extend(item.values())- elif isinstance(item, (list, tuple)):+ elif type(item) is list or type(item) is tuple:stack.extend(item)+ else:+ # EXACT types only, deliberately. A `dict` subclass may override `values()` while+ # `json.dumps` reads the real items, which would let this "lower bound" exceed the true+ # length -- measured: a 13-character state refused as `60000 > 50000`. Anything else,+ # including a subclass, a set or a cycle, is handed to the encoder, which decides the+ # size or raises and becomes the 400 below. `json.loads` only ever builds exact types,+ # so no HTTP request takes this branch.+ return 0return 0@@ -272,7 +286,7 @@ def _state_length(state: Any) -> int:Cost. At the gate's own ceiling the dump is 0.133 ms against 0.080 ms for the `str()` itreplaces -- orders of magnitude under the forward pass that state then gets, so the serializedtext is not worth threading through `predict()` to be encoded only once. A state large enough- for that difference to matter never reaches the dump: `_provably_over_state_limit` refuses it+ for that difference to matter never reaches the dump: `_state_length_lower_bound_over` refuses itfirst, which is what keeps an oversized body from costing more to reject than it did upstream.(Best of 200 after 20 warm-up iterations, CPython 3.12 on a 10-core arm64 laptop.)"""diff --git a/tests/test_example_server_limits.py b/tests/test_example_server_limits.pyindex a1cc7845..605ac577 100644--- a/tests/test_example_server_limits.py+++ b/tests/test_example_server_limits.py@@ -266,8 +266,8 @@ def main():# other `str(state)`. It cannot catch them being wrong together: `examples/server.py` binds# `_state_length` from `laya.serve` by `getattr`, so a regression inside that helper moves# both and parity stays green. Measured: regressing `_state_length` fails 0 of these parity- # checks and 4 of the absolute ones below. Those are the real coverage -- do not prune them- # as redundant.+ # checks and both of the absolute ones below. Those are the real coverage -- do not prune+ # them as redundant.("a state whose repr is half its JSON", {"body": '"' * 49988}, one),]for label, st, qs in parity_cases:@@ -277,10 +277,15 @@ def main():# Parity alone cannot see a bug both surfaces share, and `getattr` guarantees they share one:# with `_state_length` measuring `str(state)` again, both agree on accepting a state that# serializes to 99 988 characters and every parity check above stays green (measured: 0 of them- # fail, 4 of these do). So the verdict itself is asserted, not just the agreement.+ # fail, both of these do). So the verdict itself is asserted, not just the agreement.quote_heavy = {"body": '"' * 49988}- assert len(str(quote_heavy)) <= MAX_STATE_CHARS, "the fixture must pass a str()-based gate"- assert len(json.dumps(quote_heavy, ensure_ascii=False)) > MAX_STATE_CHARS, "and fail a real one"+ # Recorded through `ok` like everything else in this file: a bare `assert` here would raise out+ # of `main()` and abandon the ~30 checks that follow instead of recording one failure.+ ok("the quote-heavy fixture passes a str()-based gate",+ len(str(quote_heavy)) <= MAX_STATE_CHARS, len(str(quote_heavy)))+ ok("the quote-heavy fixture fails a serialization-based gate",+ len(json.dumps(quote_heavy, ensure_ascii=False)) > MAX_STATE_CHARS,+ len(json.dumps(quote_heavy, ensure_ascii=False)))for who, verdict in (("laya.serve", serve_verdict(quote_heavy, one)),("the demo server", demo_verdict(quote_heavy, one))):ok("%s refuses a state whose JSON is twice its repr" % who, verdict == "refused", verdict)diff --git a/tests/test_serve.py b/tests/test_serve.pyindex 89b582fd..bbdc6ea6 100644--- a/tests/test_serve.py+++ b/tests/test_serve.py@@ -1097,22 +1097,31 @@ def test_an_oversized_integer_state_never_reaches_the_gate(monkeypatch, default_def test_the_probe_refuses_an_oversized_value_without_serializing_the_state(monkeypatch):"""An oversized state is refused without the 2 MiB dump whose result cannot change the verdict.- This is the check that goes red if `_provably_over_state_limit` is removed: without it the gate+ This is the check that goes red if `_state_length_lower_bound_over` is removed: without it the gatestill answers 413, so the *absence of the dump* is the only observable difference."""import laya.serve as serve_modcalls = []- real_dumps = serve_mod.json.dumps+ real_dumps = json.dumpsdef counting_dumps(obj, **kw):calls.append(kw)return real_dumps(obj, **kw)- monkeypatch.setattr(serve_mod.json, "dumps", counting_dumps)- state = {"body": "a" * (2 * 1024 * 1024)}- assert serve_mod._state_length(state) > serve_mod.MAX_STATE_CHARS- assert calls == [], "the state was serialized although one value already exceeds the cap"+ # Scoped to the module's own binding: `serve_mod.json` IS the stdlib module, so patching an+ # attribute on it would replace `json.dumps` for the whole process for the test's duration.+ monkeypatch.setattr(serve_mod, "json", SimpleNamespace(dumps=counting_dumps))+ # F2: a conversation LIST is this library's dominant state shape, and the probe's list branch+ # can be deleted with every other check still green unless one of these is list-shaped.+ for state in ({"body": "a" * (2 * 1024 * 1024)},+ ["z" * 60000],+ [{"role": "user", "content": "x" * 70000}],+ ({"role": "user", "content": "x" * 70000},)):+ calls.clear()+ assert serve_mod._state_length(state) > serve_mod.MAX_STATE_CHARS, state.__class__.__name__+ assert calls == [], "a %s state was serialized although one value already exceeds the cap" \+ % type(state).__name__def test_the_probe_verdict_always_matches_the_exact_measurement():@@ -1156,11 +1165,13 @@ def test_the_early_refusal_reports_a_number_it_measured():"""import laya.serve as serve_modcap = serve_mod.MAX_STATE_CHARS- for state in ({"body": "x" * 60000}, {"body": "y" * (2 * 1024 * 1024)}, {"a": ["z" * 80000]}):+ # `cap + 1` is NOT the tell: {"body": "a" * 50001} legitimately sums to exactly that. The+ # property that distinguishes a measured bound from a fabricated one is `cap < reported <= exact`.+ for state in ({"body": "x" * 60000}, {"body": "y" * (2 * 1024 * 1024)}, {"a": ["z" * 80000]},+ {"body": "a" * (cap + 1)}, ["w" * 60000]):exact = len(json.dumps(state, ensure_ascii=False))reported = serve_mod._state_length(state)assert reported > cap, reported- assert reported != cap + 1, "the refusal reported cap+1, a number nothing measured"assert reported <= exact, \"reported %d for a state of %d characters -- a 413 must never overstate" % (reported, exact)@@ -1176,3 +1187,21 @@ def test_the_probe_is_bounded_and_falls_through_rather_than_guessing():wide = {("k%06d" % i): "x" for i in range(200000)}assert serve_mod._state_length_lower_bound_over(wide, serve_mod.MAX_STATE_CHARS) == 0assert serve_mod._state_length(wide) == len(json.dumps(wide, ensure_ascii=False))+++def test_the_probe_walks_exact_container_types_only():+ """A subclass may override `values()` while `json.dumps` reads the real items, so walking one+ would let the "lower bound" exceed the true length. Measured before this was narrowed: a+ 13-character state refused as `60000 > 50000`. No HTTP request can reach it -- `json.loads`+ builds exact types -- but the 400 branch exists for in-process callers.+ """+ import laya.serve as serve_mod++ class Lying(dict):+ def values(self):+ return ["x" * 60000]++ state = Lying(x="tiny")+ exact = len(json.dumps(state, ensure_ascii=False))+ assert serve_mod._state_length_lower_bound_over(state, serve_mod.MAX_STATE_CHARS) == 0+ assert serve_mod._state_length(state) == exact == 13