Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(serve): the LAYA_DEFAULT_MODEL default column said english
Aashish4fb39558f981
diff --git a/docs/http-api.md b/docs/http-api.mdindex a93644ab..2b9e1068 100644--- a/docs/http-api.md+++ b/docs/http-api.md@@ -36,7 +36,7 @@ Everything is environment variables, so one image serves a laptop dev run and a| `LAYA_IDLE_UNLOAD_SECONDS` | unload resident checkpoints after this many idle seconds; the next request loads its checkpoint again. Zero disables unloading | `0` || `LAYA_MAX_LOADED` | checkpoints kept resident at once; a cap below what routing chooses rebuilds one per switch, and `preload()` raises it to hold whatever it builds | `2` || `LAYA_MAX_TOKEN_BUDGET` | server-side ceiling on the per-request `max_len` and `head_max_len` overrides; a larger value is a `422`. Unparseable or non-positive input logs a warning and falls back | `8192` |-| `LAYA_DEFAULT_MODEL` | checkpoint a state with no language evidence falls back to; aliases such as `ml` resolve the way core resolves them, and an unresolvable name stops the server at startup | `english` |+| `LAYA_DEFAULT_MODEL` | checkpoint a state with no language evidence falls back to; aliases such as `ml` resolve the way core resolves them, and an unresolvable name stops the server at startup | `multilingual` || `LAYA_EXTRA_MODELS` | JSON object `{name: source}` registering extra checkpoints beside the bundled ones: a Hub repo id or local checkpoint directory as a string, or a `["repo", "subfolder"]` pair. Names get the same `model=` pin a built-in does; a malformed value or bad name stops the server at startup | none || `LAYA_API_KEY` | if set, require `Authorization: Bearer <key>` | none || `LAYA_LOG_LEVEL` | uvicorn log level | `info` |diff --git a/laya/serve.py b/laya/serve.pyindex fd3da64d..8637ac0e 100644--- a/laya/serve.py+++ b/laya/serve.py@@ -30,7 +30,7 @@ env var meaning defaulKeep <= physical cores; oversubscribing the default)logical/hyperthread count is a large regression.``LAYA_AUTO_TASK`` auto-route to the typed-decisions checkpoint 0-``LAYA_DEFAULT_MODEL`` fallback checkpoint when a state carries no (english)+``LAYA_DEFAULT_MODEL`` fallback checkpoint when a state carries no (multilingual)language evidence; aliases like ml work``LAYA_MAX_LOADED`` checkpoints kept resident at once. Below what 2routing can choose, one reloads per switch.diff --git a/tests/test_env_docs.py b/tests/test_env_docs.pyindex a83d8771..ce2fb8bd 100644--- a/tests/test_env_docs.py+++ b/tests/test_env_docs.py@@ -27,6 +27,18 @@ the moment an ``os.environ`` read is added in another, so this derives the set itranscribing it: it walks ``laya/`` with ``ast`` and compares what it finds against whatthe docs mention, in both directions.+A default column goes stale the same way and is harder to see, because the row still reads+correctly. Both surfaces that document the package's own defaults say ``LAYA_DEFAULT_MODEL``+falls back to ``english``. Since 0.4.0 it falls back to ``multilingual``: the breaking routing+change moved ``Router(default=...)`` and four pages with it, but not ``docs/http-api.md`` nor the+table in ``laya/serve.py``'s docstring, and the server leaves the keyword out when the variable is+unset precisely so that the value "cannot drift" from ``Router``'s own. ``tests/test_serve.py``+already pins the resolved ``router.default``; it never reads a table, so nothing compared the two+columns to the code they describe. The operator who trusts the stale cell is asking for the+checkpoint that change exists to stop -- off English it collapses while staying confident, so no+downstream ``min_confidence`` gate catches it. So the default column is derived too, from three+read shapes, and held on the two surfaces that describe the package rather than a container.+Why the whole string-constant sweep rather than only ``os.environ.get("LAYA_...")``: thenarrow form misses two of the fourteen names the package actually consumes, because``_env_bool("LAYA_AUTO_TASK", ...)`` and ``os.environ.get(_ENV_KEY)`` hide the literal from@@ -256,6 +268,312 @@ def offered(text: str, var: str) -> Set[str]:return out+# ------------------------------------------------------------------ a documented default+# Two cells promised ``english`` for ``LAYA_DEFAULT_MODEL`` while ``Router(default="multilingual")``+# has answered those requests since 0.4.0: the breaking routing change moved the fallback and the+# two environment tables were not part of it. Every other row on both pages is right, so this is+# not a page nobody reads -- it is a page that reads well and went stale, which is the failure a+# prose review cannot catch and a comparison against the code can. The comparison derives the+# code's answer from three read shapes rather than transcribing a table of defaults here:+#+# R1 the read site carries it: ``os.environ.get("LAYA_X", D)`` and ``_env_bool("LAYA_X", D)``.+# R2 a resolver reads with no literal default and answers the unset path with one, found through+# the guard on the variable that read assigned (`raw = ...` then `if not raw: return DEFAULT_X`).+# R3 a resolver sends *nothing* when unset -- it returns ``{}`` or ``None`` and the caller leaves+# the keyword out of ``Router(**options)`` -- so the answer is ``Router``'s own parameter+# default. The keyword comes from the one-key dict literal the resolver returns when the+# variable *is* set, or from the `options["key"] = value` line that forwards it.+#+# Only the two surfaces that document *the package's* defaults are held to this. ``docs/docker.md``+# and ``docs/cli-mcp.md`` document the Compose service and the stdio server, whose defaults are+# allowed to differ, which is the row in "what this cannot check" below.+ROUTER_CLASS = "Router"+PACKAGE_DEFAULT_SURFACES = ("laya/serve.py docstring", "docs/http-api.md")+TABLE_RULE = re.compile(r"[ =]+")+DEFAULT_GUTTER = 4+++def is_read(node):+ """``(name, default_node)`` when this call is a ``LAYA_*`` read, else None.++ ``_env_bool`` is in the set because ``LAYA_AUTO_TASK`` and ``LAYA_PRELOAD`` go through it, and+ the narrow ``os.environ`` matcher this file already reports as insufficient misses them.+ """+ if not (isinstance(node, ast.Call) and node.args):+ return None+ func = node.func+ environ = isinstance(func, ast.Attribute) and func.attr == "get" \+ and isinstance(func.value, ast.Attribute) and func.value.attr == "environ"+ if not (environ or (isinstance(func, ast.Name) and func.id == "_env_bool")):+ return None+ first = node.args[0]+ if not (isinstance(first, ast.Constant) and isinstance(first.value, str)):+ return None+ found = NAME.findall(first.value)+ if len(found) != 1:+ return None+ return found[0], (node.args[1] if len(node.args) > 1 else None)+++def own_nodes(func):+ """Nodes in this function's own body, in source order, nested functions left out.++ Nesting matters twice over: ``create_app`` would otherwise "read" every name its request+ handlers read and be judged ambiguous, and a caller's ``options["k"] = v`` line has to be seen+ after the ``v = F(...)`` line that gives it meaning.+ """+ out, stack = [], [func]+ while stack:+ node = stack.pop(0)+ out.append(node)+ for child in ast.iter_child_nodes(node):+ if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef, ast.Lambda)) \+ and child is not func:+ continue+ stack.append(child)+ return out+++def literal_of(node, consts):+ """A constant, or a module-level constant's value; ``ast.Constant`` covers None and "".++ Returns ``(found, value)`` rather than ``value`` because ``False`` and ``0`` are answers the+ documentation has to be able to agree with.+ """+ if isinstance(node, ast.Constant):+ return True, node.value+ if isinstance(node, ast.Name) and node.id in consts:+ return True, consts[node.id]+ return False, None+++def module_consts(tree):+ out = {}+ for stmt in tree.body:+ if isinstance(stmt, ast.Assign) and len(stmt.targets) == 1 \+ and isinstance(stmt.targets[0], ast.Name) \+ and isinstance(stmt.value, ast.Constant):+ out[stmt.targets[0].id] = stmt.value.value+ return out+++def read_names(func):+ return {site[0] for node in own_nodes(func) for site in [is_read(node)] if site}+++def router_param_defaults():+ """``Router.__init__``'s parameter defaults, as written in laya/router.py.++ ``None`` is dropped: it is the same "no opinion" the resolver expressed, so it cannot be the+ scalar a default column names.+ """+ tree = ast.parse(read(os.path.join(ROOT, "laya", "router.py")))+ for cls in [n for n in ast.walk(tree) if isinstance(n, ast.ClassDef) and n.name == ROUTER_CLASS]:+ for func in cls.body:+ if isinstance(func, ast.FunctionDef) and func.name == "__init__":+ names = [a.arg for a in func.args.args[1:]]+ defaults = func.args.defaults+ pad = len(names) - len(defaults)+ out = {}+ for i, dflt in enumerate(defaults):+ ok, value = literal_of(dflt, {})+ if ok and value is not None:+ out[names[pad + i]] = value+ return out+ return {}+++def forwarded_keyword(func, funcs, name):+ """The ``Router`` keyword a send-nothing resolver answers for, recovered from the caller.++ Two shapes, both real in laya/serve.py: ``options.update(F(...))`` where F returns a one-key+ dict literal, and ``value = F(...)`` followed by ``options["key"] = value``. The second one is+ only a keyword when the same function then does ``Router(**options)``, and which local that is+ comes out of the call rather than out of its name."""+ for node in own_nodes(func):+ if isinstance(node, ast.Return) and isinstance(node.value, ast.Dict) \+ and len(node.value.keys) == 1:+ key = node.value.keys[0]+ if isinstance(key, ast.Constant) and isinstance(key.value, str):+ return key.value+ for holder in funcs.values():+ if holder is func:+ continue+ nodes = own_nodes(holder)+ expanded = set()+ for node in nodes:+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) \+ and node.func.id == ROUTER_CLASS:+ expanded.update(kw.value.id for kw in node.keywords+ if kw.arg is None and isinstance(kw.value, ast.Name))+ called = {}+ for node in nodes:+ if isinstance(node, ast.Assign) and len(node.targets) == 1 \+ and isinstance(node.targets[0], ast.Name) \+ and isinstance(node.value, ast.Call) \+ and isinstance(node.value.func, ast.Name):+ inner = funcs.get(node.value.func.id)+ if inner is not None:+ called[node.targets[0].id] = inner+ for node in nodes:+ if not isinstance(node, ast.Assign) or len(node.targets) != 1:+ continue+ target = node.targets[0]+ if isinstance(target, ast.Subscript) and isinstance(target.value, ast.Name) \+ and target.value.id in expanded \+ and isinstance(target.slice, ast.Constant) \+ and isinstance(target.slice.value, str) \+ and isinstance(node.value, ast.Name):+ inner = called.get(node.value.id)+ if inner is not None and read_names(inner) == {name}:+ return target.slice.value+ return None+++def code_defaults():+ """What laya/serve.py does when a ``LAYA_*`` is unset.++ Returns ``(derived, unchecked, conflicts)``: the names answered with a single value, the names+ this file could not answer with the reason for each, and the names read with two different+ literal defaults at two read sites.+ """+ tree = ast.parse(read(os.path.join(PACKAGE, "serve.py")))+ consts = module_consts(tree)+ funcs = {}+ for func in [n for n in ast.walk(tree)+ if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef))]:+ funcs.setdefault(func.name, func)++ read_defaults: Dict[str, Set[object]] = {}+ for node in ast.walk(tree):+ site = is_read(node)+ if not site:+ continue+ ok, value = literal_of(site[1], consts) if site[1] is not None else (False, None)+ # "" is not a default a page can name; it is the read's way of saying "unset", and the+ # function's own guard decides what that means.+ if ok and value is not None and value != "":+ read_defaults.setdefault(site[0], set()).add(value)+ # Two reads of one name that disagree about what unset means is a code bug, and naming either+ # answer would let the page match the code and still be wrong, so it is failed on.+ conflicts = {name: sorted(repr(v) for v in values)+ for name, values in read_defaults.items() if len(values) > 1}+ derived = {name: next(iter(values)) for name, values in read_defaults.items()+ if len(values) == 1}++ unchecked = {}+ router = router_param_defaults()+ for name in sorted({n for func in funcs.values() for n in read_names(func)}):+ if name in derived or name in conflicts:+ continue+ candidates = [f for f in funcs.values() if name in read_names(f)]+ if len(candidates) != 1:+ unchecked[name] = "read by %d functions, so no single unset path to follow" % len(candidates)+ continue+ func = candidates[0]+ locals_ = set()+ for node in own_nodes(func):+ if isinstance(node, ast.Assign) and len(node.targets) == 1 \+ and isinstance(node.targets[0], ast.Name) \+ and any(site and site[0] == name+ for c in ast.walk(node.value)+ for site in [is_read(c)] if isinstance(c, ast.Call)):+ locals_.add(node.targets[0].id)+ unset = None+ for node in own_nodes(func):+ if not isinstance(node, ast.If):+ continue+ if not locals_ & {n.id for n in ast.walk(node.test) if isinstance(n, ast.Name)}:+ continue+ returns = [s for s in node.body if isinstance(s, ast.Return)]+ if returns:+ unset = returns[0].value+ break+ if unset is None:+ unchecked[name] = "no default at the read and no guard on the read's own variable"+ continue+ ok, value = literal_of(unset, consts)+ if ok and value is not None:+ derived[name] = value+ continue+ keyword = forwarded_keyword(func, funcs, name)+ if keyword is None:+ unchecked[name] = "the unset path sends nothing and no keyword names it"+ elif keyword not in router:+ unchecked[name] = "the unset path hands %r to %s, which defaults it to no value" \+ % (keyword, ROUTER_CLASS)+ else:+ derived[name] = router[keyword]+ return derived, unchecked, conflicts+++def docstring_default_table():+ """The environment table in laya/serve.py's module docstring, as ``{name: default cell}``.++ Hand-aligned, and its name column overflows the rule above it, so slicing at the rule's spans+ loses the long names. The default column is the one every row keeps to the right, so each row's+ text after its final run of two or more spaces is taken, and kept only when it starts in that+ column's gutter -- which is also what tells a wrapped meaning cell from a wrapped default.+ """+ doc = ast.get_docstring(ast.parse(read(os.path.join(PACKAGE, "serve.py")))) or ""+ lines = doc.splitlines()+ rules = [i for i, line in enumerate(lines) if TABLE_RULE.fullmatch(line) and "=" in line]+ if len(rules) < 3:+ return {}+ gutter = [m.start() for m in re.finditer(r"=+", lines[rules[0]])][-1] - DEFAULT_GUTTER+ rows, current = {}, None+ for line in lines[rules[1] + 1:rules[2]]:+ runs = list(re.finditer(r"\s{2,}", line))+ piece = line[runs[-1].end():].strip() if runs else ""+ on_column = bool(runs) and runs[-1].end() >= gutter and bool(piece)+ found = NAME.findall(line[:40])+ if len(found) == 1 and line.startswith("``"):+ current = found[0]+ rows[current] = [piece] if on_column else []+ elif current and on_column:+ rows[current].append(piece)+ return {name: " ".join(cells) for name, cells in rows.items()}+++def markdown_default_table(rel_path):+ """A markdown configuration table's default column, for rows whose first cell is one name."""+ rows = {}+ for line in read(os.path.join(ROOT, *rel_path.split("/"))).splitlines():+ if not line.startswith("|"):+ continue+ cells = [c.strip() for c in line.strip().strip("|").split("|")]+ if len(cells) < 2:+ continue+ found = NAME.findall(cells[0])+ if len(found) == 1 and cells[0].strip("`") == found[0]:+ rows[found[0]] = cells[-1]+ return rows+++def default_text(value):+ """How a scalar default reads in a table: booleans as the 1/0 the pages spell, 0.0 as 0."""+ if isinstance(value, bool):+ return "1" if value else "0"+ if isinstance(value, float) and value.is_integer():+ return str(int(value))+ return str(value)+++def default_cell(cell):+ """A default column's text, stripped of the markup and of the gloss some rows carry.++ ``(`` ``english`` ``)``, ``0 (off)`` and ``none`` all state one value; the parentheses are the+ table's own convention for "this is the value nobody set anything to", and the trailing pair on+ ``0 (off)`` is a note about that value rather than a second value. A row whose whole cell is+ parenthesised is therefore read inside the parentheses, and one that ends with them keeps the+ part before.+ """+ text = cell.strip().strip("`").strip()+ if re.fullmatch(r"\(.*\)", text):+ return text[1:-1].strip()+ return re.sub(r"\s*\([^)]*\)\s*$", "", text).strip()++def main() -> int:pkg = package_names()docs = markdown_names()@@ -365,11 +683,56 @@ def main() -> int:"claims compared: %s -- if laya/agent.py stops carrying a sentence that lists an ""accepted dtype, the prose site this sweep was written for moved" % sorted(claim_sites))+ # ------------------------------------------------- what a documented default says+ derived, unchecked, conflicts = code_defaults()+ check("defaults/no name is read with two different literal defaults", conflicts, {})+ surfaces = {"laya/serve.py docstring": docstring_default_table(),+ "docs/http-api.md": markdown_default_table("docs/http-api.md")}+ for name, value in sorted(derived.items()):+ for surface in PACKAGE_DEFAULT_SURFACES:+ table = surfaces[surface]+ if name not in table:+ continue+ check("defaults/%s's %s cell says what the code does" % (surface, name),+ default_cell(table[name]), default_text(value))+ # Non-vacuity by name. Without these the whole section could be standing on nothing: a table+ # that stopped parsing returns {}, every `continue` above fires, and the file is green.+ check_true("defaults/the drifted cell is derived from Router, not asserted from a copy",+ derived.get("LAYA_DEFAULT_MODEL") == "multilingual",+ "got %r -- if the routing fallback moved again, the two pages this was written for "+ "moved with it or are now stale; check them before changing this line"+ % derived.get("LAYA_DEFAULT_MODEL"))+ counts = {s: len(surfaces[s]) for s in PACKAGE_DEFAULT_SURFACES}+ check_true("defaults/both package surfaces still carry a default column",+ all(count >= 12 and "LAYA_PRELOAD" in surfaces[s]+ for s, count in counts.items()), counts)+ check_true("defaults/LAYA_DEFAULT_MODEL is a row on both package surfaces",+ all("LAYA_DEFAULT_MODEL" in surfaces[s] for s in PACKAGE_DEFAULT_SURFACES),+ {s: sorted(surfaces[s]) for s in PACKAGE_DEFAULT_SURFACES})+ check_true("defaults/at least eight defaults were derived, not skipped",+ len(derived) >= 8, sorted(derived))+ documented = {name for surface in PACKAGE_DEFAULT_SURFACES for name in surfaces[surface]}+ for name in sorted(documented - set(derived)):+ # A name the server never reads has no unset path here to follow, so it joins the reported+ # set rather than passing quietly -- but only if nothing more specific already explains it,+ # which is why this cannot turn a failed derivation into a silent skip.+ unchecked.setdefault(name, "laya/serve.py does not read it, so this file has no unset "+ "path to follow for it")+ unaccounted = sorted(name for surface in PACKAGE_DEFAULT_SURFACES+ for name in surfaces[surface]+ if name not in derived and name not in unchecked)+ check_true("defaults/every documented default is derived or named as unchecked",+ not unaccounted, unaccounted)+# ------------------------------------------------- what this cannot checkunbacked = [("a documented default matches the code default",- "docs/docker.md documents the Compose service's default, which is deliberately not "- "the package default (LAYA_PRELOAD is 0 there and 1 in laya/serve.py)"),+ "held to laya/serve.py and laya/router.py for %d of the names on the two package surfaces; "+ "%s stay out because their unset answer is not a value a table can print. Also out: "+ "docs/docker.md, which documents the Compose service's defaults and deliberately differs "+ "from the package (LAYA_PRELOAD is 0 there and 1 in laya/serve.py), and docs/cli-mcp.md, "+ "which documents the stdio server's"+ % (len(derived), sorted(set(surfaces["docs/http-api.md"]) - set(derived)))),("a name only mentioned in a comment","comments are not string constants, so an undocumented read in a comment is invisible ""here; the same is true of a name built by concatenation, and of an accepted spelling "@@ -391,6 +754,9 @@ def main() -> int:print("laya/agent.py compares %s against %s" % (var, sorted(accepted[var])))print("%d row(s) and %d completeness claim(s) held to that set: %s / %s"% (len(rows_sites), len(claim_sites), sorted(rows_sites), sorted(claim_sites)))+ print("%d documented default(s) held to laya/serve.py and laya/router.py" % len(derived))+ for name, why in sorted(unchecked.items()):+ print("not derived: %s -- %s" % (name, why))for name, why in sorted(DEFERRED.items()):print("deferred: %s -- %s; delete the DEFERRED row once it is documented" % (name, why))print("not checkable from this repository (not asserted either way):")