Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
Merge branch 'pr738' into integration-0411
Nandakishor79f463cb0300
diff --cc README.mdindex a6e1604b,27679545..0c08b126--- a/README.md+++ b/README.md@@@ -1356,7 -1322,7 +1356,7 @@@ TileLang fast path and `compile=True` rLaya can be exposed as an [MCP](https://modelcontextprotocol.io) stdio server, so any MCPclient (OpenClaw, Claude Desktop, Cursor, ...) can call typed decisions as tools- (`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_tournament`, `laya_preset`, `laya_status`) without writing glue code.-(`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_predict_long`, `laya_preset`, `laya_status`) without writing glue code.++(`laya_predict`, `laya_predict_batch`, `laya_predict_long`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_tournament`, `laya_preset`, `laya_status`) without writing glue code.This is an **optional extra**: the core package has no `mcp` dependency.```bash@@@ -1417,16 -1382,12 +1417,18 @@@ it shortlists a many-option choice quessimilarity (mean-pooled from the answering checkpoint's own encoder, so no extra model isdownloaded), answers in one forward pass, and returns per-question shortlist metadata(kept labels, cosine scores, `k`, option count, and whether the question passed through-unshortlisted). The guardrails shown on every decision-tool point clients to `laya_shortlist` for >20-option choices. As with the SDK, use it for structured+unshortlisted). `laya_tournament` is the in-process MCP form of [`predict_tournament`](#honest-limits):+the decision model narrows large choices in balanced groups of `group_size=16` by default,+then answers the complete request with the finalists. It returns `tournament` metadata per choice+(`labels`, original count `n`, elimination `rounds`), finalist-only probabilities, and total latency.+Small choices pass through with `rounds: 0`; narrowed questions report discarded `option_order`+values under `option_order_dropped`. Routing uses the original request; language, budget and+confidence controls reach every round. The guardrails shown on every decision+tool point clients to `laya_shortlist` or `laya_tournament` for >20-option choices. As with the SDK, use it for structureddecisions only; not for open Q&A or- text generation. Tests: `tests/test_mcp.py` (CI, no weights) and+ text generation. `laya_predict_long` is the MCP form of [`predict_long`](#long-documents-predict_long): it+ scans a state past the context window in overlapping windows and answers with per-answer+ window attribution plus the window count. Tests: `tests/test_mcp.py` (CI, no weights) and`tests/test_mcp_local_e2e.py` (local, real weights and a real stdio handshake).---diff --cc docs/cli-mcp.mdindex 9db36158,f967aa8e..71319df3--- a/docs/cli-mcp.md+++ b/docs/cli-mcp.md@@@ -168,9 -165,8 +168,10 @@@ Laya does not open a network port| `laya_predict_batch` | Answers many requests in one call. Requests are routed first and grouped by checkpoint, so matching question schemas share forward passes; answers come back in input order. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?, max_len?, head_max_len?}`, optional `batch_size`, `hooks_timeout`, `min_confidence`, `sort_by_length` || `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout` || `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model`, `min_confidence` |+ | `laya_predict_long` | Scans a state longer than the context window in overlapping windows, then answers with per-answer window attribution and the window count. | `state`, `questions`, optional `model`, `window`, `stride`, `task`, `lang` |+On `laya_route`, `laya_predict`, `laya_shortlist`, `laya_tournament`, and `laya_decide`, `model` may be omitted or set to `null`; either form resolves to `auto`.+The three batch and schema tools exist because the same operations are available on the SDK and`laya-serve`: handling many requests, or serving a caller that already knows the answer shape,does not require dropping to Python. For the schema-driven form in more depth, see@@@ -182,56 -178,16 +183,64 @@@ default is `k=20`. It uses mean-pooledso it does not download a second model, and returns the kept labels, cosine scores, `k`,option count, and whether the question passed through unshortlisted for each shortlistedquestion. A question with `k` at or past its option count is passed through unchanged, so-its cosine scores come back `null` rather than empty.+its cosine scores come back `null` rather than empty. A question that narrows cannot keep+a caller-supplied `option_order`: the order covers every option while the answer covers the+kept labels only, and remapping it onto them would move options between slots, so the order+is dropped and the question is named under `option_order_dropped` with its old and kept+option counts.++`laya_tournament` uses the decision model itself to narrow choices, without an embedding pass.+With the default `group_size=16`, 40 labels form balanced groups of 13, 13 and 14. One forward+pass answers all three groups; their winners meet in a final pass that also answers the rest+of the request. Larger sets can take more elimination rounds. The checkpoint is selected from+the original request and pinned for every round, keeping routing controls and detected-language+calibration consistent. Token budgets and `min_confidence` reach every pass.+A missing or out-of-group winner returns `internal_error` before another pass, so a broken+round cannot silently advance a label the model did not select from that group.++The result has the usual `answers`, `routing`, `latency_ms` and readable `device`, plus+`tournament` metadata for each choice: `labels` are the finalists in criteria order, `n` is the+original option count, and `rounds` counts elimination rounds before the final pass. A choice+that fits `group_size` reports `rounds: 0`; non-choice questions have no tournament entry.+Probabilities and confidence come from the final pass over the finalists only, so a threshold+fitted on the full label set needs to be measured again. `latency_ms` covers routing and all+passes. Narrowing reports a discarded `option_order` under `option_order_dropped` with `n` and+`kept` counts, just as shortlisting does; passthrough questions keep their order.++For a small example, call `laya_tournament` with these arguments. `group_size=2` forces one+elimination round here; keep the default 16 for a larger catalog unless you have measured another size.++```json+{+ "state": {"body": "Please refund the duplicate charge."},+ "questions": {+ "department": {+ "type": "choice",+ "instructions": "Which team should handle the `body`?",+ "criteria": {+ "billing": "payments, invoices, refunds",+ "technical": "bugs, outages, integrations",+ "account": "login, passwords, profile settings",+ "sales": "pricing, upgrades, product enquiries"+ }+ }+ },+ "group_size": 2+}+```++This makes two prediction calls and returns two finalists with `n: 4` and `rounds: 1`.+`group_size` must be an integer of at least 2; invalid values return `invalid_group_size`.+The tool is available only when the MCP server loads checkpoints in-process.+ When the state itself does not fit one window, `laya_predict` answers from its first window and+ never reads the rest. `laya_predict_long` tokenizes the state once, scores it in overlapping+ windows sized by `window` (default: the checkpoint budget) stepped by `stride` (default: half+ the window), and aggregates per question -- strongest window for `noul`, most-confident window+ for `choice`/`score`. Each answer names its deciding window, and the result counts the windows+ scored. There is no `min_confidence` here and no `max_len`/`head_max_len`: core's scan takes+ neither.+`state` must be a non-empty JSON object. `questions` must be a non-empty object whose values useLaya's typed question schema. `laya_preset` accepts the same five presets the CLI does: `email`,`guard`, `moderation`, `triage`, and the router workflow, whose canonical name on this surface isdiff --cc laya/mcp/tools.pyindex a757c11d,0e870869..b81368ef--- a/laya/mcp/tools.py+++ b/laya/mcp/tools.py@@@ -851,110 -834,126 +851,232 @@@ def laya_shortlistreturn out+def laya_tournament(+ state: Any,+ questions: Any,+ model: Any = "auto",+ group_size: Any = 16,+ *,+ task: Any = None,+ lang: Any = None,+ lang_guess: Any = None,+ max_len: Any = None,+ head_max_len: Any = None,+ min_confidence: Any = None,+ router: Any = None,+ agent: Any = None,+) -> dict:+ """Answer large choices with core's balanced tournament in-process.++ Select the checkpoint from the original request, then pin it for every round.+ Each round uses ``laya_predict``'s validation, answer shape and device reporting.+ Probabilities and confidence describe the final call's finalists; ``tournament``+ adds their ``labels``, the original option count ``n`` and elimination ``rounds``.+ Small choices report zero rounds; non-choice questions reach the final call.+ ``latency_ms`` covers routing and all rounds. A narrowed question's discarded+ ``option_order`` is reported with its original and kept option counts.++ Remote callers use ``laya_predict`` with the HTTP server's tournament enabled;+ its probability and usage contract differs from this core helper's contract.+ """+ if router is not None and agent is None and hasattr(router, "base_url") and hasattr(router, "health"):+ raise ToolError(+ "unsupported_remote",+ "laya_tournament runs core tournament rounds in-process; it is not available when "+ "LAYA_BASE_URL points at laya-serve. Use laya_predict with LAYA_TOURNAMENT_GROUP "+ "enabled on that server, or run the MCP server without LAYA_BASE_URL.",+ )+ state_d = validate_state(state)+ questions_d = validate_questions(questions)+ model_name = validate_model(model)+ if isinstance(group_size, bool) or not isinstance(group_size, int) or group_size < 2:+ raise ToolError("invalid_group_size", f"group_size must be an integer of at least 2, got {group_size!r}")+ routing_overrides = _overrides(validate_task(task), validate_lang(lang), None, None)+ hint = validate_lang_guess(lang_guess)+ if hint is not None:+ routing_overrides["lang_guess"] = hint+ forward = _overrides(None, None, validate_budget(max_len, "max_len"),+ validate_budget(head_max_len, "head_max_len"))+ min_conf = validate_min_confidence(min_confidence)+ if min_conf is not None:+ forward["min_confidence"] = min_conf+ for name in ("task", "lang_guess"):+ if model_name != AUTO and name in routing_overrides:+ raise ToolError(f"invalid_{name}",+ f"{name} routes between checkpoints; with a pinned model there is nothing to route")+ if "lang" in routing_overrides:+ forward["lang"] = routing_overrides["lang"]++ from laya.shortlist import predict_tournament++ started = time.perf_counter()+ routing = None+ predict_agent = agent+ if model_name == AUTO and router is not None:+ if not hasattr(router, "route"):+ raise ToolError("internal_error", "router has no route() method")+ decision = router.route(state_d, questions_d, **routing_overrides)+ routing = _decision_to_dict(decision)+ model_name = routing["model"]+ if not isinstance(model_name, str) or not model_name:+ raise ToolError("internal_error", "router.route returned no model")+ detection = (decision.get("detection") if isinstance(decision, dict)+ else getattr(decision, "detection", None))+ detected = _detected_language(detection)+ if detected is not None and "lang" not in forward:+ forward["lang"] = detected+ predict_agent = None++ class Predictor:+ def predict(self, round_state, round_questions):+ result = laya_predict(round_state, round_questions, model=model_name,+ router=router, agent=predict_agent, **forward)+ for name, spec in round_questions.items():+ if spec["type"] != "choice":+ continue+ answer = result["answers"].get(name)+ if (not isinstance(answer, dict) or not isinstance(answer.get("choice"), str)+ or answer["choice"] not in spec["criteria"]):+ raise ToolError("internal_error",+ f"tournament prediction for question {name!r} must return a choice from its criteria")+ return result++ out = predict_tournament(Predictor(), state_d, questions_d, group_size=group_size)+ if routing is not None:+ out["routing"] = routing+ out["latency_ms"] = round((time.perf_counter() - started) * 1000.0, 3)+ dropped_orders = {+ name: {"n": entry["n"], "kept": len(entry["labels"])}+ for name, entry in out["tournament"].items()+ if entry["rounds"] and "option_order" in questions_d[name]+ }+ if dropped_orders:+ out["option_order_dropped"] = dropped_orders+ return out+++++++ def laya_predict_long(+ state: Any,+ questions: Any,+ model: Any = "auto",+ *,+ window: Any = None,+ stride: Any = None,+ task: Any = None,+ lang: Any = None,+ router: Any = None,+ agent: Any = None,+ ) -> dict:+ """Scan a state longer than the context window, then answer.++ The in-process ``predict_long`` pattern over MCP: the state is tokenized once, scored in+ overlapping windows in shared forward passes, and aggregated per question (strongest window+ for ``noul``, most-confident window for ``choice``/``score``). Use this instead of+ ``laya_predict`` when the state does not fit one window -- ``laya_predict`` answers from+ its first window and never reads the rest.++ ``router`` is used when model == "auto"; ``agent`` for a direct checkpoint. ``task``/``lang``+ are the router's own overrides (an explicit ``model`` outranks an explicit ``task``, which+ outranks an explicit ``lang``); ``window``/``stride`` size the scan the way core's+ ``predict_long`` sizes it, and only set values are forwarded, so a checkpoint that predates+ those keywords keeps working. ``max_len``/``head_max_len``/``min_confidence`` are not+ accepted here because core's ``predict_long`` takes none of them: a window is sized by+ ``window`` or the checkpoint budget, and there is no abstention gate on the scan.++ Each answer carries the deciding window's ``window`` attribution (index and token offsets+ into the tokenized state), and ``windows`` counts the windows the model scored.+ """+ state_d = validate_state(state)+ questions_d = validate_questions(questions)+ model_name = validate_model(model)+ # Only what the caller actually set is forwarded: `_overrides` is not reused because its keys+ # are the single-window budgets, and `predict_long` rejects those -- the scan is sized by+ # `window`/`stride` or the checkpoint budget instead.+ scan = {name: value for name, value in (+ ("task", validate_task(task)),+ ("lang", validate_lang(lang)),+ ("window", validate_budget(window, "window")),+ ("stride", validate_budget(stride, "stride")),+ ) if value is not None}+ # `task` picks a checkpoint by saying what the work is, so it means nothing once one is pinned:+ # same rule as `laya_predict` -- the router checks an explicit `model` first and never reaches+ # the task, and a checkpoint's `predict_long` does not accept the keyword at all.+ if model_name != AUTO and "task" in scan:+ raise ToolError(+ "invalid_task",+ "task routes between checkpoints; with a pinned model there is nothing to route",+ )+ auto_without_router = False++ def _run() -> Any:+ if model_name == "auto":+ if router is None:+ if agent is None:+ raise ToolError("models_not_ready", "Router is not loaded (auto mode)")+ # An Agent answers but does not route: run it directly and report+ # a null model below, instead of echoing 'auto' (#444). `Agent.predict_long`+ # has no `task` parameter -- only a Router routes between checkpoints -- so the+ # keyword is dropped here exactly as `laya_predict` drops it (`tools.py:415`);+ # forwarding it would surface as a raw TypeError, not an actionable refusal.+ nonlocal auto_without_router+ auto_without_router = True+ return _scan(agent, state_d, questions_d,+ **{k: v for k, v in scan.items() if k != "task"})+ return _scan(router, state_d, questions_d, **scan)+ if agent is not None:+ return _scan(agent, state_d, questions_d, **scan)+ if router is None:+ raise ToolError("models_not_ready", "no agent/router loaded")+ return _scan(router, state_d, questions_d, model=model_name, **scan)++ started = time.perf_counter()+ result = _run()+ latency_ms = (time.perf_counter() - started) * 1000.0++ if not isinstance(result, dict):+ raise ToolError("internal_error", "predict_long returned non-object")+ answers = _normalize_answers(result.get("answers"))+ usage = result.get("usage") or {}+ windows = usage.get("windows", 0) if isinstance(usage, dict) else 0+ if auto_without_router:+ routing = {"model": None, "repo": None, "reason": "auto routing without router"}+ elif model_name == "auto":+ # A bare agent's scan carries no 'routing' key because the Agent itself does no+ # routing; 'auto' is a routing directive, not a checkpoint name (#444). A Router's+ # scan carries the decision it routed with.+ routing = (result.get("routing") or {"model": None, "repo": None,+ "reason": "auto routing without router"})+ else:+ routing = {"model": model_name, "repo": None, "reason": "explicit model"}+ device = None if auto_without_router else _reading_device(router, agent, model_name,+ result.get("routing"))+ out: dict[str, Any] = {+ "answers": answers,+ "routing": routing,+ "windows": windows,+ "latency_ms": round(latency_ms, 3),+ }+ if device:+ out["device"] = device+ return out+++ def _scan(target: Any, state: Any, questions: Any, **kwargs: Any) -> Any:+ """Call ``predict_long`` on a router or agent, refusing clearly without it.++ A checkpoint old enough to predate the scan (or a hand-rolled stand-in) has no+ ``predict_long``; falling back to ``predict`` would silently answer from the first+ window, which is exactly what this tool exists to avoid.+ """+ fn = getattr(target, "predict_long", None)+ if not callable(fn):+ raise ToolError("internal_error", "%s has no predict_long() method"+ % type(target).__name__)+ return fn(state, questions, **kwargs)++def laya_preset(preset: Any,state: Any,diff --cc tests/test_mcp.pyindex 65d14438,ea62e453..c5e4f6a5--- a/tests/test_mcp.py+++ b/tests/test_mcp.py@@@ -2207,8 -1944,11 +2353,11 @@@ def test_controls_signature_and_schema(import inspectcontrols = ["task", "lang", "lang_guess", "max_len", "head_max_len"]- for fn, want in ((laya_predict, controls), (laya_shortlist, controls),+ for fn, want in ((laya_predict, controls), (laya_shortlist, controls), (laya_tournament, controls),- (laya_preset, controls), (laya_route, ["model", "task", "lang", "lang_guess"])):+ (laya_preset, controls), (laya_route, ["model", "task", "lang", "lang_guess"]),+ # predict_long sizes the scan by window/stride, not the single-window+ # budgets, so only the routing overrides are shared controls here.+ (laya_predict_long, ["task", "lang"])):params = inspect.signature(fn).parametersfor name in want:ok("signature/%s_has_%s" % (fn.__name__, name), name in params, repr(sorted(params)))@@@ -2221,8 -1961,9 +2370,9 @@@repr(list(inspect.signature(laya_predict).parameters)))by_name = {t.name: t for t in asyncio.run(mcp_server.list_tools())}- for name, want in (("laya_predict", controls), ("laya_shortlist", controls),+ for name, want in (("laya_predict", controls), ("laya_shortlist", controls), ("laya_tournament", controls),- ("laya_preset", controls), ("laya_route", ["model", "task", "lang"])):+ ("laya_preset", controls), ("laya_route", ["model", "task", "lang"]),+ ("laya_predict_long", ["task", "lang"])):tool = by_name[name]schema = tool.input_schema if hasattr(tool, "input_schema") else tool.inputSchemaprops = schema.get("properties", {})@@@ -3003,10 -2724,11 +3153,12 @@@ def test_server_registration()tools = asyncio.run(mcp_server.list_tools())names = sorted(t.name for t in tools)ok("server/tool_names", names == ["laya_decide", "laya_predict", "laya_predict_batch",- "laya_preset", "laya_route", "laya_route_batch",- "laya_shortlist", "laya_status", "laya_tournament"], repr(names))+ "laya_predict_long", "laya_preset", "laya_route",- "laya_route_batch", "laya_shortlist", "laya_status"],- repr(names))++ "laya_route_batch", "laya_shortlist", "laya_status",++ "laya_tournament"], repr(names))decision = {"laya_predict", "laya_route", "laya_preset", "laya_shortlist",- "laya_predict_batch", "laya_route_batch", "laya_decide", "laya_tournament"}- "laya_predict_batch", "laya_route_batch", "laya_decide", "laya_predict_long"}++ "laya_predict_batch", "laya_route_batch", "laya_decide", "laya_tournament",++ "laya_predict_long"}for t in tools:desc = (t.description or "").lower()ok("server/desc_%s_nonempty" % t.name, bool(desc.strip()), repr(desc))@@@ -3209,8 -2882,8 +3361,8 @@@ def test_cli_mcp_page_tool_argument_rowreturn cells[2]return None- for tool in ("laya_status", "laya_route", "laya_predict",+ for tool in ("laya_status", "laya_route", "laya_predict", "laya_predict_long",- "laya_shortlist", "laya_preset", "laya_decide"):+ "laya_shortlist", "laya_tournament", "laya_preset", "laya_decide"):cell = row_cell(tool)ok("docs-mcp/args_row_%s_present" % tool, cell is not None, "row not found")if cell is None:diff --cc tests/test_mcp_local_e2e.pyindex 214ec78e,1857e81f..6b772e78--- a/tests/test_mcp_local_e2e.py+++ b/tests/test_mcp_local_e2e.py@@@ -2,8 -2,8 +2,9 @@@Launches ``python -m laya.mcp.server`` as a subprocess and speaks MCP overstdin/stdout (newline-delimited JSON-RPC), exactly like a real MCP client.- Exercises laya_predict, laya_predict_batch, laya_route, laya_route_batch, laya_decide,- laya_preset, laya_shortlist, laya_tournament and laya_status against the+ Exercises laya_predict, laya_predict_batch, laya_predict_long, laya_route,-laya_route_batch, laya_decide, laya_preset, laya_shortlist and laya_status against the++laya_route_batch, laya_decide, laya_preset, laya_shortlist, laya_tournament and++laya_status against thelive checkpoints (downloaded via huggingface_hub on first run, cachedafterwards).@@@ -117,8 -117,9 +118,9 @@@ def main()tools = client.request("tools/list", {})names = sorted(t["name"] for t in tools.get("tools", []))ok("e2e/tool_names", names == ["laya_decide", "laya_predict", "laya_predict_batch",- "laya_preset", "laya_route", "laya_route_batch",- "laya_shortlist", "laya_status", "laya_tournament"], repr(names))+ "laya_predict_long", "laya_preset", "laya_route",- "laya_route_batch", "laya_shortlist", "laya_status"],- repr(names))++ "laya_route_batch", "laya_shortlist", "laya_status",++ "laya_tournament"], repr(names))ticket = {"state": {