Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

Merge branch 'pr738' into integration-0411

Nandakishor79f463cb0300
diff --cc README.md
index a6e1604b,27679545..0c08b126
--- a/README.md
+++ b/README.md
@@@ -1356,7 -1322,7 +1356,7 @@@ TileLang fast path and `compile=True` r
Laya can be exposed as an [MCP](https://modelcontextprotocol.io) stdio server, so any MCP
client (OpenClaw, Claude Desktop, Cursor, ...) can call typed decisions as tools
- (`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_tournament`, `laya_preset`, `laya_status`) without writing glue code.
-(`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_predict_long`, `laya_preset`, `laya_status`) without writing glue code.
++(`laya_predict`, `laya_predict_batch`, `laya_predict_long`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_tournament`, `laya_preset`, `laya_status`) without writing glue code.
This is an **optional extra**: the core package has no `mcp` dependency.
```bash
@@@ -1417,16 -1382,12 +1417,18 @@@ it shortlists a many-option choice ques
similarity (mean-pooled from the answering checkpoint's own encoder, so no extra model is
downloaded), answers in one forward pass, and returns per-question shortlist metadata
(kept labels, cosine scores, `k`, option count, and whether the question passed through
-unshortlisted). The guardrails shown on every decision
-tool point clients to `laya_shortlist` for >20-option choices. As with the SDK, use it for structured
+unshortlisted). `laya_tournament` is the in-process MCP form of [`predict_tournament`](#honest-limits):
+the decision model narrows large choices in balanced groups of `group_size=16` by default,
+then answers the complete request with the finalists. It returns `tournament` metadata per choice
+(`labels`, original count `n`, elimination `rounds`), finalist-only probabilities, and total latency.
+Small choices pass through with `rounds: 0`; narrowed questions report discarded `option_order`
+values under `option_order_dropped`. Routing uses the original request; language, budget and
+confidence controls reach every round. The guardrails shown on every decision
+tool point clients to `laya_shortlist` or `laya_tournament` for >20-option choices. As with the SDK, use it for structured
decisions only; not for open Q&A or
- text generation. Tests: `tests/test_mcp.py` (CI, no weights) and
+ text generation. `laya_predict_long` is the MCP form of [`predict_long`](#long-documents-predict_long): it
+ scans a state past the context window in overlapping windows and answers with per-answer
+ window attribution plus the window count. Tests: `tests/test_mcp.py` (CI, no weights) and
`tests/test_mcp_local_e2e.py` (local, real weights and a real stdio handshake).
---
diff --cc docs/cli-mcp.md
index 9db36158,f967aa8e..71319df3
--- a/docs/cli-mcp.md
+++ b/docs/cli-mcp.md
@@@ -168,9 -165,8 +168,10 @@@ Laya does not open a network port
| `laya_predict_batch` | Answers many requests in one call. Requests are routed first and grouped by checkpoint, so matching question schemas share forward passes; answers come back in input order. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?, max_len?, head_max_len?}`, optional `batch_size`, `hooks_timeout`, `min_confidence`, `sort_by_length` |
| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout` |
| `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model`, `min_confidence` |
+ | `laya_predict_long` | Scans a state longer than the context window in overlapping windows, then answers with per-answer window attribution and the window count. | `state`, `questions`, optional `model`, `window`, `stride`, `task`, `lang` |
+On `laya_route`, `laya_predict`, `laya_shortlist`, `laya_tournament`, and `laya_decide`, `model` may be omitted or set to `null`; either form resolves to `auto`.
+
The three batch and schema tools exist because the same operations are available on the SDK and
`laya-serve`: handling many requests, or serving a caller that already knows the answer shape,
does not require dropping to Python. For the schema-driven form in more depth, see
@@@ -182,56 -178,16 +183,64 @@@ default is `k=20`. It uses mean-pooled
so it does not download a second model, and returns the kept labels, cosine scores, `k`,
option count, and whether the question passed through unshortlisted for each shortlisted
question. A question with `k` at or past its option count is passed through unchanged, so
-its cosine scores come back `null` rather than empty.
+its cosine scores come back `null` rather than empty. A question that narrows cannot keep
+a caller-supplied `option_order`: the order covers every option while the answer covers the
+kept labels only, and remapping it onto them would move options between slots, so the order
+is dropped and the question is named under `option_order_dropped` with its old and kept
+option counts.
+
+`laya_tournament` uses the decision model itself to narrow choices, without an embedding pass.
+With the default `group_size=16`, 40 labels form balanced groups of 13, 13 and 14. One forward
+pass answers all three groups; their winners meet in a final pass that also answers the rest
+of the request. Larger sets can take more elimination rounds. The checkpoint is selected from
+the original request and pinned for every round, keeping routing controls and detected-language
+calibration consistent. Token budgets and `min_confidence` reach every pass.
+A missing or out-of-group winner returns `internal_error` before another pass, so a broken
+round cannot silently advance a label the model did not select from that group.
+
+The result has the usual `answers`, `routing`, `latency_ms` and readable `device`, plus
+`tournament` metadata for each choice: `labels` are the finalists in criteria order, `n` is the
+original option count, and `rounds` counts elimination rounds before the final pass. A choice
+that fits `group_size` reports `rounds: 0`; non-choice questions have no tournament entry.
+Probabilities and confidence come from the final pass over the finalists only, so a threshold
+fitted on the full label set needs to be measured again. `latency_ms` covers routing and all
+passes. Narrowing reports a discarded `option_order` under `option_order_dropped` with `n` and
+`kept` counts, just as shortlisting does; passthrough questions keep their order.
+
+For a small example, call `laya_tournament` with these arguments. `group_size=2` forces one
+elimination round here; keep the default 16 for a larger catalog unless you have measured another size.
+
+```json
+{
+ "state": {"body": "Please refund the duplicate charge."},
+ "questions": {
+ "department": {
+ "type": "choice",
+ "instructions": "Which team should handle the `body`?",
+ "criteria": {
+ "billing": "payments, invoices, refunds",
+ "technical": "bugs, outages, integrations",
+ "account": "login, passwords, profile settings",
+ "sales": "pricing, upgrades, product enquiries"
+ }
+ }
+ },
+ "group_size": 2
+}
+```
+
+This makes two prediction calls and returns two finalists with `n: 4` and `rounds: 1`.
+`group_size` must be an integer of at least 2; invalid values return `invalid_group_size`.
+The tool is available only when the MCP server loads checkpoints in-process.
+ When the state itself does not fit one window, `laya_predict` answers from its first window and
+ never reads the rest. `laya_predict_long` tokenizes the state once, scores it in overlapping
+ windows sized by `window` (default: the checkpoint budget) stepped by `stride` (default: half
+ the window), and aggregates per question -- strongest window for `noul`, most-confident window
+ for `choice`/`score`. Each answer names its deciding window, and the result counts the windows
+ scored. There is no `min_confidence` here and no `max_len`/`head_max_len`: core's scan takes
+ neither.
+
`state` must be a non-empty JSON object. `questions` must be a non-empty object whose values use
Laya's typed question schema. `laya_preset` accepts the same five presets the CLI does: `email`,
`guard`, `moderation`, `triage`, and the router workflow, whose canonical name on this surface is
diff --cc laya/mcp/tools.py
index a757c11d,0e870869..b81368ef
--- a/laya/mcp/tools.py
+++ b/laya/mcp/tools.py
@@@ -851,110 -834,126 +851,232 @@@ def laya_shortlist
return out
+def laya_tournament(
+ state: Any,
+ questions: Any,
+ model: Any = "auto",
+ group_size: Any = 16,
+ *,
+ task: Any = None,
+ lang: Any = None,
+ lang_guess: Any = None,
+ max_len: Any = None,
+ head_max_len: Any = None,
+ min_confidence: Any = None,
+ router: Any = None,
+ agent: Any = None,
+) -> dict:
+ """Answer large choices with core's balanced tournament in-process.
+
+ Select the checkpoint from the original request, then pin it for every round.
+ Each round uses ``laya_predict``'s validation, answer shape and device reporting.
+ Probabilities and confidence describe the final call's finalists; ``tournament``
+ adds their ``labels``, the original option count ``n`` and elimination ``rounds``.
+ Small choices report zero rounds; non-choice questions reach the final call.
+ ``latency_ms`` covers routing and all rounds. A narrowed question's discarded
+ ``option_order`` is reported with its original and kept option counts.
+
+ Remote callers use ``laya_predict`` with the HTTP server's tournament enabled;
+ its probability and usage contract differs from this core helper's contract.
+ """
+ if router is not None and agent is None and hasattr(router, "base_url") and hasattr(router, "health"):
+ raise ToolError(
+ "unsupported_remote",
+ "laya_tournament runs core tournament rounds in-process; it is not available when "
+ "LAYA_BASE_URL points at laya-serve. Use laya_predict with LAYA_TOURNAMENT_GROUP "
+ "enabled on that server, or run the MCP server without LAYA_BASE_URL.",
+ )
+ state_d = validate_state(state)
+ questions_d = validate_questions(questions)
+ model_name = validate_model(model)
+ if isinstance(group_size, bool) or not isinstance(group_size, int) or group_size < 2:
+ raise ToolError("invalid_group_size", f"group_size must be an integer of at least 2, got {group_size!r}")
+ routing_overrides = _overrides(validate_task(task), validate_lang(lang), None, None)
+ hint = validate_lang_guess(lang_guess)
+ if hint is not None:
+ routing_overrides["lang_guess"] = hint
+ forward = _overrides(None, None, validate_budget(max_len, "max_len"),
+ validate_budget(head_max_len, "head_max_len"))
+ min_conf = validate_min_confidence(min_confidence)
+ if min_conf is not None:
+ forward["min_confidence"] = min_conf
+ for name in ("task", "lang_guess"):
+ if model_name != AUTO and name in routing_overrides:
+ raise ToolError(f"invalid_{name}",
+ f"{name} routes between checkpoints; with a pinned model there is nothing to route")
+ if "lang" in routing_overrides:
+ forward["lang"] = routing_overrides["lang"]
+
+ from laya.shortlist import predict_tournament
+
+ started = time.perf_counter()
+ routing = None
+ predict_agent = agent
+ if model_name == AUTO and router is not None:
+ if not hasattr(router, "route"):
+ raise ToolError("internal_error", "router has no route() method")
+ decision = router.route(state_d, questions_d, **routing_overrides)
+ routing = _decision_to_dict(decision)
+ model_name = routing["model"]
+ if not isinstance(model_name, str) or not model_name:
+ raise ToolError("internal_error", "router.route returned no model")
+ detection = (decision.get("detection") if isinstance(decision, dict)
+ else getattr(decision, "detection", None))
+ detected = _detected_language(detection)
+ if detected is not None and "lang" not in forward:
+ forward["lang"] = detected
+ predict_agent = None
+
+ class Predictor:
+ def predict(self, round_state, round_questions):
+ result = laya_predict(round_state, round_questions, model=model_name,
+ router=router, agent=predict_agent, **forward)
+ for name, spec in round_questions.items():
+ if spec["type"] != "choice":
+ continue
+ answer = result["answers"].get(name)
+ if (not isinstance(answer, dict) or not isinstance(answer.get("choice"), str)
+ or answer["choice"] not in spec["criteria"]):
+ raise ToolError("internal_error",
+ f"tournament prediction for question {name!r} must return a choice from its criteria")
+ return result
+
+ out = predict_tournament(Predictor(), state_d, questions_d, group_size=group_size)
+ if routing is not None:
+ out["routing"] = routing
+ out["latency_ms"] = round((time.perf_counter() - started) * 1000.0, 3)
+ dropped_orders = {
+ name: {"n": entry["n"], "kept": len(entry["labels"])}
+ for name, entry in out["tournament"].items()
+ if entry["rounds"] and "option_order" in questions_d[name]
+ }
+ if dropped_orders:
+ out["option_order_dropped"] = dropped_orders
+ return out
+
+
++
++
+ def laya_predict_long(
+ state: Any,
+ questions: Any,
+ model: Any = "auto",
+ *,
+ window: Any = None,
+ stride: Any = None,
+ task: Any = None,
+ lang: Any = None,
+ router: Any = None,
+ agent: Any = None,
+ ) -> dict:
+ """Scan a state longer than the context window, then answer.
+
+ The in-process ``predict_long`` pattern over MCP: the state is tokenized once, scored in
+ overlapping windows in shared forward passes, and aggregated per question (strongest window
+ for ``noul``, most-confident window for ``choice``/``score``). Use this instead of
+ ``laya_predict`` when the state does not fit one window -- ``laya_predict`` answers from
+ its first window and never reads the rest.
+
+ ``router`` is used when model == "auto"; ``agent`` for a direct checkpoint. ``task``/``lang``
+ are the router's own overrides (an explicit ``model`` outranks an explicit ``task``, which
+ outranks an explicit ``lang``); ``window``/``stride`` size the scan the way core's
+ ``predict_long`` sizes it, and only set values are forwarded, so a checkpoint that predates
+ those keywords keeps working. ``max_len``/``head_max_len``/``min_confidence`` are not
+ accepted here because core's ``predict_long`` takes none of them: a window is sized by
+ ``window`` or the checkpoint budget, and there is no abstention gate on the scan.
+
+ Each answer carries the deciding window's ``window`` attribution (index and token offsets
+ into the tokenized state), and ``windows`` counts the windows the model scored.
+ """
+ state_d = validate_state(state)
+ questions_d = validate_questions(questions)
+ model_name = validate_model(model)
+ # Only what the caller actually set is forwarded: `_overrides` is not reused because its keys
+ # are the single-window budgets, and `predict_long` rejects those -- the scan is sized by
+ # `window`/`stride` or the checkpoint budget instead.
+ scan = {name: value for name, value in (
+ ("task", validate_task(task)),
+ ("lang", validate_lang(lang)),
+ ("window", validate_budget(window, "window")),
+ ("stride", validate_budget(stride, "stride")),
+ ) if value is not None}
+ # `task` picks a checkpoint by saying what the work is, so it means nothing once one is pinned:
+ # same rule as `laya_predict` -- the router checks an explicit `model` first and never reaches
+ # the task, and a checkpoint's `predict_long` does not accept the keyword at all.
+ if model_name != AUTO and "task" in scan:
+ raise ToolError(
+ "invalid_task",
+ "task routes between checkpoints; with a pinned model there is nothing to route",
+ )
+ auto_without_router = False
+
+ def _run() -> Any:
+ if model_name == "auto":
+ if router is None:
+ if agent is None:
+ raise ToolError("models_not_ready", "Router is not loaded (auto mode)")
+ # An Agent answers but does not route: run it directly and report
+ # a null model below, instead of echoing 'auto' (#444). `Agent.predict_long`
+ # has no `task` parameter -- only a Router routes between checkpoints -- so the
+ # keyword is dropped here exactly as `laya_predict` drops it (`tools.py:415`);
+ # forwarding it would surface as a raw TypeError, not an actionable refusal.
+ nonlocal auto_without_router
+ auto_without_router = True
+ return _scan(agent, state_d, questions_d,
+ **{k: v for k, v in scan.items() if k != "task"})
+ return _scan(router, state_d, questions_d, **scan)
+ if agent is not None:
+ return _scan(agent, state_d, questions_d, **scan)
+ if router is None:
+ raise ToolError("models_not_ready", "no agent/router loaded")
+ return _scan(router, state_d, questions_d, model=model_name, **scan)
+
+ started = time.perf_counter()
+ result = _run()
+ latency_ms = (time.perf_counter() - started) * 1000.0
+
+ if not isinstance(result, dict):
+ raise ToolError("internal_error", "predict_long returned non-object")
+ answers = _normalize_answers(result.get("answers"))
+ usage = result.get("usage") or {}
+ windows = usage.get("windows", 0) if isinstance(usage, dict) else 0
+ if auto_without_router:
+ routing = {"model": None, "repo": None, "reason": "auto routing without router"}
+ elif model_name == "auto":
+ # A bare agent's scan carries no 'routing' key because the Agent itself does no
+ # routing; 'auto' is a routing directive, not a checkpoint name (#444). A Router's
+ # scan carries the decision it routed with.
+ routing = (result.get("routing") or {"model": None, "repo": None,
+ "reason": "auto routing without router"})
+ else:
+ routing = {"model": model_name, "repo": None, "reason": "explicit model"}
+ device = None if auto_without_router else _reading_device(router, agent, model_name,
+ result.get("routing"))
+ out: dict[str, Any] = {
+ "answers": answers,
+ "routing": routing,
+ "windows": windows,
+ "latency_ms": round(latency_ms, 3),
+ }
+ if device:
+ out["device"] = device
+ return out
+
+
+ def _scan(target: Any, state: Any, questions: Any, **kwargs: Any) -> Any:
+ """Call ``predict_long`` on a router or agent, refusing clearly without it.
+
+ A checkpoint old enough to predate the scan (or a hand-rolled stand-in) has no
+ ``predict_long``; falling back to ``predict`` would silently answer from the first
+ window, which is exactly what this tool exists to avoid.
+ """
+ fn = getattr(target, "predict_long", None)
+ if not callable(fn):
+ raise ToolError("internal_error", "%s has no predict_long() method"
+ % type(target).__name__)
+ return fn(state, questions, **kwargs)
+
+
def laya_preset(
preset: Any,
state: Any,
diff --cc tests/test_mcp.py
index 65d14438,ea62e453..c5e4f6a5
--- a/tests/test_mcp.py
+++ b/tests/test_mcp.py
@@@ -2207,8 -1944,11 +2353,11 @@@ def test_controls_signature_and_schema(
import inspect
controls = ["task", "lang", "lang_guess", "max_len", "head_max_len"]
- for fn, want in ((laya_predict, controls), (laya_shortlist, controls),
+ for fn, want in ((laya_predict, controls), (laya_shortlist, controls), (laya_tournament, controls),
- (laya_preset, controls), (laya_route, ["model", "task", "lang", "lang_guess"])):
+ (laya_preset, controls), (laya_route, ["model", "task", "lang", "lang_guess"]),
+ # predict_long sizes the scan by window/stride, not the single-window
+ # budgets, so only the routing overrides are shared controls here.
+ (laya_predict_long, ["task", "lang"])):
params = inspect.signature(fn).parameters
for name in want:
ok("signature/%s_has_%s" % (fn.__name__, name), name in params, repr(sorted(params)))
@@@ -2221,8 -1961,9 +2370,9 @@@
repr(list(inspect.signature(laya_predict).parameters)))
by_name = {t.name: t for t in asyncio.run(mcp_server.list_tools())}
- for name, want in (("laya_predict", controls), ("laya_shortlist", controls),
+ for name, want in (("laya_predict", controls), ("laya_shortlist", controls), ("laya_tournament", controls),
- ("laya_preset", controls), ("laya_route", ["model", "task", "lang"])):
+ ("laya_preset", controls), ("laya_route", ["model", "task", "lang"]),
+ ("laya_predict_long", ["task", "lang"])):
tool = by_name[name]
schema = tool.input_schema if hasattr(tool, "input_schema") else tool.inputSchema
props = schema.get("properties", {})
@@@ -3003,10 -2724,11 +3153,12 @@@ def test_server_registration()
tools = asyncio.run(mcp_server.list_tools())
names = sorted(t.name for t in tools)
ok("server/tool_names", names == ["laya_decide", "laya_predict", "laya_predict_batch",
- "laya_preset", "laya_route", "laya_route_batch",
- "laya_shortlist", "laya_status", "laya_tournament"], repr(names))
+ "laya_predict_long", "laya_preset", "laya_route",
- "laya_route_batch", "laya_shortlist", "laya_status"],
- repr(names))
++ "laya_route_batch", "laya_shortlist", "laya_status",
++ "laya_tournament"], repr(names))
decision = {"laya_predict", "laya_route", "laya_preset", "laya_shortlist",
- "laya_predict_batch", "laya_route_batch", "laya_decide", "laya_tournament"}
- "laya_predict_batch", "laya_route_batch", "laya_decide", "laya_predict_long"}
++ "laya_predict_batch", "laya_route_batch", "laya_decide", "laya_tournament",
++ "laya_predict_long"}
for t in tools:
desc = (t.description or "").lower()
ok("server/desc_%s_nonempty" % t.name, bool(desc.strip()), repr(desc))
@@@ -3209,8 -2882,8 +3361,8 @@@ def test_cli_mcp_page_tool_argument_row
return cells[2]
return None
- for tool in ("laya_status", "laya_route", "laya_predict",
+ for tool in ("laya_status", "laya_route", "laya_predict", "laya_predict_long",
- "laya_shortlist", "laya_preset", "laya_decide"):
+ "laya_shortlist", "laya_tournament", "laya_preset", "laya_decide"):
cell = row_cell(tool)
ok("docs-mcp/args_row_%s_present" % tool, cell is not None, "row not found")
if cell is None:
diff --cc tests/test_mcp_local_e2e.py
index 214ec78e,1857e81f..6b772e78
--- a/tests/test_mcp_local_e2e.py
+++ b/tests/test_mcp_local_e2e.py
@@@ -2,8 -2,8 +2,9 @@@
Launches ``python -m laya.mcp.server`` as a subprocess and speaks MCP over
stdin/stdout (newline-delimited JSON-RPC), exactly like a real MCP client.
- Exercises laya_predict, laya_predict_batch, laya_route, laya_route_batch, laya_decide,
- laya_preset, laya_shortlist, laya_tournament and laya_status against the
+ Exercises laya_predict, laya_predict_batch, laya_predict_long, laya_route,
-laya_route_batch, laya_decide, laya_preset, laya_shortlist and laya_status against the
++laya_route_batch, laya_decide, laya_preset, laya_shortlist, laya_tournament and
++laya_status against the
live checkpoints (downloaded via huggingface_hub on first run, cached
afterwards).
@@@ -117,8 -117,9 +118,9 @@@ def main()
tools = client.request("tools/list", {})
names = sorted(t["name"] for t in tools.get("tools", []))
ok("e2e/tool_names", names == ["laya_decide", "laya_predict", "laya_predict_batch",
- "laya_preset", "laya_route", "laya_route_batch",
- "laya_shortlist", "laya_status", "laya_tournament"], repr(names))
+ "laya_predict_long", "laya_preset", "laya_route",
- "laya_route_batch", "laya_shortlist", "laya_status"],
- repr(names))
++ "laya_route_batch", "laya_shortlist", "laya_status",
++ "laya_tournament"], repr(names))
ticket = {
"state": {