Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(mcp): explain tournament rounds and shared-server alternatives

Rishit Sharmac490ce5d2142
diff --git a/README.md b/README.md
index 78720624..3ca7dfa2 100644
--- a/README.md
+++ b/README.md
@@ -1347,7 +1347,7 @@ TileLang fast path and `compile=True` refuse such a checkpoint, and `backend="au
Laya can be exposed as an [MCP](https://modelcontextprotocol.io) stdio server, so any MCP
client (OpenClaw, Claude Desktop, Cursor, ...) can call typed decisions as tools
-(`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_preset`, `laya_status`) without writing glue code.
+(`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_tournament`, `laya_preset`, `laya_status`) without writing glue code.
This is an **optional extra**: the core package has no `mcp` dependency.
```bash
@@ -1358,8 +1358,9 @@ laya-mcp-server # or: python -m laya.mcp.server
To share a `laya-serve` on your own hardware across editor sessions, set
`LAYA_BASE_URL=http://127.0.0.1:8000` in each MCP client's environment. Its prediction tools use
HTTP and import no torch; routing stays local. Set `LAYA_API_KEY` to the server's bearer key and
-`LAYA_REMOTE_TIMEOUT` to the allowed HTTP wait (300 seconds by default). `laya_shortlist` is
-unavailable in this mode. See [shared-server MCP setup](https://github.com/NandhaKishorM/laya/blob/main/docs/cli-mcp.md#share-one-model-server-across-mcp-sessions).
+`LAYA_REMOTE_TIMEOUT` to the allowed HTTP wait (300 seconds by default). `laya_shortlist` and
+`laya_tournament` are unavailable in this mode; use `laya_predict` with the HTTP server's
+`LAYA_TOURNAMENT_GROUP` enabled for shared-server tournament selection. See [shared-server MCP setup](https://github.com/NandhaKishorM/laya/blob/main/docs/cli-mcp.md#share-one-model-server-across-mcp-sessions).
Example MCP client configuration (stdio transport):
@@ -1407,8 +1408,14 @@ it shortlists a many-option choice question to its `k` most likely labels by emb
similarity (mean-pooled from the answering checkpoint's own encoder, so no extra model is
downloaded), answers in one forward pass, and returns per-question shortlist metadata
(kept labels, cosine scores, `k`, option count, and whether the question passed through
-unshortlisted). The guardrails shown on every decision
-tool point clients to `laya_shortlist` for >20-option choices. As with the SDK, use it for structured
+unshortlisted). `laya_tournament` is the in-process MCP form of [`predict_tournament`](#honest-limits):
+the decision model narrows large choices in balanced groups of `group_size=16` by default,
+then answers the complete request with the finalists. It returns `tournament` metadata per choice
+(`labels`, original count `n`, elimination `rounds`), finalist-only probabilities, and total latency.
+Small choices pass through with `rounds: 0`; narrowed questions report discarded `option_order`
+values under `option_order_dropped`. Routing uses the original request; language, budget and
+confidence controls reach every round. The guardrails shown on every decision
+tool point clients to `laya_shortlist` or `laya_tournament` for >20-option choices. As with the SDK, use it for structured
decisions only; not for open Q&A or
text generation. Tests: `tests/test_mcp.py` (CI, no weights) and
`tests/test_mcp_local_e2e.py` (local, real weights and a real stdio handshake).
diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md
index 9703dc7b..9efbc614 100644
--- a/docs/cli-mcp.md
+++ b/docs/cli-mcp.md
@@ -167,15 +167,15 @@ Laya does not open a network port.
| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout` |
| `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model`, `min_confidence` |
-On `laya_route`, `laya_predict`, `laya_shortlist`, and `laya_decide`, `model` may be omitted or set to `null`; either form resolves to `auto`.
+On `laya_route`, `laya_predict`, `laya_shortlist`, `laya_tournament`, and `laya_decide`, `model` may be omitted or set to `null`; either form resolves to `auto`.
The three batch and schema tools exist because the same operations are available on the SDK and
`laya-serve`: handling many requests, or serving a caller that already knows the answer shape,
does not require dropping to Python. For the schema-driven form in more depth, see
[Schema-driven decisions](structured.md).
-The shared guardrail says not to send choice questions with more than 20 options without
-shortlisting. `laya_shortlist` keeps the `k` most likely labels before the forward pass; its
+The shared guardrail says to narrow choice questions with more than 20 options using
+`laya_shortlist` or `laya_tournament`. `laya_shortlist` keeps the `k` most likely labels before the forward pass; its
default is `k=20`. It uses mean-pooled embeddings from the answering checkpoint's own encoder,
so it does not download a second model, and returns the kept labels, cosine scores, `k`,
option count, and whether the question passed through unshortlisted for each shortlisted
@@ -186,6 +186,48 @@ kept labels only, and remapping it onto them would move options between slots, s
is dropped and the question is named under `option_order_dropped` with its old and kept
option counts.
+`laya_tournament` uses the decision model itself to narrow choices, without an embedding pass.
+With the default `group_size=16`, 40 labels form balanced groups of 13, 13 and 14. One forward
+pass answers all three groups; their winners meet in a final pass that also answers the rest
+of the request. Larger sets can take more elimination rounds. The checkpoint is selected from
+the original request and pinned for every round, keeping routing controls and detected-language
+calibration consistent. Token budgets and `min_confidence` reach every pass.
+
+The result has the usual `answers`, `routing`, `latency_ms` and readable `device`, plus
+`tournament` metadata for each choice: `labels` are the finalists in criteria order, `n` is the
+original option count, and `rounds` counts elimination rounds before the final pass. A choice
+that fits `group_size` reports `rounds: 0`; non-choice questions have no tournament entry.
+Probabilities and confidence come from the final pass over the finalists only, so a threshold
+fitted on the full label set needs to be measured again. `latency_ms` covers routing and all
+passes. Narrowing reports a discarded `option_order` under `option_order_dropped` with `n` and
+`kept` counts, just as shortlisting does; passthrough questions keep their order.
+
+For a small example, call `laya_tournament` with these arguments. `group_size=2` forces one
+elimination round here; keep the default 16 for a larger catalog unless you have measured another size.
+
+```json
+{
+ "state": {"body": "Please refund the duplicate charge."},
+ "questions": {
+ "department": {
+ "type": "choice",
+ "instructions": "Which team should handle the `body`?",
+ "criteria": {
+ "billing": "payments, invoices, refunds",
+ "technical": "bugs, outages, integrations",
+ "account": "login, passwords, profile settings",
+ "sales": "pricing, upgrades, product enquiries"
+ }
+ }
+ },
+ "group_size": 2
+}
+```
+
+This makes two prediction calls and returns two finalists with `n: 4` and `rounds: 1`.
+`group_size` must be an integer of at least 2; invalid values return `invalid_group_size`.
+The tool is available only when the MCP server loads checkpoints in-process.
+
`state` must be a non-empty JSON object. `questions` must be a non-empty object whose values use
Laya's typed question schema. `laya_preset` accepts the same five presets the CLI does: `email`,
`guard`, `moderation`, `triage`, and the router workflow, whose canonical name on this surface is
@@ -200,7 +242,7 @@ Every single-request tool takes the same per-call routing controls the batch req
and `lang_guess` (a soft language hint that sits below `lang` and above the built-in detector, so a
probable-but-uncertain code can nudge which checkpoint is chosen without forcing it the way `lang`
does). `lang_guess` only participates in routing, so like `task` it is refused on a call that pins
-`model` -- a pinned checkpoint has nothing left to route. `laya_predict` and `laya_shortlist` also
+`model` -- a pinned checkpoint has nothing left to route. `laya_predict`, `laya_shortlist`, and `laya_tournament` also
take `max_len`/`head_max_len` for the answering token budget and `min_confidence` for the abstention
gate.
@@ -284,8 +326,12 @@ Set the same `LAYA_API_KEY` in both processes when the server requires a bearer
`LAYA_DEFAULT_MODEL` and `LAYA_AUTO_TASK` aligned so local routing previews match the server's
actual routing. Device and preload settings belong to the HTTP server. The first call after an
idle unload waits for a cold load; raise `LAYA_REMOTE_TIMEOUT` if that takes longer than 300 seconds.
-`laya_shortlist` and prediction hook overrides return `unsupported_remote`, since their code needs
-the model process. HTTP errors retain the server's detail text as MCP tool errors. With
+`laya_shortlist`, `laya_tournament`, and prediction hook overrides return `unsupported_remote`, since these
+paths run in the model process. For shared-server tournament selection, enable
+[`LAYA_TOURNAMENT_GROUP` on the HTTP server](http-api.md#large-choices-laya_tournament_group) and call
+`laya_predict`: HTTP keeps probabilities for every original label (zero for eliminated labels),
+reports `x_tournament`, and counts usage across rounds. The in-process `laya_tournament` returns
+finalist-only probabilities and core's `tournament` metadata. HTTP errors retain the server's detail text as MCP tool errors. With
`LAYA_BASE_URL` unset, the MCP server keeps loading checkpoints in its own process.
The stock `laya-mcp-server` launcher creates its Router without installing hooks. Install prediction