Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(mcp): explain tournament rounds and shared-server alternatives
Rishit Sharmac490ce5d2142
diff --git a/README.md b/README.mdindex 78720624..3ca7dfa2 100644--- a/README.md+++ b/README.md@@ -1347,7 +1347,7 @@ TileLang fast path and `compile=True` refuse such a checkpoint, and `backend="auLaya can be exposed as an [MCP](https://modelcontextprotocol.io) stdio server, so any MCPclient (OpenClaw, Claude Desktop, Cursor, ...) can call typed decisions as tools-(`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_preset`, `laya_status`) without writing glue code.+(`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_decide`, `laya_shortlist`, `laya_tournament`, `laya_preset`, `laya_status`) without writing glue code.This is an **optional extra**: the core package has no `mcp` dependency.```bash@@ -1358,8 +1358,9 @@ laya-mcp-server # or: python -m laya.mcp.serverTo share a `laya-serve` on your own hardware across editor sessions, set`LAYA_BASE_URL=http://127.0.0.1:8000` in each MCP client's environment. Its prediction tools useHTTP and import no torch; routing stays local. Set `LAYA_API_KEY` to the server's bearer key and-`LAYA_REMOTE_TIMEOUT` to the allowed HTTP wait (300 seconds by default). `laya_shortlist` is-unavailable in this mode. See [shared-server MCP setup](https://github.com/NandhaKishorM/laya/blob/main/docs/cli-mcp.md#share-one-model-server-across-mcp-sessions).+`LAYA_REMOTE_TIMEOUT` to the allowed HTTP wait (300 seconds by default). `laya_shortlist` and+`laya_tournament` are unavailable in this mode; use `laya_predict` with the HTTP server's+`LAYA_TOURNAMENT_GROUP` enabled for shared-server tournament selection. See [shared-server MCP setup](https://github.com/NandhaKishorM/laya/blob/main/docs/cli-mcp.md#share-one-model-server-across-mcp-sessions).Example MCP client configuration (stdio transport):@@ -1407,8 +1408,14 @@ it shortlists a many-option choice question to its `k` most likely labels by embsimilarity (mean-pooled from the answering checkpoint's own encoder, so no extra model isdownloaded), answers in one forward pass, and returns per-question shortlist metadata(kept labels, cosine scores, `k`, option count, and whether the question passed through-unshortlisted). The guardrails shown on every decision-tool point clients to `laya_shortlist` for >20-option choices. As with the SDK, use it for structured+unshortlisted). `laya_tournament` is the in-process MCP form of [`predict_tournament`](#honest-limits):+the decision model narrows large choices in balanced groups of `group_size=16` by default,+then answers the complete request with the finalists. It returns `tournament` metadata per choice+(`labels`, original count `n`, elimination `rounds`), finalist-only probabilities, and total latency.+Small choices pass through with `rounds: 0`; narrowed questions report discarded `option_order`+values under `option_order_dropped`. Routing uses the original request; language, budget and+confidence controls reach every round. The guardrails shown on every decision+tool point clients to `laya_shortlist` or `laya_tournament` for >20-option choices. As with the SDK, use it for structureddecisions only; not for open Q&A ortext generation. Tests: `tests/test_mcp.py` (CI, no weights) and`tests/test_mcp_local_e2e.py` (local, real weights and a real stdio handshake).diff --git a/docs/cli-mcp.md b/docs/cli-mcp.mdindex 9703dc7b..9efbc614 100644--- a/docs/cli-mcp.md+++ b/docs/cli-mcp.md@@ -167,15 +167,15 @@ Laya does not open a network port.| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout` || `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model`, `min_confidence` |-On `laya_route`, `laya_predict`, `laya_shortlist`, and `laya_decide`, `model` may be omitted or set to `null`; either form resolves to `auto`.+On `laya_route`, `laya_predict`, `laya_shortlist`, `laya_tournament`, and `laya_decide`, `model` may be omitted or set to `null`; either form resolves to `auto`.The three batch and schema tools exist because the same operations are available on the SDK and`laya-serve`: handling many requests, or serving a caller that already knows the answer shape,does not require dropping to Python. For the schema-driven form in more depth, see[Schema-driven decisions](structured.md).-The shared guardrail says not to send choice questions with more than 20 options without-shortlisting. `laya_shortlist` keeps the `k` most likely labels before the forward pass; its+The shared guardrail says to narrow choice questions with more than 20 options using+`laya_shortlist` or `laya_tournament`. `laya_shortlist` keeps the `k` most likely labels before the forward pass; itsdefault is `k=20`. It uses mean-pooled embeddings from the answering checkpoint's own encoder,so it does not download a second model, and returns the kept labels, cosine scores, `k`,option count, and whether the question passed through unshortlisted for each shortlisted@@ -186,6 +186,48 @@ kept labels only, and remapping it onto them would move options between slots, sis dropped and the question is named under `option_order_dropped` with its old and keptoption counts.+`laya_tournament` uses the decision model itself to narrow choices, without an embedding pass.+With the default `group_size=16`, 40 labels form balanced groups of 13, 13 and 14. One forward+pass answers all three groups; their winners meet in a final pass that also answers the rest+of the request. Larger sets can take more elimination rounds. The checkpoint is selected from+the original request and pinned for every round, keeping routing controls and detected-language+calibration consistent. Token budgets and `min_confidence` reach every pass.++The result has the usual `answers`, `routing`, `latency_ms` and readable `device`, plus+`tournament` metadata for each choice: `labels` are the finalists in criteria order, `n` is the+original option count, and `rounds` counts elimination rounds before the final pass. A choice+that fits `group_size` reports `rounds: 0`; non-choice questions have no tournament entry.+Probabilities and confidence come from the final pass over the finalists only, so a threshold+fitted on the full label set needs to be measured again. `latency_ms` covers routing and all+passes. Narrowing reports a discarded `option_order` under `option_order_dropped` with `n` and+`kept` counts, just as shortlisting does; passthrough questions keep their order.++For a small example, call `laya_tournament` with these arguments. `group_size=2` forces one+elimination round here; keep the default 16 for a larger catalog unless you have measured another size.++```json+{+ "state": {"body": "Please refund the duplicate charge."},+ "questions": {+ "department": {+ "type": "choice",+ "instructions": "Which team should handle the `body`?",+ "criteria": {+ "billing": "payments, invoices, refunds",+ "technical": "bugs, outages, integrations",+ "account": "login, passwords, profile settings",+ "sales": "pricing, upgrades, product enquiries"+ }+ }+ },+ "group_size": 2+}+```++This makes two prediction calls and returns two finalists with `n: 4` and `rounds: 1`.+`group_size` must be an integer of at least 2; invalid values return `invalid_group_size`.+The tool is available only when the MCP server loads checkpoints in-process.+`state` must be a non-empty JSON object. `questions` must be a non-empty object whose values useLaya's typed question schema. `laya_preset` accepts the same five presets the CLI does: `email`,`guard`, `moderation`, `triage`, and the router workflow, whose canonical name on this surface is@@ -200,7 +242,7 @@ Every single-request tool takes the same per-call routing controls the batch reqand `lang_guess` (a soft language hint that sits below `lang` and above the built-in detector, so aprobable-but-uncertain code can nudge which checkpoint is chosen without forcing it the way `lang`does). `lang_guess` only participates in routing, so like `task` it is refused on a call that pins-`model` -- a pinned checkpoint has nothing left to route. `laya_predict` and `laya_shortlist` also+`model` -- a pinned checkpoint has nothing left to route. `laya_predict`, `laya_shortlist`, and `laya_tournament` alsotake `max_len`/`head_max_len` for the answering token budget and `min_confidence` for the abstentiongate.@@ -284,8 +326,12 @@ Set the same `LAYA_API_KEY` in both processes when the server requires a bearer`LAYA_DEFAULT_MODEL` and `LAYA_AUTO_TASK` aligned so local routing previews match the server'sactual routing. Device and preload settings belong to the HTTP server. The first call after anidle unload waits for a cold load; raise `LAYA_REMOTE_TIMEOUT` if that takes longer than 300 seconds.-`laya_shortlist` and prediction hook overrides return `unsupported_remote`, since their code needs-the model process. HTTP errors retain the server's detail text as MCP tool errors. With+`laya_shortlist`, `laya_tournament`, and prediction hook overrides return `unsupported_remote`, since these+paths run in the model process. For shared-server tournament selection, enable+[`LAYA_TOURNAMENT_GROUP` on the HTTP server](http-api.md#large-choices-laya_tournament_group) and call+`laya_predict`: HTTP keeps probabilities for every original label (zero for eliminated labels),+reports `x_tournament`, and counts usage across rounds. The in-process `laya_tournament` returns+finalist-only probabilities and core's `tournament` metadata. HTTP errors retain the server's detail text as MCP tool errors. With`LAYA_BASE_URL` unset, the MCP server keeps loading checkpoints in its own process.The stock `laya-mcp-server` launcher creates its Router without installing hooks. Install prediction