Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(readme): list the batch MCP tools and their measured speedup
Aashishb847034e145e
diff --git a/README.md b/README.mdindex 54c20cf9..36f22179 100644--- a/README.md+++ b/README.md@@ -774,7 +774,7 @@ override on your own data rather than treating `A`/`B` as a universal fix.Laya can be exposed as an [MCP](https://modelcontextprotocol.io) stdio server, so any MCPclient (OpenClaw, Claude Desktop, Cursor, ...) can call typed decisions as tools-(`laya_predict`, `laya_route`, `laya_shortlist`, `laya_preset`, `laya_status`) without writing glue code.+(`laya_predict`, `laya_predict_batch`, `laya_route`, `laya_route_batch`, `laya_shortlist`, `laya_preset`, `laya_status`) without writing glue code.This is an **optional extra**: the core package has no `mcp` dependency.```bash@@ -807,12 +807,21 @@ package:| `LAYA_THREADS` | (torch default) | Same as `laya.serve`: cap torch intra-op threads for CPU inference; keep it at or below the physical core count |The tools return structured JSON (answers with probabilities, routing metadata, device,-`latency_ms`). `laya_shortlist` is the MCP form of [`predict_shortlist`](#honest-limits):+`latency_ms`). `laya_predict_batch` and `laya_route_batch` are the MCP form of+[`Router.predict_batch` / `route_batch`](#batch-mode-score-many-states-in-one-forward-pass):+one tool call takes an array of `{state, questions, model?, lang?}` requests, routes them+first, groups them by checkpoint, and shares forward passes between requests with the same+question schema, returning the answers in input order. On 16 mixed-language tickets through+the tool functions themselves, one batch call beat 16 `laya_predict` calls by **2.1-2.3x on+MPS** (983-1082 ms -> 467-477 ms) and **~1.25x on CPU** (1861-2471 ms -> 1470-1911 ms), with+**0/16 decision flips** (choice label, rounded score, noul sign) against the loop. Prefer it+whenever a client has more than a few requests: each saved round trip is also an MCP+request/response. `laya_shortlist` is the MCP form of [`predict_shortlist`](#honest-limits):it shortlists a many-option choice question to its `k` most likely labels by embeddingsimilarity (mean-pooled from the answering checkpoint's own encoder, so no extra model isdownloaded), answers in one forward pass, and returns per-question shortlist metadata(kept labels, cosine scores, `k`, option count). The guardrails shown on every decision-tool point clients here for >20-option choices. As with the SDK, use it for structured+tool point clients to `laya_shortlist` for >20-option choices. As with the SDK, use it for structureddecisions only; not for open Q&A ortext generation. Tests: `tests/test_mcp.py` (CI, no weights) and`tests/test_mcp_local_e2e.py` (local, real weights and a real stdio handshake).