Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(mcp): extend routing diagnostics to batches
xp78955db5e127
diff --git a/docs/cli-mcp.md b/docs/cli-mcp.mdindex ec4c1f55..365080ed 100644--- a/docs/cli-mcp.md+++ b/docs/cli-mcp.md@@ -166,7 +166,7 @@ Laya does not open a network port.| `laya_tournament` | Narrows large choices in balanced rounds using the decision model, then answers the complete request. In-process only. | `state`, `questions`, optional `model`, `group_size` (default `16`), `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence` || `laya_preset` | Runs a built-in workflow using its built-in question set. | `preset`, `state`, optional `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence` || `laya_predict_batch` | Answers many requests in one call. Requests are routed first and grouped by checkpoint, so matching question schemas share forward passes; answers come back in input order. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?, max_len?, head_max_len?}`, optional `batch_size`, `hooks_timeout`, `min_confidence`, `sort_by_length` |-| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout` |+| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. Opt into per-decision diagnostics with `explain`. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout`, `explain` (default `false`, for the whole batch) || `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model`, `min_confidence` || `laya_predict_long` | Scans a state longer than the context window in overlapping windows, then answers with per-answer window attribution and the window count. | `state`, `questions`, optional `model`, `window`, `stride`, `task`, `lang` |@@ -176,20 +176,45 @@ On `laya_route`, `laya_predict`, `laya_shortlist`, `laya_tournament`, and `laya_`stage`, `reason`, and `source`. When no source exists, `source` is `null`. When the final routingdecision has no diagnostic (for example, an explicit model or language override, or a hookreplacement), `diagnostic` is `null`. The diagnostic comes from that routing call; it does not-run detection again after a hook. Omitted or `false` keeps the existing three-field result.+run detection again after a hook. `laya_route_batch` takes the same flag once for the whole+batch, adds `diagnostic` to each decision, and preserves input order and `model_counts`.+Omitted or `false` keeps each existing three-field decision.Only booleans are accepted: strings such as `"false"`, numbers, and `null` are rejected with anerror naming `explain`. Direct Python helper calls report `invalid_explain`; MCP rejects themduring argument validation. An injected router that rejects the requested `explain` keywordreports `unsupported_explain`, with no retry without the flag. Errors inside a router retainthe existing internal-error path.-The added diagnostic contains no caller **values** or `mixed_segment` text. Its bounded+The built-in detector's diagnostic contains no caller **values** or `mixed_segment` text. Its bounded`source.path` can contain caller-chosen field names; the existing routing `reason` can alreadycontain an input segment. This is not a promise that the whole response is free of caller text.Even with `LAYA_BASE_URL`, it describes routing performed by the MCP process, not the HTTPserver's actual prediction on a potentially different version or configuration. Shared-serverrouting uses no HTTP request or model. Ordinary local startup keeps its configured preload policy.+For a local check of the MCP helpers without starting a server or loading checkpoints:++```python+from laya import Router+from laya.mcp.tools import laya_route, laya_route_batch++router = Router()+request = {+ "state": {"message": "Turn lights off"},+ "questions": {"category": {"type": "choice", "instructions": "Classify the message.",+ "criteria": {"A": "Support", "B": "Other"}}},+}+single = laya_route(**request, explain=True, router=router)+print(single["diagnostic"])+# {'stage': 'whole_state', 'reason': 'insufficient_words', 'source': None}++batch = laya_route_batch([request, dict(request, lang="en")], explain=True, router=router)+assert batch["decisions"][0] == single+assert batch["decisions"][1]["diagnostic"] is None # explicit language bypasses detection+assert batch["model_counts"] == {"multilingual": 1, "english": 1}+assert not router.loaded+```+The three batch and schema tools exist because the same operations are available on the SDK and`laya-serve`: handling many requests, or serving a caller that already knows the answer shape,does not require dropping to Python. For the schema-driven form in more depth, seediff --git a/laya/mcp/server.py b/laya/mcp/server.pyindex 246810bd..04c6171f 100644--- a/laya/mcp/server.py+++ b/laya/mcp/server.py@@ -394,14 +394,16 @@ def laya_predict_batch_tool(requests: list, batch_size: int = 0, hooks_timeout:+ batch_item_key_doc(omit=("max_len", "head_max_len")) + ". Use this to ""inspect or aggregate the routing of a workload before paying model-load cost. ""Returns one {model, repo, reason} decision per request in input order, plus "- "model_counts. hooks_timeout: positive number or unset -- override the "+ "model_counts. explain=true adds each final decision's language diagnostic, or null; "+ "defaults to false. In shared-server mode this is local routing evidence, not the HTTP "+ "server's prediction. hooks_timeout: positive number or unset -- override the ""Router's own hook deadline for this sweep's on_route dispatch, so a slow ""or hung operator hook cannot stall a routing pass the caller only wants ""the checkpoint labels for. "+ _GUARDRAILS),)-def laya_route_batch_tool(requests: list, hooks_timeout: float = 0) -> str:+def laya_route_batch_tool(requests: list, hooks_timeout: float = 0, explain: StrictBool = False) -> str:"""Route many requests to checkpoints without a forward pass."""router = _router_or_error()# hooks_timeout=0 means "unset": MCP clients send defaults eagerly, and@@ -414,6 +416,7 @@ def laya_route_batch_tool(requests: list, hooks_timeout: float = 0) -> str:laya_route_batch,requests=requests,hooks_timeout=hooks_timeout,+ explain=explain,router=router,)diff --git a/laya/mcp/tools.py b/laya/mcp/tools.pyindex 72379fe6..f832f0f4 100644--- a/laya/mcp/tools.py+++ b/laya/mcp/tools.py@@ -1455,7 +1455,7 @@ def laya_predict_batch(}-def laya_route_batch(requests: Any, hooks_timeout: Any = None, *, router: Any = None) -> dict:+def laya_route_batch(requests: Any, hooks_timeout: Any = None, *, explain: Any = False, router: Any = None) -> dict:"""Routing decisions for many requests: no forward pass, no checkpoint loads.The batch form of ``laya_route``, mirroring ``Router.route_batch``: it@@ -1464,22 +1464,25 @@ def laya_route_batch(requests: Any, hooks_timeout: Any = None, *, router: Any =``hooks_timeout`` overrides the Router's own value for this call's``on_route`` dispatch, exactly as it does for ``laya_predict_batch``: anoperator-installed hook that hangs should not stall a whole routing sweep.+ ``explain=True`` adds a diagnostic (or ``None``) to each final decision,+ using the same contract as ``laya_route`` and preserving input order."""+ explain = _validate_explain(explain)items = validate_batch_requests(requests)timeout = _validate_hooks_timeout(hooks_timeout)if router is None:raise ToolError("models_not_ready", "Router is not loaded")if not hasattr(router, "route_batch"):raise ToolError("internal_error", "router has no route_batch() method")- decisions = router.route_batch(items, hooks_timeout=timeout) if timeout is not None \- else router.route_batch(items)+ overrides = {"hooks_timeout": timeout} if timeout is not None else {}+ decisions = _call_route(router.route_batch, items, explain=explain, **overrides)if not isinstance(decisions, list) or len(decisions) != len(items):raise ToolError("internal_error","route_batch returned %r decisions for %d requests" % (len(decisions) if isinstance(decisions, list) else type(decisions).__name__, len(items)),)- out = [_decision_to_dict(decision) for decision in decisions]+ out = [_route_to_dict(decision, explain=explain) for decision in decisions]model_counts: dict[str, int] = {}for entry in out:name = entry["model"]diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.pyindex 2ea1d8cf..a6e24ae6 100644--- a/tests/test_hooks_api.py+++ b/tests/test_hooks_api.py@@ -283,9 +283,10 @@ for label, fn in (("analyse", analyse), ("Router.route", Router.route),check_param(label, fn, "explain", False, inspect.Parameter.KEYWORD_ONLY)check("latin_profile keeps its signature", list(sig(latin_profile)), ["text"])-from laya.mcp.tools import laya_route # noqa: E402+from laya.mcp.tools import laya_route, laya_route_batch # noqa: E402check_param("laya_route", laya_route, "explain", False, inspect.Parameter.KEYWORD_ONLY)+check_param("laya_route_batch", laya_route_batch, "explain", False, inspect.Parameter.KEYWORD_ONLY)_plain_analysis = analyse("Turn lights off")_explained_analysis = analyse("Turn lights off", explain=True)check("analyse opts into one extra key", set(_explained_analysis) - set(_plain_analysis), {"diagnostic"})diff --git a/tests/test_mcp_diagnostics.py b/tests/test_mcp_diagnostics.pyindex 92a5dd98..13e829d5 100644--- a/tests/test_mcp_diagnostics.py+++ b/tests/test_mcp_diagnostics.py@@ -11,7 +11,9 @@ pytest.importorskip("mcp")from mcp.server.mcpserver.exceptions import ToolError as McpToolError # noqa: E402from laya import BaseHook, Router # noqa: E402from laya.mcp import server as server_mod # noqa: E402-from laya.mcp.tools import ToolError, laya_predict, laya_route, laya_tournament # noqa: E402+from laya.mcp.tools import ( # noqa: E402+ ToolError, laya_predict, laya_route, laya_route_batch, laya_tournament,+)STATE = {"message": "Turn lights off"}@@ -217,3 +219,178 @@ def test_real_mcp_parser_rejects_non_boolean_without_dispatch(bad, monkeypatch):with pytest.raises(McpToolError, match="explain"):_call_tool({"state": STATE, "questions": QUESTIONS, "explain": bad})assert router.calls == []+++class LegacyBatchRouter:+ def __init__(self):+ self.batch_calls = []++ def route_batch(self, requests):+ self.batch_calls.append(requests)+ return [dict(ROUTING) for _ in requests]+++class RecordingBatchRouter(RecordingRouter):+ def __init__(self, **kwargs):+ super().__init__(**kwargs)+ self.batch_calls = []++ def route_batch(self, requests, **kwargs):+ self.batch_calls.append((requests, dict(kwargs)))+ return super().route_batch(requests, **kwargs)+++def _batch_requests():+ return [{"state": copy.deepcopy(STATE), "questions": copy.deepcopy(QUESTIONS)},+ {"state": {"message": "xyzzy frobnitz quux blorb"}, "questions": copy.deepcopy(QUESTIONS)}]+++def _call_batch_tool(arguments):+ result = asyncio.run(server_mod.server.call_tool("laya_route_batch", arguments))+ assert not result.is_error+ return json.loads(result.content[0].text)++[email protected]("flag", [{}, {"explain": False}])+def test_batch_default_preserves_legacy_router_and_exact_output(flag):+ router = LegacyBatchRouter()+ out = laya_route_batch(_batch_requests(), router=router, **flag)+ expected = {"decisions": [dict(ROUTING), dict(ROUTING)], "model_counts": {"english": 2}}+ assert json.dumps(out) == json.dumps(expected)+ assert len(router.batch_calls) == 1+++def test_batch_diagnostics_preserve_order_overrides_and_model_counts():+ router = RecordingBatchRouter()+ requests = _batch_requests()+ requests[1]["lang"] = "de"+ requests.append({"state": copy.deepcopy(STATE), "questions": copy.deepcopy(QUESTIONS), "model": "english"})+ requests.append({"state": {"note": "This is an English note about the order. " * 20,+ "payload": {"customer": "Ich kann mich nicht anmelden und brauche Hilfe"}},+ "questions": copy.deepcopy(QUESTIONS)})+ original = copy.deepcopy(requests)+ plain = laya_route_batch(requests, router=router)+ explained = laya_route_batch(requests, router=router, explain=True)+ assert [d.pop("diagnostic") for d in explained["decisions"]] == [+ DIAGNOSTIC, None, None,+ {"stage": "segment", "reason": "named_segment",+ "source": {"leaf_index": 1, "path": ["payload", "customer"], "line": 1}},+ ]+ assert explained == plain+ assert [d["model"] for d in explained["decisions"]] == ["multilingual", "multilingual", "english", "multilingual"]+ assert explained["model_counts"] == {"multilingual": 3, "english": 1}+ assert router.batch_calls[0][1] == {}+ assert router.batch_calls[1][1] == {"explain": True}+ assert len(router.calls) == 2 * len(requests)+ assert requests == original+++def test_batch_uses_each_final_hook_decision_and_preserves_hooks_timeout():+ class ReplaceSecondRoute(BaseHook):+ def on_route(self, ctx):+ if ctx.states[0]["message"].startswith("xyzzy"):+ ctx.decision = dict(ROUTING)++ router = RecordingBatchRouter(hooks=[ReplaceSecondRoute()])+ out = laya_route_batch(_batch_requests(), hooks_timeout=2.5, router=router, explain=True)+ assert out["decisions"][0]["diagnostic"] == DIAGNOSTIC+ assert out["decisions"][1] == dict(ROUTING, diagnostic=None)+ assert out["model_counts"] == {"multilingual": 1, "english": 1}+ assert router.batch_calls[0][1] == {"hooks_timeout": 2.5, "explain": True}+ assert len(router.batch_calls) == 1+ assert len(router.calls) == 2+ assert all(call[2]["hooks_timeout"] == 2.5 for call in router.calls)+ assert all(call[2]["explain"] is True for call in router.calls)++[email protected]("bad", [None, 0, "false"])+def test_batch_rejects_non_boolean_before_dispatch(bad):+ router = RecordingBatchRouter()+ with pytest.raises(ToolError, match="explain") as info:+ laya_route_batch(_batch_requests(), router=router, explain=bad)+ assert info.value.code == "invalid_explain"+ assert router.batch_calls == []+ assert router.calls == []+++def test_batch_legacy_router_refuses_explanation_without_fallback():+ router = LegacyBatchRouter()+ with pytest.raises(ToolError, match="explain") as info:+ laya_route_batch(_batch_requests(), router=router, explain=True)+ assert info.value.code == "unsupported_explain"+ assert router.batch_calls == []+++def test_batch_internal_type_error_is_not_relabelled_or_retried():+ error = TypeError("wrapped.route_batch() got an unexpected keyword argument 'explain'")++ class BrokenBatchRouter:+ calls = 0++ def route_batch(self, requests, **kwargs):+ self.calls += 1+ raise error++ router = BrokenBatchRouter()+ with pytest.raises(TypeError) as info:+ laya_route_batch(_batch_requests(), router=router, explain=True)+ assert info.value is error+ assert router.calls == 1++[email protected]("decisions", [[], [dict(ROUTING)], tuple([dict(ROUTING)] * 2)])+def test_explained_batch_preserves_result_cardinality_validation(decisions):+ class InvalidBatchRouter:+ def route_batch(self, requests, *, explain=False):+ assert explain is True+ return decisions++ with pytest.raises(ToolError) as info:+ laya_route_batch(_batch_requests(), router=InvalidBatchRouter(), explain=True)+ assert info.value.code == "internal_error"+++def test_explained_batch_preserves_timeout_validation_before_dispatch():+ router = RecordingBatchRouter()+ with pytest.raises(ToolError) as info:+ laya_route_batch(_batch_requests(), hooks_timeout=True, router=router, explain=True)+ assert info.value.code == "invalid_hooks_timeout"+ assert router.batch_calls == []+++def test_mcp_batch_schema_exposes_optional_boolean_explain():+ tool = next(t for t in asyncio.run(server_mod.server.list_tools()) if t.name == "laya_route_batch")+ schema = tool.input_schema+ assert schema["properties"]["explain"]["type"] == "boolean"+ assert schema["properties"]["explain"]["default"] is False+ assert schema["required"] == ["requests"]++[email protected](("flag", "timeout", "expected_kwargs"), [+ ({}, {}, {}),+ ({"explain": False}, {}, {}),+ ({"explain": True}, {}, {"explain": True}),+ ({"explain": True}, {"hooks_timeout": 2.5}, {"explain": True, "hooks_timeout": 2.5}),+])+def test_real_mcp_batch_calls_honor_explain_and_timeout(flag, timeout, expected_kwargs, monkeypatch):+ router = RecordingBatchRouter()+ monkeypatch.setattr(server_mod, "_ROUTER", router)+ out = _call_batch_tool({"requests": _batch_requests(), **flag, **timeout})+ if flag.get("explain", False):+ assert [d["diagnostic"]["reason"] for d in out["decisions"]] == ["insufficient_words", "no_language_evidence"]+ else:+ assert all(set(d) == {"model", "repo", "reason"} for d in out["decisions"])+ assert out["model_counts"] == {"multilingual": 2}+ assert len(router.batch_calls) == 1+ assert router.batch_calls[0][1] == expected_kwargs+ assert len(router.calls) == 2++[email protected]("bad", [None, 1, "false"])+def test_real_mcp_batch_parser_rejects_non_boolean_before_dispatch(bad, monkeypatch):+ router = RecordingBatchRouter()+ monkeypatch.setattr(server_mod, "_ROUTER", router)+ with pytest.raises(McpToolError, match="explain"):+ _call_batch_tool({"requests": _batch_requests(), "explain": bad})+ assert router.batch_calls == []+ assert router.calls == []diff --git a/tests/test_mcp_remote.py b/tests/test_mcp_remote.pyindex a9c06e19..d9452427 100644--- a/tests/test_mcp_remote.py+++ b/tests/test_mcp_remote.py@@ -402,11 +402,17 @@ def test_remote_decide_and_preset_use_the_server(serve):assert info.value.code == "busy"-def test_remote_route_tools_do_not_contact_the_server(serve):[email protected]("explain", [False, True])+def test_remote_route_tools_do_not_contact_the_server(serve, explain):router = RemoteRouter(serve)state = {"text": "Mein Konto wurde zweimal belastet"}- assert laya_route(state, _questions(), router=router)["model"] == "multilingual"- assert laya_route_batch([{"state": state, "questions": _questions()}], router=router)["decisions"][0]["model"] == "multilingual"+ single = laya_route(state, _questions(), router=router, explain=explain)+ batch = laya_route_batch([{"state": state, "questions": _questions()}], router=router, explain=explain)+ assert single["model"] == "multilingual"+ assert batch == {"decisions": [single], "model_counts": {"multilingual": 1}}+ assert ("diagnostic" in single) is explain+ if explain:+ assert set(single["diagnostic"]) == {"stage", "reason", "source"}assert not FakeServe.requests@@ -448,6 +454,18 @@ def test_remote_mcp_startup_and_every_tool_avoid_torch(serve):for call in calls:out = json.loads(call())assert "error" not in out, out+ # Exercise actual MCP parsing, with both model imports and HTTP forbidden.+ import asyncio+ def no_http(*args, **kwargs):+ raise AssertionError("routing diagnostics attempted an HTTP request")+ server._ROUTER._request = no_http+ for name, arguments in (("laya_route", {"state": state, "questions": questions}),+ ("laya_route_batch", {"requests": items})):+ result = asyncio.run(server.server.call_tool(name, dict(arguments, explain=True)))+ assert not result.is_error, result+ out = json.loads(result.content[0].text)+ decisions = out["decisions"] if name == "laya_route_batch" else [out]+ assert all(set(d["diagnostic"]) == {"stage", "reason", "source"} for d in decisions)for tool in (server.laya_shortlist_tool, server.laya_tournament_tool):try:tool(state, questions)