Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(mcp): extend routing diagnostics to batches

xp78955db5e127
diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md
index ec4c1f55..365080ed 100644
--- a/docs/cli-mcp.md
+++ b/docs/cli-mcp.md
@@ -166,7 +166,7 @@ Laya does not open a network port.
| `laya_tournament` | Narrows large choices in balanced rounds using the decision model, then answers the complete request. In-process only. | `state`, `questions`, optional `model`, `group_size` (default `16`), `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence` |
| `laya_preset` | Runs a built-in workflow using its built-in question set. | `preset`, `state`, optional `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence` |
| `laya_predict_batch` | Answers many requests in one call. Requests are routed first and grouped by checkpoint, so matching question schemas share forward passes; answers come back in input order. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?, max_len?, head_max_len?}`, optional `batch_size`, `hooks_timeout`, `min_confidence`, `sort_by_length` |
-| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout` |
+| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. Opt into per-decision diagnostics with `explain`. | `requests`, each `{state, questions, model?, task?, lang?, lang_guess?}`, optional `hooks_timeout`, `explain` (default `false`, for the whole batch) |
| `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model`, `min_confidence` |
| `laya_predict_long` | Scans a state longer than the context window in overlapping windows, then answers with per-answer window attribution and the window count. | `state`, `questions`, optional `model`, `window`, `stride`, `task`, `lang` |
@@ -176,20 +176,45 @@ On `laya_route`, `laya_predict`, `laya_shortlist`, `laya_tournament`, and `laya_
`stage`, `reason`, and `source`. When no source exists, `source` is `null`. When the final routing
decision has no diagnostic (for example, an explicit model or language override, or a hook
replacement), `diagnostic` is `null`. The diagnostic comes from that routing call; it does not
-run detection again after a hook. Omitted or `false` keeps the existing three-field result.
+run detection again after a hook. `laya_route_batch` takes the same flag once for the whole
+batch, adds `diagnostic` to each decision, and preserves input order and `model_counts`.
+Omitted or `false` keeps each existing three-field decision.
Only booleans are accepted: strings such as `"false"`, numbers, and `null` are rejected with an
error naming `explain`. Direct Python helper calls report `invalid_explain`; MCP rejects them
during argument validation. An injected router that rejects the requested `explain` keyword
reports `unsupported_explain`, with no retry without the flag. Errors inside a router retain
the existing internal-error path.
-The added diagnostic contains no caller **values** or `mixed_segment` text. Its bounded
+The built-in detector's diagnostic contains no caller **values** or `mixed_segment` text. Its bounded
`source.path` can contain caller-chosen field names; the existing routing `reason` can already
contain an input segment. This is not a promise that the whole response is free of caller text.
Even with `LAYA_BASE_URL`, it describes routing performed by the MCP process, not the HTTP
server's actual prediction on a potentially different version or configuration. Shared-server
routing uses no HTTP request or model. Ordinary local startup keeps its configured preload policy.
+For a local check of the MCP helpers without starting a server or loading checkpoints:
+
+```python
+from laya import Router
+from laya.mcp.tools import laya_route, laya_route_batch
+
+router = Router()
+request = {
+ "state": {"message": "Turn lights off"},
+ "questions": {"category": {"type": "choice", "instructions": "Classify the message.",
+ "criteria": {"A": "Support", "B": "Other"}}},
+}
+single = laya_route(**request, explain=True, router=router)
+print(single["diagnostic"])
+# {'stage': 'whole_state', 'reason': 'insufficient_words', 'source': None}
+
+batch = laya_route_batch([request, dict(request, lang="en")], explain=True, router=router)
+assert batch["decisions"][0] == single
+assert batch["decisions"][1]["diagnostic"] is None # explicit language bypasses detection
+assert batch["model_counts"] == {"multilingual": 1, "english": 1}
+assert not router.loaded
+```
+
The three batch and schema tools exist because the same operations are available on the SDK and
`laya-serve`: handling many requests, or serving a caller that already knows the answer shape,
does not require dropping to Python. For the schema-driven form in more depth, see
diff --git a/laya/mcp/server.py b/laya/mcp/server.py
index 246810bd..04c6171f 100644
--- a/laya/mcp/server.py
+++ b/laya/mcp/server.py
@@ -394,14 +394,16 @@ def laya_predict_batch_tool(requests: list, batch_size: int = 0, hooks_timeout:
+ batch_item_key_doc(omit=("max_len", "head_max_len")) + ". Use this to "
"inspect or aggregate the routing of a workload before paying model-load cost. "
"Returns one {model, repo, reason} decision per request in input order, plus "
- "model_counts. hooks_timeout: positive number or unset -- override the "
+ "model_counts. explain=true adds each final decision's language diagnostic, or null; "
+ "defaults to false. In shared-server mode this is local routing evidence, not the HTTP "
+ "server's prediction. hooks_timeout: positive number or unset -- override the "
"Router's own hook deadline for this sweep's on_route dispatch, so a slow "
"or hung operator hook cannot stall a routing pass the caller only wants "
"the checkpoint labels for. "
+ _GUARDRAILS
),
)
-def laya_route_batch_tool(requests: list, hooks_timeout: float = 0) -> str:
+def laya_route_batch_tool(requests: list, hooks_timeout: float = 0, explain: StrictBool = False) -> str:
"""Route many requests to checkpoints without a forward pass."""
router = _router_or_error()
# hooks_timeout=0 means "unset": MCP clients send defaults eagerly, and
@@ -414,6 +416,7 @@ def laya_route_batch_tool(requests: list, hooks_timeout: float = 0) -> str:
laya_route_batch,
requests=requests,
hooks_timeout=hooks_timeout,
+ explain=explain,
router=router,
)
diff --git a/laya/mcp/tools.py b/laya/mcp/tools.py
index 72379fe6..f832f0f4 100644
--- a/laya/mcp/tools.py
+++ b/laya/mcp/tools.py
@@ -1455,7 +1455,7 @@ def laya_predict_batch(
}
-def laya_route_batch(requests: Any, hooks_timeout: Any = None, *, router: Any = None) -> dict:
+def laya_route_batch(requests: Any, hooks_timeout: Any = None, *, explain: Any = False, router: Any = None) -> dict:
"""Routing decisions for many requests: no forward pass, no checkpoint loads.
The batch form of ``laya_route``, mirroring ``Router.route_batch``: it
@@ -1464,22 +1464,25 @@ def laya_route_batch(requests: Any, hooks_timeout: Any = None, *, router: Any =
``hooks_timeout`` overrides the Router's own value for this call's
``on_route`` dispatch, exactly as it does for ``laya_predict_batch``: an
operator-installed hook that hangs should not stall a whole routing sweep.
+ ``explain=True`` adds a diagnostic (or ``None``) to each final decision,
+ using the same contract as ``laya_route`` and preserving input order.
"""
+ explain = _validate_explain(explain)
items = validate_batch_requests(requests)
timeout = _validate_hooks_timeout(hooks_timeout)
if router is None:
raise ToolError("models_not_ready", "Router is not loaded")
if not hasattr(router, "route_batch"):
raise ToolError("internal_error", "router has no route_batch() method")
- decisions = router.route_batch(items, hooks_timeout=timeout) if timeout is not None \
- else router.route_batch(items)
+ overrides = {"hooks_timeout": timeout} if timeout is not None else {}
+ decisions = _call_route(router.route_batch, items, explain=explain, **overrides)
if not isinstance(decisions, list) or len(decisions) != len(items):
raise ToolError(
"internal_error",
"route_batch returned %r decisions for %d requests" % (
len(decisions) if isinstance(decisions, list) else type(decisions).__name__, len(items)),
)
- out = [_decision_to_dict(decision) for decision in decisions]
+ out = [_route_to_dict(decision, explain=explain) for decision in decisions]
model_counts: dict[str, int] = {}
for entry in out:
name = entry["model"]
diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.py
index 2ea1d8cf..a6e24ae6 100644
--- a/tests/test_hooks_api.py
+++ b/tests/test_hooks_api.py
@@ -283,9 +283,10 @@ for label, fn in (("analyse", analyse), ("Router.route", Router.route),
check_param(label, fn, "explain", False, inspect.Parameter.KEYWORD_ONLY)
check("latin_profile keeps its signature", list(sig(latin_profile)), ["text"])
-from laya.mcp.tools import laya_route # noqa: E402
+from laya.mcp.tools import laya_route, laya_route_batch # noqa: E402
check_param("laya_route", laya_route, "explain", False, inspect.Parameter.KEYWORD_ONLY)
+check_param("laya_route_batch", laya_route_batch, "explain", False, inspect.Parameter.KEYWORD_ONLY)
_plain_analysis = analyse("Turn lights off")
_explained_analysis = analyse("Turn lights off", explain=True)
check("analyse opts into one extra key", set(_explained_analysis) - set(_plain_analysis), {"diagnostic"})
diff --git a/tests/test_mcp_diagnostics.py b/tests/test_mcp_diagnostics.py
index 92a5dd98..13e829d5 100644
--- a/tests/test_mcp_diagnostics.py
+++ b/tests/test_mcp_diagnostics.py
@@ -11,7 +11,9 @@ pytest.importorskip("mcp")
from mcp.server.mcpserver.exceptions import ToolError as McpToolError # noqa: E402
from laya import BaseHook, Router # noqa: E402
from laya.mcp import server as server_mod # noqa: E402
-from laya.mcp.tools import ToolError, laya_predict, laya_route, laya_tournament # noqa: E402
+from laya.mcp.tools import ( # noqa: E402
+ ToolError, laya_predict, laya_route, laya_route_batch, laya_tournament,
+)
STATE = {"message": "Turn lights off"}
@@ -217,3 +219,178 @@ def test_real_mcp_parser_rejects_non_boolean_without_dispatch(bad, monkeypatch):
with pytest.raises(McpToolError, match="explain"):
_call_tool({"state": STATE, "questions": QUESTIONS, "explain": bad})
assert router.calls == []
+
+
+class LegacyBatchRouter:
+ def __init__(self):
+ self.batch_calls = []
+
+ def route_batch(self, requests):
+ self.batch_calls.append(requests)
+ return [dict(ROUTING) for _ in requests]
+
+
+class RecordingBatchRouter(RecordingRouter):
+ def __init__(self, **kwargs):
+ super().__init__(**kwargs)
+ self.batch_calls = []
+
+ def route_batch(self, requests, **kwargs):
+ self.batch_calls.append((requests, dict(kwargs)))
+ return super().route_batch(requests, **kwargs)
+
+
+def _batch_requests():
+ return [{"state": copy.deepcopy(STATE), "questions": copy.deepcopy(QUESTIONS)},
+ {"state": {"message": "xyzzy frobnitz quux blorb"}, "questions": copy.deepcopy(QUESTIONS)}]
+
+
+def _call_batch_tool(arguments):
+ result = asyncio.run(server_mod.server.call_tool("laya_route_batch", arguments))
+ assert not result.is_error
+ return json.loads(result.content[0].text)
+
+
[email protected]("flag", [{}, {"explain": False}])
+def test_batch_default_preserves_legacy_router_and_exact_output(flag):
+ router = LegacyBatchRouter()
+ out = laya_route_batch(_batch_requests(), router=router, **flag)
+ expected = {"decisions": [dict(ROUTING), dict(ROUTING)], "model_counts": {"english": 2}}
+ assert json.dumps(out) == json.dumps(expected)
+ assert len(router.batch_calls) == 1
+
+
+def test_batch_diagnostics_preserve_order_overrides_and_model_counts():
+ router = RecordingBatchRouter()
+ requests = _batch_requests()
+ requests[1]["lang"] = "de"
+ requests.append({"state": copy.deepcopy(STATE), "questions": copy.deepcopy(QUESTIONS), "model": "english"})
+ requests.append({"state": {"note": "This is an English note about the order. " * 20,
+ "payload": {"customer": "Ich kann mich nicht anmelden und brauche Hilfe"}},
+ "questions": copy.deepcopy(QUESTIONS)})
+ original = copy.deepcopy(requests)
+ plain = laya_route_batch(requests, router=router)
+ explained = laya_route_batch(requests, router=router, explain=True)
+ assert [d.pop("diagnostic") for d in explained["decisions"]] == [
+ DIAGNOSTIC, None, None,
+ {"stage": "segment", "reason": "named_segment",
+ "source": {"leaf_index": 1, "path": ["payload", "customer"], "line": 1}},
+ ]
+ assert explained == plain
+ assert [d["model"] for d in explained["decisions"]] == ["multilingual", "multilingual", "english", "multilingual"]
+ assert explained["model_counts"] == {"multilingual": 3, "english": 1}
+ assert router.batch_calls[0][1] == {}
+ assert router.batch_calls[1][1] == {"explain": True}
+ assert len(router.calls) == 2 * len(requests)
+ assert requests == original
+
+
+def test_batch_uses_each_final_hook_decision_and_preserves_hooks_timeout():
+ class ReplaceSecondRoute(BaseHook):
+ def on_route(self, ctx):
+ if ctx.states[0]["message"].startswith("xyzzy"):
+ ctx.decision = dict(ROUTING)
+
+ router = RecordingBatchRouter(hooks=[ReplaceSecondRoute()])
+ out = laya_route_batch(_batch_requests(), hooks_timeout=2.5, router=router, explain=True)
+ assert out["decisions"][0]["diagnostic"] == DIAGNOSTIC
+ assert out["decisions"][1] == dict(ROUTING, diagnostic=None)
+ assert out["model_counts"] == {"multilingual": 1, "english": 1}
+ assert router.batch_calls[0][1] == {"hooks_timeout": 2.5, "explain": True}
+ assert len(router.batch_calls) == 1
+ assert len(router.calls) == 2
+ assert all(call[2]["hooks_timeout"] == 2.5 for call in router.calls)
+ assert all(call[2]["explain"] is True for call in router.calls)
+
+
[email protected]("bad", [None, 0, "false"])
+def test_batch_rejects_non_boolean_before_dispatch(bad):
+ router = RecordingBatchRouter()
+ with pytest.raises(ToolError, match="explain") as info:
+ laya_route_batch(_batch_requests(), router=router, explain=bad)
+ assert info.value.code == "invalid_explain"
+ assert router.batch_calls == []
+ assert router.calls == []
+
+
+def test_batch_legacy_router_refuses_explanation_without_fallback():
+ router = LegacyBatchRouter()
+ with pytest.raises(ToolError, match="explain") as info:
+ laya_route_batch(_batch_requests(), router=router, explain=True)
+ assert info.value.code == "unsupported_explain"
+ assert router.batch_calls == []
+
+
+def test_batch_internal_type_error_is_not_relabelled_or_retried():
+ error = TypeError("wrapped.route_batch() got an unexpected keyword argument 'explain'")
+
+ class BrokenBatchRouter:
+ calls = 0
+
+ def route_batch(self, requests, **kwargs):
+ self.calls += 1
+ raise error
+
+ router = BrokenBatchRouter()
+ with pytest.raises(TypeError) as info:
+ laya_route_batch(_batch_requests(), router=router, explain=True)
+ assert info.value is error
+ assert router.calls == 1
+
+
[email protected]("decisions", [[], [dict(ROUTING)], tuple([dict(ROUTING)] * 2)])
+def test_explained_batch_preserves_result_cardinality_validation(decisions):
+ class InvalidBatchRouter:
+ def route_batch(self, requests, *, explain=False):
+ assert explain is True
+ return decisions
+
+ with pytest.raises(ToolError) as info:
+ laya_route_batch(_batch_requests(), router=InvalidBatchRouter(), explain=True)
+ assert info.value.code == "internal_error"
+
+
+def test_explained_batch_preserves_timeout_validation_before_dispatch():
+ router = RecordingBatchRouter()
+ with pytest.raises(ToolError) as info:
+ laya_route_batch(_batch_requests(), hooks_timeout=True, router=router, explain=True)
+ assert info.value.code == "invalid_hooks_timeout"
+ assert router.batch_calls == []
+
+
+def test_mcp_batch_schema_exposes_optional_boolean_explain():
+ tool = next(t for t in asyncio.run(server_mod.server.list_tools()) if t.name == "laya_route_batch")
+ schema = tool.input_schema
+ assert schema["properties"]["explain"]["type"] == "boolean"
+ assert schema["properties"]["explain"]["default"] is False
+ assert schema["required"] == ["requests"]
+
+
[email protected](("flag", "timeout", "expected_kwargs"), [
+ ({}, {}, {}),
+ ({"explain": False}, {}, {}),
+ ({"explain": True}, {}, {"explain": True}),
+ ({"explain": True}, {"hooks_timeout": 2.5}, {"explain": True, "hooks_timeout": 2.5}),
+])
+def test_real_mcp_batch_calls_honor_explain_and_timeout(flag, timeout, expected_kwargs, monkeypatch):
+ router = RecordingBatchRouter()
+ monkeypatch.setattr(server_mod, "_ROUTER", router)
+ out = _call_batch_tool({"requests": _batch_requests(), **flag, **timeout})
+ if flag.get("explain", False):
+ assert [d["diagnostic"]["reason"] for d in out["decisions"]] == ["insufficient_words", "no_language_evidence"]
+ else:
+ assert all(set(d) == {"model", "repo", "reason"} for d in out["decisions"])
+ assert out["model_counts"] == {"multilingual": 2}
+ assert len(router.batch_calls) == 1
+ assert router.batch_calls[0][1] == expected_kwargs
+ assert len(router.calls) == 2
+
+
[email protected]("bad", [None, 1, "false"])
+def test_real_mcp_batch_parser_rejects_non_boolean_before_dispatch(bad, monkeypatch):
+ router = RecordingBatchRouter()
+ monkeypatch.setattr(server_mod, "_ROUTER", router)
+ with pytest.raises(McpToolError, match="explain"):
+ _call_batch_tool({"requests": _batch_requests(), "explain": bad})
+ assert router.batch_calls == []
+ assert router.calls == []
diff --git a/tests/test_mcp_remote.py b/tests/test_mcp_remote.py
index a9c06e19..d9452427 100644
--- a/tests/test_mcp_remote.py
+++ b/tests/test_mcp_remote.py
@@ -402,11 +402,17 @@ def test_remote_decide_and_preset_use_the_server(serve):
assert info.value.code == "busy"
-def test_remote_route_tools_do_not_contact_the_server(serve):
[email protected]("explain", [False, True])
+def test_remote_route_tools_do_not_contact_the_server(serve, explain):
router = RemoteRouter(serve)
state = {"text": "Mein Konto wurde zweimal belastet"}
- assert laya_route(state, _questions(), router=router)["model"] == "multilingual"
- assert laya_route_batch([{"state": state, "questions": _questions()}], router=router)["decisions"][0]["model"] == "multilingual"
+ single = laya_route(state, _questions(), router=router, explain=explain)
+ batch = laya_route_batch([{"state": state, "questions": _questions()}], router=router, explain=explain)
+ assert single["model"] == "multilingual"
+ assert batch == {"decisions": [single], "model_counts": {"multilingual": 1}}
+ assert ("diagnostic" in single) is explain
+ if explain:
+ assert set(single["diagnostic"]) == {"stage", "reason", "source"}
assert not FakeServe.requests
@@ -448,6 +454,18 @@ def test_remote_mcp_startup_and_every_tool_avoid_torch(serve):
for call in calls:
out = json.loads(call())
assert "error" not in out, out
+ # Exercise actual MCP parsing, with both model imports and HTTP forbidden.
+ import asyncio
+ def no_http(*args, **kwargs):
+ raise AssertionError("routing diagnostics attempted an HTTP request")
+ server._ROUTER._request = no_http
+ for name, arguments in (("laya_route", {"state": state, "questions": questions}),
+ ("laya_route_batch", {"requests": items})):
+ result = asyncio.run(server.server.call_tool(name, dict(arguments, explain=True)))
+ assert not result.is_error, result
+ out = json.loads(result.content[0].text)
+ decisions = out["decisions"] if name == "laya_route_batch" else [out]
+ assert all(set(d["diagnostic"]) == {"stage", "reason", "source"} for d in decisions)
for tool in (server.laya_shortlist_tool, server.laya_tournament_tool):
try:
tool(state, questions)