Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(mcp): expose opt-in diagnostics for single routing calls

xpd515f4ed92db
diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md
index b9d8183e..ec4c1f55 100644
--- a/docs/cli-mcp.md
+++ b/docs/cli-mcp.md
@@ -160,7 +160,7 @@ Laya does not open a network port.
| Tool | What it does | Main inputs |
|---|---|---|
| `laya_status` | Reports the device per loaded checkpoint (`checkpoint_devices`), the configured or actual device, CUDA availability, loaded checkpoints, preload state, readiness, and package versions. | none |
-| `laya_route` | Selects a checkpoint and returns its model, repository, and reason without running a forward pass. | `state`, `questions`, optional `model`, `task`, `lang`, `lang_guess` |
+| `laya_route` | Selects a checkpoint and returns its model, repository, and reason without running a forward pass. Opt into language diagnostics with `explain`. | `state`, `questions`, optional `model`, `task`, `lang`, `lang_guess`, `explain` (default `false`) |
| `laya_predict` | Runs typed questions and returns answers, routing metadata, latency, and the answering device when readable. | `state`, `questions`, optional `model` (`auto`, `english`, `multilingual`, or `typed-decisions`), `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence` |
| `laya_shortlist` | Shortlists a many-option choice question, then answers it and returns the shortlist metadata. | `state`, `questions`, optional `model`, `k` (default `20`), `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence` |
| `laya_tournament` | Narrows large choices in balanced rounds using the decision model, then answers the complete request. In-process only. | `state`, `questions`, optional `model`, `group_size` (default `16`), `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence` |
@@ -172,6 +172,24 @@ Laya does not open a network port.
On `laya_route`, `laya_predict`, `laya_shortlist`, `laya_tournament`, and `laya_decide`, `model` may be omitted or set to `null`; either form resolves to `auto`.
+`laya_route` accepts `explain: true` to add a top-level `diagnostic` with the core detector's
+`stage`, `reason`, and `source`. When no source exists, `source` is `null`. When the final routing
+decision has no diagnostic (for example, an explicit model or language override, or a hook
+replacement), `diagnostic` is `null`. The diagnostic comes from that routing call; it does not
+run detection again after a hook. Omitted or `false` keeps the existing three-field result.
+Only booleans are accepted: strings such as `"false"`, numbers, and `null` are rejected with an
+error naming `explain`. Direct Python helper calls report `invalid_explain`; MCP rejects them
+during argument validation. An injected router that rejects the requested `explain` keyword
+reports `unsupported_explain`, with no retry without the flag. Errors inside a router retain
+the existing internal-error path.
+
+The added diagnostic contains no caller **values** or `mixed_segment` text. Its bounded
+`source.path` can contain caller-chosen field names; the existing routing `reason` can already
+contain an input segment. This is not a promise that the whole response is free of caller text.
+Even with `LAYA_BASE_URL`, it describes routing performed by the MCP process, not the HTTP
+server's actual prediction on a potentially different version or configuration. Shared-server
+routing uses no HTTP request or model. Ordinary local startup keeps its configured preload policy.
+
The three batch and schema tools exist because the same operations are available on the SDK and
`laya-serve`: handling many requests, or serving a caller that already knows the answer shape,
does not require dropping to Python. For the schema-driven form in more depth, see
diff --git a/laya/mcp/server.py b/laya/mcp/server.py
index 1c0d01ba..246810bd 100644
--- a/laya/mcp/server.py
+++ b/laya/mcp/server.py
@@ -47,6 +47,7 @@ from typing import Any
try:
from mcp.server.mcpserver import MCPServer
from mcp.server.mcpserver.exceptions import ToolError as McpToolError
+ from pydantic import StrictBool
except ImportError as exc: # mcp extra not installed
raise ImportError(
"the laya[mcp] extra (mcp>=2.2.0) is required to run the MCP server: "
@@ -264,7 +265,8 @@ def laya_status_tool() -> str:
"Use this to explain routing (english vs multilingual vs typed-decisions) to the user. "
"Passing a laya_predict call's model/task/lang/lang_guess here reproduces the routing block it "
"reported, without paying for the forward pass; leaving model unset (or 'auto') routes as "
- "normal. "
+ "normal. explain=true adds the final routing decision's language diagnostic, or null; "
+ "it describes this MCP process's routing even in remote mode. "
+ _GUARDRAILS
+ _CONTROLS_DOC
),
@@ -276,6 +278,7 @@ def laya_route_tool(
task: str | None = None,
lang: str | None = None,
lang_guess: str | None = None,
+ explain: StrictBool = False,
) -> str:
"""Decide which Laya checkpoint would answer, without running a forward pass."""
router = _router_or_error()
@@ -287,6 +290,7 @@ def laya_route_tool(
task=task,
lang=lang,
lang_guess=lang_guess,
+ explain=explain,
router=router,
)
diff --git a/laya/mcp/tools.py b/laya/mcp/tools.py
index b81368ef..72379fe6 100644
--- a/laya/mcp/tools.py
+++ b/laya/mcp/tools.py
@@ -536,6 +536,34 @@ def _decision_to_dict(decision: Any) -> dict:
}
+def _validate_explain(explain: Any) -> bool:
+ if not isinstance(explain, bool):
+ raise ToolError("invalid_explain", "explain must be a boolean")
+ return explain
+
+
+def _call_route(route: Callable, *args, explain: bool, **kwargs) -> Any:
+ if not explain:
+ return route(*args, **kwargs)
+ try:
+ return route(*args, explain=True, **kwargs)
+ except TypeError as exc:
+ # Only a keyword binding refusal, not a TypeError raised inside the router.
+ # Never retry without the explanation the caller explicitly requested.
+ if exc.__traceback__.tb_next is None and "unexpected keyword argument 'explain'" in str(exc):
+ raise ToolError("unsupported_explain", "router does not support explain=True") from exc
+ raise
+
+
+def _route_to_dict(decision: Any, *, explain: bool) -> dict:
+ # Predict tools also use _decision_to_dict; only routing tools expose this opt-in field.
+ out = _decision_to_dict(decision)
+ if explain:
+ detection = decision.get("detection") if isinstance(decision, dict) else getattr(decision, "detection", None)
+ out["diagnostic"] = detection.get("diagnostic") if isinstance(detection, dict) else None
+ return out
+
+
def laya_route(
state: Any,
questions: Any,
@@ -544,6 +572,7 @@ def laya_route(
task: Any = None,
lang: Any = None,
lang_guess: Any = None,
+ explain: Any = False,
router: Any = None,
) -> dict:
"""Routing decision only: no forward pass.
@@ -553,7 +582,11 @@ def laya_route(
running anything.
Passing a predict call's controls here reproduces the ``routing`` block it returned, which is
what makes a route a cheap explanation of a decision rather than a different decision.
+
+ ``explain=True`` adds the final decision's built-in detection diagnostic, or ``None`` when
+ none exists. It never re-runs detection or exposes the rest of the detection payload.
"""
+ explain = _validate_explain(explain)
state_d = validate_state(state)
questions_d = validate_questions(questions)
# `auto` is this layer's word for "do not pin"; core has no such name, so it becomes an absent
@@ -589,7 +622,8 @@ def laya_route(
raise ToolError("models_not_ready", "Router is not loaded")
if not hasattr(router, "route"):
raise ToolError("internal_error", "router has no route() method")
- return _decision_to_dict(router.route(state_d, questions_d, **overrides))
+ decision = _call_route(router.route, state_d, questions_d, explain=explain, **overrides)
+ return _route_to_dict(decision, explain=explain)
# Where the reusable shortlist embedder lives. Kept private and on the agent, so the
diff --git a/scripts/test_suites.py b/scripts/test_suites.py
index a720e352..0870e0ec 100644
--- a/scripts/test_suites.py
+++ b/scripts/test_suites.py
@@ -119,6 +119,7 @@ PYTEST_SUITES = [
"tests/test_evals_tournament.py",
"tests/test_serve.py",
"tests/test_mcp_remote.py",
+ "tests/test_mcp_diagnostics.py",
"tests/test_router_batch.py",
"tests/test_predict_batch.py",
"tests/test_system_one_lang.py",
diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.py
index 05e81bba..2ea1d8cf 100644
--- a/tests/test_hooks_api.py
+++ b/tests/test_hooks_api.py
@@ -282,6 +282,10 @@ for label, fn in (("analyse", analyse), ("Router.route", Router.route),
("Router.route_batch", Router.route_batch)):
check_param(label, fn, "explain", False, inspect.Parameter.KEYWORD_ONLY)
check("latin_profile keeps its signature", list(sig(latin_profile)), ["text"])
+
+from laya.mcp.tools import laya_route # noqa: E402
+
+check_param("laya_route", laya_route, "explain", False, inspect.Parameter.KEYWORD_ONLY)
_plain_analysis = analyse("Turn lights off")
_explained_analysis = analyse("Turn lights off", explain=True)
check("analyse opts into one extra key", set(_explained_analysis) - set(_plain_analysis), {"diagnostic"})
diff --git a/tests/test_mcp_diagnostics.py b/tests/test_mcp_diagnostics.py
new file mode 100644
index 00000000..92a5dd98
--- /dev/null
+++ b/tests/test_mcp_diagnostics.py
@@ -0,0 +1,219 @@
+"""Opt-in MCP routing diagnostics: real core decisions and MCP parsing, no weights."""
+import asyncio
+import copy
+import json
+from types import SimpleNamespace
+
+import pytest
+
+pytest.importorskip("mcp")
+
+from mcp.server.mcpserver.exceptions import ToolError as McpToolError # noqa: E402
+from laya import BaseHook, Router # noqa: E402
+from laya.mcp import server as server_mod # noqa: E402
+from laya.mcp.tools import ToolError, laya_predict, laya_route, laya_tournament # noqa: E402
+
+
+STATE = {"message": "Turn lights off"}
+QUESTIONS = {"category": {"type": "choice", "instructions": "Classify the message.",
+ "criteria": {"A": "Support", "B": "Other"}}}
+DIAGNOSTIC = {"stage": "whole_state", "reason": "insufficient_words", "source": None}
+ROUTING = {"model": "english", "repo": "example/english", "reason": "test route"}
+INVALID_FLAGS = [None, 0, 1, 0.0, 1.0, "false", "true", "", [], {}]
+
+
+class LegacyRouter:
+ """An injected router predating explain; even an unrequested keyword is incompatible."""
+
+ def __init__(self):
+ self.calls = []
+
+ def route(self, state, questions):
+ self.calls.append((state, questions))
+ return dict(ROUTING)
+
+
+class RecordingRouter(Router):
+ def __init__(self, **kwargs):
+ super().__init__(**kwargs)
+ self.calls = []
+
+ def route(self, state, questions=None, **kwargs):
+ self.calls.append((state, questions, dict(kwargs)))
+ return super().route(state, questions, **kwargs)
+
+
+def _call_tool(arguments):
+ result = asyncio.run(server_mod.server.call_tool("laya_route", arguments))
+ assert not result.is_error
+ return json.loads(result.content[0].text)
+
+
[email protected]("flag", [{}, {"explain": False}])
+def test_default_route_preserves_legacy_router_and_exact_output(flag):
+ router = LegacyRouter()
+ out = laya_route(STATE, QUESTIONS, router=router, **flag)
+ assert json.dumps(out) == json.dumps(ROUTING)
+ assert len(router.calls) == 1
+
+
[email protected](("state", "expected"), [
+ (STATE, DIAGNOSTIC),
+ ({"message": "xyzzy frobnitz quux blorb"},
+ {"stage": "whole_state", "reason": "no_language_evidence", "source": None}),
+ ({"note": "This is an English note about the order. " * 20,
+ "payload": {"customer": "Ich kann mich nicht anmelden und brauche Hilfe"}},
+ {"stage": "segment", "reason": "named_segment",
+ "source": {"leaf_index": 1, "path": ["payload", "customer"], "line": 1}}),
+])
+def test_route_returns_the_core_diagnostic_without_detection_values(state, expected):
+ router = RecordingRouter()
+ original = copy.deepcopy(state)
+ plain = laya_route(state, QUESTIONS, router=router)
+ explained = laya_route(state, QUESTIONS, router=router, explain=True)
+ assert explained.pop("diagnostic") == expected
+ assert explained == plain
+ assert set(explained) == {"model", "repo", "reason"}
+ assert router.calls[0][2] == {}
+ assert router.calls[1][2] == {"explain": True}
+ assert len(router.calls) == 2
+ assert state == original
+
+
[email protected]("override", [
+ {"model": "english"}, {"task": "typed_decisions"}, {"lang": "de"}, {"lang_guess": "de"},
+])
+def test_route_override_has_explicit_null_diagnostic(override):
+ router = Router()
+ plain = laya_route(STATE, QUESTIONS, router=router, **override)
+ explained = laya_route(STATE, QUESTIONS, router=router, explain=True, **override)
+ assert explained.pop("diagnostic") is None
+ assert explained == plain
+
+
[email protected]("replacement", [
+ dict(ROUTING),
+ dict(ROUTING, detection={"diagnostic": {"stage": "whole_state", "reason": "no_letters", "source": None}}),
+])
+def test_route_uses_final_hook_decision_not_stale_detection(replacement):
+ class ReplaceRoute(BaseHook):
+ def on_route(self, ctx):
+ ctx.decision = copy.deepcopy(replacement)
+
+ router = RecordingRouter(hooks=[ReplaceRoute()])
+ out = laya_route(STATE, QUESTIONS, router=router, explain=True)
+ assert out == dict(ROUTING, diagnostic=replacement.get("detection", {}).get("diagnostic"))
+ assert len(router.calls) == 1
+
+
+def test_object_style_route_decision_preserves_only_the_diagnostic():
+ class ObjectRouter:
+ def route(self, state, questions, *, explain=False):
+ assert explain is True
+ return SimpleNamespace(**ROUTING, detection={"mixed_segment": "private value",
+ "diagnostic": copy.deepcopy(DIAGNOSTIC)})
+
+ out = laya_route(STATE, QUESTIONS, router=ObjectRouter(), explain=True)
+ assert out == dict(ROUTING, diagnostic=DIAGNOSTIC)
+ assert "private value" not in json.dumps(out)
+
+
[email protected]("bad", INVALID_FLAGS)
+def test_route_rejects_non_boolean_before_dispatch(bad):
+ router = RecordingRouter()
+ with pytest.raises(ToolError, match="explain") as info:
+ laya_route(STATE, QUESTIONS, router=router, explain=bad)
+ assert info.value.code == "invalid_explain"
+ assert router.calls == []
+
+
+def test_legacy_router_refuses_requested_explanation_without_fallback():
+ router = LegacyRouter()
+ with pytest.raises(ToolError, match="explain") as info:
+ laya_route(STATE, QUESTIONS, router=router, explain=True)
+ assert info.value.code == "unsupported_explain"
+ assert router.calls == []
+
+
[email protected]("message", [
+ "internal routing failure",
+ "wrapped.route() got an unexpected keyword argument 'explain'",
+])
+def test_internal_type_error_is_not_relabelled_as_unsupported(message):
+ error = TypeError(message)
+
+ class BrokenRouter:
+ def route(self, state, questions, **kwargs):
+ raise error
+
+ with pytest.raises(TypeError) as info:
+ laya_route(STATE, QUESTIONS, router=BrokenRouter(), explain=True)
+ assert info.value is error
+
+
+def test_forwarding_router_type_error_is_not_relabelled_or_retried():
+ inner = LegacyRouter()
+
+ class ForwardingRouter:
+ def __init__(self):
+ self.calls = 0
+
+ def route(self, state, questions, **kwargs):
+ self.calls += 1
+ return inner.route(state, questions, **kwargs)
+
+ router = ForwardingRouter()
+ with pytest.raises(TypeError, match="unexpected keyword argument 'explain'"):
+ laya_route(STATE, QUESTIONS, router=router, explain=True)
+ assert router.calls == 1
+ assert inner.calls == []
+
+
+def test_prediction_routing_is_unchanged_after_an_explained_route():
+ class PredictRouter:
+ def route(self, state, questions, **kwargs):
+ return dict(ROUTING, detection={"diagnostic": copy.deepcopy(DIAGNOSTIC)})
+
+ def predict(self, state, questions, **kwargs):
+ return {"answers": {"category": {"type": "choice", "choice": "A"}},
+ "routing": dict(ROUTING)}
+
+ router = PredictRouter()
+ before = json.dumps(laya_predict(STATE, QUESTIONS, router=router)["routing"])
+ assert laya_route(STATE, QUESTIONS, router=router, explain=True)["diagnostic"] == DIAGNOSTIC
+ after = json.dumps(laya_predict(STATE, QUESTIONS, router=router)["routing"])
+ assert before == after == json.dumps(ROUTING)
+ # Tournament is a real caller of the shared projection; two options need no narrowing.
+ tournament = laya_tournament(STATE, QUESTIONS, router=router)
+ assert json.dumps(tournament["routing"]) == json.dumps(ROUTING)
+
+
+def test_mcp_route_schema_exposes_optional_boolean_explain():
+ tool = next(t for t in asyncio.run(server_mod.server.list_tools()) if t.name == "laya_route")
+ schema = tool.input_schema
+ assert schema["properties"]["explain"]["type"] == "boolean"
+ assert schema["properties"]["explain"]["default"] is False
+ assert schema["required"] == ["state", "questions"]
+
+
[email protected]("flag", [{}, {"explain": False}, {"explain": True}])
+def test_real_mcp_route_calls_honor_boolean(flag, monkeypatch):
+ router = RecordingRouter()
+ monkeypatch.setattr(server_mod, "_ROUTER", router)
+ out = _call_tool({"state": STATE, "questions": QUESTIONS, **flag})
+ if flag.get("explain", False):
+ assert out["diagnostic"] == DIAGNOSTIC
+ assert router.calls[0][2] == {"explain": True}
+ else:
+ assert set(out) == {"model", "repo", "reason"}
+ assert router.calls[0][2] == {}
+ assert len(router.calls) == 1
+
+
[email protected]("bad", INVALID_FLAGS)
+def test_real_mcp_parser_rejects_non_boolean_without_dispatch(bad, monkeypatch):
+ router = RecordingRouter()
+ monkeypatch.setattr(server_mod, "_ROUTER", router)
+ with pytest.raises(McpToolError, match="explain"):
+ _call_tool({"state": STATE, "questions": QUESTIONS, "explain": bad})
+ assert router.calls == []