Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(mcp): share a local model server through LAYA_BASE_URL
Rishit Sharmac8ddefa4836a
diff --git a/README.md b/README.mdindex 061732d3..c55ff1c3 100644--- a/README.md+++ b/README.md@@ -1187,6 +1187,12 @@ pip install "laya[mcp]"laya-mcp-server # or: python -m laya.mcp.server```+To share a `laya-serve` on your own hardware across editor sessions, set+`LAYA_BASE_URL=http://127.0.0.1:8000` in each MCP client's environment. Its prediction tools use+HTTP and import no torch; routing stays local. Set `LAYA_API_KEY` to the server's bearer key and+`LAYA_REMOTE_TIMEOUT` to the allowed HTTP wait (300 seconds by default). `laya_shortlist` is+unavailable in this mode. See [shared-server MCP setup](docs/cli-mcp.md#share-one-model-server-across-mcp-sessions).+Example MCP client configuration (stdio transport):```jsondiff --git a/docs/cli-mcp.md b/docs/cli-mcp.mdindex b09d5588..97edf633 100644--- a/docs/cli-mcp.md+++ b/docs/cli-mcp.md@@ -229,10 +229,48 @@ the server is ready.| `LAYA_THREADS` | PyTorch default | Caps Torch intra-op threads for CPU inference; keep it at or below the physical core count. || `LAYA_AUTO_TASK` | `0` | Set to `1` to let a request auto-route to the `typed-decisions` checkpoint. Same meaning as in `laya.serve`; it does not preload that checkpoint, so `LAYA_MODELS` still decides what is built at startup. || `LAYA_DEFAULT_MODEL` | `english` | The checkpoint a state with no language evidence falls back to, same meaning as in `laya.serve`. Unlike `laya.serve`, an unresolvable name does not stop the server: it comes back as a `router construction failed` tool error on the next call, because a stdio server has no startup to refuse. |+| `LAYA_BASE_URL` | unset | Send predictions to a `laya-serve` on your own hardware instead of loading checkpoints in each MCP process. A bare `host:port` is read as HTTP. |+| `LAYA_REMOTE_TIMEOUT` | `300` | HTTP timeout in seconds when `LAYA_BASE_URL` is set, including the server's cold load. Invalid or non-positive values use the default. |-The stock `laya-mcp-server` launcher creates its Router without installing hooks. If you need-prediction hooks, use a custom launcher that installs them, for example with-`laya.hooks.set_default_hooks`, before the server builds its Router. The environment variables+### Share one model server across MCP sessions++Run one local HTTP server and point each MCP client's environment at it:++```bash+LAYA_HOST=127.0.0.1 LAYA_PRELOAD=0 LAYA_IDLE_UNLOAD_SECONDS=300 laya-serve+```++```json+{+ "mcpServers": {+ "laya": {+ "command": "laya-mcp-server",+ "env": {"LAYA_BASE_URL": "http://127.0.0.1:8000"}+ }+ }+}+```++Install `laya[serve]` where the HTTP server runs. MCP still uses stdio with the editor; its+prediction tools use HTTP to reach your server. `laya_predict`, `laya_predict_batch`, `laya_decide`+and `laya_preset` use the server's original state, instructions and option descriptions.+Heterogeneous batches send one `/v1/systemone` request per item, preserving input order;+`batch_size` and `sort_by_length` do not change the server's execution. `laya_status` reports the+server's `/health`; `laya_route` and `laya_route_batch` stay local and need no model or HTTP request.+The MCP process imports no torch and loads no checkpoint, including when `LAYA_THREADS` or+`LAYA_PRELOAD` is set.++Set the same `LAYA_API_KEY` in both processes when the server requires a bearer token. Keep+`LAYA_DEFAULT_MODEL` and `LAYA_AUTO_TASK` aligned so local routing previews match the server's+actual routing. Device and preload settings belong to the HTTP server. The first call after an+idle unload waits for a cold load; raise `LAYA_REMOTE_TIMEOUT` if that takes longer than 300 seconds.+`laya_shortlist` and prediction hook overrides return `unsupported_remote`, since their code needs+the model process. HTTP errors retain the server's detail text as MCP tool errors. With+`LAYA_BASE_URL` unset, the MCP server keeps loading checkpoints in its own process.++The stock `laya-mcp-server` launcher creates its Router without installing hooks. Install prediction+hooks in the process that runs inference: the MCP process in local mode, or the HTTP server in+shared-server mode. A custom launcher can use `laya.hooks.set_default_hooks` before building its Router. The environment variablesabove configure model lifecycle, not hook registration. The client still decides when to call atool and what to do with the returned decision.diff --git a/laya/mcp/remote.py b/laya/mcp/remote.pynew file mode 100644index 00000000..0631e622--- /dev/null+++ b/laya/mcp/remote.py@@ -0,0 +1,239 @@+"""A Router that answers over HTTP from a running ``laya-serve`` instead of loading weights.++``laya-mcp-server`` builds a ``Router`` and loads checkpoints into its own process. That is the+right default for one agent on one machine, but it means every MCP client that opens a session+holds its own copy of the model: two Claude sessions plus a ``laya-serve`` for scripts is three+times 1.3 GB for the same checkpoint. Set ``LAYA_BASE_URL`` and the MCP server uses this class+instead: ``predict`` and ``predict_batch`` become ``POST /v1/systemone`` calls to that server,+and the MCP process itself stays at a few tens of megabytes, with no torch import at all.++Routing stays local and free. ``RemoteRouter`` is a ``Router``, so ``route`` / ``route_batch``+run the same language and script detection the server runs, without a model. The server makes+its own routing decision for the actual answer and reports it in ``routing``; the two agree when+the MCP process and the server see the same ``LAYA_DEFAULT_MODEL`` / ``LAYA_AUTO_TASK``.++Not available remotely: ``load`` (and therefore ``laya_shortlist``, which needs the answering+checkpoint's embeddings in-process), ``preload`` and ``unload`` (no-ops: the server owns its+checkpoints' lifetime, see ``LAYA_IDLE_UNLOAD_SECONDS``), and per-call hooks (a hook is a callable+that has to live where inference runs, exactly as ``laya.serve`` already refuses them).++Standard library only: ``urllib`` carries the requests, so the extra pulls in nothing new.+"""+from __future__ import annotations++import json+import math+import os+import urllib.error+import urllib.request+import urllib.parse+from typing import Any, Dict, List, Optional, Sequence, Union++from ..router import Router+from ..serve import BODY_CONTROLS, BODY_REFUSALS++DEFAULT_TIMEOUT_S = 300.0++# Request fields `laya.serve` reads off a /v1/systemone body, in the order the single-request+# handler validates them. Anything not in this list has no wire form and is refused or dropped.+_WIRE_CONTROLS = BODY_CONTROLS+++class RemoteError(RuntimeError):+ """An HTTP-level failure talking to laya-serve. ``code`` is a tool-error style slug."""++ def __init__(self, code: str, message: str):+ super().__init__(message)+ self.code = code+ self.message = message+++def _timeout_from_env() -> float:+ raw = os.environ.get("LAYA_REMOTE_TIMEOUT", "").strip()+ if not raw:+ return DEFAULT_TIMEOUT_S+ try:+ seconds = float(raw)+ except ValueError:+ return DEFAULT_TIMEOUT_S+ return seconds if math.isfinite(seconds) and seconds > 0 else DEFAULT_TIMEOUT_S+++class RemoteRouter(Router):+ """Route locally, answer from a ``laya-serve`` at ``base_url``.++ ``base_url`` is the server origin (``http://127.0.0.1:8000``); the ``/v1/systemone`` paths are+ appended here. ``api_key``, when given, is sent as ``Authorization: Bearer`` and must match the+ server's ``LAYA_API_KEY``.+ """++ def __init__(self, base_url: str, api_key: Optional[str] = None,+ timeout: Optional[float] = None, **router_kwargs: Any):+ # device/preload make no sense for a router that never builds an Agent.+ router_kwargs.pop("device", None)+ router_kwargs.pop("preload", None)+ super().__init__(**router_kwargs)+ self.base_url = base_url.rstrip("/")+ parsed = urllib.parse.urlsplit(self.base_url)+ if parsed.scheme not in ("http", "https") or not parsed.hostname or parsed.query or parsed.fragment:+ raise ValueError("base_url must be an HTTP(S) server URL without a query or fragment")+ self.api_key = api_key or None+ self.timeout = _timeout_from_env() if timeout is None else float(timeout)+ if not math.isfinite(self.timeout) or self.timeout <= 0:+ raise ValueError("timeout must be a finite positive number")++ # ------------------------------------------------------------------ transport+ def _request(self, path: str, body: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:+ data = None if body is None else json.dumps(body).encode("utf-8")+ headers = {"Accept": "application/json"}+ if data is not None:+ headers["Content-Type"] = "application/json"+ if self.api_key:+ headers["Authorization"] = "Bearer " + self.api_key+ req = urllib.request.Request(self.base_url + path, data=data, headers=headers,+ method="POST" if data is not None else "GET")+ try:+ with urllib.request.urlopen(req, timeout=self.timeout) as resp:+ raw = resp.read()+ except urllib.error.HTTPError as exc:+ detail = _error_detail(exc)+ if exc.code == 401:+ raise RemoteError("unauthorized", "laya-serve refused the bearer token: " + detail) from exc+ if exc.code in (400, 413, 422):+ raise RemoteError("invalid_request", detail) from exc+ if exc.code == 503:+ raise RemoteError("busy", "laya-serve is busy: " + detail) from exc+ raise RemoteError("upstream_error", "laya-serve answered HTTP %d: %s" % (exc.code, detail)) from exc+ except urllib.error.URLError as exc:+ if isinstance(exc.reason, TimeoutError):+ raise RemoteError("timeout", "laya-serve did not answer within %.0fs" % self.timeout) from exc+ raise RemoteError("unreachable", "cannot reach laya-serve at %s: %s" % (self.base_url, exc.reason)) from exc+ except TimeoutError as exc:+ raise RemoteError("timeout", "laya-serve did not answer within %.0fs" % self.timeout) from exc+ try:+ payload = json.loads(raw.decode("utf-8"))+ except (UnicodeDecodeError, json.JSONDecodeError) as exc:+ raise RemoteError("upstream_error", "laya-serve returned a non-JSON body") from exc+ if not isinstance(payload, dict):+ raise RemoteError("upstream_error", "laya-serve returned a non-object body")+ return payload++ # ------------------------------------------------------------------ server state+ def health(self) -> Dict[str, Any]:+ """The server's ``GET /health`` payload (liveness only when the bearer is wrong)."""+ return self._request("/health")++ @property+ def loaded(self) -> List[str]:+ """Checkpoints resident on the server right now, or [] when it cannot be asked."""+ try:+ value = self.health().get("loaded")+ except RemoteError:+ return []+ return [v for v in value if isinstance(v, str)] if isinstance(value, list) else []++ @property+ def loaded_revisions(self) -> Dict[str, Optional[str]]:+ try:+ value = self.health().get("revisions")+ except RemoteError:+ return {}+ return dict(value) if isinstance(value, dict) else {}++ # ------------------------------------------------------------------ lifecycle (server-owned)+ def load(self, name: str):+ raise RemoteError(+ "unsupported_remote",+ "checkpoint %r lives in laya-serve at %s; a remote router cannot hand out an Agent"+ % (name, self.base_url),+ )++ def preload(self, names: Optional[List[str]] = None):+ return self # the server decides what is resident++ def unload(self, name: Optional[str] = None):+ return None # the server decides when to free memory (LAYA_IDLE_UNLOAD_SECONDS)++ def attach(self, name: str, agent: Any):+ raise RemoteError("unsupported_remote", "a remote router holds no agents to attach to")++ # ------------------------------------------------------------------ answering+ def predict(+ self,+ state: Union[str, dict, list],+ questions: Dict[str, Any],+ model: Optional[str] = None,+ task: Optional[str] = None,+ lang: Optional[str] = None,+ lang_guess: Optional[Any] = None,+ hooks=None,+ on_predict_start=None,+ on_predict_end=None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ min_confidence: Optional[float] = None,+ ) -> Dict[str, Any]:+ if any(value is not None for value in (hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)):+ raise RemoteError("unsupported_remote",+ "hooks are callables and cannot be sent to laya-serve; install them on the server")+ body: Dict[str, Any] = {"state": state, "questions": questions}+ controls = {"model": model, "task": task, "lang": lang, "lang_guess": lang_guess,+ "max_len": max_len, "head_max_len": head_max_len, "min_confidence": min_confidence}+ for key in _WIRE_CONTROLS:+ value = controls[key]+ if value is not None and not (key == "model" and value == "auto"):+ if key == "lang_guess" and not isinstance(value, str):+ raise RemoteError("unsupported_remote", "a callable lang_guess cannot be sent over HTTP")+ body[key] = value+ return self._request("/v1/systemone", body)++ def predict_batch(+ self,+ requests: Sequence[Dict[str, Any]],+ batch_size: Optional[int] = None,+ hooks_timeout: Optional[float] = None,+ min_confidence: Optional[float] = None,+ sort_by_length: bool = False,+ ) -> List[Dict[str, Any]]:+ """One ``/v1/systemone`` call per request, answers in input order.++ ``laya-serve``'s ``/v1/systemone/batch`` shares one ``questions`` object across every state,+ while a Router batch lets each request carry its own questions and controls, so the general+ form is the per-request call. ``batch_size`` and ``sort_by_length`` shape forward passes and+ have no meaning here; they are accepted and ignored so callers written against ``Router``+ keep working. The server applies its own batching inside each call.+ """+ results: List[Dict[str, Any]] = []+ if hooks_timeout is not None:+ raise RemoteError("unsupported_remote", "hooks_timeout belongs to the server's hooks")+ for item in requests:+ if not isinstance(item, dict):+ raise TypeError("predict_batch requests must be dicts, got %s" % type(item).__name__)+ if any(item.get(key) is not None for key in BODY_REFUSALS):+ raise RemoteError("unsupported_remote", "per-request hooks cannot be sent to laya-serve")+ kwargs = {k: item[k] for k in ("model", "task", "lang", "lang_guess", "max_len", "head_max_len")+ if k in item}+ if min_confidence is not None:+ kwargs["min_confidence"] = min_confidence+ results.append(self.predict(item["state"], item["questions"], **kwargs))+ return results++ def __repr__(self) -> str:+ return "RemoteRouter(base_url=%r, timeout=%g)" % (self.base_url, self.timeout)+++def _error_detail(exc: urllib.error.HTTPError) -> str:+ """The server's ``detail`` string when the error body is FastAPI-shaped, else the status text."""+ try:+ body = json.loads(exc.read().decode("utf-8"))+ except Exception: # noqa: BLE001 -- any unreadable body falls back to the HTTP reason+ return str(exc.reason)+ if isinstance(body, dict):+ detail = body.get("detail")+ if isinstance(detail, str):+ return detail+ if detail is not None:+ return json.dumps(detail, ensure_ascii=False)+ return str(exc.reason)diff --git a/laya/mcp/server.py b/laya/mcp/server.pyindex 3eb05a65..39035cb4 100644--- a/laya/mcp/server.py+++ b/laya/mcp/server.py@@ -18,6 +18,8 @@ Environment (same meaning as laya.serve where it exists):LAYA_DEFAULT_MODEL the checkpoint a state with no language evidence falls back to(same as laya.serve, including its aliases); an unset value leavesit to Router, an unresolvable one is a tool error.+ LAYA_BASE_URL answer from a running laya-serve over HTTP; unset keeps models local+ LAYA_REMOTE_TIMEOUT HTTP timeout in seconds for remote mode (default 300)"""from __future__ import annotations@@ -126,6 +128,20 @@ def _models_from_env() -> list[str]:return names or list(_DEFAULT_MODELS)+def _remote_base_url() -> str | None:+ """``LAYA_BASE_URL``: a running laya-serve to answer from instead of loading checkpoints here.++ Set it to the server origin (``http://127.0.0.1:8000``). Blank or unset means the default,+ in-process router. Scheme-less values are accepted as http so a bare ``host:port`` works.+ """+ raw = os.environ.get("LAYA_BASE_URL", "").strip()+ if not raw:+ return None+ if "://" not in raw:+ raw = "http://" + raw+ return raw.rstrip("/")++def _ensure_router() -> Any:"""Build the Router from the environment, following the laya.serve contract.@@ -145,6 +161,19 @@ def _ensure_router() -> Any:with _ROUTER_LOCK:if _ROUTER is not None:return _ROUTER+ base_url = _remote_base_url()+ if base_url:+ # Remote mode: no checkpoint is built here, no torch is imported. The RemoteRouter+ # routes locally (pure Python) and answers over HTTP from a running laya-serve.+ try:+ from .remote import RemoteRouter+ router = RemoteRouter(base_url, api_key=os.environ.get("LAYA_API_KEY") or None,+ auto_task_detection=_env_bool("LAYA_AUTO_TASK", False),+ **_default_model_option())+ except Exception as exc:+ raise ToolError("internal_error", f"remote router construction failed: {exc}") from exc+ _ROUTER = router+ return routertry:from laya import Routerexcept Exception as exc:@@ -205,7 +234,11 @@ def _wrap(fn, **kwargs) -> str:@server.tool(name="laya_status")def laya_status_tool() -> str:"""Report the device actually in use per loaded checkpoint (or the configured preference, flagged as such, when nothing is loaded), torch CUDA availability, loaded checkpoints, and package versions."""- return _wrap(laya_status, router=_ROUTER, preload=_env_bool("LAYA_PRELOAD", True))+ # In remote mode the router is cheap (no checkpoint, no torch), so build it here if a status+ # call comes first: the remote report then describes the server instead of a torch probe of+ # this process, and `device_report()`'s torch import never happens in a remote MCP process.+ router = _ROUTER if (_ROUTER is not None or not _remote_base_url()) else _router_or_error()+ return _wrap(laya_status, router=router, preload=_env_bool("LAYA_PRELOAD", True))@server.tool(@@ -478,7 +511,11 @@ def main() -> None:if os.name == "nt":os.environ.setdefault("HF_HUB_DISABLE_SYMLINKS", "1")os.environ.setdefault("HF_HUB_DISABLE_SYMLINKS_WARNING", "1")- if _env_bool("LAYA_PRELOAD", True):+ if _remote_base_url():+ # Nothing to preload: the server at LAYA_BASE_URL owns the checkpoints. Say so once, so a+ # reader of the client's stderr knows which process is answering.+ print(f"[laya-mcp] remote mode: answering from laya-serve at {_remote_base_url()}", file=sys.stderr)+ elif _env_bool("LAYA_PRELOAD", True):try:_ensure_router()except Exception as exc:diff --git a/laya/mcp/tools.py b/laya/mcp/tools.pyindex 2ac1c22f..80539d47 100644--- a/laya/mcp/tools.py+++ b/laya/mcp/tools.py@@ -342,6 +342,14 @@ def _overrides(task: Any, lang: Any, max_len: Any, head_max_len: Any) -> dict:return {name: value for name, value in values.items() if value is not None}+def _raise_if_remote_error(exc: BaseException) -> None:+ """Turn transport failures into the ToolError the client can read."""+ from .remote import RemoteError++ if isinstance(exc, RemoteError):+ raise ToolError(exc.code, exc.message) from exc++def _normalize_answers(raw: Any) -> dict:if not isinstance(raw, dict):raise ToolError("internal_error", "predict returned non-object answers")@@ -457,7 +465,11 @@ def laya_predict(return router.predict(state_d, questions_d, model=model_name, **budget)started = time.perf_counter()- result = _run()+ try:+ result = _run()+ except Exception as exc:+ _raise_if_remote_error(exc)+ raiselatency_ms = (time.perf_counter() - started) * 1000.0norm = _normalize_result(result)@@ -589,6 +601,13 @@ def _resident_or_load(router: Any, name: str) -> Any:resident = router_agent(router, name)if resident is not None:return resident+ if hasattr(router, "base_url") and hasattr(router, "health"):+ raise ToolError(+ "unsupported_remote",+ "laya_shortlist embeds options with the answering checkpoint in-process; it is not "+ "available when LAYA_BASE_URL points the MCP server at a remote laya-serve. Use "+ "laya_predict with head_max_len raised, or run the MCP server without LAYA_BASE_URL.",+ )load = getattr(router, "load", None)if load is None:raise ToolError("models_not_ready", f"checkpoint {name!r} is not loaded")@@ -657,6 +676,15 @@ def laya_shortlist((``lang_guess`` only routes, so like ``task`` it is stripped before the answering pass);``min_confidence`` flags a kept-label answer the checkpoint is unsure of."""+ # Before the heavy imports below: a remote router has no in-process encoder to embed with,+ # and the refusal should not cost the MCP process a torch import on the way to saying so.+ if router is not None and agent is None and hasattr(router, "base_url") and hasattr(router, "health"):+ raise ToolError(+ "unsupported_remote",+ "laya_shortlist embeds options with the answering checkpoint in-process; it is not "+ "available when LAYA_BASE_URL points the MCP server at a remote laya-serve. Use "+ "laya_predict with head_max_len raised, or run the MCP server without LAYA_BASE_URL.",+ )# Lazy: keeps numpy/shortlist out of module import for laya.mcp.tools.from laya.shortlist import (DEFAULT_SHORTLIST_K,@@ -822,7 +850,39 @@ def laya_preset()+def _remote_status(router: Any, preload: bool) -> dict:+ """``laya_status`` for a remote router: the server's own /health, no torch import here."""+ versions: dict[str, str | None] = {"laya": None}+ try:+ versions["laya"] = getattr(__import__("laya"), "__version__", "unknown")+ except Exception:+ pass+ out: dict[str, Any] = {+ "mode": "remote",+ "base_url": router.base_url,+ "router_preload": False,+ "router_ready": True,+ "package_versions": versions,+ }+ try:+ health = router.health()+ except Exception as exc: # noqa: BLE001 -- the status tool reports, it never raises+ out["server"] = None+ out["server_error"] = f"{type(exc).__name__}: {exc}"+ out["loaded"] = []+ return out+ out["server"] = health+ loaded = health.get("loaded")+ out["loaded"] = [v for v in loaded if isinstance(v, str)] if isinstance(loaded, list) else []+ for key in ("device", "device_is_preference", "checkpoint_devices"):+ if key in health:+ out[key] = health[key]+ return out++def laya_status(*, router: Any = None, loaded: list[str] | None = None, preload: bool = True) -> dict:+ if router is not None and hasattr(router, "base_url") and hasattr(router, "health"):+ return _remote_status(router, preload)report = device_report()versions: dict[str, str | None] = {"laya": None,@@ -1038,7 +1098,11 @@ def laya_predict_batch(if sort_by_length:kwargs["sort_by_length"] = Trueresults = router.predict_batch(items, **kwargs)- except TypeError as exc:+ except Exception as exc:+ _raise_if_remote_error(exc)+ if not isinstance(exc, TypeError):+ raise+# A Router without batch support raises at the call itself; anything# else is a real bug and must surface unchanged.raise ToolError(@@ -1161,6 +1225,9 @@ def laya_decide(model=model_name, min_confidence=min_conf)except SchemaError as exc:raise ToolError("invalid_schema", str(exc)) from exc+ except Exception as exc:+ _raise_if_remote_error(exc)+ raiselatency_ms = (time.perf_counter() - started) * 1000.0answers = _normalize_answers(details.answers)diff --git a/scripts/test_suites.py b/scripts/test_suites.pyindex ffe1ab20..fbab819a 100644--- a/scripts/test_suites.py+++ b/scripts/test_suites.py@@ -99,6 +99,7 @@ SCRIPT_SUITES = [PYTEST_SUITES = ["tests/test_serve.py",+ "tests/test_mcp_remote.py","tests/test_router_batch.py","tests/test_predict_batch.py","tests/test_system_one_lang.py",diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.pyindex a8aa4957..e9e2b753 100644--- a/tests/test_hooks_api.py+++ b/tests/test_hooks_api.py@@ -105,6 +105,20 @@ check_param("load", load, "compile", False)check_param("ONNXAgent.__init__", ONNXAgent.__init__, "token", None)# --------------------------------------------------------------- predict surfaces+from laya.mcp.remote import RemoteRouter # noqa: E402++check_true("RemoteRouter/is a Router", issubclass(RemoteRouter, Router))+check_param("RemoteRouter.__init__", RemoteRouter.__init__, "base_url", inspect.Parameter.empty)+check_param("RemoteRouter.__init__", RemoteRouter.__init__, "api_key", None)+check_param("RemoteRouter.__init__", RemoteRouter.__init__, "timeout", None)+for method in ("predict", "predict_batch"):+ check("RemoteRouter/%s signature" % method,+ [(p.name, p.kind, p.default) for p in sig(getattr(RemoteRouter, method)).values()],+ [(p.name, p.kind, p.default) for p in sig(getattr(Router, method)).values()])+for method in ("route", "route_batch"):+ check_true("RemoteRouter/%s stays local" % method,+ getattr(RemoteRouter, method) is getattr(Router, method))+for label, fn in (("Agent.predict_batch", Agent.predict_batch),("Agent.system_one", Agent.system_one),("Agent.predict_long", Agent.predict_long),diff --git a/tests/test_mcp_remote.py b/tests/test_mcp_remote.pynew file mode 100644index 00000000..ef412075--- /dev/null+++ b/tests/test_mcp_remote.py@@ -0,0 +1,375 @@+"""Remote mode of the MCP server (LAYA_BASE_URL): answers come from a laya-serve over HTTP.++A stdlib HTTP server stands in for laya-serve, so these tests need no checkpoint, no torch+and no external network. Run: python -m pytest tests/test_mcp_remote.py+"""+import json+import threading+from http.server import BaseHTTPRequestHandler, HTTPServer++import pytest++pytest.importorskip("mcp")++from laya.mcp import server as mcp_server # noqa: E402+from laya.mcp.tools import ( # noqa: E402+ ToolError, laya_decide, laya_predict, laya_predict_batch, laya_preset, laya_route, laya_route_batch,+ laya_shortlist, laya_status,+)+from laya.mcp.remote import DEFAULT_TIMEOUT_S, RemoteError, RemoteRouter # noqa: E402++ANSWER = {+ "model": "laya-rl-agent",+ "answers": {"dept": {"type": "choice", "choice": "billing",+ "probabilities": {"billing": 0.9, "tech": 0.1}, "confidence": 0.9}},+ "usage": {"input_tokens": 40, "output_tokens": 0},+ "routing": {"model": "english", "repo": "convaiinnovations/laya", "reason": "English Latin text"},+}+HEALTH = {"status": "ok", "loaded": ["english"], "revisions": {"english": "abc"},+ "device": "mps", "device_is_preference": False, "checkpoint_devices": {"english": "mps"}}+++class FakeServe(BaseHTTPRequestHandler):+ """Records every request; answers /health and /v1/systemone like laya-serve."""++ requests: list = []+ status_override: int | None = None+ detail_override: str = "bad request"+ require_bearer: str | None = None++ def log_message(self, *args): # quiet+ pass++ def _send(self, status, payload):+ body = json.dumps(payload).encode()+ self.send_response(status)+ self.send_header("Content-Type", "application/json")+ self.send_header("Content-Length", str(len(body)))+ self.end_headers()+ self.wfile.write(body)++ def do_GET(self):+ FakeServe.requests.append(("GET", self.path, None, dict(self.headers)))+ if self.path == "/health":+ if FakeServe.require_bearer and self.headers.get("Authorization") != "Bearer " + FakeServe.require_bearer:+ return self._send(200, {"status": "ok"})+ return self._send(200, HEALTH)+ self._send(404, {"detail": "Not Found"})++ def do_POST(self):+ length = int(self.headers.get("Content-Length") or 0)+ body = json.loads(self.rfile.read(length)) if length else None+ FakeServe.requests.append(("POST", self.path, body, dict(self.headers)))+ if FakeServe.require_bearer and self.headers.get("Authorization") != "Bearer " + FakeServe.require_bearer:+ return self._send(401, {"detail": "invalid or missing bearer token"})+ if FakeServe.status_override:+ return self._send(FakeServe.status_override, {"detail": FakeServe.detail_override})+ if self.path == "/v1/systemone":+ return self._send(200, ANSWER)+ self._send(404, {"detail": "Not Found"})+++def serve():+ FakeServe.requests = []+ FakeServe.status_override = None+ FakeServe.require_bearer = None+ httpd = HTTPServer(("127.0.0.1", 0), FakeServe)+ thread = threading.Thread(target=lambda: httpd.serve_forever(poll_interval=0.01), daemon=True)+ thread.start()+ try:+ yield "http://127.0.0.1:%d" % httpd.server_address[1]+ finally:+ httpd.shutdown()+ httpd.server_close()+++def _questions():+ return {"dept": {"type": "choice", "instructions": "which team?", "criteria": {"billing": None, "tech": None}}}+++def test_predict_posts_the_serve_body_and_returns_it_unchanged(serve):+ router = RemoteRouter(serve)+ result = router.predict({"text": "billed twice"}, _questions(), lang="en", max_len=512, min_confidence=0.3)+ assert result == ANSWER+ method, path, body, headers = FakeServe.requests[-1]+ assert (method, path) == ("POST", "/v1/systemone")+ assert body == {"state": {"text": "billed twice"}, "questions": _questions(),+ "lang": "en", "max_len": 512, "min_confidence": 0.3}+ assert "Authorization" not in headers+++def test_auto_model_is_not_sent_but_a_pinned_one_is(serve):+ router = RemoteRouter(serve)+ router.predict("x", _questions(), model="auto")+ assert "model" not in FakeServe.requests[-1][2]+ router.predict("x", _questions(), model="multilingual")+ assert FakeServe.requests[-1][2]["model"] == "multilingual"+++def test_bearer_token_is_sent_when_configured(serve):+ FakeServe.require_bearer = "s3cret"+ router = RemoteRouter(serve, api_key="s3cret")+ assert router.predict("x", _questions()) == ANSWER+ assert FakeServe.requests[-1][3]["Authorization"] == "Bearer s3cret"+ with pytest.raises(RemoteError) as info:+ RemoteRouter(serve, api_key="wrong").predict("x", _questions())+ assert info.value.code == "unauthorized"++[email protected]("status, code", [(400, "invalid_request"), (422, "invalid_request"),+ (503, "busy"), (500, "upstream_error")])+def test_http_errors_map_to_tool_error_codes(serve, status, code):+ FakeServe.status_override = status+ FakeServe.detail_override = "question 'dept': criteria must be an object"+ with pytest.raises(RemoteError) as info:+ RemoteRouter(serve).predict("x", _questions())+ assert info.value.code == code+ assert "criteria must be an object" in info.value.message+++def test_unreachable_server_is_a_clean_error():+ with pytest.raises(RemoteError) as info:+ RemoteRouter("http://127.0.0.1:9", timeout=2).predict("x", _questions())+ assert info.value.code == "unreachable"+++def test_loaded_and_revisions_come_from_health(serve):+ router = RemoteRouter(serve)+ assert router.loaded == ["english"]+ assert router.loaded_revisions == {"english": "abc"}+ assert RemoteRouter("http://127.0.0.1:9", timeout=2).loaded == []+++def test_routing_stays_local_and_needs_no_server(monkeypatch):+ router = RemoteRouter("http://127.0.0.1:9", timeout=2)+ def no_http(*args, **kwargs):+ raise AssertionError("routing attempted an HTTP request")+ monkeypatch.setattr(router, "_request", no_http)+ decision = router.route({"text": "Mein Konto wurde zweimal belastet"}, _questions())+ assert decision.model == "multilingual"+ assert router.route_batch([{"state": "Mein Konto wurde zweimal belastet", "questions": _questions()}])[0].model == "multilingual"+++def test_predict_batch_is_one_call_per_request_in_order(serve):+ router = RemoteRouter(serve)+ items = [{"state": "a", "questions": _questions()},+ {"state": "b", "questions": _questions(), "model": "english", "max_len": 256}]+ results = router.predict_batch(items, batch_size=8, sort_by_length=True, min_confidence=0.5)+ assert results == [ANSWER, ANSWER]+ posts = [r for r in FakeServe.requests if r[0] == "POST"]+ assert [p[2]["state"] for p in posts] == ["a", "b"]+ assert posts[1][2]["model"] == "english" and posts[1][2]["max_len"] == 256+ assert all(p[2]["min_confidence"] == 0.5 for p in posts)+++def test_lifecycle_is_server_owned(serve):+ router = RemoteRouter(serve)+ assert router.preload(["english"]) is router+ assert router.unload() is None+ with pytest.raises(RemoteError) as info:+ router.load("english")+ assert info.value.code == "unsupported_remote"+++def test_tools_translate_remote_errors(serve):+ FakeServe.status_override = 422+ FakeServe.detail_override = "question 'dept': unknown type"+ with pytest.raises(ToolError) as info:+ laya_predict({"text": "x"}, _questions(), router=RemoteRouter(serve))+ assert info.value.code == "invalid_request" and "unknown type" in info.value.message+ with pytest.raises(ToolError) as info:+ laya_predict_batch([{"state": {"text": "x"}, "questions": _questions()}], router=RemoteRouter(serve))+ assert info.value.code == "invalid_request"+++def test_predict_tool_shape_through_a_remote_router(serve):+ out = laya_predict({"text": "billed twice"}, _questions(), router=RemoteRouter(serve))+ assert out["answers"] == ANSWER["answers"]+ assert out["routing"]["model"] == "english"+ assert "device" not in out # no in-process agent to read a device from+ assert out["latency_ms"] >= 0+++def test_status_reports_the_server_not_local_torch(serve):+ out = laya_status(router=RemoteRouter(serve), preload=False)+ assert out["mode"] == "remote" and out["base_url"] == serve+ assert out["loaded"] == ["english"] and out["device"] == "mps"+ assert out["server"]["status"] == "ok"+ assert "torch_version" not in out+ down = laya_status(router=RemoteRouter("http://127.0.0.1:9", timeout=2))+ assert down["mode"] == "remote" and down["loaded"] == [] and "server_error" in down+++def test_shortlist_is_refused_in_remote_mode(serve):+ many = {"pick": {"type": "choice", "instructions": "pick", "criteria": {"opt%d" % i: None for i in range(60)}}}+ with pytest.raises(ToolError) as info:+ laya_shortlist({"text": "x"}, many, k=5, router=RemoteRouter(serve))+ assert info.value.code == "unsupported_remote"+++def test_server_builds_a_remote_router_from_the_environment(monkeypatch, serve):+ monkeypatch.setenv("LAYA_BASE_URL", serve + "/")+ monkeypatch.setenv("LAYA_API_KEY", "k")+ monkeypatch.setattr(mcp_server, "_ROUTER", None)+ router = mcp_server._ensure_router()+ assert isinstance(router, RemoteRouter)+ assert router.base_url == serve and router.api_key == "k"+ monkeypatch.setattr(mcp_server, "_ROUTER", None)++[email protected]("raw, expected", [("", None), (" ", None), ("127.0.0.1:8123", "http://127.0.0.1:8123"),+ ("http://h:1/", "http://h:1"), ("https://laya.example", "https://laya.example")])+def test_base_url_normalisation(monkeypatch, raw, expected):+ monkeypatch.setenv("LAYA_BASE_URL", raw)+ assert mcp_server._remote_base_url() == expected++[email protected]("raw", ["bad", "0", "-1", "nan", "inf", "1e999"])+def test_remote_timeout_rejects_invalid_environment(monkeypatch, raw):+ monkeypatch.setenv("LAYA_REMOTE_TIMEOUT", raw)+ assert RemoteRouter("http://localhost:8000").timeout == DEFAULT_TIMEOUT_S++[email protected]("timeout", [0, -1, float("nan"), float("inf")])+def test_remote_timeout_rejects_invalid_explicit_values(timeout):+ with pytest.raises(ValueError, match="finite positive"):+ RemoteRouter("http://localhost:8000", timeout=timeout)++[email protected]("key, value", [+ ("hooks", []), ("on_predict_start", lambda ctx: None), ("on_predict_end", lambda ctx: None),+ ("hooks_raise", False), ("hooks_timeout", 1), ("lang_guess", lambda state: "en"),+])+def test_remote_predict_refuses_controls_without_a_wire_form(serve, key, value):+ with pytest.raises(RemoteError) as info:+ RemoteRouter(serve).predict("x", _questions(), **{key: value})+ assert info.value.code == "unsupported_remote"+ assert not FakeServe.requests+++def test_remote_batch_refuses_hook_timeouts_and_item_hooks(serve):+ router = RemoteRouter(serve)+ items = [{"state": "x", "questions": _questions()}]+ with pytest.raises(RemoteError, match="hooks_timeout"):+ router.predict_batch(items, hooks_timeout=1)+ items[0]["hooks"] = [lambda ctx: None]+ with pytest.raises(RemoteError, match="per-request hooks"):+ router.predict_batch(items)+ assert not FakeServe.requests+++def test_remote_decide_and_preset_use_the_server(serve):+ router = RemoteRouter(serve)+ schema = {"type": "object", "properties": {"dept": {"type": "string", "enum": ["billing", "tech"]}}}+ result = laya_decide({"text": "billed twice"}, schema, router=router)+ assert result["values"] == {"dept": "billing"}+ assert FakeServe.requests[-1][2]["questions"]["dept"]["criteria"] == {"billing": None, "tech": None}+ laya_preset("triage", {"text": "billed twice"}, router=router, preset_builder=mcp_server._preset_builder)+ assert FakeServe.requests[-1][1] == "/v1/systemone"+ FakeServe.status_override = 503+ for fn in (lambda: laya_decide({"text": "x"}, schema, router=router),+ lambda: laya_preset("triage", {"text": "x"}, router=router, preset_builder=mcp_server._preset_builder)):+ with pytest.raises(ToolError) as info:+ fn()+ assert info.value.code == "busy"+++def test_remote_route_tools_do_not_contact_the_server(serve):+ router = RemoteRouter(serve)+ state = {"text": "Mein Konto wurde zweimal belastet"}+ assert laya_route(state, _questions(), router=router)["model"] == "multilingual"+ assert laya_route_batch([{"state": state, "questions": _questions()}], router=router)["decisions"][0]["model"] == "multilingual"+ assert not FakeServe.requests++[email protected]("url", ["file:///tmp/model", "localhost:8000", "http://", "http://host?q=x", "http://host#x"])+def test_remote_router_requires_an_http_server_url(url):+ with pytest.raises(ValueError, match="HTTP"):+ RemoteRouter(url)+++def test_remote_mcp_startup_and_every_tool_avoid_torch(serve):+ import os+ import subprocess+ import sys+ import textwrap++ code = textwrap.dedent('''+ import builtins, json, sys+ original_import = builtins.__import__+ def no_torch(name, *args, **kwargs):+ if name == "torch" or name.startswith("torch."):+ raise AssertionError("remote MCP imported torch")+ return original_import(name, *args, **kwargs)+ builtins.__import__ = no_torch+ from laya.mcp import server+ server.server.run = lambda: None+ server.main()+ state = {"text": "billed twice"}+ questions = {"dept": {"type": "choice", "instructions": "which team?",+ "criteria": {"billing": "payments", "tech": "bugs"}}}+ schema = {"type": "object", "properties": {"dept": {"type": "string", "enum": ["billing", "tech"]}}}+ items = [{"state": state, "questions": questions}]+ calls = [lambda: server.laya_status_tool(),+ lambda: server.laya_predict_tool(state, questions),+ lambda: server.laya_predict_batch_tool(items),+ lambda: server.laya_decide_tool(state, schema),+ lambda: server.laya_preset_tool("triage", state),+ lambda: server.laya_route_tool(state, questions),+ lambda: server.laya_route_batch_tool(items)]+ for call in calls:+ out = json.loads(call())+ assert "error" not in out, out+ try:+ server.laya_shortlist_tool(state, questions)+ except Exception as exc:+ assert "unsupported_remote" in str(exc), exc+ else:+ raise AssertionError("remote shortlist did not refuse the call")+ assert "torch" not in sys.modules+ assert not server._ROUTER._agents+ ''')+ env = dict(os.environ, LAYA_BASE_URL=serve, LAYA_THREADS="2", LAYA_PRELOAD="1")+ result = subprocess.run([sys.executable, "-c", code], env=env, capture_output=True, text=True, timeout=15)+ assert result.returncode == 0, result.stderr+++def test_remote_timeout_is_forwarded_and_http_failures_are_validated(monkeypatch, serve):+ monkeypatch.setenv("LAYA_REMOTE_TIMEOUT", "0.25")+ router = RemoteRouter(serve)+ assert router.timeout == 0.25+ assert RemoteRouter(serve, timeout=3).timeout == 3+ from urllib.error import URLError+ import urllib.request+ def timed_out(*args, **kwargs):+ raise URLError(TimeoutError("test timeout"))+ monkeypatch.setattr(urllib.request, "urlopen", timed_out)+ with pytest.raises(RemoteError) as info:+ router.predict("x", _questions())+ assert info.value.code == "timeout"+++def test_remote_transport_preserves_options_and_all_wire_controls(serve):+ questions = {"dept": {"type": "choice", "instructions": "Which team handles `message`?",+ "criteria": {"billing": "refunds", "tech": "bugs"}}}+ controls = {"task": "typed_decisions", "lang": "de", "lang_guess": "de", "max_len": 512,+ "head_max_len": 128, "min_confidence": 0.7}+ state = {"message": "Rückerstattung bitte"}+ RemoteRouter(serve).predict(state, questions, **controls)+ assert FakeServe.requests[-1][2] == dict(state=state, questions=questions, **controls)++[email protected]("raw", [b"not JSON", b"\xff", b"[]", b"null"])+def test_remote_transport_reports_malformed_responses(monkeypatch, raw):+ from contextlib import contextmanager+ from types import SimpleNamespace+ import urllib.request++ @contextmanager+ def response(*args, **kwargs):+ yield SimpleNamespace(read=lambda: raw)+ monkeypatch.setattr(urllib.request, "urlopen", response)+ with pytest.raises(RemoteError) as info:+ RemoteRouter("http://localhost:8000").predict("x", _questions())+ assert info.value.code == "upstream_error"