Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(mcp): say what LAYA_DEVICE actually goes through

Aashishc0439ab246bd
diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md
index 32c25fc3..b8d70cef 100644
--- a/docs/cli-mcp.md
+++ b/docs/cli-mcp.md
@@ -287,7 +287,7 @@ the server is ready.
| Variable | Default | Meaning |
|---|---|---|
-| `LAYA_DEVICE` | automatic | Device value passed to PyTorch, such as `cpu` or `cuda`. |
+| `LAYA_DEVICE` | automatic | The device checkpoints are asked to build on, such as `cpu` or `cuda`. Not the value verbatim: `laya.mcp.device.env_device` strips and lower-cases it, so `CUDA:0` works, and a string PyTorch cannot parse is refused with an error naming this variable. Because there is no startup to refuse, that comes back as a `router construction failed` tool error. It is a preference, not a guarantee: a GPU that cannot be had falls back to CPU silently, and `laya_status` reports the loaded checkpoint's real device. |
| `LAYA_PRELOAD` | `1` | Build the configured checkpoints at startup. Set to `0` for lazy loading. |
| `LAYA_MODELS` | `english,multilingual` | Comma-separated checkpoints to preload. An empty value keeps the MCP default rather than preloading every checkpoint. |
| `LAYA_THREADS` | PyTorch default | Caps Torch intra-op threads for CPU inference; keep it at or below the physical core count. |
diff --git a/laya/mcp/server.py b/laya/mcp/server.py
index f2139760..a18ee4a9 100644
--- a/laya/mcp/server.py
+++ b/laya/mcp/server.py
@@ -7,7 +7,18 @@ wired to any MCP client (OpenClaw, Claude Desktop, cursor, ...):
python -m laya.mcp.server
Environment (same meaning as laya.serve where it exists):
- LAYA_DEVICE torch device for the checkpoints (value goes straight to torch)
+ LAYA_DEVICE the device the checkpoints are asked to build on. Not the value verbatim:
+ laya.mcp.device.env_device strips and lower-cases it (`CUDA:0` -> `cuda:0`)
+ and refuses a string torch cannot parse with a ValueError naming this variable
+ (`gpu`, `cuda:`, `cuda: 0`), so the failure reads as a bad environment value
+ rather than a torch internal one -- here as a `router construction failed` tool
+ error, because a stdio server has no startup to refuse. A preference, not a
+ guarantee: as in laya.serve an unavailable GPU falls back to CPU silently, and
+ laya_status then answers the loaded agent's real device (before any load it
+ answers this preference, flagged with `device_is_preference`). It answers one
+ device, though: with two checkpoints resident on different devices the top-level
+ `device` is the first entry of `loaded` (the least recently used), so
+ `checkpoint_devices` is the field that says where each one actually runs.
LAYA_PRELOAD "0" to build the checkpoints lazily instead of at startup (default on)
LAYA_MODELS comma list to preload; MCP default is "english,multilingual" so
typed-decisions stays lazy (in laya.serve an empty value preloads all)