Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(mcp): normalise LAYA_DEVICE once, so torch and laya_status agree
Bruce-Yiib86b81bf7c68
diff --git a/README.md b/README.mdindex 4aadb0c4..a7ef4a4c 100644--- a/README.md+++ b/README.md@@ -926,7 +926,7 @@ package:| Variable | Default | Meaning ||---|---|---|-| `LAYA_DEVICE` | (auto) | Same as `laya.serve`: the value is passed straight to torch |+| `LAYA_DEVICE` | (auto) | Same as `laya.serve`: the device type is lower-cased (`CUDA` → `cuda`, `CUDA:0` → `cuda:0`) and passed to torch, whose device parser is case-sensitive || `LAYA_PRELOAD` | `1` | Same as `laya.serve`: build the checkpoints at startup, not lazily || `LAYA_MODELS` | `english,multilingual` | Comma list to preload (serve contract). MCP difference: an empty value preloads `english,multilingual` so `typed-decisions` stays lazy; in `laya.serve` empty means every checkpoint || `LAYA_THREADS` | (torch default) | Same as `laya.serve`: cap torch intra-op threads for CPU inference; keep it at or below the physical core count |diff --git a/laya/mcp/device.py b/laya/mcp/device.pyindex ee65bca0..2bf58078 100644--- a/laya/mcp/device.py+++ b/laya/mcp/device.py@@ -1,7 +1,10 @@"""Device resolution: LAYA_DEVICE has the same meaning as in laya.serve.-The environment variable is passed straight to torch (``Router(device=...)``);-``resolve_device`` is a best-effort label of the *configured preference*+The environment variable is normalised once, in `env_device`, and passed on to torch+(``Router(device=...)``); `resolve_device` reports that same string, so the device a client is+told about and the device torch is asked for cannot drift apart. Normalisation lower-cases the+device type, because torch's parser is case-sensitive.+`resolve_device` is a best-effort label of the *configured preference*(LAYA_DEVICE or torch auto-detection), not of the device a loaded checkpointactually runs on. For the real device read ``Agent.device`` via``agent_device`` / ``router_agent``: the Agent falls back to CPU silently when@@ -18,12 +21,27 @@ _ENV_KEY = "LAYA_DEVICE"def env_device() -> str | None:- """LAYA_DEVICE verbatim (empty or unset = None), ready for Router(device=...)."""+ """LAYA_DEVICE (empty or unset = None), ready for Router(device=...).++ Lower-cased, not verbatim. torch's device parser is case-sensitive -- it accepts ``cuda``+ and rejects ``CUDA`` with "Expected one of cpu, cuda, ..." -- so a value this function+ returned unchanged could fail the Router build while `laya_status` reported the+ lower-cased form as the working device. The same normalisation `resolve_device` applies+ is applied here, so the value handed to torch and the value reported are one string.++ Splitting on the device *type* and the optional index keeps that true for ``CUDA:0``:+ torch wants the type lower-cased and does not care about the index, so+ ``CUDA:0 -> cuda:0``.+ """value = os.environ.get(_ENV_KEY)if value is None:return Nonevalue = value.strip()- return value or None+ if not value:+ return None+ kind, sep, index = value.partition(":")+ kind = kind.lower()+ return "%s%s%s" % (kind, sep, index) if sep else kinddef resolve_device(force: str | None = None) -> str:@@ -43,7 +61,7 @@ def resolve_device(force: str | None = None) -> str:return forcevalue = env_device()if value is not None:- return value.lower()+ return valuetry:import torchdiff --git a/tests/test_mcp.py b/tests/test_mcp.pyindex ce228995..f5262ec5 100644--- a/tests/test_mcp.py+++ b/tests/test_mcp.py@@ -89,12 +89,22 @@ def test_device():else:os.environ["LAYA_DEVICE"] = oldok("device/fallback", resolve_device(None) in ("cuda", "mps", "xpu", "cpu"))- # laya.serve contract: LAYA_DEVICE goes verbatim to torch; the label is lowercased.+ # laya.serve contract: LAYA_DEVICE is normalised to what torch's device parser accepts --+ # the device type is lower-cased, the optional index is left alone -- and the same string is+ # both handed to torch and reported by `resolve_device`.old_dev = os.environ.get("LAYA_DEVICE")try:os.environ["LAYA_DEVICE"] = "cuda:1"- ok("device/env_raw_for_torch", env_device() == "cuda:1")+ ok("device/env_for_torch", env_device() == "cuda:1")ok("device/env_label", resolve_device() == "cuda:1")+ # torch's parser is case-sensitive, so the type is normalised rather than passed through.+ os.environ["LAYA_DEVICE"] = "CUDA"+ ok("device/env_upper_type_lowercased", env_device() == "cuda")+ ok("device/env_upper_matches_label", env_device() == resolve_device())+ os.environ["LAYA_DEVICE"] = "CUDA:0"+ ok("device/env_upper_index_kept", env_device() == "cuda:0")+ os.environ["LAYA_DEVICE"] = " CPU "+ ok("device/env_padded_upper_lowercased", env_device() == "cpu")os.environ["LAYA_DEVICE"] = " "ok("device/env_blank_none", env_device() is None)finally: