Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(mcp): reject a LAYA_DEVICE value torch cannot parse where it is read
Aashishe1aeb932112b
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 39843b20..43ffce5a 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -145,6 +145,7 @@ jobs:python tests/test_confidence.pypython tests/test_cli.pypython tests/test_mcp.py+ python tests/test_mcp_device.pypython tests/test_langchain.pypython tests/test_portability.pypython tests/test_training.py@@ -308,6 +309,7 @@ jobs:test_criteria_normalization test_docker_entrypoint \test_empty_questions test_router_memory test_shortlist_cosine \test_temperature_loading test_confidence test_cli test_mcp \+ test_mcp_device \test_langchain test_portability test_training \test_example_server_limits test_blank_lang_routing \test_export_onnx_safety test_load_errors test_revision_pinning; dodiff --git a/laya/mcp/device.py b/laya/mcp/device.pyindex 2bf58078..d82ec3be 100644--- a/laya/mcp/device.py+++ b/laya/mcp/device.py@@ -4,6 +4,14 @@ The environment variable is normalised once, in `env_device`, and passed on to t(``Router(device=...)``); `resolve_device` reports that same string, so the device a client istold about and the device torch is asked for cannot drift apart. Normalisation lower-cases thedevice type, because torch's parser is case-sensitive.++Normalisation is not sufficient for that guarantee on its own. ``cuda:`` -- or ``gpu``, or+``cuda: 0`` -- normalises to a string torch's parser still refuses, and the refusal lands+wherever the first checkpoint build happens: on the lazy Router that is inside a request+handler, after the server started healthily and ``laya_status`` reported the string as the+device in use. `env_device` therefore parses the value it is about to return with+``torch.device`` and raises a named ``ValueError`` for one torch cannot read, so the typo is+attributed to the variable at the point the process reads it.`resolve_device` is a best-effort label of the *configured preference*(LAYA_DEVICE or torch auto-detection), not of the device a loaded checkpointactually runs on. For the real device read ``Agent.device`` via@@ -32,6 +40,12 @@ def env_device() -> str | None:Splitting on the device *type* and the optional index keeps that true for ``CUDA:0``:torch wants the type lower-cased and does not care about the index, so``CUDA:0 -> cuda:0``.++ The normalised value is then parsed by torch (``_check_torch_device``): a value torch+ cannot read raises ``ValueError`` naming ``LAYA_DEVICE``, before the string can reach+ ``Router(device=...)`` or be reported as the device in use. ``cuda:`` (trailing colon),+ ``gpu`` and ``cuda: 0`` (space after the colon) are the common typos; ``CUDA:0`` is not+ one, because it is normalised first."""value = os.environ.get(_ENV_KEY)if value is None:@@ -41,7 +55,30 @@ def env_device() -> str | None:return Nonekind, sep, index = value.partition(":")kind = kind.lower()- return "%s%s%s" % (kind, sep, index) if sep else kind+ normalised = "%s%s%s" % (kind, sep, index) if sep else kind+ _check_torch_device(normalised)+ return normalised+++def _check_torch_device(value: str) -> None:+ """Raise a named ValueError for a device string ``torch.device`` refuses.++ ``torch.device`` is used as the parser itself rather than a lookalike of it: devices+ this module has never heard of (``meta``, ``privateuseone:1``) keep working, and a+ torch that cannot be imported leaves the value alone, because nothing downstream+ could have used it anyway.+ """+ try:+ import torch+ except Exception: # pragma: no cover - torch is a hard dependency of the package+ return+ try:+ torch.device(value)+ except Exception as exc:+ raise ValueError(+ "%s=%r is not a torch device: %s -- use a device such as 'cpu', 'cuda', "+ "'cuda:0' or 'mps'" % (_ENV_KEY, value, exc)+ ) from excdef resolve_device(force: str | None = None) -> str:@@ -56,6 +93,10 @@ def resolve_device(force: str | None = None) -> str:This is what the Router is asked to build on, not what a loaded agentcomputes on: use ``agent_device`` for the real device (a GPU -> CPUfallback happens silently at agent build time).++ A ``LAYA_DEVICE`` value torch cannot parse raises here too (see+ ``env_device``): there is no path on which this function labels a device+ torch would refuse, so the label and the build cannot disagree."""if force:return forcediff --git a/tests/test_mcp_device.py b/tests/test_mcp_device.pynew file mode 100644index 00000000..1b29a7cc--- /dev/null+++ b/tests/test_mcp_device.py@@ -0,0 +1,141 @@+"""LAYA_DEVICE normalisation *and* parseability: the laya.mcp.device env contract.++The variable is read once, in `env_device`, and the same string is handed to torch+(``Router(device=...)``) and reported by `resolve_device` / `laya_status`. Case and+whitespace normalisation was already covered (``CUDA:0 -> cuda:0``); what this suite+adds is the other half of "torch can use what we return": ``cuda:`` (or ``gpu``, or+``cuda: 0``) normalises cleanly and is still refused by torch -- so on a lazy Router it+survived startup, was reported as the configured device, and the refusal landed inside+the first request that built a checkpoint. `env_device` now parses the value it returns+and raises a named ValueError where the environment is read.++No model weights, no network, no GPU: torch is imported for its parser only.+Run: python tests/test_mcp_device.py+"""+import os+import sys+from pathlib import Path++sys.path.insert(0, str(Path(__file__).resolve().parents[1]))++try:+ import torch # noqa: F401 (the parser under test)+except ImportError:+ print("SKIP: torch not importable, so there is no device parser to check")+ sys.exit(0)++from laya.mcp.device import env_device, resolve_device # noqa: E402++PASS, FAIL = [], []+++def ok(name, cond, detail=""):+ (PASS if cond else FAIL).append("%s%s" % (name, (": " + detail) if detail and not cond else ""))+ print(" %s %s%s" % ("PASS" if cond else "FAIL", name, (" " + detail) if detail else ""), flush=True)+++def env_device_with(raw):+ """`env_device()` with LAYA_DEVICE set to raw (None = unset); the old value is restored."""+ old = os.environ.get("LAYA_DEVICE")+ try:+ if raw is None:+ os.environ.pop("LAYA_DEVICE", None)+ else:+ os.environ["LAYA_DEVICE"] = raw+ return env_device()+ finally:+ if old is None:+ os.environ.pop("LAYA_DEVICE", None)+ else:+ os.environ["LAYA_DEVICE"] = old+++def rejection(raw):+ """The ValueError message `env_device()` raises for raw, or "" if it does not raise."""+ try:+ env_device_with(raw)+ except ValueError as exc:+ return str(exc)+ except BaseException as exc: # noqa: BLE001 - a wrong exception is a failed case, not an abort+ return "wrong exception %r" % (exc,)+ return ""+++def test_unset_and_blank_stay_none():+ ok("unset -> None", env_device_with(None) is None)+ ok("empty -> None", env_device_with("") is None)+ ok("spaces -> None", env_device_with(" ") is None)+++def test_normalisation_still_applies_first():+ ok("cpu verbatim", env_device_with("cpu") == "cpu")+ ok("case + spaces + index", env_device_with(" CUDA:0 ") == "cuda:0")+ ok("index preserved", env_device_with("cuda:1") == "cuda:1")+++def test_the_parser_is_the_judge_not_an_allowlist():+ # torch.device("meta") parses; a hand-written {cpu, cuda, mps} allowlist would not,+ # and would make a device this module has never heard of impossible to configure.+ ok("meta (valid for torch, never loadable here)", env_device_with("meta") == "meta")+++def test_unparseable_values_are_rejected_by_name():+ for raw in ("cuda:", "gpu", "cuda: 0"):+ message = rejection(raw)+ ok("%r -> ValueError" % raw, bool(message) and "wrong exception" not in message, message or "no error")+ ok("%r names LAYA_DEVICE" % raw, "LAYA_DEVICE" in message, message)+ ok("%r quotes the offending value" % raw, raw.strip() in message, message)+ ok("'cuda:' carries torch's own reason",+ "Invalid device string" in rejection("cuda:"), rejection("cuda:"))+ ok("'gpu' carries torch's own list of device types",+ "Expected one of" in rejection("gpu"), rejection("gpu"))+++def test_no_reporting_path_labels_a_refused_device():+ old = os.environ.get("LAYA_DEVICE")+ try:+ os.environ["LAYA_DEVICE"] = "cuda:"+ try:+ resolve_device()+ except ValueError:+ ok("resolve_device() refuses it too", True)+ else:+ ok("resolve_device() refuses it too", False, "returned a label for 'cuda:'")+ finally:+ if old is None:+ os.environ.pop("LAYA_DEVICE", None)+ else:+ os.environ["LAYA_DEVICE"] = old+ # force= is the documented tests hook: it is returned verbatim, exactly like the+ # CLI --device flag, which never reads the environment.+ ok("force= hook stays verbatim", resolve_device("cuda:") == "cuda:")+++def test_reported_equals_handed_to_torch():+ old = os.environ.get("LAYA_DEVICE")+ try:+ os.environ["LAYA_DEVICE"] = "CUDA:1"+ ok("env_device == resolve_device", env_device() == resolve_device() == "cuda:1")+ finally:+ if old is None:+ os.environ.pop("LAYA_DEVICE", None)+ else:+ os.environ["LAYA_DEVICE"] = old+++def main():+ for fn in (test_unset_and_blank_stay_none,+ test_normalisation_still_applies_first,+ test_the_parser_is_the_judge_not_an_allowlist,+ test_unparseable_values_are_rejected_by_name,+ test_no_reporting_path_labels_a_refused_device,+ test_reported_equals_handed_to_torch):+ fn()+ print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))+ for name in FAIL:+ print(" FAIL " + name)+ return 1 if FAIL else 0+++if __name__ == "__main__":+ sys.exit(main())