Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(compose): forward extra models and idle unload knobs to laya-serve
naveenprasaath235067f79e611e
diff --git a/compose.http.yaml b/compose.http.yamlindex 7d85a4f1..1e4d56b4 100644--- a/compose.http.yaml+++ b/compose.http.yaml@@ -51,6 +51,12 @@ services:# default (english); set `multilingual` for mostly-non-English traffic. An unresolvable# name stops the container at startup rather than serving a configuration nobody asked for.LAYA_DEFAULT_MODEL: "${LAYA_DEFAULT_MODEL:-}"+ # Extra checkpoints beside the bundled ones, as a JSON object {name: source}. Empty registers+ # nothing; a malformed value or bad name stops the container at startup.+ LAYA_EXTRA_MODELS: "${LAYA_EXTRA_MODELS:-}"+ # Unload resident checkpoints after this many idle seconds. Empty keeps the server default (0,+ # disabled); the next request loads its checkpoint again.+ LAYA_IDLE_UNLOAD_SECONDS: "${LAYA_IDLE_UNLOAD_SECONDS:-}"# Empty keeps Router's own default (2). Set 3 when LAYA_AUTO_TASK is on: that adds a# third checkpoint routing can choose on demand, and a cap below it reloads one per switch.LAYA_MAX_LOADED: "${LAYA_MAX_LOADED:-}"diff --git a/docs/docker.md b/docs/docker.mdindex dcabf649..a872f43e 100644--- a/docs/docker.md+++ b/docs/docker.md@@ -339,6 +339,8 @@ These apply to the `laya-serve` service only.| `LAYA_THREADS` | `OMP_NUM_THREADS` | caps torch intra-op threads; keep at or below physical cores || `LAYA_AUTO_TASK` | `0` | `1` lets the router reach `typed-decisions` automatically || `LAYA_DEFAULT_MODEL` | `multilingual` | Checkpoint a state with no language evidence falls back to (no letters, or Latin text too short to identify). Set `english` for mostly English traffic; an unresolvable name stops the container at startup instead of serving a configuration nobody asked for |+| `LAYA_EXTRA_MODELS` | (none) | JSON object `{name: source}` registering extra checkpoints beside the bundled ones: a Hub repo id or local checkpoint directory as a string, or a `["repo", "subfolder"]` pair |+| `LAYA_IDLE_UNLOAD_SECONDS` | `0` | unload resident checkpoints after this many idle seconds; the next request loads its checkpoint again. Zero disables unloading || `LAYA_MAX_LOADED` | `2` | Checkpoints kept resident; `LAYA_AUTO_TASK` makes a third reachable on demand, and a cap below what routing chooses rebuilds one per switch || `LAYA_MAX_CONCURRENT` | `16` | requests admitted at once; later ones get `503` (a value that does not parse, or is not positive, falls back to `16`) || `LAYA_LOG_LEVEL` | `info` | uvicorn log level |diff --git a/tests/test_compose_env.py b/tests/test_compose_env.pyindex a3152016..be435448 100644--- a/tests/test_compose_env.py+++ b/tests/test_compose_env.py@@ -537,7 +537,8 @@ if server_doc:_env = env_blocks(read("compose.http.yaml")).get(_service, {})check_true("compose/compose.http.yaml defines the %s service" % _service,_service in env_blocks(read("compose.http.yaml")), sorted(_env))- for _name in ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION"):+ for _name in ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION",+ "LAYA_EXTRA_MODELS", "LAYA_IDLE_UNLOAD_SECONDS"):check_true("compose/compose.http.yaml/%s forwards the %s this change adds"% (_service, _name), _name in _env,"it is documented and read by the runtime, and the service that needs it "@@ -610,8 +611,10 @@ if shutil.which("docker"):# container is the failure the textual checks above are reasoning about. Both arms: set, and# left alone. The unset arm is the one an operator actually boots, and it must show the empty# default arriving as empty rather than as the literal `${LAYA_MAX_CONCURRENT:-}`.- _KN = ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION")- _arms = [(dict(zip(_KN, ("4", "16384", "reviewed"))), dict(zip(_KN, ("4", "16384", "reviewed")))),+ _KN = ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION",+ "LAYA_EXTRA_MODELS", "LAYA_IDLE_UNLOAD_SECONDS")+ _arms = [(dict(zip(_KN, ("4", "16384", "reviewed", '{"custom":"./model"}', "30"))),+ dict(zip(_KN, ("4", "16384", "reviewed", '{"custom":"./model"}', "30")))),({}, dict.fromkeys(_KN, ""))]for _set, _expect in _arms:proc = subprocess.run(["docker", "compose", "-f", "compose.yaml", "-f", "compose.http.yaml",