Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

fix(compose): forward extra models and idle unload knobs to laya-serve

naveenprasaath235067f79e611e
diff --git a/compose.http.yaml b/compose.http.yaml
index 7d85a4f1..1e4d56b4 100644
--- a/compose.http.yaml
+++ b/compose.http.yaml
@@ -51,6 +51,12 @@ services:
# default (english); set `multilingual` for mostly-non-English traffic. An unresolvable
# name stops the container at startup rather than serving a configuration nobody asked for.
LAYA_DEFAULT_MODEL: "${LAYA_DEFAULT_MODEL:-}"
+ # Extra checkpoints beside the bundled ones, as a JSON object {name: source}. Empty registers
+ # nothing; a malformed value or bad name stops the container at startup.
+ LAYA_EXTRA_MODELS: "${LAYA_EXTRA_MODELS:-}"
+ # Unload resident checkpoints after this many idle seconds. Empty keeps the server default (0,
+ # disabled); the next request loads its checkpoint again.
+ LAYA_IDLE_UNLOAD_SECONDS: "${LAYA_IDLE_UNLOAD_SECONDS:-}"
# Empty keeps Router's own default (2). Set 3 when LAYA_AUTO_TASK is on: that adds a
# third checkpoint routing can choose on demand, and a cap below it reloads one per switch.
LAYA_MAX_LOADED: "${LAYA_MAX_LOADED:-}"
diff --git a/docs/docker.md b/docs/docker.md
index dcabf649..a872f43e 100644
--- a/docs/docker.md
+++ b/docs/docker.md
@@ -339,6 +339,8 @@ These apply to the `laya-serve` service only.
| `LAYA_THREADS` | `OMP_NUM_THREADS` | caps torch intra-op threads; keep at or below physical cores |
| `LAYA_AUTO_TASK` | `0` | `1` lets the router reach `typed-decisions` automatically |
| `LAYA_DEFAULT_MODEL` | `multilingual` | Checkpoint a state with no language evidence falls back to (no letters, or Latin text too short to identify). Set `english` for mostly English traffic; an unresolvable name stops the container at startup instead of serving a configuration nobody asked for |
+| `LAYA_EXTRA_MODELS` | (none) | JSON object `{name: source}` registering extra checkpoints beside the bundled ones: a Hub repo id or local checkpoint directory as a string, or a `["repo", "subfolder"]` pair |
+| `LAYA_IDLE_UNLOAD_SECONDS` | `0` | unload resident checkpoints after this many idle seconds; the next request loads its checkpoint again. Zero disables unloading |
| `LAYA_MAX_LOADED` | `2` | Checkpoints kept resident; `LAYA_AUTO_TASK` makes a third reachable on demand, and a cap below what routing chooses rebuilds one per switch |
| `LAYA_MAX_CONCURRENT` | `16` | requests admitted at once; later ones get `503` (a value that does not parse, or is not positive, falls back to `16`) |
| `LAYA_LOG_LEVEL` | `info` | uvicorn log level |
diff --git a/tests/test_compose_env.py b/tests/test_compose_env.py
index a3152016..be435448 100644
--- a/tests/test_compose_env.py
+++ b/tests/test_compose_env.py
@@ -537,7 +537,8 @@ if server_doc:
_env = env_blocks(read("compose.http.yaml")).get(_service, {})
check_true("compose/compose.http.yaml defines the %s service" % _service,
_service in env_blocks(read("compose.http.yaml")), sorted(_env))
- for _name in ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION"):
+ for _name in ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION",
+ "LAYA_EXTRA_MODELS", "LAYA_IDLE_UNLOAD_SECONDS"):
check_true("compose/compose.http.yaml/%s forwards the %s this change adds"
% (_service, _name), _name in _env,
"it is documented and read by the runtime, and the service that needs it "
@@ -610,8 +611,10 @@ if shutil.which("docker"):
# container is the failure the textual checks above are reasoning about. Both arms: set, and
# left alone. The unset arm is the one an operator actually boots, and it must show the empty
# default arriving as empty rather than as the literal `${LAYA_MAX_CONCURRENT:-}`.
- _KN = ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION")
- _arms = [(dict(zip(_KN, ("4", "16384", "reviewed"))), dict(zip(_KN, ("4", "16384", "reviewed")))),
+ _KN = ("LAYA_MAX_CONCURRENT", "LAYA_MAX_TOKEN_BUDGET", "LAYA_REVISION",
+ "LAYA_EXTRA_MODELS", "LAYA_IDLE_UNLOAD_SECONDS")
+ _arms = [(dict(zip(_KN, ("4", "16384", "reviewed", '{"custom":"./model"}', "30"))),
+ dict(zip(_KN, ("4", "16384", "reviewed", '{"custom":"./model"}', "30")))),
({}, dict.fromkeys(_KN, ""))]
for _set, _expect in _arms:
proc = subprocess.run(["docker", "compose", "-f", "compose.yaml", "-f", "compose.http.yaml",