Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
build(docker): keep laya-serve on loopback and add a healthcheck (#262)
Maxfield Allison706bb18d05bc
diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.ymlindex f0233af4..0def775e 100644--- a/.github/workflows/docker.yml+++ b/.github/workflows/docker.yml@@ -20,6 +20,8 @@ jobs:docker compose -f compose.yaml -f compose.example.yml config --quietdocker compose -f compose.yaml -f compose.example.yml -f compose.cuda.yaml config --quietdocker compose -f compose.yaml -f compose.cuda.yaml config --quiet+ docker compose -f compose.yaml -f compose.http.yaml config --quiet+ docker compose -f compose.yaml -f compose.http.yaml -f compose.cuda.yaml config --quietdocker build --pull -t laya:smoke .- name: Check installed runtime and fresh cache permissionsrun: |@@ -67,3 +69,30 @@ jobs:- name: Remove test cacheif: always()run: docker volume rm laya-smoke-cache || true++ http:+ # Start laya-serve the way the docs do and wait for its healthcheck. No preload,+ # so no weights download; the key is a throwaway written for this job.+ runs-on: ubuntu-latest+ timeout-minutes: 20+ env:+ COMPOSE_FILE: compose.yaml:compose.http.yaml+ LAYA_PRELOAD: "0"+ LAYA_API_KEY: ci-test-key+ steps:+ - uses: actions/checkout@v4+ - name: Start laya-serve and wait until healthy+ run: docker compose up --build -d --wait --wait-timeout 300 laya-serve+ - name: Check binding, health and bearer auth+ run: |+ docker compose port laya-serve 8000 | grep -q '^127.0.0.1:'+ curl -fsS localhost:8000/health+ code() { curl -s -o /dev/null -w '%{http_code}' -X POST localhost:8000/v1/systemone \+ -H 'content-type: application/json' -d '{}' "$@"; }+ test "$(code)" = 401+ test "$(code -H 'authorization: Bearer wrong')" = 401+ - name: Stop+ if: always()+ run: |+ docker compose logs laya-serve || true+ docker compose down --volumes || truediff --git a/compose.cuda.yaml b/compose.cuda.yamlindex dfc0710e..71dc27ed 100644--- a/compose.cuda.yaml+++ b/compose.cuda.yaml@@ -1,7 +1,6 @@# Use with compose.yaml, and with compose.http.yaml when serving over HTTP. The-# `laya-serve` override repeats the CUDA build args and device reservation, because a-# Compose override replaces a service's `build` block whole rather than merging it, so-# adding this file without repeating them would silently serve on CPU.+# `laya-serve` entry repeats the CUDA build args and device reservation because it is+# a separate service: overrides for `laya` never reach it, so it would serve on CPU.services:laya:build:diff --git a/compose.http.yaml b/compose.http.yamlindex cb062316..a3eb8272 100644--- a/compose.http.yaml+++ b/compose.http.yaml@@ -27,9 +27,11 @@ services:# secrets still runs first.command: ["laya-serve"]ports:- # Host port first, container port second. Both come from LAYA_PORT, which- # `laya-serve` itself reads, so they cannot drift apart.- - "${LAYA_PORT:-8000}:${LAYA_PORT:-8000}"+ # Host address, host port, container port. The port comes from LAYA_PORT, which+ # `laya-serve` itself reads, so the two sides cannot drift apart. The host side+ # stays on loopback unless LAYA_BIND_ADDRESS says otherwise, because the API has+ # no authentication until LAYA_API_KEY is set.+ - "${LAYA_BIND_ADDRESS:-127.0.0.1}:${LAYA_PORT:-8000}:${LAYA_PORT:-8000}"environment:LAYA_HOST: "${LAYA_HOST:-0.0.0.0}"LAYA_PORT: "${LAYA_PORT:-8000}"@@ -54,6 +56,15 @@ services:# - ${LAYA_API_KEY_PATH:?Set LAYA_API_KEY_PATH}:/run/secrets/laya_api_key:roinit: truerestart: unless-stopped+ # The server preloads before it binds, so /health answers once the preloaded+ # checkpoints are ready. `up --wait` therefore waits for a usable server.+ healthcheck:+ test: ["CMD", "python", "-c", "import os, urllib.request; urllib.request.urlopen('http://127.0.0.1:' + os.environ['LAYA_PORT'] + '/health', timeout=4)"]+ interval: 30s+ timeout: 5s+ retries: 3+ start_period: 10m+ start_interval: 2svolumes:model-cache:diff --git a/docs/docker.md b/docs/docker.mdindex 4d084d3a..37489043 100644--- a/docs/docker.md+++ b/docs/docker.md@@ -179,8 +179,8 @@ curl -s localhost:8000/v1/systemone -H 'content-type: application/json' \```For NVIDIA, add the CUDA override. It repeats the build args and the device reservation-for `laya-serve`, because a Compose override replaces a service's `build` block whole-rather than merging it:+for `laya-serve`, because `laya-serve` is a separate service and overrides for `laya`+never reach it:```bashdocker compose -f compose.yaml -f compose.http.yaml -f compose.cuda.yaml up --build laya-serve@@ -191,6 +191,15 @@ named `model-cache` volume as the quickstart, so serving after a quickstart runwith the checkpoints already on disk. Stop with `docker compose ... down`, using the sameCompose files.+The port is published on `127.0.0.1` only. The API has no authentication until+`LAYA_API_KEY` is set, so set a key before exposing it with+`LAYA_BIND_ADDRESS=0.0.0.0`, and put a TLS reverse proxy in front for remote clients.+`/health` does not require authentication in either case.++The service has a healthcheck on `/health`. The server preloads before it starts+listening, so with `LAYA_PRELOAD=1` a healthy container has its checkpoints loaded.+`docker compose ... up -d --wait laya-serve` returns once it is healthy.+### Server configurationThese apply to the `laya-serve` service only.@@ -199,6 +208,7 @@ These apply to the `laya-serve` service only.|---|---|---|| `LAYA_HOST` | `0.0.0.0` | bind address inside the container || `LAYA_PORT` | `8000` | container port, and the host port published for it |+| `LAYA_BIND_ADDRESS` | `127.0.0.1` | host address the port is published on || `LAYA_PRELOAD` | `0` | `1` builds every checkpoint at startup instead of on first request || `LAYA_MODELS` | (all) | comma list to preload: `english,multilingual,typed-decisions` || `LAYA_THREADS` | `OMP_NUM_THREADS` | caps torch intra-op threads; keep at or below physical cores |diff --git a/tests/test_packaging.py b/tests/test_packaging.pyindex 585124b2..e3995249 100644--- a/tests/test_packaging.py+++ b/tests/test_packaging.py@@ -161,7 +161,12 @@ check_true("compose.http/runs the server command",check_true("compose.http/publishes a port", re.search(r"^\s*ports:", http, re.M) is not None)# Host and container port must come from the same variable, or they drift apart and the# published port stops reaching the server.-port_map = re.search(r'-\s*"(\$\{[A-Z_]+:-(\d+)\}):(\$\{[A-Z_]+:-(\d+)\})"', http)+port_map = re.search(r'-\s*"(?:\$\{LAYA_BIND_ADDRESS:-[^}]+\}:)?(\$\{[A-Z_]+:-(\d+)\}):(\$\{[A-Z_]+:-(\d+)\})"', http)+# The API is unauthenticated until LAYA_API_KEY is set, so exposure beyond the host is opt-in.+check_true("compose.http/publishes on loopback unless LAYA_BIND_ADDRESS is set",+ '"${LAYA_BIND_ADDRESS:-127.0.0.1}:' in http)+check_true("compose.http/has a healthcheck on /health",+ "healthcheck:" in http and "/health" in http)check_true("compose.http/port mapping is present", port_map is not None, http[:300])if port_map:check("compose.http/host port equals container port", port_map.group(2), port_map.group(4))@@ -179,7 +184,7 @@ check_true("compose.http/leaves the quickstart service alone",check_true("Dockerfile/installs the serve extra", '".[serve]"' in dockerfile, dockerfile[:400])check_true("Dockerfile/still runs pip check", "pip check" in dockerfile)-# A Compose override replaces a service's `build` block whole. If the CUDA override does+# Overrides for `laya` never reach `laya-serve`, a separate service. If the CUDA override does# not repeat the args for laya-serve, that service silently serves on CPU.check_true("compose.cuda/covers laya-serve too",re.search(r"^\s{2}laya-serve:", cuda, re.M) is not None,