Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

build(docker): keep laya-serve on loopback and add a healthcheck (#262)

Maxfield Allison706bb18d05bc
diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml
index f0233af4..0def775e 100644
--- a/.github/workflows/docker.yml
+++ b/.github/workflows/docker.yml
@@ -20,6 +20,8 @@ jobs:
docker compose -f compose.yaml -f compose.example.yml config --quiet
docker compose -f compose.yaml -f compose.example.yml -f compose.cuda.yaml config --quiet
docker compose -f compose.yaml -f compose.cuda.yaml config --quiet
+ docker compose -f compose.yaml -f compose.http.yaml config --quiet
+ docker compose -f compose.yaml -f compose.http.yaml -f compose.cuda.yaml config --quiet
docker build --pull -t laya:smoke .
- name: Check installed runtime and fresh cache permissions
run: |
@@ -67,3 +69,30 @@ jobs:
- name: Remove test cache
if: always()
run: docker volume rm laya-smoke-cache || true
+
+ http:
+ # Start laya-serve the way the docs do and wait for its healthcheck. No preload,
+ # so no weights download; the key is a throwaway written for this job.
+ runs-on: ubuntu-latest
+ timeout-minutes: 20
+ env:
+ COMPOSE_FILE: compose.yaml:compose.http.yaml
+ LAYA_PRELOAD: "0"
+ LAYA_API_KEY: ci-test-key
+ steps:
+ - uses: actions/checkout@v4
+ - name: Start laya-serve and wait until healthy
+ run: docker compose up --build -d --wait --wait-timeout 300 laya-serve
+ - name: Check binding, health and bearer auth
+ run: |
+ docker compose port laya-serve 8000 | grep -q '^127.0.0.1:'
+ curl -fsS localhost:8000/health
+ code() { curl -s -o /dev/null -w '%{http_code}' -X POST localhost:8000/v1/systemone \
+ -H 'content-type: application/json' -d '{}' "$@"; }
+ test "$(code)" = 401
+ test "$(code -H 'authorization: Bearer wrong')" = 401
+ - name: Stop
+ if: always()
+ run: |
+ docker compose logs laya-serve || true
+ docker compose down --volumes || true
diff --git a/compose.cuda.yaml b/compose.cuda.yaml
index dfc0710e..71dc27ed 100644
--- a/compose.cuda.yaml
+++ b/compose.cuda.yaml
@@ -1,7 +1,6 @@
# Use with compose.yaml, and with compose.http.yaml when serving over HTTP. The
-# `laya-serve` override repeats the CUDA build args and device reservation, because a
-# Compose override replaces a service's `build` block whole rather than merging it, so
-# adding this file without repeating them would silently serve on CPU.
+# `laya-serve` entry repeats the CUDA build args and device reservation because it is
+# a separate service: overrides for `laya` never reach it, so it would serve on CPU.
services:
laya:
build:
diff --git a/compose.http.yaml b/compose.http.yaml
index cb062316..a3eb8272 100644
--- a/compose.http.yaml
+++ b/compose.http.yaml
@@ -27,9 +27,11 @@ services:
# secrets still runs first.
command: ["laya-serve"]
ports:
- # Host port first, container port second. Both come from LAYA_PORT, which
- # `laya-serve` itself reads, so they cannot drift apart.
- - "${LAYA_PORT:-8000}:${LAYA_PORT:-8000}"
+ # Host address, host port, container port. The port comes from LAYA_PORT, which
+ # `laya-serve` itself reads, so the two sides cannot drift apart. The host side
+ # stays on loopback unless LAYA_BIND_ADDRESS says otherwise, because the API has
+ # no authentication until LAYA_API_KEY is set.
+ - "${LAYA_BIND_ADDRESS:-127.0.0.1}:${LAYA_PORT:-8000}:${LAYA_PORT:-8000}"
environment:
LAYA_HOST: "${LAYA_HOST:-0.0.0.0}"
LAYA_PORT: "${LAYA_PORT:-8000}"
@@ -54,6 +56,15 @@ services:
# - ${LAYA_API_KEY_PATH:?Set LAYA_API_KEY_PATH}:/run/secrets/laya_api_key:ro
init: true
restart: unless-stopped
+ # The server preloads before it binds, so /health answers once the preloaded
+ # checkpoints are ready. `up --wait` therefore waits for a usable server.
+ healthcheck:
+ test: ["CMD", "python", "-c", "import os, urllib.request; urllib.request.urlopen('http://127.0.0.1:' + os.environ['LAYA_PORT'] + '/health', timeout=4)"]
+ interval: 30s
+ timeout: 5s
+ retries: 3
+ start_period: 10m
+ start_interval: 2s
volumes:
model-cache:
diff --git a/docs/docker.md b/docs/docker.md
index 4d084d3a..37489043 100644
--- a/docs/docker.md
+++ b/docs/docker.md
@@ -179,8 +179,8 @@ curl -s localhost:8000/v1/systemone -H 'content-type: application/json' \
```
For NVIDIA, add the CUDA override. It repeats the build args and the device reservation
-for `laya-serve`, because a Compose override replaces a service's `build` block whole
-rather than merging it:
+for `laya-serve`, because `laya-serve` is a separate service and overrides for `laya`
+never reach it:
```bash
docker compose -f compose.yaml -f compose.http.yaml -f compose.cuda.yaml up --build laya-serve
@@ -191,6 +191,15 @@ named `model-cache` volume as the quickstart, so serving after a quickstart run
with the checkpoints already on disk. Stop with `docker compose ... down`, using the same
Compose files.
+The port is published on `127.0.0.1` only. The API has no authentication until
+`LAYA_API_KEY` is set, so set a key before exposing it with
+`LAYA_BIND_ADDRESS=0.0.0.0`, and put a TLS reverse proxy in front for remote clients.
+`/health` does not require authentication in either case.
+
+The service has a healthcheck on `/health`. The server preloads before it starts
+listening, so with `LAYA_PRELOAD=1` a healthy container has its checkpoints loaded.
+`docker compose ... up -d --wait laya-serve` returns once it is healthy.
+
### Server configuration
These apply to the `laya-serve` service only.
@@ -199,6 +208,7 @@ These apply to the `laya-serve` service only.
|---|---|---|
| `LAYA_HOST` | `0.0.0.0` | bind address inside the container |
| `LAYA_PORT` | `8000` | container port, and the host port published for it |
+| `LAYA_BIND_ADDRESS` | `127.0.0.1` | host address the port is published on |
| `LAYA_PRELOAD` | `0` | `1` builds every checkpoint at startup instead of on first request |
| `LAYA_MODELS` | (all) | comma list to preload: `english,multilingual,typed-decisions` |
| `LAYA_THREADS` | `OMP_NUM_THREADS` | caps torch intra-op threads; keep at or below physical cores |
diff --git a/tests/test_packaging.py b/tests/test_packaging.py
index 585124b2..e3995249 100644
--- a/tests/test_packaging.py
+++ b/tests/test_packaging.py
@@ -161,7 +161,12 @@ check_true("compose.http/runs the server command",
check_true("compose.http/publishes a port", re.search(r"^\s*ports:", http, re.M) is not None)
# Host and container port must come from the same variable, or they drift apart and the
# published port stops reaching the server.
-port_map = re.search(r'-\s*"(\$\{[A-Z_]+:-(\d+)\}):(\$\{[A-Z_]+:-(\d+)\})"', http)
+port_map = re.search(r'-\s*"(?:\$\{LAYA_BIND_ADDRESS:-[^}]+\}:)?(\$\{[A-Z_]+:-(\d+)\}):(\$\{[A-Z_]+:-(\d+)\})"', http)
+# The API is unauthenticated until LAYA_API_KEY is set, so exposure beyond the host is opt-in.
+check_true("compose.http/publishes on loopback unless LAYA_BIND_ADDRESS is set",
+ '"${LAYA_BIND_ADDRESS:-127.0.0.1}:' in http)
+check_true("compose.http/has a healthcheck on /health",
+ "healthcheck:" in http and "/health" in http)
check_true("compose.http/port mapping is present", port_map is not None, http[:300])
if port_map:
check("compose.http/host port equals container port", port_map.group(2), port_map.group(4))
@@ -179,7 +184,7 @@ check_true("compose.http/leaves the quickstart service alone",
check_true("Dockerfile/installs the serve extra", '".[serve]"' in dockerfile, dockerfile[:400])
check_true("Dockerfile/still runs pip check", "pip check" in dockerfile)
-# A Compose override replaces a service's `build` block whole. If the CUDA override does
+# Overrides for `laya` never reach `laya-serve`, a separate service. If the CUDA override does
# not repeat the args for laya-serve, that service silently serves on CPU.
check_true("compose.cuda/covers laya-serve too",
re.search(r"^\s{2}laya-serve:", cuda, re.M) is not None,