Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

Require the decision token on the model server too

agent-runner353287680246c3c
diff --git a/decision/README.md b/decision/README.md
index 769d5f973..d73c236e8 100644
--- a/decision/README.md
+++ b/decision/README.md
@@ -68,6 +68,11 @@ switch-core talks only to this contract; the backend behind it is configuration.
- **Auth:** with `DECISION_SERVICE_TOKEN` set, every request except
`GET /health` needs `Authorization: Bearer <token>`. Unset means no auth,
for use inside a cluster behind a NetworkPolicy.
+- **The model server's token:** with `MODEL_SERVER_TOKEN` set, the service
+ sends it as `Authorization: Bearer <token>` to the llama.cpp and SystemOne
+ servers, so they can require it: llama.cpp with `--api-key` (or
+ `LLAMA_API_KEY`), the Strands server with `STRANDS_API_KEY`. Neither guards
+ `/health`. Never logged.
- **Per-request backend override**, so a workspace can choose Haiku without a
restart. The headers are never logged.
- `X-Decision-Backend`: `default` (or absent) for the configured backend;
@@ -106,6 +111,7 @@ switch-core talks only to this contract; the backend behind it is configuration.
| `DECISION_HISTORY_LIMIT` | `8` | recent and thread messages kept, each (8 recent + 8 thread). Measured on an L4: a room at switch-core's full limits took 0.55 s with 30 and 0.23 s with 8, with the same accuracy |
| `DECISION_WITH_REASON` | `1` | `0` drops the reason from Haiku's tool call |
| `DECISION_SERVICE_TOKEN` | unset | bearer token for everything but `/health` |
+| `MODEL_SERVER_TOKEN` | unset | bearer token sent to the llama.cpp or SystemOne server |
| `DECISION_BEDROCK_ENABLED` | unset | `1` lets a request choose `bedrock` |
| `DECISION_BEDROCK_MODEL` | `us.anthropic.claude-haiku-5-5-v1:0` | Bedrock model or inference profile |
| `AWS_REGION`, `AWS_PROFILE` | `us-east-1`, unset | standard AWS credential chain |
@@ -138,7 +144,9 @@ downloads the checkpoint and its base model (about 17 GB) into `/models`.
weights once at load, which on an L4 gives the same answers in 0.26 s instead
of 0.42 s for a typical room and about 3.9 decisions a second instead of 2.4.
It's configured by env only (`STRANDS_CHECKPOINT`, `STRANDS_REVISION`,
-`STRANDS_BASE_REVISION`, `STRANDS_MODEL_NAME`, `STRANDS_MAX_BATCH`). The image
+`STRANDS_BASE_REVISION`, `STRANDS_MODEL_NAME`, `STRANDS_MAX_BATCH`, and
+`STRANDS_API_KEY`, which when set is required as a bearer token on everything
+but `/health`). The image
pins the checkpoint and its base model to Hugging Face commits
(`STRANDS_REVISION`, `STRANDS_BASE_REVISION`); the base revision must be the one
the checkpoint records in its `provenance.json`, or the server refuses to
diff --git a/decision/model-servers/strands/Dockerfile b/decision/model-servers/strands/Dockerfile
index d0c7d7ee5..2db09567f 100644
--- a/decision/model-servers/strands/Dockerfile
+++ b/decision/model-servers/strands/Dockerfile
@@ -60,5 +60,5 @@ WORKDIR /home/decision
EXPOSE 8400
HEALTHCHECK --interval=10s --timeout=5s --start-period=30m --retries=3 \
CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8400/health', timeout=3)"]
-COPY --chown=10001:10001 serve.py /opt/strands-serve/serve.py
+COPY --chown=10001:10001 serve.py bearer.py /opt/strands-serve/
ENTRYPOINT ["python", "/opt/strands-serve/serve.py"]
diff --git a/decision/model-servers/strands/bearer.py b/decision/model-servers/strands/bearer.py
new file mode 100644
index 000000000..9d93401ff
--- /dev/null
+++ b/decision/model-servers/strands/bearer.py
@@ -0,0 +1,38 @@
+"""The Strands server's bearer token: with STRANDS_API_KEY set, every request
+but GET /health must carry `Authorization: Bearer <key>`, so another pod
+reaching the server's port can't use the model. The decision service sends
+it (MODEL_SERVER_TOKEN).
+
+Apart from serve.py, which loads the model when imported, so it can be tested
+without torch or a GPU."""
+
+import hmac
+
+from fastapi import FastAPI, Request
+from fastapi.responses import JSONResponse
+
+
+def authorized(header: str | None, key: str) -> bool:
+ """Whether an Authorization header carries `key` as a bearer token."""
+ scheme, _, given = (header or "").partition(" ")
+ return scheme.lower() == "bearer" and hmac.compare_digest(
+ given.strip().encode(), key.encode()
+ )
+
+
+def require_bearer(app: FastAPI, key: str | None) -> None:
+ """Refuse every request to `app` but GET /health without `key`; with no
+ key, leave the app open."""
+ if not key:
+ return
+
+ @app.middleware("http")
+ async def check(request: Request, call_next):
+ open_path = request.method == "GET" and request.url.path == "/health"
+ if not open_path and not authorized(request.headers.get("authorization"), key):
+ return JSONResponse(
+ status_code=401,
+ content={"detail": "missing or wrong bearer token"},
+ headers={"WWW-Authenticate": "Bearer"},
+ )
+ return await call_next(request)
diff --git a/decision/model-servers/strands/serve.py b/decision/model-servers/strands/serve.py
index 02f9ef6d7..36576fb48 100644
--- a/decision/model-servers/strands/serve.py
+++ b/decision/model-servers/strands/serve.py
@@ -5,8 +5,9 @@ adapter merged into the base weights.
Reads STRANDS_CHECKPOINT, STRANDS_REVISION, STRANDS_BASE_REVISION,
STRANDS_MODEL_NAME, STRANDS_PORT (default 8400), STRANDS_HOST (default
-0.0.0.0), STRANDS_DEVICE (default cuda) and STRANDS_MAX_BATCH (default 32,
-requests evaluated together).
+0.0.0.0), STRANDS_DEVICE (default cuda), STRANDS_MAX_BATCH (default 32,
+requests evaluated together) and STRANDS_API_KEY (optional: when set, every
+request but GET /health needs it as a bearer token; see bearer.py).
STRANDS_REVISION is the Hugging Face commit of the checkpoint to download, and
STRANDS_BASE_REVISION the commit of its base model, so a new push to either
@@ -28,6 +29,7 @@ import time
import torch
import uvicorn
+from bearer import require_bearer
from fastapi import FastAPI, HTTPException
from fastapi.responses import JSONResponse
from huggingface_hub import snapshot_download
@@ -49,6 +51,7 @@ PORT = int(os.environ.get("STRANDS_PORT", "8400"))
HOST = os.environ.get("STRANDS_HOST", "0.0.0.0")
DEVICE = os.environ.get("STRANDS_DEVICE", "cuda")
MAX_BATCH = int(os.environ.get("STRANDS_MAX_BATCH", "32"))
+API_KEY = os.environ.get("STRANDS_API_KEY") or None
def pinned_checkpoint() -> str:
@@ -100,6 +103,7 @@ def load_engine() -> SystemOneEngine:
engine = load_engine()
app = FastAPI(title="Strands Decider (adapter merged)")
+require_bearer(app, API_KEY)
@app.get("/health")
diff --git a/decision/switch_decision/backends.py b/decision/switch_decision/backends.py
index 1b7a0b053..207a986ba 100644
--- a/decision/switch_decision/backends.py
+++ b/decision/switch_decision/backends.py
@@ -345,6 +345,14 @@ def model_server(settings: Settings) -> str:
)
+def model_server_headers(settings: Settings) -> dict[str, str]:
+ """The llama.cpp or SystemOne server's bearer token, when one is set. The
+ headers are never logged."""
+ if settings.model_server_token is None:
+ return {}
+ return {"Authorization": f"Bearer {settings.model_server_token}"}
+
+
async def ask(
client: httpx.AsyncClient,
settings: Settings,
@@ -367,7 +375,12 @@ async def ask(
"chat_template_kwargs": {"enable_thinking": False},
}
try:
- r = await client.post(url, json=payload, timeout=settings.timeout_s)
+ r = await client.post(
+ url,
+ json=payload,
+ headers=model_server_headers(settings),
+ timeout=settings.timeout_s,
+ )
except httpx.HTTPError as exc:
raise ModelServerError(
f"model server {url} unreachable: {type(exc).__name__}: {exc}"
@@ -379,9 +392,11 @@ async def ask(
)
return read_letter_logprobs(r.json(), letters)
options = [a.name for a in req.agents] + [NONE]
+ headers: dict[str, str] = {}
if settings.backend == "systemone":
url = settings.systemone_url
payload = systemone_request(req, settings.model, settings.history_limit)
+ headers = model_server_headers(settings)
else:
url = f"{settings.ollama_url}/api/chat"
payload = {
@@ -393,7 +408,9 @@ async def ask(
"options": {"temperature": 0},
}
try:
- r = await client.post(url, json=payload, timeout=settings.timeout_s)
+ r = await client.post(
+ url, json=payload, headers=headers, timeout=settings.timeout_s
+ )
except httpx.HTTPError as exc:
raise ModelServerError(
f"model server {url} unreachable: {type(exc).__name__}: {exc}"
diff --git a/decision/switch_decision/config.py b/decision/switch_decision/config.py
index 99271e9f7..38a251753 100644
--- a/decision/switch_decision/config.py
+++ b/decision/switch_decision/config.py
@@ -35,6 +35,9 @@ class Settings:
with_reason: bool = True
# Bearer token every request but GET /health must carry; None means no auth.
service_token: str | None = None
+ # Bearer token sent to the llama.cpp and SystemOne (Strands) model servers,
+ # which then refuse everything but /health without it; None sends none.
+ model_server_token: str | None = None
# Whether a request may choose Bedrock with X-Decision-Backend (it spends
# the service's own AWS credentials).
bedrock_enabled: bool = False
@@ -69,6 +72,7 @@ def load_settings() -> Settings:
history_limit=int(os.environ.get("DECISION_HISTORY_LIMIT", "8")),
with_reason=os.environ.get("DECISION_WITH_REASON", "1") != "0",
service_token=os.environ.get("DECISION_SERVICE_TOKEN") or None,
+ model_server_token=os.environ.get("MODEL_SERVER_TOKEN") or None,
bedrock_enabled=_flag("DECISION_BEDROCK_ENABLED"),
bedrock_model=bedrock_model,
)
diff --git a/decision/tests/test_backends.py b/decision/tests/test_backends.py
index 739ef22ff..70d247b7f 100644
--- a/decision/tests/test_backends.py
+++ b/decision/tests/test_backends.py
@@ -1,5 +1,7 @@
+import asyncio
import json
import math
+from dataclasses import replace
from types import SimpleNamespace
import httpx
@@ -454,3 +456,71 @@ def test_read_anthropic_tool_call():
assert (a.choice, a.confidence) == ("deploy-bot", 0.8)
with pytest.raises(ModelServerError, match="did not call"):
read_anthropic(SimpleNamespace(stop_reason="refusal", content=[]), OPTIONS)
+
+
+# ── the model server's bearer token ─────────────────────────────────────────
+
+
+def _ask_recording(settings: Settings, body: dict) -> httpx.Request:
+ """Run one `ask` against a fake model server; return the request it got."""
+ seen: list[httpx.Request] = []
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ seen.append(request)
+ return httpx.Response(200, json=body)
+
+ async def run() -> None:
+ async with httpx.AsyncClient(transport=httpx.MockTransport(handler)) as c:
+ await backends.ask(c, settings, _req())
+
+ asyncio.run(run())
+ assert len(seen) == 1
+ return seen[0]
+
+
+LLAMA_ANSWER = _top([("A", math.log(0.9)), ("N", math.log(0.1))])
+SYSTEMONE_ANSWER = _answer("deploy-bot", {"deploy-bot": 0.9, NONE: 0.1})
+
+
+ ("backend", "body"),
+ [("llamacpp", LLAMA_ANSWER), ("systemone", SYSTEMONE_ANSWER)],
+)
+def test_the_model_server_token_is_sent_as_a_bearer_token(backend, body):
+ s = replace(_settings(), backend=backend, model_server_token="s3cret")
+ request = _ask_recording(s, body)
+ assert request.headers["authorization"] == "Bearer s3cret"
+
+
+ ("backend", "body"),
+ [("llamacpp", LLAMA_ANSWER), ("systemone", SYSTEMONE_ANSWER)],
+)
+def test_no_model_server_token_sends_no_authorization(backend, body):
+ request = _ask_recording(replace(_settings(), backend=backend), body)
+ assert "authorization" not in request.headers
+
+
+def test_ollama_never_gets_the_model_server_token():
+ s = replace(_settings(), backend="ollama", model_server_token="s3cret")
+ body = {
+ "message": {"content": json.dumps({"agent": "deploy-bot", "confidence": 0.9})}
+ }
+ request = _ask_recording(s, body)
+ assert "authorization" not in request.headers
+
+
+def test_a_refused_model_server_token_is_a_502_without_the_token(monkeypatch, caplog):
+ tc = _client(
+ monkeypatch,
+ lambda r: httpx.Response(401, json={"detail": "missing or wrong bearer token"}),
+ )
+ monkeypatch.setattr(
+ app_module,
+ "settings",
+ replace(app_module.settings, model_server_token="s3cret"),
+ )
+ with caplog.at_level("INFO"):
+ r = tc.post("/decide", json=_payload())
+ assert r.status_code == 502 and "401" in r.json()["detail"]
+ assert "s3cret" not in caplog.text and "s3cret" not in r.text
diff --git a/decision/tests/test_config.py b/decision/tests/test_config.py
index fb6f6c170..036b9894c 100644
--- a/decision/tests/test_config.py
+++ b/decision/tests/test_config.py
@@ -11,6 +11,7 @@ def test_defaults(monkeypatch):
"LLAMACPP_URL",
"DECISION_SERVICE_TOKEN",
"DECISION_BEDROCK_ENABLED",
+ "MODEL_SERVER_TOKEN",
):
monkeypatch.delenv(k, raising=False)
s = load_settings()
@@ -21,6 +22,7 @@ def test_defaults(monkeypatch):
)
assert s.llamacpp_url == "http://127.0.0.1:8300"
assert (s.service_token, s.bedrock_enabled) == (None, False)
+ assert s.model_server_token is None
def test_ollama_backend_default_model(monkeypatch):
@@ -72,3 +74,10 @@ def test_bedrock_enabled_flag(monkeypatch, value, enabled):
def test_history_limit_defaults_to_eight(monkeypatch):
monkeypatch.delenv("DECISION_HISTORY_LIMIT", raising=False)
assert load_settings().history_limit == 8
+
+
+def test_model_server_token(monkeypatch):
+ monkeypatch.setenv("MODEL_SERVER_TOKEN", "s3cret")
+ assert load_settings().model_server_token == "s3cret"
+ monkeypatch.setenv("MODEL_SERVER_TOKEN", "")
+ assert load_settings().model_server_token is None
diff --git a/decision/tests/test_strands_bearer.py b/decision/tests/test_strands_bearer.py
new file mode 100644
index 000000000..af8c4ff1c
--- /dev/null
+++ b/decision/tests/test_strands_bearer.py
@@ -0,0 +1,67 @@
+"""The Strands server's bearer token (model-servers/strands/bearer.py), tested
+without serve.py, which needs torch and loads the model when imported."""
+
+import importlib.util
+from pathlib import Path
+
+import pytest
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+
+_PATH = Path(__file__).parents[1] / "model-servers" / "strands" / "bearer.py"
+_spec = importlib.util.spec_from_file_location("strands_bearer", _PATH)
+assert _spec is not None and _spec.loader is not None
+bearer = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(bearer)
+
+
+def _app(key: str | None) -> TestClient:
+ app = FastAPI()
+ bearer.require_bearer(app, key)
+
+ @app.get("/health")
+ def health() -> dict:
+ return {"status": "ok"}
+
+ @app.post("/v1/systemone")
+ def systemone() -> dict:
+ return {"answers": {}}
+
+ return TestClient(app)
+
+
+ ("header", "ok"),
+ [
+ ("Bearer s3cret", True),
+ ("bearer s3cret ", True),
+ ("Bearer wrong", False),
+ ("Bearer s3cret2", False),
+ ("Basic s3cret", False),
+ ("s3cret", False),
+ ("", False),
+ (None, False),
+ ],
+)
+def test_authorized(header, ok):
+ assert bearer.authorized(header, "s3cret") is ok
+
+
+def test_with_a_key_only_health_is_open():
+ tc = _app("s3cret")
+ assert tc.get("/health").status_code == 200
+ assert tc.post("/v1/systemone", json={}).status_code == 401
+ wrong = {"Authorization": "Bearer wrong"}
+ assert tc.post("/v1/systemone", json={}, headers=wrong).status_code == 401
+ right = {"Authorization": "Bearer s3cret"}
+ assert tc.post("/v1/systemone", json={}, headers=right).status_code == 200
+ # Only GET /health: any other method or path needs the key.
+ assert tc.post("/health").status_code == 401
+ assert tc.get("/docs").status_code == 401
+
+
[email protected]("key", [None, ""])
+def test_without_a_key_everything_is_open(key):
+ tc = _app(key)
+ assert tc.post("/v1/systemone", json={}).status_code == 200
+ assert tc.get("/health").status_code == 200
diff --git a/deploy/remote/helm/switch/README.md b/deploy/remote/helm/switch/README.md
index 05b49777a..c00ea38c2 100644
--- a/deploy/remote/helm/switch/README.md
+++ b/deploy/remote/helm/switch/README.md
@@ -294,8 +294,12 @@ no further setup.
- **The service token.** The decision service requires a bearer token on
everything but `/health` (`DECISION_SERVICE_TOKEN`), and switch-core sends it
- (`DECISION_API_KEY`), so another pod in the cluster gets 401 even where the
- NetworkPolicy isn't enforced. The chart generates it into the Secret
+ (`DECISION_API_KEY`). The model server beside it, which listens on the pod's
+ IP for its probes, requires the same token (`LLAMA_API_KEY` for llama.cpp,
+ `STRANDS_API_KEY` for Strands), and the decision container sends it
+ (`MODEL_SERVER_TOKEN`). So another pod in the cluster gets 401 from either
+ port even where the NetworkPolicy isn't enforced; only `/health` answers
+ without it. The chart generates it into the Secret
`<release>-decision-token` on the first install, and reads it back with
`lookup` on every upgrade, so it stays the same. `helm template` can't read
the cluster, so it renders a new value each time; that is expected, and only
@@ -366,8 +370,10 @@ no further setup.
- **Other settings.** `decision.env` passes anything else in the service's
environment (`DECISION_TIMEOUT_S`, `DECISION_HISTORY_LIMIT`, …).
- **The NetworkPolicy** needs a CNI that enforces it; where none does, the
- service token still keeps other pods out. It stays on by default as a
- second layer; turn it off with `decision.networkPolicy.enabled: false`.
+ token still keeps other pods from asking for decisions or using the model
+ (they can read `/health`). It stays on by
+ default as a second layer; turn it off with
+ `decision.networkPolicy.enabled: false`.
To use a decision service run elsewhere instead, leave `decision.enabled` off
and set `switchCore.decision.endpointUrl` (and `bedrockEnabled` if that service
diff --git a/deploy/remote/helm/switch/templates/decision/deployment.yaml b/deploy/remote/helm/switch/templates/decision/deployment.yaml
index 5236003b1..b7223d351 100644
--- a/deploy/remote/helm/switch/templates/decision/deployment.yaml
+++ b/deploy/remote/helm/switch/templates/decision/deployment.yaml
@@ -131,6 +131,11 @@ spec:
- name: DECISION_MODEL
value: {{ . | quote }}
{{- end }}
+ {{- if $kind }}
+ # Sent to the model server beside it, which refuses everything but
+ # /health without it.
+ {{- include "switch.decisionTokenEnv" (dict "root" $ "name" "MODEL_SERVER_TOKEN") | nindent 12 }}
+ {{- end }}
{{- if eq $kind "strands" }}
- name: SYSTEMONE_URL
value: "http://127.0.0.1:{{ $ms.strands.port }}/v1/systemone"
@@ -197,6 +202,8 @@ spec:
value: {{ required "decision.modelServer.strands.modelName is required" $st.modelName | quote }}
- name: STRANDS_PORT
value: {{ $st.port | quote }}
+ # Every request but /health needs it; the decision container sends it.
+ {{- include "switch.decisionTokenEnv" (dict "root" $ "name" "STRANDS_API_KEY") | nindent 12 }}
{{- with include "switch.decisionHfToken" . }}
{{- . | trim | nindent 12 }}
{{- end }}
@@ -247,6 +254,9 @@ spec:
env:
- name: MODEL_PATH
value: "/models/{{ include "switch.decisionModelFile" . }}"
+ # llama-server's --api-key: every request but /health needs it; the
+ # decision container sends it.
+ {{- include "switch.decisionTokenEnv" (dict "root" $ "name" "LLAMA_API_KEY") | nindent 12 }}
ports:
- name: model
containerPort: {{ $llama.port }}
diff --git a/scripts/check_chart_decision.py b/scripts/check_chart_decision.py
index 208d150cb..7cbf4993f 100755
--- a/scripts/check_chart_decision.py
+++ b/scripts/check_chart_decision.py
@@ -5,7 +5,8 @@ The decision service (auto-route) is optional in the chart. Off, nothing of it
renders and switch-core gets no endpoint. On, the chart runs it beside its
model server (Strands Decider by default, or Qwen on llama.cpp), points
switch-core at it and admits only switch-core to it, both by NetworkPolicy and
-by a bearer token the chart generates. Its model weights are pinned to fixed
+by a bearer token the chart generates, which its model server requires
+too. Its model weights are pinned to fixed
revisions, the same in docker-compose. An endpoint run elsewhere
can be named instead, but not as well as the chart's own. These are the
promises deploy/remote/helm/switch/README.md makes about `decision.*`; a render
@@ -199,6 +200,8 @@ def test_strands_decider_is_the_default_model_server() -> None:
"STRANDS_BASE_REVISION": env["STRANDS_BASE_REVISION"],
"STRANDS_MODEL_NAME": "strands-decider-E4B",
"STRANDS_PORT": "8400",
+ # From the decision token Secret: test_the_model_server_requires_the_token_too.
+ "STRANDS_API_KEY": None,
}, env
assert [p["containerPort"] for p in server["ports"]] == [8400]
resources = server["resources"]
@@ -230,7 +233,10 @@ def test_qwen_on_llama_cpp_is_the_alternative() -> None:
server = by_name["model-server"]
assert server["image"] == "ghcr.io/ggml-org/llama.cpp:server-cuda"
- assert env_of(server) == {"MODEL_PATH": "/models/Qwen3.5-4B-Q4_K_M.gguf"}
+ assert env_of(server) == {
+ "MODEL_PATH": "/models/Qwen3.5-4B-Q4_K_M.gguf",
+ "LLAMA_API_KEY": None,
+ }
assert server["resources"]["limits"]["nvidia.com/gpu"] == 1
assert server["volumeMounts"] == [
{"name": "weights", "mountPath": "/models", "readOnly": True}
@@ -371,9 +377,62 @@ def test_the_service_requires_a_token_that_switch_core_sends() -> None:
decision = containers(objects["Deployment"])["decision"]
assert secret_refs_of(decision)["DECISION_SERVICE_TOKEN"] == ref
assert secret_refs_of(core_container(docs))["DECISION_API_KEY"] == ref
- # The model server is reached only on localhost and takes no token.
+ # The model server listens on the pod's IP (its probes reach it there),
+ # so it takes the same token, and the decision container sends it.
model_server = containers(objects["Deployment"])["model-server"]
assert "DECISION_SERVICE_TOKEN" not in secret_refs_of(model_server)
+ assert secret_refs_of(decision)["MODEL_SERVER_TOKEN"] == ref
+
+
+def test_the_model_server_requires_the_token_too() -> None:
+ ref = {"name": f"{RELEASE}-decision-token", "key": "token"}
+ for kind, variable in (
+ ("strands", "STRANDS_API_KEY"),
+ ("llamacpp", "LLAMA_API_KEY"),
+ ):
+ docs = render(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ f"decision.modelServer.kind={kind}",
+ *FLAGS,
+ )
+ pod = containers(decision_objects(docs)["Deployment"])
+ assert secret_refs_of(pod["model-server"])[variable] == ref, kind
+ assert secret_refs_of(pod["decision"])["MODEL_SERVER_TOKEN"] == ref, kind
+ # The probes stay on /health, which neither server guards.
+ for probe in ("startupProbe", "livenessProbe"):
+ assert pod["model-server"][probe]["httpGet"]["path"] == "/health", kind
+
+ # Your own Secret: both sides read it.
+ docs = render(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ "decision.modelServer.kind=llamacpp",
+ "--set",
+ "decision.token.existingSecret=my-decision-token",
+ "--set",
+ "decision.token.existingSecretKey=bearer",
+ *FLAGS,
+ )
+ pod = containers(decision_objects(docs)["Deployment"])
+ mine = {"name": "my-decision-token", "key": "bearer"}
+ assert secret_refs_of(pod["model-server"])["LLAMA_API_KEY"] == mine
+ assert secret_refs_of(pod["decision"])["MODEL_SERVER_TOKEN"] == mine
+
+ # No model server, nothing to send it to.
+ docs = render(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ "decision.modelServer.enabled=false",
+ "--set",
+ "decision.backend=anthropic",
+ *FLAGS,
+ )
+ decision = containers(decision_objects(docs)["Deployment"])["decision"]
+ assert "MODEL_SERVER_TOKEN" not in secret_refs_of(decision)
def test_an_existing_token_secret_replaces_the_generated_one() -> None:
@@ -527,6 +586,7 @@ TESTS: list[Callable[[], None]] = [
test_an_endpoint_elsewhere,
test_the_service_requires_a_token_that_switch_core_sends,
test_an_existing_token_secret_replaces_the_generated_one,
+ test_the_model_server_requires_the_token_too,
test_the_qwen_weights_are_pinned_and_a_bad_file_is_deleted,
test_the_strands_checkpoint_and_base_are_pinned,
test_compose_pins_the_same_weights_as_the_chart,