Skip to content

armgit / Soup

Public Imported from GitHub
0

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

fix(serve): require the tool token on the adapter routes; let soup loop send it (#1139) (#1184)

Dipak Chaudhari46b65d03b443
diff --git a/changelog.d/0.76.0/1184.fixed.md b/changelog.d/0.76.0/1184.fixed.md
new file mode 100644
index 00000000..2669dd22
--- /dev/null
+++ b/changelog.d/0.76.0/1184.fixed.md
@@ -0,0 +1 @@
+- **`soup loop watch` can now present the `soup serve` tool token (#1139 by @dchaudhari7177 in #1184).** Since 0.75.1 `soup serve` requires its `--tool-auth-token` on the adapter routes, so the loop's deploy stage, which activates the adapter over HTTP, was refused on any server started with a token, and reported only a bare `activate POST failed`. Pass the new `--tool-auth-token` or set `SOUP_TOOL_AUTH_TOKEN`; a `--detach` child gets the token through its environment, never its argv, and the audit log records it as `<redacted>`. A refused activation now says whether the endpoint wanted a token that was not sent or rejected the one that was, and names both ways to supply it. Without a token the loop sends no `Authorization` header, as before. The activate POST to a plain-http (loopback) endpoint now ignores `HTTP_PROXY` / `ALL_PROXY`, so a proxy never sees the token in clear text; an https endpoint still reads the environment, including `SSL_CERT_FILE` / `SSL_CERT_DIR`.
diff --git a/docs/adapters-and-governance.md b/docs/adapters-and-governance.md
index 89182c20..34de4b43 100644
--- a/docs/adapters-and-governance.md
+++ b/docs/adapters-and-governance.md
@@ -133,6 +133,16 @@ soup loop replay iter-20260515T120000-abcdef01 --extract ./iter-dump
# Background subprocess (writes PID, no shell)
soup loop watch --detach
+# The deploy stage activates the adapter on SOUP_LOOP_SERVE_ENDPOINT. A server
+# started with --tool-auth-token requires that token on the adapter routes, so
+# give the loop the same one: --tool-auth-token, or SOUP_TOOL_AUTH_TOKEN to keep
+# it out of the shell history. A --detach child gets it through its environment
+# (so does the train stage's `soup train` child, which ignores it). soup serve has
+# no TLS and the loop refuses plain http off-loopback, so run the loop on the
+# serving box against loopback, or put a TLS-terminating proxy in front.
+soup serve --model base-model --adapters chat=./chat --host 0.0.0.0 --tool-auth-token "$TOKEN" &
+SOUP_LOOP_SERVE_ENDPOINT=http://127.0.0.1:8000 SOUP_TOOL_AUTH_TOKEN="$TOKEN" soup loop watch --detach
+
# Start serve with the candidate loaded, then promote it at 5% traffic.
soup serve --model ./base --adapters candidate=./candidate
soup loop canary candidate --traffic 5% --autoroll-on-regress
diff --git a/docs/cli-reference.md b/docs/cli-reference.md
index 563c2edc..b3ee895a 100644
--- a/docs/cli-reference.md
+++ b/docs/cli-reference.md
@@ -2744,6 +2744,7 @@ Usage: `soup loop watch [OPTIONS]`
| `--poll-interval` | float | `60.0` | | Seconds between iterations \[1, 3600]. |
| `--pre-wired` | flag | off | | Force the pre-wired production stages even if loop.yaml didn't set it. |
| `--pack-cans` | flag | off | | Pack each iteration as a v0.26 Soup Can + Registry entry (v0.71.4). |
+| `--tool-auth-token` | text | | | The soup serve --tool-auth-token value, sent as a Bearer token when the deploy stage activates an adapter. Defaults to \$SOUP_TOOL_AUTH_TOKEN, which keeps it out of the shell history. Env: `SOUP_TOOL_AUTH_TOKEN`. |
## `soup mcp`
diff --git a/docs/commands.md b/docs/commands.md
index 1f590dfb..17f17d5d 100644
--- a/docs/commands.md
+++ b/docs/commands.md
@@ -173,7 +173,7 @@ soup adapters checkout <name> Restore a snapshotted branch's c
soup adapters diff <a> <b> Per-layer ΔW Frobenius diff + effective-rank drift
soup loop init <model> --eval <s> --baseline <b> [--pre-wired] [--thumbs-db <db>] Create .soup/loop.yaml (data flywheel; --pre-wired = real stages; --thumbs-db = harvest `serve --record-thumbs` thumbs)
soup loop status Counters + status + pre_wired flag
-soup loop watch [--detach] [--max-iterations N] [--pre-wired] [--pack-cans] Harvest → train → gate → deploy daemon (pre-wired stages + Soup Can packing)
+soup loop watch [--detach] [--max-iterations N] [--pre-wired] [--pack-cans] [--tool-auth-token T] Harvest → train → gate → deploy daemon (pre-wired stages + Soup Can packing)
soup loop pause Atomic status flip: pause watch daemon at next iteration boundary
soup loop resume Atomic status flip: resume a paused loop
soup loop canary <adapter> --traffic 5% Route keyed serve traffic to a loaded adapter; watch auto-rolls back on MAJOR
diff --git a/docs/serving-and-export.md b/docs/serving-and-export.md
index 818db8af..50df0b94 100644
--- a/docs/serving-and-export.md
+++ b/docs/serving-and-export.md
@@ -609,6 +609,17 @@ curl http://localhost:8000/v1/adapters
# → {"adapters": [{"name": "chat", "active": true}, ...], "active": "chat"}
```
+A server started with `--tool-auth-token` requires that token on all three adapter
+routes, loopback included, and answers 401 without it. A non-loopback bind always has
+one, since `soup serve` refuses to start there without it:
+
+```bash
+soup serve --model base-model --adapters chat=./chat-adapter --host 0.0.0.0 --tool-auth-token "$TOKEN"
+
+curl -X POST -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/v1/adapters/activate/chat
+curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/v1/adapters
+```
+
Names are validated against `^[a-zA-Z0-9][a-zA-Z0-9-]*$`; activate/deactivate calls are thread-safe behind a lock. Activate/deactivate also check the `Host` and `Origin` headers (see [Server-Side Tool Endpoints](#server-side-tool-endpoints)).
### Multi-Tenant Vector Bank (`soup serve --bank`)
@@ -991,7 +1002,10 @@ Three POST routes are now available on `soup serve`:
Tool routes, `/v1/thumbs` and adapter activate/deactivate accept requests only when the
`Host` header names the bound address (any loopback name for a loopback bind) and any
-`Origin` header names the same; otherwise they answer 421 or 403.
+`Origin` header names the same; otherwise they answer 421 or 403. That check runs first.
+The tool routes and all three adapter routes (`GET /v1/adapters`, activate, deactivate)
+then also require `Authorization: Bearer <token>` whenever `--tool-auth-token` is set,
+on loopback too, and answer 401 without it.
The generation routes (`/v1/chat/completions`, `/v1/messages`) and the adapter listing
(`GET /v1/adapters`) carry the narrower half of that check: a request whose `Origin` names
diff --git a/src/soup_cli/commands/loop.py b/src/soup_cli/commands/loop.py
index 830fdf06..40bc6688 100644
--- a/src/soup_cli/commands/loop.py
+++ b/src/soup_cli/commands/loop.py
@@ -227,12 +227,29 @@ def watch_cmd(
False, "--pack-cans",
help="Pack each iteration as a v0.26 Soup Can + Registry entry (v0.71.4).",
),
+ tool_auth_token: Optional[str] = typer.Option(
+ None,
+ "--tool-auth-token",
+ envvar="SOUP_TOOL_AUTH_TOKEN",
+ help=(
+ "The soup serve --tool-auth-token value, sent as a Bearer token when the "
+ "deploy stage activates an adapter. Defaults to $SOUP_TOOL_AUTH_TOKEN, "
+ "which keeps it out of the shell history."
+ ),
+ ),
) -> None:
"""Run the harvest → train → gate → deploy daemon."""
if detach and foreground:
console.print("[red]--detach and --foreground are mutually exclusive[/]")
raise typer.Exit(code=2)
state = _safe_read() # ensure state exists before forking
+ if tool_auth_token:
+ # The deploy stage reads the token from the environment, as it reads
+ # SOUP_LOOP_SERVE_ENDPOINT; a --detach child inherits it from there
+ # rather than from argv, where any local user could read it (#1139).
+ from soup_cli.utils.loop_stages import TOOL_AUTH_TOKEN_ENV
+
+ os.environ[TOOL_AUTH_TOKEN_ENV] = tool_auth_token
if pre_wired or state.pre_wired:
# #1440: a configured thumbs database the harvest would refuse stops the
# watch here, before it starts or detaches, instead of mid-iteration.
diff --git a/src/soup_cli/utils/loop_stages.py b/src/soup_cli/utils/loop_stages.py
index 448367a0..485a13aa 100644
--- a/src/soup_cli/utils/loop_stages.py
+++ b/src/soup_cli/utils/loop_stages.py
@@ -400,12 +400,26 @@ def _endpoint_is_local(endpoint: str) -> bool:
return ip.is_loopback or ip.is_private or ip.is_link_local
+#: Environment variable holding the ``soup serve --tool-auth-token`` value.
+#: ``soup loop watch --tool-auth-token`` sets it, including for a ``--detach``
+#: child, so the token never has to appear in argv (#1139).
+TOOL_AUTH_TOKEN_ENV = "SOUP_TOOL_AUTH_TOKEN"
+
+
+class _ActivateRefusedError(Exception):
+ """The serve endpoint answered the activate POST with 401."""
+
+
def _post_activate(endpoint: str, name: str) -> bool:
"""POST to ``<endpoint>/v1/adapters/activate/<name>``; True on 2xx.
``endpoint`` is already SSRF-validated by the caller; ``name`` is
re-validated here against the activate-route pattern before URL
interpolation (defence-in-depth against a crafted adapter dir name).
+ The tool token from ``SOUP_TOOL_AUTH_TOKEN``, when set, goes as a Bearer
+ header: a server started with ``--tool-auth-token`` requires it on this
+ route (#1139). A 401 raises ``_ActivateRefusedError`` so the caller can name
+ the missing credential instead of reporting a bare failure.
"""
if _DEPLOY_POSTER is not None:
return _DEPLOY_POSTER(endpoint, name)
@@ -416,11 +430,20 @@ def _post_activate(endpoint: str, name: str) -> bool:
except ImportError:
return False
url = endpoint.rstrip("/") + f"/v1/adapters/activate/{name}"
+ token = os.environ.get(TOOL_AUTH_TOKEN_ENV)
+ headers = {"Authorization": f"Bearer {token}"} if token else {}
try:
- resp = httpx.post(url, timeout=5.0)
- return 200 <= resp.status_code < 300
+ # A plain-http endpoint (loopback only) ignores HTTP_PROXY / ALL_PROXY, which
+ # would otherwise receive the Bearer token in clear text. An https endpoint keeps
+ # trust_env: a proxy only sees a CONNECT tunnel there, and SSL_CERT_FILE /
+ # SSL_CERT_DIR are how a TLS proxy's private CA gets trusted.
+ plain_http = url.lower().startswith("http://")
+ resp = httpx.post(url, headers=headers, timeout=5.0, trust_env=not plain_http)
except Exception: # noqa: BLE001 — deploy must never crash the loop
return False
+ if resp.status_code == 401:
+ raise _ActivateRefusedError()
+ return 200 <= resp.status_code < 300
def deploy_to_canary(
@@ -477,7 +500,22 @@ def deploy_to_canary(
"notes": "SOUP_LOOP_SERVE_ENDPOINT must be loopback/LAN (SSRF guard)",
}
name = os.path.basename(str(adapter_path).rstrip("/\\")) or "canary"
- ok = _post_activate(endpoint, name)
+ try:
+ ok = _post_activate(endpoint, name)
+ except _ActivateRefusedError:
+ if os.environ.get(TOOL_AUTH_TOKEN_ENV):
+ note = (
+ "activate POST refused (401): the serve endpoint rejected the tool "
+ "token; check soup loop watch --tool-auth-token / SOUP_TOOL_AUTH_TOKEN "
+ "against soup serve --tool-auth-token"
+ )
+ else:
+ note = (
+ "activate POST refused (401): the serve endpoint requires its tool "
+ "token; pass soup loop watch --tool-auth-token or set SOUP_TOOL_AUTH_TOKEN"
+ )
+ _LOG.warning(note)
+ return {"deployed": False, "canary_verdict": None, "notes": note}
# activate_adapter is a full hot-swap (100% of traffic), not a canary
# split; canary_router.route()/BucketStats never ran, so there is no
# verdict to report here even when the POST itself succeeded.
diff --git a/tests/test_issue1139_adapter_route_auth.py b/tests/test_issue1139_adapter_route_auth.py
new file mode 100644
index 00000000..4d1eaae2
--- /dev/null
+++ b/tests/test_issue1139_adapter_route_auth.py
@@ -0,0 +1,296 @@
+"""#1139 — soup loop can send the serve tool token the adapter routes require.
+
+`soup serve` gates `/v1/adapters/activate` and `/deactivate` on its
+`--tool-auth-token` (#1126), so `soup loop watch` gains `--tool-auth-token`
+(falling back to `SOUP_TOOL_AUTH_TOKEN`) for its deploy stage to present. The
+server-side tests pin the contract the loop depends on.
+"""
+
+from __future__ import annotations
+
+from unittest.mock import MagicMock
+
+import pytest
+
+pytest.importorskip("fastapi")
+
+from fastapi.testclient import TestClient # noqa: E402
+
+TOKEN = "s3cret-tool-token"
+LAN_HOST = "192.168.1.5"
+
+
+def _client(host: str, auth_token: str | None) -> TestClient:
+ from soup_cli.commands.serve import _create_app
+
+ app = _create_app(
+ model_obj=MagicMock(),
+ tokenizer=MagicMock(),
+ device="cpu",
+ model_name="test",
+ max_tokens_default=512,
+ adapter_map={"chat": "./x"},
+ auth_token=auth_token,
+ host=host,
+ )
+ # The Host header names the bind, so the Host/Origin guard lets the request through.
+ return TestClient(app, base_url=f"http://{host}")
+
+
+def _bearer(token: str) -> dict:
+ return {"Authorization": f"Bearer {token}"}
+
+
[email protected]("host", [LAN_HOST, "127.0.0.1"], ids=["lan", "loopback"])
+def test_a_configured_token_gates_activate_and_deactivate(host):
+ client = _client(host, TOKEN)
+
+ for path in ("/v1/adapters/activate/chat", "/v1/adapters/deactivate"):
+ assert client.post(path).status_code == 401, f"{path} without the token"
+ assert client.post(path, headers=_bearer("wrong")).status_code == 401, (
+ f"{path} with the wrong token"
+ )
+ assert client.post(path, headers=_bearer(TOKEN)).status_code == 200, (
+ f"{path} with the token"
+ )
+
+
+def test_the_loopback_default_without_a_token_is_unchanged():
+ client = _client("127.0.0.1", None)
+
+ resp = client.post("/v1/adapters/activate/chat")
+ assert resp.status_code == 200
+ assert resp.json()["active"] == "chat"
+ assert client.post("/v1/adapters/deactivate").status_code == 200
+
+
+def test_the_token_gates_the_adapter_listing_but_not_the_model_list():
+ client = _client(LAN_HOST, TOKEN)
+
+ assert client.get("/v1/models").status_code == 200
+ assert client.get("/v1/adapters").status_code == 401
+ assert client.get("/v1/adapters", headers=_bearer(TOKEN)).status_code == 200
+
+
+def test_no_token_on_a_lan_bind_refuses_the_adapter_routes():
+ assert _client(LAN_HOST, None).post("/v1/adapters/activate/chat").status_code == 401
+
+
+def test_the_host_guard_runs_before_the_token_check():
+ client = _client(LAN_HOST, TOKEN)
+
+ # No token: a token check that ran first would answer 401.
+ resp = client.post("/v1/adapters/activate/chat", headers={"Host": "evil.example"})
+ assert resp.status_code == 421
+
+
[email protected]("path", ["/v1/adapters/activate/chat", "/v1/adapters/deactivate"])
+def test_adapter_token_check_uses_compare_digest(monkeypatch, path):
+ import secrets
+
+ calls, real = [], secrets.compare_digest
+ monkeypatch.setattr(secrets, "compare_digest", lambda a, b: calls.append(1) or real(a, b))
+ client = _client(LAN_HOST, TOKEN)
+ assert client.post(path, headers=_bearer(TOKEN)).status_code == 200
+ assert calls
+
+
+# ---------------------------------------------------------------------------
+# client side: the loop's deploy stage
+# ---------------------------------------------------------------------------
+
+
+def deploy_env(tmp_path, monkeypatch):
+ import soup_cli.utils.loop_stages as ls
+ from soup_cli.utils.loop_state import LoopState
+
+ monkeypatch.chdir(tmp_path)
+ monkeypatch.setenv("SOUP_REGISTRY_DB_PATH", str(tmp_path / "registry.db"))
+ monkeypatch.setenv("SOUP_LOOP_SERVE_ENDPOINT", f"https://{LAN_HOST}:8000")
+ monkeypatch.delenv(ls.TOOL_AUTH_TOKEN_ENV, raising=False)
+ monkeypatch.setattr(ls, "_DEPLOY_POSTER", None)
+
+ def deploy():
+ state = LoopState(
+ served_model="m", eval_suite="e", baseline="b", status="running"
+ )
+ return ls.deploy_to_canary(
+ state, {"gate_verdict": "OK", "adapter_path": "adapters/chat"}
+ )
+
+ return deploy
+
+
+def _route_httpx_to(monkeypatch, client: TestClient) -> list:
+ """Send the deploy stage's ``httpx.post`` to a real app through ``client``."""
+ import httpx
+
+ sent = []
+
+ def post(url, headers=None, timeout=None, **_options):
+ sent.append(dict(headers or {}))
+ return client.post(url, headers=headers or {})
+
+ monkeypatch.setattr(httpx, "post", post)
+ return sent
+
+
+def test_the_loop_token_reaches_a_server_started_with_it(deploy_env, monkeypatch):
+ import soup_cli.utils.loop_stages as ls
+
+ sent = _route_httpx_to(monkeypatch, _client(LAN_HOST, TOKEN))
+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, TOKEN)
+
+ out = deploy_env()
+
+ assert out["deployed"] is True
+ assert sent == [_bearer(TOKEN)]
+
+
[email protected]("token", [None, "wrong-0xC0FFEE"], ids=["missing", "wrong"])
+def test_a_refused_activate_names_the_credential(deploy_env, monkeypatch, caplog, token):
+ import soup_cli.utils.loop_stages as ls
+
+ _route_httpx_to(monkeypatch, _client(LAN_HOST, TOKEN))
+ if token:
+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, token)
+
+ out = deploy_env()
+
+ assert out["deployed"] is False
+ assert "401" in out["notes"]
+ assert "--tool-auth-token" in out["notes"]
+ assert "SOUP_TOOL_AUTH_TOKEN" in out["notes"]
+ expected = "rejected the tool token" if token else "requires its tool token"
+ assert expected in out["notes"]
+ if token:
+ assert token not in out["notes"]
+ assert token not in caplog.text
+
+
+def test_no_token_sends_no_authorization_header(deploy_env, monkeypatch):
+ sent = _route_httpx_to(monkeypatch, _client("127.0.0.1", None))
+ monkeypatch.setenv("SOUP_LOOP_SERVE_ENDPOINT", "http://127.0.0.1:8000")
+
+ out = deploy_env()
+
+ assert out["deployed"] is True
+ assert sent == [{}]
+
+
+# ---------------------------------------------------------------------------
+# the CLI option
+# ---------------------------------------------------------------------------
+
+
+def test_watch_detach_hands_the_token_to_the_child_through_the_environment(
+ tmp_path, monkeypatch
+):
+ import os
+
+ from typer.testing import CliRunner
+
+ import soup_cli.commands.loop as loop_cmd
+ from soup_cli.cli import app
+ from soup_cli.utils.loop_stages import TOOL_AUTH_TOKEN_ENV
+
+ monkeypatch.chdir(tmp_path)
+ # recorded so monkeypatch restores the real value after the command sets it
+ monkeypatch.setenv(TOOL_AUTH_TOKEN_ENV, "placeholder")
+ spawned = {}
+
+ class _Proc:
+ pid = 4242
+
+ def popen(argv, **kwargs):
+ spawned["argv"] = list(argv)
+ spawned["env"] = dict(os.environ) if kwargs.get("env") is None else kwargs["env"]
+ return _Proc()
+
+ monkeypatch.setattr(loop_cmd.subprocess, "Popen", popen)
+ runner = CliRunner()
+ runner.invoke(app, ["loop", "init", "m", "--eval", "e", "--baseline", "b"])
+
+ result = runner.invoke(app, ["loop", "watch", "--detach", "--tool-auth-token", TOKEN])
+
+ assert result.exit_code == 0, result.output
+ assert TOKEN not in spawned["argv"], "the token must not appear in the child's argv"
+ assert spawned["env"][TOOL_AUTH_TOKEN_ENV] == TOKEN
+
+
+def test_the_token_is_masked_in_the_audit_log():
+ from soup_cli.cli import _mask_audit_args
+
+ masked = _mask_audit_args("loop", ["watch", "--tool-auth-token", TOKEN])
+
+ assert TOKEN not in " ".join(masked)
+ assert "--tool-auth-token" in masked
+
+
+def test_the_activate_post_ignores_environment_proxies(monkeypatch):
+ import socket
+ import threading
+
+ import soup_cli.utils.loop_stages as ls
+
+ hits = []
+ proxy = socket.socket()
+ proxy.bind(("127.0.0.1", 0))
+ proxy.listen(1)
+ proxy.settimeout(3)
+
+ def accept_one():
+ try:
+ conn, _ = proxy.accept()
+ except OSError:
+ return
+ hits.append(conn.recv(65536))
+ conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\nConnection: close\r\n\r\n")
+ conn.close()
+
+ thread = threading.Thread(target=accept_one, daemon=True)
+ thread.start()
+
+ dead = socket.socket()
+ dead.bind(("127.0.0.1", 0))
+ dead_port = dead.getsockname()[1]
+ dead.close() # nothing listens here, so only a proxy could answer
+
+ monkeypatch.setattr(ls, "_DEPLOY_POSTER", None)
+ monkeypatch.setenv("HTTP_PROXY", f"http://127.0.0.1:{proxy.getsockname()[1]}")
+ monkeypatch.delenv("NO_PROXY", raising=False)
+ monkeypatch.delenv("no_proxy", raising=False)
+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, "proxy-leak-token")
+
+ assert ls._post_activate(f"http://127.0.0.1:{dead_port}", "chat") is False
+
+ proxy.close()
+ thread.join(5)
+ assert not hits, "the tool token must not be sent to an environment proxy"
+
+
+def test_an_https_endpoint_keeps_the_environment_for_its_ca(monkeypatch):
+ # A LAN endpoint is https behind a TLS-terminating proxy, whose certificate for a
+ # private IP comes from a private CA. httpx trusts that CA only through SSL_CERT_FILE /
+ # SSL_CERT_DIR, which trust_env=False switches off; on https a proxy sees only CONNECT.
+ import httpx
+
+ import soup_cli.utils.loop_stages as ls
+
+ calls = []
+
+ def fake_post(url, **kwargs):
+ calls.append((url, kwargs.get("trust_env", True)))
+ return httpx.Response(200)
+
+ monkeypatch.setattr(ls, "_DEPLOY_POSTER", None)
+ monkeypatch.setattr(httpx, "post", fake_post)
+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, TOKEN)
+
+ assert ls._post_activate("https://192.168.1.5:8443", "chat") is True
+ assert ls._post_activate("http://127.0.0.1:8000", "chat") is True
+ assert calls == [
+ ("https://192.168.1.5:8443/v1/adapters/activate/chat", True),
+ ("http://127.0.0.1:8000/v1/adapters/activate/chat", False),
+ ]