Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
Python cliconsumer-gpudpofine-tuningggufhuggingfacellmllmopslocal-ailocal-llmloralow-vrammachine-learningollamapeftpythonpytorchqlorasfttransformers
fix(serve): require the tool token on the adapter routes; let soup loop send it (#1139) (#1184)
Dipak Chaudhari46b65d03b443
diff --git a/changelog.d/0.76.0/1184.fixed.md b/changelog.d/0.76.0/1184.fixed.mdnew file mode 100644index 00000000..2669dd22--- /dev/null+++ b/changelog.d/0.76.0/1184.fixed.md@@ -0,0 +1 @@+- **`soup loop watch` can now present the `soup serve` tool token (#1139 by @dchaudhari7177 in #1184).** Since 0.75.1 `soup serve` requires its `--tool-auth-token` on the adapter routes, so the loop's deploy stage, which activates the adapter over HTTP, was refused on any server started with a token, and reported only a bare `activate POST failed`. Pass the new `--tool-auth-token` or set `SOUP_TOOL_AUTH_TOKEN`; a `--detach` child gets the token through its environment, never its argv, and the audit log records it as `<redacted>`. A refused activation now says whether the endpoint wanted a token that was not sent or rejected the one that was, and names both ways to supply it. Without a token the loop sends no `Authorization` header, as before. The activate POST to a plain-http (loopback) endpoint now ignores `HTTP_PROXY` / `ALL_PROXY`, so a proxy never sees the token in clear text; an https endpoint still reads the environment, including `SSL_CERT_FILE` / `SSL_CERT_DIR`.diff --git a/docs/adapters-and-governance.md b/docs/adapters-and-governance.mdindex 89182c20..34de4b43 100644--- a/docs/adapters-and-governance.md+++ b/docs/adapters-and-governance.md@@ -133,6 +133,16 @@ soup loop replay iter-20260515T120000-abcdef01 --extract ./iter-dump# Background subprocess (writes PID, no shell)soup loop watch --detach+# The deploy stage activates the adapter on SOUP_LOOP_SERVE_ENDPOINT. A server+# started with --tool-auth-token requires that token on the adapter routes, so+# give the loop the same one: --tool-auth-token, or SOUP_TOOL_AUTH_TOKEN to keep+# it out of the shell history. A --detach child gets it through its environment+# (so does the train stage's `soup train` child, which ignores it). soup serve has+# no TLS and the loop refuses plain http off-loopback, so run the loop on the+# serving box against loopback, or put a TLS-terminating proxy in front.+soup serve --model base-model --adapters chat=./chat --host 0.0.0.0 --tool-auth-token "$TOKEN" &+SOUP_LOOP_SERVE_ENDPOINT=http://127.0.0.1:8000 SOUP_TOOL_AUTH_TOKEN="$TOKEN" soup loop watch --detach+# Start serve with the candidate loaded, then promote it at 5% traffic.soup serve --model ./base --adapters candidate=./candidatesoup loop canary candidate --traffic 5% --autoroll-on-regressdiff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex 563c2edc..b3ee895a 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -2744,6 +2744,7 @@ Usage: `soup loop watch [OPTIONS]`| `--poll-interval` | float | `60.0` | | Seconds between iterations \[1, 3600]. || `--pre-wired` | flag | off | | Force the pre-wired production stages even if loop.yaml didn't set it. || `--pack-cans` | flag | off | | Pack each iteration as a v0.26 Soup Can + Registry entry (v0.71.4). |+| `--tool-auth-token` | text | | | The soup serve --tool-auth-token value, sent as a Bearer token when the deploy stage activates an adapter. Defaults to \$SOUP_TOOL_AUTH_TOKEN, which keeps it out of the shell history. Env: `SOUP_TOOL_AUTH_TOKEN`. |## `soup mcp`diff --git a/docs/commands.md b/docs/commands.mdindex 1f590dfb..17f17d5d 100644--- a/docs/commands.md+++ b/docs/commands.md@@ -173,7 +173,7 @@ soup adapters checkout <name> Restore a snapshotted branch's csoup adapters diff <a> <b> Per-layer ΔW Frobenius diff + effective-rank driftsoup loop init <model> --eval <s> --baseline <b> [--pre-wired] [--thumbs-db <db>] Create .soup/loop.yaml (data flywheel; --pre-wired = real stages; --thumbs-db = harvest `serve --record-thumbs` thumbs)soup loop status Counters + status + pre_wired flag-soup loop watch [--detach] [--max-iterations N] [--pre-wired] [--pack-cans] Harvest → train → gate → deploy daemon (pre-wired stages + Soup Can packing)+soup loop watch [--detach] [--max-iterations N] [--pre-wired] [--pack-cans] [--tool-auth-token T] Harvest → train → gate → deploy daemon (pre-wired stages + Soup Can packing)soup loop pause Atomic status flip: pause watch daemon at next iteration boundarysoup loop resume Atomic status flip: resume a paused loopsoup loop canary <adapter> --traffic 5% Route keyed serve traffic to a loaded adapter; watch auto-rolls back on MAJORdiff --git a/docs/serving-and-export.md b/docs/serving-and-export.mdindex 818db8af..50df0b94 100644--- a/docs/serving-and-export.md+++ b/docs/serving-and-export.md@@ -609,6 +609,17 @@ curl http://localhost:8000/v1/adapters# → {"adapters": [{"name": "chat", "active": true}, ...], "active": "chat"}```+A server started with `--tool-auth-token` requires that token on all three adapter+routes, loopback included, and answers 401 without it. A non-loopback bind always has+one, since `soup serve` refuses to start there without it:++```bash+soup serve --model base-model --adapters chat=./chat-adapter --host 0.0.0.0 --tool-auth-token "$TOKEN"++curl -X POST -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/v1/adapters/activate/chat+curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/v1/adapters+```+Names are validated against `^[a-zA-Z0-9][a-zA-Z0-9-]*$`; activate/deactivate calls are thread-safe behind a lock. Activate/deactivate also check the `Host` and `Origin` headers (see [Server-Side Tool Endpoints](#server-side-tool-endpoints)).### Multi-Tenant Vector Bank (`soup serve --bank`)@@ -991,7 +1002,10 @@ Three POST routes are now available on `soup serve`:Tool routes, `/v1/thumbs` and adapter activate/deactivate accept requests only when the`Host` header names the bound address (any loopback name for a loopback bind) and any-`Origin` header names the same; otherwise they answer 421 or 403.+`Origin` header names the same; otherwise they answer 421 or 403. That check runs first.+The tool routes and all three adapter routes (`GET /v1/adapters`, activate, deactivate)+then also require `Authorization: Bearer <token>` whenever `--tool-auth-token` is set,+on loopback too, and answer 401 without it.The generation routes (`/v1/chat/completions`, `/v1/messages`) and the adapter listing(`GET /v1/adapters`) carry the narrower half of that check: a request whose `Origin` namesdiff --git a/src/soup_cli/commands/loop.py b/src/soup_cli/commands/loop.pyindex 830fdf06..40bc6688 100644--- a/src/soup_cli/commands/loop.py+++ b/src/soup_cli/commands/loop.py@@ -227,12 +227,29 @@ def watch_cmd(False, "--pack-cans",help="Pack each iteration as a v0.26 Soup Can + Registry entry (v0.71.4).",),+ tool_auth_token: Optional[str] = typer.Option(+ None,+ "--tool-auth-token",+ envvar="SOUP_TOOL_AUTH_TOKEN",+ help=(+ "The soup serve --tool-auth-token value, sent as a Bearer token when the "+ "deploy stage activates an adapter. Defaults to $SOUP_TOOL_AUTH_TOKEN, "+ "which keeps it out of the shell history."+ ),+ ),) -> None:"""Run the harvest → train → gate → deploy daemon."""if detach and foreground:console.print("[red]--detach and --foreground are mutually exclusive[/]")raise typer.Exit(code=2)state = _safe_read() # ensure state exists before forking+ if tool_auth_token:+ # The deploy stage reads the token from the environment, as it reads+ # SOUP_LOOP_SERVE_ENDPOINT; a --detach child inherits it from there+ # rather than from argv, where any local user could read it (#1139).+ from soup_cli.utils.loop_stages import TOOL_AUTH_TOKEN_ENV++ os.environ[TOOL_AUTH_TOKEN_ENV] = tool_auth_tokenif pre_wired or state.pre_wired:# #1440: a configured thumbs database the harvest would refuse stops the# watch here, before it starts or detaches, instead of mid-iteration.diff --git a/src/soup_cli/utils/loop_stages.py b/src/soup_cli/utils/loop_stages.pyindex 448367a0..485a13aa 100644--- a/src/soup_cli/utils/loop_stages.py+++ b/src/soup_cli/utils/loop_stages.py@@ -400,12 +400,26 @@ def _endpoint_is_local(endpoint: str) -> bool:return ip.is_loopback or ip.is_private or ip.is_link_local+#: Environment variable holding the ``soup serve --tool-auth-token`` value.+#: ``soup loop watch --tool-auth-token`` sets it, including for a ``--detach``+#: child, so the token never has to appear in argv (#1139).+TOOL_AUTH_TOKEN_ENV = "SOUP_TOOL_AUTH_TOKEN"+++class _ActivateRefusedError(Exception):+ """The serve endpoint answered the activate POST with 401."""++def _post_activate(endpoint: str, name: str) -> bool:"""POST to ``<endpoint>/v1/adapters/activate/<name>``; True on 2xx.``endpoint`` is already SSRF-validated by the caller; ``name`` isre-validated here against the activate-route pattern before URLinterpolation (defence-in-depth against a crafted adapter dir name).+ The tool token from ``SOUP_TOOL_AUTH_TOKEN``, when set, goes as a Bearer+ header: a server started with ``--tool-auth-token`` requires it on this+ route (#1139). A 401 raises ``_ActivateRefusedError`` so the caller can name+ the missing credential instead of reporting a bare failure."""if _DEPLOY_POSTER is not None:return _DEPLOY_POSTER(endpoint, name)@@ -416,11 +430,20 @@ def _post_activate(endpoint: str, name: str) -> bool:except ImportError:return Falseurl = endpoint.rstrip("/") + f"/v1/adapters/activate/{name}"+ token = os.environ.get(TOOL_AUTH_TOKEN_ENV)+ headers = {"Authorization": f"Bearer {token}"} if token else {}try:- resp = httpx.post(url, timeout=5.0)- return 200 <= resp.status_code < 300+ # A plain-http endpoint (loopback only) ignores HTTP_PROXY / ALL_PROXY, which+ # would otherwise receive the Bearer token in clear text. An https endpoint keeps+ # trust_env: a proxy only sees a CONNECT tunnel there, and SSL_CERT_FILE /+ # SSL_CERT_DIR are how a TLS proxy's private CA gets trusted.+ plain_http = url.lower().startswith("http://")+ resp = httpx.post(url, headers=headers, timeout=5.0, trust_env=not plain_http)except Exception: # noqa: BLE001 — deploy must never crash the loopreturn False+ if resp.status_code == 401:+ raise _ActivateRefusedError()+ return 200 <= resp.status_code < 300def deploy_to_canary(@@ -477,7 +500,22 @@ def deploy_to_canary("notes": "SOUP_LOOP_SERVE_ENDPOINT must be loopback/LAN (SSRF guard)",}name = os.path.basename(str(adapter_path).rstrip("/\\")) or "canary"- ok = _post_activate(endpoint, name)+ try:+ ok = _post_activate(endpoint, name)+ except _ActivateRefusedError:+ if os.environ.get(TOOL_AUTH_TOKEN_ENV):+ note = (+ "activate POST refused (401): the serve endpoint rejected the tool "+ "token; check soup loop watch --tool-auth-token / SOUP_TOOL_AUTH_TOKEN "+ "against soup serve --tool-auth-token"+ )+ else:+ note = (+ "activate POST refused (401): the serve endpoint requires its tool "+ "token; pass soup loop watch --tool-auth-token or set SOUP_TOOL_AUTH_TOKEN"+ )+ _LOG.warning(note)+ return {"deployed": False, "canary_verdict": None, "notes": note}# activate_adapter is a full hot-swap (100% of traffic), not a canary# split; canary_router.route()/BucketStats never ran, so there is no# verdict to report here even when the POST itself succeeded.diff --git a/tests/test_issue1139_adapter_route_auth.py b/tests/test_issue1139_adapter_route_auth.pynew file mode 100644index 00000000..4d1eaae2--- /dev/null+++ b/tests/test_issue1139_adapter_route_auth.py@@ -0,0 +1,296 @@+"""#1139 — soup loop can send the serve tool token the adapter routes require.++`soup serve` gates `/v1/adapters/activate` and `/deactivate` on its+`--tool-auth-token` (#1126), so `soup loop watch` gains `--tool-auth-token`+(falling back to `SOUP_TOOL_AUTH_TOKEN`) for its deploy stage to present. The+server-side tests pin the contract the loop depends on.+"""++from __future__ import annotations++from unittest.mock import MagicMock++import pytest++pytest.importorskip("fastapi")++from fastapi.testclient import TestClient # noqa: E402++TOKEN = "s3cret-tool-token"+LAN_HOST = "192.168.1.5"+++def _client(host: str, auth_token: str | None) -> TestClient:+ from soup_cli.commands.serve import _create_app++ app = _create_app(+ model_obj=MagicMock(),+ tokenizer=MagicMock(),+ device="cpu",+ model_name="test",+ max_tokens_default=512,+ adapter_map={"chat": "./x"},+ auth_token=auth_token,+ host=host,+ )+ # The Host header names the bind, so the Host/Origin guard lets the request through.+ return TestClient(app, base_url=f"http://{host}")+++def _bearer(token: str) -> dict:+ return {"Authorization": f"Bearer {token}"}++[email protected]("host", [LAN_HOST, "127.0.0.1"], ids=["lan", "loopback"])+def test_a_configured_token_gates_activate_and_deactivate(host):+ client = _client(host, TOKEN)++ for path in ("/v1/adapters/activate/chat", "/v1/adapters/deactivate"):+ assert client.post(path).status_code == 401, f"{path} without the token"+ assert client.post(path, headers=_bearer("wrong")).status_code == 401, (+ f"{path} with the wrong token"+ )+ assert client.post(path, headers=_bearer(TOKEN)).status_code == 200, (+ f"{path} with the token"+ )+++def test_the_loopback_default_without_a_token_is_unchanged():+ client = _client("127.0.0.1", None)++ resp = client.post("/v1/adapters/activate/chat")+ assert resp.status_code == 200+ assert resp.json()["active"] == "chat"+ assert client.post("/v1/adapters/deactivate").status_code == 200+++def test_the_token_gates_the_adapter_listing_but_not_the_model_list():+ client = _client(LAN_HOST, TOKEN)++ assert client.get("/v1/models").status_code == 200+ assert client.get("/v1/adapters").status_code == 401+ assert client.get("/v1/adapters", headers=_bearer(TOKEN)).status_code == 200+++def test_no_token_on_a_lan_bind_refuses_the_adapter_routes():+ assert _client(LAN_HOST, None).post("/v1/adapters/activate/chat").status_code == 401+++def test_the_host_guard_runs_before_the_token_check():+ client = _client(LAN_HOST, TOKEN)++ # No token: a token check that ran first would answer 401.+ resp = client.post("/v1/adapters/activate/chat", headers={"Host": "evil.example"})+ assert resp.status_code == 421++[email protected]("path", ["/v1/adapters/activate/chat", "/v1/adapters/deactivate"])+def test_adapter_token_check_uses_compare_digest(monkeypatch, path):+ import secrets++ calls, real = [], secrets.compare_digest+ monkeypatch.setattr(secrets, "compare_digest", lambda a, b: calls.append(1) or real(a, b))+ client = _client(LAN_HOST, TOKEN)+ assert client.post(path, headers=_bearer(TOKEN)).status_code == 200+ assert calls+++# ---------------------------------------------------------------------------+# client side: the loop's deploy stage+# ---------------------------------------------------------------------------+++def deploy_env(tmp_path, monkeypatch):+ import soup_cli.utils.loop_stages as ls+ from soup_cli.utils.loop_state import LoopState++ monkeypatch.chdir(tmp_path)+ monkeypatch.setenv("SOUP_REGISTRY_DB_PATH", str(tmp_path / "registry.db"))+ monkeypatch.setenv("SOUP_LOOP_SERVE_ENDPOINT", f"https://{LAN_HOST}:8000")+ monkeypatch.delenv(ls.TOOL_AUTH_TOKEN_ENV, raising=False)+ monkeypatch.setattr(ls, "_DEPLOY_POSTER", None)++ def deploy():+ state = LoopState(+ served_model="m", eval_suite="e", baseline="b", status="running"+ )+ return ls.deploy_to_canary(+ state, {"gate_verdict": "OK", "adapter_path": "adapters/chat"}+ )++ return deploy+++def _route_httpx_to(monkeypatch, client: TestClient) -> list:+ """Send the deploy stage's ``httpx.post`` to a real app through ``client``."""+ import httpx++ sent = []++ def post(url, headers=None, timeout=None, **_options):+ sent.append(dict(headers or {}))+ return client.post(url, headers=headers or {})++ monkeypatch.setattr(httpx, "post", post)+ return sent+++def test_the_loop_token_reaches_a_server_started_with_it(deploy_env, monkeypatch):+ import soup_cli.utils.loop_stages as ls++ sent = _route_httpx_to(monkeypatch, _client(LAN_HOST, TOKEN))+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, TOKEN)++ out = deploy_env()++ assert out["deployed"] is True+ assert sent == [_bearer(TOKEN)]++[email protected]("token", [None, "wrong-0xC0FFEE"], ids=["missing", "wrong"])+def test_a_refused_activate_names_the_credential(deploy_env, monkeypatch, caplog, token):+ import soup_cli.utils.loop_stages as ls++ _route_httpx_to(monkeypatch, _client(LAN_HOST, TOKEN))+ if token:+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, token)++ out = deploy_env()++ assert out["deployed"] is False+ assert "401" in out["notes"]+ assert "--tool-auth-token" in out["notes"]+ assert "SOUP_TOOL_AUTH_TOKEN" in out["notes"]+ expected = "rejected the tool token" if token else "requires its tool token"+ assert expected in out["notes"]+ if token:+ assert token not in out["notes"]+ assert token not in caplog.text+++def test_no_token_sends_no_authorization_header(deploy_env, monkeypatch):+ sent = _route_httpx_to(monkeypatch, _client("127.0.0.1", None))+ monkeypatch.setenv("SOUP_LOOP_SERVE_ENDPOINT", "http://127.0.0.1:8000")++ out = deploy_env()++ assert out["deployed"] is True+ assert sent == [{}]+++# ---------------------------------------------------------------------------+# the CLI option+# ---------------------------------------------------------------------------+++def test_watch_detach_hands_the_token_to_the_child_through_the_environment(+ tmp_path, monkeypatch+):+ import os++ from typer.testing import CliRunner++ import soup_cli.commands.loop as loop_cmd+ from soup_cli.cli import app+ from soup_cli.utils.loop_stages import TOOL_AUTH_TOKEN_ENV++ monkeypatch.chdir(tmp_path)+ # recorded so monkeypatch restores the real value after the command sets it+ monkeypatch.setenv(TOOL_AUTH_TOKEN_ENV, "placeholder")+ spawned = {}++ class _Proc:+ pid = 4242++ def popen(argv, **kwargs):+ spawned["argv"] = list(argv)+ spawned["env"] = dict(os.environ) if kwargs.get("env") is None else kwargs["env"]+ return _Proc()++ monkeypatch.setattr(loop_cmd.subprocess, "Popen", popen)+ runner = CliRunner()+ runner.invoke(app, ["loop", "init", "m", "--eval", "e", "--baseline", "b"])++ result = runner.invoke(app, ["loop", "watch", "--detach", "--tool-auth-token", TOKEN])++ assert result.exit_code == 0, result.output+ assert TOKEN not in spawned["argv"], "the token must not appear in the child's argv"+ assert spawned["env"][TOOL_AUTH_TOKEN_ENV] == TOKEN+++def test_the_token_is_masked_in_the_audit_log():+ from soup_cli.cli import _mask_audit_args++ masked = _mask_audit_args("loop", ["watch", "--tool-auth-token", TOKEN])++ assert TOKEN not in " ".join(masked)+ assert "--tool-auth-token" in masked+++def test_the_activate_post_ignores_environment_proxies(monkeypatch):+ import socket+ import threading++ import soup_cli.utils.loop_stages as ls++ hits = []+ proxy = socket.socket()+ proxy.bind(("127.0.0.1", 0))+ proxy.listen(1)+ proxy.settimeout(3)++ def accept_one():+ try:+ conn, _ = proxy.accept()+ except OSError:+ return+ hits.append(conn.recv(65536))+ conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\nConnection: close\r\n\r\n")+ conn.close()++ thread = threading.Thread(target=accept_one, daemon=True)+ thread.start()++ dead = socket.socket()+ dead.bind(("127.0.0.1", 0))+ dead_port = dead.getsockname()[1]+ dead.close() # nothing listens here, so only a proxy could answer++ monkeypatch.setattr(ls, "_DEPLOY_POSTER", None)+ monkeypatch.setenv("HTTP_PROXY", f"http://127.0.0.1:{proxy.getsockname()[1]}")+ monkeypatch.delenv("NO_PROXY", raising=False)+ monkeypatch.delenv("no_proxy", raising=False)+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, "proxy-leak-token")++ assert ls._post_activate(f"http://127.0.0.1:{dead_port}", "chat") is False++ proxy.close()+ thread.join(5)+ assert not hits, "the tool token must not be sent to an environment proxy"+++def test_an_https_endpoint_keeps_the_environment_for_its_ca(monkeypatch):+ # A LAN endpoint is https behind a TLS-terminating proxy, whose certificate for a+ # private IP comes from a private CA. httpx trusts that CA only through SSL_CERT_FILE /+ # SSL_CERT_DIR, which trust_env=False switches off; on https a proxy sees only CONNECT.+ import httpx++ import soup_cli.utils.loop_stages as ls++ calls = []++ def fake_post(url, **kwargs):+ calls.append((url, kwargs.get("trust_env", True)))+ return httpx.Response(200)++ monkeypatch.setattr(ls, "_DEPLOY_POSTER", None)+ monkeypatch.setattr(httpx, "post", fake_post)+ monkeypatch.setenv(ls.TOOL_AUTH_TOKEN_ENV, TOKEN)++ assert ls._post_activate("https://192.168.1.5:8443", "chat") is True+ assert ls._post_activate("http://127.0.0.1:8000", "chat") is True+ assert calls == [+ ("https://192.168.1.5:8443/v1/adapters/activate/chat", True),+ ("http://127.0.0.1:8000/v1/adapters/activate/chat", False),+ ]