Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
Python cliconsumer-gpudpofine-tuningggufhuggingfacellmllmopslocal-ailocal-llmloralow-vrammachine-learningollamapeftpythonpytorchqlorasfttransformers
fix(serve): require --tool-auth-token for the python and bash tool routes on every bind (#2174)
Alpamys Makazhan38aded14c0d6
diff --git a/changelog.d/0.76.0/2174.changed.md b/changelog.d/0.76.0/2174.changed.mdnew file mode 100644index 00000000..945b2d40--- /dev/null+++ b/changelog.d/0.76.0/2174.changed.md@@ -0,0 +1 @@+- **`soup serve`: `/v1/tools/python` and `/v1/tools/bash` require `--tool-auth-token` on every bind, as they already did beyond loopback.** A server started without a token now answers both routes with 401 on a loopback bind too, before the request's content is used; the reply names the fix (start `soup serve` with `--tool-auth-token <secret>` or set `SOUP_TOOL_AUTH_TOKEN`, and send `Authorization: Bearer <secret>`). `--tool-auth-token` now also reads `SOUP_TOOL_AUTH_TOKEN`, which keeps the token off the command line. `/v1/tools/web_search`, `/v1/thumbs`, the adapter routes and the generation routes are unchanged, and `soup serve` still starts without a token on loopback. To keep calling the two routes after upgrading, pass the token to the server and send it with each request (#2174)diff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex b3ee895a..c7f8075f 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -3448,7 +3448,7 @@ Usage: `soup serve [OPTIONS]`| `--bank-strength` | float | `1.0` | | Vector-bank delta strength multiplier. Ignored when --bank is unset. || `--mole` | text | | | Path to a MoLE training output dir (mole_gate.pt + mole_manifest.json). Serves the base + N frozen task LoRAs with per-token gate blending at decode time. Requires --backend transformers; not combinable with --bank / --steer / --adapters. (v0.71.17 #259) || `--kv-cache-type` | text | | | KV-cache type for decoding: q8_0 (8-bit quantized, needs hqq) / bf16 / f16 (cache dtype) / fp8 (vLLM+Hopper only). Transformers backend only — vLLM / SGLang routing is in the blocked tail. v0.71.14 (#140). |-| `--tool-auth-token` | text | | | Require 'Authorization: Bearer <token>' on the code-exec tool endpoints (/v1/tools/python, /v1/tools/bash, /v1/tools/web_search). Strongly recommended whenever --host is not loopback, since those endpoints run caller-supplied Python in a best-effort sandbox. |+| `--tool-auth-token` | text | | | Require 'Authorization: Bearer <token>' on the tool endpoints (/v1/tools/python, /v1/tools/bash, /v1/tools/web_search). Without it, /v1/tools/python and /v1/tools/bash answer 401 on every bind and a non-loopback --host is refused. Env: `SOUP_TOOL_AUTH_TOKEN`. |## `soup ship`diff --git a/docs/serving-and-export.md b/docs/serving-and-export.mdindex 0c216050..a6693437 100644--- a/docs/serving-and-export.md+++ b/docs/serving-and-export.md@@ -996,7 +996,7 @@ Three POST routes are now available on `soup serve`:- **`POST /v1/tools/bash`** — runs a bash command with strict OS-level networkisolation (`unshare` on supported Linux runtimes, `sandbox-exec` on macOS).- Enforces mandatory Bearer token authentication when bound to a non-loopback host,+ Enforces the `--tool-auth-token` Bearer token on every bind,a 5-second wall-clock timeout, a minimal secret-scrubbed environment, and a combined10KB stdout/stderr streaming kill limit. Returns the real stdout, stderr, exit code,and timeout state. This is not a filesystem sandbox: the child retains the server@@ -1004,6 +1004,27 @@ Three POST routes are now available on `soup serve`:HTTP 501 when strict OS isolation is unavailable (including on Windows or restrictedLinux containers).+**`/v1/tools/python` and `/v1/tools/bash` need `--tool-auth-token` on every bind.** A+server started without it answers both routes with 401 on loopback too, as it already did+on any other bind, and the reply names the fix. The token is checked before the OS-level+isolation requirement above, so without it the reply is 401 even on a host where the route+would answer 501; with it, the isolation rule decides. Start the server with a token, from+the option or from `SOUP_TOOL_AUTH_TOKEN` (which keeps it off the command line and out of+the shell history), and send it as a Bearer header:++```bash+export SOUP_TOOL_AUTH_TOKEN="$(python -c 'import secrets; print(secrets.token_urlsafe(32))')"+soup serve --model ./output++curl -X POST http://127.0.0.1:8000/v1/tools/python \+ -H "Authorization: Bearer $SOUP_TOOL_AUTH_TOKEN" \+ -H "Content-Type: application/json" \+ -d '{"code": "print(1)"}'+```++`/v1/tools/web_search`, `/v1/thumbs` and the adapter routes keep their rule: on a loopback+bind they take a token only when the server has one.+Tool routes, `/v1/thumbs` and adapter activate/deactivate accept requests only when the`Host` header names the bound address (any loopback name for a loopback bind) and any`Origin` header names the same; otherwise they answer 421 or 403. That check runs first.diff --git a/src/soup_cli/commands/serve.py b/src/soup_cli/commands/serve.pyindex 53575eb7..bafdd47e 100644--- a/src/soup_cli/commands/serve.py+++ b/src/soup_cli/commands/serve.py@@ -309,11 +309,12 @@ def serve(tool_auth_token: Optional[str] = typer.Option(None,"--tool-auth-token",+ envvar="SOUP_TOOL_AUTH_TOKEN",help=(- "Require 'Authorization: Bearer <token>' on the code-exec tool "- "endpoints (/v1/tools/python, /v1/tools/bash, /v1/tools/web_search). "- "Strongly recommended whenever --host is not loopback, since those "- "endpoints run caller-supplied Python in a best-effort sandbox."+ "Require 'Authorization: Bearer <token>' on the tool endpoints "+ "(/v1/tools/python, /v1/tools/bash, /v1/tools/web_search). Without it, "+ "/v1/tools/python and /v1/tools/bash answer 401 on every bind and a "+ "non-loopback --host is refused."),),):@@ -1245,6 +1246,13 @@ _METRICS_BACKENDS = frozenset({"transformers", "vllm"})# The binds `soup serve` accepts without --tool-auth-token (the same three# spellings the refusal in serve() and _check_tool_auth compare against)._LOOPBACK_BINDS = frozenset({"127.0.0.1", "localhost", "::1"})+# What /v1/tools/python and /v1/tools/bash answer, on any bind, when the server+# was started without --tool-auth-token: the 401 detail names the fix.+_CODE_TOOL_TOKEN_REQUIRED = (+ "Authentication required: /v1/tools/python and /v1/tools/bash need a token on "+ "every bind. Start soup serve with --tool-auth-token <secret> (or set "+ "SOUP_TOOL_AUTH_TOKEN) and send 'Authorization: Bearer <secret>'."+)def _non_loopback_notices(host: str, backend: str) -> list[str]:@@ -1725,12 +1733,13 @@ def _create_app("""Create the FastAPI application with OpenAI-compatible endpoints.Args:- auth_token: optional Bearer-token gate for the v0.53.7 tool- endpoints (``/v1/tools/python``, ``/v1/tools/bash`` and- ``/v1/tools/web_search``).- When ``None`` (default), endpoints inherit the server's- loopback-only CORS trust boundary. When set, callers must- supply ``Authorization: Bearer <token>``.+ auth_token: Bearer token for the v0.53.7 tool endpoints+ (``/v1/tools/python``, ``/v1/tools/bash`` and+ ``/v1/tools/web_search``), ``/v1/thumbs`` and the adapter routes.+ When set, callers must supply ``Authorization: Bearer <token>``.+ When ``None`` (default), ``/v1/tools/python`` and ``/v1/tools/bash``+ answer 401 on every bind; the other routes take no token on a+ loopback bind and answer 401 on any other."""import secrets as _secretsimport threading as _threading@@ -1789,6 +1798,12 @@ def _create_app(status_code=401, detail="Invalid or missing bearer token")+ def _check_code_tool_auth(authorization: Optional[str]) -> None:+ """The python and bash tool routes take a token on every bind, loopback included."""+ if not auth_token:+ raise HTTPException(status_code=401, detail=_CODE_TOOL_TOKEN_REQUIRED)+ _check_tool_auth(authorization)+from collections.abc import AsyncIteratorfrom contextlib import asynccontextmanager@@ -1824,7 +1839,8 @@ def _create_app(# None of the three covers a non-browser client, which sends no Origin at# all and can name the bind address in Host — for those, on a non-loopback# bind, the bearer token checked by `_check_tool_auth` is the only thing- # protecting these routes.+ # protecting these routes. The python and bash tool routes take that token+ # on every bind, loopback included (`_check_code_tool_auth`).app.add_middleware(CORSMiddleware,allow_origin_regex=r"^https?://(localhost|127\.0\.0\.1)(:\d+)?$",@@ -2308,7 +2324,7 @@ def _create_app(payload: dict,authorization: Optional[str] = Header(default=None),) -> dict:- _check_tool_auth(authorization)+ _check_code_tool_auth(authorization)if not isinstance(payload, dict):raise HTTPException(status_code=400, detail="Invalid request")code = payload.get("code")@@ -2347,7 +2363,7 @@ def _create_app(payload: dict,authorization: Optional[str] = Header(default=None),) -> dict:- _check_tool_auth(authorization)+ _check_code_tool_auth(authorization)if not isinstance(payload, dict):raise HTTPException(status_code=400, detail="Invalid request")command = payload.get("command")diff --git a/tests/conftest.py b/tests/conftest.pyindex f0dea438..b4cb2920 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -171,6 +171,17 @@ def _isolate_experiments_db(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) ->monkeypatch.setenv("SOUP_DB_PATH", str(tmp_path / "experiments.db"))[email protected](autouse=True)+def _no_ambient_tool_auth_token(monkeypatch: pytest.MonkeyPatch) -> None:+ """Start every test without ``SOUP_TOOL_AUTH_TOKEN`` from the developer's shell.++ ``soup serve --tool-auth-token`` reads it, so an exported token would turn a test+ of a refused bind, or of a route without a token, into one that passes the check.+ A test that needs it sets it itself.+ """+ monkeypatch.delenv("SOUP_TOOL_AUTH_TOKEN", raising=False)++@pytest.fixture(autouse=True)def _restore_working_directory():"""Put the working directory back after every test.diff --git a/tests/test_code_review_critical.py b/tests/test_code_review_critical.pyindex 6849fb07..6f3531be 100644--- a/tests/test_code_review_critical.py+++ b/tests/test_code_review_critical.py@@ -146,14 +146,26 @@ def test_tool_python_endpoint_requires_bearer_when_token_set():)-def test_tool_python_gate_is_noop_without_token():+def test_tool_python_refuses_without_a_configured_token():pytest.importorskip("fastapi")from fastapi.testclient import TestClientclient = TestClient(_build_tool_app(None), base_url="http://127.0.0.1")- # No token configured -> auth passes -> empty body fails the code check- # with 400 (NOT 401). Proves the gate is opt-in, and reachable.- assert client.post("/v1/tools/python", json={}).status_code == 400+ # No token configured -> 401 on loopback too, before the body is read, and+ # the detail names the option that fixes it.+ resp = client.post("/v1/tools/python", json={})+ assert resp.status_code == 401+ assert "--tool-auth-token" in resp.json()["detail"]+++def test_tool_python_gate_is_reachable_with_the_right_token():+ pytest.importorskip("fastapi")+ from fastapi.testclient import TestClient++ client = TestClient(_build_tool_app("secret"), base_url="http://127.0.0.1")+ # Right token -> auth passes -> empty body fails the code check with 400.+ resp = client.post("/v1/tools/python", json={}, headers={"Authorization": "Bearer secret"})+ assert resp.status_code == 400# ─────────────────────────── CRITICAL 3: eval benchmark injection ──────────────diff --git a/tests/test_serve_code_tool_token.py b/tests/test_serve_code_tool_token.pynew file mode 100644index 00000000..778ee697--- /dev/null+++ b/tests/test_serve_code_tool_token.py@@ -0,0 +1,306 @@+"""``soup serve``: the code-execution tool routes take ``--tool-auth-token`` on every bind.++``/v1/tools/python`` and ``/v1/tools/bash`` answer 401 unless the server was started+with ``--tool-auth-token`` (or ``SOUP_TOOL_AUTH_TOKEN``) and the request sends+``Authorization: Bearer <token>``. They already did so beyond loopback; they now do so+on a loopback bind too. ``/v1/tools/web_search``, ``/v1/thumbs`` and the adapter routes+keep their rule: a token only when one is configured.++The token check comes after the Host/Origin check and before the request's content is+used or a sandbox starts, so a refused request runs nothing. An empty token, from the+option, the environment or ``_create_app``, counts as no token.+"""++from __future__ import annotations++from unittest.mock import MagicMock++import pytest++pytest.importorskip("fastapi")+pytest.importorskip("httpx")++from fastapi.testclient import TestClient # noqa: E402+from typer.testing import CliRunner # noqa: E402++from soup_cli.cli import app as cli_app # noqa: E402+from tests.conftest import strip_ansi # noqa: E402++runner = CliRunner()++TOKEN = "s3cret-tool-token"+AUTH = {"Authorization": f"Bearer {TOKEN}"}++# The exact refusal a client reads when the server has no token configured.+TOKEN_REQUIRED = (+ "Authentication required: /v1/tools/python and /v1/tools/bash need a token on "+ "every bind. Start soup serve with --tool-auth-token <secret> (or set "+ "SOUP_TOOL_AUTH_TOKEN) and send 'Authorization: Bearer <secret>'."+)+BAD_BEARER = "Invalid or missing bearer token"++CODE_ROUTES = [+ ("/v1/tools/python", {"code": "print(1)"}),+ ("/v1/tools/bash", {"command": "echo 1"}),+]+CODE_ROUTE_IDS = ["python", "bash"]+LOOPBACK_BINDS = ["127.0.0.1", "localhost", "::1"]+++def _plain(text: str) -> str:+ """ANSI-stripped, whitespace-collapsed CLI output, safe to substring-match."""+ return " ".join(strip_ansi(text).split())+++def _app(**kwargs):+ from soup_cli.commands.serve import _create_app++ base = dict(+ model_obj=MagicMock(),+ tokenizer=MagicMock(),+ device="cpu",+ model_name="test-model",+ max_tokens_default=64,+ )+ base.update(kwargs)+ return _create_app(**base)+++def sandbox_calls(monkeypatch):+ """Stand in for both sandboxes and record every run they are asked for.++ The isolation strategy is pinned to one that lets both routes run, so what is+ measured here is the token check alone, on any host.+ """+ from soup_cli.trainer import rewards++ calls: list[tuple[str, str]] = []++ def _fake_python(code, **_options):+ calls.append(("python", code))+ return "ran\n"++ def _fake_bash(command, **_options):+ calls.append(("bash", command))+ return rewards.SandboxProcessResult(returncode=0, stdout="ran\n", stderr="")++ monkeypatch.setattr(rewards, "_get_isolation_strategy", lambda: "namespaces")+ monkeypatch.setattr(rewards, "_run_code_sandbox", _fake_python)+ monkeypatch.setattr(rewards, "_run_bash_sandbox", _fake_bash)+ return calls+++class TestNoTokenConfigured:+ @pytest.mark.parametrize("bind", LOOPBACK_BINDS)+ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_code_routes_refuse_on_a_loopback_bind(self, bind, path, body, sandbox_calls):+ client = TestClient(_app(host=bind), base_url="http://127.0.0.1:8000")++ resp = client.post(path, json=body)++ assert resp.status_code == 401, (path, resp.text)+ assert resp.json()["detail"] == TOKEN_REQUIRED+ assert sandbox_calls == []++ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_code_routes_refuse_beyond_loopback(self, path, body, sandbox_calls):+ client = TestClient(_app(host="0.0.0.0"), base_url="http://10.0.0.7:8000")++ resp = client.post(path, json=body)++ assert resp.status_code == 401, (path, resp.text)+ assert resp.json()["detail"] == TOKEN_REQUIRED+ assert sandbox_calls == []++ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_a_bearer_header_does_not_stand_in_for_a_server_token(+ self, path, body, sandbox_calls+ ):+ client = TestClient(_app(), base_url="http://127.0.0.1:8000")++ resp = client.post(path, json=body, headers=AUTH)++ assert resp.status_code == 401, (path, resp.text)+ assert resp.json()["detail"] == TOKEN_REQUIRED+ assert sandbox_calls == []++ @pytest.mark.parametrize(+ "header",+ [None, AUTH, {"Authorization": "Bearer"}],+ ids=["no-header", "bearer", "empty-bearer"],+ )+ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_an_empty_token_is_no_token(self, header, path, body, sandbox_calls):+ client = TestClient(_app(auth_token=""), base_url="http://127.0.0.1:8000")++ resp = client.post(path, json=body, headers=header)++ assert resp.status_code == 401, (path, resp.text)+ assert resp.json()["detail"] == TOKEN_REQUIRED+ assert sandbox_calls == []++ @pytest.mark.parametrize(("path", "_body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_the_token_is_checked_before_the_body_content(self, path, _body, sandbox_calls):+ client = TestClient(_app(), base_url="http://127.0.0.1:8000")++ resp = client.post(path, json={})++ assert resp.status_code == 401, (path, resp.text)+ assert resp.json()["detail"] == TOKEN_REQUIRED++ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_the_token_is_checked_before_the_isolation_requirement(+ self, path, body, sandbox_calls, monkeypatch+ ):+ from soup_cli.trainer import rewards++ # A host without OS-level isolation answers 501 once a request is let in.+ monkeypatch.setattr(rewards, "_get_isolation_strategy", lambda: "best-effort")+ client = TestClient(_app(), base_url="http://127.0.0.1:8000")++ resp = client.post(path, json=body)++ assert resp.status_code == 401, (path, resp.text)+ assert resp.json()["detail"] == TOKEN_REQUIRED+ assert sandbox_calls == []++ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_the_host_check_still_answers_first(self, path, body, sandbox_calls):+ client = TestClient(_app(), base_url="http://evil.example:8000")++ resp = client.post(path, json=body)++ assert resp.status_code == 421, (path, resp.text)+ assert sandbox_calls == []++ def test_web_search_keeps_its_rule_on_loopback(self):+ from soup_cli.utils.server_tools import WebSearchConfig++ def fake_backend(query, max_results, allowlist):+ return [{"url": "https://example.com/a", "snippet": "s"}]++ client = TestClient(+ _app(+ web_search_config=WebSearchConfig(domain_allowlist=("example.com",)),+ web_search_backend=fake_backend,+ ),+ base_url="http://127.0.0.1:8000",+ )++ resp = client.post("/v1/tools/web_search", json={"query": "q"})++ assert resp.status_code == 200, resp.text+ assert [r["url"] for r in resp.json()["results"]] == ["https://example.com/a"]++ def test_adapter_listing_keeps_its_rule_on_loopback(self):+ client = TestClient(+ _app(adapter_map={"chat": "/fake/chat"}), base_url="http://127.0.0.1:8000"+ )++ resp = client.get("/v1/adapters")++ assert resp.status_code == 200, resp.text+ assert [a["name"] for a in resp.json()["adapters"]] == ["chat"]+++class TestTokenConfigured:+ @pytest.mark.parametrize("bind", ["127.0.0.1", "0.0.0.0"])+ @pytest.mark.parametrize(+ "header",+ [None, "Bearer wrong", TOKEN, f"Bearer {TOKEN}x", f"bearer {TOKEN}"],+ ids=["missing", "wrong", "no-scheme", "suffix", "lowercase-scheme"],+ )+ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_code_routes_refuse_a_bad_bearer(self, bind, header, path, body, sandbox_calls):+ client = TestClient(_app(host=bind, auth_token=TOKEN), base_url="http://127.0.0.1:8000")+ headers = {"Authorization": header} if header is not None else {}++ resp = client.post(path, json=body, headers=headers)++ assert resp.status_code == 401, (path, resp.text)+ assert resp.json()["detail"] == BAD_BEARER+ assert sandbox_calls == []++ @pytest.mark.parametrize("bind", LOOPBACK_BINDS + ["0.0.0.0"])+ @pytest.mark.parametrize(("path", "body"), CODE_ROUTES, ids=CODE_ROUTE_IDS)+ def test_code_routes_run_with_the_right_bearer(self, bind, path, body, sandbox_calls):+ client = TestClient(_app(host=bind, auth_token=TOKEN), base_url="http://127.0.0.1:8000")++ resp = client.post(path, json=body, headers=AUTH)++ assert resp.status_code == 200, (path, resp.text)+ assert resp.json()["stdout"] == "ran\n"+ assert [route for route, _ in sandbox_calls] == [path.rsplit("/", 1)[1]]++ def test_with_the_right_bearer_the_body_is_checked(self, sandbox_calls):+ client = TestClient(_app(auth_token=TOKEN), base_url="http://127.0.0.1:8000")++ resp = client.post("/v1/tools/python", json={}, headers=AUTH)++ assert resp.status_code == 400, resp.text+ assert sandbox_calls == []+++class TestCommandLine:+ def test_help_says_the_code_routes_need_the_token_on_every_bind(self, monkeypatch):+ monkeypatch.setenv("COLUMNS", "200")++ result = runner.invoke(cli_app, ["serve", "--help"])++ assert result.exit_code == 0, (result.output, repr(result.exception))+ out = _plain(result.output.replace("│", " "))+ assert (+ "Without it, /v1/tools/python and /v1/tools/bash answer 401 on every bind "+ "and a non-loopback --host is refused." in out+ )+ assert "SOUP_TOOL_AUTH_TOKEN" in out+ assert "Strongly recommended whenever --host is not loopback" not in out++ def test_the_env_var_supplies_the_token(self, monkeypatch):+ monkeypatch.setenv("COLUMNS", "400")+ monkeypatch.setenv("SOUP_TOOL_AUTH_TOKEN", "env-token-value-123")++ # The model path does not exist, so the command stops before a server+ # starts; the bind check runs before that.+ result = runner.invoke(+ cli_app, ["serve", "--model", "no-such-model-dir", "--host", "0.0.0.0"]+ )++ out = _plain(result.output)+ assert "requires --tool-auth-token" not in out, out+ assert "Note: bound to '0.0.0.0', not loopback" in out, out+ assert "env-token-value-123" not in out++ @pytest.mark.parametrize(+ ("env", "option"),+ [("", []), (None, ["--tool-auth-token", ""])],+ ids=["empty-env-var", "empty-option"],+ )+ def test_an_empty_token_is_no_token_at_start(self, env, option, monkeypatch):+ monkeypatch.setenv("COLUMNS", "400")+ if env is not None:+ monkeypatch.setenv("SOUP_TOOL_AUTH_TOKEN", env)++ result = runner.invoke(+ cli_app, ["serve", "--model", "no-such-model-dir", "--host", "0.0.0.0", *option]+ )++ assert result.exit_code == 2, (result.output, repr(result.exception))+ out = _plain(result.output)+ assert "binding non-loopback host '0.0.0.0' requires --tool-auth-token <secret>" in out+ assert "Note: bound to" not in out++ def test_without_the_env_var_a_non_loopback_bind_is_still_refused(self, monkeypatch):+ monkeypatch.setenv("COLUMNS", "400")+ monkeypatch.delenv("SOUP_TOOL_AUTH_TOKEN", raising=False)++ result = runner.invoke(+ cli_app, ["serve", "--model", "no-such-model-dir", "--host", "0.0.0.0"]+ )++ assert result.exit_code == 2, (result.output, repr(result.exception))+ assert (+ "binding non-loopback host '0.0.0.0' requires --tool-auth-token <secret>"+ in _plain(result.output)+ )diff --git a/tests/test_serve_python_tool_isolation.py b/tests/test_serve_python_tool_isolation.pyindex 9bcd5768..3cbe9aad 100644--- a/tests/test_serve_python_tool_isolation.py+++ b/tests/test_serve_python_tool_isolation.py@@ -35,6 +35,8 @@ TOKEN = "s3cret-token"def _client(**kwargs) -> TestClient:+ # /v1/tools/python and /v1/tools/bash answer 401 without a token on every bind+ # (tests/test_serve_code_tool_token.py), so the app has one and the client sends it.from soup_cli.commands.serve import _create_appbase = dict(@@ -43,9 +45,14 @@ def _client(**kwargs) -> TestClient:device="cpu",model_name="test-model",max_tokens_default=128,+ auth_token=TOKEN,)base.update(kwargs)- return TestClient(_create_app(**base), base_url="http://127.0.0.1")+ return TestClient(+ _create_app(**base),+ base_url="http://127.0.0.1",+ headers={"Authorization": f"Bearer {TOKEN}"},+ )@pytest.fixturediff --git a/tests/test_v0536.py b/tests/test_v0536.pyindex 35367cb9..140018cb 100644--- a/tests/test_v0536.py+++ b/tests/test_v0536.py@@ -262,6 +262,12 @@ def _fake_model_and_tokenizer():return model, tokenizer+# /v1/tools/python and /v1/tools/bash answer 401 without a token on every bind+# (tests/test_serve_code_tool_token.py), so the app is built with one.+_TOOL_TOKEN = "tool-test-token"+_TOOL_AUTH = {"Authorization": f"Bearer {_TOOL_TOKEN}"}++def _build_app(**overrides):pytest.importorskip("fastapi")from soup_cli.commands.serve import _create_app@@ -273,6 +279,7 @@ def _build_app(**overrides):device="cpu",model_name="test-model",max_tokens_default=64,+ auth_token=_TOOL_TOKEN,)kwargs.update(overrides)return _create_app(**kwargs)@@ -508,9 +515,9 @@ def test_tool_endpoint_returns_501(path: str, monkeypatch):monkeypatch.setattr(rewards, "_get_isolation_strategy", lambda: "namespaces")monkeypatch.setattr(os, "unshare", lambda *args, **kwargs: None, raising=False)app = _build_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)response = client.post(path, json={"code": "print(1)"})- assert response.status_code != 501+ assert response.status_code not in (401, 501), response.text# ----------------------------------------------------------------------diff --git a/tests/test_v0537.py b/tests/test_v0537.pyindex eaf3e6d7..3c1c63e6 100644--- a/tests/test_v0537.py+++ b/tests/test_v0537.py@@ -876,7 +876,14 @@ class TestInstantiateTrainerPlugins:# ----- #103 /v1/tools/{python,bash,web_search} live ----------------------def _create_test_app(host: str = "127.0.0.1", auth_token: str | None = None, **kwargs):+# /v1/tools/python and /v1/tools/bash answer 401 without a token on every bind+# (tests/test_serve_code_tool_token.py), so the apps below are built with one by+# default and the live-route tests send it.+_TOOL_TOKEN = "tool-test-token"+_TOOL_AUTH = {"Authorization": f"Bearer {_TOOL_TOKEN}"}+++def _create_test_app(host: str = "127.0.0.1", auth_token: str | None = _TOOL_TOKEN, **kwargs):"""Build a test FastAPI app via _create_app (lazy fastapi import)."""try:import fastapi # noqa: F401@@ -925,7 +932,7 @@ class TestToolEndpointsLive:from fastapi.testclient import TestClient_allow_live_python_tool(monkeypatch)app = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)resp = client.post("/v1/tools/python",json={"code": "print('hello')"},@@ -940,14 +947,14 @@ class TestToolEndpointsLive:def test_python_tool_rejects_missing_code(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)resp = client.post("/v1/tools/python", json={})assert resp.status_code == 400def test_python_tool_rejects_oversize_code(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)oversize = "x" * (64 * 1024 + 1)resp = client.post("/v1/tools/python", json={"code": oversize})assert resp.status_code == 400@@ -955,7 +962,7 @@ class TestToolEndpointsLive:def test_python_tool_non_dict_rejected(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)# FastAPI body parsing accepts list as dict-typed param? It coerces.# Use empty code which is rejected as ValueError-equivalent.resp = client.post("/v1/tools/python", json={"code": ""})@@ -969,7 +976,7 @@ class TestToolEndpointsLive:from soup_cli.trainer.rewards import SandboxProcessResultapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)monkeypatch.setattr("soup_cli.trainer.rewards._get_isolation_strategy", lambda: "namespaces")@@ -990,14 +997,14 @@ class TestToolEndpointsLive:def test_bash_tool_rejects_missing_command(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)resp = client.post("/v1/tools/bash", json={})assert resp.status_code == 400def test_bash_tool_rejects_oversize_command(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)oversize = "x" * (64 * 1024 + 1)resp = client.post("/v1/tools/bash", json={"command": oversize})assert resp.status_code == 400@@ -1010,7 +1017,7 @@ class TestToolEndpointsLive:from soup_cli.trainer.rewards import SandboxProcessResultapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)monkeypatch.setattr("soup_cli.trainer.rewards._get_isolation_strategy", lambda: "namespaces")@@ -1041,7 +1048,7 @@ class TestToolEndpointsLive:def test_bash_tool_windows_returns_501(self, monkeypatch):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)monkeypatch.setattr("soup_cli.trainer.rewards._get_isolation_strategy", lambda: "best-effort")@@ -1054,7 +1061,7 @@ class TestToolEndpointsLive:def test_bash_tool_auth_required_on_non_loopback_host(self):from fastapi.testclient import TestClient- app = _create_test_app(host="0.0.0.0")+ app = _create_test_app(host="0.0.0.0", auth_token=None)client = TestClient(app, base_url="http://127.0.0.1")resp = client.post("/v1/tools/bash",@@ -1226,7 +1233,7 @@ class TestToolEndpointsLive:def test_bash_tool_restricted_linux_fails_closed_501(self, monkeypatch):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)monkeypatch.setattr("soup_cli.trainer.rewards._get_isolation_strategy", lambda: "namespaces")@@ -1248,7 +1255,7 @@ class TestToolEndpointsLive:from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)monkeypatch.setattr("soup_cli.trainer.rewards._get_isolation_strategy", lambda: "namespaces")@@ -1270,7 +1277,7 @@ class TestToolEndpointsLive:def test_web_search_default_deny_all(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)resp = client.post("/v1/tools/web_search",json={"query": "anything"},@@ -1281,7 +1288,7 @@ class TestToolEndpointsLive:def test_web_search_rejects_oversize_query(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)resp = client.post("/v1/tools/web_search",json={"query": "x" * 2000},@@ -1291,7 +1298,7 @@ class TestToolEndpointsLive:def test_web_search_rejects_bool_max_results(self):from fastapi.testclient import TestClientapp = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)resp = client.post("/v1/tools/web_search",json={"query": "q", "max_results": True},@@ -1342,7 +1349,7 @@ class TestToolEndpointsLive:from fastapi.testclient import TestClient_allow_live_python_tool(monkeypatch)app = _create_test_app()- client = TestClient(app, base_url="http://127.0.0.1")+ client = TestClient(app, base_url="http://127.0.0.1", headers=_TOOL_AUTH)# Python: should return 200 with sandbox response.resp = client.post("/v1/tools/python", json={"code": "print(1)"})assert resp.status_code != 501@@ -1947,16 +1954,21 @@ class TestReviewFixesAuthToken:_allow_live_python_tool(monkeypatch)from soup_cli.commands.serve import _create_app+ from soup_cli.utils.server_tools import WebSearchConfig+ # web_search keeps the opt-in gate on loopback; /v1/tools/python and+ # /v1/tools/bash take a token on every bind (test_serve_code_tool_token.py).app = _create_app(model_obj=MagicMock(),tokenizer=MagicMock(),device="cpu",model_name="test-model",max_tokens_default=128,+ web_search_config=WebSearchConfig(domain_allowlist=("example.com",)),+ web_search_backend=lambda query, max_results, allowlist: [],)client = TestClient(app, base_url="http://127.0.0.1")- resp = client.post("/v1/tools/python", json={"code": "print(1)"})+ resp = client.post("/v1/tools/web_search", json={"query": "q"})assert resp.status_code == 200