Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
feat(gateway): run LLM calls with no own key on the OpenCompany Gateway
trohitg3f32e4e159f0
diff --git a/.env.template b/.env.templateindex c5b74b13..4ee232a6 100644--- a/.env.template+++ b/.env.template@@ -333,4 +333,11 @@ WEBSOCKET_API_KEY=# Default port for AI proxy servers (e.g., Ollama, Claude Code CLI proxy)AI_PROXY_DEFAULT_PORT=11434+# OpenCompany Gateway (docs-internal/llm_gateway.md). When enabled, the+# Connectors panel offers "OpenCompany Gateway": with a gateway key there, LLM+# calls for a provider this install has no key of its own for go through the+# gateway, billed to the key owner's monthly allowance.+OPENCOMPANY_GATEWAY_ENABLED=false+OPENCOMPANY_GATEWAY_URL=https://api.gw.opencompany.sh+diff --git a/CLAUDE.md b/CLAUDE.mdindex 229ae60b..70e2ed9e 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -51,7 +51,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire| **[Translate / Transliterate / Detect](./docs-internal/speech_provider_rfc.md#8-the-pattern-generalised-nodestranslate)** | The second application of the multi-vendor pattern, and the one that retired `nodes/sarvam/`. Three nodes (`translateText`, `transliterateText`, `detectLanguage`) in [`server/nodes/translate/`](./server/nodes/translate/), backed by **three registries — one per capability**, because the asymmetry is sharper than speech's: DeepL translates and does nothing else, while Sarvam and LLM-backed providers do all three. Membership makes an unsupported selection unrepresentable rather than a runtime failure inside a paid call. Providers: **deepl** (`Authorization: DeepL-Auth-Key`, `text` is an array, free keys ending `:fx` route to a different host, and it returns `billed_characters` so cost is the provider's own figure), **sarvam** (all three over dedicated REST), **openai** (all three via `ChatUnifier` — proves the Protocol does not care what a provider does internally; billing deliberately NOT recorded because that path bills tokens the LLM layer already costs). Capabilities in `server/config/translate_defaults.json`. || **[Desktop App](./docs-internal/desktop_app.md)** | Electron shell in [`desktop/`](./desktop/) — a standalone bun package (NOT a root workspace member; `cli/tests/test_release_pipeline_config.py` locks `workspaces`). Bundles pinned `uv`, python-build-standalone CPython 3.12 and bun 1.4.0 — the only JavaScript runtime shipped; no Node, no npm (`desktop/runtimes.json`, verified against upstream checksum manifests by `scripts/fetch-runtimes.ts`, which also fetches bun's `LICENSE.md` from the bun repo — the release zip carries none — and prunes stale runtime dirs such as a leftover `node/`), stages the backend sibling layout from `bun pm pack --dry-run` (`scripts/stage.ts`, which runs the sidecar package's own `bun run build` — `bun build --target=bun`, express inlined — and copies `dist/index.js`), sets `OPENCOMPANY_BUN_BIN` / `BUN_INSTALL=<userData>/bun` / `BUN_INSTALL_CACHE_DIR` and prepends the bundled bun dir (then uv) to PATH for the backend (`env.ts`; `OPENCOMPANY_NODE_BIN` no longer exists), provisions the venv into Electron `userData` on first launch with `uv sync --frozen --python <bundled>` (stamp = sha256 of uv.lock + pyproject + Python + uv versions; falls back to `uv python install 3.12`), pre-warms `PYTHONPYCACHEPREFIX` (with a prefix set Python ignores in-tree .pyc — 32 s vs 3 s first boot), spawns uvicorn on `127.0.0.1:5678` (persisted port; attaches to a running `company serve` instead of double-spawning), waits on `/health/ready`, loads the backend-served SPA (zero frontend changes), routes `window.open` / `_blank` to the OS browser. Quit = `POST /api/desktop/shutdown` + stdin EOF, 30 s, then tree-kill. electron-builder: NSIS x64, DMG+zip arm64/x64, AppImage+deb x64, unsigned first (`mac.identity: null`; macOS auto-update notify-only until signed — Squirrel.Mac refuses unsigned). CI: `desktop-release.yml` (tag-triggered: `prepare` drafts the GitHub Release from the annotated tag's message, three legs build and upload version-less OS-named installers such as `OpenCompany-windows-x64.exe` / `OpenCompany-macos-arm64.dmg` / `OpenCompany-linux-x86_64.AppImage` so the README can link `releases/latest/download/<file>`, each leg refuses a desktop version that differs from the tag, `finalize` undrafts) and `desktop-ci.yml` (typecheck/unit/invariants/Playwright smoke). **Gotcha: editor-hosted terminals export `ELECTRON_RUN_AS_NODE=1`, which makes Electron boot as plain Node (`electron.app` undefined) — strip it before launching; the e2e test does.** || **[Desktop Host Contract](./docs-internal/desktop_host_contract.md)** | The backend's side of being owned by a GUI shell. `core/approot.py` is the ONE module that locates the app tree (`OPENCOMPANY_APP_ROOT` / `OPENCOMPANY_CLIENT_DIST` / `OPENCOMPANY_ENV_FILE` / `OPENCOMPANY_ENV_TEMPLATE`); `tests/test_no_root_climb.py` forbids any other `Path(__file__).parents[N]` climb above `server/` and any `"client"/"dist"` / `.env.template` / `package.json` path join. `core/env_defaults.apply_file_defaults_to_environ()` does the CLI's `.env.template < .env < process env` layering at the top of `main.py`, and `Settings.env_file` is the absolute `(template, .env)` tuple — so a shell spawns `python -m uvicorn` from any cwd without the CLI package. Desktop mode (`OPENCOMPANY_DESKTOP=1`, `core/desktop.py`): parent-PID watchdog with create-time PID-reuse guard, stdin-EOF watchdog (polling via `PeekNamedPipe`/`select`, never a blocking read — a daemon thread parked in `ReadFile` at finalisation crashed exit with 0xC0000005), ctypes Job Object so children die with the backend (no pywin32 in the server venv), token-gated `POST /api/desktop/shutdown` (`routers/desktop.py`, public path; raises the signal uvicorn handles so the lifespan reaps Temporal/the bun sidecar/edgymeow), 45 s hard deadline. `GET /health/ready` (public) = DB up AND (Temporal disabled OR worker manager started), with `phase` written by `services/temporal/lifecycle.py` (`installing_temporal` / `starting_temporal` / `connecting` / `starting_workers` / `ready`). Plugin shutdown hooks now run under a 10 s per-hook timeout (`HOOK_TIMEOUT_SECONDS`), and the current browser hook uses `peek_browser_runtime()` so shutdown never constructs or installs the runtime. Historical incident: the retired `get_browser_service()` lazily installed agent-browser at shutdown and wedged graceful exit (masked under the CLI by its 5 s grace + tree-kill). `server/uv.lock` is committed; `predeploy.yml` runs `uv lock --check`. |-| **[LLM Gateway](./docs-internal/llm_gateway.md)** | Hosted LiteLLM Proxy in `gateway/` (official AWS module, mock-model local stack, contract and pin tests, `gateway-ci.yml`): pinned LiteLLM 1.103.4 and price list, served providers, and the contract OpenCompany relies on (wire model names, `budget_exceeded` refusals, `/user/info` allowance with Default keys, attribution headers, cache replay). Operator runbook: `gateway/README.md`. OpenCompany-side routing not implemented yet. |+| **[LLM Gateway](./docs-internal/llm_gateway.md)** | Hosted LiteLLM Proxy in `gateway/` (official AWS module, mock-model local stack, contract and pin tests, `gateway-ci.yml`): pinned LiteLLM 1.103.4 and price list, served providers, and the contract OpenCompany relies on (wire model names, `budget_exceeded` refusals, `/user/info` allowance with Default keys, attribution headers, cache replay). Operator runbook: `gateway/README.md`. OpenCompany's side, `server/services/gateway/`: the Connectors card (`OPENCOMPANY_GATEWAY_ENABLED`), own-keys-first routing in ChatUnifier from `"gateway"` blocks in `llm_defaults.json`, attribution from the log context, gateway error wording. || **[CI/CD Pipeline](./docs-internal/ci_cd.md)** | GitHub Actions workflows, predeploy validation, release publishing, and the composite setup action (bun via SHA-pinned `oven-sh/setup-bun` reading the root `packageManager` pin + Node 22 as a CI-only convenience so bun runs vite / vitest / eslint on it via their node shebangs + Python 3.12 + uv; `bun install --frozen-lockfile` everywhere; publish is `bun publish --access public` after a `bun pm whoami` credential preflight with `~/.npmrc` written from `NPM_TOKEN` — no npm CLI, no `setup-node` step of their own in the publish jobs (they get Node through the composite action), and npm provenance attestation went away with npm). Also **Cutting a release**: the annotated tag is the version and its message is the release notes; `company version sync vX.Y.Z` bumps every version file for the `chore(release)` commit the tag sits on, and `desktop-release.yml` drafts the GitHub Release from the tag message and refuses a version that differs from the tag || **[Performance](./docs-internal/performance.md)** | Cold-start measurements (Application startup complete 2.90 s warm, 21.5 s cold post-`company clean` — was 71 s before the 2026-07-14 boot-delay fixes; first WS connect 8.29 s) + warm and cold per-phase timelines + optimisation history (lazy LangChain imports, lazy LLM-SDK exception refs, pre-bundled sidecar (esbuild at the time, `bun build --target=bun` today), bytecode-at-build, PartySocket reconnect, TanStack Query auth bootstrap, `manualChunks`, Vite dep-cache preservation, off-loop Temporal build-id hash) + bottleneck inventory + reproduction commands + anti-patterns to never reintroduce (incl. eager SDK import at LLM provider registration, unconditional `.vite` wipe, synchronous `Worker()` on the event loop). Pairs with the build-time pipeline doc below. || **[Release Build Pipeline](./docs-internal/release_build_pipeline.md)** | Build pipeline for the registry tarball (installed with `bun add -g`; bun is the only JavaScript runtime and package manager anything shipped needs): `typescript@7` (native Go compiler, exact-pinned in **root** devDependencies — client keeps `typescript@^5.x` because typescript-eslint's peer range excludes 6/7, and one manifest cannot declare `typescript` twice) for type-check, measured 5.9× faster than tsc 5.9.3 (820 ms vs 4830 ms); `bun run --filter react-flow-client typecheck` delegates up to the root gate, Vite `manualChunks` + `target: 'es2022'`, `bun build --target=bun` sidecar bundle (self-contained `dist/index.js` with express inlined, run as `bun dist/index.js`; drops the `tsx` interpreter cost and needs no `node_modules` at runtime), bytecode pre-compile in two halves — `[tool.uv] compile-bytecode = true` (uv sync compiles `.venv/` site-packages) + `python -m compileall` for project source (plain `.pyc`, **no `-O`** — per PEP 488 the non-`-O` runtimes never load `.opt-1.pyc`; excludes `.venv/`, `tests/`). Single source of truth for the bytecode-compile path list: `cli.commands.build.COMPILEALL_SOURCE_DIRS`. Dev PM is **[email protected]** (tracked text `bun.lock`, `bunfig.toml` pins the isolated linker, ranged security pins in top-level `overrides`); `company build` step [1/6] is `bun install`. |diff --git a/client/src/components/credentials/panels/ApiKeyPanel.tsx b/client/src/components/credentials/panels/ApiKeyPanel.tsxindex 85b5efc8..cd31339f 100644--- a/client/src/components/credentials/panels/ApiKeyPanel.tsx+++ b/client/src/components/credentials/panels/ApiKeyPanel.tsx@@ -169,6 +169,9 @@ const ApiKeyPanel: React.FC<CredentialPanelProps> = ({{field?.help && !endpoints && (<p className="mt-2 text-xs text-muted-foreground">{field.help}</p>)}+ {panel.statusSummary && validated && (+ <p className="mt-2 text-sm">{panel.statusSummary}</p>+ )}</CardContent></Card>diff --git a/client/src/components/credentials/useCredentialPanel.ts b/client/src/components/credentials/useCredentialPanel.tsindex 600cb29e..eb74cccd 100644--- a/client/src/components/credentials/useCredentialPanel.ts+++ b/client/src/components/credentials/useCredentialPanel.ts@@ -230,8 +230,21 @@ export function useCredentialPanel(config: ProviderConfig, visible: boolean) {execute(type, () => sendRequest(type, data ?? {}, timeoutMs)),};+ // A key-based card that declares a status hook (``ws.status``) shows the+ // line it returns as ``summary`` once a key is stored: the OpenCompany+ // Gateway's remaining allowance, for one. OAuth cards keep their own flow.+ const statusHook = config.kind !== 'oauth' ? config.ws?.status : undefined;+ const statusQuery = useQuery<{ summary?: unknown }, Error>({+ queryKey: ['credential-status', config.id],+ queryFn: () => sendRequest(statusHook as string, {}),+ enabled: Boolean(visible && isConnected && statusHook && stored),+ staleTime: 60_000,+ });+ const statusSummary = typeof statusQuery.data?.summary === 'string' ? statusQuery.data.summary : null;+return {form, values, loading, error, stored, setStored, setError,+ statusSummary,sources: credentialValuesQuery.data?.sources ?? {},verificationCode,execute, actions, isConnected,diff --git a/docs-internal/llm_gateway.md b/docs-internal/llm_gateway.mdindex 6015494f..89fe572e 100644--- a/docs-internal/llm_gateway.md+++ b/docs-internal/llm_gateway.md@@ -2,7 +2,19 @@OpenCompany's hosted LLM gateway is the stock open-source LiteLLM Proxy, deployed from `gateway/` with BerriAI's official Terraform module. Invited people call LLMs on platform-owned vendor keys, each with a monthly allowance, and spend is recorded per person and per session. The repository holds only configuration and tests; there is no custom gateway code. The operator runbook (deployment, adding people, upgrades) is [gateway/README.md](../gateway/README.md).-**Status:** the gateway (Phase 0) is in `gateway/`. The OpenCompany side (routing, the Connectors entry, attribution headers) is not implemented yet; the contract below is what it will rely on, and `gateway/tests/test_gateway_contract.py` pins each part of it against the local stack.+**Status:** the gateway is in `gateway/`; OpenCompany's side is `server/services/gateway/` and covers chat through ChatUnifier (agents, chat runs, employee setup, translate). Embeddings, speech, vision, RLM and Claude Code still call vendors directly. `gateway/tests/test_gateway_contract.py` pins the contract below against the local stack; `server/tests/gateway/` covers OpenCompany's side.++## OpenCompany's side++**Turning it on.** `OPENCOMPANY_GATEWAY_ENABLED=true` (`gateway_enabled`, default off) lists the **OpenCompany Gateway** card in Connectors (`requires_setting` in `server/config/credential_providers.json`, honoured by `CredentialRegistry.get_all_providers`). `OPENCOMPANY_GATEWAY_URL` (`gateway_url`) points at the gateway. Off, behaviour is exactly as before. Distributed mode keeps it off: the gateway key has no 1Password support yet.++**The key.** One gateway key per install, saved like any API key (`OpenCompanyGatewayCredential`, `services/gateway/credential.py`, default `validate`). Its probe calls the gateway's `/user/info` with the key: 401 means a wrong key; 403 means an LLM-API key, and the person is told to create a Default one. The card's `ws.status` hook (`gateway_status`, `services/gateway/handlers.py`) returns the allowance as one line, which the API-key panel shows under the key (`statusSummary` in `useCredentialPanel`).++**Routing: own keys first** (`services/gateway/routing.py`). A call whose provider has a key of the install's own goes to that vendor as always. Otherwise, if the gateway is enabled, its key is stored and the provider has a `"gateway"` block in `server/config/llm_defaults.json`, `ChatUnifier.chat` builds the client with the gateway key and base URL (instead of the `{ref}_proxy` row) and passes the block's `model_prefix` to `OpenAIProvider`, which applies it to the wire model only. The gateway key stays inside the router and ChatUnifier: callers pass no key (`_resolve_activity_api_key` returns `""`), and it never enters node parameters or Temporal payloads. `ChatUnifier.fetch_models` never routes (it validates a key against its own vendor); `AIService.fetch_models` lists a gateway-covered provider's curated models with no network call. The checks that used to refuse "no key configured" accept gateway coverage (`GatewayRouter.covers`, or `gateway_covers` outside the DI graph): the three in `services/ai.py`, `_resolve_activity_api_key` and `prepare_agent_payload`, `Connections.ai_providers` (employee setup and the chat model picker), the global model list, and translate.++**Attribution** (`services/gateway/attribution.py`). Ids come from the structlog log context: `BaseNode.execute` binds the workflow, execution, root execution, chat run, employee job and user ids; the Temporal activity interceptor binds `temporal_workflow_id`, `temporal_run_id`, `temporal_activity_id` and `temporal_attempt`; the LLM-step and compaction activities bind the `attribution` dict their payload carries (`_attribution_ids` in `agent_workflow.py`). `log_context` restores outer values on exit, so a tool node run inside an agent keeps the agent's ids. Headers: session = chat run, else root execution, else execution, else Temporal workflow; tags `source:`, `workflow:`, `user:`; every allowlisted id as metadata. Inside a Temporal activity, calls send a replay-cache namespace from the key's fingerprint and the Temporal workflow, run and activity ids (not the attempt), so a retried activity replays at $0 and a re-run never replays another run's answer. Gemini's native route gets the headers but no cache body (unverified there).++**Errors** (`services/gateway/errors.py`, `services/llm/protocol.py`). `_classify_error` maps the `budget_exceeded` type to BILLING before any status check, so the run fails once with no retry loop, and a deployment-spawned run pauses its deployment. Failures of gateway calls get gateway wording through `public_message` and the new `public_hint` (allowance used up; model not on the gateway, from `model_cost_map_missing`; reconnect the gateway key). Compaction inside the agent workflow raises such failures instead of retrying them (`stop_on_user_action`).## Layoutdiff --git a/docs-internal/native_llm_sdk.md b/docs-internal/native_llm_sdk.mdindex 2c2ef17f..7aefe762 100644--- a/docs-internal/native_llm_sdk.md+++ b/docs-internal/native_llm_sdk.md@@ -147,6 +147,14 @@ the stored key as stored (no provider rewrites a key any more). The OpenAI SDKthen sends to the configured local endpoint — traffic never reachesapi.openai.com.+**OpenCompany Gateway** — a `chat` call that arrives with no key (the install+has none for that provider) goes to the hosted gateway when it is connected and+the provider's `llm_defaults.json` block has a `"gateway"` entry: the gateway's+URL replaces the `{ref}_proxy` row, the gateway key is sent, `OpenAIProvider`+prefixes the wire model (`groq/...`), and every provider passes the per-request+`extra_headers` / `extra_body` (attribution, replay cache). A call with a key is+untouched. See [LLM Gateway](./llm_gateway.md).+**Provider detection** ([`server/constants.py:detect_ai_provider`](../server/constants.py)) MUST list `ollama` / `lmstudio` substrings, or a chat-model node falls through to `'openai'`. Agents need no edit: their `provider` field is loader-driven.**Open-world providers** — local model names like `qwen/qwen3.6-27b` don't contain provider substrings, so the cloud-style pattern check would always reject them. A provider block declares `"open_world_models": true` (openrouter, groq, ollama, lmstudio, openai_compatible) and `is_model_valid_for_provider` accepts any id for it. The upstream server still rejects genuinely missing models with a clear 404.diff --git a/gateway/tests/test_pins.py b/gateway/tests/test_pins.pyindex f5ed90bf..b51ec031 100644--- a/gateway/tests/test_pins.py+++ b/gateway/tests/test_pins.py@@ -3,7 +3,8 @@- LiteLLM's version and the price-list commit agree across Terraform, thelocal stack and CI.- The pinned price list prices the default model of every provider the- gateway serves, so OpenCompany's defaults are never refused as unpriced.+ gateway serves (a "gateway" block in server/config/llm_defaults.json), so+ OpenCompany's defaults are never refused as unpriced."""from __future__ import annotations@@ -23,18 +24,10 @@ COMPOSE = yaml.safe_load((REPO / "gateway/local/docker-compose.yml").read_text(eCI = yaml.safe_load((REPO / ".github/workflows/gateway-ci.yml").read_text(encoding="utf-8"))LITELLM = COMPOSE["services"]["litellm"]-# Providers the gateway serves: OpenCompany provider id (server/config/-# llm_defaults.json) -> the prefix of its keys in LiteLLM's price list.-SERVED_PROVIDERS = {- "openai": "",- "anthropic": "",- "gemini": "gemini/",- "groq": "groq/",- "cerebras": "cerebras/",- "xai": "xai/",- "kimi": "moonshot/",- "mistral": "mistral/",-}+# The providers the gateway serves are the ones with a "gateway" block in+# server/config/llm_defaults.json (OpenCompany's services/gateway/routing.py).+PROVIDERS = json.loads((REPO / "server/config/llm_defaults.json").read_text(encoding="utf-8"))["providers"]+SERVED = {name: config["gateway"] for name, config in PROVIDERS.items() if isinstance(config.get("gateway"), dict)}def _terraform_string(name: str) -> str:@@ -61,11 +54,14 @@ def test_served_default_models_are_priced() -> None:if os.environ.get("CI"):raisepytest.skip(f"price list unreachable: {exc}")- providers = json.loads((REPO / "server/config/llm_defaults.json").read_text(encoding="utf-8"))["providers"]+ assert SERVED, "no provider in llm_defaults.json has a gateway block"unpriced = []- for provider, prefix in SERVED_PROVIDERS.items():- key = prefix + providers[provider]["default_model"]- entry = prices.get(key) or {}+ for provider, block in SERVED.items():+ model = PROVIDERS[provider]["default_model"]+ # LiteLLM prices a model under its wire name, its bare id, or (for+ # the native Gemini route, whose wire name is bare) gemini/<id>.+ candidates = (block.get("model_prefix", "") + model, model, f"{provider}/{model}")+ entry = next((prices[key] for key in candidates if key in prices), {})if "input_cost_per_token" not in entry and "output_cost_per_token" not in entry:- unpriced.append(key)+ unpriced.append(f"{provider}: {model}")assert not unpriced, f"served default models with no price in the pinned list: {unpriced}"diff --git a/server/config/credential_providers.json b/server/config/credential_providers.jsonindex 2a3e5e22..e0de1ff3 100644--- a/server/config/credential_providers.json+++ b/server/config/credential_providers.json@@ -113,6 +113,34 @@}]},+ "opencompany_gateway": {+ "_description": "The hosted OpenCompany Gateway (gateway/, docs-internal/llm_gateway.md). One key per install, stored like any API key; LLM calls for a provider the install has no key of its own for go through the gateway on the key owner's monthly allowance (services/gateway). Not an LLM provider itself: no llm_defaults.json block, so it never appears in model pickers. Listed only while OPENCOMPANY_GATEWAY_ENABLED is on (requires_setting). ws.status feeds the card's allowance line.",+ "name": "OpenCompany Gateway",+ "publisher": "OpenCompany",+ "verified": true,+ "consumer_category": "ai",+ "description": "Use your OpenCompany allowance for AI models you have no key for.",+ "category": "ai",+ "category_label": "AI Providers",+ "color": "dracula.purple",+ "kind": "apiKey",+ "icon_ref": "lucide:Sparkles",+ "requires_setting": "gateway_enabled",+ "ws": {+ "status": "gateway_status"+ },+ "fields": [+ {+ "key": "apiKey",+ "label": "Gateway key",+ "type": "password",+ "secret": true,+ "required": true,+ "placeholder": "sk-...",+ "help": "Create a key in the OpenCompany Gateway console (Virtual Keys, key type Default) and paste it here. Your own provider keys still take precedence."+ }+ ]+ },"openai": {"extends": "_ai_base","name": "OpenAI",diff --git a/server/config/llm_defaults.json b/server/config/llm_defaults.jsonindex 1b86ec91..b0d017b6 100644--- a/server/config/llm_defaults.json+++ b/server/config/llm_defaults.json@@ -26,6 +26,7 @@"providers": {"openai": {"display_name": "OpenAI",+ "gateway": {"model_prefix": "openai/", "base_path": "/v1"},"streaming": true,"default_model": "gpt-6-sol","popular_models": ["gpt-6-sol", "gpt-6-astra", "gpt-6-astra-pro", "gpt-6-sol-pro", "gpt-6-luna", "gpt-6-luna-pro", "gpt-5.6-sol", "gpt-5.6-sol-pro", "gpt-5.6-terra", "gpt-5.6-terra-pro", "gpt-5.6-luna", "gpt-5.6-luna-pro", "gpt-5.5", "gpt-5.4", "gpt-4.1"],@@ -86,6 +87,7 @@},"anthropic": {"display_name": "Anthropic",+ "gateway": {"model_prefix": "", "base_path": ""},"streaming": true,"default_model": "claude-opus-5-5","popular_models": ["claude-opus-5-5", "claude-fable-5-1", "claude-fable-5", "claude-opus-5", "claude-sonnet-5-5", "claude-sonnet-5", "claude-opus-4-8", "claude-opus-4-7", "claude-sonnet-4-6"],@@ -157,6 +159,7 @@},"gemini": {"display_name": "Gemini",+ "gateway": {"model_prefix": "", "base_path": ""},"default_model": "gemini-3.8-flash","popular_models": ["gemini-3.8-flash", "gemini-3.7-flash", "gemini-3.6-flash", "gemini-3.5-flash", "gemini-3.5-flash-lite", "gemini-3.1-pro-preview", "gemini-3.1-flash-lite", "gemini-3-flash-preview", "gemini-2.5-pro", "gemini-2.5-flash", "gemini-2.5-flash-lite"],"detection_patterns": ["gemini", "google", "gemma"],@@ -237,6 +240,7 @@},"groq": {"display_name": "Groq",+ "gateway": {"model_prefix": "groq/", "base_path": "/v1"},"default_model": "openai/gpt-oss-120b","popular_models": [],"_popular_note": "Empty per the 1M-context policy — no Groq-served model reaches 1M (all top out at 131072). Models remain reachable via live fetch_models.",@@ -272,6 +276,7 @@},"cerebras": {"display_name": "Cerebras",+ "gateway": {"model_prefix": "cerebras/", "base_path": "/v1"},"default_model": "gpt-oss-120b","popular_models": [],"_popular_note": "Empty per the 1M-context policy — no Cerebras model reaches 1M (all 131072). Models remain reachable via live fetch_models.",@@ -298,6 +303,7 @@},"xai": {"display_name": "xAI",+ "gateway": {"model_prefix": "xai/", "base_path": "/v1"},"default_model": "grok-4.20","popular_models": ["grok-4.20", "grok-4.20-multi-agent", "grok-4.3"],"_popular_note": "grok-4.7, grok-4.6 and grok-4.5 are excluded per the 1M-context policy (500K each); reachable via live fetch_models. Default stays grok-4.20 (2M) so the default is selectable from the global list.",@@ -356,6 +362,7 @@},"kimi": {"display_name": "Kimi",+ "gateway": {"model_prefix": "moonshot/", "base_path": "/v1"},"default_model": "kimi-k3","popular_models": ["kimi-k3"],"_popular_note": "Only kimi-k3 meets the 1M-context policy (k2.x are 262144); k2.x remain reachable via live fetch_models.",@@ -384,6 +391,7 @@},"mistral": {"display_name": "Mistral",+ "gateway": {"model_prefix": "mistral/", "base_path": "/v1"},"default_model": "mistral-large-latest","popular_models": [],"_popular_note": "Empty per the 1M-context policy — Mistral tops out at 262144 (large/medium/small). Models remain reachable via live fetch_models.",diff --git a/server/core/config.py b/server/core/config.pyindex f4f54c3b..c9ea3b65 100644--- a/server/core/config.py+++ b/server/core/config.py@@ -356,6 +356,12 @@ class Settings(BaseSettings):# AI Proxy Configuration (Ollama-style proxy)ai_proxy_default_port: int = Field(default=11434, env="AI_PROXY_DEFAULT_PORT")+ # OpenCompany Gateway (hosted LiteLLM, gateway/): with a gateway key in+ # Connectors, LLM calls for a provider this install has no key of its+ # own for go through the gateway on the person's allowance.+ gateway_enabled: bool = Field(default=False, validation_alias="OPENCOMPANY_GATEWAY_ENABLED")+ gateway_url: str = Field(default="https://api.gw.opencompany.sh", validation_alias="OPENCOMPANY_GATEWAY_URL")+# Residential Proxy Configurationproxy_enabled: bool = Field(default=False, env="PROXY_ENABLED")proxy_budget_daily_usd: float = Field(default=50.0, env="PROXY_BUDGET_DAILY_USD")diff --git a/server/core/container.py b/server/core/container.pyindex 56ac75e3..ff4f804b 100644--- a/server/core/container.py+++ b/server/core/container.py@@ -54,7 +54,18 @@ def _create_temporal_client(server_address: str, namespace: str):return TemporalClientWrapper(server_address=server_address, namespace=namespace)-def _build_chat_unifier(auth_service):+def _build_gateway_router(settings, auth_service):+ """The OpenCompany Gateway router (services/gateway/routing.py).++ Imported here, not at module scope, so the container's import doesn't+ register the gateway's Connectors handlers as a side effect.+ """+ from services.gateway.routing import GatewayRouter++ return GatewayRouter(settings=settings, auth_service=auth_service, defaults=LLM_DEFAULTS)+++def _build_chat_unifier(auth_service, gateway_router):"""Construct the chat-model unifier with the shared JSON defaults.Wrapped in a factory function so the DI container doesn't have to@@ -62,7 +73,7 @@ def _build_chat_unifier(auth_service):loaded once at import time by ``services.llm.config`` and reflectsthe on-disk ``server/config/llm_defaults.json``."""- return ChatUnifier(defaults=LLM_DEFAULTS, auth_service=auth_service)+ return ChatUnifier(defaults=LLM_DEFAULTS, auth_service=auth_service, gateway=gateway_router)# Wave 12 C4 sub-piece C: plugin-owned service factories.@@ -151,7 +162,11 @@ class Container(containers.DeclarativeContainer):# JSON-driven typed-exception translation + ``incompatible_models``# filter. ``AIService`` delegates to this rather than calling# provider classes directly.- chat_unifier = providers.Singleton(_build_chat_unifier, auth_service=auth_service)+ # Own keys first; a provider with no key of the install's own runs on+ # the OpenCompany Gateway when it is connected (services/gateway).+ gateway_router = providers.Singleton(_build_gateway_router, settings=settings, auth_service=auth_service)++ chat_unifier = providers.Singleton(_build_chat_unifier, auth_service=auth_service, gateway_router=gateway_router)ai_service = providers.Singleton(AIService,@@ -160,6 +175,7 @@ class Container(containers.DeclarativeContainer):cache=cache,settings=settings,chat_unifier=chat_unifier,+ gateway_router=gateway_router,)maps_service = providers.Factory(_build_maps_service, auth_service=auth_service, settings=settings)diff --git a/server/core/logging.py b/server/core/logging.pyindex a8b132aa..f24c49c6 100644--- a/server/core/logging.py+++ b/server/core/logging.py@@ -407,23 +407,20 @@ async def log_context(**fields: Any) -> AsyncIterator[None]:async with log_context(workflow_id=wf_id, node_id=node_id):await do_work() # all logs inside carry workflow_id + node_id- Bindings clear on ``__aexit__`` even if the inner block raises.+ On ``__aexit__`` each field goes back to the value it had before, even if+ the inner block raises: a node executed inside another node (a tool call)+ must not wipe its parent's fields. The LLM gateway's attribution headers+ read the same fields (services.gateway.attribution)."""- structlog.contextvars.bind_contextvars(**fields)- try:+ with structlog.contextvars.bound_contextvars(**fields):yield- finally:- structlog.contextvars.unbind_contextvars(*fields.keys())@contextmanagerdef log_context_sync(**fields: Any) -> Iterator[None]:"""Synchronous variant of :func:`log_context` for sync callers."""- structlog.contextvars.bind_contextvars(**fields)- try:+ with structlog.contextvars.bound_contextvars(**fields):yield- finally:- structlog.contextvars.unbind_contextvars(*fields.keys())def log_execution_time(logger: structlog.BoundLogger, operation: str, start_time: float, end_time: float, **kwargs) -> None:diff --git a/server/main.py b/server/main.pyindex 0e49485c..e2f86738 100644--- a/server/main.py+++ b/server/main.py@@ -154,6 +154,10 @@ async def lifespan(app: FastAPI):# save_api_key / delete_api_key).import services.credentials # noqa: F401+ # services/gateway/__init__.py registers the OpenCompany Gateway's+ # Connectors credential and its gateway_status handler.+ import services.gateway # noqa: F401+# Wave 13.7: services/workflow_storage/__init__.py self-registers the# 5 workflow-record CRUD handlers (save_workflow / import_workflow /# get_workflow / get_all_workflows / delete_workflow).diff --git a/server/nodes/translate/_base.py b/server/nodes/translate/_base.pyindex a5cdf049..f3d4d03a 100644--- a/server/nodes/translate/_base.py+++ b/server/nodes/translate/_base.py@@ -20,10 +20,21 @@ async def provider_api_key(ctx: NodeContext, provider: str) -> str:``BaseNode.execute`` turns into a credential envelope plus a CloudEventsbroadcast — lighting up the right provider in the Credentials modal."""+ from services.gateway.routing import gateway_covers+credential_id = translate_config.credential_id(provider)- async with ctx.connection(credential_id) as conn:- secrets = await conn.credentials()+ try:+ async with ctx.connection(credential_id) as conn:+ secrets = await conn.credentials()+ except PermissionError:+ # No key of the install's own: an LLM provider the OpenCompany+ # Gateway serves still runs, through ChatUnifier (services/gateway).+ if await gateway_covers(provider):+ return ""+ raiseapi_key = str(secrets.get("api_key") or "")+ if not api_key and await gateway_covers(provider):+ return ""if not api_key:raise NodeUserError(f"No API key stored for '{credential_id}'. Add one in the "diff --git a/server/services/ai.py b/server/services/ai.pyindex 68b93f61..9aae4695 100644--- a/server/services/ai.py+++ b/server/services/ai.py@@ -349,6 +349,7 @@ class AIService:cache,settings: Settings,chat_unifier=None,+ gateway_router=None,):self.auth = auth_serviceself.database = database@@ -361,6 +362,10 @@ class AIService:# the DI container can still instantiate ``AIService`` (the chat# paths will fall back to direct factory calls).self.chat_unifier = chat_unifier+ # services/gateway/routing.py: a provider with no key of the+ # install's own may still run on the OpenCompany Gateway.+ self.gateway_router = gateway_router+ # RLM service (lazy import to avoid circular deps)# RLM service (lazy import to avoid circular deps)from services.rlm import RLMService@@ -370,6 +375,11 @@ class AIService:"""Detect AI provider from model name."""return detect_provider_from_model(model)+ async def _gateway_covers(self, provider: str) -> bool:+ """No key of the install's own, but the OpenCompany Gateway serves+ ``provider``: ChatUnifier sends the call there (services/gateway)."""+ return self.gateway_router is not None and await self.gateway_router.covers(provider)+async def _prepare_context(self,*,@@ -647,7 +657,8 @@ class AIService:)# Trigger compaction if threshold exceeded- if tracking.get("needs_compaction") and memory_content and api_key:+ # No own key is fine when the OpenCompany Gateway serves the provider.+ if tracking.get("needs_compaction") and memory_content and (api_key or await self._gateway_covers(provider)):logger.info(f"[Compaction] Threshold exceeded for session {session_id}, triggering compaction")if broadcaster:@@ -708,6 +719,11 @@ class AIService:"ChatUnifier is not injected. AIService must be constructed via ""the DI container (core.container.Container).")+ if not api_key and await self._gateway_covers(provider):+ # No key of the install's own: the gateway serves this provider.+ # Its curated models, with no network call: a gateway key must+ # never reach a vendor's model-list URL.+ return self._get_curated_models(provider)try:return await self.chat_unifier.fetch_models(provider=provider, api_key=api_key)except NodeUserError:@@ -750,7 +766,7 @@ class AIService:provider = detect_ai_provider(node_type, flattened)- if not api_key:+ if not api_key and not await self._gateway_covers(provider):_require_saved_endpoint(provider)raise ValueError("API key is required")@@ -961,7 +977,7 @@ class AIService:else:logger.info(f"No model specified, using default: {model}")- if not api_key:+ if not api_key and not await self._gateway_covers(provider):_require_saved_endpoint(provider)raise ValueError("API key is required for AI Agent")@@ -1804,7 +1820,7 @@ class AIService:else:logger.info(f"No model specified, using default: {model}")- if not api_key:+ if not api_key and not await self._gateway_covers(provider):_require_saved_endpoint(provider)raise ValueError("API key is required for Zeenie")diff --git a/server/services/compaction.py b/server/services/compaction.pyindex d27882d4..7a92b07b 100644--- a/server/services/compaction.py+++ b/server/services/compaction.py@@ -20,6 +20,7 @@ from datetime import datetime, timezonefrom typing import Dict, Any, Optional, TYPE_CHECKINGfrom core.logging import get_logger+from services.llm.protocol import LLMErrorfrom services.pricing import get_pricing_serviceif TYPE_CHECKING:@@ -304,10 +305,16 @@ class CompactionService:model: str,*,explicit_max_retries: int = 2,+ stop_on_user_action: bool = False,) -> Dict[str, Any]:"""Perform compaction by summarizing conversation history.Uses the AI service to generate a structured summary following Claude Code pattern.++ A failed summarizer call returns ``success: False``. With+ ``stop_on_user_action``, a failure only the user can fix (a spent+ allowance or exhausted credits, a rejected key, missing access) raises+ the :class:`LLMError` instead, since retrying compaction cannot help."""if not self._ai_service:logger.warning("[Compaction] AI service not wired")@@ -367,6 +374,7 @@ Provide a concise but complete summary."""max_tokens=summary_tokens,sdk_max_retries=0,explicit_max_retries=explicit_max_retries,+ translate_errors=False,)compaction_usage = asdict(response.usage)await self._record_usage_metric(@@ -394,6 +402,16 @@ Provide a concise but complete summary.""""usage": compaction_usage,}+ except LLMError as e:+ if stop_on_user_action and e.requires_user_action:+ raise+ # user_message: the raw SDK text can carry payload fragments.+ logger.error(f"[Compaction] Failed: {e.user_message}")+ return {+ "success": False,+ "error": e.user_message,+ "usage": compaction_usage,+ }except Exception as e:logger.error(f"[Compaction] Failed: {e}")return {diff --git a/server/services/credential_registry.py b/server/services/credential_registry.pyindex d16a4442..ec334ed4 100644--- a/server/services/credential_registry.py+++ b/server/services/credential_registry.py@@ -125,14 +125,20 @@ class CredentialRegistry:# ----- public API -----def get_all_providers(self) -> List[Dict[str, Any]]:- """Return all concrete (non-abstract) providers in JSON order."""+ """Return all concrete (non-abstract) providers in JSON order.++ An entry declaring ``requires_setting`` is listed only while that+ ``Settings`` attribute is ``True`` (the OpenCompany Gateway's card,+ hidden unless the gateway is enabled). ``get_provider`` still+ resolves it by id.+ """resolved = self._resolve_all()# Preserve insertion order of the raw providers dict.raw = self._load_raw().get("providers", {})out: List[Dict[str, Any]] = []for provider_id in raw:entry = resolved[provider_id]- if entry.get("_abstract"):+ if entry.get("_abstract") or not _setting_enabled(entry):continueout.append(entry)return out@@ -245,6 +251,17 @@ class CredentialRegistry:# ----- deep-merge helper (n8n mergeNodeProperties pattern) -----+def _setting_enabled(entry: Dict[str, Any]) -> bool:+ """Whether an entry's ``requires_setting`` (if any) is on. ``is True``:+ under test the container's Settings is a MagicMock."""+ name = entry.get("requires_setting")+ if not name:+ return True+ from core.container import container++ return getattr(container.settings(), str(name), False) is True++def _deep_merge(parent: Dict[str, Any], child: Dict[str, Any]) -> Dict[str, Any]:"""Return a new dict: parent deep-merged with child overrides.diff --git a/server/services/employees/connections.py b/server/services/employees/connections.pyindex 03ac018f..edda2659 100644--- a/server/services/employees/connections.py+++ b/server/services/employees/connections.py@@ -88,10 +88,12 @@ class Connections:async def ai_providers(self) -> List[str]:"""LLM providers an agent could run on right now: one with a key- stored or a local server saved, and each saved OpenAI-compatible- endpoint by its reference (``openai_compatible:<slug>``). The bare- ``openai_compatible`` id is never one: an agent set to it has no- server to call."""+ stored or a local server saved, one the OpenCompany Gateway serves+ when it is connected (services/gateway), and each saved+ OpenAI-compatible endpoint by its reference+ (``openai_compatible:<slug>``). The bare ``openai_compatible`` id is+ never one: an agent set to it has no server to call."""+ from services.gateway.routing import gateway_coversfrom services.llm.config import ENDPOINT_PROVIDER, PROVIDER_CONFIGSusable = []@@ -100,6 +102,8 @@ class Connections:continueif self.provider(provider_id) is not None and await self.is_connected(provider_id):usable.append(provider_id)+ elif await gateway_covers(provider_id):+ usable.append(provider_id)try:from services.llm.endpoints import list_endpointsdiff --git a/server/services/gateway/__init__.py b/server/services/gateway/__init__.pynew file mode 100644index 00000000..64f0a9cb--- /dev/null+++ b/server/services/gateway/__init__.py@@ -0,0 +1,20 @@+"""OpenCompany Gateway: LLM calls on a hosted, capped allowance.++The gateway itself is the stock LiteLLM Proxy in ``gateway/``+(docs-internal/llm_gateway.md). This package is OpenCompany's side:++- ``routing``: which calls go to the gateway (own keys first) and where.+- ``attribution``: the headers that put each call on its spend row.+- ``errors``: what the user is told when a gateway call fails.+- ``credential`` / ``handlers``: the Connectors card for the gateway key.++Side-effect import (server/main.py) registers the credential and the+``gateway_status`` handler.+"""++from services.ws_handler_registry import register_ws_handlers as _register_ws_handlers++from . import credential as _credential # noqa: F401 (registers OpenCompanyGatewayCredential)+from .handlers import WS_HANDLERS as _WS_HANDLERS++_register_ws_handlers(_WS_HANDLERS)diff --git a/server/services/gateway/allowance.py b/server/services/gateway/allowance.pynew file mode 100644index 00000000..69def972--- /dev/null+++ b/server/services/gateway/allowance.py@@ -0,0 +1,62 @@+"""The monthly allowance a gateway key draws on, read from the gateway.++LiteLLM's ``GET /user/info`` with the key itself (no ``user_id``) returns+its owner's spend, limit and reset time. Only Default-type keys may call it;+LLM-API keys get 401/403 (gateway/tests pins both).+"""++from __future__ import annotations++from dataclasses import dataclass+from datetime import datetime+from typing import Any, Mapping, Optional++import httpx++TIMEOUT_SECONDS = 10.0+++@dataclass(frozen=True)+class Allowance:+ spend: float+ max_budget: Optional[float]+ reset_at: Optional[datetime]+ account: Optional[str]++ @property+ def summary(self) -> str:+ """``$12.40 of $50.00 used · resets Nov 1``."""+ used = f"${self.spend:,.2f}"+ text = f"{used} of ${self.max_budget:,.2f} used" if self.max_budget is not None else f"{used} used"+ if self.reset_at is not None:+ text += f" · resets {self.reset_at.strftime('%b')} {self.reset_at.day}"+ return text+++def _reset_time(value: Any) -> Optional[datetime]:+ if not isinstance(value, str) or not value:+ return None+ try:+ return datetime.fromisoformat(value.replace("Z", "+00:00"))+ except ValueError:+ return None+++def parse_user_info(body: Mapping[str, Any]) -> Allowance:+ info = body.get("user_info") if isinstance(body.get("user_info"), Mapping) else {}+ max_budget = info.get("max_budget")+ return Allowance(+ spend=float(info.get("spend") or 0.0),+ max_budget=float(max_budget) if max_budget is not None else None,+ reset_at=_reset_time(info.get("budget_reset_at")),+ account=info.get("user_email") or body.get("user_id") or info.get("user_id"),+ )+++async def user_info(gateway_url: str, api_key: str) -> httpx.Response:+ """``GET /user/info`` with ``api_key``; the caller reads the status."""+ async with httpx.AsyncClient(timeout=TIMEOUT_SECONDS) as client:+ return await client.get(+ f"{gateway_url.rstrip('/')}/user/info",+ headers={"Authorization": f"Bearer {api_key}"},+ )diff --git a/server/services/gateway/attribution.py b/server/services/gateway/attribution.pynew file mode 100644index 00000000..c76d1a79--- /dev/null+++ b/server/services/gateway/attribution.py@@ -0,0 +1,86 @@+"""What a call the OpenCompany Gateway serves says about itself.++Three LiteLLM headers put each call on its spend row: the session (the chat+run, else the root execution, else the execution, else the Temporal+workflow), a few low-cardinality tags, and every id as metadata. The ids come+from the log context (``core.logging.log_context``), bound by+``BaseNode.execute``, the Temporal activity interceptor and the agent+activities, so whatever a log line carries, the spend row carries. Only the+allowlisted ids below are sent; nothing else from the log context leaves the+install.++Inside a Temporal activity a call also opts in to the gateway's response+cache under a namespace that stays the same across that activity's retries,+so a retried call replays at $0 instead of being billed twice.+"""++from __future__ import annotations++import hashlib+import json+from typing import Any, Dict, Optional++import structlog++from core.encryption import fingerprint_credential++#: First present id names the session.+SESSION_KEYS = ("chat_run_id", "root_execution_id", "execution_id", "temporal_workflow_id")+#: Ids sent as spend-log metadata. Never extend this with free text.+METADATA_KEYS = (+ "chat_run_id",+ "root_execution_id",+ "execution_id",+ "workflow_id",+ "node_id",+ "node_type",+ "employee_job_id",+ "user_id",+ "temporal_workflow_id",+ "temporal_run_id",+ "temporal_activity_id",+ "temporal_attempt",+)+#: How long a replayable response stays cached (LiteLLM cache_params ttl).+REPLAY_TTL_SECONDS = 172_800+++def _ids() -> Dict[str, str]:+ bound = structlog.contextvars.get_contextvars()+ return {key: str(bound[key]) for key in METADATA_KEYS if bound.get(key) not in (None, "")}+++def request_headers() -> Dict[str, str]:+ """The attribution headers for the current log context."""+ ids = _ids()+ headers: Dict[str, str] = {}+ session = next((ids[key] for key in SESSION_KEYS if key in ids), None)+ if session:+ headers["x-litellm-session-id"] = session+ source = "chat" if "chat_run_id" in ids else ("workflow" if "workflow_id" in ids else "direct")+ tags = [f"source:{source}"]+ if "workflow_id" in ids:+ tags.append(f"workflow:{ids['workflow_id']}")+ if "user_id" in ids:+ tags.append(f"user:{ids['user_id']}")+ headers["x-litellm-tags"] = ",".join(tags)+ if ids:+ headers["x-litellm-spend-logs-metadata"] = json.dumps(ids, sort_keys=True, separators=(",", ":"))+ return headers+++def replay_cache(api_key: str) -> Optional[Dict[str, Any]]:+ """The replay-cache request body inside a Temporal activity, else ``None``.++ The namespace is the gateway key's fingerprint plus the Temporal workflow,+ run and activity ids: an activity's retries share it, and a re-run of the+ same workflow id (a trigger re-firing a failed event) gets a new run id,+ so it never replays another run's answer.+ """+ bound = structlog.contextvars.get_contextvars()+ scope = [bound.get(key) for key in ("temporal_workflow_id", "temporal_run_id", "temporal_activity_id")]+ if not all(scope):+ return None+ digest = hashlib.sha256("\x1f".join(str(part) for part in scope).encode("utf-8")).hexdigest()[:32]+ namespace = f"oc1-{fingerprint_credential(api_key)[:16]}-{digest}"+ return {"cache": {"use-cache": True, "namespace": namespace, "ttl": REPLAY_TTL_SECONDS}}diff --git a/server/services/gateway/credential.py b/server/services/gateway/credential.pynew file mode 100644index 00000000..a252d490--- /dev/null+++ b/server/services/gateway/credential.py@@ -0,0 +1,39 @@+"""The gateway key as a Connectors credential.++A standard API-key connector: the key is stored install-wide like every+other key (``Credential.validate`` in services/plugin/credential.py). The+probe reads the key's allowance from the gateway, which also tells a+Default key (needed to show the allowance) from an LLM-API one.+"""++from __future__ import annotations++from services.plugin.credential import ApiKeyCredential, ProbeResult++from .allowance import parse_user_info, user_info+from .routing import GATEWAY_CREDENTIAL_ID+++class OpenCompanyGatewayCredential(ApiKeyCredential):+ id = GATEWAY_CREDENTIAL_ID+ display_name = "OpenCompany Gateway"+ category = "AI"+ key_location = "bearer"+ docs_url = "https://docs.litellm.ai/docs/proxy/virtual_keys"++ @classmethod+ async def _probe(cls, api_key: str) -> ProbeResult:+ from core.container import container++ response = await user_info(container.settings().gateway_url, api_key)+ if response.status_code == 403:+ return ProbeResult(+ valid=False,+ message=(+ "This key can't read its allowance. In the OpenCompany Gateway console, "+ "create a key of type Default and paste that one."+ ),+ )+ # 401 and 5xx: Credential.validate words them (classify_credential_error).+ response.raise_for_status()+ return ProbeResult(valid=True, message=f"Connected: {parse_user_info(response.json()).summary}.")diff --git a/server/services/gateway/errors.py b/server/services/gateway/errors.pynew file mode 100644index 00000000..3c73fdcf--- /dev/null+++ b/server/services/gateway/errors.py@@ -0,0 +1,42 @@+"""What the user is told when a call the OpenCompany Gateway served fails.++The category texts name the vendor and point at its billing or key ("OpenAI+blocked this request because a spending limit ... was exhausted"), which is+the wrong account for a call billed to an OpenCompany allowance and made+with the gateway key. ChatUnifier passes every failure of a gateway call+through :func:`explain`; the category, and so retry and pause behaviour,+stays as classified.+"""++from __future__ import annotations++from typing import Optional++from services.llm.protocol import LLMError, LLMErrorCategory, error_type++_RECONNECT = "Reconnect the OpenCompany Gateway in Settings > Connectors, then resume."+_OWN_KEY = "add your own key for this provider in Settings > Connectors"+++def explain(error: LLMError, exc: Optional[BaseException]) -> LLMError:+ """Set gateway wording on ``error`` (in place) and return it."""+ kind = error_type(exc) if exc is not None else None+ if error.category is LLMErrorCategory.BILLING:+ error.public_message = "Your OpenCompany allowance for this month is used up."+ error.public_hint = f"Ask your OpenCompany admin to raise it, or {_OWN_KEY}, then resume."+ elif kind == "model_cost_map_missing":+ error.public_message = "This model isn't available on the OpenCompany Gateway yet."+ error.public_hint = f"Choose another model, or {_OWN_KEY}."+ elif error.category is LLMErrorCategory.AUTHENTICATION:+ error.public_message = "The OpenCompany Gateway rejected the gateway key."+ error.public_hint = _RECONNECT+ elif error.category is LLMErrorCategory.PERMISSION:+ error.public_message = "The OpenCompany Gateway denied this request."+ error.public_hint = _RECONNECT+ elif error.category is LLMErrorCategory.RATE_LIMIT:+ error.public_message = "The OpenCompany Gateway is rate-limiting requests. Retry after a short delay."+ elif error.category in (LLMErrorCategory.CONNECTION, LLMErrorCategory.TIMEOUT):+ error.public_message = "Could not reach the OpenCompany Gateway."+ elif error.category is LLMErrorCategory.SERVER:+ error.public_message = "The OpenCompany Gateway is temporarily unavailable."+ return errordiff --git a/server/services/gateway/handlers.py b/server/services/gateway/handlers.pynew file mode 100644index 00000000..a721f0c2--- /dev/null+++ b/server/services/gateway/handlers.py@@ -0,0 +1,49 @@+"""WebSocket handlers for the OpenCompany Gateway's Connectors card.++``gateway_status`` is the card's ``ws.status`` hook+(server/config/credential_providers.json): the allowance the stored gateway+key draws on, as one ``summary`` line the card shows.+"""++from __future__ import annotations++from typing import Any, Dict++import httpx++from core.logging import get_logger+from services.ws_handler_registry import ws_handler++from .allowance import parse_user_info, user_info+from .routing import GATEWAY_CREDENTIAL_ID++logger = get_logger(__name__)+++@ws_handler()+async def handle_gateway_status(data: Dict[str, Any], websocket: Any) -> Dict[str, Any]:+ from core.container import container++ api_key = await container.auth_service().get_api_key(GATEWAY_CREDENTIAL_ID)+ if not api_key:+ return {"connected": False}+ try:+ response = await user_info(container.settings().gateway_url, api_key)+ response.raise_for_status()+ except httpx.HTTPError as exc:+ logger.warning("Gateway allowance read failed", error=type(exc).__name__)+ return {"connected": True, "summary": "Couldn't read the allowance from the OpenCompany Gateway."}+ allowance = parse_user_info(response.json())+ return {+ "connected": True,+ "summary": allowance.summary,+ "account_label": allowance.account,+ "spend": allowance.spend,+ "max_budget": allowance.max_budget,+ "budget_reset_at": allowance.reset_at.isoformat() if allowance.reset_at else None,+ }+++WS_HANDLERS = {+ "gateway_status": handle_gateway_status,+}diff --git a/server/services/gateway/routing.py b/server/services/gateway/routing.pynew file mode 100644index 00000000..d95b5f5d--- /dev/null+++ b/server/services/gateway/routing.py@@ -0,0 +1,93 @@+"""Which LLM calls go through the OpenCompany Gateway, and where to.++Own keys first: a provider this install has a key for is called directly, as+always. Otherwise, when the gateway is enabled and connected and serves the+provider, ChatUnifier sends the call to the gateway with the install's+gateway key. The key never leaves this module and ChatUnifier: it is not+injected into node parameters, Temporal payloads or requests to a vendor.++Which providers the gateway serves, and how it names their models, is data:+the ``"gateway"`` block of each served provider in+``server/config/llm_defaults.json``. OpenAI-compatible providers use the+gateway's ``/v1`` routes with a LiteLLM provider prefix on the model name+(``groq/...``); Anthropic and Gemini use the gateway's native routes with+bare model ids.+"""++from __future__ import annotations++from dataclasses import dataclass+from typing import Any, List, Mapping, Optional++from services.llm.config import split_provider_ref++#: Catalogue entry and credential id of the gateway key (Connectors).+GATEWAY_CREDENTIAL_ID = "opencompany_gateway"+++@dataclass(frozen=True)+class GatewayRoute:+ """Where one LLM call goes when the gateway serves it."""++ api_key: str+ base_url: str+ model_prefix: str+++class GatewayRouter:+ """Decides, per provider, whether the gateway serves a call."""++ def __init__(self, *, settings: Any, auth_service: Any, defaults: Mapping[str, Any]) -> None:+ self._settings = settings+ self._auth = auth_service+ self._defaults = defaults++ def enabled(self) -> bool:+ # ``is True``: under test the container's Settings is a MagicMock,+ # whose attributes are all truthy. Distributed mode keeps credentials+ # in 1Password only, which the gateway key doesn't support yet.+ return (+ getattr(self._settings, "gateway_enabled", False) is True+ and getattr(self._settings, "distributed_mode", False) is not True+ )++ def _block(self, provider: str) -> Optional[Mapping[str, Any]]:+ providers = self._defaults.get("providers") or {}+ config = providers.get(split_provider_ref(provider)[0]) or {}+ block = config.get("gateway") if isinstance(config, Mapping) else None+ return block if isinstance(block, Mapping) else None++ def served_providers(self) -> List[str]:+ """Provider ids the gateway serves (a ``"gateway"`` block in llm_defaults.json)."""+ providers = self._defaults.get("providers") or {}+ return [name for name in providers if self._block(name) is not None]++ async def route(self, provider: str) -> Optional[GatewayRoute]:+ """The route for a call to ``provider`` with no key of the install's own."""+ if not self.enabled():+ return None+ block = self._block(provider)+ if block is None:+ return None+ api_key = await self._auth.get_api_key(GATEWAY_CREDENTIAL_ID)+ if not api_key:+ return None+ base_url = str(self._settings.gateway_url).rstrip("/") + str(block.get("base_path") or "")+ return GatewayRoute(api_key=api_key, base_url=base_url, model_prefix=str(block.get("model_prefix") or ""))++ async def covers(self, provider: str) -> bool:+ """Whether a call to ``provider`` can run on the gateway: what a "no key+ configured" check asks before refusing."""+ return await self.route(provider) is not None+++async def gateway_covers(provider: str) -> bool:+ """:meth:`GatewayRouter.covers` on the container's router, for call sites+ outside the DI graph. Under test the container is a MagicMock (whose+ router would claim to cover everything) or a stub without a router, so+ only a real :class:`GatewayRouter` is asked."""+ from core.container import container++ factory = getattr(container, "gateway_router", None)+ router = factory() if callable(factory) else None+ return isinstance(router, GatewayRouter) and await router.covers(provider)diff --git a/server/services/llm/protocol.py b/server/services/llm/protocol.pyindex 8f2880ec..4be87292 100644--- a/server/services/llm/protocol.py+++ b/server/services/llm/protocol.py@@ -364,6 +364,11 @@ class LLMError(Exception):# Must already be safe for public surfaces: no credential, no query# string, no userinfo.public_message: Optional[str] = None+ # Replaces the category's fixed recovery advice, under the same safety+ # rule. The LLM gateway sets both for calls it served+ # (services/gateway/errors.py): "check the provider's billing" would+ # point at the wrong account.+ public_hint: Optional[str] = Nonedef __post_init__(self) -> None:Exception.__init__(self, self.message)@@ -464,6 +469,8 @@ class LLMError(Exception):@propertydef hint(self) -> Optional[str]:"""Fixed recovery advice; never copy the provider's raw response."""+ if self.public_hint:+ return self.public_hintreturn {LLMErrorCategory.BILLING: "Review the provider project's billing, spending cap, and credits. Resume after access is restored; repeating the request will not fix it.",LLMErrorCategory.QUOTA: "Check the model's quota for your API project. Resume after the quota resets or is increased, or choose a model with available quota.",@@ -832,6 +839,16 @@ def _error_body(exc: BaseException) -> Mapping[str, Any]:return {}+def error_type(exc: BaseException) -> Optional[str]:+ """The ``type`` of an SDK error's JSON body (``budget_exceeded``), if any.++ ``LLMError.provider_code`` prefers the body's ``code``, which LiteLLM sets+ to the HTTP status, so the type has to be read from the body itself.+ """+ value = _error_body(exc).get("type")+ return str(value) if value else None++def _google_error_details(exc: BaseException) -> List[Mapping[str, Any]]:"""Read Google's typed details without exposing provider bodies publicly."""body = getattr(exc, "details", None)@@ -876,6 +893,11 @@ def _classify_error(body = _error_body(exc)message += " " + str(body.get("message") or "").lower()codes = {str(value).lower() for value in (getattr(exc, "code", None), body.get("code"), body.get("type"), body.get("status")) if value is not None}+ # The LLM gateway (LiteLLM) refuses a spent allowance with this type, as+ # 429 before LiteLLM 1.104 and 402 or 422 after. Whatever the status, only+ # the user can fix it.+ if "budget_exceeded" in codes:+ return LLMErrorCategory.BILLING# A quota/credit exhaustion can arrive as 429, but backoff cannot fix it.# Match narrow billing signatures before the generic HTTP classifiers.if status in {None, 400, 402, 403, 429} and (@@ -953,12 +975,18 @@ class LLMProvider(Protocol):context_management: Optional[Dict[str, Any]] = None,on_event: Optional[StreamSink] = None,effort: Optional[str] = None,+ extra_headers: Optional[Mapping[str, str]] = None,+ extra_body: Optional[Mapping[str, Any]] = None,) -> LLMResponse:"""One turn. A provider that streams (``streaming`` inllm_defaults.json) hands each delta to ``on_event`` as it arrives andstill returns the same response; the others never receive one.``effort`` (``low`` / ``high``) is the owner's chat choice, sent only- for a model its provider lists in ``effort_models``."""+ for a model its provider lists in ``effort_models``.++ ``extra_headers`` and ``extra_body`` ride on this request only: the+ LLM gateway's attribution headers and replay-cache body. ChatUnifier+ passes them only for calls that go through the gateway."""...async def fetch_models(self, api_key: str) -> List[str]: ...diff --git a/server/services/llm/providers/anthropic.py b/server/services/llm/providers/anthropic.pyindex a9c222d0..ce63630c 100644--- a/server/services/llm/providers/anthropic.py+++ b/server/services/llm/providers/anthropic.py@@ -2,7 +2,7 @@from __future__ import annotations-from typing import Any, Dict, List, Optional+from typing import Any, Dict, List, Mapping, Optionalfrom core.logging import get_loggerfrom services.llm.protocol import (@@ -58,6 +58,8 @@ class AnthropicProvider:context_management: Optional[Dict[str, Any]] = None,on_event: Optional[StreamSink] = None,effort: Optional[str] = None,+ extra_headers: Optional[Mapping[str, str]] = None,+ extra_body: Optional[Mapping[str, Any]] = None,) -> LLMResponse:system, api_msgs = self._split_system(messages)policy = self._model_policy(model)@@ -104,6 +106,13 @@ class AnthropicProvider:if tools:params["tools"] = [self._to_api_tool(t) for t in tools]+ # Per-request extras: the LLM gateway's attribution headers and+ # replay-cache body (ChatUnifier sends them for gateway calls only).+ if extra_headers:+ params["extra_headers"] = dict(extra_headers)+ if extra_body:+ params["extra_body"] = dict(extra_body)+# Always stream. The SDK refuses a non-streaming request whose# ``max_tokens`` implies more than ten minutes of generation# (``_calculate_nonstreaming_timeout``), and ``resolve_max_tokens``diff --git a/server/services/llm/providers/gemini.py b/server/services/llm/providers/gemini.pyindex 5f09ca00..94cbd9ee 100644--- a/server/services/llm/providers/gemini.py+++ b/server/services/llm/providers/gemini.py@@ -5,7 +5,7 @@ from __future__ import annotationsimport base64import hashlibimport json-from typing import Any, Dict, List, Optional+from typing import Any, Dict, List, Mapping, Optionalfrom core.logging import get_loggerfrom services.llm.protocol import (@@ -80,8 +80,13 @@ class GeminiProvider:tools: Optional[List[ToolDef]] = None,context_management: Optional[Dict[str, Any]] = None,effort: Optional[str] = None,+ extra_headers: Optional[Mapping[str, str]] = None,+ extra_body: Optional[Mapping[str, Any]] = None,) -> LLMResponse:del context_management # Gemini stays on portable Context checkpoints.+ # The LLM gateway's replay cache is verified only on its OpenAI and+ # Anthropic routes, not the native Gemini one; never sent here.+ del extra_bodyfrom google.genai import typessystem_instruction, contents = self._split_system_and_contents(messages)@@ -118,6 +123,11 @@ class GeminiProvider:if tools:config["tools"] = [self._to_api_tool(t) for t in tools]+ if extra_headers:+ # Request-level options patch the client's: headers merge, the+ # base URL stays.+ config["http_options"] = types.HttpOptions(headers=dict(extra_headers))+resp = await self._client.aio.models.generate_content(model=model,contents=contents,diff --git a/server/services/llm/providers/openai.py b/server/services/llm/providers/openai.pyindex f2516136..0b4bdd6a 100644--- a/server/services/llm/providers/openai.py+++ b/server/services/llm/providers/openai.py@@ -7,7 +7,7 @@ Also used as base for OpenRouter (same API shape, different base_url).from __future__ import annotationsimport json-from typing import Any, Dict, List, Optional+from typing import Any, Dict, List, Mapping, Optionalfrom core.logging import get_loggerfrom services.llm.protocol import (@@ -79,6 +79,10 @@ def _detail(source: Dict[str, Any]) -> str:class OpenAIProvider:provider_name = "openai"+ #: Prepended to the model name on the wire only. The LLM gateway+ #: (LiteLLM) routes OpenAI-format calls by provider prefix+ #: (``groq/llama-...``); everything else here keeps the bare id.+ _model_prefix: str = ""def __init__(self,@@ -88,10 +92,12 @@ class OpenAIProvider:base_url: Optional[str] = None,provider_name: Optional[str] = None,max_retries: int = 2,+ model_prefix: str = "",):import openaiself.provider_name = provider_name or type(self).provider_name+ self._model_prefix = model_prefixkwargs: Dict[str, Any] = {"api_key": api_key,"max_retries": max(0, int(max_retries)),@@ -110,6 +116,29 @@ class OpenAIProvider:"""The base URL requests actually go to."""return str(self._client.base_url)+ def _wire_model(self, model: str) -> str:+ """The model name a request carries (see ``_model_prefix``)."""+ return f"{self._model_prefix}{model}" if self._model_prefix else model++ def _bare_model(self, model: str) -> str:+ """A model name the server echoed, without the wire prefix."""+ prefix = self._model_prefix+ return model[len(prefix):] if prefix and model.startswith(prefix) else model++ @staticmethod+ def _add_request_extras(+ params: Dict[str, Any],+ extra_headers: Optional[Mapping[str, str]],+ extra_body: Optional[Mapping[str, Any]],+ ) -> None:+ """Per-request headers and body fields (the LLM gateway's+ attribution and replay cache), merged with any ``extra_body``+ the request already carries."""+ if extra_headers:+ params["extra_headers"] = dict(extra_headers)+ if extra_body:+ params.setdefault("extra_body", {}).update(extra_body)+def _raise_if_error_body(self, resp: Any) -> None:"""Turn a 2xx that carried an error body into a ``PROTOCOL`` failure.@@ -158,6 +187,8 @@ class OpenAIProvider:context_management: Optional[Dict[str, Any]] = None,on_event: Optional[StreamSink] = None,effort: Optional[str] = None,+ extra_headers: Optional[Mapping[str, str]] = None,+ extra_body: Optional[Mapping[str, Any]] = None,) -> LLMResponse:model = self._clean_model(model)policy = self._model_policy(model, thinking)@@ -175,10 +206,12 @@ class OpenAIProvider:context_management=context_management,on_event=on_event,effort=effort,+ extra_headers=extra_headers,+ extra_body=extra_body,)params: Dict[str, Any] = {- "model": model,+ "model": self._wire_model(model),"messages": [self._to_api_message(m) for m in messages],}@@ -221,6 +254,7 @@ class OpenAIProvider:# Toolsif tools:params["tools"] = [self._to_api_tool(t) for t in tools]+ self._add_request_extras(params, extra_headers, extra_body)if on_event is not None:return self._normalize(await self._stream_completion(params, on_event), model)@@ -264,11 +298,13 @@ class OpenAIProvider:context_management: Optional[Dict[str, Any]],on_event: Optional[StreamSink] = None,effort: Optional[str] = None,+ extra_headers: Optional[Mapping[str, str]] = None,+ extra_body: Optional[Mapping[str, Any]] = None,) -> LLMResponse:"""Run a self-contained Responses API turn for reasoning tool use."""params: Dict[str, Any] = {- "model": model,+ "model": self._wire_model(model),"input": self._to_responses_input(messages),"max_output_tokens": max_tokens,# Durable encrypted reasoning state is recorded in our message;@@ -301,6 +337,7 @@ class OpenAIProvider:"compact_threshold": threshold,}]+ self._add_request_extras(params, extra_headers, extra_body)if on_event is not None:return self._normalize_responses(await self._stream_response(params, on_event), model)response = await self._client.responses.create(**params)@@ -754,7 +791,7 @@ class OpenAIProvider:thinking=thinking_text,tool_calls=tool_calls,usage=usage,- model=str(getattr(resp, "model", "") or model),+ model=self._bare_model(str(getattr(resp, "model", "") or model)),finish_reason=str(getattr(resp, "status", "") or "stop").lower(),diff --git a/server/services/llm/providers/openrouter.py b/server/services/llm/providers/openrouter.pyindex 17d57017..ffeb3c58 100644--- a/server/services/llm/providers/openrouter.py+++ b/server/services/llm/providers/openrouter.py@@ -2,7 +2,7 @@from __future__ import annotations-from typing import Any, Dict, List, Optional+from typing import Any, Dict, List, Mapping, Optionalfrom core.logging import get_loggerfrom services.llm.protocol import (@@ -51,6 +51,8 @@ class OpenRouterProvider(OpenAIProvider):tools: Optional[List[ToolDef]] = None,context_management: Optional[Dict[str, Any]] = None,effort: Optional[str] = None,+ extra_headers: Optional[Mapping[str, str]] = None,+ extra_body: Optional[Mapping[str, Any]] = None,) -> LLMResponse:# OpenAI-compatible endpoints must prove support independently; the# OpenRouter adapter currently uses portable Context checkpoints.@@ -66,6 +68,8 @@ class OpenRouterProvider(OpenAIProvider):tools=tools,context_management=None,effort=effort,+ extra_headers=extra_headers,+ extra_body=extra_body,)async def fetch_models(self, api_key: str) -> List[str]:diff --git a/server/services/llm/unifier.py b/server/services/llm/unifier.pyindex 1096e708..57ca4d25 100644--- a/server/services/llm/unifier.py+++ b/server/services/llm/unifier.py@@ -42,6 +42,7 @@ from services.plugin import NodeUserErrorif TYPE_CHECKING:from services.auth import AuthService+ from services.gateway.routing import GatewayRoute, GatewayRouterlogger = get_logger(__name__)@@ -55,6 +56,24 @@ def _provider_failure(provider: str, exc: BaseException) -> LLMError:return error+def _gateway_request_extras(route: "GatewayRoute") -> Dict[str, Any]:+ """Per-request headers and body for a call the gateway serves."""+ from services.gateway.attribution import replay_cache, request_headers++ extras: Dict[str, Any] = {"extra_headers": request_headers()}+ cache = replay_cache(route.api_key)+ if cache is not None:+ extras["extra_body"] = cache+ return extras+++def _explain_gateway_failure(error: LLMError, exc: BaseException) -> None:+ """Gateway wording for a failed gateway call (services/gateway/errors.py)."""+ from services.gateway.errors import explain++ explain(error, exc)++@dataclass(eq=False)class _ClientEntry:"""One cached SDK client plus its in-flight lease count."""@@ -72,6 +91,11 @@ class ChatUnifier:Public surface: ``chat()``, ``fetch_models()``, ``is_registered()``.Everything else is private.++ ``gateway`` (services/gateway/routing.py) sends a chat call that comes+ with no key of the install's own to the OpenCompany Gateway when it serves+ the provider. ``fetch_models`` never goes there: it validates a key+ against its own vendor."""def __init__(@@ -80,9 +104,11 @@ class ChatUnifier:auth_service: "AuthService",*,client_cache_size: int = 32,+ gateway: Optional["GatewayRouter"] = None,):self._defaults = defaultsself._auth = auth_service+ self._gateway = gatewayself._client_cache_size = max(0, int(client_cache_size))self._client_cache: OrderedDict[str, _ClientEntry] = OrderedDict()self._retired_entries: List[_ClientEntry] = []@@ -129,15 +155,21 @@ class ChatUnifier:``effort`` is the owner's chat choice (``low`` / ``high``). It reachesthe provider only for a model listed in its ``effort_models``; anyother model answers as it would without it.++ An empty ``api_key`` means the install has no key of its own for the+ provider. If the OpenCompany Gateway serves it and is connected, the+ call goes there with the gateway key, attribution headers and, inside+ a Temporal activity, a replay-cache namespace (services/gateway)."""provider_id = split_provider_ref(provider)[0]spec = get_provider(provider_id)streams = on_event is not None and self._streams(provider_id)- api_key = self._resolve_credential(provider, api_key)+ route = await self._gateway.route(provider) if not api_key and self._gateway is not None else None+ api_key = route.api_key if route is not None else self._resolve_credential(provider, api_key)entry: Optional[_ClientEntry] = Nonetry:entry = await self._acquire_client(- spec, api_key, provider_ref=provider, sdk_max_retries=sdk_max_retries+ spec, api_key, provider_ref=provider, sdk_max_retries=sdk_max_retries, route=route)kwargs: Dict[str, Any] = {"model": model,@@ -151,6 +183,8 @@ class ChatUnifier:kwargs["on_event"] = on_eventif effort and supports_effort(provider, model):kwargs["effort"] = effort+ if route is not None:+ kwargs.update(_gateway_request_extras(route))response = await entry.client.chat(messages, **kwargs)if on_event is not None and not streams:await _replay_response(response, on_event)@@ -160,12 +194,16 @@ class ChatUnifier:# carried an error body instead of a completion. Logged whether# or not it is translated: agent steps take it untranslated, and# only this layer knows where the call went (RFC-0003 D10).+ if route is not None:+ _explain_gateway_failure(error, error)self._log_failure("LLM provider request failed", error, entry, provider)if not translate_errors:raiseraise error.as_node_error() from errorexcept spec.sdk_exception_types + (httpx.TransportError, httpx.HTTPStatusError) as e:error = _provider_failure(provider, e)+ if route is not None:+ _explain_gateway_failure(error, e)self._log_failure("LLM provider request failed", error, entry, provider)if not translate_errors:raise error from e@@ -188,6 +226,8 @@ class ChatUnifier:category=category,retryable=category == LLMErrorCategory.CONNECTION,)+ if route is not None:+ _explain_gateway_failure(error, e)logger.warning("LLM client construction failed",provider=error.provider,@@ -322,6 +362,7 @@ class ChatUnifier:*,sdk_max_retries: int,provider_ref: Optional[str] = None,+ route: Optional["GatewayRoute"] = None,) -> _ClientEntry:"""Return an entry leased until ``_release_client`` is called."""@@ -331,6 +372,7 @@ class ChatUnifier:provider_ref=provider_ref,sdk_max_retries=sdk_max_retries,acquire=True,+ route=route,)async def _get_or_create_entry(@@ -341,22 +383,30 @@ class ChatUnifier:sdk_max_retries: int,acquire: bool,provider_ref: Optional[str] = None,+ route: Optional["GatewayRoute"] = None,) -> _ClientEntry:# The base-URL row is keyed by the full reference, so each named# endpoint reads its own URL while sharing one registration.ref = provider_ref or spec.name- proxy_url = await self._auth.resolve_api_key(base_url_key(ref))- unconfigured = unconfigured_endpoint_message(ref)- if unconfigured and not proxy_url:- # A named endpoint exists only as its credential rows. Without- # the URL row the SDK would default to api.openai.com.- raise NodeUserError(unconfigured)+ if route is not None:+ # A gateway call goes to the gateway, never to a saved base URL.+ proxy_url: Optional[str] = route.base_url+ else:+ proxy_url = await self._auth.resolve_api_key(base_url_key(ref))+ unconfigured = unconfigured_endpoint_message(ref)+ if unconfigured and not proxy_url:+ # A named endpoint exists only as its credential rows. Without+ # the URL row the SDK would default to api.openai.com.+ raise NodeUserError(unconfigured)+ model_prefix = route.model_prefix if route is not None else ""factory_kwargs = {"api_key": api_key,"proxy_url": proxy_url,"max_retries": max(0, int(sdk_max_retries)),**spec.client_kwargs,}+ if model_prefix:+ factory_kwargs["model_prefix"] = model_prefixif not self._client_cache_size:return _ClientEntry(client=spec.factory(**factory_kwargs),@@ -369,6 +419,7 @@ class ChatUnifier:api_key=api_key,proxy_url=proxy_url,sdk_max_retries=sdk_max_retries,+ model_prefix=model_prefix,)evicted: Optional[_ClientEntry] = Noneasync with self._cache_lock:@@ -412,6 +463,7 @@ class ChatUnifier:api_key: str,proxy_url: Optional[str],sdk_max_retries: int,+ model_prefix: str = "",) -> str:material = json.dumps({@@ -419,6 +471,7 @@ class ChatUnifier:"proxy_url": proxy_url or "","client_kwargs": spec.client_kwargs,"max_retries": max(0, int(sdk_max_retries)),+ "model_prefix": model_prefix,},sort_keys=True,separators=(",", ":"),diff --git a/server/services/plugin/base.py b/server/services/plugin/base.pyindex 2fa01b32..763ece3f 100644--- a/server/services/plugin/base.py+++ b/server/services/plugin/base.py@@ -582,16 +582,26 @@ class BaseNode:the same identifiers as the log context."""start_time = time.time()- workflow_id_attr: Optional[str] = None- if isinstance(context.raw, dict):- workflow_id_attr = context.raw.get("workflow_id")+ raw = context.raw if isinstance(context.raw, dict) else {}+ workflow_id_attr: Optional[str] = raw.get("workflow_id")++ from services.chat.stream import chat_run_id_oflog_fields: Dict[str, Any] = {"node_id": node_id,"node_type": self.type,}- if workflow_id_attr is not None:- log_fields["workflow_id"] = workflow_id_attr+ # Which run this work belongs to. Logs carry them, and so do the LLM+ # gateway's attribution headers (services.gateway.attribution).+ run_fields = {+ "workflow_id": workflow_id_attr,+ "execution_id": getattr(context, "execution_id", None),+ "root_execution_id": raw.get("root_execution_id"),+ "chat_run_id": chat_run_id_of(raw),+ "employee_job_id": raw.get("employee_job_id"),+ "user_id": getattr(context, "user_id", None),+ }+ log_fields.update({key: value for key, value in run_fields.items() if value})async with log_context(**log_fields):with tracer.start_as_current_span(diff --git a/server/services/settings/handlers.py b/server/services/settings/handlers.pyindex 34276fb7..8ae1ce62 100644--- a/server/services/settings/handlers.py+++ b/server/services/settings/handlers.py@@ -158,13 +158,19 @@ async def handle_get_validated_ai_providers(except Exception:llm_defaults = {"providers": {}}+ from services.gateway.routing import gateway_covers+ from services.llm.config import curated_models+providers = []for provider in AI_PROVIDERS:- if not await auth_service.has_valid_key(provider, data.get("session_id", "default"), principal=principal):+ if await auth_service.has_valid_key(provider, data.get("session_id", "default"), principal=principal):+ stored_models = await auth_service.get_stored_models(provider, data.get("session_id", "default"), principal=principal)+ elif await gateway_covers(provider):+ # No key of the install's own; the OpenCompany Gateway serves it.+ stored_models = curated_models(provider)+ else:continue- stored_models = await auth_service.get_stored_models(provider, data.get("session_id", "default"), principal=principal)-provider_config = llm_defaults.get("providers", {}).get(provider, {})default_model = provider_config.get("default_model", "")popular_models = provider_config.get("popular_models") or [diff --git a/server/services/temporal/_interceptors.py b/server/services/temporal/_interceptors.pyindex f525b887..1ef2f0cb 100644--- a/server/services/temporal/_interceptors.py+++ b/server/services/temporal/_interceptors.py@@ -34,13 +34,18 @@ from temporalio.worker import (WorkflowInterceptorClassInput,)-from core.logging import get_logger+from core.logging import get_logger, log_contextlogger = get_logger(__name__)class ActivityObservabilityInterceptor(ActivityInboundInterceptor):- """Log start / retry / end (with outcome) for every activity."""+ """Log start / retry / end (with outcome) for every activity.++ Also binds the Temporal identifiers to the log context for the whole+ activity, under their own keys (``workflow_id`` elsewhere is the graph+ id). The LLM gateway's attribution reads them (services.gateway).+ """async def execute_activity(self, input: ExecuteActivityInput) -> Any:info = activity.info()@@ -57,8 +62,15 @@ class ActivityObservabilityInterceptor(ActivityInboundInterceptor):logger.warning("activity_retry", **extra)else:logger.debug("activity_start", **extra)+ bound = {+ "temporal_workflow_id": info.workflow_id,+ "temporal_run_id": info.workflow_run_id,+ "temporal_activity_id": info.activity_id,+ "temporal_attempt": info.attempt,+ }try:- result = await self.next.execute_activity(input)+ async with log_context(**bound):+ result = await self.next.execute_activity(input)except Exception as exc:logger.warning("activity_end",diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.pyindex bd3550ac..327e6c16 100644--- a/server/services/temporal/agent_activities.py+++ b/server/services/temporal/agent_activities.py@@ -221,6 +221,12 @@ async def _resolve_activity_api_key(payload: Dict[str, Any]) -> str:if isinstance(candidate, str) and candidate and parameters.get("provider", "openai") == provider:api_key = candidate+ from services.gateway.routing import gateway_covers++ if not api_key and await gateway_covers(provider):+ # No own key: ChatUnifier sends the call through the OpenCompany+ # Gateway with the install's gateway key (services/gateway).+ return ""if not api_key:raise _unsaved_endpoint_error(provider) or ApplicationError(f"API key for provider {provider!r} is not configured",@@ -720,13 +726,18 @@ async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:``heartbeat_timeout``."""+ from core.logging import log_context+activity.logger.info(f"Agent LLM step: provider={payload.get('provider')} " f"model={payload.get('model')} messages={len(payload.get('messages', []))}")activity.heartbeat(f"LLM step starting: {payload.get('model')}")await _publish_llm_attempt_status(payload)- result = await _execute_native_llm_step(payload)+ # The run's ids reach the log context before the provider call starts, so+ # its logs and the LLM gateway's spend row carry them.+ async with log_context(**(payload.get("attribution") or {})):+ result = await _execute_native_llm_step(payload)activity.heartbeat("LLM step: model returned")return result@@ -1230,6 +1241,11 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:api_key = bool(source) or await auth.has_valid_key(provider) or await auth.has_valid_key(f"{provider}_api_key")else:api_key = await auth.has_valid_key(provider) or await auth.has_valid_key(f"{provider}_api_key")+ if not api_key:+ from services.gateway.routing import gateway_covers++ # No own key, but the OpenCompany Gateway serves the provider.+ api_key = await gateway_covers(provider)if not api_key:endpoint_error = _unsaved_endpoint_error(provider)if endpoint_error:@@ -2570,17 +2586,27 @@ async def compact_context(payload: Dict[str, Any]) -> Dict[str, Any]:f"({len(rendered)} rendered chars) via "f"{payload['provider']}/{payload['model']}")- result = await svc.compact_context(- session_id=str(payload.get("session_id") or "default"),- node_id=payload["node_id"],- memory_content=rendered,- provider=payload["provider"],- api_key=await _resolve_activity_api_key(payload),- model=payload["model"],- # Temporal owns activity retries; never repeat an ambiguous provider- # request inside the activity or SDK.- explicit_max_retries=0,- )+ from core.logging import log_context+ from services.llm.protocol import LLMError++ try:+ async with log_context(**(payload.get("attribution") or {})):+ result = await svc.compact_context(+ session_id=str(payload.get("session_id") or "default"),+ node_id=payload["node_id"],+ memory_content=rendered,+ provider=payload["provider"],+ api_key=await _resolve_activity_api_key(payload),+ model=payload["model"],+ # Temporal owns activity retries; never repeat an ambiguous provider+ # request inside the activity or SDK.+ explicit_max_retries=0,+ # A spent allowance or a rejected key stops the run with its+ # reason, like an LLM step, instead of retrying compaction.+ stop_on_user_action=True,+ )+ except LLMError as error:+ raise _as_temporal_llm_error(error) from Noneif result.get("success") and result.get("summary"):usage = result.get("usage") or {}activity.logger.info(diff --git a/server/services/temporal/agent_workflow.py b/server/services/temporal/agent_workflow.pyindex fa22cfd1..f187948a 100644--- a/server/services/temporal/agent_workflow.py+++ b/server/services/temporal/agent_workflow.py@@ -344,6 +344,30 @@ def _inherited_scope(context: Dict[str, Any], *, delegation: bool = False) -> Direturn inherited+def _attribution_ids(+ context: Dict[str, Any],+ payload: Dict[str, Any],+ *,+ workflow_id: Optional[str],+ execution_id: str,+ root_execution_id: str,+) -> Dict[str, str]:+ """The ids an LLM step or compaction binds to its log context.++ Logs carry them, and so does the LLM gateway's spend row+ (services/gateway/attribution.py). Deterministic workflow state only.+ """+ ids = {+ "workflow_id": workflow_id,+ "execution_id": execution_id,+ "root_execution_id": root_execution_id,+ "chat_run_id": payload.get("chat_run_id"),+ "employee_job_id": context.get("employee_job_id"),+ "user_id": context.get("user_id"),+ }+ return {key: str(value) for key, value in ids.items() if value}++def _child_control_scope(control: ExecutionControl) -> Dict[str, Any]:"""Fresh intent travels to children; each root owns its chain identity."""return {@@ -1135,6 +1159,16 @@ class AgentWorkflow:if payload.get("effort")else {}),+ # Which run the step belongs to, for logs and the LLM+ # gateway's spend rows. Read only by the activity's log+ # context; activity input only, like the keys above.+ "attribution": _attribution_ids(+ context,+ payload,+ workflow_id=agent_workflow_id,+ execution_id=execution_id,+ root_execution_id=root_execution_id,+ ),}if context.get("native_workspace_version") == 1 or context.get("browser_runtime_version") == 1 or context.get("browser_routing_version") == 1:llm_payload["user_id"] = str(context.get("user_id") or "owner")@@ -2658,6 +2692,13 @@ class AgentWorkflow:"messages": compact_source,"provider": payload["provider"],"model": payload["model"],+ "attribution": _attribution_ids(+ context,+ payload,+ workflow_id=agent_workflow_id,+ execution_id=execution_id,+ root_execution_id=root_execution_id,+ ),}if context.get("native_workspace_version") == 1 or context.get("browser_runtime_version") == 1 or context.get("browser_routing_version") == 1:compact_payload["user_id"] = str(context.get("user_id") or "owner")diff --git a/server/tests/gateway/__init__.py b/server/tests/gateway/__init__.pynew file mode 100644index 00000000..e69de29bdiff --git a/server/tests/gateway/test_attribution.py b/server/tests/gateway/test_attribution.pynew file mode 100644index 00000000..df42cb5b--- /dev/null+++ b/server/tests/gateway/test_attribution.py@@ -0,0 +1,102 @@+"""services/gateway/attribution.py: what a gateway call says about itself,+from the log context, allowlisted ids only."""++import importlib.util+import json+from pathlib import Path++import pytest+import structlog++from services.gateway.attribution import REPLAY_TTL_SECONDS, replay_cache, request_headers+++def test_headers_name_the_chat_run_and_carry_only_allowlisted_ids():+ with structlog.contextvars.bound_contextvars(+ chat_run_id="r_1",+ root_execution_id="root",+ workflow_id="wf",+ user_id="owner",+ node_id="agent-1",+ prompt="a secret the user typed",+ ):+ headers = request_headers()+ assert headers["x-litellm-session-id"] == "r_1"+ assert headers["x-litellm-tags"] == "source:chat,workflow:wf,user:owner"+ assert json.loads(headers["x-litellm-spend-logs-metadata"]) == {+ "chat_run_id": "r_1",+ "root_execution_id": "root",+ "workflow_id": "wf",+ "user_id": "owner",+ "node_id": "agent-1",+ }+ assert "a secret the user typed" not in json.dumps(headers)+++ ("bound", "session"),+ [+ ({"root_execution_id": "root", "execution_id": "exec", "temporal_workflow_id": "t"}, "root"),+ ({"execution_id": "exec", "temporal_workflow_id": "t"}, "exec"),+ ({"temporal_workflow_id": "t"}, "t"),+ ],+)+def test_session_falls_back_from_the_chat_run(bound, session):+ with structlog.contextvars.bound_contextvars(**bound):+ assert request_headers()["x-litellm-session-id"] == session+++def test_outside_any_run_only_the_source_is_sent():+ assert request_headers() == {"x-litellm-tags": "source:direct"}+++def test_replay_cache_needs_a_temporal_activity():+ with structlog.contextvars.bound_contextvars(chat_run_id="r_1"):+ assert replay_cache("sk-gw") is None+++def _namespace(api_key="sk-gw", **ids):+ with structlog.contextvars.bound_contextvars(**ids):+ body = replay_cache(api_key)+ assert body is not None+ assert body["cache"]["use-cache"] is True+ assert body["cache"]["ttl"] == REPLAY_TTL_SECONDS+ return body["cache"]["namespace"]+++def test_replay_namespace_is_stable_across_attempts_and_new_per_run_and_key():+ ids = {"temporal_workflow_id": "wf", "temporal_run_id": "run-1", "temporal_activity_id": "llm-step-1"}+ first = _namespace(**ids, temporal_attempt=1)+ assert first.startswith("oc1-")+ assert _namespace(**ids, temporal_attempt=4) == first+ assert _namespace(**{**ids, "temporal_run_id": "run-2"}) != first+ assert _namespace(**{**ids, "temporal_activity_id": "llm-step-2"}) != first+ assert _namespace("sk-other", **ids) != first+++def test_log_context_restores_the_outer_ids():+ """A node executed inside another (a tool call) must not wipe its+ parent's ids. tests/conftest.py stubs core.logging, so load the file."""+ path = Path(__file__).resolve().parents[2] / "core" / "logging.py"+ spec = importlib.util.spec_from_file_location("_core_logging_under_test", path)+ module = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(module)++ with module.log_context_sync(chat_run_id="r_outer", node_id="agent"):+ with module.log_context_sync(node_id="tool"):+ assert structlog.contextvars.get_contextvars()["node_id"] == "tool"+ assert structlog.contextvars.get_contextvars()["node_id"] == "agent"+ assert structlog.contextvars.get_contextvars()["chat_run_id"] == "r_outer"+ assert "node_id" not in structlog.contextvars.get_contextvars()+++async def test_async_log_context_restores_the_outer_ids():+ path = Path(__file__).resolve().parents[2] / "core" / "logging.py"+ spec = importlib.util.spec_from_file_location("_core_logging_under_test_async", path)+ module = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(module)++ async with module.log_context(execution_id="outer"):+ async with module.log_context(execution_id="inner"):+ assert structlog.contextvars.get_contextvars()["execution_id"] == "inner"+ assert structlog.contextvars.get_contextvars()["execution_id"] == "outer"diff --git a/server/tests/gateway/test_connector.py b/server/tests/gateway/test_connector.pynew file mode 100644index 00000000..354810a7--- /dev/null+++ b/server/tests/gateway/test_connector.py@@ -0,0 +1,111 @@+"""The OpenCompany Gateway's Connectors card: the key probe, the allowance+line, and the card listed only while the gateway is enabled."""++from types import SimpleNamespace+from unittest.mock import AsyncMock, MagicMock++import httpx+import pytest++from services.gateway import credential as credential_module+from services.gateway import handlers as handlers_module+from services.gateway.allowance import parse_user_info++USER_INFO = {+ "user_id": "alice",+ "user_info": {+ "spend": 12.4,+ "max_budget": 50,+ "budget_reset_at": "2026-11-01T00:00:00Z",+ "user_email": "[email protected]",+ },+}+++def _response(status, body=None):+ return httpx.Response(status, json=body or {}, request=httpx.Request("GET", "https://gw.example/user/info"))+++def test_the_allowance_reads_as_one_line():+ allowance = parse_user_info(USER_INFO)+ assert allowance.summary == "$12.40 of $50.00 used · resets Nov 1"+ assert allowance.account == "[email protected]"+ assert parse_user_info({"user_info": {"spend": 3}}).summary == "$3.00 used"+++async def test_the_probe_reads_the_allowance(monkeypatch):+ monkeypatch.setattr(credential_module, "user_info", AsyncMock(return_value=_response(200, USER_INFO)))+ result = await credential_module.OpenCompanyGatewayCredential._probe("sk-gateway")+ assert result.valid is True+ assert result.message == "Connected: $12.40 of $50.00 used · resets Nov 1."+++async def test_the_probe_asks_for_a_default_key(monkeypatch):+ monkeypatch.setattr(credential_module, "user_info", AsyncMock(return_value=_response(403)))+ result = await credential_module.OpenCompanyGatewayCredential._probe("sk-llm-api")+ assert result.valid is False+ assert "type Default" in result.message+++async def test_a_rejected_key_is_left_to_validate_to_word(monkeypatch):+ monkeypatch.setattr(credential_module, "user_info", AsyncMock(return_value=_response(401)))+ with pytest.raises(httpx.HTTPStatusError):+ await credential_module.OpenCompanyGatewayCredential._probe("sk-wrong")+++async def test_status_without_a_stored_key(monkeypatch):+ from core.container import container++ auth = MagicMock()+ auth.get_api_key = AsyncMock(return_value=None)+ monkeypatch.setattr(container, "auth_service", MagicMock(return_value=auth))+ result = await handlers_module.handle_gateway_status({}, MagicMock())+ assert result == {"success": True, "connected": False}+++async def test_status_shows_the_allowance(monkeypatch):+ from core.container import container++ auth = MagicMock()+ auth.get_api_key = AsyncMock(return_value="sk-gateway")+ monkeypatch.setattr(container, "auth_service", MagicMock(return_value=auth))+ monkeypatch.setattr(handlers_module, "user_info", AsyncMock(return_value=_response(200, USER_INFO)))+ result = await handlers_module.handle_gateway_status({}, MagicMock())+ assert result["connected"] is True+ assert result["summary"] == "$12.40 of $50.00 used · resets Nov 1"+ assert result["account_label"] == "[email protected]"+++async def test_status_survives_an_unreachable_gateway(monkeypatch):+ from core.container import container++ auth = MagicMock()+ auth.get_api_key = AsyncMock(return_value="sk-gateway")+ monkeypatch.setattr(container, "auth_service", MagicMock(return_value=auth))+ monkeypatch.setattr(handlers_module, "user_info", AsyncMock(side_effect=httpx.ConnectError("down")))+ result = await handlers_module.handle_gateway_status({}, MagicMock())+ assert result["connected"] is True+ assert "Couldn't read the allowance" in result["summary"]+++def test_the_card_is_listed_only_while_the_gateway_is_enabled(monkeypatch):+ from core.container import container+ from services.credential_registry import CredentialRegistry++ monkeypatch.setattr(container, "settings", MagicMock(return_value=SimpleNamespace(gateway_enabled=True)))+ listed = {provider["id"]: provider for provider in CredentialRegistry().get_all_providers()}+ assert listed["opencompany_gateway"]["kind"] == "apiKey"+ assert listed["opencompany_gateway"]["consumer_category"] == "ai"+ assert listed["opencompany_gateway"]["ws"] == {"status": "gateway_status"}++ monkeypatch.setattr(container, "settings", MagicMock(return_value=SimpleNamespace(gateway_enabled=False)))+ assert "opencompany_gateway" not in {provider["id"] for provider in CredentialRegistry().get_all_providers()}+ # Still resolvable by id, for the connection state of a stored key.+ assert CredentialRegistry().get_provider("opencompany_gateway") is not None+++def test_the_card_is_no_llm_provider():+ """No llm_defaults.json block, so it never shows up in model pickers."""+ from services.llm.config import PROVIDER_CONFIGS++ assert "opencompany_gateway" not in PROVIDER_CONFIGSdiff --git a/server/tests/gateway/test_errors.py b/server/tests/gateway/test_errors.pynew file mode 100644index 00000000..9ae13cd1--- /dev/null+++ b/server/tests/gateway/test_errors.py@@ -0,0 +1,75 @@+"""A spent allowance is BILLING whatever the status, and gateway calls get+gateway wording (services/llm/protocol.py, services/gateway/errors.py)."""++import httpx+import openai+import pytest++from services.gateway.errors import explain+from services.llm.protocol import LLMError, LLMErrorCategory, error_type+++def _gateway_error(status: int, error_type_: str, message: str = "refused") -> openai.APIStatusError:+ """What the OpenAI SDK raises for a LiteLLM error body."""+ body = {"message": message, "type": error_type_, "param": None, "code": str(status)}+ response = httpx.Response(+ status,+ request=httpx.Request("POST", "https://gw.example/v1/chat/completions"),+ json={"error": body},+ )+ return openai.APIStatusError(message, response=response, body=body)++[email protected]("status", [402, 422, 429])+def test_a_spent_allowance_is_billing_whatever_the_status(status):+ """LiteLLM answers 429 before 1.104, 402 with our setting, 422 by default after."""+ error = LLMError.from_exception("openai", _gateway_error(status, "budget_exceeded", "Budget has been exceeded!"))+ assert error.category is LLMErrorCategory.BILLING+ assert error.retryable is False+ assert error.requires_user_action is True+++def test_the_body_type_is_readable_although_provider_code_is_the_status():+ exc = _gateway_error(403, "model_cost_map_missing")+ assert LLMError.from_exception("openai", exc).provider_code == "403"+ assert error_type(exc) == "model_cost_map_missing"+++def test_a_spent_allowance_is_worded_as_the_opencompany_allowance():+ exc = _gateway_error(429, "budget_exceeded")+ error = explain(LLMError.from_exception("openai", exc), exc)+ assert error.user_message == "Your OpenCompany allowance for this month is used up."+ assert "raise it" in error.hint+ node_error = error.as_node_error()+ assert node_error.requires_user_action is True+++def test_an_unpriced_model_says_so():+ exc = _gateway_error(403, "model_cost_map_missing")+ error = explain(LLMError.from_exception("openai", exc), exc)+ assert error.category is LLMErrorCategory.PERMISSION+ assert error.user_message == "This model isn't available on the OpenCompany Gateway yet."+ assert error.hint.startswith("Choose another model")+++def test_a_rejected_gateway_key_points_at_the_gateway_card():+ exc = _gateway_error(401, "auth_error")+ error = explain(LLMError.from_exception("openai", exc), exc)+ assert error.category is LLMErrorCategory.AUTHENTICATION+ assert "OpenCompany Gateway" in error.user_message+ assert "Reconnect the OpenCompany Gateway" in error.hint+++def test_wording_never_changes_retry_behaviour():+ exc = _gateway_error(503, "service_unavailable")+ error = LLMError.from_exception("openai", exc)+ before = (error.category, error.retryable)+ explain(error, exc)+ assert (error.category, error.retryable) == before+ assert error.user_message == "The OpenCompany Gateway is temporarily unavailable."+++def test_public_hint_replaces_the_category_hint():+ error = LLMError(message="x", provider="openai", category=LLMErrorCategory.BILLING, public_hint="custom")+ assert error.hint == "custom"+ assert LLMError(message="x", provider="openai", category=LLMErrorCategory.BILLING).hint.startswith("Review the provider")diff --git a/server/tests/gateway/test_routing.py b/server/tests/gateway/test_routing.pynew file mode 100644index 00000000..c27115e7--- /dev/null+++ b/server/tests/gateway/test_routing.py@@ -0,0 +1,115 @@+"""services/gateway/routing.py: own keys first; the gateway serves only the+providers llm_defaults.json marks, and only when enabled and connected."""++from types import SimpleNamespace+from unittest.mock import AsyncMock, MagicMock++import pytest++from services.gateway.routing import GATEWAY_CREDENTIAL_ID, GatewayRoute, GatewayRouter, gateway_covers++DEFAULTS = {+ "providers": {+ "openai": {"gateway": {"model_prefix": "openai/", "base_path": "/v1"}},+ "anthropic": {"gateway": {"model_prefix": "", "base_path": ""}},+ "deepseek": {"default_model": "deepseek-v4.1-flash"},+ }+}+++def _router(*, enabled=True, distributed=False, key="sk-gateway", defaults=DEFAULTS):+ settings = SimpleNamespace(gateway_enabled=enabled, distributed_mode=distributed, gateway_url="https://gw.example/")+ auth = MagicMock()+ auth.get_api_key = AsyncMock(return_value=key)+ return GatewayRouter(settings=settings, auth_service=auth, defaults=defaults), auth+++async def test_a_served_provider_gets_the_gateway_route():+ router, auth = _router()+ assert await router.route("openai") == GatewayRoute(+ api_key="sk-gateway", base_url="https://gw.example/v1", model_prefix="openai/"+ )+ auth.get_api_key.assert_awaited_with(GATEWAY_CREDENTIAL_ID)+++async def test_native_routes_use_the_gateway_root_and_bare_ids():+ router, _ = _router()+ route = await router.route("anthropic")+ assert route is not None+ assert (route.base_url, route.model_prefix) == ("https://gw.example", "")++[email protected]("provider", ["deepseek", "openai_compatible:lab", "unknown"])+async def test_a_provider_without_a_gateway_block_is_never_routed(provider):+ router, _ = _router()+ assert await router.route(provider) is None+++async def test_no_route_without_a_stored_gateway_key():+ router, _ = _router(key=None)+ assert await router.route("openai") is None+ assert await router.covers("openai") is False++[email protected]("flag", [False, MagicMock(), "true", 1])+async def test_only_a_true_flag_turns_the_gateway_on(flag):+ router, auth = _router(enabled=flag)+ assert await router.route("openai") is None+ auth.get_api_key.assert_not_awaited()+++async def test_distributed_mode_turns_the_gateway_off():+ router, _ = _router(distributed=True)+ assert await router.route("openai") is None+++def test_served_providers_come_from_the_gateway_blocks():+ router, _ = _router()+ assert router.served_providers() == ["openai", "anthropic"]+++def test_llm_defaults_serves_exactly_the_priced_providers():+ """OpenRouter, DeepSeek and Sarvam stay off until their default models+ are priced upstream (gateway/tests/test_pins.py checks the prices)."""+ from services.llm.config import LLM_DEFAULTS++ router, _ = _router(defaults=LLM_DEFAULTS)+ assert set(router.served_providers()) == {+ "openai", "anthropic", "gemini", "groq", "cerebras", "xai", "kimi", "mistral",+ }+++async def test_gateway_covers_trusts_only_a_real_router():+ # tests/conftest.py stubs core.container with a MagicMock, whose router+ # would claim to cover every provider.+ assert await gateway_covers("openai") is False+++async def test_activity_key_lookup_returns_no_key_when_the_gateway_covers(monkeypatch):+ """The gateway key never reaches activity code or a Temporal payload:+ the activity passes no key and ChatUnifier routes the call."""+ from core.container import container+ from services.temporal import agent_activities++ auth = MagicMock(distributed_credentials=False)+ auth.resolve_api_key = AsyncMock(return_value=None)+ monkeypatch.setattr(container, "auth_service", MagicMock(return_value=auth))+ monkeypatch.setattr("services.gateway.routing.gateway_covers", AsyncMock(return_value=True))++ assert await agent_activities._resolve_activity_api_key({"provider": "openai", "user_id": "owner"}) == ""+++async def test_activity_key_lookup_still_refuses_when_nothing_covers(monkeypatch):+ from temporalio.exceptions import ApplicationError++ from core.container import container+ from services.temporal import agent_activities++ auth = MagicMock(distributed_credentials=False)+ auth.resolve_api_key = AsyncMock(return_value=None)+ monkeypatch.setattr(container, "auth_service", MagicMock(return_value=auth))+ monkeypatch.setattr("services.gateway.routing.gateway_covers", AsyncMock(return_value=False))++ with pytest.raises(ApplicationError) as raised:+ await agent_activities._resolve_activity_api_key({"provider": "openai", "user_id": "owner"})+ assert raised.value.type == "MissingAgentProviderCredential"diff --git a/server/tests/gateway/test_unifier_route.py b/server/tests/gateway/test_unifier_route.pynew file mode 100644index 00000000..e291eaac--- /dev/null+++ b/server/tests/gateway/test_unifier_route.py@@ -0,0 +1,145 @@+"""ChatUnifier sends a call with no key of the install's own to the gateway+route, and never asks the gateway when there is one."""++from unittest.mock import AsyncMock, MagicMock++import httpx+import openai+import pytest+import structlog++from services.gateway.routing import GatewayRoute+from services.llm import registry+from services.llm.protocol import LLMError, LLMErrorCategory, LLMResponse, Message+from services.llm.registry import ProviderSpec+from services.llm.unifier import ChatUnifier+from services.plugin import NodeUserError++ROUTE = GatewayRoute(api_key="sk-gateway", base_url="https://gw.example/v1", model_prefix="groq/")+HI = [Message(role="user", content="hi")]+++class _Router:+ def __init__(self, route):+ self._route = route+ self.asked = []++ async def route(self, provider):+ self.asked.append(provider)+ return self._route+++def _provider(monkeypatch, name, *, chat=None):+ """A fake provider in the registry for this test only (monkeypatch removes it)."""+ created = []++ class Client:+ def __init__(self, **kwargs):+ self.kwargs = kwargs+ self.chat = chat or AsyncMock(return_value=LLMResponse(content="ok"))+ created.append(self)++ monkeypatch.setitem(+ registry._REGISTRY, name, ProviderSpec(name=name, factory=Client, sdk_exception_refs=("openai:OpenAIError",))+ )+ return created+++def _unifier(router):+ auth = MagicMock()+ auth.resolve_api_key = AsyncMock(return_value=None)+ return ChatUnifier(defaults={"providers": {}}, auth_service=auth, gateway=router), auth+++async def test_a_call_with_no_own_key_goes_to_the_gateway(monkeypatch):+ created = _provider(monkeypatch, "gateway-route-test")+ router = _Router(ROUTE)+ unifier, auth = _unifier(router)++ with structlog.contextvars.bound_contextvars(chat_run_id="r_1", workflow_id="wf"):+ await unifier.chat(provider="gateway-route-test", api_key="", messages=HI, model="llama-3.3-70b")++ client = created[0]+ assert client.kwargs["api_key"] == "sk-gateway"+ assert client.kwargs["proxy_url"] == "https://gw.example/v1"+ assert client.kwargs["model_prefix"] == "groq/"+ sent = client.chat.call_args.kwargs+ # The provider adds the prefix on the wire; capability lookups keep bare ids.+ assert sent["model"] == "llama-3.3-70b"+ assert sent["extra_headers"]["x-litellm-session-id"] == "r_1"+ # Outside a Temporal activity there is nothing to replay.+ assert "extra_body" not in sent+ # A gateway call never reads a saved {provider}_proxy base URL.+ auth.resolve_api_key.assert_not_awaited()+++async def test_an_own_key_is_used_as_before_and_the_gateway_is_not_asked(monkeypatch):+ created = _provider(monkeypatch, "gateway-own-key-test")+ router = _Router(ROUTE)+ unifier, _ = _unifier(router)++ await unifier.chat(provider="gateway-own-key-test", api_key="sk-own", messages=HI, model="m")++ assert router.asked == []+ assert created[0].kwargs["api_key"] == "sk-own"+ assert "model_prefix" not in created[0].kwargs+ assert "extra_headers" not in created[0].chat.call_args.kwargs+++async def test_inside_an_activity_the_call_opts_into_replay(monkeypatch):+ created = _provider(monkeypatch, "gateway-replay-test")+ unifier, _ = _unifier(_Router(ROUTE))++ with structlog.contextvars.bound_contextvars(+ temporal_workflow_id="wf", temporal_run_id="run", temporal_activity_id="llm-step-1"+ ):+ await unifier.chat(provider="gateway-replay-test", api_key="", messages=HI, model="m")++ cache = created[0].chat.call_args.kwargs["extra_body"]["cache"]+ assert cache["use-cache"] is True and cache["namespace"].startswith("oc1-")+++async def test_no_own_key_and_no_gateway_still_refuses(monkeypatch):+ _provider(monkeypatch, "gateway-none-test")+ unifier, _ = _unifier(_Router(None))+ with pytest.raises(NodeUserError, match="No API key is configured"):+ await unifier.chat(provider="gateway-none-test", api_key="", messages=HI, model="m")+++async def test_a_gateway_failure_gets_gateway_wording(monkeypatch):+ body = {"message": "Budget has been exceeded!", "type": "budget_exceeded", "param": None, "code": "429"}+ response = httpx.Response(429, request=httpx.Request("POST", "https://gw.example/v1/chat/completions"), json={"error": body})+ refused = openai.RateLimitError("Budget has been exceeded!", response=response, body=body)+ _provider(monkeypatch, "gateway-failure-test", chat=AsyncMock(side_effect=refused))+ unifier, _ = _unifier(_Router(ROUTE))++ with pytest.raises(LLMError) as raised:+ await unifier.chat(+ provider="gateway-failure-test", api_key="", messages=HI, model="m", translate_errors=False+ )+ assert raised.value.category is LLMErrorCategory.BILLING+ assert raised.value.retryable is False+ assert raised.value.user_message == "Your OpenCompany allowance for this month is used up."+++async def test_openai_provider_prefixes_the_wire_model_only():+ from services.llm.providers.openai import OpenAIProvider++ provider = OpenAIProvider("sk-gateway", proxy_url="https://gw.example/v1", model_prefix="openai/")+ resp = MagicMock()+ resp.choices = [MagicMock(message=MagicMock(content="ok", reasoning_content=None, tool_calls=None), finish_reason="stop")]+ resp.usage = MagicMock(prompt_tokens=1, completion_tokens=1, total_tokens=2, completion_tokens_details=None)+ provider._client.chat.completions.create = AsyncMock(return_value=resp)++ result = await provider.chat(+ HI,+ model="gpt-4.1-mini",+ extra_headers={"x-litellm-session-id": "s"},+ extra_body={"cache": {"use-cache": True}},+ )++ sent = provider._client.chat.completions.create.call_args.kwargs+ assert sent["model"] == "openai/gpt-4.1-mini"+ assert sent["extra_headers"] == {"x-litellm-session-id": "s"}+ assert sent["extra_body"] == {"cache": {"use-cache": True}}+ assert result.model == "gpt-4.1-mini"