Skip to content

Self-improving AI Employees for everyone

feat(gateway): run the RLM agent and Claude Code on the OpenCompany Gateway

trohitg197c4572e93e
diff --git a/CLAUDE.md b/CLAUDE.md
index 8582b354..118ce180 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -51,7 +51,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire
| **[Translate / Transliterate / Detect](./docs-internal/speech_provider_rfc.md#8-the-pattern-generalised-nodestranslate)** | The second application of the multi-vendor pattern, and the one that retired `nodes/sarvam/`. Three nodes (`translateText`, `transliterateText`, `detectLanguage`) in [`server/nodes/translate/`](./server/nodes/translate/), backed by **three registries — one per capability**, because the asymmetry is sharper than speech's: DeepL translates and does nothing else, while Sarvam and LLM-backed providers do all three. Membership makes an unsupported selection unrepresentable rather than a runtime failure inside a paid call. Providers: **deepl** (`Authorization: DeepL-Auth-Key`, `text` is an array, free keys ending `:fx` route to a different host, and it returns `billed_characters` so cost is the provider's own figure), **sarvam** (all three over dedicated REST), **openai** (all three via `ChatUnifier` — proves the Protocol does not care what a provider does internally; billing deliberately NOT recorded because that path bills tokens the LLM layer already costs). Capabilities in `server/config/translate_defaults.json`. |
| **[Desktop App](./docs-internal/desktop_app.md)** | Electron shell in [`desktop/`](./desktop/) — a standalone bun package (NOT a root workspace member; `cli/tests/test_release_pipeline_config.py` locks `workspaces`). Bundles pinned `uv`, python-build-standalone CPython 3.12 and bun 1.4.0 — the only JavaScript runtime shipped; no Node, no npm (`desktop/runtimes.json`, verified against upstream checksum manifests by `scripts/fetch-runtimes.ts`, which also fetches bun's `LICENSE.md` from the bun repo — the release zip carries none — and prunes stale runtime dirs such as a leftover `node/`), stages the backend sibling layout from `bun pm pack --dry-run` (`scripts/stage.ts`, which runs the sidecar package's own `bun run build` — `bun build --target=bun`, express inlined — and copies `dist/index.js`), sets `OPENCOMPANY_BUN_BIN` / `BUN_INSTALL=<userData>/bun` / `BUN_INSTALL_CACHE_DIR` and prepends the bundled bun dir (then uv) to PATH for the backend (`env.ts`; `OPENCOMPANY_NODE_BIN` no longer exists), provisions the venv into Electron `userData` on first launch with `uv sync --frozen --python <bundled>` (stamp = sha256 of uv.lock + pyproject + Python + uv versions; falls back to `uv python install 3.12`), pre-warms `PYTHONPYCACHEPREFIX` (with a prefix set Python ignores in-tree .pyc — 32 s vs 3 s first boot), spawns uvicorn on `127.0.0.1:5678` (persisted port; attaches to a running `company serve` instead of double-spawning), waits on `/health/ready`, loads the backend-served SPA (zero frontend changes), routes `window.open` / `_blank` to the OS browser. Quit = `POST /api/desktop/shutdown` + stdin EOF, 30 s, then tree-kill. electron-builder: NSIS x64, DMG+zip arm64/x64, AppImage+deb x64, unsigned first (`mac.identity: null`; macOS auto-update notify-only until signed — Squirrel.Mac refuses unsigned). CI: `desktop-release.yml` (tag-triggered: `prepare` drafts the GitHub Release from the annotated tag's message, three legs build and upload version-less OS-named installers such as `OpenCompany-windows-x64.exe` / `OpenCompany-macos-arm64.dmg` / `OpenCompany-linux-x86_64.AppImage` so the README can link `releases/latest/download/<file>`, each leg refuses a desktop version that differs from the tag, `finalize` undrafts) and `desktop-ci.yml` (typecheck/unit/invariants/Playwright smoke). **Gotcha: editor-hosted terminals export `ELECTRON_RUN_AS_NODE=1`, which makes Electron boot as plain Node (`electron.app` undefined) — strip it before launching; the e2e test does.** |
| **[Desktop Host Contract](./docs-internal/desktop_host_contract.md)** | The backend's side of being owned by a GUI shell. `core/approot.py` is the ONE module that locates the app tree (`OPENCOMPANY_APP_ROOT` / `OPENCOMPANY_CLIENT_DIST` / `OPENCOMPANY_ENV_FILE` / `OPENCOMPANY_ENV_TEMPLATE`); `tests/test_no_root_climb.py` forbids any other `Path(__file__).parents[N]` climb above `server/` and any `"client"/"dist"` / `.env.template` / `package.json` path join. `core/env_defaults.apply_file_defaults_to_environ()` does the CLI's `.env.template < .env < process env` layering at the top of `main.py`, and `Settings.env_file` is the absolute `(template, .env)` tuple — so a shell spawns `python -m uvicorn` from any cwd without the CLI package. Desktop mode (`OPENCOMPANY_DESKTOP=1`, `core/desktop.py`): parent-PID watchdog with create-time PID-reuse guard, stdin-EOF watchdog (polling via `PeekNamedPipe`/`select`, never a blocking read — a daemon thread parked in `ReadFile` at finalisation crashed exit with 0xC0000005), ctypes Job Object so children die with the backend (no pywin32 in the server venv), token-gated `POST /api/desktop/shutdown` (`routers/desktop.py`, public path; raises the signal uvicorn handles so the lifespan reaps Temporal/the bun sidecar/edgymeow), 45 s hard deadline. `GET /health/ready` (public) = DB up AND (Temporal disabled OR worker manager started), with `phase` written by `services/temporal/lifecycle.py` (`installing_temporal` / `starting_temporal` / `connecting` / `starting_workers` / `ready`). Plugin shutdown hooks now run under a 10 s per-hook timeout (`HOOK_TIMEOUT_SECONDS`), and the current browser hook uses `peek_browser_runtime()` so shutdown never constructs or installs the runtime. Historical incident: the retired `get_browser_service()` lazily installed agent-browser at shutdown and wedged graceful exit (masked under the CLI by its 5 s grace + tree-kill). `server/uv.lock` is committed; `predeploy.yml` runs `uv lock --check`. |
-| **[LLM Gateway](./docs-internal/llm_gateway.md)** | Hosted LiteLLM Proxy in `gateway/` (official AWS module, mock-model local stack, contract and pin tests, `gateway-ci.yml`): pinned LiteLLM 1.103.4 and price list, served providers, and the contract OpenCompany relies on (wire model names, `budget_exceeded` refusals, `/user/info` allowance with Default keys, attribution headers, cache replay). Operator runbook: `gateway/README.md`. OpenCompany's side, `server/services/gateway/`: the Connectors card (`OPENCOMPANY_GATEWAY_ENABLED`), own-keys-first routing in ChatUnifier from `"gateway"` blocks in `llm_defaults.json`, and for embeddings, OpenAI speech (`"gateway": true` in `speech_defaults.json`) and vision through `key_or_route`; attribution from the log context, gateway error wording. |
+| **[LLM Gateway](./docs-internal/llm_gateway.md)** | Hosted LiteLLM Proxy in `gateway/` (official AWS module, mock-model local stack, contract and pin tests, `gateway-ci.yml`): pinned LiteLLM 1.103.4 and price list, served providers, and the contract OpenCompany relies on (wire model names, `budget_exceeded` refusals, `/user/info` allowance with Default keys, attribution headers, cache replay). Operator runbook: `gateway/README.md`. OpenCompany's side, `server/services/gateway/`: the Connectors card (`OPENCOMPANY_GATEWAY_ENABLED`), own-keys-first routing in ChatUnifier from `"gateway"` blocks in `llm_defaults.json`, and for embeddings, OpenAI speech (`"gateway": true` in `speech_defaults.json`) and vision through `key_or_route`; the RLM agent on OpenAI-format routes (`openai_prefix`); Claude Code after its login and the install's Anthropic key (`api_access`, no warm reuse); attribution from the log context, gateway error wording. |
| **[CI/CD Pipeline](./docs-internal/ci_cd.md)** | GitHub Actions workflows, predeploy validation, release publishing, and the composite setup action (bun via SHA-pinned `oven-sh/setup-bun` reading the root `packageManager` pin + Node 22 as a CI-only convenience so bun runs vite / vitest / eslint on it via their node shebangs + Python 3.12 + uv; `bun install --frozen-lockfile` everywhere; publish is `bun publish --access public` after a `bun pm whoami` credential preflight with `~/.npmrc` written from `NPM_TOKEN` — no npm CLI, no `setup-node` step of their own in the publish jobs (they get Node through the composite action), and npm provenance attestation went away with npm). Also **Cutting a release**: the annotated tag is the version and its message is the release notes; `company version sync vX.Y.Z` bumps every version file for the `chore(release)` commit the tag sits on, and `desktop-release.yml` drafts the GitHub Release from the tag message and refuses a version that differs from the tag |
| **[Performance](./docs-internal/performance.md)** | Cold-start measurements (Application startup complete 2.90 s warm, 21.5 s cold post-`company clean` — was 71 s before the 2026-07-14 boot-delay fixes; first WS connect 8.29 s) + warm and cold per-phase timelines + optimisation history (lazy LangChain imports, lazy LLM-SDK exception refs, pre-bundled sidecar (esbuild at the time, `bun build --target=bun` today), bytecode-at-build, PartySocket reconnect, TanStack Query auth bootstrap, `manualChunks`, Vite dep-cache preservation, off-loop Temporal build-id hash) + bottleneck inventory + reproduction commands + anti-patterns to never reintroduce (incl. eager SDK import at LLM provider registration, unconditional `.vite` wipe, synchronous `Worker()` on the event loop). Pairs with the build-time pipeline doc below. |
| **[Release Build Pipeline](./docs-internal/release_build_pipeline.md)** | Build pipeline for the registry tarball (installed with `bun add -g`; bun is the only JavaScript runtime and package manager anything shipped needs): `typescript@7` (native Go compiler, exact-pinned in **root** devDependencies — client keeps `typescript@^5.x` because typescript-eslint's peer range excludes 6/7, and one manifest cannot declare `typescript` twice) for type-check, measured 5.9× faster than tsc 5.9.3 (820 ms vs 4830 ms); `bun run --filter react-flow-client typecheck` delegates up to the root gate, Vite `manualChunks` + `target: 'es2022'`, `bun build --target=bun` sidecar bundle (self-contained `dist/index.js` with express inlined, run as `bun dist/index.js`; drops the `tsx` interpreter cost and needs no `node_modules` at runtime), bytecode pre-compile in two halves — `[tool.uv] compile-bytecode = true` (uv sync compiles `.venv/` site-packages) + `python -m compileall` for project source (plain `.pyc`, **no `-O`** — per PEP 488 the non-`-O` runtimes never load `.opt-1.pyc`; excludes `.venv/`, `tests/`). Single source of truth for the bytecode-compile path list: `cli.commands.build.COMPILEALL_SOURCE_DIRS`. Dev PM is **[email protected]** (tracked text `bun.lock`, `bunfig.toml` pins the isolated linker, ranged security pins in top-level `overrides`); `company build` step [1/6] is `bun install`. |
diff --git a/docs-internal/llm_gateway.md b/docs-internal/llm_gateway.md
index b9bf93c2..2ac810f8 100644
--- a/docs-internal/llm_gateway.md
+++ b/docs-internal/llm_gateway.md
@@ -2,7 +2,7 @@
OpenCompany's hosted LLM gateway is the stock open-source LiteLLM Proxy, deployed from `gateway/` with BerriAI's official Terraform module. Invited people call LLMs on platform-owned vendor keys, each with a monthly allowance, and spend is recorded per person and per session. The repository holds only configuration and tests; there is no custom gateway code. The operator runbook (deployment, adding people, upgrades) is [gateway/README.md](../gateway/README.md).
-**Status:** the gateway is in `gateway/`; OpenCompany's side is `server/services/gateway/` and covers chat through ChatUnifier (agents, chat runs, employee setup, translate), OpenAI embeddings (long-term memory, the Embedding Generator), OpenAI speech (Text to Speech, Speech to Text, chat dictation) and the vision tool. RLM and Claude Code still call vendors directly. `gateway/tests/test_gateway_contract.py` pins the contract below against the local stack; `server/tests/gateway/` covers OpenCompany's side.
+**Status:** the gateway is in `gateway/`; OpenCompany's side is `server/services/gateway/` and covers chat through ChatUnifier (agents, chat runs, employee setup, translate), OpenAI embeddings (long-term memory, the Embedding Generator), OpenAI speech (Text to Speech, Speech to Text, chat dictation), the vision tool, the RLM agent and Claude Code. Codex and the Gemini CLI stay on their own logins. `gateway/tests/test_gateway_contract.py` pins the contract below against the local stack; `server/tests/gateway/` covers OpenCompany's side.
## OpenCompany's side
@@ -19,6 +19,16 @@ OpenCompany's hosted LLM gateway is the stock open-source LiteLLM Proxy, deploye
- **Vision** (`nodes/vision/vision_analyze`): routes all three SDK clients. The node now declares its OpenAI, Anthropic and Gemini credentials. It never did, so every real run failed at `ctx.connection`.
- **Error wording.** Failures of these calls get gateway wording from `describe` (`services/gateway/errors.py`), which reads LiteLLM's error type from an SDK or `httpx` error.
+**The RLM agent** (`services/rlm/`) uses the rlms library, which reaches a custom URL only in OpenAI's format.
+- **Route:** with no own key, `BackendAdapter` gives it the gateway's `/v1` route (`gateway_route(provider, openai_format=True)`) for every provider. The model goes under LiteLLM's name: the block's `model_prefix`, or for Anthropic and Gemini, which OpenCompany otherwise calls on native routes, the block's `openai_prefix`. Its client sends the run's attribution headers as `default_headers`.
+- **Scope:** the same applies to a connected small model. The supported providers are unchanged; there is no replay cache.
+
+**Claude Code** (`nodes/agent/claude_code_agent/_oauth.py`, `api_access`).
+- **When the gateway serves Anthropic:** own access still comes first: the CLI's login (`claude auth status`), then the install's Anthropic key (`ANTHROPIC_API_KEY`), then the gateway. On the gateway, the spawn gets `ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN` and the run's attribution headers in `ANTHROPIC_CUSTOM_HEADERS`.
+- **Without the gateway,** or with API settings in the server's own environment, nothing changes.
+- **No reuse:** a gateway process carries its run's headers, so it is terminated after its turn instead of staying warm. The next turn resumes the conversation from the memory node's session id, as after an idle eviction.
+- **Credential tool:** the MCP `getCredential` tool never returns the gateway key, allowlisted or not.
+
**Attribution** (`services/gateway/attribution.py`). Ids come from the structlog log context: `BaseNode.execute` binds the workflow, execution, root execution, chat run, employee job and user ids; the Temporal activity interceptor binds `temporal_workflow_id`, `temporal_run_id`, `temporal_activity_id` and `temporal_attempt`; the LLM-step and compaction activities bind the `attribution` dict their payload carries (`_attribution_ids` in `agent_workflow.py`). `log_context` restores outer values on exit, so a tool node run inside an agent keeps the agent's ids. Headers: session = chat run, else root execution, else execution, else Temporal workflow; tags `source:`, `workflow:`, `user:`; every allowlisted id as metadata. Inside a Temporal activity, calls send a replay-cache namespace from the key's fingerprint and the Temporal workflow, run and activity ids (not the attempt), so a retried activity replays at $0 and a re-run never replays another run's answer. Gemini's native route is the exception: LiteLLM 1.103.4 records its session id and tags but drops the metadata header, and ignores the cache body, so OpenCompany sends no cache body there and a retried Gemini step is billed again.
**Errors** (`services/gateway/errors.py`, `services/llm/protocol.py`). `_classify_error` maps the `budget_exceeded` type to BILLING before any status check, so the run fails once with no retry loop, and a deployment-spawned run pauses its deployment. Failures of gateway calls get gateway wording through `public_message` and the new `public_hint` (allowance used up; model not on the gateway, from `model_cost_map_missing`; reconnect the gateway key). Compaction inside the agent workflow raises such failures instead of retrying them (`stop_on_user_action`).
@@ -56,7 +66,7 @@ Not served: OpenRouter, DeepSeek and Sarvam (no price for their default models),
| Behaviour | Detail | Test |
|---|---|---|
-| Wire model names | OpenAI-format calls use `<litellm prefix>/<model>` on `/v1` (`openai/`, `groq/`, `cerebras/`, `xai/`, `moonshot/`, `mistral/`). Anthropic SDK calls send bare `claude-*` ids to `/v1/messages`; Gemini SDK calls use the native `/v1beta/models/<id>:generateContent` route with the key in `x-goog-api-key` | `test_anthropic_messages_route_accepts_bare_model_ids`, `test_gemini_native_route_accepts_virtual_key` |
+| Wire model names | OpenAI-format calls use `<litellm prefix>/<model>` on `/v1` (`openai/`, `groq/`, `cerebras/`, `xai/`, `moonshot/`, `mistral/`). Anthropic SDK calls and Claude Code send bare `claude-*` ids to `/v1/messages`; Gemini SDK calls use the native `/v1beta/models/<id>:generateContent` route with the key in `x-goog-api-key`. Both are also served in OpenAI's format as `anthropic/` and `gemini/` (the RLM agent) | `test_anthropic_messages_route_accepts_bare_model_ids`, `test_gemini_native_route_accepts_virtual_key`, `test_native_providers_answer_in_openai_format` |
| Spent allowance | Refused with error type `budget_exceeded` on all three routes; HTTP 429 on 1.103.x, 402 from 1.104. Recognise it by type | `test_budget_refusal_is_typed_on_every_route` |
| Unpriced model | HTTP 403, error type `model_cost_map_missing` | `test_unpriced_model_is_refused` |
| Allowance | `GET /user/info` with the person's key returns `user_info.spend`, `max_budget` and `budget_reset_at`. Needs a Default-type key; LLM-API keys are refused; another user's id is refused | `test_user_info_needs_a_default_key_and_stays_scoped` |
diff --git a/gateway/README.md b/gateway/README.md
index ece4e5f9..8a65d438 100644
--- a/gateway/README.md
+++ b/gateway/README.md
@@ -116,4 +116,5 @@ Why 1.103.x: it is the only stable line with the streamed-alias billing fix (Lit
- `gpt-4o-mini-tts` (OpenCompany's Text to Speech default) carries a price override in `litellm/config.yaml`. The speech route returns no usage for the audio, so LiteLLM would otherwise bill the input text only, about 1% of the cost. The override's per-token rate assumes about 200 text tokens per minute of speech; revisit it if OpenAI's price changes.
- Groq speech is not served: 1.103.4 bills Groq transcription at $0. The `groq/*` wildcard still lets a person call Groq Whisper directly with their key, billed $0 to them; Groq charges about $0.11 per hour of audio.
- A Default key can create and delete that person's own keys, but not raise their allowance. Revoke a leaked key in the console.
+- Claude Code on the gateway holds the install's gateway key in its process environment, where its own tools (Bash) can read it, as with any API key a CLI agent runs on. The allowance caps what a leaked key can spend.
- The open-source tier has no SSO beyond 5 users, no audit log, no automatic key rotation, and no route permissions for tool pass-through. Those need a LiteLLM Enterprise license.
diff --git a/gateway/local/config.local.yaml b/gateway/local/config.local.yaml
index 493f4a5e..76fb3aea 100644
--- a/gateway/local/config.local.yaml
+++ b/gateway/local/config.local.yaml
@@ -25,6 +25,18 @@ model_list:
model: gemini/gemini-*
api_key: local-unused
mock_response: pong
+ # OpenAI-format names for the providers OpenCompany calls on native routes
+ # (the RLM agent's library speaks only OpenAI's format).
+ - model_name: anthropic/*
+ litellm_params:
+ model: anthropic/*
+ api_key: sk-local-unused
+ mock_response: pong
+ - model_name: gemini/*
+ litellm_params:
+ model: gemini/*
+ api_key: local-unused
+ mock_response: pong
# Real upstreams (fake_upstream.py) for what mocks can't show: spend rows on
# /v1/messages, the native Gemini route and the audio routes, and streamed
# pricing. Exact names win over the wildcards above.
diff --git a/gateway/tests/test_gateway_contract.py b/gateway/tests/test_gateway_contract.py
index 6b766bff..fe501676 100644
--- a/gateway/tests/test_gateway_contract.py
+++ b/gateway/tests/test_gateway_contract.py
@@ -368,6 +368,14 @@ def test_gemini_native_route_writes_a_priced_row_for_the_session(person: Callabl
assert "source:chat" in (rows[0].get("request_tags") or []), rows[0]
[email protected]("model", ["anthropic/claude-haiku-4-5", "gemini/gemini-2.5-flash"])
+def test_native_providers_answer_in_openai_format(person: Callable[..., Person], model: str) -> None:
+ """The RLM agent's library reaches every provider on /v1/chat/completions, under the
+ "openai_prefix" of its gateway block in server/config/llm_defaults.json."""
+ resp = _chat(person().key, body={"model": model})
+ assert resp.status_code == 200, resp.text
+
+
def test_embeddings_route_writes_an_attributed_row(person: Callable[..., Person], admin: httpx.Client) -> None:
"""OpenAI embeddings (long-term memory, the Embedding Generator) bill the person with attribution."""
session = f"r_{uuid.uuid4().hex}"
diff --git a/server/config/llm_defaults.json b/server/config/llm_defaults.json
index b0d017b6..d6d911b4 100644
--- a/server/config/llm_defaults.json
+++ b/server/config/llm_defaults.json
@@ -87,7 +87,7 @@
},
"anthropic": {
"display_name": "Anthropic",
- "gateway": {"model_prefix": "", "base_path": ""},
+ "gateway": {"model_prefix": "", "base_path": "", "openai_prefix": "anthropic/"},
"streaming": true,
"default_model": "claude-opus-5-5",
"popular_models": ["claude-opus-5-5", "claude-fable-5-1", "claude-fable-5", "claude-opus-5", "claude-sonnet-5-5", "claude-sonnet-5", "claude-opus-4-8", "claude-opus-4-7", "claude-sonnet-4-6"],
@@ -159,7 +159,7 @@
},
"gemini": {
"display_name": "Gemini",
- "gateway": {"model_prefix": "", "base_path": ""},
+ "gateway": {"model_prefix": "", "base_path": "", "openai_prefix": "gemini/"},
"default_model": "gemini-3.8-flash",
"popular_models": ["gemini-3.8-flash", "gemini-3.7-flash", "gemini-3.6-flash", "gemini-3.5-flash", "gemini-3.5-flash-lite", "gemini-3.1-pro-preview", "gemini-3.1-flash-lite", "gemini-3-flash-preview", "gemini-2.5-pro", "gemini-2.5-flash", "gemini-2.5-flash-lite"],
"detection_patterns": ["gemini", "google", "gemma"],
diff --git a/server/nodes/agent/claude_code_agent/_oauth.py b/server/nodes/agent/claude_code_agent/_oauth.py
index fb1ec656..240e09d6 100644
--- a/server/nodes/agent/claude_code_agent/_oauth.py
+++ b/server/nodes/agent/claude_code_agent/_oauth.py
@@ -38,8 +38,9 @@ from __future__ import annotations
import asyncio
import json
import os
+from dataclasses import dataclass, field
from services.process_environment import without_onepassword_environment
-from typing import Any, Dict
+from typing import Any, Dict, Mapping
from core.js_runtime import add_package, shared_tree_bin
from core.logging import get_logger
@@ -211,3 +212,53 @@ async def claude_auth_logout() -> bool:
"""Run ``claude auth logout``. True on exit 0."""
envelope = await _run_auth("logout", timeout=ONESHOT_TIMEOUT_SECONDS)
return bool(envelope.get("success"))
+
+
+#: How a claude process reaches the API other than through its login. Set in
+#: the server's own environment, they are the install's choice: left alone.
+_API_ENV_VARIABLES = ("ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_BASE_URL")
+
+
+@dataclass(frozen=True)
+class ApiAccess:
+ """What a spawned claude needs, beyond its login, to reach the API."""
+
+ env: Dict[str, str] = field(default_factory=dict)
+ #: The env carries this run's attribution headers, so the process must
+ #: not serve another run (no warm reuse).
+ per_run: bool = False
+
+
+async def api_access(base_env: Mapping[str, str]) -> ApiAccess:
+ """Where the OpenCompany Gateway serves Anthropic, own access still comes
+ first: the CLI's login (``claude auth status``: a Claude subscription or
+ Console account), then the install's Anthropic key, then the gateway.
+ Without the gateway, claude uses its login as always.
+
+ On the gateway, claude calls the gateway's Anthropic route with the
+ gateway key (``ANTHROPIC_BASE_URL``, ``ANTHROPIC_AUTH_TOKEN``) and sends
+ the run's attribution headers (``ANTHROPIC_CUSTOM_HEADERS``).
+ """
+ if any(base_env.get(name) for name in _API_ENV_VARIABLES):
+ return ApiAccess()
+ from services.gateway.routing import gateway_route
+
+ route = await gateway_route("anthropic")
+ if route is None or await claude_auth_status():
+ return ApiAccess()
+ from core.container import container
+
+ own_key = await container.auth_service().resolve_api_key("anthropic")
+ if own_key:
+ return ApiAccess(env={"ANTHROPIC_API_KEY": own_key})
+ from services.gateway.attribution import request_headers
+
+ headers = "\n".join(f"{name}: {value}" for name, value in request_headers().items())
+ return ApiAccess(
+ env={
+ "ANTHROPIC_BASE_URL": route.base_url,
+ "ANTHROPIC_AUTH_TOKEN": route.api_key,
+ "ANTHROPIC_CUSTOM_HEADERS": headers,
+ },
+ per_run=True,
+ )
diff --git a/server/services/cli_agent/mcp_server.py b/server/services/cli_agent/mcp_server.py
index bebbc873..680c06f6 100644
--- a/server/services/cli_agent/mcp_server.py
+++ b/server/services/cli_agent/mcp_server.py
@@ -528,6 +528,16 @@ def _build_tools(mcp: Any) -> None: # FastMCP type
)
async def get_credential(name: str) -> Dict[str, Any]:
ctx = _require_batch()
+ from services.gateway.routing import GATEWAY_CREDENTIAL_ID
+
+ if name == GATEWAY_CREDENTIAL_ID:
+ # Only OpenCompany's own clients spend the gateway key: in an
+ # agent's hands it would spend the allowance outside any run.
+ return {
+ "error": "the OpenCompany Gateway key is never handed out",
+ "name": name,
+ "status": 403,
+ }
if name not in ctx.allowed_credentials:
return {
"error": f"credential {name!r} not in allowlist for this batch",
diff --git a/server/services/cli_agent/service.py b/server/services/cli_agent/service.py
index 379d2cec..703466c9 100644
--- a/server/services/cli_agent/service.py
+++ b/server/services/cli_agent/service.py
@@ -674,7 +674,7 @@ class AICliService:
env = {**os.environ, "PYTHONUNBUFFERED": "1"}
# Project-local claude auth dir — same as ``AICliSession.env``.
- from nodes.agent.claude_code_agent._oauth import OPENCOMPANY_CLAUDE_DIR
+ from nodes.agent.claude_code_agent._oauth import OPENCOMPANY_CLAUDE_DIR, api_access
env["CLAUDE_CONFIG_DIR"] = str(OPENCOMPANY_CLAUDE_DIR)
# Composio-style parent-run-ID for MCP correlation.
@@ -683,6 +683,10 @@ class AICliService:
env["MACHINA_PARENT_RUN_ID"] = parent_run_id # legacy child-process contract
try:
+ # Without its own login, claude runs on the install's Anthropic
+ # key or the OpenCompany Gateway.
+ access = await api_access(env)
+ env.update(access.env)
pooled = await pool.acquire(
session_key,
spec=task,
@@ -713,7 +717,13 @@ class AICliService:
workflow_id=workflow_id,
)
finally:
- await pool.release(pooled)
+ if access.per_run:
+ # Its env carries this run's attribution headers. The next
+ # turn spawns fresh and resumes the conversation, as after
+ # an idle eviction.
+ await pool.terminate(session_key)
+ else:
+ await pool.release(pooled)
@staticmethod
async def _clear_stale_session_id(
diff --git a/server/services/gateway/routing.py b/server/services/gateway/routing.py
index 99253510..1eb9ed2c 100644
--- a/server/services/gateway/routing.py
+++ b/server/services/gateway/routing.py
@@ -13,7 +13,10 @@ gateway's ``/v1`` routes with a LiteLLM provider prefix on the model name
(``groq/...``); Anthropic and Gemini use the gateway's native routes with
bare model ids. The same route serves a provider's embeddings, speech and
vision calls, which go out from their own SDK clients
-(:func:`key_or_route`).
+(:func:`key_or_route`). The RLM agent's library speaks only OpenAI's format,
+so it asks for ``openai_format`` routes; Claude Code reaches the Anthropic
+route through its own environment variables
+(``nodes/agent/claude_code_agent/_oauth.py``).
"""
from __future__ import annotations
@@ -64,8 +67,13 @@ class GatewayRouter:
providers = self._defaults.get("providers") or {}
return [name for name in providers if self._block(name) is not None]
- async def route(self, provider: str) -> Optional[GatewayRoute]:
- """The route for a call to ``provider`` with no key of the install's own."""
+ async def route(self, provider: str, *, openai_format: bool = False) -> Optional[GatewayRoute]:
+ """The route for a call to ``provider`` with no key of the install's own.
+
+ ``openai_format`` is for a client that speaks only OpenAI's format
+ (the RLM library): the gateway's ``/v1`` routes, where a provider
+ OpenCompany otherwise calls on its native route goes by the block's
+ ``openai_prefix`` (``anthropic/``, ``gemini/``)."""
if not self.enabled():
return None
block = self._block(provider)
@@ -74,8 +82,11 @@ class GatewayRouter:
api_key = await self._auth.get_api_key(GATEWAY_CREDENTIAL_ID)
if not api_key:
return None
- base_url = str(self._settings.gateway_url).rstrip("/") + str(block.get("base_path") or "")
- return GatewayRoute(api_key=api_key, base_url=base_url, model_prefix=str(block.get("model_prefix") or ""))
+ base_path, model_prefix = str(block.get("base_path") or ""), str(block.get("model_prefix") or "")
+ if openai_format and "openai_prefix" in block:
+ base_path, model_prefix = "/v1", str(block["openai_prefix"])
+ base_url = str(self._settings.gateway_url).rstrip("/") + base_path
+ return GatewayRoute(api_key=api_key, base_url=base_url, model_prefix=model_prefix)
async def covers(self, provider: str) -> bool:
"""Whether a call to ``provider`` can run on the gateway: what a "no key
@@ -83,7 +94,7 @@ class GatewayRouter:
return await self.route(provider) is not None
-async def gateway_route(provider: str) -> Optional[GatewayRoute]:
+async def gateway_route(provider: str, *, openai_format: bool = False) -> Optional[GatewayRoute]:
""":meth:`GatewayRouter.route` on the container's router, for call sites
outside the DI graph. Under test the container is a MagicMock (whose
router would claim to cover everything) or a stub without a router, so
@@ -92,7 +103,9 @@ async def gateway_route(provider: str) -> Optional[GatewayRoute]:
factory = getattr(container, "gateway_router", None)
router = factory() if callable(factory) else None
- return await router.route(provider) if isinstance(router, GatewayRouter) else None
+ if not isinstance(router, GatewayRouter):
+ return None
+ return await router.route(provider, openai_format=openai_format)
async def gateway_covers(provider: str) -> bool:
diff --git a/server/services/rlm/adapters.py b/server/services/rlm/adapters.py
index e4d67ff9..15fc2880 100644
--- a/server/services/rlm/adapters.py
+++ b/server/services/rlm/adapters.py
@@ -5,12 +5,15 @@
- ToolBridgeAdapter: Connected tool nodes -> RLM custom_tools dict
"""
+from __future__ import annotations
+
import asyncio
import inspect
import re
import threading
from concurrent.futures import TimeoutError as FutureTimeoutError
from typing import (
+ TYPE_CHECKING,
Any,
Awaitable,
Callable,
@@ -23,6 +26,9 @@ from typing import (
from core.logging import get_logger
from .constants import PROVIDER_TO_BACKEND, PROVIDER_BASE_URLS
+if TYPE_CHECKING:
+ from services.gateway.routing import GatewayRoute
+
logger = get_logger(__name__)
@@ -30,7 +36,25 @@ class BackendAdapter:
"""Maps OpenCompany provider/model/api_key to RLM backend constructor args."""
@staticmethod
- def adapt(provider: str, model: str, api_key: str) -> Tuple[str, Dict[str, Any]]:
+ def adapt(
+ provider: str,
+ model: str,
+ api_key: Optional[str],
+ *,
+ route: Optional[GatewayRoute] = None,
+ ) -> Tuple[str, Dict[str, Any]]:
+ if route is not None:
+ # On the OpenCompany Gateway every provider speaks OpenAI's
+ # format under LiteLLM's model name. The attribution headers are
+ # the run's: an RLM completion runs within one node execution.
+ from services.gateway.attribution import request_headers
+
+ return "openai", {
+ "model_name": f"{route.model_prefix}{model}",
+ "api_key": route.api_key,
+ "base_url": route.base_url,
+ "default_headers": request_headers(),
+ }
backend = PROVIDER_TO_BACKEND.get(provider, "openai")
kwargs = {"model_name": model, "api_key": api_key}
if provider in PROVIDER_BASE_URLS:
@@ -75,11 +99,16 @@ class ChatModelExtractor:
if not api_key and auth:
api_key = await auth.resolve_api_key(provider)
+ route = None
if not api_key:
+ from services.gateway.routing import gateway_route
+
+ route = await gateway_route(provider, openai_format=True)
+ if not api_key and route is None:
logger.warning(f"[RLM] Skipping chat model node {node_type}: no API key")
continue
- backend, kwargs = BackendAdapter.adapt(provider, model, api_key)
+ backend, kwargs = BackendAdapter.adapt(provider, model, api_key, route=route)
backends.append(backend)
kwargs_list.append(kwargs)
logger.info(f"[RLM] Extracted small LM: {provider}/{model} -> {backend}")
diff --git a/server/services/rlm/service.py b/server/services/rlm/service.py
index d1ed8545..88575151 100644
--- a/server/services/rlm/service.py
+++ b/server/services/rlm/service.py
@@ -46,6 +46,7 @@ class RLMService:
"""
start_time = time.time()
context_bridge = None
+ route = None
if (context_data or {}).get("kind") == "context":
from services.cli_agent.context_bridge import (
SpecializedAgentContextBridge,
@@ -110,6 +111,13 @@ class RLMService:
if not api_key and self.auth:
api_key = await self.auth.resolve_api_key(provider)
if not api_key:
+ # Own keys first. The RLM library reaches a custom URL only
+ # in OpenAI's format, which the gateway serves for every
+ # provider it serves.
+ from services.gateway.routing import gateway_route
+
+ route = await gateway_route(provider, openai_format=True)
+ if not api_key and route is None:
raise ValueError(f"API key required for RLM Agent (provider: {provider})")
await broadcast_status(
@@ -131,7 +139,7 @@ class RLMService:
supplementary_context += f"\n\n## Conversation History\n{memory_text}"
# === Adapter: Map provider to RLM backend ===
- backend, backend_kwargs = BackendAdapter.adapt(provider, model, api_key)
+ backend, backend_kwargs = BackendAdapter.adapt(provider, model, api_key, route=route)
# === Adapter: Extract chat model nodes as other_backends ===
other_backends, other_backend_kwargs = await ChatModelExtractor.extract(tool_data, self.auth)
@@ -294,6 +302,11 @@ class RLMService:
except Exception as e:
public_error = str(e)
+ if route is not None:
+ from services.gateway.errors import describe
+
+ # The vendor's text would point at the wrong key and account.
+ public_error = describe(e) or public_error
logger.error(
"[RLM] Execution failed type=%s",
type(e).__name__,
diff --git a/server/tests/gateway/test_claude_code_route.py b/server/tests/gateway/test_claude_code_route.py
new file mode 100644
index 00000000..c7adc140
--- /dev/null
+++ b/server/tests/gateway/test_claude_code_route.py
@@ -0,0 +1,120 @@
+"""Claude Code: its own login first, then the install's Anthropic key, then
+the gateway (nodes/agent/claude_code_agent/_oauth.py, services/cli_agent)."""
+
+from types import SimpleNamespace
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+import structlog
+
+from nodes.agent.claude_code_agent import _oauth
+from nodes.agent.claude_code_agent._oauth import ApiAccess, api_access
+from services.gateway.routing import GatewayRoute
+
+ROUTE = GatewayRoute(api_key="sk-gateway", base_url="https://gw.example", model_prefix="")
+
+
+def install(monkeypatch):
+ """Configure the login, the stored Anthropic key and the gateway route."""
+
+ def configure(*, logged_in=False, own_key=None, route=ROUTE):
+ from core.container import container
+
+ auth = MagicMock()
+ auth.resolve_api_key = AsyncMock(return_value=own_key)
+ monkeypatch.setattr(container, "auth_service", MagicMock(return_value=auth))
+ monkeypatch.setattr("services.gateway.routing.gateway_route", AsyncMock(return_value=route))
+ status = AsyncMock(return_value=logged_in)
+ monkeypatch.setattr(_oauth, "claude_auth_status", status)
+ return status
+
+ return configure
+
+
+async def test_the_clis_own_login_comes_first(install):
+ install(logged_in=True, own_key="sk-own")
+ assert await api_access({}) == ApiAccess()
+
+
+async def test_without_a_login_the_install_key_comes_next(install):
+ install(own_key="sk-own")
+ assert await api_access({}) == ApiAccess(env={"ANTHROPIC_API_KEY": "sk-own"})
+
+
+async def test_without_a_login_or_key_claude_runs_on_the_gateway(install):
+ install()
+ with structlog.contextvars.bound_contextvars(chat_run_id="run-1", workflow_id="wf-1"):
+ access = await api_access({})
+ assert access.per_run is True
+ assert access.env["ANTHROPIC_BASE_URL"] == "https://gw.example"
+ assert access.env["ANTHROPIC_AUTH_TOKEN"] == "sk-gateway"
+ headers = dict(line.split(": ", 1) for line in access.env["ANTHROPIC_CUSTOM_HEADERS"].splitlines())
+ assert headers["x-litellm-session-id"] == "run-1"
+ assert "workflow:wf-1" in headers["x-litellm-tags"]
+
+
+async def test_without_the_gateway_nothing_changes(install):
+ """Not even an Anthropic key: claude uses its login as always."""
+ status = install(own_key="sk-own", route=None)
+ assert await api_access({}) == ApiAccess()
+ status.assert_not_awaited()
+
+
+async def test_api_settings_in_the_servers_environment_are_left_alone(install):
+ status = install()
+ assert await api_access({"ANTHROPIC_BASE_URL": "https://proxy.example"}) == ApiAccess()
+ status.assert_not_awaited()
+
+
[email protected]("per_run", [True, False])
+async def test_a_gateway_process_never_serves_another_run(monkeypatch, tmp_path, per_run):
+ from services.cli_agent.service import AICliService
+ from services.cli_agent.types import ClaudeTaskSpec
+
+ access = ApiAccess(env={"ANTHROPIC_AUTH_TOKEN": "sk-gateway"} if per_run else {}, per_run=per_run)
+ monkeypatch.setattr(_oauth, "api_access", AsyncMock(return_value=access))
+ pool = SimpleNamespace(
+ start_reaper=AsyncMock(),
+ acquire=AsyncMock(return_value="pooled"),
+ send_turn=AsyncMock(return_value="result"),
+ release=AsyncMock(),
+ terminate=AsyncMock(),
+ )
+ monkeypatch.setattr("services.cli_agent.service.get_session_pool", lambda _: pool)
+ await AICliService()._run_pooled_turn(
+ task=ClaudeTaskSpec(prompt="task"), session_key="session", cwd=tmp_path, workspace_dir=tmp_path,
+ defaults={}, mcp_port=8000, mcp_bearer_token="abcdefgh-token", connected_tools=[],
+ connected_skill_names=[], workflow_id="workflow",
+ )
+ env = pool.acquire.call_args.kwargs["env"]
+ assert (env.get("ANTHROPIC_AUTH_TOKEN") == "sk-gateway") is per_run
+ if per_run:
+ pool.terminate.assert_awaited_once_with("session")
+ pool.release.assert_not_awaited()
+ else:
+ pool.release.assert_awaited_once_with("pooled")
+ pool.terminate.assert_not_awaited()
+
+
+async def test_an_agent_never_gets_the_gateway_key():
+ from services.cli_agent import mcp_server
+
+ tools = {}
+
+ class FakeMCP:
+ def tool(self, name=None, **_):
+ def register(fn):
+ tools[name] = fn
+ return fn
+
+ return register
+
+ mcp_server._build_tools(FakeMCP())
+ token = mcp_server._current_batch.set(SimpleNamespace(allowed_credentials={"opencompany_gateway"}))
+ try:
+ refused = await tools["getCredential"]("opencompany_gateway")
+ finally:
+ mcp_server._current_batch.reset(token)
+ assert refused["status"] == 403
+ assert "value" not in refused
diff --git a/server/tests/gateway/test_rlm_route.py b/server/tests/gateway/test_rlm_route.py
new file mode 100644
index 00000000..18c860a6
--- /dev/null
+++ b/server/tests/gateway/test_rlm_route.py
@@ -0,0 +1,111 @@
+"""The RLM agent runs on the gateway's OpenAI-format routes with no own key
+(services/rlm): its library reaches a custom URL only in OpenAI's format."""
+
+import sys
+from types import ModuleType, SimpleNamespace
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import structlog
+
+from services.gateway.routing import GatewayRoute
+from services.rlm.adapters import BackendAdapter, ChatModelExtractor
+from services.rlm.service import RLMService
+
+from .test_errors import _gateway_error
+
+ROUTE = GatewayRoute(api_key="sk-gateway", base_url="https://gw.example/v1", model_prefix="anthropic/")
+
+
+def _auth(key=None):
+ auth = MagicMock()
+ auth.resolve_api_key = AsyncMock(return_value=key)
+ return auth
+
+
+def test_on_the_gateway_every_provider_uses_the_openai_backend():
+ with structlog.contextvars.bound_contextvars(chat_run_id="run-1"):
+ backend, kwargs = BackendAdapter.adapt("anthropic", "claude-x", None, route=ROUTE)
+ assert backend == "openai"
+ assert (kwargs["model_name"], kwargs["api_key"], kwargs["base_url"]) == (
+ "anthropic/claude-x",
+ "sk-gateway",
+ "https://gw.example/v1",
+ )
+ assert kwargs["default_headers"]["x-litellm-session-id"] == "run-1"
+
+
+def test_an_own_key_keeps_the_native_backend():
+ assert BackendAdapter.adapt("anthropic", "claude-x", "sk-own") == (
+ "anthropic",
+ {"model_name": "claude-x", "api_key": "sk-own"},
+ )
+
+
+async def _execute(monkeypatch, *, own_key=None, error=None):
+ """Run RLMService.execute on a fake RLM library; returns (result, RLM kwargs)."""
+ route = AsyncMock(return_value=ROUTE)
+ monkeypatch.setattr("services.gateway.routing.gateway_route", route)
+ seen = {}
+ rlm = ModuleType("rlm")
+
+ class RLM:
+ def __init__(self, **kwargs):
+ seen.update(kwargs)
+
+ def completion(self, prompt):
+ if error is not None:
+ raise error
+ usage = SimpleNamespace(total_cost=None, total_input_tokens=1, total_output_tokens=1)
+ return SimpleNamespace(response="done", metadata={}, usage_summary=usage)
+
+ rlm.RLM = RLM
+ logger = ModuleType("rlm.logger")
+ logger.RLMLogger = lambda: object()
+ # setitem, not patch.dict: restoring the whole of sys.modules would drop
+ # modules first imported during the run (services.handlers.tools) and
+ # split later tests' patches from the copies the code imports.
+ monkeypatch.setitem(sys.modules, "rlm", rlm)
+ monkeypatch.setitem(sys.modules, "rlm.logger", logger)
+ with (
+ patch("services.llm.config.is_model_valid_for_provider", return_value=True),
+ patch("services.skill_runtime.skill_tool_info", return_value=None),
+ ):
+ result = await RLMService(auth=_auth(own_key)).execute(
+ "rlm-1",
+ {"prompt": "solve", "provider": "anthropic", "model": "claude-x"},
+ workflow_id="wf",
+ context={"execution_id": "run-1"},
+ database=MagicMock(),
+ )
+ return result, seen, route
+
+
+async def test_rlm_without_an_own_key_runs_on_the_gateway(monkeypatch):
+ result, seen, route = await _execute(monkeypatch)
+ assert result["success"] is True
+ assert seen["backend"] == "openai"
+ assert seen["backend_kwargs"]["model_name"] == "anthropic/claude-x"
+ route.assert_awaited_once_with("anthropic", openai_format=True)
+
+
+async def test_rlm_with_an_own_key_never_asks_the_gateway(monkeypatch):
+ result, seen, route = await _execute(monkeypatch, own_key="sk-own")
+ assert result["success"] is True
+ assert seen["backend"] == "anthropic"
+ route.assert_not_awaited()
+
+
+async def test_a_spent_allowance_is_worded_as_the_opencompany_allowance(monkeypatch):
+ result, _, _ = await _execute(monkeypatch, error=_gateway_error(429, "budget_exceeded"))
+ assert result["success"] is False
+ assert result["error"].startswith("Your OpenCompany allowance for this month is used up.")
+
+
+async def test_a_connected_small_model_without_a_key_runs_on_the_gateway(monkeypatch):
+ route = GatewayRoute(api_key="sk-gateway", base_url="https://gw.example/v1", model_prefix="openai/")
+ monkeypatch.setattr("services.gateway.routing.gateway_route", AsyncMock(return_value=route))
+ backends, kwargs = await ChatModelExtractor.extract(
+ [{"node_type": "openaiChatModel", "parameters": {"model": "gpt-4o-mini"}}], _auth()
+ )
+ assert backends == ["openai"]
+ assert kwargs[0]["model_name"] == "openai/gpt-4o-mini"
diff --git a/server/tests/gateway/test_routing.py b/server/tests/gateway/test_routing.py
index a39f9da9..d38cf49a 100644
--- a/server/tests/gateway/test_routing.py
+++ b/server/tests/gateway/test_routing.py
@@ -64,6 +64,26 @@ async def test_distributed_mode_turns_the_gateway_off():
assert await router.route("openai") is None
+async def test_openai_format_routes_put_native_providers_on_v1():
+ """For the RLM library, which speaks only OpenAI's format to a custom URL."""
+ defaults = {"providers": {**DEFAULTS["providers"], "anthropic": {"gateway": {
+ "model_prefix": "", "base_path": "", "openai_prefix": "anthropic/"}}}}
+ router, _ = _router(defaults=defaults)
+ assert await router.route("anthropic", openai_format=True) == GatewayRoute(
+ api_key="sk-gateway", base_url="https://gw.example/v1", model_prefix="anthropic/"
+ )
+ assert await router.route("openai", openai_format=True) == await router.route("openai")
+
+
+async def test_every_served_provider_has_an_openai_format_route():
+ from services.llm.config import LLM_DEFAULTS
+
+ router, _ = _router(defaults=LLM_DEFAULTS)
+ for provider in router.served_providers():
+ route = await router.route(provider, openai_format=True)
+ assert route.base_url.endswith("/v1") and route.model_prefix.endswith("/"), provider
+
+
def test_served_providers_come_from_the_gateway_blocks():
router, _ = _router()
assert router.served_providers() == ["openai", "anthropic"]
diff --git a/server/tests/services/test_process_environment.py b/server/tests/services/test_process_environment.py
index f6fa4929..c2aa182d 100644
--- a/server/tests/services/test_process_environment.py
+++ b/server/tests/services/test_process_environment.py
@@ -233,6 +233,10 @@ async def test_cli_pooled_turn_supplies_filtered_env_and_keeps_parent_ids(
send_turn=AsyncMock(return_value="result"), release=AsyncMock(),
)
monkeypatch.setattr("services.cli_agent.service.get_session_pool", lambda _: pool)
+ # The login / Anthropic key / gateway choice is covered in tests/gateway.
+ from nodes.agent.claude_code_agent import _oauth
+
+ monkeypatch.setattr(_oauth, "api_access", AsyncMock(return_value=_oauth.ApiAccess()))
result = await AICliService()._run_pooled_turn(
task=ClaudeTaskSpec(prompt="safe task"), session_key="session", cwd=tmp_path,
workspace_dir=tmp_path, defaults={}, mcp_port=8000, mcp_bearer_token="abcdefgh-token",