Skip to content

AI-powered virtual executive team — a single coherent executive persona backed by 8 specialist agents (FastAPI + Next.js).

feat: support local / self-hosted OpenAI-compatible models

dtmaiale89babe7e9779
diff --git a/.env.example b/.env.example
index 7a8a61b9..3eb53743 100644
--- a/.env.example
+++ b/.env.example
@@ -1,4 +1,6 @@
-# Required
+# Required UNLESS you run entirely on local or OpenRouter models (see
+# LOCAL_MODELS_ENABLED / OPENROUTER_ENABLED below). At least one provider must
+# be configured or the app refuses to start.
ANTHROPIC_API_KEY=sk-ant-your-key-here
# --- Auth (UI: Auth.js / NextAuth v5 with Google) -----------------------------
@@ -54,6 +56,35 @@ OPENROUTER_ENABLED=false
# Required when OPENROUTER_ENABLED=true. Get one at https://openrouter.ai.
OPENROUTER_API_KEY=
+# --- Local / self-hosted models ----------------------------------------------
+# Route selected model slugs to a local OpenAI-compatible server (Ollama,
+# LM Studio, vLLM, llama.cpp, …) instead of the Anthropic API. Off by default;
+# a fresh checkout behaves identically to before.
+LOCAL_MODELS_ENABLED=false
+# Base URL of the local server, INCLUDING the version path. Required when
+# LOCAL_MODELS_ENABLED=true.
+# Ollama http://localhost:11434/v1
+# LM Studio http://localhost:1234/v1
+# vLLM http://localhost:8000/v1
+LOCAL_BASE_URL=
+# Optional bearer token. Ollama / LM Studio need none (leave blank); vLLM or a
+# gateway in front may require one.
+LOCAL_API_KEY=
+# Comma-separated model slugs to surface in the Council UI and route locally.
+# Sent to the server verbatim, so they must match what it serves
+# (e.g. `ollama list`). Example: llama3.3,qwen2.5:14b
+LOCAL_MODELS=
+# Local generation (especially on CPU) can be slow — generous timeout (seconds).
+LOCAL_TIMEOUT_S=300
+#
+# To run with NO Anthropic key, configure the three lines above AND point the
+# model settings at local slugs:
+# DEFAULT_MODEL=llama3.3
+# DEEP_REASONING_MODEL=llama3.3
+# ROUTING_MODEL=llama3.3
+# (Server-side web search has no local equivalent and is disabled for local
+# models; tool-use quality depends on the model you choose.)
+
# --- Honcho per-person memory ------------------------------------------------
# External memory layer (https://honcho.dev) that gives the Executive a
# per-person peer card shared across Slack, Discord, Telegram, email, and the
diff --git a/README.md b/README.md
index 0bd6e9dd..ae8f83fc 100644
--- a/README.md
+++ b/README.md
@@ -259,11 +259,14 @@ The deployed UI is gated behind Google sign-in with an email allow-list, and the
## Configuration
-All settings via environment variables. Minimum required: `ANTHROPIC_API_KEY`.
+All settings via environment variables. Minimum required: `ANTHROPIC_API_KEY` —
+*unless* you configure a local or OpenRouter backend instead (see [Running on
+Local Models](#running-on-local-models)). At least one provider must be set or
+the app refuses to start.
| Variable | Required | Default | Description |
|---|---|---|---|
-| `ANTHROPIC_API_KEY` | Yes | — | Anthropic API key |
+| `ANTHROPIC_API_KEY` | Yes¹ | — | Anthropic API key |
| `DEFAULT_MODEL` | No | `claude-sonnet-4-6` | Executive + most specialists |
| `DEEP_REASONING_MODEL` | No | `claude-opus-4-7` | CSO, CFO, GC, Board |
| `VECTOR_STORE_PATH` | No | `./chroma_db` | ChromaDB directory |
@@ -287,12 +290,55 @@ All settings via environment variables. Minimum required: `ANTHROPIC_API_KEY`.
| `GOOGLE_OAUTH_CLIENT_SECRET` | No | — | Google OAuth client secret (Gmail MCP) |
| `OPENROUTER_ENABLED` | No | `false` | Route Claude calls through OpenRouter and unlock non-Anthropic models per-agent in the Council UI |
| `OPENROUTER_API_KEY` | No | — | Required when `OPENROUTER_ENABLED=true` |
+| `LOCAL_MODELS_ENABLED` | No | `false` | Route selected slugs to a local OpenAI-compatible server (Ollama, LM Studio, vLLM, llama.cpp) |
+| `LOCAL_BASE_URL` | No | — | Local server URL incl. version path, e.g. `http://localhost:11434/v1`. Required when `LOCAL_MODELS_ENABLED=true` |
+| `LOCAL_API_KEY` | No | — | Optional bearer token (vLLM / gateways); Ollama & LM Studio need none |
+| `LOCAL_MODELS` | No | — | Comma-separated local model slugs to surface in the Council UI and route locally, e.g. `llama3.3,qwen2.5` |
+| `LOCAL_TIMEOUT_S` | No | `300` | Per-call timeout for local generation, in seconds |
| `HONCHO_ENABLED` | No | `false` | Per-person memory layer ([honcho.dev](https://honcho.dev)) — a peer card shared across all channels |
| `HONCHO_API_KEY` | No | — | Required when `HONCHO_ENABLED=true` |
| `HONCHO_BASE_URL` | No | — | Self-hosted Honcho endpoint |
See [.env.example](.env.example) for the full list.
+> ¹ `ANTHROPIC_API_KEY` is required only when you serve Claude models directly.
+> It can be omitted entirely if you run on local models (`LOCAL_MODELS_ENABLED`)
+> or route through OpenRouter (`OPENROUTER_ENABLED`).
+
+## Running on Local Models
+
+Open Executive can run against any **OpenAI-compatible** local server — Ollama,
+LM Studio, vLLM, or llama.cpp — instead of (or alongside) the Anthropic API.
+Local model slugs route to your server through the same provider abstraction the
+hosted models use; no agent or orchestrator code changes.
+
+```bash
+# 1. Pull a capable, tool-use-friendly model (example: Ollama)
+ollama pull llama3.3
+
+# 2. In .env — point at the local server and list the slugs to expose
+LOCAL_MODELS_ENABLED=true
+LOCAL_BASE_URL=http://localhost:11434/v1 # Ollama default
+LOCAL_MODELS=llama3.3
+
+# 3. (Optional) run with NO Anthropic key — make local the default everywhere
+DEFAULT_MODEL=llama3.3
+DEEP_REASONING_MODEL=llama3.3
+ROUTING_MODEL=llama3.3
+# ...and leave ANTHROPIC_API_KEY unset
+```
+
+The listed slugs appear in the **Council UI** model dropdown, so you can also run
+a hybrid setup — keep the Executive on Claude while flipping individual
+specialists to a local model per-agent.
+
+**Caveats.** Server-side web search (`ENABLE_WEB_SEARCH`) and Anthropic prompt
+caching / extended thinking have no local equivalent and are automatically
+disabled for local models. Multi-agent routing leans heavily on tool use, so
+pick a model that's strong at it (e.g. Llama 3.3 70B, Qwen2.5) — small models
+may route poorly. `LOCAL_API_KEY` is only needed if your server (vLLM, or a
+gateway) requires a bearer token; Ollama and LM Studio need none.
+
## Adding a New Specialist Agent
1. Create `packages/core/openexecutive/agents/your_agent.py` extending `BaseAgent`
diff --git a/packages/core/openexecutive/architecture/architecture-facts.yaml b/packages/core/openexecutive/architecture/architecture-facts.yaml
index 4228bede..dd2a8ba1 100644
--- a/packages/core/openexecutive/architecture/architecture-facts.yaml
+++ b/packages/core/openexecutive/architecture/architecture-facts.yaml
@@ -37,8 +37,21 @@ system:
matrix) was removed — its 429s surfaced as user-visible errors and the
per-agent free-model surface was more than it earned; BYO-model routing
through OpenRouter is otherwise unchanged.
- Non-Claude slugs always route through OpenRouter; selecting one
- with OPENROUTER_ENABLED=false is rejected at the API layer.
+ When LOCAL_MODELS_ENABLED=true (plus LOCAL_BASE_URL), the slugs listed
+ in LOCAL_MODELS route to a self-hosted OpenAI-compatible server
+ (Ollama / LM Studio / vLLM / llama.cpp) through the same provider seam,
+ reusing the OpenRouter request/response/stream translator. Those slugs
+ are added to the Council UI dropdown and get the non-Claude feature
+ spec (no cache_control / thinking / web_search). This makes
+ ANTHROPIC_API_KEY optional: a deployment with only local (or
+ OpenRouter) models configured boots with no Anthropic key — the
+ Settings validator just requires that at least one provider is
+ reachable. Requesting a Claude model with no key and OpenRouter off
+ returns an actionable 400.
+ Non-Claude slugs route through OpenRouter, except those listed in
+ LOCAL_MODELS (which route to the local backend); selecting an
+ OpenRouter-only slug with OPENROUTER_ENABLED=false is rejected at the
+ API layer.
Provider selection is per-call by model, not global — so the
Executive can be on GPT-5 while the CFO specialist stays on
claude-opus-4-7 within the same turn.
@@ -53,8 +66,9 @@ system:
- Tool definitions are sorted by name before being sent to the API so cache hits survive across processes.
- Company-specific data lives under packages/core/company/ and is gitignored. Never commit it.
- Audit writes never break a chat turn or tool call — exceptions are swallowed with a warning.
- - Provider selection is per-call by model (openexecutive.providers.get_provider(model)). A single chat turn can mix Anthropic-routed Claude calls and OpenRouter-routed non-Claude calls — no global "current provider" state.
- - When a non-Claude model is selected, Anthropic-only fields (cache_control on system/tool/user blocks, adaptive thinking + output_config, server-side web_search) are stripped by the feature gate before the request leaves the process. Callers do NOT need to know which model they're routing to.
+ - Provider selection is per-call by model (openexecutive.providers.get_provider(model)). A single chat turn can mix Anthropic-routed Claude calls, OpenRouter-routed non-Claude calls, and locally-routed self-hosted models — no global "current provider" state.
+ - When a non-Claude or local model is selected, Anthropic-only fields (cache_control on system/tool/user blocks, adaptive thinking + output_config, server-side web_search) are stripped by the feature gate before the request leaves the process. Callers do NOT need to know which model they're routing to.
+ - ANTHROPIC_API_KEY is optional: a Settings validator requires at least one provider (Anthropic key, OpenRouter, or local models) to be configured, so an Anthropic-free deployment is supported but a zero-provider one fails fast at startup.
- External-monitor adapters are read-only — they never write to upstream systems. The whole `monitoring/` subsystem is observation-only; surfacing happens through the alert pipeline, never via a direct upstream side-effect.
- External signals never auto-execute on the principal's behalf. Every signal that surfaces is proposed via the alert pipeline (and thus subject to triage / mute / severity gates); the scheduler never dispatches an outbound message directly off a signal.
- An external signal whose `dedup_key` already exists is dropped at ingest via the UNIQUE(dedup_key) constraint on `external_signals`. Adapter retries / re-polls cannot multiply alerts for the same upstream event.
diff --git a/packages/core/openexecutive/architecture/prebuilt/overview.json b/packages/core/openexecutive/architecture/prebuilt/overview.json
index f37d9597..59660c52 100644
--- a/packages/core/openexecutive/architecture/prebuilt/overview.json
+++ b/packages/core/openexecutive/architecture/prebuilt/overview.json
@@ -1,7 +1,7 @@
{
"section_id": "overview",
"title": "System Overview",
- "markdown": "Open Executive is a multi-agent system presenting a single executive persona to the user while a council of domain specialists works behind it. Clients and integration channels reach a FastAPI backend, which drives the `Executive` orchestrator (`openexecutive.orchestrator.executive`). The internal sub-agent architecture is never exposed — there is one voice.\n\n### Topology\n\n- **Clients** — the Next.js web UI plus inbound integration channels (email, Slack, Discord, Telegram) all enter through the FastAPI API layer (`POST /chat` streams the live turn as server-sent events).\n- **Orchestrator** — the `Executive` runs an Anthropic tool-use loop. For cross-domain questions it emits multiple parallel `consult_specialist` tool calls that fan out concurrently via `openexecutive.orchestrator.router.route_parallel`, bounded by `MAX_PARALLEL_SPECIALISTS`.\n- **Specialists** — seven consult-routable domain agents (cso, cfo, cmo, coo, chro, cpo, gc). Each runs its own domain-filtered ChromaDB query before seeing the question, so retrieval is scoped per declared domain alias.\n- **Knowledge layer** — ChromaDB holds curated `BUILTIN` knowledge, ingested `COMPANY` documents, recorded `FAILURES`, and `RECENT_RESEARCH` artifacts; SQLite backs sessions, audit, departments, and people.\n\nThe Executive synthesizes all specialist input into one coherent reply. LLM calls resolve per-call through `openexecutive.providers.get_provider(model)`, defaulting to the Anthropic SDK with Anthropic prompt caching on the system blocks.",
+ "markdown": "Open Executive is a multi-agent system presenting a single executive persona to the user while a council of domain specialists works behind it. Clients and integration channels reach a FastAPI backend, which drives the `Executive` orchestrator (`openexecutive.orchestrator.executive`). The internal sub-agent architecture is never exposed — there is one voice.\n\n### Topology\n\n- **Clients** — the Next.js web UI plus inbound integration channels (email, Slack, Discord, Telegram) all enter through the FastAPI API layer (`POST /chat` streams the live turn as server-sent events).\n- **Orchestrator** — the `Executive` runs an Anthropic tool-use loop. For cross-domain questions it emits multiple parallel `consult_specialist` tool calls that fan out concurrently via `openexecutive.orchestrator.router.route_parallel`, bounded by `MAX_PARALLEL_SPECIALISTS`.\n- **Specialists** — seven consult-routable domain agents (cso, cfo, cmo, coo, chro, cpo, gc). Each runs its own domain-filtered ChromaDB query before seeing the question, so retrieval is scoped per declared domain alias.\n- **Knowledge layer** — ChromaDB holds curated `BUILTIN` knowledge, ingested `COMPANY` documents, recorded `FAILURES`, and `RECENT_RESEARCH` artifacts; SQLite backs sessions, audit, departments, and people.\n\nThe Executive synthesizes all specialist input into one coherent reply. LLM calls resolve per-call through `openexecutive.providers.get_provider(model)`, defaulting to the Anthropic SDK with Anthropic prompt caching on the system blocks. Setting `OPENROUTER_ENABLED` routes calls through OpenRouter, and `LOCAL_MODELS_ENABLED` routes the slugs listed in `LOCAL_MODELS` to a self-hosted OpenAI-compatible server (Ollama, LM Studio, vLLM) via the same provider seam — so `ANTHROPIC_API_KEY` becomes optional when one of those is configured.",
"mermaid": "flowchart TD\n Clients[\"Web UI &<br/>integrations\"]\n API[\"FastAPI API\"]\n Exec[\"Executive<br/>orchestrator\"]\n Router[\"route_parallel\"]\n Spec[\"Domain specialists<br/>(cso, cfo, cmo, ...)\"]\n KB[(\"ChromaDB knowledge\")]\n Claude{{\"Anthropic Claude\"}}\n Clients --> API\n API --> Exec\n Exec -->|\"consult_specialist\"| Router\n Router --> Spec\n Spec -->|\"domain-filtered RAG\"| KB\n Spec --> Exec\n Exec --> Claude\n Exec -->|\"synthesized reply\"| API\n class Clients entry\n class API entry\n class Exec compute\n class Router compute\n class Spec compute\n class KB storage\n class Claude external",
"generated_at": "2026-06-01T00:00:00+00:00"
}
diff --git a/packages/core/openexecutive/config.py b/packages/core/openexecutive/config.py
index 5451c601..79631993 100644
--- a/packages/core/openexecutive/config.py
+++ b/packages/core/openexecutive/config.py
@@ -29,7 +29,12 @@ class Settings(BaseSettings):
extra="ignore",
)
- anthropic_api_key: str = Field(..., alias="ANTHROPIC_API_KEY")
+ # Optional: a deployment can run entirely on local / OpenRouter models
+ # with no Anthropic key. The `_validate_provider_available` model
+ # validator below ensures at least one backend is reachable, and the
+ # registry raises an actionable error if a Claude model is requested
+ # while this is unset.
+ anthropic_api_key: str | None = Field(None, alias="ANTHROPIC_API_KEY")
default_model: str = Field("claude-sonnet-4-6", alias="DEFAULT_MODEL")
deep_reasoning_model: str = Field("claude-opus-4-7", alias="DEEP_REASONING_MODEL")
@@ -103,6 +108,70 @@ class Settings(BaseSettings):
)
return self
+ # ---- Local / self-hosted models ------------------------------------
+ # Route selected model slugs to a local OpenAI-compatible server
+ # (Ollama, LM Studio, vLLM, llama.cpp, …) instead of the Anthropic API.
+ # Default OFF so a fresh checkout's behavior is identical to before.
+ #
+ # To run with NO Anthropic key, also point the model settings at local
+ # slugs, e.g.:
+ # LOCAL_MODELS_ENABLED=true
+ # LOCAL_BASE_URL=http://localhost:11434/v1 # Ollama
+ # LOCAL_MODELS=llama3.3,qwen2.5
+ # DEFAULT_MODEL=llama3.3
+ # DEEP_REASONING_MODEL=llama3.3
+ # ROUTING_MODEL=llama3.3
+ local_models_enabled: bool = Field(False, alias="LOCAL_MODELS_ENABLED")
+ # Base URL of the local OpenAI-compatible server, including the version
+ # path, e.g. http://localhost:11434/v1 (Ollama) or http://localhost:1234/v1
+ # (LM Studio). Required when LOCAL_MODELS_ENABLED is on.
+ local_base_url: str | None = Field(None, alias="LOCAL_BASE_URL")
+ # Optional bearer token. Ollama / LM Studio need none; vLLM or a gateway
+ # in front of it may. Omitted from requests entirely when unset.
+ local_api_key: str | None = Field(None, alias="LOCAL_API_KEY")
+ # Comma-separated model slugs to surface in the Council UI and route to
+ # the local backend, e.g. "llama3.3,qwen2.5:14b". These are sent to the
+ # server verbatim, so they must match the names it serves.
+ local_models: Annotated[list[str], NoDecode] = Field(
+ default_factory=list, alias="LOCAL_MODELS"
+ )
+ # Local generation (especially CPU inference) can be far slower than a
+ # hosted API. Default generous so a slow first token doesn't time out.
+ local_timeout_s: float = Field(300.0, alias="LOCAL_TIMEOUT_S")
+
+ @field_validator("local_models", mode="before")
+ @classmethod
+ def _parse_local_models(cls, v: Any) -> list[str]:
+ if isinstance(v, list):
+ return [str(x).strip() for x in v if str(x).strip()]
+ if isinstance(v, str) and v.strip():
+ return [x.strip() for x in v.split(",") if x.strip()]
+ return []
+
+ @model_validator(mode="after")
+ def _validate_local_models(self) -> "Settings":
+ if self.local_models_enabled and not self.local_base_url:
+ raise ValueError(
+ "LOCAL_MODELS_ENABLED=true requires LOCAL_BASE_URL to be set "
+ "(e.g. http://localhost:11434/v1 for Ollama)"
+ )
+ return self
+
+ @model_validator(mode="after")
+ def _validate_provider_available(self) -> "Settings":
+ # At least one backend must be reachable, or every model call fails.
+ if not (
+ self.anthropic_api_key
+ or self.openrouter_enabled
+ or self.local_models_enabled
+ ):
+ raise ValueError(
+ "No LLM provider configured. Set ANTHROPIC_API_KEY, or enable "
+ "OpenRouter (OPENROUTER_ENABLED=true + OPENROUTER_API_KEY), or "
+ "enable local models (LOCAL_MODELS_ENABLED=true + LOCAL_BASE_URL)."
+ )
+ return self
+
# ---- Honcho memory provider ----------------------------------------
# External per-person memory layer (https://honcho.dev). When enabled,
# the Executive queries Honcho for a `<peer_memory>` block keyed off the
diff --git a/packages/core/openexecutive/providers/openai_compatible.py b/packages/core/openexecutive/providers/openai_compatible.py
new file mode 100644
index 00000000..1ce3adad
--- /dev/null
+++ b/packages/core/openexecutive/providers/openai_compatible.py
@@ -0,0 +1,258 @@
+"""Generic HTTP-backed LLMProvider for any OpenAI-compatible chat API.
+
+This is the shared engine behind both ``OpenRouterProvider`` and local /
+self-hosted backends (Ollama, LM Studio, vLLM, llama.cpp). It speaks the
+OpenAI ``/chat/completions`` wire format; the translator package converts
+request/response/stream shapes so call sites stay Anthropic-native.
+
+What's intentionally configurable so a single class serves every
+OpenAI-compatible endpoint:
+
+* ``base_url`` — ``https://openrouter.ai/api/v1`` for OpenRouter,
+ ``http://localhost:11434/v1`` for Ollama, etc.
+* ``api_key`` — optional. Many local servers need no auth, so when it's
+ ``None`` we omit the ``Authorization`` header entirely rather than send
+ ``Bearer None``.
+* ``default_headers`` — provider-specific attribution headers (OpenRouter
+ sets ``HTTP-Referer`` / ``X-Title``); local backends pass nothing.
+
+Streaming notes:
+
+* SSE chunks arrive as JSON objects in ``data: ...`` lines.
+* The Executive's loop iterates ``async for event in stream`` looking for
+ ``content_block_delta`` with ``delta.type == "text_delta"`` (the streaming
+ text path) and then awaits ``stream.get_final_message()`` to read the
+ fully assembled message (with tool_use blocks). We yield text deltas
+ inline and accumulate tool_call fragments into the final message.
+"""
+from __future__ import annotations
+
+import contextlib
+import json
+import logging
+from collections.abc import AsyncIterator, Awaitable
+from contextlib import AbstractAsyncContextManager
+from types import SimpleNamespace
+from typing import Any
+
+import httpx
+
+from openexecutive.providers.feature_gate import FeatureSpec, apply_feature_gates
+from openexecutive.providers.translator import (
+ StreamAccumulator,
+ from_openai_response,
+ to_openai_request,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class OpenAICompatibleProvider:
+ """LLMProvider implementation backed by any OpenAI-compatible endpoint.
+
+ Holds one ``httpx.AsyncClient`` for the lifetime of the process; the
+ client pools its own TCP connections and is async-safe.
+ """
+
+ def __init__(
+ self,
+ *,
+ base_url: str,
+ api_key: str | None = None,
+ default_headers: dict[str, str] | None = None,
+ timeout_s: float = 180.0,
+ slug_lookup: dict[str, str] | None = None,
+ spec_lookup: dict[str, FeatureSpec] | None = None,
+ ) -> None:
+ self._api_key = api_key
+ self._base_url = base_url.rstrip("/")
+ self._client = httpx.AsyncClient(
+ base_url=self._base_url,
+ headers=default_headers or {},
+ timeout=timeout_s,
+ )
+ # internal_name → backend slug. Populated by the registry so call
+ # sites can use the same model names everywhere. Unknown models pass
+ # through unchanged (the common case for local model names).
+ self._slug_lookup = slug_lookup or {}
+ self._spec_lookup = spec_lookup or {}
+
+ # ------------------------------------------------------------------
+ # internal helpers
+ # ------------------------------------------------------------------
+
+ def _resolve(self, anthropic_model: str) -> tuple[str, FeatureSpec]:
+ slug = self._slug_lookup.get(anthropic_model, anthropic_model)
+ spec = self._spec_lookup.get(
+ anthropic_model,
+ # Default for unknown models: assume non-Claude, no Anthropic-only
+ # features. Safer than the optimistic default — a misconfigured
+ # slug then quietly gets the right gating.
+ FeatureSpec(
+ supports_cache_control=False,
+ supports_thinking=False,
+ supports_web_search=False,
+ supports_tool_use=True,
+ ),
+ )
+ return slug, spec
+
+ def _auth_headers(self) -> dict[str, str]:
+ # Local backends (Ollama, LM Studio) typically need no auth — omit
+ # the header entirely rather than send a bogus ``Bearer None``.
+ if not self._api_key:
+ return {}
+ return {"Authorization": f"Bearer {self._api_key}"}
+
+ # ------------------------------------------------------------------
+ # LLMProvider surface
+ # ------------------------------------------------------------------
+
+ def messages_create(self, **kwargs: Any) -> Awaitable[Any]:
+ return self._messages_create(kwargs)
+
+ async def _messages_create(self, kwargs: dict[str, Any]) -> Any:
+ # Strip the SDK's own ``timeout`` kwarg — httpx already has it from
+ # the client; passing it into the body would break the request.
+ request_timeout = kwargs.pop("timeout", None)
+ model = kwargs.pop("model", "")
+ slug, spec = self._resolve(model)
+ gated = apply_feature_gates(spec, kwargs)
+ body = to_openai_request(slug, gated)
+
+ try:
+ resp = await self._client.post(
+ "/chat/completions",
+ json=body,
+ headers=self._auth_headers(),
+ timeout=request_timeout if request_timeout is not None else httpx.USE_CLIENT_DEFAULT,
+ )
+ resp.raise_for_status()
+ except httpx.HTTPStatusError as exc:
+ logger.error(
+ "OpenAI-compatible backend %s returned %s: %s",
+ slug,
+ exc.response.status_code,
+ exc.response.text[:500],
+ )
+ raise
+ return from_openai_response(resp.json())
+
+ def messages_stream(self, **kwargs: Any) -> AbstractAsyncContextManager[Any]:
+ request_timeout = kwargs.pop("timeout", None)
+ model = kwargs.pop("model", "")
+ slug, spec = self._resolve(model)
+ gated = apply_feature_gates(spec, kwargs)
+ body = to_openai_request(slug, gated)
+ body["stream"] = True
+ return _OpenAICompatibleStream(
+ client=self._client,
+ body=body,
+ headers=self._auth_headers(),
+ timeout=request_timeout,
+ )
+
+ async def aclose(self) -> None:
+ await self._client.aclose()
+
+
+class _OpenAICompatibleStream:
+ """Async context manager that wraps an OpenAI-compatible SSE response so
+ it quacks like ``anthropic.AsyncMessageStreamManager``.
+
+ Consumers do:
+
+ async with provider.messages_stream(...) as stream:
+ async for event in stream:
+ ... # event has Anthropic shape
+ final = await stream.get_final_message()
+ """
+
+ def __init__(
+ self,
+ *,
+ client: httpx.AsyncClient,
+ body: dict[str, Any],
+ headers: dict[str, str],
+ timeout: float | None,
+ ) -> None:
+ self._client = client
+ self._body = body
+ self._headers = headers
+ self._timeout = timeout
+ self._response_cm: contextlib.AbstractAsyncContextManager[httpx.Response] | None = None
+ self._response: httpx.Response | None = None
+ self._accumulator = StreamAccumulator()
+ self._finalized: SimpleNamespace | None = None
+
+ async def __aenter__(self) -> _OpenAICompatibleStream:
+ # ``client.stream(...)`` is a context manager; we open it here and
+ # close it in ``__aexit__`` so callers get the same scoping as
+ # Anthropic's manager.
+ self._response_cm = self._client.stream(
+ "POST",
+ "/chat/completions",
+ json=self._body,
+ headers=self._headers,
+ timeout=self._timeout if self._timeout is not None else httpx.USE_CLIENT_DEFAULT,
+ )
+ self._response = await self._response_cm.__aenter__()
+ # If the backend returned a non-2xx we must close the response context
+ # ourselves — the outer ``async with`` will NOT call __aexit__ when
+ # __aenter__ raises, so a naive ``raise_for_status`` would leak the
+ # connection back to the pool half-read.
+ if self._response.status_code >= 400:
+ text = await self._response.aread()
+ logger.error(
+ "OpenAI-compatible stream returned %s: %s",
+ self._response.status_code,
+ text.decode("utf-8", errors="replace")[:500],
+ )
+ try:
+ self._response.raise_for_status()
+ finally:
+ await self._response_cm.__aexit__(None, None, None)
+ self._response_cm = None
+ self._response = None
+ return self
+
+ async def __aexit__(self, *exc_info: Any) -> None:
+ if self._response_cm is not None:
+ await self._response_cm.__aexit__(*exc_info)
+ self._response_cm = None
+ self._response = None
+
+ def __aiter__(self) -> AsyncIterator[Any]:
+ return self._iter_events()
+
+ async def _iter_events(self) -> AsyncIterator[Any]:
+ if self._response is None:
+ return
+ async for raw_line in self._response.aiter_lines():
+ line = raw_line.strip()
+ if not line or not line.startswith("data:"):
+ continue
+ payload = line[len("data:"):].strip()
+ if payload == "[DONE]":
+ break
+ try:
+ chunk = json.loads(payload)
+ except json.JSONDecodeError:
+ logger.debug("OpenAI-compatible: undecodable SSE payload %s", payload[:120])
+ continue
+ for event in self._accumulator.feed(chunk):
+ yield event
+ # Cache the final message so a later get_final_message() call returns
+ # immediately without re-reading the stream.
+ self._finalized = self._accumulator.finalize()
+
+ async def get_final_message(self) -> Any:
+ if self._finalized is None:
+ # The consumer didn't iterate the stream; drive it to completion.
+ async for _ in self._iter_events():
+ pass
+ # _iter_events sets _finalized in finalize(); if for some reason we
+ # got here without it being set, fall back to a fresh finalize().
+ if self._finalized is None:
+ self._finalized = self._accumulator.finalize()
+ return self._finalized
diff --git a/packages/core/openexecutive/providers/openrouter_provider.py b/packages/core/openexecutive/providers/openrouter_provider.py
index 7010bd35..b1549e1b 100644
--- a/packages/core/openexecutive/providers/openrouter_provider.py
+++ b/packages/core/openexecutive/providers/openrouter_provider.py
@@ -1,47 +1,28 @@
-"""HTTP-backed LLMProvider that targets the OpenRouter chat-completions API.
+"""OpenRouter backend — a thin specialization of ``OpenAICompatibleProvider``.
-Implements the same ``messages_create`` / ``messages_stream`` surface as
-``AnthropicProvider``, but routes through OpenRouter's OpenAI-format
-endpoint. The translator package handles the request/response/stream
-shape conversion so call sites stay Anthropic-native.
+OpenRouter speaks the OpenAI ``/chat/completions`` format, so the entire
+request/response/stream machinery lives in the generic
+``OpenAICompatibleProvider`` base. The only OpenRouter-specific bits are the
+default base URL and the attribution headers (``HTTP-Referer`` / ``X-Title``)
+that surface this app in your OpenRouter dashboard alongside the cost data.
-Streaming notes:
-
-* OpenRouter SSE chunks arrive as JSON objects in ``data: ...`` lines.
-* The Executive's loop iterates ``async for event in stream`` looking for
- ``content_block_delta`` with ``delta.type == "text_delta"`` (the streaming
- text path) and then awaits ``stream.get_final_message()`` to read the
- fully assembled message (with tool_use blocks). We yield text deltas
- inline and accumulate tool_call fragments into the final message.
+``_OpenRouterStream`` is re-exported as an alias of the generic stream class
+for backward compatibility with existing imports.
"""
from __future__ import annotations
-import contextlib
-import json
-import logging
-from collections.abc import AsyncIterator, Awaitable
-from contextlib import AbstractAsyncContextManager
-from types import SimpleNamespace
-from typing import Any
-
-import httpx
-
-from openexecutive.providers.feature_gate import FeatureSpec, apply_feature_gates
-from openexecutive.providers.translator import (
- StreamAccumulator,
- from_openai_response,
- to_openai_request,
+from openexecutive.providers.feature_gate import FeatureSpec
+from openexecutive.providers.openai_compatible import (
+ OpenAICompatibleProvider,
+ _OpenAICompatibleStream,
)
-logger = logging.getLogger(__name__)
-
+# Backward-compatible alias — the stream class is fully generic.
+_OpenRouterStream = _OpenAICompatibleStream
-class OpenRouterProvider:
- """LLMProvider implementation backed by OpenRouter.
- Holds one ``httpx.AsyncClient`` for the lifetime of the process; the
- client pools its own TCP connections and is async-safe.
- """
+class OpenRouterProvider(OpenAICompatibleProvider):
+ """LLMProvider implementation backed by OpenRouter."""
def __init__(
self,
@@ -54,193 +35,16 @@ class OpenRouterProvider:
slug_lookup: dict[str, str] | None = None,
spec_lookup: dict[str, FeatureSpec] | None = None,
) -> None:
- self._api_key = api_key
- self._base_url = base_url.rstrip("/")
# OpenRouter attribution headers — surfaced in your dashboard alongside
# the cost data so you can attribute usage back to this app.
- headers = {"HTTP-Referer": referer or "https://github.com/", "X-Title": app_title}
- self._client = httpx.AsyncClient(
- base_url=self._base_url,
- headers=headers,
- timeout=timeout_s,
- )
- # internal_name → OpenRouter slug. Populated by the registry from
- # MODEL_SPECS so call sites can use the same model names everywhere.
- self._slug_lookup = slug_lookup or {}
- self._spec_lookup = spec_lookup or {}
-
- # ------------------------------------------------------------------
- # internal helpers
- # ------------------------------------------------------------------
-
- def _resolve(self, anthropic_model: str) -> tuple[str, FeatureSpec]:
- slug = self._slug_lookup.get(anthropic_model, anthropic_model)
- spec = self._spec_lookup.get(
- anthropic_model,
- # Default for unknown models: assume non-Claude, no Anthropic-only
- # features. Safer than the optimistic default — a misconfigured
- # slug then quietly gets the right gating.
- FeatureSpec(
- supports_cache_control=False,
- supports_thinking=False,
- supports_web_search=False,
- supports_tool_use=True,
- ),
- )
- return slug, spec
-
- def _auth_headers(self) -> dict[str, str]:
- return {"Authorization": f"Bearer {self._api_key}"}
-
- # ------------------------------------------------------------------
- # LLMProvider surface
- # ------------------------------------------------------------------
-
- def messages_create(self, **kwargs: Any) -> Awaitable[Any]:
- return self._messages_create(kwargs)
-
- async def _messages_create(self, kwargs: dict[str, Any]) -> Any:
- # Strip the SDK's own ``timeout`` kwarg — httpx already has it from
- # the client; passing it into the body would break the request.
- request_timeout = kwargs.pop("timeout", None)
- model = kwargs.pop("model", "")
- slug, spec = self._resolve(model)
- gated = apply_feature_gates(spec, kwargs)
- body = to_openai_request(slug, gated)
-
- try:
- resp = await self._client.post(
- "/chat/completions",
- json=body,
- headers=self._auth_headers(),
- timeout=request_timeout if request_timeout is not None else httpx.USE_CLIENT_DEFAULT,
- )
- resp.raise_for_status()
- except httpx.HTTPStatusError as exc:
- logger.error(
- "OpenRouter %s returned %s: %s",
- slug,
- exc.response.status_code,
- exc.response.text[:500],
- )
- raise
- return from_openai_response(resp.json())
-
- def messages_stream(self, **kwargs: Any) -> AbstractAsyncContextManager[Any]:
- request_timeout = kwargs.pop("timeout", None)
- model = kwargs.pop("model", "")
- slug, spec = self._resolve(model)
- gated = apply_feature_gates(spec, kwargs)
- body = to_openai_request(slug, gated)
- body["stream"] = True
- return _OpenRouterStream(
- client=self._client,
- body=body,
- headers=self._auth_headers(),
- timeout=request_timeout,
- )
-
- async def aclose(self) -> None:
- await self._client.aclose()
-
-
-class _OpenRouterStream:
- """Async context manager that wraps an OpenRouter SSE response so it
- quacks like ``anthropic.AsyncMessageStreamManager``.
-
- Consumers do:
-
- async with provider.messages_stream(...) as stream:
- async for event in stream:
- ... # event has Anthropic shape
- final = await stream.get_final_message()
- """
-
- def __init__(
- self,
- *,
- client: httpx.AsyncClient,
- body: dict[str, Any],
- headers: dict[str, str],
- timeout: float | None,
- ) -> None:
- self._client = client
- self._body = body
- self._headers = headers
- self._timeout = timeout
- self._response_cm: contextlib.AbstractAsyncContextManager[httpx.Response] | None = None
- self._response: httpx.Response | None = None
- self._accumulator = StreamAccumulator()
- self._finalized: SimpleNamespace | None = None
-
- async def __aenter__(self) -> _OpenRouterStream:
- # ``client.stream(...)`` is a context manager; we open it here and
- # close it in ``__aexit__`` so callers get the same scoping as
- # Anthropic's manager.
- self._response_cm = self._client.stream(
- "POST",
- "/chat/completions",
- json=self._body,
- headers=self._headers,
- timeout=self._timeout if self._timeout is not None else httpx.USE_CLIENT_DEFAULT,
+ super().__init__(
+ base_url=base_url,
+ api_key=api_key,
+ default_headers={
+ "HTTP-Referer": referer or "https://github.com/",
+ "X-Title": app_title,
+ },
+ timeout_s=timeout_s,
+ slug_lookup=slug_lookup,
+ spec_lookup=spec_lookup,
)
- self._response = await self._response_cm.__aenter__()
- # If OpenRouter returned a non-2xx we must close the response context
- # ourselves — the outer ``async with`` will NOT call __aexit__ when
- # __aenter__ raises, so a naive ``raise_for_status`` would leak the
- # connection back to the pool half-read.
- if self._response.status_code >= 400:
- text = await self._response.aread()
- logger.error(
- "OpenRouter stream returned %s: %s",
- self._response.status_code,
- text.decode("utf-8", errors="replace")[:500],
- )
- try:
- self._response.raise_for_status()
- finally:
- await self._response_cm.__aexit__(None, None, None)
- self._response_cm = None
- self._response = None
- return self
-
- async def __aexit__(self, *exc_info: Any) -> None:
- if self._response_cm is not None:
- await self._response_cm.__aexit__(*exc_info)
- self._response_cm = None
- self._response = None
-
- def __aiter__(self) -> AsyncIterator[Any]:
- return self._iter_events()
-
- async def _iter_events(self) -> AsyncIterator[Any]:
- if self._response is None:
- return
- async for raw_line in self._response.aiter_lines():
- line = raw_line.strip()
- if not line or not line.startswith("data:"):
- continue
- payload = line[len("data:"):].strip()
- if payload == "[DONE]":
- break
- try:
- chunk = json.loads(payload)
- except json.JSONDecodeError:
- logger.debug("OpenRouter: undecodable SSE payload %s", payload[:120])
- continue
- for event in self._accumulator.feed(chunk):
- yield event
- # Cache the final message so a later get_final_message() call returns
- # immediately without re-reading the stream.
- self._finalized = self._accumulator.finalize()
-
- async def get_final_message(self) -> Any:
- if self._finalized is None:
- # The consumer didn't iterate the stream; drive it to completion.
- async for _ in self._iter_events():
- pass
- # _iter_events sets _finalized in finalize(); if for some reason we
- # got here without it being set, fall back to a fresh finalize().
- if self._finalized is None:
- self._finalized = self._accumulator.finalize()
- return self._finalized
diff --git a/packages/core/openexecutive/providers/registry.py b/packages/core/openexecutive/providers/registry.py
index 6f539cfd..f05ad9f5 100644
--- a/packages/core/openexecutive/providers/registry.py
+++ b/packages/core/openexecutive/providers/registry.py
@@ -9,11 +9,14 @@ route differently.
"""
from __future__ import annotations
+from typing import Any
+
from fastapi import HTTPException
from openexecutive.config import get_settings
from openexecutive.providers.anthropic_provider import AnthropicProvider
from openexecutive.providers.feature_gate import FeatureSpec
+from openexecutive.providers.openai_compatible import OpenAICompatibleProvider
from openexecutive.providers.openrouter_provider import OpenRouterProvider
from openexecutive.providers.provider import LLMProvider
@@ -73,17 +76,35 @@ _DEFAULT_NON_CLAUDE_SPEC = FeatureSpec(
)
+def _local_models(settings: Any) -> list[str]:
+ """Configured local model slugs, or ``[]`` when local routing is off.
+
+ Read defensively: lightweight test settings stubs may omit the field.
+ """
+ if not getattr(settings, "local_models_enabled", False):
+ return []
+ return list(getattr(settings, "local_models", []) or [])
+
+
def allowed_models() -> list[str]:
"""Flat allowlist the Council UI's dropdown reads.
- Always exposes the Anthropic-direct trio. The OpenRouter set is folded
- in when ``OPENROUTER_ENABLED`` is on so the dropdown can't offer a
- model the runtime won't actually serve.
+ The dropdown can't offer a model the runtime won't actually serve, so
+ each family is folded in only when it's reachable:
+
+ * Anthropic-direct trio — when an ``ANTHROPIC_API_KEY`` is set, OR when
+ ``OPENROUTER_ENABLED`` is on (Claude is then reachable via OpenRouter).
+ * OpenRouter set — when ``OPENROUTER_ENABLED`` is on.
+ * Local models — when ``LOCAL_MODELS_ENABLED`` is on.
"""
settings = get_settings()
+ models: list[str] = []
+ if getattr(settings, "anthropic_api_key", None) or settings.openrouter_enabled:
+ models.extend(ANTHROPIC_DIRECT_MODELS)
if settings.openrouter_enabled:
- return [*ANTHROPIC_DIRECT_MODELS, *OPENROUTER_MODELS]
- return list(ANTHROPIC_DIRECT_MODELS)
+ models.extend(OPENROUTER_MODELS)
+ models.extend(_local_models(settings))
+ return models
def allowed_models_for(agent_id: str | None) -> list[str]:
@@ -106,16 +127,58 @@ def _is_claude(model: str) -> bool:
# are async-safe. Recreating them per call burns ~10 ms each.
_anthropic_provider: AnthropicProvider | None = None
_openrouter_provider: OpenRouterProvider | None = None
+_local_provider: OpenAICompatibleProvider | None = None
def _anthropic() -> AnthropicProvider:
global _anthropic_provider
if _anthropic_provider is None:
settings = get_settings()
- _anthropic_provider = AnthropicProvider(api_key=settings.anthropic_api_key)
+ api_key = settings.anthropic_api_key
+ if not api_key:
+ # Reachable only when a Claude model is requested with no key and
+ # OpenRouter off — e.g. an Anthropic-free deployment that left a
+ # model setting pointed at Claude. Fail with actionable guidance.
+ raise HTTPException(
+ status_code=400,
+ detail=(
+ "A Claude model was requested but ANTHROPIC_API_KEY is not "
+ "set. Set it, enable OpenRouter, or point the model setting "
+ "at a configured local model."
+ ),
+ )
+ _anthropic_provider = AnthropicProvider(api_key=api_key)
return _anthropic_provider
+def _local() -> OpenAICompatibleProvider:
+ global _local_provider
+ if _local_provider is None:
+ settings = get_settings()
+ base_url = getattr(settings, "local_base_url", None)
+ if not base_url:
+ # The Settings model_validator already prevents LOCAL_MODELS_ENABLED
+ # without a base URL, but defense in depth — a misconfigured env
+ # could otherwise produce a request against an empty host.
+ raise HTTPException(
+ status_code=400,
+ detail="Local model routing requires LOCAL_BASE_URL",
+ )
+ # Local models get the non-Claude feature spec: no cache_control,
+ # thinking, or server-side web_search — those are Anthropic-only and
+ # would 400 (or be silently ignored) on an OpenAI-compatible server.
+ spec_lookup: dict[str, FeatureSpec] = {
+ m: _DEFAULT_NON_CLAUDE_SPEC for m in _local_models(settings)
+ }
+ _local_provider = OpenAICompatibleProvider(
+ base_url=base_url,
+ api_key=getattr(settings, "local_api_key", None),
+ timeout_s=getattr(settings, "local_timeout_s", 300.0),
+ spec_lookup=spec_lookup,
+ )
+ return _local_provider
+
+
def _openrouter() -> OpenRouterProvider:
global _openrouter_provider
if _openrouter_provider is None:
@@ -155,22 +218,31 @@ def get_provider(model: str) -> LLMProvider:
* Claude family — Anthropic direct by default; OpenRouter when
``OPENROUTER_ENABLED`` is on.
- * Non-Claude (anything in ``OPENROUTER_MODELS``, or any unknown slug)
- — OpenRouter only. Raises HTTP 400 when ``OPENROUTER_ENABLED`` is
- off, since we have no other backend that speaks those models.
+ * Local models (slugs listed in ``LOCAL_MODELS`` with
+ ``LOCAL_MODELS_ENABLED`` on) — the self-hosted OpenAI-compatible
+ backend at ``LOCAL_BASE_URL``.
+ * Other non-Claude (anything in ``OPENROUTER_MODELS``, or any unknown
+ slug) — OpenRouter only. Raises HTTP 400 when ``OPENROUTER_ENABLED``
+ is off, since we have no other backend that speaks those models.
"""
settings = get_settings()
if _is_claude(model):
if settings.openrouter_enabled:
return _openrouter()
return _anthropic()
- # Non-Claude requires OpenRouter to be enabled.
+ # Local models take precedence for their configured slugs — a local
+ # endpoint can serve them with no external dependency.
+ if model in _local_models(settings):
+ return _local()
+ # Other non-Claude slugs require OpenRouter to be enabled.
if not settings.openrouter_enabled:
raise HTTPException(
status_code=400,
detail=(
- f"Model {model!r} requires OPENROUTER_ENABLED=true; "
- f"set OPENROUTER_API_KEY and toggle the flag to route this model."
+ f"Model {model!r} requires OPENROUTER_ENABLED=true (set "
+ f"OPENROUTER_API_KEY and toggle the flag), or list it in "
+ f"LOCAL_MODELS with LOCAL_MODELS_ENABLED=true to serve it "
+ f"from a local OpenAI-compatible backend."
),
)
return _openrouter()
@@ -178,6 +250,7 @@ def get_provider(model: str) -> LLMProvider:
def _reset_for_tests() -> None:
"""Drop cached provider singletons. Test-only — pytest fixtures call this."""
- global _anthropic_provider, _openrouter_provider
+ global _anthropic_provider, _openrouter_provider, _local_provider
_anthropic_provider = None
_openrouter_provider = None
+ _local_provider = None
diff --git a/packages/core/tests/unit/test_config_local_models.py b/packages/core/tests/unit/test_config_local_models.py
new file mode 100644
index 00000000..0960f750
--- /dev/null
+++ b/packages/core/tests/unit/test_config_local_models.py
@@ -0,0 +1,77 @@
+"""Config-layer tests for local-model settings and provider-availability.
+
+These build ``Settings`` directly with ``_env_file=None`` so the repo-root
+``.env`` can't leak values in, and drive the provider-related env vars
+explicitly via monkeypatch.
+"""
+from __future__ import annotations
+
+import pytest
+
+from openexecutive.config import Settings
+
+_PROVIDER_VARS = (
+ "ANTHROPIC_API_KEY",
+ "OPENROUTER_ENABLED",
+ "OPENROUTER_API_KEY",
+ "LOCAL_MODELS_ENABLED",
+ "LOCAL_BASE_URL",
+ "LOCAL_MODELS",
+ "LOCAL_API_KEY",
+)
+
+
+def _build(monkeypatch: pytest.MonkeyPatch, **env: str) -> Settings:
+ # EXEC_EMAIL_ADDRESS is a required field unrelated to provider routing —
+ # keep a valid baseline so construction reaches the provider validators.
+ monkeypatch.setenv("EXEC_EMAIL_ADDRESS", "[email protected]")
+ for key in _PROVIDER_VARS:
+ monkeypatch.delenv(key, raising=False)
+ for key, value in env.items():
+ monkeypatch.setenv(key, value)
+ return Settings(_env_file=None)
+
+
+def test_local_models_csv_parses_and_trims(monkeypatch: pytest.MonkeyPatch) -> None:
+ s = _build(
+ monkeypatch,
+ ANTHROPIC_API_KEY="sk-test",
+ LOCAL_MODELS_ENABLED="true",
+ LOCAL_BASE_URL="http://localhost:11434/v1",
+ LOCAL_MODELS="llama3.3, qwen2.5:14b ,",
+ )
+ assert s.local_models == ["llama3.3", "qwen2.5:14b"]
+
+
+def test_local_defaults_off_and_empty(monkeypatch: pytest.MonkeyPatch) -> None:
+ s = _build(monkeypatch, ANTHROPIC_API_KEY="sk-test")
+ assert s.local_models_enabled is False
+ assert s.local_models == []
+
+
+def test_local_enabled_requires_base_url(monkeypatch: pytest.MonkeyPatch) -> None:
+ with pytest.raises(ValueError, match="LOCAL_BASE_URL"):
+ _build(monkeypatch, ANTHROPIC_API_KEY="sk-test", LOCAL_MODELS_ENABLED="true")
+
+
+def test_anthropic_free_boot_with_local(monkeypatch: pytest.MonkeyPatch) -> None:
+ """The headline of this enhancement: boot with NO Anthropic key."""
+ s = _build(
+ monkeypatch,
+ LOCAL_MODELS_ENABLED="true",
+ LOCAL_BASE_URL="http://localhost:11434/v1",
+ LOCAL_MODELS="llama3.3",
+ )
+ assert s.anthropic_api_key is None
+ assert s.local_models_enabled is True
+
+
+def test_no_provider_configured_is_rejected(monkeypatch: pytest.MonkeyPatch) -> None:
+ # ValidationError subclasses ValueError, so this catches the model_validator.
+ with pytest.raises(ValueError, match="No LLM provider"):
+ _build(monkeypatch)
+
+
+def test_anthropic_key_alone_still_boots(monkeypatch: pytest.MonkeyPatch) -> None:
+ s = _build(monkeypatch, ANTHROPIC_API_KEY="sk-test")
+ assert s.anthropic_api_key == "sk-test"
diff --git a/packages/core/tests/unit/test_openai_compatible_provider.py b/packages/core/tests/unit/test_openai_compatible_provider.py
new file mode 100644
index 00000000..65b37fe4
--- /dev/null
+++ b/packages/core/tests/unit/test_openai_compatible_provider.py
@@ -0,0 +1,107 @@
+"""Tests for the generic OpenAI-compatible provider used by local backends.
+
+OpenRouter's request/response/stream behavior is pinned in
+``test_openrouter_provider_lifecycle.py`` (OpenRouter now subclasses this
+provider). This file covers the behavior that's specific to the local /
+self-hosted path: optional auth, verbatim model passthrough, and the
+Anthropic-only feature stripping that a plain OpenAI-compatible server needs.
+"""
+from __future__ import annotations
+
+import asyncio
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock
+
+from openexecutive.providers.feature_gate import FeatureSpec
+from openexecutive.providers.openai_compatible import OpenAICompatibleProvider
+
+_LOCAL_SPEC = FeatureSpec(
+ supports_cache_control=False,
+ supports_thinking=False,
+ supports_web_search=False,
+ supports_tool_use=True,
+)
+
+
+def _local_provider(api_key: str | None = None) -> OpenAICompatibleProvider:
+ return OpenAICompatibleProvider(
+ base_url="http://localhost:11434/v1",
+ api_key=api_key,
+ spec_lookup={"llama3.3": _LOCAL_SPEC},
+ )
+
+
+def _ok_response() -> MagicMock:
+ fake = MagicMock()
+ fake.json.return_value = {
+ "id": "x",
+ "choices": [
+ {
+ "message": {"role": "assistant", "content": "ok"},
+ "finish_reason": "stop",
+ }
+ ],
+ }
+ fake.raise_for_status = MagicMock()
+ return fake
+
+
+def _run_create(provider: OpenAICompatibleProvider, **kwargs: Any) -> dict[str, Any]:
+ captured: dict[str, Any] = {}
+
+ async def _fake_post(url: str, **post_kwargs: Any) -> Any:
+ captured["url"] = url
+ captured["headers"] = post_kwargs.get("headers", {})
+ captured["json"] = post_kwargs.get("json", {})
+ return _ok_response()
+
+ provider._client.post = AsyncMock(side_effect=_fake_post) # type: ignore[method-assign]
+ asyncio.run(
+ provider.messages_create(
+ model=kwargs.pop("model", "llama3.3"),
+ max_tokens=kwargs.pop("max_tokens", 8),
+ messages=kwargs.pop(
+ "messages", [{"role": "user", "content": "hi"}]
+ ),
+ **kwargs,
+ )
+ )
+ return captured
+
+
+def test_no_auth_header_when_api_key_absent() -> None:
+ """Local servers (Ollama, LM Studio) need no auth — we must NOT send a
+ bogus ``Authorization: Bearer None`` that a strict server could reject."""
+ captured = _run_create(_local_provider(api_key=None))
+ assert "Authorization" not in captured["headers"]
+
+
+def test_auth_header_present_when_api_key_given() -> None:
+ """vLLM or a gateway in front of a local server may require a token."""
+ captured = _run_create(_local_provider(api_key="vllm-secret"))
+ assert captured["headers"]["Authorization"] == "Bearer vllm-secret"
+
+
+def test_unknown_model_passes_through_verbatim() -> None:
+ """Local model names aren't translated — they're sent as-is so they match
+ what the server actually serves."""
+ captured = _run_create(_local_provider(), model="llama3.3")
+ assert captured["json"]["model"] == "llama3.3"
+
+
+def test_anthropic_only_fields_stripped_for_local_model() -> None:
+ """thinking / output_config / cache_control have no OpenAI-format
+ equivalent and would 400 a plain server — the feature gate drops them."""
+ captured = _run_create(
+ _local_provider(),
+ thinking={"type": "adaptive"},
+ output_config={"effort": "low"},
+ system=[
+ {"type": "text", "text": "P", "cache_control": {"type": "ephemeral"}}
+ ],
+ )
+ body = captured["json"]
+ assert "thinking" not in body
+ assert "output_config" not in body
+ # system flattened into a plain string message — no cache_control survives.
+ assert isinstance(body["messages"][0]["content"], str)
diff --git a/packages/core/tests/unit/test_provider_registry.py b/packages/core/tests/unit/test_provider_registry.py
index c87055cf..1cf09e2d 100644
--- a/packages/core/tests/unit/test_provider_registry.py
+++ b/packages/core/tests/unit/test_provider_registry.py
@@ -156,15 +156,30 @@ def test_anthropic_provider_constructs_lazy_only_once(
# --------------------------------------------------------------------------
-def _settings_stub(*, enabled: bool, key: str | None = "sk-or-v1-xxx") -> Any:
+def _settings_stub(
+ *,
+ enabled: bool = False,
+ key: str | None = "sk-or-v1-xxx",
+ anthropic_key: str | None = "sk-test",
+ local_enabled: bool = False,
+ local_base_url: str | None = None,
+ local_models: list[str] | None = None,
+ local_api_key: str | None = None,
+ local_timeout_s: float = 300.0,
+) -> Any:
return SimpleNamespace(
- anthropic_api_key="sk-test",
+ anthropic_api_key=anthropic_key,
openrouter_enabled=enabled,
openrouter_api_key=key,
openrouter_base_url="https://openrouter.ai/api/v1",
openrouter_app_title="Open Executive",
openrouter_referer=None,
openrouter_timeout_s=180.0,
+ local_models_enabled=local_enabled,
+ local_base_url=local_base_url,
+ local_models=local_models or [],
+ local_api_key=local_api_key,
+ local_timeout_s=local_timeout_s,
)
@@ -277,3 +292,93 @@ def test_allowed_models_includes_openrouter_set_only_when_enabled(
assert on == [*ANTHROPIC_DIRECT_MODELS, *OPENROUTER_MODELS]
# Both sets contain entries (catch the case where one list was emptied).
assert len(on) > len(off)
+
+
+# --------------------------------------------------------------------------
+# Local / self-hosted model routing
+# --------------------------------------------------------------------------
+
+
+def _local_stub(monkeypatch: pytest.MonkeyPatch, **kwargs: Any) -> None:
+ monkeypatch.setattr(
+ "openexecutive.providers.registry.get_settings",
+ lambda: _settings_stub(
+ local_enabled=True,
+ local_base_url="http://localhost:11434/v1",
+ local_models=["llama3.3", "qwen2.5"],
+ **kwargs,
+ ),
+ )
+ registry_mod._reset_for_tests()
+
+
+def test_local_model_routes_to_local_provider(monkeypatch: pytest.MonkeyPatch) -> None:
+ from openexecutive.providers.openai_compatible import OpenAICompatibleProvider
+ from openexecutive.providers.openrouter_provider import OpenRouterProvider
+
+ _local_stub(monkeypatch, enabled=False)
+ provider = get_provider("llama3.3")
+ assert isinstance(provider, OpenAICompatibleProvider)
+ # The local backend is the plain base class, never the OpenRouter subclass.
+ assert not isinstance(provider, OpenRouterProvider)
+ # Cached singleton — same object across calls.
+ assert get_provider("qwen2.5") is provider
+
+
+def test_local_provider_distinct_from_openrouter_singleton(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """With OpenRouter also on, a local slug and an OpenRouter slug resolve to
+ different provider instances — a local request can't leak to OpenRouter."""
+ _local_stub(monkeypatch, enabled=True)
+ local = get_provider("llama3.3")
+ openrouter = get_provider("openai/gpt-5")
+ assert local is not openrouter
+
+
+def test_unlisted_model_does_not_route_local(monkeypatch: pytest.MonkeyPatch) -> None:
+ """Local routing claims ONLY its configured slugs; an unlisted non-Claude
+ slug with OpenRouter off is still a 400."""
+ from fastapi import HTTPException
+
+ _local_stub(monkeypatch, enabled=False)
+ with pytest.raises(HTTPException) as exc_info:
+ get_provider("some-unlisted-model")
+ assert exc_info.value.status_code == 400
+
+
+def test_allowed_models_includes_local_when_enabled(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ from openexecutive.providers import allowed_models
+ from openexecutive.providers.registry import ANTHROPIC_DIRECT_MODELS
+
+ _local_stub(monkeypatch, enabled=False)
+ assert allowed_models() == [*ANTHROPIC_DIRECT_MODELS, "llama3.3", "qwen2.5"]
+
+
+def test_anthropic_free_deployment_hides_claude_and_serves_local(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """No Anthropic key + OpenRouter off + local on: the Council UI must not
+ offer Claude (unreachable), only the local slugs — which route locally."""
+ from openexecutive.providers import allowed_models
+ from openexecutive.providers.openai_compatible import OpenAICompatibleProvider
+
+ _local_stub(monkeypatch, enabled=False, anthropic_key=None)
+ assert allowed_models() == ["llama3.3", "qwen2.5"]
+ assert isinstance(get_provider("llama3.3"), OpenAICompatibleProvider)
+
+
+def test_claude_without_key_or_openrouter_raises_actionable_400(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """An Anthropic-free deployment that left a model setting pointed at Claude
+ gets a clear 400 — not a None-key SDK crash deep in a request."""
+ from fastapi import HTTPException
+
+ _local_stub(monkeypatch, enabled=False, anthropic_key=None)
+ with pytest.raises(HTTPException) as exc_info:
+ get_provider("claude-sonnet-4-6")
+ assert exc_info.value.status_code == 400
+ assert "ANTHROPIC_API_KEY" in exc_info.value.detail