Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
feat(llm): add named OpenAI-compatible endpoints
trohitg316d90cf249b
diff --git a/docs-internal/node-logic-flows/README.md b/docs-internal/node-logic-flows/README.mdindex 460abaf3..24e5a7db 100644--- a/docs-internal/node-logic-flows/README.md+++ b/docs-internal/node-logic-flows/README.md@@ -52,6 +52,7 @@ match its doc here, and each doc must still describe what the code does.- [Mistral Chat Model (`mistralChatModel`)](./ai_chat_models/mistralChatModel.md)- [Ollama Chat Model (`ollamaChatModel`)](./ai_chat_models/ollamaChatModel.md)- [OpenAI Chat Model (`openaiChatModel`)](./ai_chat_models/openaiChatModel.md)+- [OpenAI-compatible Chat Model (`openaiCompatibleChatModel`)](./ai_chat_models/openaiCompatibleChatModel.md)- [OpenRouter Chat Model (`openrouterChatModel`)](./ai_chat_models/openrouterChatModel.md)- [Sarvam AI Chat Model (`sarvamChatModel`)](./ai_chat_models/sarvamChatModel.md)diff --git a/docs-internal/node-logic-flows/ai_chat_models/openaiCompatibleChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/openaiCompatibleChatModel.mdnew file mode 100644index 00000000..dac61cc5--- /dev/null+++ b/docs-internal/node-logic-flows/ai_chat_models/openaiCompatibleChatModel.md@@ -0,0 +1,111 @@+# OpenAI-compatible Chat Model (`openaiCompatibleChatModel`)++| Field | Value |+|------|-------|+| **Category** | ai_chat_models |+| **Backend handler** | [`server/nodes/model/openai_compatible_chat_model/__init__.py`](../../../server/nodes/model/openai_compatible_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |+| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |+| **Tests** | [`server/tests/nodes/test_openai_compatible_node.py`](../../../server/tests/nodes/test_openai_compatible_node.py) |+| **Skill (if any)** | n/a |+| **Dual-purpose tool** | no (group `('model',)`) |++## Purpose++Chat with any OpenAI-compatible server the user has saved as a named endpoint under Credentials > OpenAI-compatible: llama.cpp, vLLM, a LiteLLM proxy, a second Ollama host, and so on (RFC-0003 D13). The node's `endpoint` parameter holds the endpoint's provider reference, `openai_compatible:<slug>`, which `detect_ai_provider` returns as the provider. `ChatUnifier` reads the endpoint's key from the `{ref}` credential row and its resolved base URL from `{ref}_proxy`, then calls the server through `OpenAIProvider`. The `ChatModelBase.chat` operation calls `AIService.execute_chat`, as for every chat model.++## Inputs (handles)++| Handle | Connection type | Required | Purpose |+|--------|-----------------|----------|---------|+| `input-main` | main | no | Upstream data; not consumed directly |++## Parameters++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `endpoint` | string | `""` | yes | - | The saved endpoint, as its reference `openai_compatible:<slug>`. Options from the `openaiCompatibleEndpoints` loader; rendered first |+| `prompt` | string | `""` | yes | - | User message |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (first option) | no | - | Options from the `openaiCompatibleModels` loader: the models stored when the endpoint was saved or refreshed. Reloads when `endpoint` changes. Open-world: the name is not pattern-checked |+| `temperature` | number\|null | `null` | no | - | 0-2 |+| `max_tokens` | number\|null | `null` | no | - | 1-200000. Unset: the model's registered max output (a quarter of its context, 512-4096, when only the context is known), else 2048 |+| `top_p` | number\|null | `1.0` | no | - | |+| `api_key` | string\|null | `null` (injected) | no | - | Injected from the endpoint's `{ref}` row: the key the user saved, or the placeholder `sk-no-key-required` |++The field is named `endpoint`, not `provider`, on purpose: a `provider` sibling of `model` triggers the parameter panel's stored-key effect, which would copy the endpoint's key into this node's `api_key`.++## Outputs (handles)++| Handle | Shape | Description |+|--------|-------|-------------|+| `output-model` | object | Model output; standard envelope payload |++### Output payload++```ts+{+ response: string;+ thinking: string | null;+ thinking_enabled: boolean;+ model: string;+ provider: string; // the endpoint reference, openai_compatible:<slug>+ finish_reason: string;+ timestamp: string;+ input: { prompt: string; system_prompt: string };+}+```++Wrapped in `{ success, node_id, node_type, result, execution_time }`.++## Logic Flow++```mermaid+flowchart TD+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]+ B --> C[AIService.execute_chat]+ C --> D{key + prompt?}+ D -- no --> X[error envelope]+ D -- yes --> E[detect_ai_provider -> parameters.endpoint<br/>openai_compatible:slug]+ E --> F[ChatUnifier.chat -> registry.get_provider openai_compatible]+ F --> G{ref_proxy row?}+ G -- no --> Y[NodeUserError: endpoint not configured]+ G -- yes --> H[OpenAIProvider<br/>base_url=resolved URL, key from the ref row]+ H --> I[provider.chat]+ I --> J[success envelope]+ H -- typed SDK error --> Z[NodeUserError]+```++## Decision Logic++- **Provider routing**: `detect_ai_provider` recognises this node type before the `lmstudio` / `ollama` tokens and returns `parameters["endpoint"]`. With no endpoint chosen it returns the bare `openai_compatible`, which holds no key, so the run stops before any request is sent.+- **Base URL**: the URL resolved when the endpoint was saved (`services/llm/endpoints.py::resolve_base_url`); nothing is probed at call time. A call whose `{ref}_proxy` row is missing is refused instead of going to the SDK's default, api.openai.com.+- **Key**: the stored key, or the declared placeholder `sk-no-key-required` (`auth.placeholder_key` in `llm_defaults.json`); never `None`, so the SDK never falls back to `OPENAI_API_KEY`.+- **Open-world model name**: `openai_compatible` declares `open_world_models: true`, so `is_model_valid_for_provider` accepts any model id.+- **Answers without a completion**: a 2xx with no `choices` and an error body raises `LLMError(PROTOCOL)`, which names the redacted URL instead of returning an empty response.++## Side Effects++- **Database writes**: none on the chat path. Saving or refreshing the endpoint under Credentials writes the `{ref}` and `{ref}_proxy` rows and registers per-model context and price in `DATA_DIR/local_models.json`.+- **Broadcasts**: none on the chat path.+- **External API calls**: `POST {resolved base URL}/chat/completions` via the `openai` SDK.+- **File I/O**: none.+- **Subprocess**: none.++## External Dependencies++- **Credentials**: the `OpenAICompatibleCredential` rows `openai_compatible:<slug>` (key, models, per-model params) and `openai_compatible:<slug>_proxy` (resolved URL).+- **Services**: `services/llm/providers/openai.py` (`OpenAIProvider`); `services/llm/endpoints.py` (URL rooting at save time); `nodes/model/_local_validator.py` (the save path and kind detection); `nodes/model/_option_loaders.py` (the endpoint and model dropdowns).+- **Python packages**: `openai`.+- **Environment variables**: none.++## Edge cases & known limits++- **The model list is a snapshot**: it is read when the endpoint is saved or refreshed. A model the server loads later appears only after Refresh in the Credentials panel.+- **Sizing**: the kind is detected once at save time. An endpoint detected as Ollama or LM Studio is sized by their SDK probes, llama.cpp by `n_ctx` from `/props`; any other server by vLLM's `max_model_len` on the model list, then LiteLLM's model table by model id. Failing all of these the `openai_compatible` defaults apply (8192 context, 2048 output).+- **Removed endpoint**: a node that still names it has no key row, so the run fails before any request is sent. Pick another endpoint, or add it again.+- **Error boundary**: typed OpenAI SDK failures become user-safe `NodeUserError` values in `ChatUnifier`, which `BaseNode.execute()` turns into the standard failure envelope.++## Related++- **Peer nodes**: [`ollamaChatModel`](./ollamaChatModel.md) and [`lmstudioChatModel`](./lmstudioChatModel.md), which share the save path; the cloud chat-model docs in this folder.+- **Architecture docs**: [Native LLM SDK](../../native_llm_sdk.md) and [RFC-0003](../../../RFC-0003-OPENAI-COMPATIBLE-PROVIDER-CONTRACT.md).diff --git a/server/config/credential_providers.json b/server/config/credential_providers.jsonindex 4fbc7dbf..5eae9962 100644--- a/server/config/credential_providers.json+++ b/server/config/credential_providers.json@@ -231,6 +231,44 @@],"instructions": "Start the LM Studio local server from the Developer tab. API key is not required."},+ "openai_compatible": {+ "_description": "Named OpenAI-compatible endpoints (RFC-0003 D13): llama.cpp, vLLM, a LiteLLM proxy, a second Ollama host, and so on. Each saved endpoint is its own provider reference, openai_compatible:<slug>, with the same two rows Ollama uses. The panel adds and refreshes endpoints through validate_api_key (api_key carries the Base URL, the other fields ride alongside under their own keys, and a refresh adds ref) and removes one through delete_api_key with its reference. `stored` and `endpoints` are filled on every catalogue fetch by OpenAICompatibleCredential.catalogue_extras, so the fields below are an add-endpoint form, never stored values.",+ "name": "OpenAI-compatible",+ "category": "ai",+ "category_label": "AI Providers",+ "color": "dracula.pink",+ "kind": "apiKey",+ "icon_ref": "lucide:Plug",+ "fields": [+ {+ "key": "openai_compatible_url",+ "label": "Base URL",+ "type": "string",+ "placeholder": "http://localhost:8080/v1",+ "default": "http://localhost:8080",+ "required": true,+ "help": "Where the server listens. With or without /v1: saving asks the server which one it answers on."+ },+ {+ "key": "openai_compatible_label",+ "label": "Label",+ "type": "text",+ "placeholder": "e.g. home-llama",+ "required": false,+ "help": "How the endpoint appears in provider dropdowns. Defaults to the server's host and port."+ },+ {+ "key": "openai_compatible_api_key",+ "label": "API key (optional)",+ "type": "password",+ "secret": true,+ "placeholder": "Only if the server requires one",+ "required": false,+ "help": "For servers started with an API key, such as vLLM or llama.cpp with --api-key, or a LiteLLM proxy virtual key."+ }+ ],+ "instructions": "Point at any server that speaks the OpenAI API: llama.cpp (llama-server; start it with --jinja for tool calling), vLLM, a LiteLLM proxy, SGLang, LocalAI, or another Ollama or LM Studio host. Each saved endpoint appears in every agent's provider dropdown and in the OpenAI-compatible chat-model node."+ },"_cli_base": {"_abstract": true,"category": "ai",diff --git a/server/config/pricing.json b/server/config/pricing.jsonindex 2c7dd9dc..cb4bb1fe 100644--- a/server/config/pricing.json+++ b/server/config/pricing.json@@ -121,6 +121,10 @@"lmstudio": {"_description": "Local inference — same zero-cost convention as ollama.","_default": {"input": 0.00, "output": 0.00}+ },+ "openai_compatible": {+ "_description": "Named OpenAI-compatible endpoints (openai_compatible:<slug>). Self-hosted servers cost nothing at the API layer. When saving an endpoint found a model id in the LiteLLM table, PricingService.get_pricing uses that per-model price from the model registry before this _default.",+ "_default": {"input": 0.00, "output": 0.00}}},diff --git a/server/constants.py b/server/constants.pyindex c017e51d..343d4407 100644--- a/server/constants.py+++ b/server/constants.py@@ -26,6 +26,8 @@ AI_CHAT_MODEL_TYPES: FrozenSet[str] = frozenset(# format; routed through OpenAIProvider with a localhost base_url)."ollamaChatModel","lmstudioChatModel",+ # A user-named OpenAI-compatible endpoint (RFC-0003 D13).+ "openaiCompatibleChatModel",])@@ -466,6 +468,13 @@ def detect_ai_provider(node_type: str, parameters: dict = None) -> str:return "anthropic"if "gemini" in nt:return "gemini"+ # A named endpoint: the node's ``endpoint`` parameter holds its+ # provider reference, ``openai_compatible:<slug>``. Without one the+ # bare id reaches the unifier, which reports "not configured".+ if "openaicompatible" in nt:+ from services.llm.config import ENDPOINT_PROVIDER++ return (parameters or {}).get("endpoint") or ENDPOINT_PROVIDER# Local-server providers — match the LMStudioChatModelNode /# OllamaChatModelNode plugin types so the runtime path reads the# correct {provider}_proxy credential and the openai SDK is pointeddiff --git a/server/nodes/agent/_provider.py b/server/nodes/agent/_provider.pynew file mode 100644index 00000000..bbead7b8--- /dev/null+++ b/server/nodes/agent/_provider.py@@ -0,0 +1,36 @@+"""The ``provider`` field every LLM agent declares.++A provider reference: a registered provider id, or a named+OpenAI-compatible endpoint ``openai_compatible:<slug>`` (RFC-0003 D13).+Options come from the ``aiProviders`` loader (``nodes/model``), so a newly+saved endpoint appears without a NodeSpec change. The value travels as-is+through key injection, Temporal payloads and the unifier, which is why it+is one string rather than a provider plus a separate endpoint field.+"""++from __future__ import annotations++from typing import Annotated++from pydantic import AfterValidator, Field+++def _check_provider_ref(value: str) -> str:+ """Accept a registered provider, or a named endpoint of the endpoint provider."""+ from services.llm.config import ENDPOINT_PROVIDER, split_provider_ref+ from services.llm.registry import has_provider++ name, slug = split_provider_ref(value)+ if not has_provider(name) or (name == ENDPOINT_PROVIDER) != bool(slug):+ raise ValueError(f"unknown LLM provider {value!r}")+ return value+++ProviderRef = Annotated[+ str,+ AfterValidator(_check_provider_ref),+ Field(json_schema_extra={"loadOptionsMethod": "aiProviders"}),+]+++__all__ = ["ProviderRef"]diff --git a/server/nodes/agent/_specialized.py b/server/nodes/agent/_specialized.pyindex 1296b758..abf2f4a0 100644--- a/server/nodes/agent/_specialized.py+++ b/server/nodes/agent/_specialized.py@@ -11,13 +11,14 @@ one-file edit.from __future__ import annotations-from typing import Any, Literal, Optional+from typing import Any, Optionalfrom pydantic import BaseModel, ConfigDict, Fieldfrom services.plugin import ActionNode, NodeContext, Operation, TaskQueuefrom ._handles import STD_AGENT_HINTS, std_agent_handles+from ._provider import ProviderRefclass SpecializedAgentParams(BaseModel):@@ -35,23 +36,7 @@ class SpecializedAgentParams(BaseModel):"rows": 4,},)- provider: Literal[- "openai",- "anthropic",- "gemini",- "openrouter",- "xai",- "groq",- "cerebras",- "deepseek",- "kimi",- "mistral",- "sarvam",- # Local-server providers — see ai_agent.Params for the proxy_url- # rationale. Same fix; same reason.- "ollama",- "lmstudio",- ] = "openai"+ provider: ProviderRef = "openai"model: str = Field(default="",json_schema_extra={"placeholder": "Select a model..."},diff --git a/server/nodes/agent/ai_agent/__init__.py b/server/nodes/agent/ai_agent/__init__.pyindex 88338d2a..5b0d754c 100644--- a/server/nodes/agent/ai_agent/__init__.py+++ b/server/nodes/agent/ai_agent/__init__.py@@ -8,13 +8,14 @@ streaming + memory persistence + delegation.from __future__ import annotations-from typing import Any, Literal, Optional+from typing import Any, Optionalfrom pydantic import BaseModel, ConfigDict, Fieldfrom services.plugin import ActionNode, NodeContext, Operation, TaskQueuefrom .._handles import STD_AGENT_HINTS, std_agent_handles+from .._provider import ProviderRefclass AIAgentParams(BaseModel):@@ -38,26 +39,7 @@ class AIAgentParams(BaseModel):"rows": 4,},)- provider: Literal[- "openai",- "anthropic",- "gemini",- "openrouter",- "xai",- "groq",- "cerebras",- "deepseek",- "kimi",- "mistral",- "sarvam",- # Local-server providers — agent execution reads- # ``{provider}_proxy`` to point the native OpenAI client at the- # user's localhost server. Without these entries the dropdown- # silently falls back to ``"openai"`` and execute_agent ends- # up calling api.openai.com instead.- "ollama",- "lmstudio",- ] = "openai"+ provider: ProviderRef = "openai"model: str = Field(default="",json_schema_extra={"placeholder": "Select a model..."},diff --git a/server/nodes/agent/chat_agent/__init__.py b/server/nodes/agent/chat_agent/__init__.pyindex 8c555643..32bb0b54 100644--- a/server/nodes/agent/chat_agent/__init__.py+++ b/server/nodes/agent/chat_agent/__init__.py@@ -7,13 +7,14 @@ applies a different default system message + skill prompt assembly.from __future__ import annotations-from typing import Any, Literal, Optional+from typing import Any, Optionalfrom pydantic import BaseModel, ConfigDict, Fieldfrom services.plugin import ActionNode, NodeContext, Operation, TaskQueuefrom .._handles import STD_AGENT_HINTS, std_agent_handles+from .._provider import ProviderRefclass ChatAgentParams(BaseModel):@@ -25,23 +26,7 @@ class ChatAgentParams(BaseModel):convention."""- provider: Literal[- "openai",- "anthropic",- "gemini",- "openrouter",- "xai",- "groq",- "cerebras",- "deepseek",- "kimi",- "mistral",- "sarvam",- # Local-server providers — see ai_agent.Params for the proxy_url- # rationale. Same fix; same reason.- "ollama",- "lmstudio",- ] = "openai"+ provider: ProviderRef = "openai"model: str = Field(default="",json_schema_extra={"placeholder": "Select a model..."},diff --git a/server/nodes/model/__init__.py b/server/nodes/model/__init__.pyindex edd80913..11f04f92 100644--- a/server/nodes/model/__init__.py+++ b/server/nodes/model/__init__.py@@ -1 +1,14 @@-"""Plugins for the 'model' palette group. See ../__init__.py for the package layout."""+"""Plugins for the 'model' palette group. See ../__init__.py for the package layout.++Also owns the LLM provider dropdowns: the agent ``provider`` field and the+OpenAI-compatible chat-model node load their options from here, because+this folder owns the credentials those options are built from.+"""++from services.ws_handler_registry import register_option_loader++from ._option_loaders import load_ai_providers, load_endpoint_models, load_endpoints++register_option_loader("aiProviders", load_ai_providers)+register_option_loader("openaiCompatibleEndpoints", load_endpoints)+register_option_loader("openaiCompatibleModels", load_endpoint_models)diff --git a/server/nodes/model/_credentials.py b/server/nodes/model/_credentials.pyindex 8e351b38..5fdafe95 100644--- a/server/nodes/model/_credentials.py+++ b/server/nodes/model/_credentials.py@@ -8,17 +8,19 @@ The native SDK client pulls the key directly from:mod:`services.auth`; this class is the Credentials-modal + discoverymanifest, not the runtime client.-Servers the user runs (Ollama, LM Studio) store their Base URL under-``{provider}_proxy`` and need no key unless the server enforces one;-without one, the placeholder the vendor documents is sent-(``auth.placeholder_key`` in llm_defaults.json, resolved by-:func:`services.llm.config.resolve_credential`). Both save through one-path, ``_local_validator.save_llm_server`` (RFC-0003 §6).+Servers the user runs (Ollama, LM Studio, and named OpenAI-compatible+endpoints) store their Base URL under ``{provider}_proxy`` and need no+key unless the server enforces one; without one, the placeholder the+vendor documents is sent (``auth.placeholder_key`` in llm_defaults.json,+resolved by :func:`services.llm.config.resolve_credential`). All three+save through one path, ``_local_validator.save_llm_server`` (RFC-0003 §6)."""from __future__ import annotations-from typing import Any, Dict+import re+from typing import Any, Dict, Optional+from urllib.parse import urlsplitfrom services.plugin.credential import ApiKeyCredential, ProbeResult@@ -198,3 +200,98 @@ class LMStudioCredential(_LocalLLM):display_name = "LM Studio"icon = "lobehub:lmstudio"docs_url = "https://lmstudio.ai/docs/local-server"+++# Mirrors the provider column limit: "openai_compatible:" + slug + "_proxy"+# must fit in EncryptedAPIKey.provider (max 50 characters).+_ENDPOINT_SLUG_MAX = 24+_SLUG_PATTERN = re.compile(r"^[a-z0-9-]+$")+++def _endpoint_slug(label: str, base_url: str) -> str:+ """Slug for a new endpoint: from its label, else from the URL's host."""+ from slugify import slugify++ source = label or urlsplit(base_url).netloc or base_url+ return slugify(source, max_length=_ENDPOINT_SLUG_MAX, separator="-")+++def _rejected(message: str) -> Dict[str, Any]:+ return {"provider": OpenAICompatibleCredential.id, "success": True, "valid": False, "message": message, "models": []}+++class OpenAICompatibleCredential(_LLMApiKey):+ """Named OpenAI-compatible endpoints (RFC-0003 D13).++ Any number of servers, each saved under its own provider reference+ ``openai_compatible:<slug>`` with the same two rows Ollama uses+ (``{ref}`` for the key, ``{ref}_proxy`` for the resolved URL). The+ LiteLLM ``model_list`` shape: a label, a base URL, an optional key.++ Saved and refreshed through the standard ``validate_api_key`` message:+ ``api_key`` carries the Base URL, the other catalogue fields ride+ alongside under their own keys, and ``ref`` marks a refresh of an+ existing endpoint. Removed through ``delete_api_key`` with the+ endpoint's reference, which also clears its URL row.+ """++ id = "openai_compatible"+ display_name = "OpenAI-compatible"+ docs_url = "https://docs.litellm.ai/docs/providers/openai_compatible"++ # Field keys of this provider's entry in config/credential_providers.json.+ label_field = "openai_compatible_label"+ key_field = "openai_compatible_api_key"++ @classmethod+ async def validate(cls, data: Dict[str, Any]) -> Dict[str, Any]:+ from services.llm.config import endpoint_ref, split_provider_ref+ from services.llm.endpoints import SERVER_META_KEY, base_url_key+ from services.plugin.deps import get_auth_service++ from ._local_validator import save_llm_server++ auth = get_auth_service()+ label = (data.get(cls.label_field) or "").strip()+ user_key: Optional[str] = (data.get(cls.key_field) or "").strip() or None+ ref = (data.get("ref") or "").strip()++ if ref:+ # Refresh. The stored URL is authoritative: the panel only ever+ # sees the redacted form, which may have lost userinfo.+ name, slug = split_provider_ref(ref)+ if name != cls.id or not _SLUG_PATTERN.match(slug):+ return _rejected(f"Unknown endpoint {ref!r}.")+ candidate = await auth.get_api_key(base_url_key(ref))+ if not candidate:+ return _rejected(f"The endpoint '{slug}' no longer exists.")+ user_key = user_key or await auth.get_api_key(ref)+ if not label:+ meta = (await auth.get_model_params(ref)).get(SERVER_META_KEY) or {}+ label = meta.get("label") or slug+ else:+ candidate = (data.get("api_key") or "").strip()+ slug = _endpoint_slug(label, candidate)+ if not slug:+ return _rejected("Give the endpoint a label with at least one letter or digit.")+ ref = endpoint_ref(slug)+ if await auth.get_api_key(ref):+ return _rejected(+ f"An endpoint named '{slug}' already exists. Choose another label, "+ "or refresh the existing endpoint."+ )+ label = label or slug++ return await save_llm_server(ref, candidate, user_key, display=label, label=label)++ @classmethod+ async def catalogue_extras(cls) -> Dict[str, Any]:+ """``stored`` (any endpoint saved) plus the endpoint list for the panel."""+ from services.llm.endpoints import list_endpoints+ from services.plugin.deps import get_auth_service++ endpoints = [+ {"ref": e.ref, "label": e.label, "base_url": e.base_url, "kind": e.kind, "model_count": len(e.models)}+ for e in await list_endpoints(get_auth_service())+ ]+ return {"stored": bool(endpoints), "endpoints": endpoints}diff --git a/server/nodes/model/_option_loaders.py b/server/nodes/model/_option_loaders.pynew file mode 100644index 00000000..a7c18e11--- /dev/null+++ b/server/nodes/model/_option_loaders.py@@ -0,0 +1,57 @@+"""Dropdown loaders for LLM provider and endpoint selection.++The provider dropdown was a ``Literal`` on each agent's Params. A saved+endpoint cannot be a literal, so the options come from here instead and a+new endpoint appears without a NodeSpec change. Imports no LLM SDK: these+run on every parameter-panel render.+"""++from __future__ import annotations++from typing import Any, Dict, List+++async def _endpoints():+ from services.llm.endpoints import list_endpoints+ from services.plugin.deps import get_auth_service++ return await list_endpoints(get_auth_service())+++async def load_ai_providers(params: Dict[str, Any] | None = None) -> List[Dict[str, str]]:+ """Every registered provider, then one option per saved endpoint.++ Ordered as ``llm_defaults.json`` lists the providers, so the order is+ declared rather than an accident of import order. The bare+ ``openai_compatible`` id is not an option: it only exists as endpoints.+ """+ from services.llm.config import ENDPOINT_PROVIDER, LLM_DEFAULTS+ from services.llm.registry import all_providers++ blocks = LLM_DEFAULTS.get("providers", {})+ registered = set(all_providers()) - {ENDPOINT_PROVIDER}+ ordered = [name for name in blocks if name in registered]+ ordered += sorted(registered - set(ordered))+ options = [{"name": blocks.get(name, {}).get("display_name") or name, "value": name} for name in ordered]+ options += [{"name": f"{e.label} (OpenAI-compatible)", "value": e.ref} for e in await _endpoints()]+ return options+++async def load_endpoints(params: Dict[str, Any] | None = None) -> List[Dict[str, str]]:+ """Saved endpoints only, for the OpenAI-compatible chat-model node."""+ return [{"name": e.label, "value": e.ref} for e in await _endpoints()]+++async def load_endpoint_models(params: Dict[str, Any] | None = None) -> List[Dict[str, str]]:+ """Models the selected endpoint listed when it was saved."""+ from services.llm.config import ENDPOINT_PROVIDER, split_provider_ref+ from services.plugin.deps import get_auth_service++ ref = (params or {}).get("endpoint") or ""+ name, slug = split_provider_ref(ref)+ if name != ENDPOINT_PROVIDER or not slug:+ return []+ return [{"name": model, "value": model} for model in await get_auth_service().get_stored_models(ref)]+++__all__ = ["load_ai_providers", "load_endpoint_models", "load_endpoints"]diff --git a/server/nodes/model/openai_compatible_chat_model/__init__.py b/server/nodes/model/openai_compatible_chat_model/__init__.pynew file mode 100644index 00000000..e7ef3128--- /dev/null+++ b/server/nodes/model/openai_compatible_chat_model/__init__.py@@ -0,0 +1,49 @@+"""OpenAI-compatible chat model: one of the user's named endpoints.++Pick an endpoint saved under Credentials > OpenAI-compatible (llama.cpp,+vLLM, a LiteLLM proxy, a second Ollama host, ...) and a model it serves.+The endpoint's reference, ``openai_compatible:<slug>``, is the provider the+runtime resolves (``constants.detect_ai_provider``), so its stored key and+resolved URL are used with no per-node configuration.+"""++from pydantic import BaseModel, Field++from .._base import ChatModelBase, ChatModelParams+from .._credentials import OpenAICompatibleCredential+++class _EndpointField(BaseModel):+ # Pydantic lays fields out in reversed-MRO order, so declaring the+ # endpoint on a base listed after ChatModelParams renders it first,+ # above the prompt and model, where it is chosen.+ endpoint: str = Field(+ default="",+ description="Saved endpoint to send requests to.",+ json_schema_extra={"loadOptionsMethod": "openaiCompatibleEndpoints"},+ )+++class OpenAICompatibleChatModelParams(ChatModelParams, _EndpointField):+ # The field is ``endpoint``, not ``provider``: a ``provider`` sibling of+ # ``model`` triggers the parameter panel's stored-key effect, which+ # would copy the endpoint's key into this node's ``api_key``.+ model: str = Field(+ default="",+ json_schema_extra={+ "placeholder": "Select a model...",+ "loadOptionsMethod": "openaiCompatibleModels",+ "loadOptionsDependsOn": ["endpoint"],+ },+ )+++class OpenAICompatibleChatModelNode(ChatModelBase):+ type = "openaiCompatibleChatModel"+ display_name = "OpenAI-compatible"+ subtitle = "Chat Model"+ group = ("model",)+ description = "Any OpenAI-compatible server saved as a named endpoint: llama.cpp, vLLM, a LiteLLM proxy, and more"++ credentials = (OpenAICompatibleCredential,)+ Params = OpenAICompatibleChatModelParamsdiff --git a/server/nodes/model/openai_compatible_chat_model/meta.json b/server/nodes/model/openai_compatible_chat_model/meta.jsonnew file mode 100644index 00000000..66f2f1bc--- /dev/null+++ b/server/nodes/model/openai_compatible_chat_model/meta.json@@ -0,0 +1,4 @@+{+ "color": "#ff79c6",+ "icon": "lucide:Plug"+}diff --git a/server/routers/websocket.py b/server/routers/websocket.pyindex d4c9ab57..11ca1a3b 100644--- a/server/routers/websocket.py+++ b/server/routers/websocket.py@@ -393,6 +393,7 @@ async def handle_get_credential_catalogue(data: Dict[str, Any], websocket: WebSoclient-side credential checks needed."""from services.credential_registry import get_credential_registry+ from services.plugin.credential import CREDENTIAL_REGISTRYregistry = get_credential_registry()since = data.get("since")@@ -449,6 +450,16 @@ async def handle_get_credential_catalogue(data: Dict[str, Any], websocket: WebSoelse:provider["account_label"] = None+ # A credential whose state is not one row per provider id (e.g.+ # several named OpenAI-compatible endpoints) contributes its own+ # fields, and may replace ``stored``. Declared on the plugin's+ # Credential class, so this handler names no provider.+ cred_cls = CREDENTIAL_REGISTRY.get(pid)+ if cred_cls is not None:+ extras = await cred_cls.catalogue_extras()+ if extras:+ provider.update(extras)+return cataloguediff --git a/server/services/auth.py b/server/services/auth.pyindex 5421b60c..ebe6c348 100644--- a/server/services/auth.py+++ b/server/services/auth.py@@ -234,6 +234,25 @@ class AuthService:logger.error("Failed to list key scopes", provider=provider, error=str(e))return []+ async def list_api_key_providers(self, session_id: str = "default") -> List[str]:+ """List every provider name holding a key in one session.++ The other axis of :meth:`list_key_scopes`. Reads the database, not+ the lazily populated cache. Used to enumerate named OpenAI-compatible+ endpoints, whose provider names are ``openai_compatible:<slug>``.++ Args:+ session_id: Session identifier++ Returns:+ Provider names, empty on failure+ """+ try:+ return await self.credentials_db.list_api_keys(session_id)+ except Exception as e:+ logger.error("Failed to list API key providers", session_id=session_id, error=str(e))+ return []+async def get_stored_models(self, provider: str, session_id: str = "default") -> List[str]:"""Get stored models for provider.diff --git a/server/services/llm/endpoints.py b/server/services/llm/endpoints.pyindex d2ed0c1c..3920706e 100644--- a/server/services/llm/endpoints.py+++ b/server/services/llm/endpoints.py@@ -25,10 +25,14 @@ pure helpers costs nothing.from __future__ import annotationsfrom dataclasses import dataclass, field-from typing import Any, List, Optional+from typing import TYPE_CHECKING, Any, List, Optionalfrom urllib.parse import urlsplit, urlunsplitfrom core.logging import get_logger+from services.llm.config import ENDPOINT_PROVIDER, split_provider_ref++if TYPE_CHECKING:+ from services.auth import AuthServicelogger = get_logger(__name__)@@ -168,11 +172,60 @@ async def resolve_base_url(candidate: str, *, api_key: str, timeout: float = 10.)+async def list_endpoint_refs(auth: "AuthService") -> List[str]:+ """Every saved named endpoint, as provider references, sorted.++ Enumerated from the credential rows themselves rather than a separate+ index, so there is nothing that can disagree with them.+ """+ prefix = f"{ENDPOINT_PROVIDER}:"+ return sorted(+ provider+ for provider in await auth.list_api_key_providers()+ if provider.startswith(prefix) and not provider.endswith(BASE_URL_SUFFIX)+ )+++@dataclass(frozen=True)+class SavedEndpoint:+ """A saved named endpoint, as the credentials panel and dropdowns show it.++ ``base_url`` is the redacted form stored for display; the full URL+ stays in the encrypted ``{ref}_proxy`` row.+ """++ ref: str+ label: str+ base_url: str+ kind: str+ models: List[str]+++async def list_endpoints(auth: "AuthService") -> List[SavedEndpoint]:+ """Every saved named endpoint with its display metadata and models."""+ endpoints: List[SavedEndpoint] = []+ for ref in await list_endpoint_refs(auth):+ meta = (await auth.get_model_params(ref)).get(SERVER_META_KEY) or {}+ endpoints.append(+ SavedEndpoint(+ ref=ref,+ label=meta.get("label") or split_provider_ref(ref)[1],+ base_url=meta.get("base_url", ""),+ kind=meta.get("kind", "generic"),+ models=list(await auth.get_stored_models(ref)),+ )+ )+ return endpoints++__all__ = ["BASE_URL_SUFFIX","SERVER_META_KEY","ResolvedBaseUrl",+ "SavedEndpoint","base_url_key",+ "list_endpoint_refs",+ "list_endpoints","redact_url","resolve_base_url",]diff --git a/server/services/llm/providers/_compat.py b/server/services/llm/providers/_compat.pyindex 73ff7956..4377c5e6 100644--- a/server/services/llm/providers/_compat.py+++ b/server/services/llm/providers/_compat.py@@ -13,6 +13,10 @@ Phase D (see the ``_COMPAT_PROVIDERS`` comment below); Sarvam followed andbrought the ``supports_model_listing`` flag with it — a provider can bewire-compatible yet ship no model-list route, which ``OpenAIProvider``handles generically off that JSON key.++One more registration, ``openai_compatible``, carries no ``base_url`` at+all: it serves every user-named endpoint, each of which stores its own+resolved URL (RFC-0003 D13)."""from __future__ import annotations@@ -20,6 +24,7 @@ from __future__ import annotationsfrom typing import Tuplefrom core.logging import get_logger+from services.llm.config import ENDPOINT_PROVIDERfrom services.llm.providers.openai import OpenAIProviderfrom services.llm.registry import ProviderSpec, register_provider@@ -111,4 +116,23 @@ def _register_compat_providers() -> None:)+def _register_endpoint_provider() -> None:+ """Register the one provider behind every named endpoint (RFC-0003 D13).++ Deliberately no ``base_url``: each endpoint's resolved URL lives on its+ own ``openai_compatible:<slug>_proxy`` row, and the unifier refuses the+ call when that row is missing rather than let the SDK default to+ api.openai.com.+ """+ register_provider(+ ProviderSpec(+ name=ENDPOINT_PROVIDER,+ factory=OpenAIProvider,+ sdk_exception_refs=("openai:OpenAIError",),+ client_kwargs={"provider_name": ENDPOINT_PROVIDER},+ )+ )++_register_compat_providers()+_register_endpoint_provider()diff --git a/server/services/plugin/credential.py b/server/services/plugin/credential.pyindex 25ececa7..36c6e232 100644--- a/server/services/plugin/credential.py+++ b/server/services/plugin/credential.py@@ -340,6 +340,19 @@ class Credential:"""raise NotImplementedError(f"Credential subclass {cls.__name__} must override _probe()")+ @classmethod+ async def catalogue_extras(cls) -> Optional[Dict[str, Any]]:+ """Extra catalogue fields computed from stored state, or ``None``.++ ``handle_get_credential_catalogue`` merges the result into this+ provider's entry after computing ``stored`` the default way, so an+ override may also replace ``stored``. For credentials whose state is+ not one row per provider id, such as a provider holding several+ named rows. Must not do network IO: the catalogue is fetched on+ every modal open.+ """+ return None+class OAuth2Credential(Credential):"""OAuth 2.0 with refresh-token support.diff --git a/server/services/pricing.py b/server/services/pricing.pyindex ee4d0a65..bfa31b19 100644--- a/server/services/pricing.py+++ b/server/services/pricing.py@@ -122,16 +122,21 @@ class PricingService:Falls back to '_default' if no match found.Args:- provider: Provider name (openai, anthropic, gemini, groq, cerebras, openrouter)+ provider: Provider reference (openai, anthropic, ..., or a named+ endpoint ``openai_compatible:<slug>``, which reads the+ ``openai_compatible`` block)model: Model name or IDReturns:ModelPricing with rates per million tokens"""+ from services.llm.config import split_provider_ref+provider_lower = provider.lower()model_lower = model.lower() if model else ""+ name, endpoint_slug = split_provider_ref(provider_lower)- provider_pricing = self._llm_registry.get(provider_lower, {})+ provider_pricing = self._llm_registry.get(name, {})# Try exact match firstif model_lower in provider_pricing:@@ -147,6 +152,16 @@ class PricingService:if model_key != "_default" and model_key in model_lower:return pricing+ # A named endpoint's per-model price, recorded when it was saved+ # (from the LiteLLM table). Scoped to endpoints so every other+ # provider keeps its curated pricing.json behaviour.+ if endpoint_slug:+ from services.model_registry import get_model_registry++ info = get_model_registry().get_model_info(model or "", provider)+ if info and (info.input_price_per_mtok or info.output_price_per_mtok):+ return ModelPricing(info.input_price_per_mtok, info.output_price_per_mtok)+# Fall back to provider defaultdefault_pricing = provider_pricing.get("_default")if default_pricing:diff --git a/server/services/rlm/service.py b/server/services/rlm/service.pyindex 34000d79..3dd54a4c 100644--- a/server/services/rlm/service.py+++ b/server/services/rlm/service.py@@ -92,6 +92,18 @@ class RLMService:provider = parameters.get("provider", "openai")model = parameters.get("model", "")+ # RLM builds its own clients from constants.PROVIDER_TO_BACKEND+ # and never reads a user base URL, so any other provider (a+ # local server, a named endpoint) would silently be sent to+ # api.openai.com with that provider's key.+ from .constants import PROVIDER_TO_BACKEND++ if provider not in PROVIDER_TO_BACKEND:+ raise ValueError(+ f"The RLM Agent cannot run on provider {provider!r}. "+ f"Choose one of: {', '.join(sorted(PROVIDER_TO_BACKEND))}."+ )+if not model or not is_model_valid_for_provider(model, provider):model = await get_default_model_async(provider, database)diff --git a/server/services/settings/handlers.py b/server/services/settings/handlers.pyindex 7ad21f97..86121f80 100644--- a/server/services/settings/handlers.py+++ b/server/services/settings/handlers.py@@ -178,6 +178,23 @@ async def handle_get_validated_ai_providers(})+ # Each named endpoint is its own provider, so one can be the global+ # default (RFC-0003 D13). The bare ``openai_compatible`` id holds no+ # key, so the loop above already skipped it.+ from services.llm.endpoints import list_endpoints++ for endpoint in await list_endpoints(auth_service):+ endpoint_defaults = await database.get_provider_defaults(endpoint.ref)+ providers.append(+ {+ "provider": endpoint.ref,+ "display_name": f"{endpoint.label} (OpenAI-compatible)",+ "models": endpoint.models,+ "popular_models": [],+ "default_model": (endpoint_defaults or {}).get("default_model", ""),+ }+ )+user_id = data.get("user_id", "default")settings = await database.get_user_settings(user_id)global_provider = settings.get("default_llm_provider") if settings else Nonediff --git a/server/tests/credentials/test_openai_compatible_credential.py b/server/tests/credentials/test_openai_compatible_credential.pynew file mode 100644index 00000000..36320846--- /dev/null+++ b/server/tests/credentials/test_openai_compatible_credential.py@@ -0,0 +1,151 @@+"""RFC-0003 D13 / AG15: named OpenAI-compatible endpoints as credentials.++Adding and refreshing go through ``validate_api_key``; removing through+``delete_api_key``; the panel's list rides the catalogue. Storage is the+real encrypted credentials DB (``auth_service`` fixture) wherever rows+matter; the save path itself is covered in tests/nodes/test_local_llm_save.py.+"""++from __future__ import annotations++from types import SimpleNamespace+from unittest.mock import AsyncMock, MagicMock++import pytest++from nodes.model import _credentials as creds+from nodes.model._credentials import OpenAICompatibleCredential++pytestmark = pytest.mark.credentials++REF = "openai_compatible:home-vllm"+++def auth(auth_service, monkeypatch):+ monkeypatch.setattr("services.plugin.deps.get_auth_service", lambda: auth_service)+ return auth_service+++def save(monkeypatch) -> AsyncMock:+ fake = AsyncMock(return_value={"success": True, "valid": True})+ monkeypatch.setattr("nodes.model._local_validator.save_llm_server", fake)+ return fake+++async def _store_endpoint(auth, ref: str, url: str, key: str, label: str, models=("m1",)) -> None:+ await auth.store_api_key(provider=f"{ref}_proxy", api_key=url, models=[])+ await auth.store_api_key(+ provider=ref,+ api_key=key,+ models=list(models),+ model_params={"_endpoint": {"label": label, "base_url": url.split("@")[-1], "kind": "generic"}},+ )+++class TestAdd:+ async def test_the_label_becomes_the_slug_of_the_reference(self, auth, save):+ await OpenAICompatibleCredential.validate(+ {"api_key": "http://host:8000", "openai_compatible_label": "Home vLLM", "openai_compatible_api_key": "sk-1"}+ )++ save.assert_awaited_once_with(REF, "http://host:8000", "sk-1", display="Home vLLM", label="Home vLLM")++ async def test_without_a_label_the_slug_comes_from_the_host(self, auth, save):+ await OpenAICompatibleCredential.validate({"api_key": "http://localhost:8080"})++ ref, url, key = save.await_args.args+ assert (ref, url, key) == ("openai_compatible:localhost-8080", "http://localhost:8080", None)++ async def test_a_slug_fits_the_provider_column_with_its_url_suffix(self, auth, save):+ await OpenAICompatibleCredential.validate({"api_key": "http://h", "openai_compatible_label": "a very long endpoint label indeed"})++ ref = save.await_args.args[0]+ assert len(f"{ref}_proxy") <= 50++ async def test_an_existing_label_is_rejected_not_overwritten(self, auth, save):+ await _store_endpoint(auth, REF, "http://host:8000/v1", "sk-old", "Home vLLM")++ result = await OpenAICompatibleCredential.validate({"api_key": "http://other", "openai_compatible_label": "home vllm"})++ assert result["valid"] is False and "already exists" in result["message"]+ save.assert_not_awaited()++ async def test_a_label_with_no_letter_or_digit_is_rejected(self, auth, save):+ result = await OpenAICompatibleCredential.validate({"api_key": "http://h", "openai_compatible_label": "!!!"})++ assert result["valid"] is False+ save.assert_not_awaited()+++class TestRefresh:+ async def test_uses_the_stored_url_and_key_not_what_the_panel_sent(self, auth, save):+ await _store_endpoint(auth, REF, "http://user:pw@host:8000/v1", "sk-stored", "Home vLLM")++ await OpenAICompatibleCredential.validate({"api_key": "http://host:8000/v1", "ref": REF})++ save.assert_awaited_once_with(+ REF, "http://user:pw@host:8000/v1", "sk-stored", display="Home vLLM", label="Home vLLM"+ )++ @pytest.mark.parametrize("ref", ["openai_compatible:gone", "ollama", "openai_compatible:Bad_Slug"])+ async def test_an_unknown_or_malformed_reference_is_rejected(self, auth, save, ref):+ result = await OpenAICompatibleCredential.validate({"api_key": "http://h", "ref": ref})++ assert result["valid"] is False+ save.assert_not_awaited()+++class TestCatalogue:+ async def test_lists_saved_endpoints_and_marks_the_provider_stored(self, auth):+ await _store_endpoint(auth, REF, "http://host:8000/v1", "sk-1", "Home vLLM", models=("a", "b"))+ await _store_endpoint(auth, "openai_compatible:lab", "http://lab:8080", "sk-no-key-required", "Lab")+ await auth.store_api_key(provider="ollama", api_key="ollama", models=["x"])++ extras = await OpenAICompatibleCredential.catalogue_extras()++ assert extras["stored"] is True+ assert extras["endpoints"] == [+ {"ref": REF, "label": "Home vLLM", "base_url": "http://host:8000/v1", "kind": "generic", "model_count": 2},+ {"ref": "openai_compatible:lab", "label": "Lab", "base_url": "http://lab:8080", "kind": "generic", "model_count": 1},+ ]++ async def test_no_endpoints_means_not_stored(self, auth):+ assert await OpenAICompatibleCredential.catalogue_extras() == {"stored": False, "endpoints": []}++ async def test_the_catalogue_handler_merges_the_extras(self, auth, monkeypatch):+ import routers.websocket as ws++ await _store_endpoint(auth, REF, "http://host:8000/v1", "sk-1", "Home vLLM")+ monkeypatch.setattr(ws.container, "auth_service", lambda: auth)++ catalogue = await ws.handle_get_credential_catalogue({}, SimpleNamespace())++ entry = next(p for p in catalogue["providers"] if p["id"] == "openai_compatible")+ assert entry["stored"] is True+ assert [e["ref"] for e in entry["endpoints"]] == [REF]+++class TestDelete:+ async def test_deleting_an_endpoint_removes_its_url_row_too(self, auth, monkeypatch):+ from services.credentials import handlers++ await _store_endpoint(auth, REF, "http://host:8000/v1", "sk-1", "Home vLLM")+ broadcaster = MagicMock()+ broadcaster.update_api_key_status = AsyncMock()+ broadcaster.broadcast_credential_event = AsyncMock()+ monkeypatch.setattr(handlers.container, "auth_service", lambda: auth)+ monkeypatch.setattr(handlers, "get_status_broadcaster", lambda: broadcaster)++ await handlers.handle_delete_api_key({"provider": REF}, SimpleNamespace())++ assert await auth.get_api_key(REF) is None+ assert await auth.get_api_key(f"{REF}_proxy") is None+ broadcaster.broadcast_credential_event.assert_awaited_once()+++def test_the_credential_is_registered_for_dispatch():+ from services.plugin.credential import CREDENTIAL_REGISTRY++ assert CREDENTIAL_REGISTRY["openai_compatible"] is creds.OpenAICompatibleCredentialdiff --git a/server/tests/llm/test_plugin_shape.py b/server/tests/llm/test_plugin_shape.pyindex 6cda29d8..5ccaa14c 100644--- a/server/tests/llm/test_plugin_shape.py+++ b/server/tests/llm/test_plugin_shape.py@@ -144,19 +144,29 @@ def test_every_provider_declares_sdk_exception_types():)-def test_all_registered_providers_are_agent_selectable():- """Agent surfaces must not silently omit a native provider."""+async def test_all_registered_providers_are_agent_selectable():+ """Agent surfaces must not silently omit a native provider.- from typing import get_args+ The ``provider`` field is loader-driven (RFC-0003 D13): a saved endpoint+ cannot be a ``Literal``. Every agent Params points at the ``aiProviders``+ loader, and the loader offers every registered provider except the bare+ endpoint id, which is only selectable as a saved endpoint.+ """+ from unittest.mock import AsyncMock, patchfrom nodes.agent._specialized import SpecializedAgentParamsfrom nodes.agent.ai_agent import AIAgentParamsfrom nodes.agent.chat_agent import ChatAgentParams+ from nodes.model._option_loaders import load_ai_providers+ from services.llm.config import ENDPOINT_PROVIDER- expected = set(all_providers())for params_type in (AIAgentParams, ChatAgentParams, SpecializedAgentParams):- annotation = params_type.model_fields["provider"].annotation- assert set(get_args(annotation)) == expected+ schema = params_type.model_json_schema()["properties"]["provider"]+ assert schema.get("loadOptionsMethod") == "aiProviders", params_type.__name__++ with patch("nodes.model._option_loaders._endpoints", AsyncMock(return_value=[])):+ options = await load_ai_providers({})+ assert {option["value"] for option in options} == set(all_providers()) - {ENDPOINT_PROVIDER}# ---------------------------------------------------------------------------diff --git a/server/tests/llm/test_provider_refs.py b/server/tests/llm/test_provider_refs.pynew file mode 100644index 00000000..7abf1708--- /dev/null+++ b/server/tests/llm/test_provider_refs.py@@ -0,0 +1,196 @@+"""RFC-0003 D6/D7/D13 at call time: provider references and credentials.++A named endpoint is the provider reference ``openai_compatible:<slug>``.+Its credential rows are keyed by the full reference; everything read from+``llm_defaults.json`` resolves through the shared ``openai_compatible``+block (AG16). No path may hand an SDK an empty key (AG8).+"""++from __future__ import annotations++import importlib.util+from pathlib import Path+from unittest.mock import AsyncMock, MagicMock, patch++import pytest+import respx++import services.llm # noqa: F401 — populate the registry+from services.llm.config import (+ ENDPOINT_PROVIDER,+ curated_models,+ endpoint_ref,+ get_default_model,+ is_model_valid_for_provider,+ resolve_credential,+ split_provider_ref,+ supports_model_listing,+)+from services.llm.protocol import LLMResponse, Message+from services.llm.registry import ProviderSpec, all_providers+from services.llm.unifier import ChatUnifier+from services.plugin import NodeUserError++REF = "openai_compatible:home"+SERVER_DIR = Path(__file__).resolve().parents[2]+++def _unifier(get_api_key) -> ChatUnifier:+ auth = MagicMock()+ auth.get_api_key = get_api_key+ return ChatUnifier(defaults={"providers": {}}, auth_service=auth)+++async def _chat(unifier: ChatUnifier, provider: str, api_key: str = "k"):+ return await unifier.chat(+ provider=provider,+ api_key=api_key,+ messages=[Message(role="user", content="hi")],+ model="m",+ )+++class TestReferences:+ def test_split_and_build_round_trip(self):+ assert split_provider_ref("ollama") == ("ollama", "")+ assert split_provider_ref(REF) == (ENDPOINT_PROVIDER, "home")+ assert endpoint_ref("home") == REF+ assert split_provider_ref("") == ("", "")+++class TestResolveCredential:+ def test_a_stored_key_wins(self):+ assert resolve_credential("ollama", "real-key") == "real-key"++ @pytest.mark.parametrize(+ ("provider", "placeholder"),+ [("ollama", "ollama"), ("lmstudio", "lm-studio"), (REF, "sk-no-key-required")],+ )+ def test_keyless_servers_get_the_placeholder_their_vendor_documents(self, provider, placeholder):+ assert resolve_credential(provider, None) == placeholder+ assert resolve_credential(provider, "") == placeholder++ @pytest.mark.parametrize("provider", ["openai", "anthropic", "deepseek"])+ def test_no_key_and_no_placeholder_is_an_error_not_none(self, provider):+ with pytest.raises(ValueError):+ resolve_credential(provider, None)+++class TestReferencesReadTheSharedBlock:+ def test_default_model_is_not_a_cloud_fallback(self):+ # An unknown provider would get the "gpt-5.2" literal.+ assert get_default_model(REF) == ""++ def test_listing_is_supported_and_nothing_is_curated(self):+ assert supports_model_listing(REF) is True+ assert curated_models(REF) == []++ @pytest.mark.parametrize("provider", [REF, "ollama", "lmstudio", "openrouter", "groq"])+ def test_open_world_providers_accept_any_model_id(self, provider):+ assert is_model_valid_for_provider("owner/any-model:7b", provider) is True++ def test_closed_world_providers_still_pattern_check(self):+ assert is_model_valid_for_provider("gpt-5.6-sol", "openai") is True+ assert is_model_valid_for_provider("llama3", "openai") is False+++class TestRegistryFallbacks:+ @pytest.fixture+ def registry(self, tmp_path, monkeypatch):+ import services.model_registry as mr++ monkeypatch.setattr(mr, "_local_models_path", lambda: tmp_path / "local_models.json")+ svc = mr.ModelRegistryService()+ svc._load_llm_defaults()+ return svc++ def test_an_unsized_endpoint_model_gets_the_conservative_defaults(self, registry):+ assert registry.get_context_length("unknown-model", REF) == 8192+ assert registry.get_max_output_tokens("unknown-model", REF) == 2048+ assert registry.get_temperature_range("unknown-model", REF) == (0.0, 2.0)++ def test_a_model_registered_at_save_time_wins(self, registry):+ registry.register_local_model(REF, "qwen3", {"context_length": 32768})++ assert registry.get_context_length("qwen3", REF) == 32768+ # Another endpoint serving the same id is a different server.+ assert registry.get_context_length("qwen3", "openai_compatible:other") == 8192+++class TestPricing:+ @pytest.fixture+ def pricing(self):+ # tests/conftest.py stubs services.pricing; load the real module.+ spec = importlib.util.spec_from_file_location("_real_pricing", SERVER_DIR / "services" / "pricing.py")+ module = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(module)+ return module.PricingService()++ def test_an_endpoint_is_free_by_default(self, pricing):+ price = pricing.get_pricing(REF, "qwen3")+ assert (price.input_per_mtok, price.output_per_mtok) == (0.0, 0.0)++ def test_an_endpoint_uses_the_price_recorded_when_it_was_saved(self, pricing):+ info = MagicMock(input_price_per_mtok=3.0, output_price_per_mtok=15.0)+ registry = MagicMock()+ registry.get_model_info.return_value = info+ with patch("services.model_registry.get_model_registry", return_value=registry):+ price = pricing.get_pricing(REF, "gpt-4o")+ assert (price.input_per_mtok, price.output_per_mtok) == (3.0, 15.0)+ registry.get_model_info.assert_called_once_with("gpt-4o", REF)++ def test_other_providers_keep_their_curated_prices(self, pricing):+ price = pricing.get_pricing("openai", "gpt-6-astra")+ assert (price.input_per_mtok, price.output_per_mtok) == (10.0, 50.0)+++class TestUnifierGuards:+ async def test_no_request_leaves_without_a_key(self, monkeypatch):+ """AG8: the SDK would read OPENAI_API_KEY and send it to the base URL."""+ monkeypatch.setenv("OPENAI_API_KEY", "sk-operator-secret")+ unifier = _unifier(AsyncMock(return_value=None))+ keyless = {name for name in all_providers() if name in ("ollama", "lmstudio", ENDPOINT_PROVIDER)}++ with respx.mock(assert_all_called=False) as mock:+ for provider in sorted(set(all_providers()) - keyless):+ with pytest.raises(NodeUserError, match="No API key"):+ await _chat(unifier, provider, api_key="")+ assert mock.calls.call_count == 0++ async def test_an_endpoint_without_its_url_row_is_not_configured(self):+ factory = MagicMock()+ spec = ProviderSpec(name=ENDPOINT_PROVIDER, factory=factory, sdk_exception_refs=("openai:OpenAIError",))+ unifier = _unifier(AsyncMock(return_value=None))++ with patch("services.llm.unifier.get_provider", return_value=spec):+ with pytest.raises(NodeUserError, match="'home' is not configured"):+ await _chat(unifier, REF)+ factory.assert_not_called()++ async def test_an_endpoint_reads_its_own_url_row_and_placeholder(self):+ looked_up = []++ async def get_api_key(name, session_id="default"):+ looked_up.append(name)+ return "http://host:8080/v1" if name == f"{REF}_proxy" else None++ client = MagicMock()+ client.chat = AsyncMock(return_value=LLMResponse(content="ok"))+ factory = MagicMock(return_value=client)+ spec = ProviderSpec(+ name=ENDPOINT_PROVIDER,+ factory=factory,+ sdk_exception_refs=("openai:OpenAIError",),+ client_kwargs={"provider_name": ENDPOINT_PROVIDER},+ )+ unifier = _unifier(get_api_key)++ with patch("services.llm.unifier.get_provider", return_value=spec) as lookup:+ await _chat(unifier, REF, api_key="")++ lookup.assert_called_once_with(ENDPOINT_PROVIDER)+ assert f"{REF}_proxy" in looked_up+ kwargs = factory.call_args.kwargs+ assert kwargs["proxy_url"] == "http://host:8080/v1"+ assert kwargs["api_key"] == "sk-no-key-required"+ assert kwargs["provider_name"] == ENDPOINT_PROVIDERdiff --git a/server/tests/llm/test_provider_self_registration.py b/server/tests/llm/test_provider_self_registration.pyindex 6c4a1006..92bd113b 100644--- a/server/tests/llm/test_provider_self_registration.py+++ b/server/tests/llm/test_provider_self_registration.py@@ -36,6 +36,8 @@ EXPECTED_REGISTERED_PROVIDERS = {# Indic-first; OpenAI-compatible chat endpoint but no model-list route,# which it declares via `supports_model_listing: false`."sarvam",+ # The one registration behind every user-named endpoint (RFC-0003 D13).+ "openai_compatible",}COMPAT_PROVIDERS = ("xai",@@ -186,6 +188,22 @@ def test_every_compat_provider_inherits_shared_response_and_tool_contract(assert spec.sdk_exception_refs == ("openai:OpenAIError",)+def test_endpoint_provider_is_registered_once_without_a_base_url():+ """RFC-0003 AG17: named endpoints share one OpenAIProvider registration.++ It must not pin a ``base_url``: each endpoint's resolved URL lives on its+ own ``openai_compatible:<slug>_proxy`` row, and a pinned default would be+ used silently when that row is missing.+ """+ from services.llm.providers.openai import OpenAIProvider++ spec = get_provider("openai_compatible")+ assert spec.factory is OpenAIProvider+ assert spec.client_kwargs == {"provider_name": "openai_compatible"}+ assert spec.sdk_exception_refs == ("openai:OpenAIError",)+ assert list(all_providers()).count("openai_compatible") == 1++def test_unknown_provider_raises_node_user_error():"""Unknown provider lookups surface as NodeUserError (user-correctable)."""from services.plugin import NodeUserErrordiff --git a/server/tests/nodes/test_openai_compatible_node.py b/server/tests/nodes/test_openai_compatible_node.pynew file mode 100644index 00000000..cf31e306--- /dev/null+++ b/server/tests/nodes/test_openai_compatible_node.py@@ -0,0 +1,102 @@+"""RFC-0003 D13 on the node side: choosing a named endpoint.++Agents carry it in their ``provider`` field; the OpenAI-compatible chat-model+node in its ``endpoint`` field. Both resolve to the provider reference the+runtime keys everything by. RLM, which builds its own clients, refuses what+it cannot route instead of sending it to api.openai.com.+"""++from __future__ import annotations++from unittest.mock import AsyncMock, patch++import pytest+from pydantic import ValidationError++import services.llm # noqa: F401 — populate the registry+from constants import AI_CHAT_MODEL_TYPES, detect_ai_provider+from nodes.agent.ai_agent import AIAgentParams+from nodes.model._option_loaders import load_ai_providers, load_endpoint_models, load_endpoints+from nodes.model.openai_compatible_chat_model import OpenAICompatibleChatModelNode+from services.llm.endpoints import SavedEndpoint++REF = "openai_compatible:home"+HOME = SavedEndpoint(ref=REF, label="Home vLLM", base_url="http://host:8000/v1", kind="generic", models=["a", "b"])+++class TestProviderRef:+ @pytest.mark.parametrize("value", ["openai", "ollama", REF])+ def test_registered_providers_and_named_endpoints_are_accepted(self, value):+ assert AIAgentParams(provider=value).provider == value++ @pytest.mark.parametrize("value", ["openai_compatible", "nope", "ollama:home", ""])+ def test_a_bare_endpoint_id_or_unknown_provider_is_rejected(self, value):+ with pytest.raises(ValidationError):+ AIAgentParams(provider=value)+++class TestDetectProvider:+ def test_agents_carry_the_reference_in_their_provider_field(self):+ assert detect_ai_provider("aiAgent", {"provider": REF}) == REF++ def test_the_chat_model_node_carries_it_in_its_endpoint_field(self):+ assert "openaiCompatibleChatModel" in AI_CHAT_MODEL_TYPES+ assert detect_ai_provider("openaiCompatibleChatModel", {"endpoint": REF}) == REF++ def test_no_endpoint_yields_the_bare_id_which_the_unifier_refuses(self):+ assert detect_ai_provider("openaiCompatibleChatModel", {}) == "openai_compatible"+++class TestLoaders:+ async def test_providers_then_endpoints_in_declared_order(self):+ with patch("nodes.model._option_loaders._endpoints", AsyncMock(return_value=[HOME])):+ options = await load_ai_providers({})++ values = [o["value"] for o in options]+ assert values[0] == "openai" # llm_defaults.json order, not import order+ assert "openai_compatible" not in values+ assert options[-1] == {"name": "Home vLLM (OpenAI-compatible)", "value": REF}++ async def test_the_node_lists_endpoints_only(self):+ with patch("nodes.model._option_loaders._endpoints", AsyncMock(return_value=[HOME])):+ assert await load_endpoints({}) == [{"name": "Home vLLM", "value": REF}]++ async def test_models_come_from_the_selected_endpoint(self, monkeypatch):+ auth = AsyncMock()+ auth.get_stored_models = AsyncMock(return_value=["a", "b"])+ monkeypatch.setattr("services.plugin.deps.get_auth_service", lambda: auth)++ assert await load_endpoint_models({"endpoint": REF}) == [+ {"name": "a", "value": "a"},+ {"name": "b", "value": "b"},+ ]+ auth.get_stored_models.assert_awaited_once_with(REF)+ assert await load_endpoint_models({"endpoint": "ollama"}) == []+ assert await load_endpoint_models({}) == []+++class TestChatModelNode:+ def test_the_endpoint_is_chosen_first_and_models_follow_it(self):+ schema = OpenAICompatibleChatModelNode.Params.model_json_schema()["properties"]++ assert list(schema)[0] == "endpoint"+ assert schema["endpoint"]["loadOptionsMethod"] == "openaiCompatibleEndpoints"+ assert schema["model"]["loadOptionsMethod"] == "openaiCompatibleModels"+ assert schema["model"]["loadOptionsDependsOn"] == ["endpoint"]+ # A `provider` sibling of `model` would trigger the panel's stored-key effect.+ assert "provider" not in schema++ def test_it_stays_single_purpose(self):+ assert OpenAICompatibleChatModelNode.usable_as_tool is False+ assert "tool" not in OpenAICompatibleChatModelNode.group+++class TestRlmGuard:+ @pytest.mark.parametrize("provider", [REF, "ollama"])+ async def test_rlm_refuses_a_provider_it_cannot_route(self, provider):+ from services.rlm.service import RLMService++ result = await RLMService(auth=None).execute("node-1", {"provider": provider, "model": "m", "prompt": "hi"})++ assert result["success"] is False+ assert "cannot run on provider" in result["error"]