Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

feat(gateway): Azure and key-in-URL upstreams on top of exact endpoints (#6101)

t0saki8483463c4241
diff --git a/docs/en/api/25-gateway.md b/docs/en/api/25-gateway.md
index fc91ac102..de8a4a490 100644
--- a/docs/en/api/25-gateway.md
+++ b/docs/en/api/25-gateway.md
@@ -96,11 +96,12 @@ Deleting an object that gateway keys still use returns 409 `in_use` with `key_co
The response includes `{ok, kind, status, reason, url}`, where `kind` is `connection` or `model`. Failed tests also include the upstream HTTP response in `response`, with `http_version`, `status`, `reason`, `headers`, `body` and an optional `truncated` flag. The body is limited to 16 KiB; `status` is left out when no response arrived. The timeout is the smaller of the configured upstream timeout and 60 seconds. Running either test requires write access; model tests make a real provider request and may incur a small charge.
-Anthropic tests send the key in both `x-api-key` and `Authorization: Bearer` and default `anthropic-version` to `2023-06-01` when the upstream headers do not set it. Chat Completions and Responses tests send only Bearer authentication.
+Anthropic tests send the key in both `x-api-key` and `Authorization: Bearer` and default `anthropic-version` to `2023-06-01` when the upstream headers do not set it. Chat Completions and Responses tests send only Bearer authentication. A `key_header` other than `authorization`, or `{api_key}` in `base_url`, changes this as described in [Credentials](../guides/22-gateway-operations.md#credentials); `url` keeps the `{api_key}` placeholder.
| Provider response | `reason` | `ok` |
| --- | --- | --- |
| The Coding Plan upstream has not been approved for gateway use. No request is sent. | `coding_plan_blocked` | `false` |
+| A connection test of an exact endpoint (`append_endpoint_path` is `false`), which has no model list. No request is sent; run a model test instead. | `exact_endpoint_untestable` | `false` |
| No API key to use: none is stored, or the upstream expects each client to send its own. No request is sent. | `upstream_key_missing` | `false` |
| 2xx with a response matching the configured model protocol | `passed` | `true` |
| 2xx with invalid JSON or a response that does not match the configured protocol | `invalid_response` | `false` |
@@ -115,7 +116,7 @@ A passed connection test confirms reachability and, when `/v1/models` is availab
`POST /gateway/admin/upstreams/import-vlm` turns OpenViking Server's VLM credentials into upstreams in the selected account. It is available only in embedded deployments; a standalone gateway returns 404 `vlm_import_unavailable`.
-Each imported upstream gets the fixed ID `vlm-<credential id>`, so importing again does not create duplicates. A repeat import refreshes the fields that come from the server (`base_url`, `api_key`, `headers`, `protocol`, `vendor`, `coding_plan`) and keeps every other field as it was edited in the gateway, including `name`, `enabled`, `priority`, `models`, `aliases`, `context_windows` and `allow_coding_plan`. Upstreams whose path has a `coding` or `plan` segment are marked as Coding Plan and listed under `needs_confirmation` until `allow_coding_plan` is set.
+Each imported upstream gets the fixed ID `vlm-<credential id>`, so importing again does not create duplicates. A repeat import refreshes the fields that come from the server (`base_url`, `api_key`, `headers`, `protocol`, `vendor`, `coding_plan`, `append_endpoint_path`, `key_header`) and keeps every other field as it was edited in the gateway, including `name`, `enabled`, `priority`, `models`, `aliases`, `context_windows` and `allow_coding_plan`. Upstreams whose path has a `coding` or `plan` segment are marked as Coding Plan and listed under `needs_confirmation` until `allow_coding_plan` is set.
```json
{
@@ -128,7 +129,6 @@ Each imported upstream gets the fixed ID `vlm-<credential id>`, so importing aga
| Skip `reason` | Meaning |
| --- | --- |
| `openai_codex` | Codex OAuth tokens refresh periodically, so the gateway cannot use them long term. |
-| `azure` | Azure OpenAI needs an `api-version` parameter, which the gateway does not send. |
| `litellm_no_api_base` | The LiteLLM credential has no `api_base`. Add it in the OpenViking configuration and import again. |
| `litellm_unsupported_route` | The LiteLLM route does not use the OpenAI Chat Completions API, such as `gemini/`, `bedrock/` or `vertex_ai/`. Add that provider as an upstream by hand. |
| `invalid_configuration` | The credential does not make a valid upstream, for example because the base URL is invalid. |
diff --git a/docs/en/guides/22-gateway-operations.md b/docs/en/guides/22-gateway-operations.md
index a6dddac11..b1f6f5cb5 100644
--- a/docs/en/guides/22-gateway-operations.md
+++ b/docs/en/guides/22-gateway-operations.md
@@ -256,9 +256,11 @@ When you choose a provider in the editor, its default Base URL is filled in:
- Chat Completions and Responses append `/chat/completions` or `/responses`. A bare domain does not get `/v1` added, so OpenAI needs `https://api.openai.com/v1`.
- Anthropic Messages appends `/v1/messages`, so the Base URL must not end in `/v1`; saving such a URL is rejected.
- A trailing slash is removed. Addresses with a version path, such as `/api/paas/v4`, work as is.
+ - The Base URL may carry a query, such as `?api-version=…`. The client's query parameters follow it, and on a name clash the Base URL's value wins.
- Requests that clients send to the Ark paths (`/gateway/api/v3/`, `/gateway/api/compatible/v1/`) follow the same rules.
+ - Azure OpenAI's v1 API uses `https://RES.openai.azure.com/openai/v1`, Claude in Microsoft Foundry uses the Anthropic Messages protocol with `https://RES.services.ai.azure.com/anthropic`, and Gemini's OpenAI-compatible API uses `https://generativelanguage.googleapis.com/v1beta/openai`; all of them follow these joining rules.
- An upstream has one Base URL. If a provider uses different addresses for different protocols, create separate upstreams. For example, Zhipu uses `https://open.bigmodel.cn/api/paas/v4` for Chat Completions and `https://open.bigmodel.cn/api/v1` for Responses. Ark Coding Plan uses `…/api/coding/v3` for Chat Completions and Responses and `…/api/coding` for Anthropic Messages.
-- **Append protocol endpoint path** is on by default. Turn it off when Base URL is already the complete request endpoint; the gateway then sends every upstream request to that exact URL without adding a protocol path.
+- **Append protocol endpoint path** is on by default. Turn it off when Base URL is already the complete request endpoint; the gateway then sends the protocol's requests to that exact URL without adding a protocol path. Sub-paths continue after it: Anthropic token counting goes to `…/count_tokens`. Other APIs, such as embeddings and the model list, are not routed to such an upstream, and the connection test skips it. For example, an Azure OpenAI deployment uses protocol *Chat Completions*, provider *Generic*, Base URL `https://RES.openai.azure.com/openai/deployments/DEPLOYMENT/chat/completions?api-version=2024-10-21`, this setting off, and **Send the key as** *api-key header*.
The editor previews the final request URL as you type. If an OpenAI or Ark address is missing its path, or an Anthropic address ends in `/v1`, it warns you and offers a one-click fix.
@@ -266,7 +268,9 @@ When you choose a provider in the editor, its default Base URL is filled in:
- **Who provides the API key**:
- **The gateway holds the API key** (the default): the gateway sends the upstream's API key, and clients need only their gateway key.
- - **Each client sends its own key**: besides the gateway key, every request carries the client's own provider API key in the `X-OpenViking-Upstream-Key` header. Without it the request fails with 401 "Upstream API key is missing". The gateway passes the key to the provider in the provider's usual header; `X-OpenViking-*` headers themselves are never forwarded.
+ - **Each client sends its own key**: besides the gateway key, every request carries the client's own provider API key in the `X-OpenViking-Upstream-Key` header. Without it the request fails with 401 "Upstream API key is missing". The gateway passes the key on as set in **Send the key as** (or fills it into the URL when the Base URL contains `{api_key}`); `X-OpenViking-*` headers themselves are never forwarded.
+- **Send the key as** (provider *Generic* only): `Authorization: Bearer` (the default), the `api-key` header (Azure OpenAI) or the `x-api-key` header.
+- **Key in the URL**: for a provider that expects the key in the address, write `{api_key}` in the Base URL's path or query, for example `?key={api_key}`. The gateway fills in the URL-encoded key when sending and then sends no auth header. Don't write the key itself into the address: the Base URL is visible to everyone who can view upstreams.
- **API key and extra headers are write-only.** Leave a field blank to keep the stored value; remove a header row to delete that header. **Changing the Base URL clears the stored API key and header values**, so enter them again whenever you change the address. A stored key is never sent to a new address.
- **Extra headers** are added to every request to this upstream, such as a provider's version or organization header, and replace any client header with the same name. `Host`, `Content-Length`, `Transfer-Encoding` and `Connection` cannot be set.
- **This key is a Coding Plan or subscription key**: providers usually allow subscription keys such as Coding Plan keys only in their own coding tools, and using one through a shared gateway can get the subscription suspended. When you check this, every request to the upstream is rejected with 403 "Coding Plan upstreams are disabled; configure a model API key", unless you also check **Allow this plan through the gateway**. The gateway cannot tell from the key itself whether it is a subscription key, so you have to mark it. Claude subscription tokens (`sk-ant-oat…`) are always rejected.
@@ -299,7 +303,8 @@ In an embedded deployment, **Use OpenViking's model settings** (**Import from Op
- The result shows three groups: imported, needs confirmation and not imported. Upstreams whose path contains `coding` or `plan` are marked as Coding Plan. Once you have confirmed that the provider's terms allow it, check **Allow this plan through the gateway** to use one.
- Importing again does not create duplicates. It refreshes the address, API key, headers, protocol and provider from the server, and keeps what you changed in the gateway: name, enabled state, priority, models, aliases, context windows and the Coding Plan confirmation.
-- Codex OAuth and Azure credentials, LiteLLM credentials without `api_base`, and LiteLLM routes that do not use the OpenAI Chat Completions API are skipped. The reasons are listed in [Import VLM settings](../api/25-gateway.md#import-vlm-settings).
+- Azure OpenAI credentials (provider `azure`, or LiteLLM `azure/<deployment>`) become exact-endpoint upstreams that send the key in the `api-key` header, with the configured `api_version`.
+- Codex OAuth credentials, LiteLLM credentials without `api_base`, and LiteLLM routes that do not use the OpenAI Chat Completions API are skipped. The reasons are listed in [Import VLM settings](../api/25-gateway.md#import-vlm-settings).
### Context profiles
@@ -731,6 +736,7 @@ URLs must be HTTP(S) and contain no credentials, query or fragment. Unknown fiel
| Provider | `vendor` | `generic` | `generic`, `anthropic`, `openai`, `deepseek`, `ark` (Volcano Engine Ark) or `byteplus` (BytePlus ModelArk). Studio offers only the protocols a provider supports; the API accepts any pair. |
| Base URL | `base_url` | | Provider address; defaults and path rules are in [Upstreams](#upstreams). |
| Append protocol endpoint path | `append_endpoint_path` | `true` | Appends the selected protocol's path; when false, uses `base_url` as the complete request URL. |
+| Send the key as | `key_header` | `authorization` | `authorization` (Bearer), `api-key` or `x-api-key`. Ignored when `base_url` contains `{api_key}`. |
| Who provides the API key | `auth_mode` | `managed` | `managed` (the gateway holds the API key) or `passthrough` (each client sends its own key). |
| API key | `api_key` | | Write-only; see [Credentials](#credentials). |
| Extra headers | `headers` | `{}` | Write-only; see [Credentials](#credentials). |
diff --git a/docs/zh/api/25-gateway.md b/docs/zh/api/25-gateway.md
index e454af42e..25999c64d 100644
--- a/docs/zh/api/25-gateway.md
+++ b/docs/zh/api/25-gateway.md
@@ -96,11 +96,12 @@ curl -X PUT "$SERVER_URL/gateway/admin/upstreams/openai?account_id=team" \
响应包含 `{ok, kind, status, reason, url}`,其中 `kind` 为 `connection` 或 `model`。测试失败时还会在 `response` 中返回完整的上游 HTTP 响应,包括 `http_version`、`status`、`reason`、`headers`、`body` 和可选的 `truncated` 标记。响应体最多 16 KiB;没有收到响应时不含 `status`。超时时间取已配置上游超时与 60 秒中的较小值。两种测试都需要写权限;模型测试会真实请求服务商并可能产生少量费用。
-Anthropic 测试同时在 `x-api-key` 和 `Authorization: Bearer` 中发送密钥;上游请求头未设置 `anthropic-version` 时,默认使用 `2023-06-01`。Chat Completions 和 Responses 测试只发送 Bearer 鉴权。
+Anthropic 测试同时在 `x-api-key` 和 `Authorization: Bearer` 中发送密钥;上游请求头未设置 `anthropic-version` 时,默认使用 `2023-06-01`。Chat Completions 和 Responses 测试只发送 Bearer 鉴权。`key_header` 不是 `authorization`,或者 `base_url` 含 `{api_key}` 时,按[凭证](../guides/22-gateway-operations.md#凭证)中的规则发送;返回的 `url` 保留 `{api_key}` 占位符。
| 服务商的响应 | `reason` | `ok` |
| --- | --- | --- |
| Coding Plan 上游尚未授权网关使用。网关不发请求。 | `coding_plan_blocked` | `false` |
+| 对完整请求地址(`append_endpoint_path` 为 `false`)做连接测试:这种上游没有模型列表,网关不发请求,请改用模型测试。 | `exact_endpoint_untestable` | `false` |
| 没有可用的 API Key:没有保存,或者上游要求每个客户端自带。网关不发请求。 | `upstream_key_missing` | `false` |
| 2xx,响应符合已配置的模型协议 | `passed` | `true` |
| 2xx,但响应不是合法 JSON 或不符合已配置的协议 | `invalid_response` | `false` |
@@ -115,7 +116,7 @@ Anthropic 测试同时在 `x-api-key` 和 `Authorization: Bearer` 中发送密
`POST /gateway/admin/upstreams/import-vlm` 把 OpenViking Server 的 VLM 凭证导入为所选账号的上游,只在嵌入部署中可用;独立网关返回 404 `vlm_import_unavailable`。
-每个导入的上游 ID 固定为 `vlm-<credential id>`,重复导入不会产生重复的上游。再次导入时,来自服务端的字段(`base_url`、`api_key`、`headers`、`protocol`、`vendor`、`coding_plan`)按服务端配置刷新,其他字段保留在网关里改过的值,包括 `name`、`enabled`、`priority`、`models`、`aliases`、`context_windows` 和 `allow_coding_plan`。路径中含 `coding` 或 `plan` 段的上游会被标为 Coding Plan,在设置 `allow_coding_plan` 之前归入 `needs_confirmation`。
+每个导入的上游 ID 固定为 `vlm-<credential id>`,重复导入不会产生重复的上游。再次导入时,来自服务端的字段(`base_url`、`api_key`、`headers`、`protocol`、`vendor`、`coding_plan`、`append_endpoint_path`、`key_header`)按服务端配置刷新,其他字段保留在网关里改过的值,包括 `name`、`enabled`、`priority`、`models`、`aliases`、`context_windows` 和 `allow_coding_plan`。路径中含 `coding` 或 `plan` 段的上游会被标为 Coding Plan,在设置 `allow_coding_plan` 之前归入 `needs_confirmation`。
```json
{
@@ -128,7 +129,6 @@ Anthropic 测试同时在 `x-api-key` 和 `Authorization: Bearer` 中发送密
| 跳过原因 `reason` | 说明 |
| --- | --- |
| `openai_codex` | Codex OAuth 令牌会定期刷新,网关无法长期使用。 |
-| `azure` | Azure OpenAI 需要 `api-version` 参数,网关不发送它。 |
| `litellm_no_api_base` | LiteLLM 凭证没有 `api_base`。在 OpenViking 配置中补上后重新导入。 |
| `litellm_unsupported_route` | LiteLLM 路由不使用 OpenAI Chat Completions API,例如 `gemini/`、`bedrock/` 或 `vertex_ai/`。请手动把这个服务商添加为上游。 |
| `invalid_configuration` | 凭证构不成有效的上游,例如 base URL 无效。 |
diff --git a/docs/zh/guides/22-gateway-operations.md b/docs/zh/guides/22-gateway-operations.md
index 41fcf0d5f..923c088a5 100644
--- a/docs/zh/guides/22-gateway-operations.md
+++ b/docs/zh/guides/22-gateway-operations.md
@@ -256,9 +256,11 @@ ov.example.com {
- Chat Completions 和 Responses 拼接 `/chat/completions` 或 `/responses`。只填域名时不会自动补 `/v1`,OpenAI 要写 `https://api.openai.com/v1`。
- Anthropic Messages 拼接 `/v1/messages`,所以 Base URL 不能以 `/v1` 结尾,否则保存时被拒绝。
- 末尾的斜杠会被去掉。`/api/paas/v4` 这类带版本路径的地址可以直接使用。
+ - Base URL 可以带查询参数,例如 `?api-version=…`。客户端的查询参数接在后面,同名参数以 Base URL 里的为准。
- 客户端用方舟路径(`/gateway/api/v3/`、`/gateway/api/compatible/v1/`)发来的请求,按同样的规则拼接。
+ - Azure OpenAI v1 接口填 `https://RES.openai.azure.com/openai/v1`,Microsoft Foundry 上的 Claude 用 Anthropic Messages 协议、填 `https://RES.services.ai.azure.com/anthropic`,Gemini 的 OpenAI 兼容接口填 `https://generativelanguage.googleapis.com/v1beta/openai`,都按拼接规则使用。
- 一个上游只有一个 Base URL。同一服务商的不同协议地址不同时,要分别建上游:例如智谱 Chat Completions 用 `https://open.bigmodel.cn/api/paas/v4`,Responses 用 `https://open.bigmodel.cn/api/v1`。方舟 Coding Plan 的 Chat Completions 和 Responses 用 `…/api/coding/v3`,Anthropic Messages 用 `…/api/coding`。
-- **拼接协议接口路径**默认开启。如果 Base URL 已经是完整的请求地址,可以关闭它;关闭后,网关的所有上游请求都直接使用该 URL,不再拼接协议路径。
+- **拼接协议接口路径**默认开启。如果 Base URL 已经是完整的请求地址,可以关闭它;关闭后,网关把该协议的请求直接发往这个 URL,不再拼接协议路径。子路径接在后面:Anthropic 的 Token 计数发往 `…/count_tokens`。Embeddings、模型列表等其他接口不会路由到这样的上游,连接测试也会跳过它。例如 Azure OpenAI 部署:协议选 *Chat Completions*,服务商选*通用*,Base URL 填 `https://RES.openai.azure.com/openai/deployments/DEPLOYMENT/chat/completions?api-version=2024-10-21`,关闭这个开关,**密钥发送方式**选 *api-key 请求头*。
编辑页会实时预览最终的请求地址;OpenAI、方舟这类地址漏掉路径,或者 Anthropic 地址多了 `/v1` 时,会给出提示和一键修正。
@@ -266,7 +268,9 @@ ov.example.com {
- **API Key 由谁提供**:
- **由网关保管 API Key**(默认):网关发送上游的 API Key,客户端只需要网关密钥。
- - **每个客户端自带 API Key**:每个请求除了网关密钥,还要在 `X-OpenViking-Upstream-Key` 请求头里带上客户端自己的服务商 API Key,缺少时返回 401 "Upstream API key is missing"。网关按服务商惯用的请求头转发这个 API Key,`X-OpenViking-*` 请求头本身从不转发。
+ - **每个客户端自带 API Key**:每个请求除了网关密钥,还要在 `X-OpenViking-Upstream-Key` 请求头里带上客户端自己的服务商 API Key,缺少时返回 401 "Upstream API key is missing"。网关按**密钥发送方式**转发这个 API Key(Base URL 含 `{api_key}` 时填进地址),`X-OpenViking-*` 请求头本身从不转发。
+- **密钥发送方式**(仅服务商为*通用*时):`Authorization: Bearer`(默认)、`api-key` 请求头(Azure OpenAI)或 `x-api-key` 请求头。
+- **密钥放在地址里**:服务商要求把密钥写在地址里时,在 Base URL 的路径或查询参数中写 `{api_key}`,例如 `?key={api_key}`。网关发送时填入 URL 编码后的密钥,并且不再发送鉴权请求头。不要把密钥本身写进地址:能查看上游的人都能看到 Base URL。
- **API Key 和额外请求头只写不读**:留空表示保留已存的值,删除请求头行就删除这个请求头。**修改 Base URL 会清除已存的 API Key 和请求头值**,所以每次改地址都要重新填写,已存的密钥不会被发往新地址。
- **额外请求头**加在发往这个上游的每个请求上,例如服务商的版本或组织请求头,并覆盖客户端发送的同名请求头。不能设置 `Host`、`Content-Length`、`Transfer-Encoding` 和 `Connection`。
- **这是 Coding Plan 或订阅密钥**:服务商通常只允许 Coding Plan 这类订阅密钥用在自家的编程工具里,通过共享网关使用可能导致订阅被封禁。勾选后,发往这个上游的请求一律被拒绝,返回 403 "Coding Plan upstreams are disabled; configure a model API key",除非同时勾选**允许通过网关使用这个套餐**。网关无法从密钥本身分辨它是不是订阅密钥,需要你自己标记。Claude 订阅令牌(`sk-ant-oat…`)始终被拒绝。
@@ -299,7 +303,8 @@ ov.example.com {
- 结果分三组显示:已导入、需要确认、未导入。路径中带 `coding` 或 `plan` 的上游被标为 Coding Plan,确认服务商条款允许后,勾选**允许通过网关使用这个套餐**才能使用。
- 重复导入不会产生重复的上游。再次导入时,地址、API Key、请求头、协议和服务商按服务端配置刷新;你在网关里改过的名称、启用状态、优先级、模型、别名、上下文窗口和 Coding Plan 确认都保留。
-- Codex OAuth 和 Azure 凭证、缺少 `api_base` 的 LiteLLM 凭证,以及不使用 OpenAI Chat Completions API 的 LiteLLM 路由会被跳过,原因见[导入 VLM 配置](../api/25-gateway.md#导入-vlm-配置)。
+- Azure OpenAI 凭证(服务商 `azure`,或 LiteLLM 的 `azure/<deployment>`)会导入为完整请求地址的上游,带上配置的 `api_version`,密钥放在 `api-key` 请求头里发送。
+- Codex OAuth 凭证、缺少 `api_base` 的 LiteLLM 凭证,以及不使用 OpenAI Chat Completions API 的 LiteLLM 路由会被跳过,原因见[导入 VLM 配置](../api/25-gateway.md#导入-vlm-配置)。
### 上下文配置
@@ -731,6 +736,7 @@ URL 必须是 HTTP(S),不能含凭证、查询参数或 fragment。未知字
| 服务商 | `vendor` | `generic` | `generic`、`anthropic`、`openai`、`deepseek`、`ark`(火山方舟)或 `byteplus`(BytePlus 方舟)。Studio 只提供服务商支持的协议,API 接受任意组合。 |
| Base URL | `base_url` | | 服务商地址;默认地址和拼接规则见[上游](#上游)。 |
| 拼接协议接口路径 | `append_endpoint_path` | `true` | 拼接所选协议的接口路径;关闭时把 `base_url` 直接作为完整请求地址。 |
+| 密钥发送方式 | `key_header` | `authorization` | `authorization`(Bearer)、`api-key` 或 `x-api-key`。`base_url` 含 `{api_key}` 时不生效。 |
| API Key 由谁提供 | `auth_mode` | `managed` | `managed`(由网关保管 API Key)或 `passthrough`(每个客户端自带 API Key)。 |
| API Key | `api_key` | | 只写,见[凭证](#凭证)。 |
| 额外请求头 | `headers` | `{}` | 只写,见[凭证](#凭证)。 |
diff --git a/openviking/server/gateway_vlm.py b/openviking/server/gateway_vlm.py
index 352b0cb43..4d6e95e8c 100644
--- a/openviking/server/gateway_vlm.py
+++ b/openviking/server/gateway_vlm.py
@@ -68,11 +68,22 @@ def vlm_upstreams(config: VLMConfig) -> dict:
headers = dict(settings.get("extra_headers") or {})
route = model.lower()
protocol = "chat"
+ exact = False
reason = None
if provider == "openai-codex":
reason = "openai_codex"
elif provider == "azure" or (provider == "litellm" and route.startswith("azure/")):
- reason = "azure"
+ from openviking.models.vlm.registry import DEFAULT_AZURE_API_VERSION
+
+ if provider == "litellm":
+ model = model.split("/", 1)[1]
+ reason = None if base else "litellm_no_api_base"
+ version = settings.get("api_version") or DEFAULT_AZURE_API_VERSION
+ base = (
+ f"{base.rstrip('/')}/openai/deployments/{model}/chat/completions"
+ f"?api-version={version}"
+ )
+ exact = True
elif provider == "volcengine":
from openviking.models.vlm.backends.volcengine_vlm import DEFAULT_VOLCENGINE_API_BASE
@@ -122,6 +133,8 @@ def vlm_upstreams(config: VLMConfig) -> dict:
"api_key": api_key,
"headers": headers,
"models": [model] if model else [],
+ "append_endpoint_path": not exact,
+ "key_header": "api-key" if exact else "authorization",
"vendor": _vendor(base),
"coding_plan": bool(re.search(r"(^|/)(coding|plan)(/|$)", urlsplit(base).path)),
}
diff --git a/openviking_gateway/management.py b/openviking_gateway/management.py
index 0e9ddcbc8..d8844b898 100644
--- a/openviking_gateway/management.py
+++ b/openviking_gateway/management.py
@@ -21,7 +21,7 @@ from .client import VikingError
from .config import MOUNT_PATH
from .models import CaptureReset, KeyRequest, Policy, Upstream, UpstreamTestRequest
from .protocols import text_content
-from .proxy import upstream_headers, upstream_url
+from .proxy import fill_key, upstream_headers, upstream_key, upstream_url
from .records import RecordKind as K
from .storage import delete_account_data, delete_user_data, digest
from .tool_catalog import select_tools
@@ -30,7 +30,16 @@ from .tool_protocols import ResponseCapture, tool_protocol
PREFIX = "/admin"
MANAGEMENT_HEADER = "x-openviking-gateway-management-key"
# Fields a VLM re-import refreshes; everything else may have been edited in the gateway.
-VLM_SOURCE_FIELDS = ("base_url", "api_key", "headers", "protocol", "vendor", "coding_plan")
+VLM_SOURCE_FIELDS = (
+ "base_url",
+ "api_key",
+ "headers",
+ "protocol",
+ "vendor",
+ "coding_plan",
+ "append_endpoint_path",
+ "key_header",
+)
UPSTREAM_TEST_PATHS = {
"anthropic": "/v1/messages",
"chat": "/v1/chat/completions",
@@ -461,12 +470,17 @@ def register_management(app):
assert data is not None
generating = data.model is not None
kind = "model" if generating else "connection"
+ if not generating and not upstream.get("append_endpoint_path", True):
+ # An exact endpoint has no model list; only a generation test can reach it.
+ url = upstream_url(upstream, UPSTREAM_TEST_PATHS[upstream["protocol"]])
+ return {"ok": False, "reason": "exact_endpoint_untestable", "url": url, "kind": kind}
path = UPSTREAM_TEST_PATHS[upstream["protocol"]] if generating else "/v1/models"
url = upstream_url(upstream, path)
if generating and upstream.get("coding_plan") and not upstream.get("allow_coding_plan"):
return {"ok": False, "reason": "coding_plan_blocked", "url": url, "kind": kind}
try:
headers = upstream_headers(upstream, {})
+ target = fill_key(url, upstream_key(upstream, {}))
except HTTPException as error:
if error.status_code != 401:
raise
@@ -491,7 +505,7 @@ def register_management(app):
try:
async with app.state.http.request(
"POST" if generating else "GET",
- url,
+ target,
data=orjson.dumps(payload) if payload is not None else None,
headers=headers,
timeout=aiohttp.ClientTimeout(
@@ -502,9 +516,7 @@ def register_management(app):
status = response.status
raw = bytearray()
while len(raw) <= UPSTREAM_TEST_RESPONSE_LIMIT:
- chunk = await response.content.read(
- UPSTREAM_TEST_RESPONSE_LIMIT + 1 - len(raw)
- )
+ chunk = await response.content.read(UPSTREAM_TEST_RESPONSE_LIMIT + 1 - len(raw))
if not chunk:
break
raw.extend(chunk)
diff --git a/openviking_gateway/models.py b/openviking_gateway/models.py
index 02335f273..d684935eb 100644
--- a/openviking_gateway/models.py
+++ b/openviking_gateway/models.py
@@ -6,6 +6,7 @@ from urllib.parse import urlsplit
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
ProtocolName = Literal["anthropic", "chat", "responses"]
+KEY_PLACEHOLDER = "{api_key}"
# OpenViking MCP tools that change data; new profiles offer only the read-only ones.
WRITE_TOOLS = (
@@ -90,6 +91,7 @@ class Upstream(BaseModel):
base_url: str
append_endpoint_path: bool = True
api_key: str = ""
+ key_header: Literal["authorization", "api-key", "x-api-key"] = "authorization"
auth_mode: Literal["managed", "passthrough"] = "managed"
headers: dict[str, str] = Field(default_factory=dict)
models: list[str] = Field(default_factory=list)
@@ -122,13 +124,14 @@ class Upstream(BaseModel):
or not parsed.hostname
or parsed.username
or parsed.password
- or parsed.query
- or parsed.fragment
+ or "#" in value
):
- raise ValueError(
- "expected an HTTP(S) upstream URL without credentials, query or fragment"
- )
- return value.rstrip("/")
+ raise ValueError("expected an HTTP(S) upstream URL without credentials or fragment")
+ rest = value.replace(KEY_PLACEHOLDER, "")
+ if "{" in rest or "}" in rest or "{" in parsed.netloc:
+ raise ValueError("base_url may only use the {api_key} placeholder, outside the host")
+ base, mark, query = value.partition("?")
+ return base.rstrip("/") + mark + query
@model_validator(mode="after")
def validate_protocol_base(self):
diff --git a/openviking_gateway/proxy.py b/openviking_gateway/proxy.py
index a6b070663..d2fec24da 100644
--- a/openviking_gateway/proxy.py
+++ b/openviking_gateway/proxy.py
@@ -8,6 +8,7 @@ import logging
import time
import uuid
from dataclasses import replace
+from urllib.parse import quote, unquote_plus
import aiohttp
import orjson
@@ -16,7 +17,7 @@ from fastapi.responses import Response, StreamingResponse
from starlette.background import BackgroundTask
from .capture import CapturePipeline
-from .models import Policy
+from .models import KEY_PLACEHOLDER, Policy
from .protocols import SSEDecoder, parse_body
from .storage import digest
from .tool_executor import ToolExecutor
@@ -42,6 +43,11 @@ PATH_PROTOCOL = {
"/v1/chat/completions": "chat",
"/v1/responses": "responses",
}
+PRIMARY_PATH = {
+ "anthropic": "/v1/messages",
+ "chat": "/v1/chat/completions",
+ "responses": "/v1/responses",
+}
def filtered_headers(headers, request=False):
@@ -64,23 +70,28 @@ def filtered_headers(headers, request=False):
}
-def upstream_url(upstream, path):
- base = upstream["base_url"].rstrip("/")
+def query_names(pairs):
+ return {unquote_plus(pair.partition("=")[0]) for pair in pairs}
+
+
+def upstream_url(upstream, path, query=""):
+ """The upstream address for a client path; ``{api_key}`` stays unfilled."""
+ base, _, stored = upstream["base_url"].partition("?")
+ base = base.rstrip("/")
if not upstream.get("append_endpoint_path", True):
- return base
- if upstream["protocol"] != "anthropic":
+ primary = PRIMARY_PATH[upstream["protocol"]]
+ if path != primary and not path.startswith(primary + "/"):
+ raise HTTPException(404, f"This upstream is an exact endpoint for {primary} only")
+ path = path[len(primary) :]
+ elif upstream["protocol"] != "anthropic":
path = path.removeprefix("/v1")
- return base + path
+ pairs = [pair for pair in stored.split("&") if pair]
+ names = query_names(pairs)
+ pairs += [pair for pair in query.split("&") if pair and not query_names([pair]) & names]
+ return base + path + ("?" + "&".join(pairs) if pairs else "")
-def upstream_headers(upstream, incoming):
- headers = filtered_headers(incoming, request=True)
- headers.update(upstream.get("headers", {}))
- # Drop configured copies of the auth headers set below; other protocols keep x-api-key.
- own = (
- {"authorization", "x-api-key"} if upstream["protocol"] == "anthropic" else {"authorization"}
- )
- headers = {k: v for k, v in headers.items() if k.lower() not in own}
+def upstream_key(upstream, incoming):
key = (
upstream.get("api_key", "")
if upstream.get("auth_mode") != "passthrough"
@@ -92,10 +103,30 @@ def upstream_headers(upstream, incoming):
)
if not key:
raise HTTPException(401, "Upstream API key is missing")
- # Anthropic-compatible services accept either header; some require Bearer.
- headers["Authorization"] = "Bearer " + key
- if upstream["protocol"] == "anthropic":
- headers["x-api-key"] = key
+ return key
+
+
+def fill_key(url, key):
+ return url.replace(KEY_PLACEHOLDER, quote(key, safe=""))
+
+
+def upstream_headers(upstream, incoming):
+ headers = filtered_headers(incoming, request=True)
+ headers.update(upstream.get("headers", {}))
+ key = upstream_key(upstream, incoming)
+ name = upstream.get("key_header", "authorization")
+ # Drop configured copies of the auth headers set below.
+ if KEY_PLACEHOLDER in upstream.get("base_url", ""):
+ own, auth = {"authorization", "x-api-key", "api-key"}, {}
+ elif name == "authorization":
+ own, auth = {"authorization"}, {"Authorization": "Bearer " + key}
+ # Anthropic-compatible services accept either header; some require Bearer.
+ if upstream["protocol"] == "anthropic":
+ own, auth = own | {"x-api-key"}, {**auth, "x-api-key": key}
+ else:
+ own, auth = {"authorization", name}, {name: key}
+ headers = {k: v for k, v in headers.items() if k.lower() not in own}
+ headers.update(auth)
headers["accept-encoding"] = "identity"
return headers
@@ -108,7 +139,11 @@ def check_coding_plan(upstream):
def matches(upstream, protocol, model):
return (
upstream.get("enabled", True)
- and (not protocol or upstream["protocol"] == protocol)
+ and (
+ upstream["protocol"] == protocol
+ if protocol
+ else upstream.get("append_endpoint_path", True)
+ )
and (
not model
or not upstream.get("models")
@@ -202,8 +237,15 @@ class ProxyRequest:
for u in await self.management.list(self.credential["account"], "upstreams")
if u["id"] in self.credential["upstream_ids"]
]
+ # A stored response is pinned below; an exact responses endpoint may hold it.
+ stored_response = self.path.startswith("/v1/responses/")
candidates = sorted(
- [u for u in self.upstreams if matches(u, self.protocol, self.model)],
+ [
+ u
+ for u in self.upstreams
+ if matches(u, self.protocol, self.model)
+ or (stored_response and matches(u, "responses", self.model))
+ ],
key=lambda u: (-u["priority"], u["id"]),
)
if not candidates:
@@ -258,8 +300,9 @@ class ProxyRequest:
self.metrics.update(self.prepared.metrics)
def url(self, upstream):
- target = upstream_url(upstream, self.path)
- return target + "?" + self.request.url.query if self.request.url.query else target
+ key = upstream_key(upstream, self.request.headers)
+ filled = {**upstream, "base_url": fill_key(upstream["base_url"], key)}
+ return upstream_url(filled, self.path, self.request.url.query)
async def outgoing(self, body, upstream, prepared):
"""Vendor fields and the model alias, for the client request and its summary."""
@@ -273,13 +316,14 @@ class ProxyRequest:
try:
check_coding_plan(upstream)
headers = upstream_headers(upstream, self.request.headers)
+ target = self.url(upstream)
except HTTPException as error:
raise SummaryError(f"summary_http_{error.status_code}") from error
headers.setdefault("content-type", "application/json")
body = await self.outgoing(body, upstream, prepared)
try:
async with self.app.state.http.post(
- self.url(upstream),
+ target,
data=orjson.dumps(body),
headers=headers,
timeout=aiohttp.ClientTimeout(total=self.config.upstream_timeout_seconds),
diff --git a/tests/fixtures/gateway_upstream_urls.json b/tests/fixtures/gateway_upstream_urls.json
index 314f69cf8..314c2d44a 100644
--- a/tests/fixtures/gateway_upstream_urls.json
+++ b/tests/fixtures/gateway_upstream_urls.json
@@ -18,5 +18,12 @@
{"name":"ark-anthropic","protocol":"anthropic","base_url":"https://ark.cn-beijing.volces.com/api/compatible","path":"/v1/messages","expected":"https://ark.cn-beijing.volces.com/api/compatible/v1/messages"},
{"name":"ark-coding-anthropic","protocol":"anthropic","base_url":"https://ark.cn-beijing.volces.com/api/coding","path":"/v1/messages","expected":"https://ark.cn-beijing.volces.com/api/coding/v1/messages"},
{"name":"kimi-trailing-slash","protocol":"anthropic","base_url":"https://api.kimi.com/coding/","path":"/v1/messages","expected":"https://api.kimi.com/coding/v1/messages"},
- {"name":"ollama","protocol":"chat","base_url":"http://localhost:11434/v1","path":"/v1/chat/completions","expected":"http://localhost:11434/v1/chat/completions"}
+ {"name":"ollama","protocol":"chat","base_url":"http://localhost:11434/v1","path":"/v1/chat/completions","expected":"http://localhost:11434/v1/chat/completions"},
+ {"name":"base-stored-query","protocol":"chat","base_url":"https://proxy.example/v1/?tenant=a","path":"/v1/chat/completions","expected":"https://proxy.example/v1/chat/completions?tenant=a"},
+ {"name":"azure-chat-exact","protocol":"chat","append_endpoint_path":false,"base_url":"https://res.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21","path":"/v1/chat/completions","expected":"https://res.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21"},
+ {"name":"stored-query-wins","protocol":"chat","append_endpoint_path":false,"base_url":"https://res.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21","path":"/v1/chat/completions","query":"api-version=2099-01-01&beta=true","expected":"https://res.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21&beta=true"},
+ {"name":"exact-anthropic-count-tokens","protocol":"anthropic","append_endpoint_path":false,"base_url":"https://res.services.ai.azure.com/anthropic/v1/messages","path":"/v1/messages/count_tokens","expected":"https://res.services.ai.azure.com/anthropic/v1/messages/count_tokens"},
+ {"name":"exact-responses-continuation","protocol":"responses","append_endpoint_path":false,"base_url":"https://res.openai.azure.com/openai/responses?api-version=2025-04-01-preview","path":"/v1/responses/resp_1","expected":"https://res.openai.azure.com/openai/responses/resp_1?api-version=2025-04-01-preview"},
+ {"name":"exact-embeddings-unrouted","protocol":"chat","append_endpoint_path":false,"base_url":"https://res.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21","path":"/v1/embeddings","error":true},
+ {"name":"key-placeholder-literal","protocol":"chat","append_endpoint_path":false,"base_url":"https://proxy.example/{api_key}/chat?key={api_key}","path":"/v1/chat/completions","expected":"https://proxy.example/{api_key}/chat?key={api_key}"}
]
diff --git a/tests/gateway/test_app.py b/tests/gateway/test_app.py
index cf44254b3..f750bfab1 100644
--- a/tests/gateway/test_app.py
+++ b/tests/gateway/test_app.py
@@ -1303,3 +1303,80 @@ async def test_model_restricted_key_rejects_bodies_without_a_readable_model(runn
response = await client.post("/v1/chat/completions", headers=unrestricted, content=compressed)
assert response.status_code != 415 and len(seen) > before + 1
assert seen[-1][0] == "/v1/chat/completions"
+
+
+ ("suffix", "key_header", "query", "auth"),
+ [
+ (
+ "/openai/deployments/dep/chat/completions?api-version=2024-10-21",
+ "api-key",
+ "api-version=2024-10-21&beta=true",
+ {"api-key": "model-secret"},
+ ),
+ (
+ "/proxy/chat?key={api_key}",
+ "authorization",
+ "key=model-secret&beta=true&api-version=old",
+ {},
+ ),
+ ],
+)
+async def test_exact_endpoint_chat_request(running_gateway, suffix, key_header, query, auth):
+ app, client, _, key, _, _ = running_gateway
+ upstream = await app.state.management.get("tenant", "upstreams", "chat")
+ base = upstream["base_url"].removesuffix("/v1") + suffix
+ await app.state.management.save(
+ "tenant",
+ "upstreams",
+ "chat",
+ {**upstream, "base_url": base, "append_endpoint_path": False, "key_header": key_header},
+ )
+ seen = []
+
+ async def handler(request):
+ if not request.path.startswith(("/api/", "/mcp", "/health")):
+ seen.append((request.path, request.query_string, request.headers))
+
+ app.state.test_backend["handler"] = handler
+ response = await client.post(
+ "/v1/chat/completions?beta=true&api-version=old",
+ headers={"Authorization": "Bearer " + key["key"]},
+ json={"model": "model", "messages": [{"role": "user", "content": "hi"}]},
+ )
+ assert response.status_code == 200, response.text
+ path, sent_query, headers = seen[-1]
+ assert (path, sent_query) == (suffix.split("?")[0], query)
+ names = {"authorization", "api-key", "x-api-key"}
+ assert {k.lower(): v for k, v in headers.items() if k.lower() in names} == auth
+
+
+async def test_stored_response_from_an_exact_endpoint(running_gateway):
+ app, client, admin, _, _, _ = running_gateway
+ upstream = await app.state.management.get("tenant", "upstreams", "responses")
+ await app.state.management.save(
+ "tenant",
+ "upstreams",
+ "responses",
+ {
+ **upstream,
+ "base_url": upstream["base_url"] + "/responses",
+ "append_endpoint_path": False,
+ },
+ )
+ minted = await client.post(
+ "/admin/keys",
+ headers=admin,
+ json={
+ "name": "exact",
+ "openviking_key": "user-key",
+ "policy_id": "default",
+ "upstream_ids": ["responses"],
+ },
+ )
+ headers = {"Authorization": "Bearer " + minted.json()["key"]}
+ body = {"model": "model", "input": "hi"}
+ assert (await client.post("/v1/responses", headers=headers, json=body)).status_code == 200
+ assert (await client.get("/v1/responses/resp-1", headers=headers)).status_code == 200
+ embeddings = await client.post("/v1/embeddings", headers=headers, json={"model": "model"})
+ assert embeddings.status_code == 404
diff --git a/tests/gateway/test_management.py b/tests/gateway/test_management.py
index 853433fb3..923e8d45e 100644
--- a/tests/gateway/test_management.py
+++ b/tests/gateway/test_management.py
@@ -396,9 +396,7 @@ async def test_upstream_connection_status_contract(running_gateway, status, body
async def test_upstream_test_reads_chunked_response_to_eof(running_gateway):
app, client, admin, _, _, _ = running_gateway
- body = json.dumps(
- {"choices": [{"message": {"role": "assistant", "content": "OK"}}]}
- ).encode()
+ body = json.dumps({"choices": [{"message": {"role": "assistant", "content": "OK"}}]}).encode()
async def handler(request):
if request.path == "/v1/chat/completions":
@@ -1040,3 +1038,37 @@ async def test_degraded_probe_backs_off_until_success(tmp_path, monkeypatch):
gaps = [round(b - a) for a, b in zip(probes, probes[1:], strict=False)]
# Every second until OV first answers; later outages back off 1, 2, 4 capped at the interval.
assert gaps[:6] == [1, 1, 4, 1, 2, 4]
+
+
+async def test_upstream_test_fills_url_key_and_reaches_exact_endpoints(running_gateway):
+ app, client, admin, _, captured, _ = running_gateway
+ upstream = await app.state.management.get("tenant", "upstreams", "chat")
+ stored = upstream["base_url"] + "?tenant=a&key={api_key}"
+ await app.state.management.save("tenant", "upstreams", "chat", {**upstream, "base_url": stored})
+ seen = []
+
+ async def handler(request):
+ if request.path.startswith("/v1/"):
+ seen.append((request.method, request.path, request.query_string, dict(request.headers)))
+
+ app.state.test_backend["handler"] = handler
+ result = (await client.post(P + "/upstreams/chat/test", headers=admin, json={})).json()
+ assert result["ok"] and result["url"] == upstream["base_url"] + "/models?tenant=a&key={api_key}"
+ assert seen[-1][2] == "tenant=a&key=model-secret" and "Authorization" not in seen[-1][3]
+
+ endpoint = upstream["base_url"] + "/chat/completions?api-version=1"
+ exact = {**upstream, "base_url": endpoint, "append_endpoint_path": False}
+ await app.state.management.save("tenant", "upstreams", "chat", exact)
+ before = len(captured)
+ result = (await client.post(P + "/upstreams/chat/test", headers=admin, json={})).json()
+ assert result == {
+ "ok": False,
+ "reason": "exact_endpoint_untestable",
+ "url": endpoint,
+ "kind": "connection",
+ }
+ assert len(captured) == before
+ model = {"model": "model"}
+ result = (await client.post(P + "/upstreams/chat/test", headers=admin, json=model)).json()
+ assert result["kind"] == "model" and result["url"] == endpoint
+ assert seen[-1][:3] == ("POST", "/v1/chat/completions", "api-version=1")
diff --git a/tests/gateway/test_upstream_urls.py b/tests/gateway/test_upstream_urls.py
index 437115895..0777abf2f 100644
--- a/tests/gateway/test_upstream_urls.py
+++ b/tests/gateway/test_upstream_urls.py
@@ -5,6 +5,7 @@ from pathlib import Path
from types import SimpleNamespace
import pytest
+from fastapi import HTTPException
from pydantic import ValidationError
from openviking_gateway.models import Upstream
@@ -17,8 +18,17 @@ VECTORS = json.loads(
@pytest.mark.parametrize("vector", VECTORS, ids=lambda value: value["name"])
def test_upstream_url_vectors(vector):
- upstream = {"protocol": vector["protocol"], "base_url": vector["base_url"]}
- assert upstream_url(upstream, vector["path"]) == vector["expected"]
+ upstream = {
+ "protocol": vector["protocol"],
+ "base_url": vector["base_url"],
+ "append_endpoint_path": vector.get("append_endpoint_path", True),
+ }
+ if vector.get("error"):
+ with pytest.raises(HTTPException) as error:
+ upstream_url(upstream, vector["path"], vector.get("query", ""))
+ assert error.value.status_code == 404
+ else:
+ assert upstream_url(upstream, vector["path"], vector.get("query", "")) == vector["expected"]
@pytest.mark.parametrize(
@@ -54,6 +64,27 @@ def test_anthropic_base_rejects_v1_suffix(base):
Upstream(name="test", protocol="anthropic", base_url=base)
+ "base",
+ [
+ "https://example.test/v1?key={token}",
+ "https://{api_key}.example.test/v1",
+ "https://example.test/v1#frag",
+ "https://user:[email protected]/v1",
+ ],
+)
+def test_base_url_rejects_other_placeholders_fragments_and_credentials(base):
+ with pytest.raises(ValidationError):
+ Upstream(name="test", protocol="chat", base_url=base)
+
+
+def test_base_url_keeps_query_and_strips_path_slash():
+ upstream = Upstream(
+ name="test", protocol="chat", base_url="https://example.test/v1/?key={api_key}"
+ )
+ assert upstream.base_url == "https://example.test/v1?key={api_key}"
+
+
def test_endpoint_path_defaults_on_and_allows_anthropic_v1_when_off():
default = Upstream(name="default", protocol="chat", base_url="https://example.test/v1")
exact = Upstream(
@@ -85,3 +116,30 @@ def test_upstream_auth_headers(protocol):
# A configured x-api-key reaches Bearer upstreams; Anthropic's own header replaces it.
assert headers.get("x-api-key") == ("secret" if protocol == "anthropic" else "gateway-plan-key")
assert headers["anthropic-version"] == "2023-06-01"
+
+
+ ("key_header", "base_url", "expected"),
+ [
+ ("api-key", "https://example.test/chat", {"api-key": "secret"}),
+ ("x-api-key", "https://example.test/chat", {"x-api-key": "secret"}),
+ ("authorization", "https://example.test/chat?key={api_key}", {}),
+ ("api-key", "https://example.test/{api_key}/chat", {}),
+ ],
+)
+def test_key_header_and_url_placeholder(key_header, base_url, expected):
+ headers = upstream_headers(
+ {
+ "protocol": "chat",
+ "base_url": base_url,
+ "api_key": "secret",
+ "key_header": key_header,
+ "headers": {"Authorization": "x", key_header.upper(): "y", "custom": "1"},
+ },
+ {},
+ )
+ auth = {
+ k: v for k, v in headers.items() if k.lower() in {"authorization", "api-key", "x-api-key"}
+ }
+ assert auth == expected
+ assert headers["custom"] == "1"
diff --git a/tests/server/test_gateway_vlm.py b/tests/server/test_gateway_vlm.py
index 209849c3b..17db0c3de 100644
--- a/tests/server/test_gateway_vlm.py
+++ b/tests/server/test_gateway_vlm.py
@@ -6,6 +6,7 @@ from unittest.mock import AsyncMock
import pytest
from openviking.config.vlm import AccountVLMProvider
+from openviking.models.vlm.registry import DEFAULT_AZURE_API_VERSION
from openviking.server.gateway_vlm import vlm_upstreams
from openviking_cli.utils.config.vlm_config import VLMConfig
@@ -81,7 +82,6 @@ def test_glm_and_byteplus():
@pytest.mark.parametrize(
"provider,model,reason",
[
- ("azure", "deployment", "azure"),
("openai-codex", "gpt", "openai_codex"),
("litellm", "some/model", "litellm_no_api_base"),
("litellm", "gemini/gemini-2.0-flash", "litellm_unsupported_route"),
@@ -96,6 +96,25 @@ def test_skipped_credentials_have_actionable_reasons(provider, model, reason):
assert result["skipped"] == [{"id": "vlm-primary", "reason": reason}]
+ "provider,model,api_version,query",
+ [
+ ("azure", "gpt-4o", "2024-10-21", "2024-10-21"),
+ ("litellm", "azure/gpt-4o", None, DEFAULT_AZURE_API_VERSION),
+ ],
+)
+def test_azure_deployments_become_exact_endpoints(provider, model, api_version, query):
+ config = configured(
+ provider, model, api_base="https://res.openai.azure.com/", api_version=api_version
+ )
+ (upstream,) = vlm_upstreams(config)["upstreams"]
+ assert upstream["base_url"] == (
+ f"https://res.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version={query}"
+ )
+ assert (upstream["append_endpoint_path"], upstream["key_header"]) == (False, "api-key")
+ assert (upstream["vendor"], upstream["models"]) == ("generic", ["gpt-4o"])
+
+
def test_litellm_openai_routes_send_the_bare_model_name():
(upstream,) = vlm_upstreams(
configured(
diff --git a/web-studio/src/i18n/locales/en/gateway/common.ts b/web-studio/src/i18n/locales/en/gateway/common.ts
index 42907921c..cf2bbfde4 100644
--- a/web-studio/src/i18n/locales/en/gateway/common.ts
+++ b/web-studio/src/i18n/locales/en/gateway/common.ts
@@ -218,8 +218,9 @@ const common = {
unknownCategory: 'Unknown category “{{name}}”',
quotasAllZero: 'Set at least one category above 0, or turn the limit off',
softBelowHard: 'Must be below the hard reminder',
- baseUrl:
- 'Enter an http:// or https:// URL without credentials, query or fragment',
+ baseUrl: 'Enter an http:// or https:// URL without credentials or fragment',
+ baseUrlPlaceholder:
+ 'The URL can only use the {api_key} placeholder, and not in the host',
subscriptionKey:
"Claude subscription tokens (sk-ant-oat…) aren't supported. Use a model API key.",
apiKeyRequired:
diff --git a/web-studio/src/i18n/locales/en/gateway/upstreams.ts b/web-studio/src/i18n/locales/en/gateway/upstreams.ts
index d6a0f0807..72188e40f 100644
--- a/web-studio/src/i18n/locales/en/gateway/upstreams.ts
+++ b/web-studio/src/i18n/locales/en/gateway/upstreams.ts
@@ -20,8 +20,6 @@ const upstreams = {
'The model configuration is invalid. Check the server settings and import again.',
openai_codex:
'Codex OAuth credentials refresh periodically and cannot be used long term by the gateway. Add an upstream manually.',
- azure:
- 'Azure OpenAI requires an api-version parameter that the gateway does not support yet. Add an upstream manually.',
litellm_no_api_base:
'This LiteLLM credential has no api_base. Add api_base to the server configuration and import again.',
litellm_unsupported_route:
@@ -119,6 +117,8 @@ const upstreams = {
'Checks the saved connection first, then sends one small non-streaming request to test the selected model.',
connection: 'Upstream connection',
checkingConnection: 'Checking connection…',
+ exactEndpoint:
+ 'An exact endpoint has no model list to check. Test a model directly.',
model: 'Model',
modelPlaceholder: 'Enter a provider model name',
configuredModel:
@@ -215,7 +215,7 @@ const upstreams = {
baseUrl: {
label: 'Base URL',
description:
- "Use base_url from the provider's SDK documentation. The gateway appends /chat/completions, /responses or /v1/messages. Add separate upstreams when a provider uses different base URLs for different protocols.",
+ "Use base_url from the provider's SDK documentation. The gateway appends /chat/completions, /responses or /v1/messages. Add separate upstreams when a provider uses different base URLs for different protocols. The URL may include a query; write the key as {api_key}.",
preview: 'Requests go to',
missingPath:
'This provider needs a path in base_url. A domain alone sends requests to the wrong address.',
@@ -230,6 +230,13 @@ const upstreams = {
description:
'Adds the path for the selected protocol to the Base URL. Turn this off when the Base URL is already the complete request endpoint.',
},
+ keyHeader: {
+ label: 'Send the key as',
+ authorization: 'Authorization: Bearer',
+ apiKey: 'api-key header (Azure OpenAI)',
+ xApiKey: 'x-api-key header',
+ inUrl: 'In the URL ({api_key}); no auth header is sent',
+ },
authMode: {
label: 'Who provides the API key',
managed: {
diff --git a/web-studio/src/i18n/locales/zh-CN/gateway/common.ts b/web-studio/src/i18n/locales/zh-CN/gateway/common.ts
index 4d9ac2d08..43709c59c 100644
--- a/web-studio/src/i18n/locales/zh-CN/gateway/common.ts
+++ b/web-studio/src/i18n/locales/zh-CN/gateway/common.ts
@@ -202,8 +202,8 @@ const common = {
unknownCategory: '未知分类“{{name}}”',
quotasAllZero: '至少把一个分类设为大于 0,或关闭分类限额',
softBelowHard: '须低于硬提醒时机',
- baseUrl:
- '请输入 http:// 或 https:// 地址,不能包含账号密码、查询参数或片段',
+ baseUrl: '请输入 http:// 或 https:// 地址,不能包含账号密码或 # 片段',
+ baseUrlPlaceholder: '地址里只能使用 {api_key} 占位符,且不能放在域名里',
subscriptionKey:
'不支持 Claude 订阅令牌(sk-ant-oat…),请使用模型 API Key。',
apiKeyRequired: '请填写 API Key,或改为由每个客户端自带 API Key',
diff --git a/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts b/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts
index 01d09f6a0..6beb7e01a 100644
--- a/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts
+++ b/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts
@@ -19,8 +19,6 @@ const upstreams = {
invalid_configuration: '模型配置无效,请检查服务端配置后重新导入。',
openai_codex:
'Codex 的 OAuth 凭证会定期刷新,网关无法长期使用。请手动添加上游。',
- azure:
- 'Azure OpenAI 需要 api-version 参数,网关暂不支持。请手动添加上游。',
litellm_no_api_base:
'这个 LiteLLM 凭证没有填写 api_base,网关无法确定地址。请在服务端配置里补上 api_base 后重新导入。',
litellm_unsupported_route:
@@ -113,6 +111,7 @@ const upstreams = {
'先检查已保存配置的连接,再发送一次最小的非流式生成请求测试所选模型。',
connection: '上游连接',
checkingConnection: '正在检查连接…',
+ exactEndpoint: '完整请求地址没有模型列表可检查,请直接测试模型。',
model: '模型',
modelPlaceholder: '输入服务商模型名',
configuredModel: '可选择已配置的模型,也可以输入其他模型名。',
@@ -202,7 +201,7 @@ const upstreams = {
baseUrl: {
label: 'Base URL',
description:
- '照服务商 SDK 文档填写 base_url,网关在后面拼接 /chat/completions、/responses 或 /v1/messages。同一服务商的不同协议地址不同时,请分别添加上游。',
+ '照服务商 SDK 文档填写 base_url,网关在后面拼接 /chat/completions、/responses 或 /v1/messages。同一服务商的不同协议地址不同时,请分别添加上游。地址里可以带查询参数,密钥请写成 {api_key}。',
preview: '请求将发往',
missingPath:
'这个服务商的 base_url 需要带路径,只填域名会请求到错误地址。',
@@ -217,6 +216,13 @@ const upstreams = {
description:
'在 Base URL 后拼接所选协议的接口路径。如果 Base URL 已经是完整的请求地址,请关闭此开关。',
},
+ keyHeader: {
+ label: '密钥发送方式',
+ authorization: 'Authorization: Bearer',
+ apiKey: 'api-key 请求头(Azure OpenAI)',
+ xApiKey: 'x-api-key 请求头',
+ inUrl: '放在地址里({api_key}),不发送鉴权请求头',
+ },
authMode: {
label: 'API Key 由谁提供',
managed: {
diff --git a/web-studio/src/routes/gateway/-components/connect-page.test.tsx b/web-studio/src/routes/gateway/-components/connect-page.test.tsx
index 6d4c5a3a8..79ac40ecd 100644
--- a/web-studio/src/routes/gateway/-components/connect-page.test.tsx
+++ b/web-studio/src/routes/gateway/-components/connect-page.test.tsx
@@ -85,6 +85,7 @@ function upstream(overrides: Partial<Upstream>): Upstream {
base_url: 'https://api.example.com/v1',
append_endpoint_path: true,
auth_mode: 'managed',
+ key_header: 'authorization',
models: [],
aliases: {},
priority: 0,
diff --git a/web-studio/src/routes/gateway/-components/upstreams-actions.tsx b/web-studio/src/routes/gateway/-components/upstreams-actions.tsx
index 5bea10155..b77cc6ca0 100644
--- a/web-studio/src/routes/gateway/-components/upstreams-actions.tsx
+++ b/web-studio/src/routes/gateway/-components/upstreams-actions.tsx
@@ -229,6 +229,8 @@ export function UpstreamTest({
testUpstream(connection, upstream.id, selectedModel),
})
const blocked = testBlockedReason(upstream)
+ // An exact endpoint has no model list; upstreams saved before the setting append the path.
+ const exact = upstream.append_endpoint_path === false
const explanation = blocked
? t(`upstreams.test.${blocked}`)
: [t('upstreams.test.hint'), note].filter(Boolean).join(' ')
@@ -254,7 +256,7 @@ export function UpstreamTest({
modelTest.reset()
setModel(configuredModels[0] ?? '')
setOpen(true)
- connectionTest.mutate()
+ if (!exact) connectionTest.mutate()
}
const submit = (event: React.SubmitEvent<HTMLFormElement>) => {
event.preventDefault()
@@ -262,7 +264,8 @@ export function UpstreamTest({
if (selectedModel) modelTest.mutate(selectedModel)
}
const pending = connectionTest.isPending || modelTest.isPending
- const connected = connectionTest.isSuccess && connectionTest.data.ok
+ const connected =
+ exact || (connectionTest.isSuccess && connectionTest.data.ok)
const modelBlocked = upstream.coding_plan && !upstream.allow_coding_plan
return (
@@ -319,7 +322,11 @@ export function UpstreamTest({
<p className="text-xs font-medium">
{t('upstreams.test.dialog.connection')}
</p>
- {connectionTest.isPending ? (
+ {exact ? (
+ <p className="text-sm text-muted-foreground">
+ {t('upstreams.test.dialog.exactEndpoint')}
+ </p>
+ ) : connectionTest.isPending ? (
<p className="flex items-center gap-2 text-sm text-muted-foreground">
<LoaderCircleIcon className="size-4 animate-spin" />
{t('upstreams.test.dialog.checkingConnection')}
diff --git a/web-studio/src/routes/gateway/-components/upstreams-form.tsx b/web-studio/src/routes/gateway/-components/upstreams-form.tsx
index f7bfc6378..c98abfd1e 100644
--- a/web-studio/src/routes/gateway/-components/upstreams-form.tsx
+++ b/web-studio/src/routes/gateway/-components/upstreams-form.tsx
@@ -24,6 +24,7 @@ import { cn } from '#/lib/utils'
import type {
AuthMode,
+ KeyHeader,
Protocol,
Upstream,
UpstreamInput,
@@ -33,6 +34,8 @@ import { formatNumber } from '../-lib/format'
import { protocolLabel, vendorLabel } from '../-lib/localize'
import {
ARK_VENDORS,
+ KEY_HEADERS,
+ KEY_PLACEHOLDER,
PROTOCOLS,
PROTOCOL_PATHS,
UPSTREAM_DEFAULTS,
@@ -64,6 +67,13 @@ export type UpstreamField = keyof UpstreamInput
/** Base URL placeholder for providers without a default, such as Generic. */
const BASE_URL_EXAMPLE = 'https://api.example.com/v1'
+/** Translation key under `upstreams.form.keyHeader` of each header option. */
+const KEY_HEADER_LABELS: Record<KeyHeader, string> = {
+ authorization: 'authorization',
+ 'api-key': 'apiKey',
+ 'x-api-key': 'xApiKey',
+}
+
/** Providers with a hint of their own under `upstreams.form.vendor.hints`. */
const VENDOR_HINTS: Vendor[] = ['deepseek', 'ark', 'byteplus']
@@ -207,6 +217,7 @@ export function UpstreamForm({
const preview = endpointPreview(draft)
const defaultUrl = defaultBaseUrl(draft.vendor, draft.protocol)
const keyStored = Boolean(stored?.has_api_key)
+ const keyInUrl = draft.base_url.includes(KEY_PLACEHOLDER)
const vendorHint = VENDOR_HINTS.includes(draft.vendor)
? t(`upstreams.form.vendor.hints.${draft.vendor}`)
: t('upstreams.form.vendor.description')
@@ -220,6 +231,9 @@ export function UpstreamForm({
function switchEndpoint(vendor: Vendor, protocol: Protocol) {
if (vendor !== draft.vendor) onChange('vendor', vendor)
if (protocol !== draft.protocol) onChange('protocol', protocol)
+ if (vendor !== 'generic' && draft.key_header !== 'authorization') {
+ onChange('key_header', 'authorization')
+ }
const baseUrl = baseUrlAfterSwitch(draft, vendor, protocol)
if (baseUrl !== draft.base_url) onFill('base_url', baseUrl)
}
@@ -444,6 +458,41 @@ export function UpstreamForm({
))}
</RadioGroup>
</SettingField>
+ {draft.vendor === 'generic' ? (
+ <SettingField
+ label={t('upstreams.form.keyHeader.label')}
+ htmlFor="upstream-key-header"
+ >
+ <Select
+ value={draft.key_header}
+ disabled={keyInUrl}
+ onValueChange={(value) => {
+ const header = KEY_HEADERS.find((item) => item === value)
+ if (header) onChange('key_header', header)
+ }}
+ >
+ <SelectTrigger
+ id="upstream-key-header"
+ className="w-full sm:w-96"
+ >
+ <SelectValue>
+ {t(
+ `upstreams.form.keyHeader.${keyInUrl ? 'inUrl' : KEY_HEADER_LABELS[draft.key_header]}`,
+ )}
+ </SelectValue>
+ </SelectTrigger>
+ <SelectContent>
+ {KEY_HEADERS.map((header) => (
+ <SelectItem key={header} value={header}>
+ {t(
+ `upstreams.form.keyHeader.${KEY_HEADER_LABELS[header]}`,
+ )}
+ </SelectItem>
+ ))}
+ </SelectContent>
+ </Select>
+ </SettingField>
+ ) : null}
{draft.auth_mode === 'managed' ? (
<SettingField
label={
diff --git a/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx b/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx
index e3a01a6c5..8972623f8 100644
--- a/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx
+++ b/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx
@@ -337,6 +337,9 @@ describe('UpstreamsPage', () => {
reason: 'passed',
}),
)
+ // Stored before append_endpoint_path existed: the list returns no such field.
+ const { append_endpoint_path: _, ...legacy } = openai
+ api.listUpstreams.mockResolvedValue([legacy as Upstream, claude, ark])
renderAt('/gateway/upstreams')
await screen.findByText('OpenAI')
@@ -344,10 +347,7 @@ describe('UpstreamsPage', () => {
rowOf('OpenAI').getByRole('button', { name: 'upstreams.test.action' }),
)
await waitFor(() =>
- expect(api.testUpstream).toHaveBeenCalledWith(
- expect.anything(),
- 'u1',
- ),
+ expect(api.testUpstream).toHaveBeenCalledWith(expect.anything(), 'u1'),
)
await screen.findByText('upstreams.test.result.connected')
fireEvent.click(
@@ -375,6 +375,36 @@ describe('UpstreamsPage', () => {
}
})
+ it('tests a model directly on an exact endpoint', async () => {
+ api.testUpstream.mockResolvedValue({
+ ok: true,
+ status: 200,
+ kind: 'model',
+ reason: 'passed',
+ })
+ api.listUpstreams.mockResolvedValue([
+ { ...openai, append_endpoint_path: false },
+ ])
+ renderAt('/gateway/upstreams')
+ await screen.findByText('OpenAI')
+
+ fireEvent.click(
+ rowOf('OpenAI').getByRole('button', { name: 'upstreams.test.action' }),
+ )
+ await screen.findByText('upstreams.test.dialog.exactEndpoint')
+ expect(api.testUpstream).not.toHaveBeenCalled()
+ fireEvent.click(
+ screen.getByRole('button', { name: 'upstreams.test.dialog.run' }),
+ )
+ await waitFor(() =>
+ expect(api.testUpstream).toHaveBeenCalledWith(
+ expect.anything(),
+ 'u1',
+ 'gpt-5',
+ ),
+ )
+ })
+
it('keeps model testing disabled when the connection test fails', async () => {
api.testUpstream.mockResolvedValue({
ok: false,
@@ -645,6 +675,50 @@ describe('UpstreamEditor', () => {
})
})
+ it('sends the key in the chosen header, or in the URL placeholder', async () => {
+ const azure =
+ 'https://res.openai.azure.com/openai/deployments/dep/chat/completions?api-version=2024-10-21'
+ renderAt('/gateway/upstreams/new')
+ await screen.findByText('upstreams.editor.newTitle')
+ fireEvent.change(screen.getByLabelText('upstreams.form.name.label'), {
+ target: { value: 'Azure' },
+ })
+ const keyHeader = screen.getByRole('combobox', {
+ name: 'upstreams.form.keyHeader.label',
+ })
+ fireEvent.change(baseUrlInput(), {
+ target: { value: 'https://proxy.example.com/v1?key={api_key}' },
+ })
+ expect(keyHeader.textContent).toContain('upstreams.form.keyHeader.inUrl')
+ expect(keyHeader.hasAttribute('data-disabled')).toBe(true)
+
+ fireEvent.change(baseUrlInput(), { target: { value: azure } })
+ fireEvent.click(
+ screen.getByRole('switch', {
+ name: 'upstreams.form.appendEndpointPath.label',
+ }),
+ )
+ fireEvent.change(
+ screen.getByLabelText('upstreams.form.apiKey.label', { exact: false }),
+ { target: { value: 'azure-key' } },
+ )
+ fireEvent.click(keyHeader)
+ const option = await screen.findByRole('option', {
+ name: 'upstreams.form.keyHeader.apiKey',
+ })
+ fireEvent.mouseMove(option)
+ fireEvent.click(option)
+ fireEvent.click(
+ screen.getByRole('button', { name: 'upstreams.editor.create' }),
+ )
+ await waitFor(() => expect(api.saveUpstream).toHaveBeenCalledTimes(1))
+ expect(lastSavedBody()).toMatchObject({
+ base_url: azure,
+ append_endpoint_path: false,
+ key_header: 'api-key',
+ })
+ })
+
it('shows settings that depend on the provider, key mode and plan', async () => {
renderAt('/gateway/upstreams/new')
await screen.findByText('upstreams.editor.newTitle')
@@ -1063,7 +1137,6 @@ describe('upstream access and import', () => {
needs_confirmation: [{ id: 'vlm-coding', name: 'Coding account' }],
skipped: [
{ id: 'oauth', reason: 'openai_codex' },
- { id: 'azure', reason: 'azure' },
{ id: 'lite', reason: 'litellm_no_api_base' },
],
})
@@ -1080,9 +1153,6 @@ describe('upstream access and import', () => {
expect(
screen.getByText('oauth: upstreams.import.reasons.openai_codex'),
).toBeTruthy()
- expect(
- screen.getByText('azure: upstreams.import.reasons.azure'),
- ).toBeTruthy()
expect(
screen.getByText('lite: upstreams.import.reasons.litellm_no_api_base'),
).toBeTruthy()
diff --git a/web-studio/src/routes/gateway/-lib/api.ts b/web-studio/src/routes/gateway/-lib/api.ts
index 55df9ffcd..5fda09912 100644
--- a/web-studio/src/routes/gateway/-lib/api.ts
+++ b/web-studio/src/routes/gateway/-lib/api.ts
@@ -17,6 +17,8 @@ export type Vendor =
| 'ark'
| 'byteplus'
export type AuthMode = 'managed' | 'passthrough'
+/** Header the upstream key is sent in. */
+export type KeyHeader = 'authorization' | 'api-key' | 'x-api-key'
/** Sources recall searches, in display order. */
export const CONTEXT_TYPES = ['memory', 'resource', 'skill'] as const
@@ -61,6 +63,7 @@ type UpstreamSettings = {
/** Append the selected protocol's endpoint path to base_url. */
append_endpoint_path: boolean
auth_mode: AuthMode
+ key_header: KeyHeader
models: string[]
/** Name clients use → model sent to the upstream. */
aliases: Record<string, string>
diff --git a/web-studio/src/routes/gateway/-lib/upstream-schema.test.ts b/web-studio/src/routes/gateway/-lib/upstream-schema.test.ts
index 3fed7d216..9fae63e42 100644
--- a/web-studio/src/routes/gateway/-lib/upstream-schema.test.ts
+++ b/web-studio/src/routes/gateway/-lib/upstream-schema.test.ts
@@ -167,21 +167,37 @@ describe('base URLs', () => {
['http://10.0.0.5:8000', true],
['ftp://example.com', false],
['https://user:[email protected]', false],
- ['https://example.com/?a=1', false],
- ['https://example.com/?', false],
+ ['https://example.com/?a=1', true],
+ ['https://example.com/{api_key}/v1?key={api_key}', true],
+ ['https://example.com/v1?key={token}', false],
+ ['https://{api_key}.example.com/v1', false],
['https://example.com/#top', false],
['example.com', false],
])('%s → %s', (url, valid) => {
expect(isValidBaseUrl(url)).toBe(valid)
})
- it.each(urlVectors)(
+ // The browser never sends a client query and never routes other paths.
+ it.each(
+ urlVectors.filter((vector) => !('query' in vector || 'error' in vector)),
+ )(
'joins the shared URL vector: $name',
- ({ base_url, protocol, path, expected }) => {
- expect(upstreamUrl(base_url, protocol as Protocol, path)).toBe(expected)
+ ({ base_url, protocol, path, expected, ...vector }) => {
+ const append =
+ 'append_endpoint_path' in vector ? vector.append_endpoint_path : true
+ expect(upstreamUrl(base_url, protocol as Protocol, path, append)).toBe(
+ expected,
+ )
},
)
+ it('names the placeholder rule when the URL breaks it', () => {
+ expect(
+ validateUpstream(input({ base_url: 'https://example.com/v1?k={key}' }))
+ .base_url?.key,
+ ).toBe('validation.baseUrlPlaceholder')
+ })
+
it.each([
['anthropic', '/v1/messages'],
['chat', '/v1/chat/completions'],
diff --git a/web-studio/src/routes/gateway/-lib/upstream-schema.ts b/web-studio/src/routes/gateway/-lib/upstream-schema.ts
index b89c3faa6..c18e57992 100644
--- a/web-studio/src/routes/gateway/-lib/upstream-schema.ts
+++ b/web-studio/src/routes/gateway/-lib/upstream-schema.ts
@@ -1,4 +1,10 @@
-import type { Protocol, Upstream, UpstreamInput, Vendor } from './api'
+import type {
+ KeyHeader,
+ Protocol,
+ Upstream,
+ UpstreamInput,
+ Vendor,
+} from './api'
import { checkNumber, checkRequired, collect } from './validation'
import type { NumberRule, ValidationErrors } from './validation'
@@ -10,6 +16,7 @@ export const UPSTREAM_DEFAULTS: UpstreamInput = {
append_endpoint_path: true,
api_key: '',
auth_mode: 'managed',
+ key_header: 'authorization',
headers: {},
models: [],
aliases: {},
@@ -25,6 +32,13 @@ export const UPSTREAM_DEFAULTS: UpstreamInput = {
}
export const PROTOCOLS: Protocol[] = ['anthropic', 'chat', 'responses']
+export const KEY_HEADERS: KeyHeader[] = [
+ 'authorization',
+ 'api-key',
+ 'x-api-key',
+]
+/** Where the gateway fills in the upstream key when sending. */
+export const KEY_PLACEHOLDER = '{api_key}'
export const VENDORS: Vendor[] = [
'generic',
'anthropic',
@@ -203,7 +217,14 @@ export function servedModels(
return [...new Set([...upstream.models, ...Object.keys(upstream.aliases)])]
}
-/** True for an absolute http(s) URL without credentials, query or fragment. */
+/** Braces other than `{api_key}`, or any brace in the scheme or host. */
+function misplacedPlaceholder(value: string): boolean {
+ const text = value.trim()
+ const origin = /^[^/?]*\/\/[^/?]*/.exec(text)?.[0] ?? ''
+ return /[{}]/.test(text.replaceAll(KEY_PLACEHOLDER, '') + origin)
+}
+
+/** True for an absolute http(s) URL without credentials or fragment. */
export function isValidBaseUrl(value: string): boolean {
let url: URL
try {
@@ -216,12 +237,17 @@ export function isValidBaseUrl(value: string): boolean {
Boolean(url.hostname) &&
!url.username &&
!url.password &&
- !url.search &&
- !url.hash &&
- !/[?#]/.test(value)
+ !value.includes('#') &&
+ !misplacedPlaceholder(value)
)
}
+/** Splits off the query and the path's trailing slashes, as the gateway stores it. */
+function splitBaseUrl(value: string): [base: string, query: string] {
+ const [base, ...query] = value.trim().split('?')
+ return [base.replace(/\/+$/, ''), query.length ? '?' + query.join('?') : '']
+}
+
/** Upstream URL joining: OpenAI paths omit /v1; Anthropic paths retain it. */
export function upstreamUrl(
baseUrl: string,
@@ -229,10 +255,14 @@ export function upstreamUrl(
path: string,
appendEndpointPath = true,
): string {
- const base = baseUrl.trim().replace(/\/+$/, '')
- if (!appendEndpointPath) return base
+ const [base, query] = splitBaseUrl(baseUrl)
+ if (!appendEndpointPath) {
+ return base + path.slice(PROTOCOL_PATHS[protocol].length) + query
+ }
return (
- base + (protocol === 'anthropic' ? path : path.replace(/^\/v1(?=\/)/, ''))
+ base +
+ (protocol === 'anthropic' ? path : path.replace(/^\/v1(?=\/)/, '')) +
+ query
)
}
@@ -245,10 +275,10 @@ export function baseUrlCorrection(
): { reason: 'anthropicVersion' | 'missingPath'; url: string } | undefined {
if (!upstream.append_endpoint_path || !isValidBaseUrl(upstream.base_url))
return undefined
- const base = upstream.base_url.trim().replace(/\/+$/, '')
+ const [base, query] = splitBaseUrl(upstream.base_url)
const path = new URL(base).pathname.replace(/\/+$/, '')
if (upstream.protocol === 'anthropic' && path.endsWith('/v1')) {
- return { reason: 'anthropicVersion', url: base.slice(0, -3) }
+ return { reason: 'anthropicVersion', url: base.slice(0, -3) + query }
}
if (
upstream.protocol !== 'anthropic' &&
@@ -257,7 +287,7 @@ export function baseUrlCorrection(
) {
return {
reason: 'missingPath',
- url: base + (upstream.vendor === 'openai' ? '/v1' : '/api/v3'),
+ url: base + (upstream.vendor === 'openai' ? '/v1' : '/api/v3') + query,
}
}
}
@@ -317,7 +347,11 @@ export function validateUpstream(
checkRequired(input.base_url) ??
(isValidBaseUrl(input.base_url)
? undefined
- : { key: 'validation.baseUrl' }),
+ : {
+ key: misplacedPlaceholder(input.base_url)
+ ? 'validation.baseUrlPlaceholder'
+ : 'validation.baseUrl',
+ }),
)
if (baseUrlCorrection(input)?.reason === 'anthropicVersion') {
collect(errors, 'base_url', {