Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

feat(gateway): test upstreams with model generation (#6118)

Caucher Wangc79600ae5ad0
diff --git a/docs/en/api/01-overview.md b/docs/en/api/01-overview.md
index b9707b347..68996bb9a 100644
--- a/docs/en/api/01-overview.md
+++ b/docs/en/api/01-overview.md
@@ -544,7 +544,7 @@ When the gateway runs inside the server, use the server's port; when it runs sta
| GET | `/gateway/admin/logs` | Read the request logs the caller can see |
| GET | `/gateway/admin/upstreams` | List the account's upstreams |
| PUT / DELETE | `/gateway/admin/upstreams/{identifier}` | Create/replace or delete an upstream |
-| POST | `/gateway/admin/upstreams/{identifier}/test` | Test an upstream connection |
+| POST | `/gateway/admin/upstreams/{identifier}/test` | Test an upstream connection or model |
| POST | `/gateway/admin/upstreams/import-vlm` | Import the server's VLM settings as upstreams (embedded only) |
| GET | `/gateway/admin/policies` | List the account's context profiles |
| PUT / DELETE | `/gateway/admin/policies/{identifier}` | Create/replace or delete a context profile |
diff --git a/docs/en/api/25-gateway.md b/docs/en/api/25-gateway.md
index 04552776a..b0159ee8f 100644
--- a/docs/en/api/25-gateway.md
+++ b/docs/en/api/25-gateway.md
@@ -14,8 +14,8 @@ The deployer can also set a gateway management key (see [Gateway management key]
| Operation | No management key set | Management key set |
| --- | --- | --- |
-| Read upstreams and context profiles, test upstreams | Any user in the account | Any user in the account; the management key alone also works, for any account |
-| Create, change or delete upstreams and context profiles | Any user in the account | Only requests with the correct management key |
+| Read upstreams and context profiles | Any user in the account | Any user in the account; the management key alone also works, for any account |
+| Create, change or delete upstreams and context profiles; test upstreams | Any user in the account | Only requests with the correct management key |
| Import OpenViking's model settings | Root in an embedded deployment (including local dev) | Only requests with the correct management key, and only in an embedded deployment |
| Issue, read and revoke gateway keys; reset saving | Users: their own keys; admins: their account; root: the selected account | Same; the management key adds no rights |
| Read logs and the overview | Users: their own records; admins and root: the whole account | Same; always needs an OpenViking identity |
@@ -66,7 +66,7 @@ If OpenViking rejects the API key, the response is 401 `openviking_http_401`.
| GET | `/gateway/admin/upstreams/{identifier}` | Read one upstream |
| PUT | `/gateway/admin/upstreams/{identifier}` | Create or replace an upstream |
| DELETE | `/gateway/admin/upstreams/{identifier}` | Delete an upstream no key uses |
-| POST | `/gateway/admin/upstreams/{identifier}/test` | Test the upstream connection |
+| POST | `/gateway/admin/upstreams/{identifier}/test` | Test the upstream connection or a model |
| POST | `/gateway/admin/upstreams/import-vlm` | Import OpenViking's model settings |
| GET | `/gateway/admin/policies` | List context profiles |
| GET | `/gateway/admin/policies/{identifier}` | Read one context profile |
@@ -90,20 +90,26 @@ curl -X PUT "$SERVER_URL/gateway/admin/upstreams/openai?account_id=team" \
Deleting an object that gateway keys still use returns 409 `in_use` with `key_count`. Revoke those keys first. If you only want requests to stop reaching an upstream right away, turn it off instead of deleting it.
-### Connection tests
+### Upstream tests
-`POST /gateway/admin/upstreams/{identifier}/test` asks the provider for its model list with the stored credentials, with a 10-second timeout, and returns `{ok, status, reason, url}`. `status` is left out when no response arrived. Read-only callers can run it too. Anthropic upstreams send the key in both `x-api-key` and `Authorization: Bearer`; Chat Completions and Responses upstreams send only Bearer.
+`POST /gateway/admin/upstreams/{identifier}/test` supports two modes. An empty `{}` body checks the connection with `GET /v1/models`. A `{"model":"<model-name>"}` body sends one small, non-streaming generation request with the stored credentials and headers. The model name is resolved through the upstream's aliases. Model requests use the configured Chat Completions, Responses or Anthropic Messages endpoint, including an exact endpoint URL when path appending is off. When gateway tools are allowed, they include the account's OpenViking tools filtered by the default context profile; a temporary tool-catalog failure falls back to a request without tools. The model test does not set a reasoning effort, so the model uses its default.
+
+The response includes `{ok, kind, status, reason, url}`, where `kind` is `connection` or `model`. Failed tests also include the upstream HTTP response in `response`, with `http_version`, `status`, `reason`, `headers`, `body` and an optional `truncated` flag. The body is limited to 16 KiB; `status` is left out when no response arrived. The timeout is the smaller of the configured upstream timeout and 60 seconds. Running either test requires write access; model tests make a real provider request and may incur a small charge.
+
+Anthropic tests send the key in both `x-api-key` and `Authorization: Bearer` and default `anthropic-version` to `2023-06-01` when the upstream headers do not set it. Chat Completions and Responses tests send only Bearer authentication.
| Provider response | `reason` | `ok` |
| --- | --- | --- |
+| The Coding Plan upstream has not been approved for gateway use. No request is sent. | `coding_plan_blocked` | `false` |
| No API key to use: none is stored, or the upstream expects each client to send its own. No request is sent. | `upstream_key_missing` | `false` |
-| 2xx with a `data` array in the body | `passed` | `true` |
-| 2xx with a body that signals failure, or 401 or 403 | `upstream_key_rejected` | `false` |
-| 404 or 405: the address answers, but the provider has no model list, so the key is not verified | `models_unavailable` | `true` |
-| 429: the address answers | `rate_limited` | `true` |
-| Network error or 5xx | `upstream_unavailable` | `false` |
-
-A passed test only shows that the address and key work. It does not show that a particular model is available, or that the upstream may be used with a Coding Plan.
+| 2xx with a response matching the configured model protocol | `passed` | `true` |
+| 2xx with invalid JSON or a response that does not match the configured protocol | `invalid_response` | `false` |
+| 401 or 403 | `upstream_key_rejected` | `false` |
+| 400, 404, 405 or 422 | `model_request_rejected` | `false` |
+| 429 | `rate_limited` | `false` |
+| Network error, timeout, redirect or 5xx | `upstream_unavailable` | `false` |
+
+A passed connection test confirms reachability and, when `/v1/models` is available, the stored credentials. A 404 or 405 still allows a model test because some providers do not expose that endpoint. A passed model test confirms that the selected model completed this request. It does not verify other models or authorize Coding Plan use.
### Import VLM settings
diff --git a/docs/en/guides/22-gateway-operations.md b/docs/en/guides/22-gateway-operations.md
index cbd8afe13..c7944604b 100644
--- a/docs/en/guides/22-gateway-operations.md
+++ b/docs/en/guides/22-gateway-operations.md
@@ -215,7 +215,7 @@ One storage directory serves one deployment on one host, and an embedded and a s
Connect Studio with an OpenViking API key and open **OpenViking Gateway** in the sidebar. Upstreams and context profiles are shared by the whole account; gateway keys and request logs belong to individual users.
- **No management key set**: every signed-in user can add and change the account's upstreams and context profiles.
-- **Management key set**: the Upstreams and Profiles tabs are read-only, for OpenViking admins and root too. Read-only users can still view and test upstreams (without their API keys and headers) and pick them when issuing keys for themselves. Choose **Enter gateway management key** to unlock editing; the header then shows **Editing upstreams and profiles**. Editing stays unlocked until you choose **Lock** or close the browser tab. If the management key is changed, Studio locks again on its own.
+- **Management key set**: the Upstreams and Profiles tabs are read-only, for OpenViking admins and root too. Read-only users can still view upstreams (without their API keys and headers) and pick them when issuing keys for themselves, but model tests require write access because they make billable provider requests. Choose **Enter gateway management key** to unlock editing and testing; the header then shows **Editing upstreams and profiles**. Editing stays unlocked until you choose **Lock** or close the browser tab. If the management key is changed, Studio locks again on its own.
- **Gateway keys**: everyone can issue keys for themselves; account admins can issue keys for users in their account; root first switches account in the sidebar. See [Gateway keys](#gateway-keys).
Saved upstreams and context profiles take effect at once for the keys that use them (a context profile only affects conversations that start afterwards). If no usable upstream or context profile exists yet and you cannot change them, ask whoever deployed the gateway.
@@ -279,7 +279,7 @@ All upstream settings except priority take effect at once, in ongoing conversati
#### Test and delete
-Choose **Test** in the upstream list. The gateway asks the provider for its model list with the stored credentials and tells you whether the address answered and the key was accepted. Read-only users can run it too. The test does not check a specific model, and passing it does not mean Coding Plan use is allowed; each result is explained in [Connection tests](../api/25-gateway.md#connection-tests).
+With write access, choose **Test** in the upstream list. The gateway checks the saved connection first. Once that succeeds, select or enter a model name and run the model test. The gateway sends one small generation request with the stored credentials and reports whether the endpoint, key, model and protocol worked together. It includes OpenViking tools when the upstream allows them and leaves reasoning effort at the model default. The provider may charge for the model request. Coding Plan upstreams stay blocked from model tests until gateway use is approved. Each result is explained in [Upstream tests](../api/25-gateway.md#upstream-tests).
An upstream that keys still use cannot be deleted. Revoke those keys first, or turn the upstream off instead.
diff --git a/docs/zh/api/01-overview.md b/docs/zh/api/01-overview.md
index 00e917525..a301f5a18 100644
--- a/docs/zh/api/01-overview.md
+++ b/docs/zh/api/01-overview.md
@@ -539,7 +539,7 @@ JSON 输出 - 错误:
| GET | `/gateway/admin/logs` | 读取调用方可见的请求日志 |
| GET | `/gateway/admin/upstreams` | 列出本账号的上游 |
| PUT / DELETE | `/gateway/admin/upstreams/{identifier}` | 创建/替换或删除上游 |
-| POST | `/gateway/admin/upstreams/{identifier}/test` | 测试上游连接 |
+| POST | `/gateway/admin/upstreams/{identifier}/test` | 测试上游连接或模型 |
| POST | `/gateway/admin/upstreams/import-vlm` | 把 Server 的 VLM 配置导入为上游(仅嵌入部署) |
| GET | `/gateway/admin/policies` | 列出本账号的上下文配置 |
| PUT / DELETE | `/gateway/admin/policies/{identifier}` | 创建/替换或删除上下文配置 |
diff --git a/docs/zh/api/25-gateway.md b/docs/zh/api/25-gateway.md
index 966aef916..a6a4a7086 100644
--- a/docs/zh/api/25-gateway.md
+++ b/docs/zh/api/25-gateway.md
@@ -14,8 +14,8 @@
| 操作 | 未设置管理密钥 | 已设置管理密钥 |
| --- | --- | --- |
-| 读取上游和上下文配置、测试上游 | 本账号的任何用户 | 本账号的任何用户;单独持有管理密钥也可以,并能指定任意账号 |
-| 创建、修改、删除上游或上下文配置 | 本账号的任何用户 | 只有带正确管理密钥的请求 |
+| 读取上游和上下文配置 | 本账号的任何用户 | 本账号的任何用户;单独持有管理密钥也可以,并能指定任意账号 |
+| 创建、修改或删除上游和上下文配置;测试上游 | 本账号的任何用户 | 只有带正确管理密钥的请求 |
| 导入 OpenViking 的模型配置 | 嵌入部署中的 root(包括本地 dev) | 只有带正确管理密钥的请求,且只限嵌入部署 |
| 签发、读取、吊销网关密钥,重置保存 | user 只能操作自己的密钥;admin 操作本账号;root 操作所选账号 | 同左,管理密钥不增加任何权限 |
| 读取日志和概览 | user 只看自己的;admin 和 root 看整个账号 | 同左,始终需要 OpenViking 身份 |
@@ -66,7 +66,7 @@ OpenViking 拒绝 API 密钥时,返回 401 `openviking_http_401`。
| GET | `/gateway/admin/upstreams/{identifier}` | 读取单个上游 |
| PUT | `/gateway/admin/upstreams/{identifier}` | 创建或替换上游 |
| DELETE | `/gateway/admin/upstreams/{identifier}` | 删除没有密钥在用的上游 |
-| POST | `/gateway/admin/upstreams/{identifier}/test` | 测试上游连接 |
+| POST | `/gateway/admin/upstreams/{identifier}/test` | 测试上游连接或模型 |
| POST | `/gateway/admin/upstreams/import-vlm` | 导入 OpenViking 的模型配置 |
| GET | `/gateway/admin/policies` | 列出上下文配置 |
| GET | `/gateway/admin/policies/{identifier}` | 读取单个上下文配置 |
@@ -90,20 +90,26 @@ curl -X PUT "$SERVER_URL/gateway/admin/upstreams/openai?account_id=team" \
删除仍有网关密钥在用的对象,返回 409 `in_use` 和 `key_count`。先吊销这些密钥;只是想让请求立即停止发往某个上游,可以停用它而不删除。
-### 连接测试
+### 上游测试
-`POST /gateway/admin/upstreams/{identifier}/test` 用已存的凭证请求服务商的模型列表,超时 10 秒,返回 `{ok, status, reason, url}`;没有收到响应时不含 `status`。只读调用方也能使用。Anthropic 上游同时在 `x-api-key` 和 `Authorization: Bearer` 中发送密钥,Chat Completions 和 Responses 上游只发送 Bearer。
+`POST /gateway/admin/upstreams/{identifier}/test` 支持两种模式:空请求体 `{}` 通过 `GET /v1/models` 检查连接;`{"model":"<模型名>"}` 使用已存的凭证和请求头发送一次很小的非流式生成请求。模型名会先经过上游的别名映射。模型请求使用已配置的 Chat Completions、Responses 或 Anthropic Messages 接口;关闭路径拼接时,也会遵循配置的完整接口 URL。上游允许 Gateway 工具时,请求会带上当前账户的 OpenViking 工具,并按默认上下文配置筛选;工具目录暂时不可用时,降级为不带工具继续测试。模型测试不设置 reasoning effort,由模型使用默认值。
+
+响应包含 `{ok, kind, status, reason, url}`,其中 `kind` 为 `connection` 或 `model`。测试失败时还会在 `response` 中返回完整的上游 HTTP 响应,包括 `http_version`、`status`、`reason`、`headers`、`body` 和可选的 `truncated` 标记。响应体最多 16 KiB;没有收到响应时不含 `status`。超时时间取已配置上游超时与 60 秒中的较小值。两种测试都需要写权限;模型测试会真实请求服务商并可能产生少量费用。
+
+Anthropic 测试同时在 `x-api-key` 和 `Authorization: Bearer` 中发送密钥;上游请求头未设置 `anthropic-version` 时,默认使用 `2023-06-01`。Chat Completions 和 Responses 测试只发送 Bearer 鉴权。
| 服务商的响应 | `reason` | `ok` |
| --- | --- | --- |
+| Coding Plan 上游尚未授权网关使用。网关不发请求。 | `coding_plan_blocked` | `false` |
| 没有可用的 API Key:没有保存,或者上游要求每个客户端自带。网关不发请求。 | `upstream_key_missing` | `false` |
-| 2xx,响应体带 `data` 数组 | `passed` | `true` |
-| 2xx 但响应体表示失败,或者 401、403 | `upstream_key_rejected` | `false` |
-| 404、405:地址可达,但服务商不提供模型列表,密钥没有得到验证 | `models_unavailable` | `true` |
-| 429:地址可达 | `rate_limited` | `true` |
-| 网络错误或 5xx | `upstream_unavailable` | `false` |
-
-测试通过只说明地址和密钥可用,不代表某个具体模型可用,也不代表上游允许使用 Coding Plan。
+| 2xx,响应符合已配置的模型协议 | `passed` | `true` |
+| 2xx,但响应不是合法 JSON 或不符合已配置的协议 | `invalid_response` | `false` |
+| 401 或 403 | `upstream_key_rejected` | `false` |
+| 400、404、405 或 422 | `model_request_rejected` | `false` |
+| 429 | `rate_limited` | `false` |
+| 网络错误、超时、重定向或 5xx | `upstream_unavailable` | `false` |
+
+连接测试通过说明地址可达;服务商提供 `/v1/models` 时也会验证已存凭证。部分服务商不提供该接口,返回 404 或 405 后仍可继续测试模型。模型测试通过说明所选模型完成了本次请求,不代表其他模型也可用,也不会授权使用 Coding Plan。
### 导入 VLM 配置
diff --git a/docs/zh/guides/22-gateway-operations.md b/docs/zh/guides/22-gateway-operations.md
index f83698d53..b89b46768 100644
--- a/docs/zh/guides/22-gateway-operations.md
+++ b/docs/zh/guides/22-gateway-operations.md
@@ -215,7 +215,7 @@ ov.example.com {
用 OpenViking API 密钥连接 Studio 后,在侧边栏打开 **OpenViking 网关**。上游和上下文配置由整个账号共享,网关密钥和请求日志属于各个用户:
- **没有设置管理密钥**:每个登录的用户都能添加和修改本账号的上游和上下文配置。
-- **设置了管理密钥**:“上游”和“上下文配置”标签页变为只读,OpenViking 的 admin 和 root 也一样。只读时仍能查看和测试上游(看不到 API Key 和请求头),并在给自己签发密钥时选用。点**输入网关管理密钥**解锁编辑,页头会显示“可修改上游和上下文配置”;解锁状态保持到点击**锁定**或关闭浏览器标签页。管理密钥被更换后,Studio 会自动锁定。
+- **设置了管理密钥**:“上游”和“上下文配置”标签页变为只读,OpenViking 的 admin 和 root 也一样。只读时仍能查看上游(看不到 API Key 和请求头),并在给自己签发密钥时选用;模型测试会真实请求服务商并可能计费,因此需要写权限。点**输入网关管理密钥**解锁编辑和测试,页头会显示“可修改上游和上下文配置”;解锁状态保持到点击**锁定**或关闭浏览器标签页。管理密钥被更换后,Studio 会自动锁定。
- **网关密钥**:每个人都能给自己签发;账号管理员可以给本账号的用户签发;root 先在侧边栏切换账号。见[网关密钥](#网关密钥)。
上游和上下文配置保存后,立即对使用它们的密钥生效(上下文配置只影响之后开始的对话)。还没有可用的上游或上下文配置、自己又不能修改时,请联系部署网关的人。
@@ -279,7 +279,7 @@ ov.example.com {
#### 测试和删除
-在上游列表里点**测试**,网关用已存的凭证请求服务商的模型列表,告诉你地址是否有响应、密钥是否被接受。只读用户也能测试。测试不验证具体模型,也不代表允许使用 Coding Plan;各种结果的含义见[连接测试](../api/25-gateway.md#连接测试)。
+有写权限时,在上游列表里点**测试**,网关先检查已保存配置的连接。连接成功后,再选择或输入模型名运行模型测试。网关使用已存的凭证发送一次很小的生成请求,验证接口地址、密钥、模型名和协议能否一起正常工作。上游允许 Gateway 工具时会带上 OpenViking 工具,reasoning effort 使用模型默认值。服务商可能对模型请求计费。Coding Plan 上游在授权网关使用前仍会阻止模型测试。各种结果的含义见[上游测试](../api/25-gateway.md#上游测试)。
有密钥在用的上游不能删除:先吊销这些密钥,或者改为停用这个上游。
diff --git a/openviking_gateway/management.py b/openviking_gateway/management.py
index 691337ecb..0e9ddcbc8 100644
--- a/openviking_gateway/management.py
+++ b/openviking_gateway/management.py
@@ -5,27 +5,39 @@
import asyncio
import copy
import hmac
+import json
import secrets
import time
from dataclasses import dataclass
from urllib.parse import quote
import aiohttp
+import orjson
from fastapi import APIRouter, HTTPException, Request
from pydantic import ValidationError
from .capture import reset_capture
from .client import VikingError
from .config import MOUNT_PATH
-from .models import CaptureReset, KeyRequest, Policy, Upstream
+from .models import CaptureReset, KeyRequest, Policy, Upstream, UpstreamTestRequest
+from .protocols import text_content
from .proxy import upstream_headers, upstream_url
from .records import RecordKind as K
from .storage import delete_account_data, delete_user_data, digest
+from .tool_catalog import select_tools
+from .tool_protocols import ResponseCapture, tool_protocol
PREFIX = "/admin"
MANAGEMENT_HEADER = "x-openviking-gateway-management-key"
# Fields a VLM re-import refreshes; everything else may have been edited in the gateway.
VLM_SOURCE_FIELDS = ("base_url", "api_key", "headers", "protocol", "vendor", "coding_plan")
+UPSTREAM_TEST_PATHS = {
+ "anthropic": "/v1/messages",
+ "chat": "/v1/chat/completions",
+ "responses": "/v1/responses",
+}
+UPSTREAM_TEST_PROMPT = "Reply with only OK."
+UPSTREAM_TEST_RESPONSE_LIMIT = 16_384
DETAILS = {
"management_key_invalid": "The gateway management key is incorrect.",
"management_key_required": "Changing upstreams and profiles requires the management key.",
@@ -45,6 +57,42 @@ DETAILS = {
}
+def upstream_test_payload(upstream, model, tools):
+ """Build one small, non-streaming generation request for the configured protocol."""
+ model = upstream.get("aliases", {}).get(model, model)
+ common = {"model": model, "stream": False}
+ if upstream["protocol"] == "responses":
+ payload = {
+ **common,
+ "input": UPSTREAM_TEST_PROMPT,
+ "max_output_tokens": 64,
+ "store": False,
+ }
+ else:
+ payload = {
+ **common,
+ "messages": [{"role": "user", "content": UPSTREAM_TEST_PROMPT}],
+ "max_tokens": 64,
+ }
+ if upstream["protocol"] == "chat" and upstream.get("vendor") == "openai":
+ payload["max_completion_tokens"] = payload.pop("max_tokens")
+ if tools:
+ tool_protocol(upstream["protocol"]).add_tools(payload, tools)
+ return payload
+
+
+def upstream_test_output(protocol, body):
+ """Return whether a JSON body contains generated text for the protocol."""
+ if not isinstance(body, dict) or body.get("error"):
+ return False
+ try:
+ capture = ResponseCapture(protocol)
+ capture.nonstream(body)
+ return bool(capture.message and text_content(capture.message).strip())
+ except (AttributeError, KeyError, TypeError):
+ return False
+
+
def fail(status, reason, detail=None, **extra):
raise HTTPException(status, {"reason": reason, "detail": detail or DETAILS[reason], **extra})
@@ -93,6 +141,17 @@ class Caller:
return self.user_id if self.role == "user" else None
+async def caller_tool_catalog(app, management, who):
+ """Read the account's MCP tools with a credential OpenViking accepts."""
+ key = who.credential
+ if who.role == "root" and who.identity.get("auth_mode") != "dev":
+ # OV refuses root on /mcp. Every caller sees the same catalog, so read it with
+ # a key issued in this account, or fall back to the last catalog read.
+ keys = await management.list(who.account_id, "keys")
+ key = next((item["openviking_key"] for item in keys), "")
+ return await app.state.viking.tools(key) if key else app.state.viking.last_tools or []
+
+
async def caller(request, *, require_identity=True, write=False):
state = request.app.state
configured = state.config.management_key
@@ -204,13 +263,7 @@ def register_management(app):
@router.get("/tools")
async def tools(request: Request):
who = await caller(request)
- key = who.credential
- if who.role == "root" and who.identity.get("auth_mode") != "dev":
- # OV refuses root on /mcp. Every caller sees the same catalog, so read it with
- # a key issued in this account, or fall back to the last catalog read.
- keys = await management.list(who.account_id, "keys")
- key = next((k["openviking_key"] for k in keys), "")
- catalog = await app.state.viking.tools(key) if key else app.state.viking.last_tools or []
+ catalog = await caller_tool_catalog(app, management, who)
return [
{
"name": t["name"],
@@ -400,53 +453,123 @@ def register_management(app):
@router.post("/upstreams/{identifier}/test")
async def test_upstream(identifier: str, request: Request):
- who = await caller(request, require_identity=False)
+ who = await caller(request, require_identity=False, write=True)
upstream = await management.get(who.account_id, "upstreams", identifier)
if not upstream:
fail(404, "not_found")
- url = upstream_url(upstream, "/v1/models")
+ data = await read_body(request, UpstreamTestRequest)
+ assert data is not None
+ generating = data.model is not None
+ kind = "model" if generating else "connection"
+ path = UPSTREAM_TEST_PATHS[upstream["protocol"]] if generating else "/v1/models"
+ url = upstream_url(upstream, path)
+ if generating and upstream.get("coding_plan") and not upstream.get("allow_coding_plan"):
+ return {"ok": False, "reason": "coding_plan_blocked", "url": url, "kind": kind}
try:
headers = upstream_headers(upstream, {})
except HTTPException as error:
if error.status_code != 401:
raise
# No stored key (passthrough keys come from clients): nothing is sent.
- return {"ok": False, "reason": "upstream_key_missing", "url": url}
+ return {"ok": False, "reason": "upstream_key_missing", "url": url, "kind": kind}
+ payload = None
+ if generating:
+ headers.setdefault("content-type", "application/json")
+ if upstream["protocol"] == "anthropic":
+ headers.setdefault("anthropic-version", "2023-06-01")
+ tools = []
+ if upstream.get("allow_gateway_tools", True):
+ policy = await management.get(who.account_id, "policies", "default")
+ try:
+ tools = select_tools(
+ await caller_tool_catalog(app, management, who),
+ policy or Policy().model_dump(),
+ )
+ except VikingError:
+ pass
+ payload = upstream_test_payload(upstream, data.model, tools)
try:
- async with app.state.http.get(
+ async with app.state.http.request(
+ "POST" if generating else "GET",
url,
+ data=orjson.dumps(payload) if payload is not None else None,
headers=headers,
- timeout=aiohttp.ClientTimeout(total=10),
+ timeout=aiohttp.ClientTimeout(
+ total=min(app.state.config.upstream_timeout_seconds, 60)
+ ),
allow_redirects=False,
) as response:
status = response.status
- if status in {404, 405}:
- reason, ok = "models_unavailable", True
- elif status == 429:
- reason, ok = "rate_limited", True
- elif 200 <= status < 300:
+ raw = bytearray()
+ while len(raw) <= UPSTREAM_TEST_RESPONSE_LIMIT:
+ chunk = await response.content.read(
+ UPSTREAM_TEST_RESPONSE_LIMIT + 1 - len(raw)
+ )
+ if not chunk:
+ break
+ raw.extend(chunk)
+ truncated = len(raw) > UPSTREAM_TEST_RESPONSE_LIMIT
+ response_text = raw[:UPSTREAM_TEST_RESPONSE_LIMIT].decode("utf-8", errors="replace")
+ result = {
+ "ok": False,
+ "status": status,
+ "url": url,
+ "kind": kind,
+ }
+ if not generating and status in {404, 405}:
+ result.update(ok=True, reason="models_unavailable")
+ elif not generating and status == 429:
+ result.update(ok=True, reason="rate_limited")
+ elif not generating and 200 <= status < 300:
try:
- body = await response.json(content_type=None)
- except ValueError:
- body = {}
- ok = (
+ body = json.loads(response_text) if not truncated else None
+ except (TypeError, ValueError):
+ body = None
+ valid = (
isinstance(body, dict)
and isinstance(body.get("data"), list)
and not body.get("error")
)
- reason = "passed" if ok else "upstream_key_rejected"
- else:
- reason, ok = (
- (
- "upstream_key_rejected"
- if status in {401, 403}
- else "upstream_unavailable"
- ),
- False,
+ result.update(
+ ok=valid,
+ reason="passed" if valid else "upstream_key_rejected",
)
- return {"ok": ok, "status": status, "reason": reason, "url": url}
+ elif 200 <= status < 300:
+ try:
+ body = json.loads(response_text) if not truncated else None
+ except (TypeError, ValueError):
+ body = None
+ valid = upstream_test_output(upstream["protocol"], body)
+ result.update(
+ ok=valid,
+ reason="passed" if valid else "invalid_response",
+ )
+ elif status in {401, 403}:
+ result["reason"] = "upstream_key_rejected"
+ elif status == 429:
+ result["reason"] = "rate_limited"
+ elif status in {400, 404, 405, 422}:
+ result["reason"] = "model_request_rejected"
+ else:
+ result["reason"] = "upstream_unavailable"
+ if not result["ok"]:
+ response_headers = {}
+ for name in response.headers:
+ if name in response_headers:
+ continue
+ values = response.headers.getall(name)
+ response_headers[name] = values[0] if len(values) == 1 else values
+ result["response"] = {
+ "http_version": (f"HTTP/{response.version.major}.{response.version.minor}"),
+ "status": status,
+ "reason": response.reason,
+ "headers": response_headers,
+ "body": response_text,
+ **({"truncated": True} if truncated else {}),
+ }
+ return result
except (aiohttp.ClientError, asyncio.TimeoutError):
- return {"ok": False, "reason": "upstream_unavailable", "url": url}
+ return {"ok": False, "reason": "upstream_unavailable", "url": url, "kind": kind}
app.include_router(router)
diff --git a/openviking_gateway/models.py b/openviking_gateway/models.py
index 54b4a6d9c..02335f273 100644
--- a/openviking_gateway/models.py
+++ b/openviking_gateway/models.py
@@ -155,6 +155,11 @@ class Upstream(BaseModel):
return value
+class UpstreamTestRequest(BaseModel):
+ model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+ model: str | None = Field(default=None, min_length=1, max_length=256)
+
+
class KeyRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str
diff --git a/tests/gateway/test_app.py b/tests/gateway/test_app.py
index b4782c2f4..cf44254b3 100644
--- a/tests/gateway/test_app.py
+++ b/tests/gateway/test_app.py
@@ -892,7 +892,7 @@ async def test_large_first_tool_request_keeps_client_token_limit(running_gateway
@pytest.mark.parametrize("vendor", ["ark", "byteplus"])
-async def test_ark_connection_test_lists_models_under_api_v3(running_gateway, vendor):
+async def test_ark_connection_test_generates_under_api_v3(running_gateway, vendor):
app, client, admin, _, seen, _ = running_gateway
upstream = await app.state.management.get("tenant", "upstreams", "chat")
await app.state.management.save(
@@ -905,9 +905,11 @@ async def test_ark_connection_test_lists_models_under_api_v3(running_gateway, ve
"base_url": upstream["base_url"].removesuffix("/v1") + "/api/v3",
},
)
- response = await client.post("/admin/upstreams/chat/test", headers=admin)
+ response = await client.post(
+ "/admin/upstreams/chat/test", headers=admin, json={"model": "model"}
+ )
assert response.json()["ok"] and response.json()["status"] == 200
- assert seen[-1][0] == "/api/v3/models"
+ assert seen[-1][0] == "/api/v3/chat/completions"
async def test_ark_burst_is_forwarded_without_local_429(running_gateway):
diff --git a/tests/gateway/test_management.py b/tests/gateway/test_management.py
index bb6362810..853433fb3 100644
--- a/tests/gateway/test_management.py
+++ b/tests/gateway/test_management.py
@@ -1,5 +1,8 @@
"""Identity, independent management authorization, and standalone hosting contracts."""
+import asyncio
+import json
+
import httpx
import pytest
from aiohttp import web
@@ -307,32 +310,230 @@ async def test_data_catchall_is_narrow(running_gateway, path):
assert (await client.get(path, headers={"X-API-Key": key["key"]})).status_code == 404
+ "status,body,ok,reason",
+ [
+ (
+ 200,
+ {"choices": [{"message": {"role": "assistant", "content": "OK"}}]},
+ True,
+ "passed",
+ ),
+ (
+ 200,
+ {"choices": [{"message": {"role": "assistant", "content": None}}]},
+ False,
+ "invalid_response",
+ ),
+ (200, {"error": {"code": "bad"}}, False, "invalid_response"),
+ (401, {}, False, "upstream_key_rejected"),
+ (403, {}, False, "upstream_key_rejected"),
+ (400, {}, False, "model_request_rejected"),
+ (404, {}, False, "model_request_rejected"),
+ (405, {}, False, "model_request_rejected"),
+ (429, {}, False, "rate_limited"),
+ (503, {}, False, "upstream_unavailable"),
+ ],
+)
+async def test_upstream_test_status_contract(running_gateway, status, body, ok, reason):
+ app, client, admin, _, _, _ = running_gateway
+
+ async def handler(request):
+ if request.path == "/v1/chat/completions":
+ return web.json_response(body, status=status)
+
+ app.state.test_backend["handler"] = handler
+ result = (
+ await client.post(P + "/upstreams/chat/test", headers=admin, json={"model": "model"})
+ ).json()
+ assert (
+ result["ok"] is ok
+ and result["kind"] == "model"
+ and result["reason"] == reason
+ and result["url"].endswith("/v1/chat/completions")
+ )
+ if ok:
+ assert "response" not in result
+ else:
+ upstream_response = result["response"]
+ assert upstream_response["status"] == status
+ assert upstream_response["http_version"].startswith("HTTP/")
+ assert upstream_response["reason"]
+ assert any(name.lower() == "content-type" for name in upstream_response["headers"])
+ assert json.loads(upstream_response["body"]) == body
+
+
@pytest.mark.parametrize(
"status,body,ok,reason",
[
(200, {"data": []}, True, "passed"),
(200, {"error": {"code": "bad"}}, False, "upstream_key_rejected"),
(401, {}, False, "upstream_key_rejected"),
- (403, {}, False, "upstream_key_rejected"),
(404, {}, True, "models_unavailable"),
(405, {}, True, "models_unavailable"),
(429, {}, True, "rate_limited"),
(503, {}, False, "upstream_unavailable"),
],
)
-async def test_upstream_test_status_contract(running_gateway, status, body, ok, reason):
+async def test_upstream_connection_status_contract(running_gateway, status, body, ok, reason):
app, client, admin, _, _, _ = running_gateway
+ requests = []
async def handler(request):
if request.path == "/v1/models":
+ requests.append((request.method, await request.read()))
return web.json_response(body, status=status)
app.state.test_backend["handler"] = handler
- result = (await client.post(P + "/upstreams/chat/test", headers=admin)).json()
- assert (
- result["ok"] is ok and result["reason"] == reason and result["url"].endswith("/v1/models")
+ result = (await client.post(P + "/upstreams/chat/test", headers=admin, json={})).json()
+
+ assert result["ok"] is ok
+ assert result["kind"] == "connection"
+ assert result["reason"] == reason
+ assert result["url"].endswith("/v1/models")
+ assert requests == [("GET", b"")]
+
+
+async def test_upstream_test_reads_chunked_response_to_eof(running_gateway):
+ app, client, admin, _, _, _ = running_gateway
+ body = json.dumps(
+ {"choices": [{"message": {"role": "assistant", "content": "OK"}}]}
+ ).encode()
+
+ async def handler(request):
+ if request.path == "/v1/chat/completions":
+ response = web.StreamResponse(headers={"Content-Type": "application/json"})
+ await response.prepare(request)
+ for start in range(0, len(body), 7):
+ await response.write(body[start : start + 7])
+ await asyncio.sleep(0)
+ await response.write_eof()
+ return response
+
+ app.state.test_backend["handler"] = handler
+ result = (
+ await client.post(P + "/upstreams/chat/test", headers=admin, json={"model": "model"})
+ ).json()
+
+ assert result["ok"] and result["reason"] == "passed"
+
+
+ "protocol,path,input_field,limit_field",
+ [
+ ("chat", "/v1/chat/completions", "messages", "max_tokens"),
+ ("responses", "/v1/responses", "input", "max_output_tokens"),
+ ("anthropic", "/v1/messages", "messages", "max_tokens"),
+ ],
+)
+async def test_upstream_test_generates_with_protocol_and_alias(
+ running_gateway, protocol, path, input_field, limit_field
+):
+ app, client, admin, _, captured, _ = running_gateway
+ upstream = await app.state.management.get("tenant", "upstreams", protocol)
+ await app.state.management.save(
+ "tenant",
+ "upstreams",
+ protocol,
+ {**upstream, "aliases": {"client-model": "model"}},
+ )
+
+ result = (
+ await client.post(
+ P + f"/upstreams/{protocol}/test",
+ headers=admin,
+ json={"model": "client-model"},
+ )
+ ).json()
+
+ assert result["ok"] and result["reason"] == "passed"
+ sent_path, raw, headers = captured[-1]
+ payload = json.loads(raw)
+ assert sent_path == path
+ assert payload["model"] == "model"
+ assert input_field in payload and payload[limit_field] == 64
+ assert payload["stream"] is False
+ assert "reasoning" not in payload and "reasoning_effort" not in payload
+ assert headers["Authorization"] == "Bearer model-secret"
+ assert payload["tools"]
+ if protocol == "chat":
+ assert payload["tools"][0]["function"]["name"] == "openviking_find"
+ assert "parameters" in payload["tools"][0]["function"]
+ elif protocol == "responses":
+ assert payload["tools"][0]["name"] == "openviking_find"
+ assert "parameters" in payload["tools"][0]
+ assert "reasoning.encrypted_content" in payload["include"]
+ else:
+ assert payload["tools"][0]["name"] == "openviking_find"
+ assert "input_schema" in payload["tools"][0]
+ if protocol == "anthropic":
+ assert headers["anthropic-version"] == "2023-06-01"
+
+
+async def test_upstream_test_honors_exact_endpoint_url(running_gateway):
+ app, client, admin, _, captured, _ = running_gateway
+ upstream = await app.state.management.get("tenant", "upstreams", "chat")
+ await app.state.management.save(
+ "tenant",
+ "upstreams",
+ "chat",
+ {
+ **upstream,
+ "append_endpoint_path": False,
+ "allow_gateway_tools": False,
+ "headers": {
+ "X-Test-Header": "stored-value",
+ "content-type": "application/vnd.test+json",
+ },
+ },
)
+ result = (
+ await client.post(
+ P + "/upstreams/chat/test",
+ headers=admin,
+ json={"model": "model"},
+ )
+ ).json()
+
+ assert result["ok"] and result["url"].endswith("/v1")
+ assert captured[-1][0] == "/v1"
+ assert captured[-1][2]["X-Test-Header"] == "stored-value"
+ sent_headers = {name.lower(): value for name, value in captured[-1][2].items()}
+ assert sent_headers["content-type"] == "application/vnd.test+json"
+ assert "tools" not in json.loads(captured[-1][1])
+
+
+async def test_upstream_test_continues_without_unavailable_tools(running_gateway):
+ app, client, admin, _, captured, _ = running_gateway
+ app.state.test_backend["tools_status"] = {"admin-key": 503}
+
+ result = (
+ await client.post(
+ P + "/upstreams/chat/test",
+ headers=admin,
+ json={"model": "model"},
+ )
+ ).json()
+
+ assert result["ok"]
+ assert "tools" not in json.loads(captured[-1][1])
+
+
+async def test_upstream_test_requires_write_access(running_gateway):
+ app, client, admin, _, captured, _ = running_gateway
+ app.state.config.management_key = MANAGER[next(iter(MANAGER))]
+
+ response = await client.post(
+ P + "/upstreams/chat/test",
+ headers=admin,
+ json={"model": "model"},
+ )
+
+ assert response.status_code == 403
+ assert response.json()["reason"] == "management_key_required"
+ assert not [path for path, _, _ in captured if path.endswith("/chat/completions")]
+
async def test_vlm_permissions_idempotency_and_confirmation(running_gateway):
app, client, admin, _, _, _ = running_gateway
@@ -552,14 +753,17 @@ async def test_connection_test_is_available_for_missing_key_and_blocked_plan(run
upstream = await app.state.management.get("tenant", "upstreams", "chat")
for change in ({"api_key": ""}, {"auth_mode": "passthrough"}):
await app.state.management.save("tenant", "upstreams", "chat", {**upstream, **change})
- response = await client.post(P + "/upstreams/chat/test", headers=admin)
+ response = await client.post(
+ P + "/upstreams/chat/test", headers=admin, json={"model": "model"}
+ )
assert response.status_code == 200 and response.json()["reason"] == "upstream_key_missing"
- assert not [path for path, _, _ in captured if path.endswith("/models")]
+ assert not [path for path, _, _ in captured if path.endswith("/chat/completions")]
await app.state.management.save(
"tenant", "upstreams", "chat", {**upstream, "coding_plan": True, "allow_coding_plan": False}
)
- response = await client.post(P + "/upstreams/chat/test", headers=admin)
- assert response.status_code == 200 and response.json()["ok"]
+ response = await client.post(P + "/upstreams/chat/test", headers=admin, json={"model": "model"})
+ assert response.status_code == 200 and response.json()["reason"] == "coding_plan_blocked"
+ assert not [path for path, _, _ in captured if path.endswith("/chat/completions")]
@pytest.mark.parametrize("mode", ["unavailable", "trusted", "unknown"])
diff --git a/web-studio/src/i18n/locales/en/gateway/upstreams.ts b/web-studio/src/i18n/locales/en/gateway/upstreams.ts
index f5e3f1a3c..d6a0f0807 100644
--- a/web-studio/src/i18n/locales/en/gateway/upstreams.ts
+++ b/web-studio/src/i18n/locales/en/gateway/upstreams.ts
@@ -80,30 +80,53 @@ const upstreams = {
},
test: {
action: 'Test',
- connection: 'Test connection',
+ connection: 'Test',
running: 'Testing…',
- hint: 'Asks the provider for its model list with the stored key and headers. This checks the address and the key, not the models or chat requests.',
+ hint: 'Checks the upstream connection, then lets you test a model.',
savedOnly: 'Tests the saved settings. Save first to test your changes.',
passthrough:
'Each client sends its own key, so there is no stored key to test with.',
noKey: 'Add an API key first.',
+ codingPlan:
+ 'This Coding Plan has not been approved for model requests through the gateway.',
result: {
- unverified: 'Reachable, key unverified · {{status}}',
- ok: 'Reachable · {{status}}',
+ connected: 'Connected · {{status}}',
+ connectionUnverified: 'Connected · {{status}}',
+ ok: 'Model works · {{status}}',
failed: 'Failed · {{status}}',
unreachable: 'Unreachable',
error: "Couldn't test",
},
explain: {
- ok: 'The provider accepted the key and returned its model list.',
+ connected: 'The provider accepted the key and returned its model list.',
+ connectionUnverified:
+ 'The address is reachable, but the provider does not offer a model list, so the key is not verified. You can still test a model.',
+ ok: 'The selected model generated a reply.',
auth: 'The provider rejected the API key or one of the headers.',
rateLimited:
- 'The address is reachable, but the provider is rate limiting requests. The key has not been verified.',
- notFound:
- 'The address is reachable, but the provider does not offer a model list. The API key has not been verified.',
+ 'The provider rate limited the request, so model generation did not complete.',
+ requestRejected:
+ 'The provider rejected the generation request (HTTP {{status}}). Check the model name, protocol and complete endpoint URL.',
+ invalidResponse:
+ 'The provider returned a success status, but its response does not match the selected protocol.',
status: 'The provider answered with HTTP {{status}}.',
unreachable:
- "The gateway couldn't connect within 10 seconds. Check the base URL and the network between the gateway and the provider.",
+ "The gateway couldn't connect or the request timed out. Check the base URL and the network between the gateway and the provider.",
+ },
+ dialog: {
+ title: 'Test “{{name}}”',
+ description:
+ 'Checks the saved connection first, then sends one small non-streaming request to test the selected model.',
+ connection: 'Upstream connection',
+ checkingConnection: 'Checking connection…',
+ model: 'Model',
+ modelPlaceholder: 'Enter a provider model name',
+ configuredModel:
+ 'Choose a configured model or enter another provider model name.',
+ unconfiguredModel:
+ 'This upstream allows any model. Enter a model name supported by the provider.',
+ response: 'Upstream response (JSON)',
+ run: 'Run test',
},
},
delete: {
diff --git a/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts b/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts
index 20d22b643..01d09f6a0 100644
--- a/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts
+++ b/web-studio/src/i18n/locales/zh-CN/gateway/upstreams.ts
@@ -78,27 +78,47 @@ const upstreams = {
},
test: {
action: '测试',
- connection: '测试连接',
+ connection: '测试',
running: '测试中…',
- hint: '用已保存的密钥和请求头向服务商请求模型列表。它只验证地址和密钥是否可用,不检查模型配置,也不发送对话请求。',
+ hint: '先检查上游连接,再选择模型发送真实生成请求。',
savedOnly: '测试的是已保存的设置。要测试改动后的设置,请先保存。',
passthrough: '这个上游由客户端自带 API Key,网关没有可用于测试的 API Key。',
noKey: '请先填写 API Key。',
+ codingPlan: '这个 Coding Plan 尚未授权通过网关发起模型请求。',
result: {
- unverified: '可达,密钥未验证 · {{status}}',
- ok: '可连通 · {{status}}',
+ connected: '连接可用 · {{status}}',
+ connectionUnverified: '连接可用 · {{status}}',
+ ok: '模型可用 · {{status}}',
failed: '失败 · {{status}}',
unreachable: '无法连接',
error: '无法测试',
},
explain: {
- ok: '服务商接受了密钥,并返回了模型列表。',
+ connected: '服务商接受了密钥,并返回了模型列表。',
+ connectionUnverified:
+ '地址可达,但服务商不提供模型列表,API Key 尚未验证;仍可继续测试模型。',
+ ok: '所选模型已成功生成回复。',
auth: '服务商拒绝了 API Key 或某个请求头。',
- rateLimited: '地址可达,但服务商限制了请求频率,密钥尚未验证。',
- notFound: '地址可达,但服务商不提供模型列表,API Key 尚未验证。',
+ rateLimited: '服务商限制了请求频率,本次模型生成没有完成。',
+ requestRejected:
+ '服务商拒绝了生成请求(HTTP {{status}})。请检查模型名、协议和完整请求地址。',
+ invalidResponse: '服务商返回成功状态,但响应不符合所选模型协议。',
status: '服务商返回了 HTTP {{status}}。',
unreachable:
- '网关在 10 秒内没有连上服务商。请检查 Base URL,以及网关到服务商之间的网络。',
+ '网关没有连上服务商或请求超时。请检查 Base URL,以及网关到服务商之间的网络。',
+ },
+ dialog: {
+ title: '测试“{{name}}”',
+ description:
+ '先检查已保存配置的连接,再发送一次最小的非流式生成请求测试所选模型。',
+ connection: '上游连接',
+ checkingConnection: '正在检查连接…',
+ model: '模型',
+ modelPlaceholder: '输入服务商模型名',
+ configuredModel: '可选择已配置的模型,也可以输入其他模型名。',
+ unconfiguredModel: '这个上游未限制模型,请输入服务商支持的模型名。',
+ response: '上游响应(JSON)',
+ run: '运行测试',
},
},
delete: {
diff --git a/web-studio/src/routes/gateway/-components/upstreams-actions.tsx b/web-studio/src/routes/gateway/-components/upstreams-actions.tsx
index d19b8aa60..5bea10155 100644
--- a/web-studio/src/routes/gateway/-components/upstreams-actions.tsx
+++ b/web-studio/src/routes/gateway/-components/upstreams-actions.tsx
@@ -9,6 +9,8 @@ import {
DialogHeader,
DialogTitle,
} from '#/components/ui/dialog'
+import { Input } from '#/components/ui/input'
+import { Label } from '#/components/ui/label'
import { useMutation } from '@tanstack/react-query'
import { LoaderCircleIcon, PlugZapIcon, Trash2Icon } from 'lucide-react'
import { useTranslation } from 'react-i18next'
@@ -27,7 +29,7 @@ import { toastGatewayError } from '../-lib/error-toast'
import { gatewayErrorMessage } from '../-lib/localize'
import type { Translate } from '../-lib/localize'
import { useGateway } from '../-lib/use-gateway'
-import { toUpstreamInput } from '../-lib/upstream-schema'
+import { servedModels, toUpstreamInput } from '../-lib/upstream-schema'
import { Notice } from './notice'
import { ExplainedButton } from './explained-button'
import { ToneBadge } from './status-badges'
@@ -48,6 +50,51 @@ type TestOutcome =
type TestSummary = { tone: Tone; label: string; explanation: string }
+function TestExplanation({ text }: { text: string }) {
+ const [message, url] = text.split('\n', 2)
+ return (
+ <div className="grid min-w-0 gap-1 text-muted-foreground">
+ <p className="text-sm">{message}</p>
+ {url ? (
+ <p className="break-all font-mono text-xs leading-5">{url}</p>
+ ) : null}
+ </div>
+ )
+}
+
+function readableResponse(
+ response: NonNullable<UpstreamTestResult['response']>,
+): string {
+ let body: unknown
+ try {
+ body = JSON.parse(response.body)
+ } catch {
+ body = response.body
+ }
+ if (
+ typeof body === 'string' &&
+ /<(?:!doctype|html|head|body|title|h[1-6]|p|pre|li|div|br)\b/i.test(body)
+ ) {
+ const document = new DOMParser().parseFromString(body, 'text/html')
+ document.querySelectorAll('script, style, noscript').forEach((node) => {
+ node.remove()
+ })
+ const sections = Array.from(
+ document.body.querySelectorAll('h1, h2, h3, p, li, pre'),
+ )
+ .map((node) => node.textContent)
+ .map((text) => text.replace(/\s+/g, ' ').trim())
+ .filter(Boolean)
+ body = {
+ ...(document.title ? { title: document.title } : {}),
+ body: sections.length
+ ? sections.join('\n')
+ : document.body.textContent.replace(/\s+/g, ' ').trim(),
+ }
+ }
+ return JSON.stringify({ ...response, body }, null, 2)
+}
+
/** Badge text and explanation for a finished connection test. */
export function describeTest(t: Translate, outcome: TestOutcome): TestSummary {
const { result } = outcome
@@ -59,23 +106,49 @@ export function describeTest(t: Translate, outcome: TestOutcome): TestSummary {
}
}
const withUrl = (message: string) =>
- result.url ? `${message} ${result.url}` : message
- if (
- result.reason === 'models_unavailable' ||
- result.status === 404 ||
- result.status === 405 ||
- result.reason === 'rate_limited' ||
- result.status === 429
- ) {
+ result.url ? `${message}\n${result.url}` : message
+ if (result.ok) {
+ if (result.kind === 'connection') {
+ const unverified =
+ result.reason === 'models_unavailable' || result.reason === 'rate_limited'
+ return {
+ tone: unverified ? 'neutral' : 'success',
+ label: t(
+ unverified
+ ? 'upstreams.test.result.connectionUnverified'
+ : 'upstreams.test.result.connected',
+ { status: result.status },
+ ),
+ explanation: withUrl(
+ t(
+ unverified
+ ? 'upstreams.test.explain.connectionUnverified'
+ : 'upstreams.test.explain.connected',
+ ),
+ ),
+ }
+ }
+ return {
+ tone: 'success',
+ label: t('upstreams.test.result.ok', { status: result.status }),
+ explanation: t('upstreams.test.explain.ok'),
+ }
+ }
+ if (result.reason === 'rate_limited' || result.status === 429) {
return {
tone: 'neutral',
- label: t('upstreams.test.result.unverified', { status: result.status }),
+ label: t('upstreams.test.result.failed', { status: result.status }),
+ explanation: withUrl(t('upstreams.test.explain.rateLimited')),
+ }
+ }
+ if (result.reason === 'model_request_rejected') {
+ return {
+ tone: 'danger',
+ label: t('upstreams.test.result.failed', { status: result.status }),
explanation: withUrl(
- t(
- result.reason === 'rate_limited' || result.status === 429
- ? 'upstreams.test.explain.rateLimited'
- : 'upstreams.test.explain.notFound',
- ),
+ t('upstreams.test.explain.requestRejected', {
+ status: result.status,
+ }),
),
}
}
@@ -86,6 +159,13 @@ export function describeTest(t: Translate, outcome: TestOutcome): TestSummary {
explanation: t('errors.reasons.upstream_key_missing'),
}
}
+ if (result.reason === 'coding_plan_blocked') {
+ return {
+ tone: 'danger',
+ label: t('upstreams.credentials.blocked'),
+ explanation: t('upstreams.test.codingPlan'),
+ }
+ }
if (result.status === undefined) {
return {
tone: 'danger',
@@ -94,14 +174,13 @@ export function describeTest(t: Translate, outcome: TestOutcome): TestSummary {
}
}
const { status } = result
- if (result.ok && result.reason !== 'upstream_key_rejected') {
+ if (result.reason === 'invalid_response') {
return {
- tone: 'success',
- label: t('upstreams.test.result.ok', { status }),
- explanation: t('upstreams.test.explain.ok'),
+ tone: 'danger',
+ label: t('upstreams.test.result.failed', { status }),
+ explanation: withUrl(t('upstreams.test.explain.invalidResponse')),
}
}
- // 404 and 405 were answered above as reachable but unverified.
const explanation =
result.reason === 'upstream_key_rejected' ||
status === 401 ||
@@ -133,18 +212,35 @@ export function UpstreamTest({
note,
}: UpstreamTestProps) {
const { t } = useTranslation('gateway')
- const { connection } = useGateway()
- const test = useMutation<UpstreamTestResult, unknown>({
+ const { connection, readOnly } = useGateway()
+ const configuredModels = React.useMemo(
+ () => servedModels(upstream),
+ [upstream],
+ )
+ const [open, setOpen] = React.useState(false)
+ const [model, setModel] = React.useState(configuredModels[0] ?? '')
+ const inputId = React.useId()
+ const listId = `${inputId}-models`
+ const connectionTest = useMutation<UpstreamTestResult, unknown>({
mutationFn: () => testUpstream(connection, upstream.id),
})
+ const modelTest = useMutation<UpstreamTestResult, unknown, string>({
+ mutationFn: (selectedModel) =>
+ testUpstream(connection, upstream.id, selectedModel),
+ })
const blocked = testBlockedReason(upstream)
const explanation = blocked
? t(`upstreams.test.${blocked}`)
: [t('upstreams.test.hint'), note].filter(Boolean).join(' ')
- const summary = test.isSuccess
- ? describeTest(t, { result: test.data })
- : test.isError
- ? describeTest(t, { error: test.error })
+ const connectionSummary = connectionTest.isSuccess
+ ? describeTest(t, { result: connectionTest.data })
+ : connectionTest.isError
+ ? describeTest(t, { error: connectionTest.error })
+ : undefined
+ const modelSummary = modelTest.isSuccess
+ ? describeTest(t, { result: modelTest.data })
+ : modelTest.isError
+ ? describeTest(t, { error: modelTest.error })
: undefined
const label = t(
variant === 'row' ? 'upstreams.test.action' : 'upstreams.test.connection',
@@ -152,19 +248,35 @@ export function UpstreamTest({
// In the list, below `md`, the button shows only its icon and the result
// only its dot; the text stays for screen readers.
const compact = variant === 'row' ? 'max-md:sr-only' : undefined
+ if (readOnly) return null
+ const start = () => {
+ connectionTest.reset()
+ modelTest.reset()
+ setModel(configuredModels[0] ?? '')
+ setOpen(true)
+ connectionTest.mutate()
+ }
+ const submit = (event: React.SubmitEvent<HTMLFormElement>) => {
+ event.preventDefault()
+ const selectedModel = model.trim()
+ if (selectedModel) modelTest.mutate(selectedModel)
+ }
+ const pending = connectionTest.isPending || modelTest.isPending
+ const connected = connectionTest.isSuccess && connectionTest.data.ok
+ const modelBlocked = upstream.coding_plan && !upstream.allow_coding_plan
return (
<div className="flex items-center gap-2">
<span aria-live="polite" className="contents">
- {summary && !test.isPending ? (
+ {modelSummary && !pending ? (
<ToneBadge
- tone={summary.tone}
+ tone={modelSummary.tone}
dot
- title={summary.explanation}
+ title={modelSummary.explanation}
className="max-w-48"
>
<span className={cn('min-w-0 truncate', compact)}>
- {summary.label}
+ {modelSummary.label}
</span>
</ToneBadge>
) : null}
@@ -174,19 +286,139 @@ export function UpstreamTest({
variant={variant === 'row' ? 'ghost' : 'outline'}
size="sm"
className={variant === 'row' ? 'h-8 px-2 text-xs' : undefined}
- disabled={Boolean(blocked) || test.isPending}
+ disabled={Boolean(blocked) || pending}
explanation={explanation}
- onClick={() => test.mutate()}
+ onClick={start}
>
- {test.isPending ? (
- <LoaderCircleIcon className="animate-spin" />
- ) : (
- <PlugZapIcon />
- )}
- <span className={compact}>
- {test.isPending ? t('upstreams.test.running') : label}
- </span>
+ <PlugZapIcon />
+ <span className={compact}>{label}</span>
</ExplainedButton>
+ <Dialog
+ open={open}
+ onOpenChange={(next) => {
+ if (!pending) setOpen(next)
+ }}
+ >
+ <DialogContent
+ showCloseButton={!pending}
+ className="gap-5 sm:max-w-lg"
+ >
+ <DialogHeader className="min-w-0 pr-8">
+ <DialogTitle>
+ {t('upstreams.test.dialog.title', { name: upstream.name })}
+ </DialogTitle>
+ <DialogDescription>
+ {t('upstreams.test.dialog.description')}
+ </DialogDescription>
+ </DialogHeader>
+ <form className="grid min-w-0 gap-5" onSubmit={submit}>
+ <div
+ className="grid min-w-0 gap-2 rounded-md border border-border p-3"
+ aria-live="polite"
+ >
+ <p className="text-xs font-medium">
+ {t('upstreams.test.dialog.connection')}
+ </p>
+ {connectionTest.isPending ? (
+ <p className="flex items-center gap-2 text-sm text-muted-foreground">
+ <LoaderCircleIcon className="size-4 animate-spin" />
+ {t('upstreams.test.dialog.checkingConnection')}
+ </p>
+ ) : connectionSummary ? (
+ <div className="grid min-w-0 gap-1">
+ <ToneBadge tone={connectionSummary.tone} dot>
+ {connectionSummary.label}
+ </ToneBadge>
+ <TestExplanation text={connectionSummary.explanation} />
+ </div>
+ ) : null}
+ </div>
+ <div className="grid min-w-0 gap-2">
+ <Label htmlFor={inputId}>
+ {t('upstreams.test.dialog.model')}
+ </Label>
+ <Input
+ id={inputId}
+ list={configuredModels.length ? listId : undefined}
+ value={model}
+ required
+ autoComplete="off"
+ disabled={!connected || modelTest.isPending}
+ placeholder={t('upstreams.test.dialog.modelPlaceholder')}
+ onChange={(event) => {
+ setModel(event.target.value)
+ modelTest.reset()
+ }}
+ />
+ {configuredModels.length ? (
+ <datalist id={listId}>
+ {configuredModels.map((item) => (
+ <option key={item} value={item} />
+ ))}
+ </datalist>
+ ) : null}
+ <p className="text-xs text-muted-foreground">
+ {modelBlocked
+ ? t('upstreams.test.codingPlan')
+ : t(
+ configuredModels.length
+ ? 'upstreams.test.dialog.configuredModel'
+ : 'upstreams.test.dialog.unconfiguredModel',
+ )}
+ </p>
+ </div>
+ {modelSummary && !modelTest.isPending ? (
+ <div
+ className="grid min-w-0 gap-3 rounded-md border border-border p-3"
+ aria-live="polite"
+ >
+ <ToneBadge tone={modelSummary.tone} dot>
+ {modelSummary.label}
+ </ToneBadge>
+ <TestExplanation text={modelSummary.explanation} />
+ {!modelTest.data?.ok && modelTest.data?.response ? (
+ <div className="grid gap-1">
+ <p className="text-xs font-medium">
+ {t('upstreams.test.dialog.response')}
+ </p>
+ <pre className="max-h-48 overflow-auto whitespace-pre-wrap break-all rounded-md bg-muted px-3 py-2 text-xs">
+ {readableResponse(modelTest.data.response)}
+ </pre>
+ </div>
+ ) : null}
+ </div>
+ ) : null}
+ <DialogFooter>
+ <Button
+ type="button"
+ variant="outline"
+ disabled={pending}
+ onClick={() => setOpen(false)}
+ >
+ {t('actions.cancel')}
+ </Button>
+ <Button
+ type="submit"
+ disabled={
+ !connected ||
+ modelBlocked ||
+ modelTest.isPending ||
+ !model.trim()
+ }
+ >
+ {modelTest.isPending ? (
+ <LoaderCircleIcon className="animate-spin" />
+ ) : (
+ <PlugZapIcon />
+ )}
+ {modelTest.isPending
+ ? t('upstreams.test.running')
+ : t('upstreams.test.dialog.run')}
+ </Button>
+ </DialogFooter>
+ </form>
+ </DialogContent>
+ </Dialog>
</div>
)
}
diff --git a/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx b/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx
index ac9519810..e3a01a6c5 100644
--- a/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx
+++ b/web-studio/src/routes/gateway/-components/upstreams-pages.test.tsx
@@ -328,17 +328,44 @@ describe('UpstreamsPage', () => {
})
it('shows the test result in the row and blocks tests without a stored key', async () => {
- api.testUpstream.mockResolvedValue({ ok: true, status: 200 })
+ api.testUpstream.mockImplementation(
+ (_connection, _id, model: string | undefined) =>
+ Promise.resolve({
+ ok: true,
+ status: 200,
+ kind: model ? 'model' : 'connection',
+ reason: 'passed',
+ }),
+ )
renderAt('/gateway/upstreams')
await screen.findByText('OpenAI')
fireEvent.click(
rowOf('OpenAI').getByRole('button', { name: 'upstreams.test.action' }),
)
+ await waitFor(() =>
+ expect(api.testUpstream).toHaveBeenCalledWith(
+ expect.anything(),
+ 'u1',
+ ),
+ )
+ await screen.findByText('upstreams.test.result.connected')
+ fireEvent.click(
+ screen.getByRole('button', {
+ name: 'upstreams.test.dialog.run',
+ }),
+ )
+ await waitFor(() =>
+ expect(api.testUpstream).toHaveBeenCalledWith(
+ expect.anything(),
+ 'u1',
+ 'gpt-5',
+ ),
+ )
expect(
- await rowOf('OpenAI').findByText('upstreams.test.result.ok'),
- ).toBeTruthy()
- expect(api.testUpstream.mock.lastCall?.[1]).toBe('u1')
+ (await screen.findAllByText('upstreams.test.result.ok')).length,
+ ).toBeGreaterThanOrEqual(1)
+ fireEvent.click(screen.getByRole('button', { name: 'actions.cancel' }))
for (const name of ['Claude', 'Ark']) {
const button = rowOf(name).getByRole('button', {
@@ -348,6 +375,84 @@ describe('UpstreamsPage', () => {
}
})
+ it('keeps model testing disabled when the connection test fails', async () => {
+ api.testUpstream.mockResolvedValue({
+ ok: false,
+ kind: 'connection',
+ reason: 'upstream_unavailable',
+ })
+ renderAt('/gateway/upstreams')
+ await screen.findByText('OpenAI')
+
+ fireEvent.click(
+ rowOf('OpenAI').getByRole('button', { name: 'upstreams.test.action' }),
+ )
+
+ await screen.findByText('upstreams.test.result.unreachable')
+ expect(
+ screen
+ .getByRole('button', { name: 'upstreams.test.dialog.run' })
+ .hasAttribute('disabled'),
+ ).toBe(true)
+ expect(api.testUpstream).toHaveBeenCalledTimes(1)
+ })
+
+ it.each([
+ [
+ '{"error":{"message":"Invalid API key"}}',
+ { error: { message: 'Invalid API key' } },
+ ],
+ [
+ '<html><head><title>Bad Gateway</title><style>body{color:red}</style></head><body><h1>502 Bad Gateway</h1><p>Upstream unavailable</p><script>alert(1)</script></body></html>',
+ {
+ title: 'Bad Gateway',
+ body: '502 Bad Gateway\nUpstream unavailable',
+ },
+ ],
+ ['upstream connection closed', 'upstream connection closed'],
+ ])(
+ 'formats a failed upstream response for display',
+ async (body, readableBody) => {
+ const response = {
+ http_version: 'HTTP/1.1',
+ status: 401,
+ reason: 'Unauthorized',
+ headers: { 'Content-Type': 'application/json' },
+ body,
+ }
+ api.testUpstream
+ .mockResolvedValueOnce({
+ ok: true,
+ status: 200,
+ kind: 'connection',
+ reason: 'passed',
+ })
+ .mockResolvedValueOnce({
+ ok: false,
+ status: 401,
+ kind: 'model',
+ reason: 'upstream_key_rejected',
+ response,
+ })
+ renderAt('/gateway/upstreams')
+ await screen.findByText('OpenAI')
+
+ fireEvent.click(
+ rowOf('OpenAI').getByRole('button', { name: 'upstreams.test.action' }),
+ )
+ await screen.findByText('upstreams.test.result.connected')
+ fireEvent.click(
+ screen.getByRole('button', {
+ name: 'upstreams.test.dialog.run',
+ }),
+ )
+
+ const label = await screen.findByText('upstreams.test.dialog.response')
+ const text = label.parentElement?.querySelector('pre')?.textContent ?? ''
+ expect(JSON.parse(text)).toEqual({ ...response, body: readableBody })
+ },
+ )
+
it('deletes an unused upstream after confirmation', async () => {
api.deleteUpstream.mockResolvedValue({ deleted: true })
renderAt('/gateway/upstreams')
@@ -759,18 +864,47 @@ describe('describeTest', () => {
it.each([
[{ ok: true, status: 200 }, 'success', 'upstreams.test.explain.ok'],
- [{ ok: false, status: 401 }, 'danger', 'upstreams.test.explain.auth'],
- [{ ok: true, status: 404 }, 'neutral', 'upstreams.test.explain.notFound'],
[
- { ok: true, status: 405, reason: 'models_unavailable' },
+ {
+ ok: true,
+ status: 200,
+ kind: 'connection' as const,
+ reason: 'passed',
+ },
+ 'success',
+ 'upstreams.test.explain.connected',
+ ],
+ [
+ {
+ ok: true,
+ status: 404,
+ kind: 'connection' as const,
+ reason: 'models_unavailable',
+ },
'neutral',
- 'upstreams.test.explain.notFound',
+ 'upstreams.test.explain.connectionUnverified',
],
+ [{ ok: false, status: 401 }, 'danger', 'upstreams.test.explain.auth'],
[
- { ok: true, status: 429, reason: 'rate_limited' },
+ { ok: false, status: 429, reason: 'rate_limited' },
'neutral',
'upstreams.test.explain.rateLimited',
],
+ [
+ {
+ ok: false,
+ status: 404,
+ reason: 'model_request_rejected',
+ model: 'missing',
+ },
+ 'danger',
+ 'upstreams.test.explain.requestRejected',
+ ],
+ [
+ { ok: false, status: 200, reason: 'invalid_response' },
+ 'danger',
+ 'upstreams.test.explain.invalidResponse',
+ ],
[
{ ok: false, status: 200, reason: 'upstream_key_rejected' },
'danger',
@@ -822,13 +956,8 @@ describe('upstream access and import', () => {
})
}
- it('shows read-only list details and retains connection testing', async () => {
+ it('shows read-only list details without write actions or model tests', async () => {
readOnly()
- api.testUpstream.mockResolvedValue({
- ok: true,
- status: 404,
- reason: 'models_unavailable',
- })
renderAt('/gateway/upstreams')
await screen.findByText('OpenAI')
expect(screen.queryByText('upstreams.add')).toBeNull()
@@ -839,12 +968,11 @@ describe('upstream access and import', () => {
expect(toggle.hasAttribute('data-disabled')).toBe(true)
fireEvent.click(toggle)
expect(api.saveUpstream).not.toHaveBeenCalled()
- fireEvent.click(
- rowOf('OpenAI').getByRole('button', { name: 'upstreams.test.action' }),
- )
expect(
- await screen.findByText('upstreams.test.result.unverified'),
- ).toBeTruthy()
+ rowOf('OpenAI').queryByRole('button', {
+ name: 'upstreams.test.action',
+ }),
+ ).toBeNull()
})
it('disables read-only editor controls, hides credentials, and tests outside the fieldset', async () => {
@@ -881,12 +1009,12 @@ describe('upstream access and import', () => {
screen.getByLabelText<HTMLInputElement>('upstreams.form.name.label')
.disabled,
).toBe(true)
- const test = screen.getByRole('button', {
- name: 'upstreams.test.connection',
- })
- expect(test.closest('fieldset')).toBeNull()
- fireEvent.click(test)
- await waitFor(() => expect(api.testUpstream).toHaveBeenCalledTimes(1))
+ expect(
+ screen.queryByRole('button', {
+ name: 'upstreams.test.connection',
+ }),
+ ).toBeNull()
+ expect(api.testUpstream).not.toHaveBeenCalled()
})
it('blocks the direct create URL for read-only users', async () => {
diff --git a/web-studio/src/routes/gateway/-lib/api.ts b/web-studio/src/routes/gateway/-lib/api.ts
index 633bdf70c..55df9ffcd 100644
--- a/web-studio/src/routes/gateway/-lib/api.ts
+++ b/web-studio/src/routes/gateway/-lib/api.ts
@@ -308,8 +308,17 @@ export type VlmImportResult = {
export type UpstreamTestResult = {
url?: string
ok: boolean
+ kind?: 'connection' | 'model'
status?: number
reason?: string
+ response?: {
+ http_version: string
+ status: number
+ reason: string
+ headers: Record<string, string | string[]>
+ body: string
+ truncated?: boolean
+ }
}
/** Conversation to resync, taken from a log record. */
@@ -544,9 +553,15 @@ export const saveUpstream = (
export const deleteUpstream = (connection: GatewayConnection, id: string) =>
send<{ deleted: boolean }>(connection, 'DELETE', ['upstreams', id])
-/** Calls the upstream's model list with its stored credentials. */
-export const testUpstream = (connection: GatewayConnection, id: string) =>
- send<UpstreamTestResult>(connection, 'POST', ['upstreams', id, 'test'])
+/** Checks connectivity, or sends one small generation request when a model is given. */
+export const testUpstream = (
+ connection: GatewayConnection,
+ id: string,
+ model?: string,
+) =>
+ send<UpstreamTestResult>(connection, 'POST', ['upstreams', id, 'test'], {
+ body: model ? { model } : {},
+ })
export const listProfiles = (connection: GatewayConnection) =>
send<Profile[]>(connection, 'GET', ['policies'])