Skip to content

A local-first multi-agent app development platform, built on PenguinHarness

feat(models): auto-detect the custom model protocol and vision support (#324)

Yaowei Zhenge19c80f6cd88
diff --git a/changelog/unreleased/2026-08-18-custom-model-protocol-detect.md b/changelog/unreleased/2026-08-18-custom-model-protocol-detect.md
new file mode 100644
index 00000000..8bc2fa95
--- /dev/null
+++ b/changelog/unreleased/2026-08-18-custom-model-protocol-detect.md
@@ -0,0 +1,36 @@
+# Custom models: protocol detection from the base URL
+
+- **Date:** 2026-08-18
+- **Type:** feature
+- **Scope:** `core`, `server`, `web`, `model-catalog`
+- **PR:** [#324](https://github.com/Prism-Shadow/penguin-harness/pull/324)
+
+[中文版](2026-08-18-custom-model-protocol-detect.zh.md)
+
+Custom and user-defined groups are no longer pinned to one protocol: a new custom model starts with no protocol selected, and a **Detect** action in the Web dialog probes which of AgentHub 0.4.2's generic protocol clients a custom base URL serves and stores it as the entry's `client_type`. The probe order is `openai-responses` (OpenAI Responses API) first, then `ant-messages` (Anthropic Messages API), then `openai-chat` (Chat Completions; bare `openai` stays its alias), and the first hit wins.
+
+## Server
+
+- New `POST /api/projects/:id/models/detect` (owner): probes the base URL sequentially in the order above — the paths and auth headers are exactly what the AgentHub clients construct (`POST {base}/responses` and `POST {base}/chat/completions` with `Authorization: Bearer`; `POST {base}/v1/messages` with `x-api-key` + `Authorization: Bearer` + `anthropic-version`) — and stops at the first protocol the endpoint serves, reporting per-probe outcomes for debugging.
+- Probes are minimal invalid requests (`{}` bodies, 5s timeout each, response bodies read up to 64 KiB so a flooding endpoint is abandoned rather than buffered): zero tokens billed, no model id needed, and keyless probing works — a 401/403 in the protocol's error shape proves the route. Classification separates route-exists (structured API errors, tolerant of OpenAI / Anthropic / vLLM / FastAPI dialects) from route-missing (404/405 by status alone), gateway junk (HTML / non-JSON / shapeless bodies, catch-all 200s), and 5xx (proves nothing about the path). Credential resolution has three layers: the key sent in the request, else the stored key named by the optional paired reference, else the environment variable belonging to the protocol **that probe** speaks — `ANTHROPIC_API_KEY` for `ant-messages`, `OPENAI_API_KEY` for the two OpenAI protocols, resolved through the same `resolveModelEnv` mapping the saved model reads. Resolution is per probe because the protocol is precisely what is being determined. Env values are read server-side only and never reach the browser, the response, or the logs.
+
+## Web App
+
+- **Detect** sits at the top-right of the base URL field, next to its label — the same placement idiom as the API key field's "get API key" link — and is always clickable: no API key is needed, because the server resolves the probe credential itself (below). Detection is now only ever user-initiated (this button) or the save-time backstop below: the earlier implicit run on base-URL blur is gone, because with dialogs as the feedback channel a passive trigger would either interrupt the user mid-form or fail with no explanation.
+- **Saving detects first.** Confirming the dialog while the protocol is still unset runs detection and then continues the save with the result, with the confirm button locked and reading "Detecting…" for the round-trip. A probe that finds nothing aborts the save and leaves the dialog open, with the toast carrying the reason, so the protocol can be picked by hand or the URL corrected — never a silent save. This is load-bearing: AgentHub resolves an unmatched client type by throwing rather than defaulting, so an entry persisted without a protocol is a model that cannot start. `rowToEntry` keeps the same guarantee for the paths that do not probe (set-default, set-vision-proxy, remove), and preset / vendor entries still persist an empty value so AgentHub keeps inferring from the model id.
+- **Both outcomes are toasts** — the same non-blocking channel the connectivity test in this dialog already uses (the Toaster portals to `document.body` at `z-[100]` against the Modal's `z-50`, so a toast raised from inside the open dialog renders above it). A hit names the protocol that was applied; every failure — unreachable, timed out, gateway junk, none of the three served, an unusable URL — shows the SAME short message naming the two things the user can act on, the API key and the base URL. Nothing enumerates protocol names or describes what the endpoint did: "the endpoint responded but…" read as success. The per-probe outcomes stay in the detect endpoint's response for debugging. The success toast belongs to the manual Detect button only — on the save path a hit proceeds silently, since the save it was serving carries straight on. No verdict is rendered in the form at all: the earlier line under the base URL field is gone, so the idle and post-detection layouts are identical.
+- **No protocol is preselected in the UI, not just in the data.** The suffix shows a "Select protocol" placeholder instead of a path, and no menu row is checked, until a pick or a detection lands (`protocolSelectorValue` now returns null for an unset protocol rather than mapping it to `openai-chat`, which had been rendering `/chat/completions` as a choice the user never made). The reserved in-field padding measures display width, so the localized placeholder is not under-reserved.
+- The path suffix the dialog already showed inside the right edge of the base URL field (`/responses`, `/v1/messages`, `/chat/completions`) became the manual protocol picker: the passive grey text turned into a borderless in-field trigger listing the three protocols with the path each appends and a checkmark on the current one, so protocol selection took no new form row. Selection needs no API key, which keeps a keyless endpoint configurable by hand.
+- The suffix tracks the choice live and reflects a run in progress (spinner, and amber when nothing matched or the probe failed) at unchanged width, so the input's reserved padding never shifts. The verdict ("Detected OpenAI Responses; applied", or why nothing matched) appears in the field's own message line only after a run.
+- Legacy entries keep their stored value: `openai` displays as Chat Completions without being rewritten until the user picks or a detection applies; vendor-pinned client types outside the generic family keep the read-only note. Moving a model to Custom now keeps a generic protocol client type instead of forcing `openai-chat` the way [the earlier grouping fix](2026-08-14-model-group-protocol.md) did.
+
+- **Custom groups infer nothing from the model id.** Typing `claude-sonnet-5` into a custom group no longer implies the Anthropic client or its `ANTHROPIC_*` key in the dialog's API-key hint: custom and user-defined groups always fall back to the compatible client. Vendor groups keep routing by id, which is how they actually work.
+- **A failed save-time detection no longer blocks the save.** It falls back to `openai-chat` and saves, with a toast saying that is what happened — detection is an accuracy improvement, not a gate, and refusing to save stranded anyone whose endpoint simply cannot be probed. Nothing unstartable is written either way, since the fallback is a real client.
+- **Vision detection**, beside the vision switch: sends one 1x1 PNG and a one-word prompt, and turns the switch on when the model answers. A model that answers specifically that it will not take an image turns the switch off (a definitive answer, not an error); a probe that fails on auth or the network leaves the setting untouched and shows the same "check the API key and the base URL". Always clickable, single-flight, toast-only — the protocol control's idiom. New custom models start with vision off. **Unlike the protocol probes this is a real, billed completion** (an image request cannot be made free), so it runs only on an explicit press, never on save. Server-side `POST /api/projects/:p/models/detect-vision`, owner only, with the connectivity test's credential chain.
+ `ModelEntry.vision` stays a plain boolean: an absent value already means "supported" throughout core, the CLI and the web app, so making it tri-state would have silently downgraded every existing un-annotated entry.
+
+## Core
+
+- `resolveModelEnv` routes `ant-messages` to the `ANTHROPIC_*` env pair (`openai-responses` / `openai-chat` already resolved to `OPENAI_*`), so the dialog's env-fallback hint follows the detected protocol per PRN-021.
+
+Docs: the models page (en/zh) gained a "Protocol detection for custom models" section; the configuration and server-api references picked up the new client types and the new endpoint.
diff --git a/changelog/unreleased/2026-08-18-custom-model-protocol-detect.zh.md b/changelog/unreleased/2026-08-18-custom-model-protocol-detect.zh.md
new file mode 100644
index 00000000..5bb3f0fc
--- /dev/null
+++ b/changelog/unreleased/2026-08-18-custom-model-protocol-detect.zh.md
@@ -0,0 +1,36 @@
+# 自定义模型:按 base URL 检测协议
+
+- **Date:** 2026-08-18
+- **Type:** feature
+- **Scope:** `core`, `server`, `web`, `model-catalog`
+- **PR:** [#324](https://github.com/Prism-Shadow/penguin-harness/pull/324)
+
+[English](2026-08-18-custom-model-protocol-detect.md)
+
+Custom 与自建分组不再被钉死在单一协议上:新建自定义模型默认不选择协议,Web 对话框新增「检测协议」操作,探测某个自定义 base URL 实际提供的是 AgentHub 0.4.2 中的哪一个通用协议客户端,并把结果写入该条目的 `client_type`。探测顺序固定为先 `openai-responses`(OpenAI Responses API)、再 `ant-messages`(Anthropic Messages API)、最后 `openai-chat`(Chat Completions;裸写的 `openai` 仍是它的别名),命中第一个即止。
+
+## Server
+
+- 新增 `POST /api/projects/:id/models/detect`(仅 owner):按上述顺序依次探测 base URL——所用路径与鉴权头,与 AgentHub 各客户端实际构造的完全一致(`POST {base}/responses` 和 `POST {base}/chat/completions` 带 `Authorization: Bearer`;`POST {base}/v1/messages` 带 `x-api-key` + `Authorization: Bearer` + `anthropic-version`)——在端点真正提供的第一个协议处停下,并逐条返回每次探测的结果以便排查。
+- 探测发出的是最小化的非法请求(`{}` 请求体,每次 5s 超时,响应体最多读取 64 KiB,遇到疯狂输出的端点直接放弃而不是全部缓冲下来):不产生任何 Token 计费,不需要 model id,没有 key 也能探——协议自身错误形态的 401/403 就足以证明该路由存在。分类区分四种情形:路由存在(结构化的 API 错误,兼容 OpenAI / Anthropic / vLLM / FastAPI 各家方言)、路由不存在(仅凭 404/405 状态码判定)、网关杂讯(HTML / 非 JSON / 无结构的响应体,以及一律返回 200 的兜底)、以及 5xx(对路径本身什么都证明不了)。与连通性测试一样,可选地附上 `(provider, modelId)` 这对模型引用,就能让已存的密钥为探测提供鉴权;密钥不会出现在 URL、结果或日志中。凭据按三层解析:请求里带的 key,其次配对引用指向的已存密钥,最后是**当前这个探测**所用协议对应的环境变量——`ant-messages` 读 `ANTHROPIC_API_KEY`,两个 OpenAI 协议读 `OPENAI_API_KEY`,走的是保存后模型读取时的同一套 `resolveModelEnv` 映射。之所以逐个探测分别解析,正是因为协议本身还没确定。环境变量只在服务端读取,不会回传浏览器、响应或日志。
+
+## Web App
+
+- 「检测协议」按钮位于 base URL 输入框右上角、与其标签同一行——与 API Key 字段「获取 API key」链接的摆放方式一致——并且始终可点击:不需要填 API Key,探测凭据由服务端自行解析(见下)。检测从此只有两个入口:用户主动点这个按钮,以及下面保存时的兜底。此前 base URL 失焦时的隐式触发已移除——反馈改走 toast 之后,被动触发要么在用户填表途中冒出提示,要么失败了也说不清原因。
+- **保存前先检测。** 确认对话框时若协议仍为空,会先跑一次检测,再带着结果继续保存;这段往返期间确认按钮锁定并显示「检测中…」。若探测一无所获,则中止保存,对话框保持打开、由 toast 说明原因,可手动选协议或修正 URL,绝不静默保存。这一步是关键:AgentHub 对无法匹配的 client type 是抛错而非回退默认值,协议为空的条目就是一个起不来的模型。对于不做探测的路径(设为默认 / 设为视觉代理 / 删除),`rowToEntry` 同样保证不会写入空协议;预置与厂商分组条目则仍然保持为空,交给 AgentHub 按 model id 推断。
+- **成功与失败都用 toast 提示**——与本对话框里连通性测试用的是同一套非阻塞通道(Toaster 以 `z-[100]` 挂到 `document.body`,高于 Modal 的 `z-50`,因此在对话框内触发的 toast 会显示在它上方)。成功时说明应用了哪种协议;所有失败情形——连不上、超时、网关杂讯、三条路径都没提供、URL 不可用——统一显示同一条简短提示,只讲用户能动手改的两件事:API Key 与 base URL。不再罗列协议名,也不再描述端点做了什么:「端点有响应,但……」会被读成成功。逐个协议的探测结果仍保留在检测接口的响应里,便于排查。成功 toast 只属于手动点击「检测协议」;保存路径上检测成功则静默继续,因为它服务的那次保存会直接进行下去。表单里完全不渲染检测结论:此前 base URL 字段下方那一行已删除,空闲状态与检测之后的布局完全一致。
+- **界面上确实没有预选协议,而不只是数据层为空。** 在手动选择或检测生效之前,后缀显示「选择协议」占位文案而非某条路径,菜单中也没有任何一项被勾选(`protocolSelectorValue` 现在对未设置的协议返回 null,而不再映射为 `openai-chat`——此前那样会把 `/chat/completions` 渲染成用户从未做过的选择)。输入框预留的内边距按显示宽度计算,本地化占位文案不会被少留一半。
+- 对话框此前就在 base URL 输入框右端内嵌显示协议路径(`/responses`、`/v1/messages`、`/chat/completions`),这次它成了手动协议选择器:原先被动的灰色文字变成输入框内的无边框触发器,列出三种协议及各自追加的路径,并在当前项上打勾,协议选择因此没有占用新的表单行。手动选择不需要 API Key,无需鉴权的端点仍可完全手工配好。
+- 后缀实时跟随当前选择,也反映正在进行的检测(转圈;三种协议都没匹配或探测失败时转为琥珀色),且宽度保持不变,输入框为它预留的内边距不会跟着位移。检测结论(「已检测到 OpenAI Responses,已应用」,或未能匹配的原因)只在真正跑过一次之后,才显示在该字段自己的提示行里。
+- 此前创建的条目保留已存的值:`openai` 显示为 Chat Completions,在用户手动选择或某次检测生效之前不会被改写;通用家族之外、锁定到特定厂商的 client_type 仍保留只读提示。把模型移动到 Custom 分组现在会保留通用协议客户端类型,而不再像[此前那次分组修复](2026-08-14-model-group-protocol.zh.md)那样强制写成 `openai-chat`。
+
+- **自定义分组不再从 model id 推断任何东西。** 在自定义分组里填 `claude-sonnet-5`,对话框的 API Key 提示不再因此指向 Anthropic 客户端与 `ANTHROPIC_*`:Custom 与自建分组一律回退到兼容客户端。厂商分组仍按 id 路由——它们本来就是这么工作的。
+- **保存时检测失败不再阻止保存。** 改为回退到 `openai-chat` 继续保存,并以 toast 说明发生了什么——检测是提升准确度的手段,不是关卡;一味拒绝保存会把那些端点本就无法探测的用户卡死。两种情况下写入的都是可用客户端,不会产生起不来的条目。
+- **视觉能力检测**,位于视觉开关旁边:发送一张 1x1 的 PNG 和一句一个词的提示,模型能作答就打开开关。若模型明确回答不接受图片,则关闭开关(这是确定的答案,不是错误);因鉴权或网络失败的探测则不改动该设置,只显示同一句「请检查 API Key 与 base URL」。始终可点击、单次并发、仅用 toast 反馈——与协议控件同一套做法。新建自定义模型默认关闭视觉。**与协议探测不同,这是一次真实计费的补全**(图片请求无法做成零成本),因此只在主动点击时运行,保存时不会触发。服务端为 `POST /api/projects/:p/models/detect-vision`(仅 owner),凭据链与连通性测试一致。
+ `ModelEntry.vision` 保持普通布尔值:在 core、CLI 与 Web 中,该字段缺省本就表示「支持」,改成三态会让所有既有的未标注条目被悄悄降级。
+
+## Core
+
+- `resolveModelEnv` 把 `ant-messages` 路由到 `ANTHROPIC_*` 这对环境变量(`openai-responses` / `openai-chat` 本就解析到 `OPENAI_*`),于是对话框的环境变量兜底提示会跟随检测到的协议,符合 PRN-021。
+
+文档:models 页(en/zh)新增「自定义模型的协议检测」一节;configuration 与 server-api 参考补上了新的 client type 与新端点。
diff --git a/packages/core/src/state/model-catalog.ts b/packages/core/src/state/model-catalog.ts
index b65582e0..11613597 100644
--- a/packages/core/src/state/model-catalog.ts
+++ b/packages/core/src/state/model-catalog.ts
@@ -1298,6 +1298,7 @@ export function resolveModelEnv(modelId: string, clientType?: string): ModelEnvI
}
if (t.includes("deepseek-v4")) return env("DEEPSEEK");
// agenthub 0.4.2's generic Anthropic Messages protocol client reads the ANTHROPIC_* pair.
+ // Order mirrors AutoLLMClient: ant-messages before the openai substring match.
if (t.includes("ant-messages")) return env("ANTHROPIC");
// The generic OpenAI-protocol clients — openai-chat (canonical since agenthub 0.4.2, with
// bare "openai" as a deprecated alias), openai-responses and openai-embedding — all read
diff --git a/packages/core/src/state/project-config.ts b/packages/core/src/state/project-config.ts
index d1284bd7..50c10d1e 100644
--- a/packages/core/src/state/project-config.ts
+++ b/packages/core/src/state/project-config.ts
@@ -69,11 +69,13 @@ export interface ModelEntry {
model_id: string;
context_window?: number;
/**
- * AgentHub client protocol (`openai-chat` / `claude-4-8` / `deepseek-v4` / …); defaults to
- * being inferred by AgentHub from the request id (`model_id`). A third-party model speaking
- * the OpenAI Chat Completions protocol should set this to `openai-chat` (the bare `openai`
- * spelling from configs saved before AgentHub 0.4.2's rename is normalized to it on read —
- * see canonicalClientType).
+ * AgentHub client protocol (`openai-responses` / `ant-messages` / `openai-chat` /
+ * `claude-4-8` / `deepseek-v4` / …); defaults to being inferred by AgentHub from the
+ * request id (`model_id`). Third-party endpoints use one of the generic protocol clients:
+ * `openai-responses` (OpenAI Responses API), `ant-messages` (Anthropic Messages API), or
+ * `openai-chat` (OpenAI Chat Completions; the bare `openai` spelling from configs saved
+ * before AgentHub 0.4.2's rename is normalized to it on read — see canonicalClientType).
+ * The Web models page can detect which one a custom base URL serves.
*/
client_type?: string;
/**
diff --git a/packages/core/test/model-catalog.test.ts b/packages/core/test/model-catalog.test.ts
index ac4eaf4b..048ffb3d 100644
--- a/packages/core/test/model-catalog.test.ts
+++ b/packages/core/test/model-catalog.test.ts
@@ -641,6 +641,14 @@ describe("resolveModelEnv (PRN-021: env fallback resolved by AgentHub routing ru
expect(resolveModelEnv("custom-model", "minimax-m3")).toBeUndefined();
});
+ it("generic protocol client types (agenthub 0.4.2) resolve regardless of the model id: ant-messages reads ANTHROPIC_*, openai-responses / openai-chat read OPENAI_*", () => {
+ expect(resolveModelEnv("any-model", "ant-messages")?.envKey).toBe("ANTHROPIC_API_KEY");
+ expect(resolveModelEnv("any-model", "ant-messages")?.envBaseUrlKey).toBe("ANTHROPIC_BASE_URL");
+ expect(resolveModelEnv("any-model", "openai-responses")?.envKey).toBe("OPENAI_API_KEY");
+ expect(resolveModelEnv("any-model", "openai-chat")?.envKey).toBe("OPENAI_API_KEY");
+ expect(resolveModelEnv("any-model", "openai-chat")?.envBaseUrlKey).toBe("OPENAI_BASE_URL");
+ });
+
it("unroutable ids return undefined (AgentHub would reject; needs explicit client_type or an OpenAI-protocol grouping)", () => {
expect(resolveModelEnv("totally-unknown-model")).toBeUndefined();
expect(resolveModelEnv("xiaomi/mimo-v2.5")).toBeUndefined();
diff --git a/packages/docs/content/configuration.en.md b/packages/docs/content/configuration.en.md
index 955b18aa..6947b0d7 100644
--- a/packages/docs/content/configuration.en.md
+++ b/packages/docs/content/configuration.en.md
@@ -59,7 +59,7 @@ Model entry (`[[models]]`) fields:
| `provider` | Provider group; together with `model_id` forms the entry's unique key |
| `model_id` | Upstream request id, sent to AgentHub unchanged |
| `context_window` | Context window size |
-| `client_type` | AgentHub client protocol; inferred from `model_id` by default — third-party OpenAI-compatible models should set `openai-chat` (the pre-0.4.2 `openai` spelling is a deprecated alias, normalized on read) |
+| `client_type` | AgentHub client protocol; inferred from `model_id` by default — custom endpoints use a generic protocol client: `openai-responses`, `ant-messages`, or `openai-chat` (the Web dialog can detect which one a base URL serves; the pre-0.4.2 `openai` spelling is a deprecated alias of `openai-chat`, normalized on read) |
| `display_name` | Display name; persisted only when it differs from the built-in catalog |
| `vision` | Whether image input is supported; defaults to supported |
| `max_tokens` | Per-model max output tokens; overrides the Agent's `model.max_tokens` when set, omitted = inherit it |
diff --git a/packages/docs/content/configuration.zh.md b/packages/docs/content/configuration.zh.md
index 26d9eb80..62a0b915 100644
--- a/packages/docs/content/configuration.zh.md
+++ b/packages/docs/content/configuration.zh.md
@@ -59,7 +59,7 @@ openrouter、fireworks、siliconflow、qwen-token-plan、qwen-pay-as-you-go 与
| `provider` | Provider 分组;与 `model_id` 共同构成条目唯一键 |
| `model_id` | 上游请求 id,原样发送给 AgentHub |
| `context_window` | 上下文窗口大小 |
-| `client_type` | AgentHub 客户端协议;缺省由 `model_id` 推断,OpenAI 兼容的第三方模型应设为 `openai-chat`(0.4.2 之前的旧写法 `openai` 为已废弃别名,读取时归一化) |
+| `client_type` | AgentHub 客户端协议;缺省由 `model_id` 推断。自定义端点使用通用协议客户端 `openai-responses` / `ant-messages` / `openai-chat`(Web 对话框可按 base URL 检测其中哪一种;0.4.2 之前的旧写法 `openai` 为 `openai-chat` 的已废弃别名,读取时归一化) |
| `display_name` | 展示名;仅在与内置目录不同时持久化 |
| `vision` | 是否支持图片输入;缺省视为支持 |
| `max_tokens` | 单模型最大输出 Token;设置后覆盖 Agent 的 `model.max_tokens`,缺省则继承 |
diff --git a/packages/docs/content/models.en.md b/packages/docs/content/models.en.md
index f09c895f..ba8f5396 100644
--- a/packages/docs/content/models.en.md
+++ b/packages/docs/content/models.en.md
@@ -23,7 +23,7 @@ Each Project's available models are recorded in the hidden `.project_config.toml
| `model_id` | Upstream request id |
| `context_window` | Context window (tokens). Load-bearing, not just display: each request's effective output cap and the compaction threshold are derived from it, so requests never ask for more output than the window still fits. Unset (or implausibly small, under 4096): the output clamp turns off and compaction derives from an assumed 128000 — set the real value for models with smaller windows |
| `max_tokens` | Optional per-model output cap (max output tokens per request). When set it overrides the Agent's `model.max_tokens`; unset inherits it. The cap is a ceiling, not the literal wire value: each request sends `min(max_tokens, context_window − estimated input − safety margin)`, so small-window models work without hand-tuning it. Omitting the field on a Web full-table save clears it |
-| `client_type` | Protocol hint (`openai-chat` for Chat Completions, `openai-responses` for the Responses API, `ant-messages` for Anthropic Messages, …); inferred by AgentHub from the model id when omitted. The pre-0.4.2 spelling `openai` is a deprecated alias and is normalized to `openai-chat` when the config is read |
+| `client_type` | Protocol hint (`openai-chat` for Chat Completions, `openai-responses` for the Responses API, `ant-messages` for Anthropic Messages, …); inferred by AgentHub from the model id when omitted. Custom endpoints use one of those three generic protocol clients, and the Web dialog can detect which one a base URL serves. The pre-0.4.2 spelling `openai` is a deprecated alias and is normalized to `openai-chat` when the config is read |
| `display_name` | Display name |
| `vision` | Whether image input is supported, default true |
| `pricing` | Three price buckets (unit `usd_per_mtok`, USD per million tokens): `cache_read` / `cache_write` / `output` |
@@ -80,11 +80,31 @@ Some models in the preset catalog: deepseek-v4-pro / deepseek-v4-flash, MiniMax-
## Local / self-hosted OpenAI-compatible endpoints (e.g. vLLM)
-A local inference server is just a `custom` entry: `client_type = "openai-chat"`, `base_url` pointing at the server (e.g. `http://127.0.0.1:8000/v1`), and the served model name as `model_id`. Two settings make it run smoothly:
+A local inference server is just a `custom` entry: `client_type = "openai-chat"`, `base_url` pointing at the server (e.g. `http://127.0.0.1:8000/v1`), and the served model name as `model_id` (`openai-chat` is also what the protocol detection below settles on for such servers, and what the base URL field's suffix menu selects by hand). Two settings make it run smoothly:
- **Enable tool calling server-side.** For vLLM, start the server with `--enable-auto-tool-choice` and the `--tool-call-parser` matching your model (e.g. `hermes` for Qwen, `llama3_json` for Llama 3.x); without them tool calls arrive as plain text and the agent loop cannot execute anything.
- **Set the entry's `context_window` to the server's real window** — for vLLM, the `--max-model-len` value (e.g. `32768`). The per-request output cap and the compaction threshold both derive from this window automatically: requests clamp `max_tokens` to what the window still fits, and compaction fires before the window overflows, so no hand-tuned `max_tokens` is needed. Left unset, the per-request output clamp is off and compaction assumes a 128000 window, so a smaller real window will reject requests.
+## Protocol detection for custom models
+
+Custom and user-defined groups speak AgentHub's generic protocol clients, and the Web dialog detects which one a base URL serves. A new custom model starts with **no protocol selected**: the suffix at the right edge of the base URL field reads "Select protocol" rather than a path, and no row in its menu is checked. **Detect** sits at the top-right of that field, next to its label, and is always available — no API key is needed to press it. Pressing it has the server probe the URL with three cheap requests in fixed order — `openai-responses` (`POST {base}/responses`, OpenAI Responses API) first, then `ant-messages` (`POST {base}/v1/messages`, Anthropic Messages API), then `openai-chat` (`POST {base}/chat/completions`) — and the first protocol the endpoint actually serves is applied to the entry's `client_type`, confirmed by a toast naming what was found. Nothing about a result is written into the form itself — where the protocol ended up is visible in the suffix, which is what actually holds it.
+
+Saving is the backstop: confirm the dialog while the protocol is still unset and detection runs first, then the save continues with what it found (the button reads "Detecting…" for the round-trip). A hit there is not announced — the save you asked for simply proceeds. If the probe finds nothing the model is **not** saved: the failure pops up and the dialog stays open, so you can pick a protocol by hand or correct the URL. This matters because AgentHub resolves an unmatched client type by raising, not by defaulting: an entry persisted with no protocol would be a model that cannot start.
+
+Probes are minimal invalid requests (`{}` bodies): they cost no tokens and need no valid model id — an error in the protocol's own shape already proves the route exists, while a `404`/`405` means the path isn't served and HTML or gateway junk counts for nothing. The probed URLs and auth headers are exactly what the AgentHub client would use after saving (`Authorization: Bearer` for the OpenAI protocols; `x-api-key` plus `Authorization: Bearer` and `anthropic-version` for `ant-messages`), so a detected protocol is one that will really work.
+
+The probe credential is resolved server-side in three layers: the API key typed in the dialog, else the key already stored for this entry, else the environment variable belonging to the protocol **that probe** speaks — `ANTHROPIC_API_KEY` for `ant-messages`, `OPENAI_API_KEY` for the two OpenAI protocols, the same variables the saved model would read. Resolution happens per probe precisely because the protocol is the thing being determined. None of these values are ever sent to the browser or echoed in the response. Detection still works with no credential anywhere — a protocol-shaped `401` identifies the route perfectly well — but an authenticated probe is far likelier to draw that protocol-shaped answer than the generic `401` or gateway HTML an anonymous request often gets.
+
+The manual override is the protocol path shown inside the right edge of the base URL field (`/responses`, `/v1/messages`, `/chat/completions`) — the path the client appends to your URL, which is one-to-one with the protocol. Clicking it opens a menu of the three protocols with the path each one appends, and picking one wins over detection — so an endpoint you already know the protocol of never has to be probed at all. Whenever detection fails — unreachable, timed out, non-API answers, or none of the three paths served — the suffix turns amber and one toast says the same thing: the protocol could not be detected, check the API key and the base URL. The per-protocol probe outcomes are still reported by the endpoint for debugging. Entries from before this feature keep `client_type = "openai"` — still an alias of `openai-chat` — and are only rewritten when you pick a protocol or a detection run applies. Detection is exposed as `POST /api/projects/:id/models/detect` (owner only, see [Server API](/server-api)).
+Nothing is inferred from the model id in these groups. Typing `claude-sonnet-5` into a custom group does not imply the Anthropic client or its `ANTHROPIC_*` key — custom and user-defined groups always fall back to the compatible client (`openai-chat`), and that is what the API-key hint reflects. Detection is an accuracy improvement on top, never a gate: if it comes back empty the model still saves, on `openai-chat`, with a toast saying so. Vendor and gateway groups are unaffected — their ids are catalog-known, so they keep routing by id or by their preset pin.
+
+### Vision detection
+
+"Supports vision" can be left off and asked of the model instead: **Detect** next to the switch sends one 1x1 PNG with a one-word prompt and turns the switch on if the model answers. A model that answers specifically that it will not take an image turns the switch off — that is a real answer, not an error — while a probe that fails on auth or the network leaves the switch exactly as you had it and shows the usual "check the API key and the base URL".
+
+Unlike protocol detection, **this probe is a real, billed completion**: an image request cannot be shaped to cost nothing the way the protocol probes are. It therefore runs only when you press the control, never on its own and never on save. The credential chain is the connectivity test's — the key typed in the dialog, else the stored one, else the protocol's environment variable, all resolved server-side. New custom models start with vision off; models added from a vendor or gateway group keep their catalog-known capability.
+
+
## Thinking levels
For MiniMax M3, `none` maps directly to `reasoning.effort = "none"`.
diff --git a/packages/docs/content/models.zh.md b/packages/docs/content/models.zh.md
index d1bc3957..018f84f2 100644
--- a/packages/docs/content/models.zh.md
+++ b/packages/docs/content/models.zh.md
@@ -23,7 +23,7 @@ description: 经 AgentHub 单一网关接入模型,以 (provider, model_id)
| `model_id` | 上游请求 id |
| `context_window` | 上下文窗口(Token 数)。不只用于展示:每次请求的实际输出上限与压缩阈值都由它推导,请求不会索要超出窗口剩余空间的输出。缺省(或小于 4096 的非常规值)时输出收敛关闭、压缩按 128000 假定推导——窗口更小的模型务必填真实值 |
| `max_tokens` | 可选的按模型输出上限(单次请求最大输出 Token 数)。设置后覆盖 Agent 的 `model.max_tokens`,缺省沿用。该值是天花板而非逐字上线值:每次请求实际发送 `min(max_tokens, context_window − 估算输入 − 安全余量)`,小窗口模型无需手工调低。Web 整表保存时省略该字段即清除 |
-| `client_type` | 协议提示(`openai-chat` 对应 Chat Completions、`openai-responses` 对应 Responses API、`ant-messages` 对应 Anthropic Messages 等);缺省由 AgentHub 按 model id 推断。0.4.2 之前的旧写法 `openai` 为已废弃别名,读取配置时归一化为 `openai-chat` |
+| `client_type` | 协议提示(`openai-chat` 对应 Chat Completions、`openai-responses` 对应 Responses API、`ant-messages` 对应 Anthropic Messages 等);缺省由 AgentHub 按 model id 推断。自定义端点使用这三种通用协议客户端之一,Web 对话框可按 base URL 检测其中哪一种。0.4.2 之前的旧写法 `openai` 为已废弃别名,读取配置时归一化为 `openai-chat` |
| `display_name` | 显示名 |
| `vision` | 是否支持图像输入,默认 true |
| `pricing` | 三档价格(单位 `usd_per_mtok`,USD 每百万 Token):`cache_read` / `cache_write` / `output` |
@@ -80,11 +80,31 @@ api_key = "sk-..."
## 本地 / 自建 OpenAI 兼容端点(如 vLLM)
-本地推理服务就是一条 `custom` 条目:`client_type = "openai-chat"`、`base_url` 指向服务地址(如 `http://127.0.0.1:8000/v1`)、`model_id` 填服务端的模型名。两处设置决定运行是否顺畅:
+本地推理服务就是一条 `custom` 条目:`client_type = "openai-chat"`、`base_url` 指向服务地址(如 `http://127.0.0.1:8000/v1`)、`model_id` 填服务端的模型名(下文的协议检测对这类服务同样会落到该协议,也可直接用 base URL 输入框右端的后缀菜单手动选它)。两处设置决定运行是否顺畅:
- **服务端要开启工具调用。** vLLM 需以 `--enable-auto-tool-choice` 启动,并按模型选择对应的 `--tool-call-parser`(如 Qwen 用 `hermes`、Llama 3.x 用 `llama3_json`);不开启时工具调用会以纯文本返回,Agent 循环无法执行任何工具。
- **条目的 `context_window` 填服务端的真实窗口**——vLLM 即 `--max-model-len` 的值(如 `32768`)。每次请求的输出上限与压缩阈值都会由该窗口自动推导:请求把 `max_tokens` 收敛到窗口剩余空间以内,压缩也会在撞上窗口硬限制之前触发,无需手工调低 `max_tokens`。不填时不做逐请求输出收敛、压缩按 128000 假定,真实窗口更小会导致请求被拒。
+## 自定义模型的协议检测
+
+Custom 与自建分组走 AgentHub 的通用协议客户端,Web 对话框会检测 base URL 实际提供的是哪一种。新建自定义模型时**默认不选择任何协议**:base URL 输入框右端的后缀显示「选择协议」而不是某条路径,其菜单中也没有任何一项被勾选。「检测协议」按钮位于该输入框右上角、与标签同一行,始终可点击——不需要先填 API Key。点击它,服务端按固定顺序向该 URL 发三个轻量探测请求——先 `openai-responses`(`POST {base}/responses`,OpenAI Responses API),再 `ant-messages`(`POST {base}/v1/messages`,Anthropic Messages API),最后 `openai-chat`(`POST {base}/chat/completions`)——第一个真正被端点提供的协议写入条目的 `client_type`,并以 toast 提示检测到的是哪一种。表单里不会留下任何结果文字——协议最终落在哪,看后缀即可,那才是真正承载它的地方。
+
+保存是兜底环节:若确认对话框时协议仍为空,会先自动检测,再带着检测结果继续保存(这段往返期间按钮显示「检测中…」)。此处检测成功不额外提示——你要的保存直接继续。若这次探测什么都没找到,模型**不会**被保存:toast 说明原因,对话框保持打开,你可以手动选协议或修正 URL。这一步是必要的:AgentHub 遇到无法匹配的 client type 会直接抛错而不是回退默认值,协议为空的条目就是一个根本起不来的模型。
+
+探测请求是刻意构造的最小非法请求(`{}` 请求体):不消耗 Token、不需要有效的模型 id——按协议自身形态返回的错误即证明路由存在;`404`/`405` 则说明该路径未提供,HTML 或网关杂讯一概不算数。探测的 URL 与鉴权头和保存后 AgentHub 客户端实际使用的完全一致(OpenAI 系协议用 `Authorization: Bearer`;`ant-messages` 同时带 `x-api-key`、`Authorization: Bearer` 与 `anthropic-version`),因此检测出的协议就是真正能跑通的协议。
+
+探测所用凭据在服务端按三层依次解析:对话框里填的 API Key,其次该条目已保存的密钥,最后是**当前这个探测**所用协议对应的环境变量——`ant-messages` 读 `ANTHROPIC_API_KEY`,两个 OpenAI 协议读 `OPENAI_API_KEY`,与保存后模型实际读取的是同一批变量。之所以逐个探测分别解析,正是因为协议本身还没确定。这些值都不会回传浏览器,也不会出现在响应里。完全没有凭据时检测同样可用——协议形态的 `401` 足以认出路由——但带上鉴权的探测,远比匿名请求更容易拿到那种协议形态的应答,而不是笼统的 `401` 或网关 HTML。
+
+手动覆盖入口是 base URL 输入框右端内嵌的那段协议路径(`/responses`、`/v1/messages`、`/chat/completions`)——客户端会追加到你填的 URL 之后,与协议一一对应。点开它即列出三种协议及各自追加的路径,手动选择优先于检测结果——已经知道协议的端点根本不必探测。只要检测失败——连不上、超时、返回的不是 API 响应、三条路径都没提供——该后缀就转为琥珀色,并统一以 toast 提示同一句话:无法检测接口协议,请检查 API Key 与 base URL。逐个协议的探测结果仍由该接口返回,便于排查。此前创建的条目保留 `client_type = "openai"`(仍是 `openai-chat` 的别名),只有手动选择或检测生效时才会改写。检测能力以 `POST /api/projects/:id/models/detect` 暴露(仅 owner,见 [Server API](/server-api))。
+这些分组不会从 model id 推断任何东西。在自定义分组里填 `claude-sonnet-5` 不代表就走 Anthropic 客户端、读 `ANTHROPIC_*`——Custom 与自建分组一律回退到兼容客户端(`openai-chat`),API Key 提示也据此显示。检测只是锦上添花,不是关卡:若检测无结果,模型仍会按 `openai-chat` 保存,并以 toast 说明。厂商与网关分组不受影响——它们的 id 在内置目录里,仍按 id 路由或沿用预设。
+
+### 视觉能力检测
+
+「支持视觉」可以先不开,交给模型自己回答:开关旁边的「检测」会发送一张 1x1 的 PNG 和一句一个词的提示,模型能正常作答就把开关打开。若模型明确回答不接受图片,则关闭开关——这是一个真实答案,不是错误;而因鉴权或网络失败的探测则什么都不改动,只显示那句「请检查 API Key 与 base URL」。
+
+与协议检测不同,**这次探测是一次真实计费的补全**:图片请求无法像协议探测那样构造成零成本。因此它只在你主动点击时运行,不会自行触发,保存时也不会。凭据链与连通性测试一致——对话框里填的 Key,其次已保存的 Key,最后是该协议的环境变量,全部在服务端解析。新建的自定义模型默认关闭视觉;从厂商或网关分组添加的模型沿用内置目录已知的能力。
+
+
## 思考等级
对于 MiniMax M3,`none` 会直接映射为 `reasoning.effort = "none"`。
diff --git a/packages/docs/content/server-api.en.md b/packages/docs/content/server-api.en.md
index 7ca3d3f6..21049892 100644
--- a/packages/docs/content/server-api.en.md
+++ b/packages/docs/content/server-api.en.md
@@ -114,6 +114,8 @@ Member writes are owner-only. The member routes also answer `403 desktop_single_
| GET | /api/projects/:projectId/models | List models (api_key masked) |
| PUT | /api/projects/:projectId/models | Full-table replace, keyed by `(provider, modelId)` |
| POST | /api/projects/:projectId/models/test | Connectivity test: `{provider, modelId, …}` → `{ok, latencyMs?, message?}` |
+| POST | /api/projects/:projectId/models/detect | Protocol auto-detection for a custom base URL: probes `openai-responses` → `ant-messages` → `openai-chat` in order and reports the first served protocol: `{baseUrl, apiKey?, …}` → `{detected?, probes}` |
+| POST | /api/projects/:projectId/models/detect-vision | Vision capability probe: sends one 1x1 image on this model's credential (a real, billed completion): `{provider, modelId, apiKey?, baseUrl?, clientType?}` → `{outcome: supported\|unsupported\|failed, message?}` |
Every endpoint that names a model takes the complete `(provider, modelId)` pair. Nothing is inferred: a request carrying only one half is a 400, never a lookup. Where the reference itself is optional (Session creation, Schedules), omitting both halves selects the Project's default model.
diff --git a/packages/docs/content/server-api.zh.md b/packages/docs/content/server-api.zh.md
index 51bf95f4..69612bc7 100644
--- a/packages/docs/content/server-api.zh.md
+++ b/packages/docs/content/server-api.zh.md
@@ -114,6 +114,8 @@ curl -c cookies.txt -H "Content-Type: application/json" \
| GET | /api/projects/:projectId/models | 模型列表(api_key 掩码显示) |
| PUT | /api/projects/:projectId/models | 全表替换,条目以 `(provider, modelId)` 为键 |
| POST | /api/projects/:projectId/models/test | 连通性测试:`{provider, modelId, …}` → `{ok, latencyMs?, message?}` |
+| POST | /api/projects/:projectId/models/detect | 自定义 base URL 的协议自动检测:按 `openai-responses` → `ant-messages` → `openai-chat` 顺序探测并返回第一个被提供的协议:`{baseUrl, apiKey?, …}` → `{detected?, probes}` |
+| POST | /api/projects/:projectId/models/detect-vision | 视觉能力探测:用该模型的凭据发送一张 1x1 图片(一次真实计费的补全):`{provider, modelId, apiKey?, baseUrl?, clientType?}` → `{outcome: supported\|unsupported\|failed, message?}` |
所有涉及模型的接口都要求完整的 `(provider, modelId)` 二元组,不做任何推断:只带一半的请求一律 400,绝不会退化为一次查找。模型引用本身可省略的场景(创建 Session、定时任务)省略的是整对,两半都不给即选用 Project 默认模型。
diff --git a/packages/server/src/api/types.ts b/packages/server/src/api/types.ts
index fe47bfd2..ad21a511 100644
--- a/packages/server/src/api/types.ts
+++ b/packages/server/src/api/types.ts
@@ -392,6 +392,42 @@ export interface ModelTestRequest {
clientType?: string;
}
+/**
+ * Vision capability probe: sends one 1x1 image and a one-word prompt on this model's
+ * credential, to decide whether the models dialog's "supports vision" switch should be
+ * turned on. The body mirrors the connectivity test (same paired reference and same
+ * not-yet-saved overrides), so an unsaved model can be probed before it exists on disk.
+ *
+ * Unlike protocol detection this is a real, billed completion — an image request cannot be
+ * shaped to cost nothing — so it only ever runs when the user presses the control.
+ */
+export interface ModelVisionDetectRequest {
+ provider: string;
+ modelId: string;
+ /** Newly entered API key (plaintext); the stored one backs the probe when omitted. */
+ apiKey?: string;
+ /** "Clear saved API key" is checked: do not fall back to the stored key. */
+ clearApiKey?: boolean;
+ /** Form's current base URL; null means "explicitly none" (as in the connectivity test). */
+ baseUrl?: string | null;
+ /** Protocol to speak, when the form has one; otherwise the stored/auto-routed client. */
+ clientType?: string;
+}
+
+/**
+ * Vision probe verdict. `supported` = the model took the image and answered; `unsupported`
+ * = it answered specifically that it will not take an image (a definitive negative, not an
+ * error); `failed` = the probe never got a usable answer (auth, network, an unrelated
+ * error), so nothing about the capability was learned and the setting is left alone.
+ */
+export type ModelVisionOutcome = "supported" | "unsupported" | "failed";
+
+/** Vision probe result; `message` carries the truncated provider error for the failed case. */
+export interface ModelVisionDetectResponse {
+ outcome: ModelVisionOutcome;
+ message?: string;
+}
+
/**
* Connectivity test result: carries round-trip latency when ok, and a reason on failure
* (truncated raw provider error). When streamed content was observed, also carries the
@@ -412,6 +448,53 @@ export interface ModelTestResponse {
message?: string;
}
+/**
+ * Protocol auto-detection (POST /api/projects/:p/models/detect, owner): probes which of
+ * AgentHub's generic protocol clients a custom base URL serves — `openai-responses` first,
+ * then `ant-messages`, then `openai-chat` — and reports the first hit. Used by the custom
+ * model dialog to fill `clientType` from the endpoint itself; costs no tokens (each probe
+ * is a minimal invalid request whose error reveals the protocol shape).
+ */
+export interface ModelProtocolDetectRequest {
+ /** Base URL to probe (as typed in the form); each probe appends its protocol path. */
+ baseUrl: string;
+ /** Newly entered API key (plaintext); used for probe auth if provided. Detection also works keyless: a protocol-shaped 401/403 still proves the route. */
+ apiKey?: string;
+ /** "Clear saved API key" is checked: do not fall back to the stored key (probe the current draft). */
+ clearApiKey?: boolean;
+ /** Optional paired reference: when it names a stored entry and no apiKey is given, that entry's saved key backs the probes (mirrors the connectivity test). */
+ provider?: string;
+ modelId?: string;
+}
+
+/**
+ * One probe's outcome: `served` = the route answered in an API shape (including auth
+ * failures — 401/403 with a protocol-shaped body proves the route exists);
+ * `route_missing` = 404/405; `server_error` = 5xx (proves nothing about the path);
+ * `junk` = HTML / non-JSON / JSON matching no API shape; `timeout` / `network_error` =
+ * the request itself failed.
+ */
+export type ProtocolProbeOutcome =
+ "served" | "route_missing" | "server_error" | "junk" | "timeout" | "network_error";
+
+/** One probe, for debugging display: the probed URL derives from baseUrl only (never contains the key). */
+export interface ModelProtocolProbeDto {
+ /** AgentHub client type this probe stands for (`openai-responses` / `ant-messages` / `openai-chat`). */
+ clientType: string;
+ /** Full URL probed (base URL + the protocol's path). */
+ url: string;
+ outcome: ProtocolProbeOutcome;
+ /** HTTP status, when a response arrived at all. */
+ status?: number;
+}
+
+/** Detection result: probes run sequentially and stop at the first served protocol, so `probes` lists only the ones actually run, in order. */
+export interface ModelProtocolDetectResponse {
+ /** The first protocol the endpoint serves (an AgentHub client type); absent when none of the three matched. */
+ detected?: string;
+ probes: ModelProtocolProbeDto[];
+}
+
/**
* PUT /api/projects/:p/models/default (owner): narrow default-model switch — flips the same
* top-level `default_model` the models page's whole-table PUT writes, without resending the
diff --git a/packages/server/src/http/routes/models.ts b/packages/server/src/http/routes/models.ts
index 032945fc..65288c80 100644
--- a/packages/server/src/http/routes/models.ts
+++ b/packages/server/src/http/routes/models.ts
@@ -1,21 +1,26 @@
/**
* Model & credential config routes:
* GET|PUT /api/projects/:p/models, PUT /api/projects/:p/models/default,
- * POST /api/projects/:p/models/test (the model reference
- * `(provider, modelId)` is sent as a pair in the request body, avoiding URL-encoding
- * issues). Any member can read (api_key is masked); only the owner can modify or test.
+ * POST /api/projects/:p/models/test, POST /api/projects/:p/models/detect,
+ * POST /api/projects/:p/models/detect-vision (the model
+ * reference `(provider, modelId)` is sent as a pair in the request body, avoiding
+ * URL-encoding issues). Any member can read (api_key is masked); only the owner can
+ * modify, test, or detect.
*/
import { Hono } from "hono";
import type {
DefaultModelResponse,
+ ModelProtocolDetectRequest,
ModelRefDto,
ModelsUpdateRequest,
ModelTestRequest,
ModelUpdateEntry,
+ ModelVisionDetectRequest,
ServerEvent,
} from "../../api/types.js";
import type { AppEnv } from "../../auth/middleware.js";
import { badRequest, readJson, requireString, requireValidId } from "../validate.js";
+import { isHttpUrl } from "../../services/protocol-detect.js";
import type { AppDeps } from "../../app.js";
/**
@@ -226,5 +231,79 @@ export function modelsRoutes(deps: AppDeps): Hono<AppEnv> {
return c.json(await deps.projectConfigService.testModel(projectId, req));
});
+ // Protocol auto-detection (owner): probes which generic protocol client the base URL
+ // serves — openai-responses, then ant-messages, then openai-chat — and returns the
+ // first hit plus per-probe outcomes (see services/protocol-detect.ts). Cheap: each
+ // probe is a minimal invalid request (no tokens billed). Like the connectivity test,
+ // an optional paired reference lets a stored key back the probes without the frontend
+ // ever seeing the plaintext.
+ //
+ // On the server fetching a caller-supplied URL: deliberate, and not a new capability.
+ // /test already hands an arbitrary baseUrl to the provider SDK behind the same
+ // owner-only guard, and in a self-hosted product pointing the server at an internal or
+ // loopback inference endpoint is the feature (vLLM / Ollama / a LAN gateway), not the
+ // attack — so no host allowlist. This route is the tighter of the two: it rejects
+ // anything that is not an absolute http(s) URL, and reduces every answer to an outcome
+ // enum plus an HTTP status, where /test surfaces the upstream message verbatim.
+ app.post("/detect", async (c) => {
+ const projectId = requireValidId(c, "projectId");
+ deps.projectService.requireProjectOwner(c.var.user.userId, projectId);
+ const body = await readJson(c);
+ const req: ModelProtocolDetectRequest = {
+ baseUrl: requireString(body, "baseUrl", { minLen: 1, maxLen: 2000 }),
+ };
+ if (!isHttpUrl(req.baseUrl)) throw badRequest("baseUrl must be an absolute http(s) URL.");
+ if (body.apiKey !== undefined) {
+ if (typeof body.apiKey !== "string") throw badRequest("apiKey must be a string.");
+ if (body.apiKey) req.apiKey = body.apiKey;
+ }
+ if (body.clearApiKey !== undefined) {
+ if (typeof body.clearApiKey !== "boolean") throw badRequest("clearApiKey must be a boolean.");
+ req.clearApiKey = body.clearApiKey;
+ }
+ // The paired reference is optional (adding a not-yet-saved model has none); when
+ // present, both fields are required so the stored-key lookup is unambiguous.
+ if (body.provider !== undefined || body.modelId !== undefined) {
+ req.provider = requireString(body, "provider", { minLen: 1, maxLen: 64 });
+ req.modelId = requireString(body, "modelId", { minLen: 1, maxLen: 200 });
+ }
+ return c.json(await deps.projectConfigService.detectProtocol(projectId, req));
+ });
+
+ /**
+ * Vision capability probe (owner only). Body mirrors the connectivity test's, since the
+ * probe is one real completion on the same credential — see detectVision. Owner-only for
+ * the same reason as /test and /detect: it spends the Project's key.
+ */
+ app.post("/detect-vision", async (c) => {
+ const projectId = requireValidId(c, "projectId");
+ deps.projectService.requireProjectOwner(c.var.user.userId, projectId);
+ const body = await readJson(c);
+ const req: ModelVisionDetectRequest = {
+ provider: requireString(body, "provider", { minLen: 1, maxLen: 64 }),
+ modelId: requireString(body, "modelId", { minLen: 1, maxLen: 200 }),
+ };
+ if (body.apiKey !== undefined) {
+ if (typeof body.apiKey !== "string") throw badRequest("apiKey must be a string.");
+ if (body.apiKey) req.apiKey = body.apiKey;
+ }
+ if (body.clearApiKey !== undefined) {
+ if (typeof body.clearApiKey !== "boolean") throw badRequest("clearApiKey must be a boolean.");
+ req.clearApiKey = body.clearApiKey;
+ }
+ // null is meaningful (explicitly no base URL), so it is kept distinct from absent.
+ if (body.baseUrl !== undefined) {
+ if (body.baseUrl !== null && typeof body.baseUrl !== "string") {
+ throw badRequest("baseUrl must be a string or null.");
+ }
+ req.baseUrl = body.baseUrl as string | null;
+ }
+ if (body.clientType !== undefined) {
+ if (typeof body.clientType !== "string") throw badRequest("clientType must be a string.");
+ if (body.clientType) req.clientType = body.clientType;
+ }
+ return c.json(await deps.projectConfigService.detectVision(projectId, req));
+ });
+
return app;
}
diff --git a/packages/server/src/services/project-config-service.ts b/packages/server/src/services/project-config-service.ts
index efa0bb75..3eb8a8a4 100644
--- a/packages/server/src/services/project-config-service.ts
+++ b/packages/server/src/services/project-config-service.ts
@@ -33,6 +33,7 @@ import {
canonicalClientType,
catalogEntryFor,
defaultProjectConfig,
+ imageUrlMessage,
projectConfigFromTable,
projectConfigPath,
renderProjectConfigToml,
@@ -44,14 +45,27 @@ import type {
ChatDefaultsDto,
ModelInfo,
ModelPricingDto,
+ ModelProtocolDetectRequest,
+ ModelProtocolDetectResponse,
ModelRefDto,
ModelsResponse,
ModelsUpdateRequest,
ModelTestRequest,
ModelTestResponse,
+ ModelVisionDetectRequest,
+ ModelVisionDetectResponse,
} from "../api/types.js";
import { badRequest } from "../http/validate.js";
import { cacheable } from "../internal/mtime-gate.js";
+import { detectModelProtocol } from "./protocol-detect.js";
+import {
+ VISION_PROBE_IMAGE,
+ VISION_PROBE_MAX_TOKENS,
+ VISION_PROBE_PROMPT,
+ VISION_PROBE_TIMEOUT_MS,
+ classifyVisionProbe,
+ classifyVisionProbeError,
+} from "./vision-detect.js";
import type { PricingRates } from "./usage-service.js";
type RawTable = Record<string, unknown>;
@@ -394,6 +408,71 @@ export class ProjectConfigService {
* Tokens (single-digit output; speed mode spends up to its 64-token cap to have a
* window worth timing), and writes no Trace and records no usage.
*/
+ /**
+ * Vision capability probe: sends one 1x1 PNG plus a one-word prompt and reports whether
+ * the model took it (see services/vision-detect.ts for the verdict rules and why this
+ * one costs real money, unlike the protocol probes).
+ *
+ * Credential resolution is the connectivity test's, verbatim — the request's key, else
+ * the stored one, unless "clear" is checked. The environment layer needs no code here:
+ * omitting apiKey lets AgentHub read the protocol's own variable, which is the same
+ * chain protocol detection spells out by hand because it bypasses the SDK. Nothing
+ * secret is returned; the failure message is the provider's own text, truncated.
+ */
+ async detectVision(
+ projectId: string,
+ req: ModelVisionDetectRequest,
+ ): Promise<ModelVisionDetectResponse> {
+ const raw = await this.readRaw(projectId);
+ // Probeable before the entry exists, so a custom model can be checked while adding it.
+ const entry = asArray(raw.models).find((m) => entryMatches(m, req.provider, req.modelId)) ?? {};
+ const savedKey = optStr(entry.api_key);
+ const apiKey = req.clearApiKey ? undefined : (req.apiKey ?? savedKey);
+ const savedBaseUrl = optStr(entry.base_url);
+ const baseUrl = req.baseUrl === null ? undefined : (req.baseUrl ?? savedBaseUrl);
+ const clientType = canonicalClientType(req.clientType ?? optStr(entry.client_type));
+ try {
+ // Inside the try for the same reason as testModel: the SDK throws on a missing
+ // credential during construction, and that must read as "probe failed", not a 500.
+ const llm = new GenerativeModel({
+ modelId: req.modelId,
+ ...(apiKey ? { apiKey } : {}),
+ ...(baseUrl ? { baseUrl } : {}),
+ ...(clientType ? { clientType } : {}),
+ tools: [],
+ thinkingLevel: "none",
+ maxTokens: VISION_PROBE_MAX_TOKENS,
+ requestTimeoutMs: VISION_PROBE_TIMEOUT_MS,
+ });
+ // Both parts must carry role "user" — a mixed-role batch is rejected before it is
+ // sent (see generative-model's UniMessage merge).
+ const gen = llm.streamGenerate({
+ newMessages: [userText(VISION_PROBE_PROMPT), imageUrlMessage(VISION_PROBE_IMAGE)],
+ });
+ let sawContent = false;
+ for (;;) {
+ const step = await gen.next();
+ if (step.done) {
+ const outcome = classifyVisionProbe(step.value, sawContent);
+ if (outcome !== "failed") return { outcome };
+ const detail =
+ "errorMessage" in step.value && step.value.errorMessage
+ ? step.value.errorMessage
+ : step.value.status;
+ return { outcome, message: String(detail).slice(0, 300) };
+ }
+ if (isProbeContent(step.value)) sawContent = true;
+ }
+ } catch (err) {
+ const outcome = classifyVisionProbeError(err);
+ if (outcome !== "failed") return { outcome };
+ return {
+ outcome,
+ message: (err instanceof Error ? err.message : String(err)).slice(0, 300),
+ };
+ }
+ }
+
async testModel(projectId: string, req: ModelTestRequest): Promise<ModelTestResponse> {
const raw = await this.readRaw(projectId);
// Testable even if the model isn't in the config yet (validate before saving when adding a custom model): in that case all parameters come from the request body.
@@ -470,6 +549,35 @@ export class ProjectConfigService {
}
}
+ /**
+ * Protocol detection for a custom base URL (see services/protocol-detect.ts for the
+ * probe order and classification). Credential resolution has three layers, in order:
+ * 1. the request body's key (what the user just typed in the dialog);
+ * 2. otherwise, when the optional paired reference names a stored entry and "clear"
+ * isn't checked, that entry's saved key (the frontend only ever sees the mask);
+ * 3. otherwise the environment variable for whichever protocol each probe speaks,
+ * resolved inside detectModelProtocol because the protocol is the thing being
+ * determined (ANTHROPIC_API_KEY for ant-messages, OPENAI_API_KEY for the two
+ * OpenAI protocols).
+ * Layers 2 and 3 are read server-side only and never travel back to the browser.
+ * Detection still runs with no credential at all: a protocol-shaped 401/403 proves the
+ * route. Never throws on probe failures — every outcome is reported per probe.
+ */
+ async detectProtocol(
+ projectId: string,
+ req: ModelProtocolDetectRequest,
+ ): Promise<ModelProtocolDetectResponse> {
+ let apiKey = req.apiKey;
+ if (apiKey === undefined && !req.clearApiKey && req.provider && req.modelId) {
+ const raw = await this.readRaw(projectId);
+ const entry = asArray(raw.models).find((m) =>
+ entryMatches(m, req.provider as string, req.modelId as string),
+ );
+ apiKey = entry !== undefined ? optStr(entry.api_key) : undefined;
+ }
+ return detectModelProtocol({ baseUrl: req.baseUrl, ...(apiKey ? { apiKey } : {}) });
+ }
+
/**
* GET models view: masks credential (inline fields), flags the default Model;
* the group is the entry's `provider` field, looked up in the built-in catalog by
diff --git a/packages/server/src/services/protocol-detect.ts b/packages/server/src/services/protocol-detect.ts
new file mode 100644
index 00000000..f7006e32
--- /dev/null
+++ b/packages/server/src/services/protocol-detect.ts
@@ -0,0 +1,313 @@
+/**
+ * Protocol auto-detection for custom model endpoints.
+ *
+ * Given a base URL (plus an optional API key), probes which of AgentHub 0.4.2's three
+ * generic protocol clients the endpoint actually serves, in this fixed order:
+ *
+ * 1. `openai-responses` — OpenAI Responses API, POST {base}/responses
+ * 2. `ant-messages` — Anthropic Messages API, POST {base}/v1/messages
+ * 3. `openai-chat` — OpenAI Chat Completions, POST {base}/chat/completions
+ *
+ * Paths and auth headers mirror exactly what the AgentHub clients construct (verified
+ * against the SDKs they wrap): the OpenAI SDK appends `/responses` / `/chat/completions`
+ * to the base URL and authenticates with `Authorization: Bearer`; the Anthropic SDK
+ * appends `/v1/messages` and AgentHub's AntMessagesClient sends the credential through
+ * BOTH `x-api-key` and `Authorization: Bearer` (plus `anthropic-version`), which every
+ * covered server/gateway accepts. Probing the same URL the client would call means a
+ * detected protocol is one that will actually work after saving.
+ *
+ * Probe design: each probe POSTs a deliberately minimal invalid body (`{}`) — zero token
+ * cost, no valid model id needed, and the error it provokes reveals whether the route is
+ * really an API endpoint. Classification:
+ * - 404 / 405 → the route does not exist (`route_missing`). This is judged by status
+ * alone: OpenAI-style servers wrap unknown paths in protocol-shaped 404 JSON, and the
+ * probe body carries no model id, so a 404 can never mean "model not found".
+ * - other 4xx (400 validation, 401/403 auth, 422, 429 …) with a JSON API-error shape →
+ * the route exists and answers like an API (`served`). Auth failures count: probing
+ * without a key still detects the protocol. The error-shape check is deliberately
+ * tolerant (OpenAI `{error:{…}}`, Anthropic `{type:"error",…}`, vLLM/FastAPI
+ * `{detail:…}` / `{object:"error",…}`, bare `{message:…}`) because the probed PATH
+ * is what discriminates protocols — the shape check only weeds out HTML error pages
+ * and gateway junk.
+ * - 2xx → `served` only with protocol-specific success markers (guards against
+ * catch-all gateways that 200 every path).
+ * - 5xx → `server_error`, not served: gateways emit 5xx regardless of path existence,
+ * so it proves nothing; detection moves on to the next protocol.
+ * - non-JSON body → `junk`; fetch failures → `timeout` / `network_error`.
+ *
+ * Probes run sequentially in the order above and stop at the first `served` hit. The API
+ * key is only ever placed in request headers — never in URLs, results, or logs.
+ *
+ * Credential resolution is layered: the caller passes the key typed in the dialog or the
+ * entry's stored one, and when there is neither, each probe falls back to the environment
+ * variable for the protocol IT speaks (`ANTHROPIC_API_KEY` for `ant-messages`,
+ * `OPENAI_API_KEY` for the two OpenAI protocols — see envApiKeyForProtocol). Detection
+ * still works with no credential at all, but an authenticated probe is far more likely to
+ * draw a protocol-shaped error than the generic 401 or gateway HTML an anonymous request
+ * often gets, so the fallback materially improves accuracy.
+ */
+import { resolveModelEnv } from "@prismshadow/penguin-core";
+import type {
+ ModelProtocolDetectResponse,
+ ModelProtocolProbeDto,
+ ProtocolProbeOutcome,
+} from "../api/types.js";
+
+/** AgentHub generic protocol client types, in the required detection order. */
+export const PROTOCOL_CLIENT_TYPES = ["openai-responses", "ant-messages", "openai-chat"] as const;
+export type ProtocolClientType = (typeof PROTOCOL_CLIENT_TYPES)[number];
+
+/** Per-probe timeout: cheap probes against live endpoints answer well under this; a hung gateway must not stall the dialog. */
+export const PROBE_TIMEOUT_MS = 5_000;
+
+/** One probe target: the protocol's client type, request path, and auth header shape. */
+interface ProbeSpec {
+ clientType: ProtocolClientType;
+ path: string;
+ headers: (apiKey?: string) => Record<string, string>;
+}
+
+/** OpenAI-protocol auth (Responses and Chat Completions): Authorization: Bearer. */
+function bearerHeaders(apiKey?: string): Record<string, string> {
+ return {
+ "content-type": "application/json",
+ ...(apiKey ? { authorization: `Bearer ${apiKey}` } : {}),
+ };
+}
+
+/**
+ * Anthropic Messages auth: the credential through both header conventions —
+ * `x-api-key` (Anthropic, DeepSeek) and `Authorization: Bearer` (OpenRouter, Z.AI) —
+ * exactly like AgentHub's AntMessagesClient; `anthropic-version` is always sent
+ * (the SDK sends it unconditionally, and some servers 400 without it).
+ */
+function anthropicHeaders(apiKey?: string): Record<string, string> {
+ return {
+ "content-type": "application/json",
+ "anthropic-version": "2023-06-01",
+ ...(apiKey ? { authorization: `Bearer ${apiKey}`, "x-api-key": apiKey } : {}),
+ };
+}
+
+/** The three probes in the required order: Responses first, Messages second, Chat Completions as the last resort. */
+export const PROTOCOL_PROBES: readonly ProbeSpec[] = [
+ { clientType: "openai-responses", path: "/responses", headers: bearerHeaders },
+ { clientType: "ant-messages", path: "/v1/messages", headers: anthropicHeaders },
+ { clientType: "openai-chat", path: "/chat/completions", headers: bearerHeaders },
+];
+
+/**
+ * Environment-variable fallback for a probe's credential, resolved PER PROTOCOL.
+ *
+ * Which env var backs a request depends on the protocol being spoken, and detection is
+ * precisely the case where the protocol is not yet known — so the resolution happens once
+ * per probe rather than once per detection: `ant-messages` reads `ANTHROPIC_API_KEY`,
+ * `openai-responses` / `openai-chat` read `OPENAI_API_KEY`. `resolveModelEnv` is the same
+ * mapping the rest of the app uses (an explicit client type wins over the model id there),
+ * so this cannot drift from what the saved model will actually read.
+ *
+ * Server-side only: the value is placed in a request header and never returned to the
+ * browser, echoed in a result, or logged.
+ */
+export function envApiKeyForProtocol(
+ clientType: ProtocolClientType,
+ env: NodeJS.ProcessEnv = process.env,
+): string | undefined {
+ const envKey = resolveModelEnv("", clientType)?.envKey;
+ if (!envKey) return undefined;
+ return env[envKey]?.trim() || undefined;
+}
+
+/** Whether a string is an absolute http(s) URL (the only base URLs worth probing). */
+export function isHttpUrl(value: string): boolean {
+ try {
+ const u = new URL(value);
+ return u.protocol === "http:" || u.protocol === "https:";
+ } catch {
+ return false;
+ }
+}
+
+/**
+ * The full URL one probe requests: trailing slashes are stripped before appending the
+ * protocol path — the same join both wrapped SDKs perform, so the probe hits exactly
+ * the URL the real client would.
+ */
+export function probeUrl(baseUrl: string, path: string): string {
+ return baseUrl.trim().replace(/\/+$/, "") + path;
+}
+
+/** A JSON body that looks like a structured API error (vs an HTML page / junk): see the header doc for why this check is tolerant. */
+function isApiErrorShape(body: Record<string, unknown>): boolean {
+ const err = body.error;
+ if (err !== null && typeof err === "object") return true;
+ if (typeof err === "string" && err !== "") return true;
+ if (body.type === "error") return true;
+ if (body.object === "error") return true;
+ if (typeof body.message === "string" && body.message !== "") return true;
+ // FastAPI/vLLM `detail`: it must actually carry a message. `{"detail": null}` and
+ // `{"detail": ""}` say nothing about the route, and accepting them (a bare `!== undefined`
+ // check did) let a JSON body with one empty field pass as proof the route exists.
+ const detail = body.detail;
+ if (typeof detail === "string") return detail !== "";
+ if (detail !== null && (typeof detail === "object" || typeof detail === "number")) return true;
+ return false;
+}
+
+/** Protocol-specific success markers for a 2xx probe answer (an empty `{}` body should never succeed, so a 2xx must prove its shape). */
+function isSuccessShape(clientType: ProtocolClientType, body: Record<string, unknown>): boolean {
+ switch (clientType) {
+ case "openai-responses":
+ return body.object === "response" || Array.isArray(body.output);
+ case "ant-messages":
+ return body.type === "message" || (Array.isArray(body.content) && "role" in body);
+ case "openai-chat":
+ return body.object === "chat.completion" || Array.isArray(body.choices);
+ }
+}
+
+/**
+ * Classifies one probe's HTTP answer (pure; exported for unit tests). `bodyText` is the
+ * raw response body; non-JSON bodies are junk regardless of status.
+ */
+export function classifyProbeResponse(
+ clientType: ProtocolClientType,
+ status: number,
+ bodyText: string,
+): ProtocolProbeOutcome {
+ if (status === 404 || status === 405) return "route_missing";
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(bodyText);
+ } catch {
+ return "junk";
+ }
+ if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) return "junk";
+ const body = parsed as Record<string, unknown>;
+ if (status >= 200 && status < 300) return isSuccessShape(clientType, body) ? "served" : "junk";
+ if (status >= 500) return "server_error";
+ return isApiErrorShape(body) ? "served" : "junk";
+}
+
+/** AbortSignal.timeout aborts surface as TimeoutError (undici) or AbortError; both mean the probe timed out. */
+function isTimeoutError(err: unknown): boolean {
+ return err instanceof Error && (err.name === "TimeoutError" || err.name === "AbortError");
+}
+
+/**
+ * Byte budget for a probe's response body. The timeout caps how LONG an endpoint may
+ * answer for, not how MUCH it may send: a fast host can push a great deal in five seconds,
+ * and `res.text()` buffers all of it before classification ever looks at it. Classification
+ * only needs the top of a structured error, so anything past this cap is junk by
+ * definition — 64 KiB is orders of magnitude above any real API error body.
+ */
+export const MAX_PROBE_BODY_BYTES = 64 * 1024;
+
+/**
+ * Reads at most {@link MAX_PROBE_BODY_BYTES} of a probe response and reports whether the
+ * body ran past that. Streams rather than calling `res.text()`, so an oversized (or
+ * endless) body is abandoned at the cap instead of being buffered whole; the caller's
+ * timeout signal still bounds the read in time.
+ */
+async function readCappedBody(res: Response): Promise<{ text: string; truncated: boolean }> {
+ const body = res.body;
+ // A bodyless response (204, HEAD-like) or a fetch mock returning no stream: fall back to
+ // text(), which is empty or already-materialized in both cases.
+ if (!body) return { text: await res.text(), truncated: false };
+ const reader = body.getReader();
+ const chunks: Uint8Array[] = [];
+ let size = 0;
+ let truncated = false;
+ try {
+ for (;;) {
+ const { done, value } = await reader.read();
+ if (done) break;
+ if (!value) continue;
+ size += value.byteLength;
+ if (size > MAX_PROBE_BODY_BYTES) {
+ truncated = true;
+ break;
+ }
+ chunks.push(value);
+ }
+ } finally {
+ // Releases the connection: without a cancel, abandoning an oversized body would leave
+ // the socket draining in the background.
+ await reader.cancel().catch(() => {});
+ }
+ return { text: truncated ? "" : Buffer.concat(chunks).toString("utf8"), truncated };
+}
+
+async function runProbe(
+ spec: ProbeSpec,
+ baseUrl: string,
+ apiKey: string | undefined,
+ fetchImpl: typeof fetch,
+ timeoutMs: number,
+): Promise<ModelProtocolProbeDto> {
+ const url = probeUrl(baseUrl, spec.path);
+ try {
+ const res = await fetchImpl(url, {
+ method: "POST",
+ headers: spec.headers(apiKey),
+ body: "{}",
+ signal: AbortSignal.timeout(timeoutMs),
+ // Redirects stay followed (fetch's default). It is tempting to pin `redirect: "manual"`
+ // so a base URL cannot bounce the server somewhere it never named, but the SDKs the
+ // real clients wrap follow redirects too — pinning it would report "no protocol" for
+ // the ordinary case of a base URL that 301s http -> https, i.e. break the invariant
+ // that a detected protocol is one that actually works after saving. The probe reply is
+ // reduced to an outcome enum plus a status and is never echoed, so a followed redirect
+ // leaks no response content.
+ });
+ // The same timeout signal covers the body read (a junk endpoint may stream forever);
+ // the byte cap covers the other direction (an endpoint that streams a lot, fast).
+ const { text, truncated } = await readCappedBody(res);
+ return {
+ clientType: spec.clientType,
+ url,
+ outcome: truncated ? "junk" : classifyProbeResponse(spec.clientType, res.status, text),
+ status: res.status,
+ };
+ } catch (err) {
+ return {
+ clientType: spec.clientType,
+ url,
+ outcome: isTimeoutError(err) ? "timeout" : "network_error",
+ };
+ }
+}
+
+/**
+ * Runs the three probes sequentially in the required order and stops at the first
+ * protocol the endpoint serves. Never throws: every failure mode is a probe outcome.
+ * The result lists only the probes actually run (their order is the probe order), so
+ * callers can render per-protocol outcomes for debugging; no secret ever appears in it.
+ */
+export async function detectModelProtocol(options: {
+ baseUrl: string;
+ /**
+ * Explicit credential (the key typed in the dialog, or the entry's stored key). When
+ * absent, each probe falls back to the environment variable for the protocol it speaks
+ * — see envApiKeyForProtocol. An authenticated probe is what makes the difference
+ * between a protocol-shaped error (which identifies the route) and the generic 401 or
+ * gateway HTML that an anonymous request often draws instead.
+ */
+ apiKey?: string;
+ /** Injection point for tests; defaults to global fetch (which routes through the admin proxy settings, see net/proxy.ts). */
+ fetchImpl?: typeof fetch;
+ timeoutMs?: number;
+ /** Environment source for the per-protocol fallback; injectable so tests need not mutate process.env. */
+ env?: NodeJS.ProcessEnv;
+}): Promise<ModelProtocolDetectResponse> {
+ const fetchImpl = options.fetchImpl ?? fetch;
+ const timeoutMs = options.timeoutMs ?? PROBE_TIMEOUT_MS;
+ const probes: ModelProtocolProbeDto[] = [];
+ for (const spec of PROTOCOL_PROBES) {
+ const apiKey = options.apiKey ?? envApiKeyForProtocol(spec.clientType, options.env);
+ const probe = await runProbe(spec, options.baseUrl, apiKey, fetchImpl, timeoutMs);
+ probes.push(probe);
+ if (probe.outcome === "served") return { detected: spec.clientType, probes };
+ }
+ return { probes };
+}
diff --git a/packages/server/src/services/vision-detect.ts b/packages/server/src/services/vision-detect.ts
new file mode 100644
index 00000000..ed201268
--- /dev/null
+++ b/packages/server/src/services/vision-detect.ts
@@ -0,0 +1,113 @@
+/**
+ * Vision capability detection for a configured model.
+ *
+ * The models dialog can leave "supports vision" unset and ask the model itself. Unlike the
+ * protocol probes (services/protocol-detect.ts), this cannot be a deliberately invalid
+ * request: the only way to learn whether an endpoint accepts image input is to send an
+ * image and see what comes back. So this probe **is a real, billed completion** on the
+ * user's credential. Everything here exists to make it the cheapest honest one:
+ *
+ * - a 1x1 PNG (67 bytes) — the smallest thing that is still a decodable image;
+ * - a prompt asking for one word, with the same anti-thinking hint the connectivity
+ * probe uses, so reasoning models do not burn the budget before answering;
+ * - `maxTokens` in the single digits and no tools, so the completion cannot run long.
+ *
+ * Verdict, deliberately three-valued — "it didn't work" and "it works and the answer is
+ * no" are different facts, and only the first is worth telling the user to go check their
+ * credentials over:
+ * - `supported` the model accepted the image and answered;
+ * - `unsupported` the model answered *about the image being unacceptable* — a modality
+ * rejection is a definitive negative, not an error;
+ * - `failed` anything else (auth, network, timeout, an unrelated 400): the probe
+ * learned nothing, so the capability stays exactly as the user left it.
+ */
+import type { LLMOutcome } from "@prismshadow/penguin-core";
+
+/** What one vision probe concluded. */
+export type VisionProbeOutcome = "supported" | "unsupported" | "failed";
+
+/**
+ * A 1x1 transparent PNG as a data URL — the smallest input that still decodes as an image
+ * for every provider. Inline rather than read from disk: it must not depend on packaging.
+ */
+export const VISION_PROBE_IMAGE =
+ "data:image/png;base64," +
+ "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=";
+
+/**
+ * Vision probe prompt: one word out, no reasoning. Mirrors the connectivity probe's
+ * trailing empty <think></think>, which many reasoning models read as "thinking already
+ * closed" — without it the tiny output budget is spent before any answer appears.
+ */
+export const VISION_PROBE_PROMPT =
+ 'Reply with the single word "ok" and nothing else. Do not think or explain.\n<think></think>';
+
+/** Output cap for the probe: enough for one word, nothing more. */
+export const VISION_PROBE_MAX_TOKENS = 8;
+
+/** Request timeout, matching the connectivity test. */
+export const VISION_PROBE_TIMEOUT_MS = 20_000;
+
+/**
+ * Error fragments that mean "this model/endpoint will not take an image", as opposed to
+ * "the request failed". Matched case-insensitively against the provider's error text.
+ *
+ * Kept broad on purpose: providers word this very differently (OpenAI-compatible servers
+ * talk about the content part type, Anthropic about the block type, gateways about the
+ * modality), and the cost of a miss is asymmetric — classifying a real modality rejection
+ * as `failed` merely shows the generic "check your key and URL" message, which is
+ * confusing but harmless, while the reverse would switch vision OFF on a working model.
+ * So every pattern here names an image/modality concept explicitly; nothing matches on a
+ * bare "invalid request".
+ */
+const UNSUPPORTED_PATTERNS: readonly string[] = [
+ "image_url",
+ "image url",
+ "does not support image",
+ "not support image",
+ "unsupported image",
+ "image input",
+ "image content",
+ "invalid image",
+ "vision",
+ "multimodal",
+ "modality",
+ "image is not supported",
+ "no support for image",
+ "content type 'image",
+ 'content type "image',
+ "unsupported content",
+ "cannot process image",
+ "image not supported",
+];
+
+/**
+ * Whether a provider error says the model refused the IMAGE specifically. Only consulted
+ * for a failed completion, so an ordinary success never reaches it.
+ */
+export function isImageRejection(message: string): boolean {
+ const m = message.toLowerCase();
+ return UNSUPPORTED_PATTERNS.some((p) => m.includes(p));
+}
+
+/**
+ * Classify one probe run. `sawContent` mirrors the connectivity test's allowance: a model
+ * that streamed real content and then tripped its tiny cap has demonstrably accepted the
+ * image, which is the whole question here.
+ */
+export function classifyVisionProbe(outcome: LLMOutcome, sawContent: boolean): VisionProbeOutcome {
+ if (outcome.status === "completed") return "supported";
+ if (sawContent) return "supported";
+ const detail =
+ "errorMessage" in outcome && outcome.errorMessage ? outcome.errorMessage : outcome.status;
+ return isImageRejection(String(detail)) ? "unsupported" : "failed";
+}
+
+/**
+ * Classify a thrown error (the SDK can throw during construction or the first chunk, e.g.
+ * on a missing credential). Same rule: only an image-specific complaint is a real "no".
+ */
+export function classifyVisionProbeError(err: unknown): VisionProbeOutcome {
+ const message = err instanceof Error ? err.message : String(err);
+ return isImageRejection(message) ? "unsupported" : "failed";
+}
diff --git a/packages/server/test/protocol-detect.test.ts b/packages/server/test/protocol-detect.test.ts
new file mode 100644
index 00000000..b74dc169
--- /dev/null
+++ b/packages/server/test/protocol-detect.test.ts
@@ -0,0 +1,574 @@
+/**
+ * Protocol auto-detection tests: classification of probe answers (route-exists vs
+ * route-missing 404/405 vs HTML/gateway junk vs 5xx), the required probe order
+ * (openai-responses → ant-messages → openai-chat, stop at the first served protocol),
+ * auth header shapes per protocol (Bearer for the OpenAI protocols; x-api-key +
+ * Bearer + anthropic-version for ant-messages, mirroring AgentHub's clients), the
+ * per-protocol environment-variable fallback used when no key was typed or stored
+ * (ANTHROPIC_* for ant-messages, OPENAI_* for the other two), and the
+ * POST /api/projects/:p/models/detect route (validation, stored-key fallback via the
+ * paired reference, and that the key never appears in the response).
+ */
+import { createServer } from "node:http";
+import type { AddressInfo } from "node:net";
+import type { IncomingMessage, Server, ServerResponse } from "node:http";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import type { ModelProtocolDetectResponse, ModelsResponse } from "../src/api/types.js";
+import {
+ MAX_PROBE_BODY_BYTES,
+ classifyProbeResponse,
+ detectModelProtocol,
+ envApiKeyForProtocol,
+ isHttpUrl,
+ probeUrl,
+} from "../src/services/protocol-detect.js";
+import { apiClient, createTestApp, provisionUser } from "./helpers.js";
+import type { TestApp } from "./helpers.js";
+
+// ---------------------------------------------------------------------------
+// Pure classification
+// ---------------------------------------------------------------------------
+
+const OPENAI_ERROR = JSON.stringify({
+ error: { message: "you must provide a model parameter", type: "invalid_request_error" },
+});
+const ANTHROPIC_ERROR = JSON.stringify({
+ type: "error",
+ error: { type: "invalid_request_error", message: "messages: Field required" },
+});
+
+describe("classifyProbeResponse", () => {
+ it("404/405 mean route-missing by status alone — even in protocol-shaped JSON (OpenAI wraps unknown paths in its error shape; the {} probe carries no model id, so 404 can never mean model-not-found)", () => {
+ expect(classifyProbeResponse("openai-responses", 404, OPENAI_ERROR)).toBe("route_missing");
+ expect(classifyProbeResponse("ant-messages", 404, ANTHROPIC_ERROR)).toBe("route_missing");
+ expect(classifyProbeResponse("openai-chat", 405, '{"detail":"Method Not Allowed"}')).toBe(
+ "route_missing",
+ );
+ });
+
+ it("4xx with an API-error JSON shape means the route exists (served) — auth failures included", () => {
+ expect(classifyProbeResponse("openai-responses", 401, OPENAI_ERROR)).toBe("served");
+ expect(classifyProbeResponse("openai-responses", 400, OPENAI_ERROR)).toBe("served");
+ expect(classifyProbeResponse("ant-messages", 401, ANTHROPIC_ERROR)).toBe("served");
+ expect(classifyProbeResponse("openai-chat", 403, OPENAI_ERROR)).toBe("served");
+ expect(classifyProbeResponse("openai-chat", 429, OPENAI_ERROR)).toBe("served");
+ // vLLM / FastAPI dialects: 422 {"detail": …} validation errors and the flat
+ // {"object":"error"} shape are real API answers from an existing route.
+ expect(classifyProbeResponse("openai-chat", 422, '{"detail":[{"loc":["body","model"]}]}')).toBe(
+ "served",
+ );
+ expect(
+ classifyProbeResponse(
+ "openai-chat",
+ 400,
+ '{"object":"error","message":"model is required","type":"BadRequestError"}',
+ ),
+ ).toBe("served");
+ // Gateways answering a bare {"error": "..."} string still count.
+ expect(classifyProbeResponse("openai-responses", 400, '{"error":"missing model"}')).toBe(
+ "served",
+ );
+ });
+
+ it("HTML and non-JSON bodies are junk regardless of status (edge/login walls are not the protocol route)", () => {
+ expect(classifyProbeResponse("openai-responses", 200, "<!doctype html><html></html>")).toBe(
+ "junk",
+ );
+ expect(
+ classifyProbeResponse("openai-chat", 401, "<html>401 Authorization Required</html>"),
+ ).toBe("junk");
+ expect(classifyProbeResponse("ant-messages", 400, "Bad Request")).toBe("junk");
+ expect(classifyProbeResponse("openai-chat", 200, '"just a string"')).toBe("junk");
+ });
+
+ it("2xx needs protocol success markers (catch-all gateways that 200 every path are junk)", () => {
+ expect(classifyProbeResponse("openai-responses", 200, '{"ok":true}')).toBe("junk");
+ expect(
+ classifyProbeResponse("openai-responses", 200, '{"object":"response","output":[]}'),
+ ).toBe("served");
+ expect(classifyProbeResponse("openai-chat", 200, '{"object":"chat.completion"}')).toBe(
+ "served",
+ );
+ expect(classifyProbeResponse("openai-chat", 200, '{"choices":[]}')).toBe("served");
+ expect(
+ classifyProbeResponse("ant-messages", 200, '{"type":"message","role":"assistant"}'),
+ ).toBe("served");
+ // Chat markers on the responses path do not make the Responses protocol served.
+ expect(classifyProbeResponse("openai-responses", 200, '{"choices":[]}')).toBe("junk");
+ });
+
+ it("5xx proves nothing about the path (gateways emit it for any URL), so it is server_error, not served", () => {
+ expect(classifyProbeResponse("openai-responses", 500, OPENAI_ERROR)).toBe("server_error");
+ expect(classifyProbeResponse("openai-chat", 503, OPENAI_ERROR)).toBe("server_error");
+ expect(classifyProbeResponse("ant-messages", 502, "<html>bad gateway</html>")).toBe("junk");
+ });
+
+ it("4xx JSON that matches no API-error shape is junk", () => {
+ expect(classifyProbeResponse("openai-chat", 400, '{"status":"bad"}')).toBe("junk");
+ });
+
+ it("a FastAPI-style `detail` only proves the route when it carries a message (null / empty say nothing)", () => {
+ expect(classifyProbeResponse("openai-chat", 400, '{"detail":"field required"}')).toBe("served");
+ expect(classifyProbeResponse("openai-chat", 400, '{"detail":[{"loc":["body"]}]}')).toBe(
+ "served",
+ );
+ expect(classifyProbeResponse("openai-chat", 400, '{"detail":null}')).toBe("junk");
+ expect(classifyProbeResponse("openai-chat", 400, '{"detail":""}')).toBe("junk");
+ });
+});
+
+describe("isHttpUrl / probeUrl", () => {
+ it("accepts absolute http(s) URLs only", () => {
+ expect(isHttpUrl("https://api.example.com/v1")).toBe(true);
+ expect(isHttpUrl("http://127.0.0.1:8000/v1")).toBe(true);
+ expect(isHttpUrl("ftp://example.com")).toBe(false);
+ expect(isHttpUrl("api.example.com/v1")).toBe(false);
+ expect(isHttpUrl("")).toBe(false);
+ });
+
+ it("joins base URL and protocol path the way the wrapped SDKs do (trailing slashes stripped)", () => {
+ expect(probeUrl("https://api.example.com/v1", "/responses")).toBe(
+ "https://api.example.com/v1/responses",
+ );
+ expect(probeUrl("https://api.example.com/v1/", "/responses")).toBe(
+ "https://api.example.com/v1/responses",
+ );
+ expect(probeUrl("https://api.example.com/anthropic/", "/v1/messages")).toBe(
+ "https://api.example.com/anthropic/v1/messages",
+ );
+ });
+});
+
+// ---------------------------------------------------------------------------
+// Probe ordering and header shapes (mocked fetch)
+// ---------------------------------------------------------------------------
+
+interface SeenCall {
+ url: string;
+ headers: Record<string, string>;
+ body: string;
+}
+
+/** Fake fetch serving fixed answers by pathname; unknown paths get an OpenAI-style JSON 404 (like real OpenAI-compatible servers). */
+function fakeFetch(
+ routes: Record<string, { status: number; body: string }>,
+ seen?: SeenCall[],
+): typeof fetch {
+ return (async (input: Parameters<typeof fetch>[0], init?: RequestInit) => {
+ const url = String(input);
+ const headers: Record<string, string> = {};
+ for (const [k, v] of Object.entries((init?.headers ?? {}) as Record<string, string>)) {
+ headers[k.toLowerCase()] = v;
+ }
+ seen?.push({ url, headers, body: String(init?.body ?? "") });
+ const route = routes[new URL(url).pathname];
+ if (!route) {
+ return new Response(OPENAI_ERROR, {
+ status: 404,
+ headers: { "content-type": "application/json" },
+ });
+ }
+ return new Response(route.body, {
+ status: route.status,
+ headers: { "content-type": "application/json" },
+ });
+ }) as typeof fetch;
+}
+
+describe("detectModelProtocol (order and stop-at-first)", () => {
+ const BASE = "https://gw.example.com/v1";
+
+ it("openai-responses wins when served, and later probes never run", async () => {
+ const seen: SeenCall[] = [];
+ const res = await detectModelProtocol({
+ baseUrl: BASE,
+ apiKey: "sk-unit",
+ fetchImpl: fakeFetch(
+ {
+ "/v1/responses": { status: 401, body: OPENAI_ERROR },
+ "/v1/v1/messages": { status: 401, body: ANTHROPIC_ERROR },
+ "/v1/chat/completions": { status: 401, body: OPENAI_ERROR },
+ },
+ seen,
+ ),
+ });
+ expect(res.detected).toBe("openai-responses");
+ expect(res.probes.map((p) => p.outcome)).toEqual(["served"]);
+ expect(seen.map((c) => c.url)).toEqual(["https://gw.example.com/v1/responses"]);
+ });
+
+ it("falls through to ant-messages when /responses is missing", async () => {
+ const res = await detectModelProtocol({
+ baseUrl: BASE,
+ fetchImpl: fakeFetch({
+ "/v1/v1/messages": { status: 401, body: ANTHROPIC_ERROR },
+ "/v1/chat/completions": { status: 401, body: OPENAI_ERROR },
+ }),
+ });
+ expect(res.detected).toBe("ant-messages");
+ expect(res.probes.map((p) => [p.clientType, p.outcome])).toEqual([
+ ["openai-responses", "route_missing"],
+ ["ant-messages", "served"],
+ ]);
+ expect(res.probes[1]!.status).toBe(401);
+ });
+
+ it("openai-chat is the last resort", async () => {
+ const res = await detectModelProtocol({
+ baseUrl: BASE,
+ fetchImpl: fakeFetch({
+ "/v1/chat/completions": { status: 400, body: OPENAI_ERROR },
+ }),
+ });
+ expect(res.detected).toBe("openai-chat");
+ expect(res.probes.map((p) => p.outcome)).toEqual(["route_missing", "route_missing", "served"]);
+ });
+
+ it("nothing matches: no detected protocol, all three probes reported", async () => {
+ const res = await detectModelProtocol({ baseUrl: BASE, fetchImpl: fakeFetch({}) });
+ expect(res.detected).toBeUndefined();
+ expect(res.probes).toHaveLength(3);
+ expect(res.probes.every((p) => p.outcome === "route_missing")).toBe(true);
+ });
+
+ it("probe failures do not stop the sequence: a timeout on /responses still finds chat completions", async () => {
+ const timeoutOnResponses: typeof fetch = (async (input: Parameters<typeof fetch>[0]) => {
+ const path = new URL(String(input)).pathname;
+ if (path === "/v1/responses") {
+ throw new DOMException("The operation timed out", "TimeoutError");
+ }
+ if (path === "/v1/v1/messages") {
+ throw new TypeError("fetch failed"); // connection refused / DNS style failure
+ }
+ return new Response(OPENAI_ERROR, {
+ status: 401,
+ headers: { "content-type": "application/json" },
+ });
+ }) as typeof fetch;
+ const res = await detectModelProtocol({ baseUrl: BASE, fetchImpl: timeoutOnResponses });
+ expect(res.detected).toBe("openai-chat");
+ expect(res.probes.map((p) => p.outcome)).toEqual(["timeout", "network_error", "served"]);
+ });
+
+ it("an oversized response body is abandoned at the cap and counts as junk, so a flooding endpoint neither detects nor buffers", async () => {
+ // Streams well past MAX_PROBE_BODY_BYTES; the reader must stop at the cap rather than
+ // materialize the whole thing, and the probe must not be credited as served.
+ let pushed = 0;
+ const flooder: typeof fetch = (async (input: Parameters<typeof fetch>[0]) => {
+ const path = new URL(String(input)).pathname;
+ if (path === "/v1/v1/messages") {
+ return new Response(OPENAI_ERROR, {
+ status: 404,
+ headers: { "content-type": "application/json" },
+ });
+ }
+ if (path !== "/v1/responses") {
+ return new Response(OPENAI_ERROR, {
+ status: 401,
+ headers: { "content-type": "application/json" },
+ });
+ }
+ const chunk = new Uint8Array(8 * 1024).fill(0x20); // 8 KiB of spaces per pull
+ return new Response(
+ new ReadableStream<Uint8Array>({
+ pull(controller) {
+ pushed += chunk.byteLength;
+ // Far more than the cap if it were ever drained in full.
+ if (pushed > MAX_PROBE_BODY_BYTES * 100) {
+ controller.close();
+ return;
+ }
+ controller.enqueue(chunk);
+ },
+ }),
+ { status: 200, headers: { "content-type": "application/json" } },
+ );
+ }) as typeof fetch;
+ const res = await detectModelProtocol({ baseUrl: BASE, fetchImpl: flooder });
+ expect(res.probes[0]?.outcome).toBe("junk");
+ expect(res.detected).toBe("openai-chat");
+ // Read stopped near the cap instead of draining the stream (the producer runs a chunk
+ // or two ahead of the reader, so this is "the cap plus slack", not the exact cap —
+ // what matters is that it is nowhere near the 100x the stream would have emitted).
+ expect(pushed).toBeLessThanOrEqual(MAX_PROBE_BODY_BYTES * 2);
+ });
+
+ it("auth headers mirror the AgentHub clients: Bearer for the OpenAI protocols; x-api-key + Bearer + anthropic-version for ant-messages; the {} body and key-free URLs everywhere", async () => {
+ const seen: SeenCall[] = [];
+ await detectModelProtocol({
+ baseUrl: `${BASE}/`, // trailing slash must not double up
+ apiKey: "sk-secret-key",
+ fetchImpl: fakeFetch({}, seen),
+ });
+ expect(seen).toHaveLength(3);
+ const [responses, messages, chat] = seen as [SeenCall, SeenCall, SeenCall];
+ expect(responses.url).toBe("https://gw.example.com/v1/responses");
+ expect(messages.url).toBe("https://gw.example.com/v1/v1/messages");
+ expect(chat.url).toBe("https://gw.example.com/v1/chat/completions");
+ for (const call of seen) {
+ expect(call.body).toBe("{}");
+ expect(call.url).not.toContain("sk-secret-key");
+ expect(call.headers.authorization).toBe("Bearer sk-secret-key");
+ }
+ expect(responses.headers["x-api-key"]).toBeUndefined();
+ expect(chat.headers["x-api-key"]).toBeUndefined();
+ expect(messages.headers["x-api-key"]).toBe("sk-secret-key");
+ expect(messages.headers["anthropic-version"]).toBe("2023-06-01");
+ });
+
+ it("keyless probes send no credential headers (anthropic-version stays), so a 401 in protocol shape still detects", async () => {
+ const seen: SeenCall[] = [];
+ const res = await detectModelProtocol({
+ baseUrl: BASE,
+ // Empty env: no key from anywhere, so this stays the genuinely keyless case
+ // regardless of what the machine running the suite happens to export.
+ env: {},
+ fetchImpl: fakeFetch({ "/v1/responses": { status: 401, body: OPENAI_ERROR } }, seen),
+ });
+ expect(res.detected).toBe("openai-responses");
+ expect(seen[0]!.headers.authorization).toBeUndefined();
+ expect(seen[0]!.headers["x-api-key"]).toBeUndefined();
+ });
+});
+
+// ---------------------------------------------------------------------------
+// Environment-variable fallback (no key typed, none stored)
+// ---------------------------------------------------------------------------
+
+describe("envApiKeyForProtocol", () => {
+ it("maps each protocol to the variable its AgentHub client actually reads", () => {
+ const env = { OPENAI_API_KEY: "sk-openai", ANTHROPIC_API_KEY: "sk-anthropic" };
+ expect(envApiKeyForProtocol("openai-responses", env)).toBe("sk-openai");
+ expect(envApiKeyForProtocol("openai-chat", env)).toBe("sk-openai");
+ // The whole point of resolving per protocol: this one is NOT an OPENAI_* reader.
+ expect(envApiKeyForProtocol("ant-messages", env)).toBe("sk-anthropic");
+ });
+
+ it("treats unset and blank as absent", () => {
+ expect(envApiKeyForProtocol("openai-chat", {})).toBeUndefined();
+ expect(envApiKeyForProtocol("openai-chat", { OPENAI_API_KEY: " " })).toBeUndefined();
+ expect(envApiKeyForProtocol("ant-messages", { OPENAI_API_KEY: "sk-openai" })).toBeUndefined();
+ });
+});
+
+describe("detectModelProtocol env fallback", () => {
+ const BASE = "https://gw.example.com/v1";
+
+ it("authenticates each probe from the env var for the protocol THAT probe speaks", async () => {
+ const seen: SeenCall[] = [];
+ await detectModelProtocol({
+ baseUrl: BASE,
+ env: { OPENAI_API_KEY: "sk-openai", ANTHROPIC_API_KEY: "sk-anthropic" },
+ fetchImpl: fakeFetch({}, seen), // every path 404s, so all three probes run
+ });
+ expect(seen).toHaveLength(3);
+ const [responses, messages, chat] = seen as [SeenCall, SeenCall, SeenCall];
+ expect(responses.headers.authorization).toBe("Bearer sk-openai");
+ expect(chat.headers.authorization).toBe("Bearer sk-openai");
+ // ant-messages carries the Anthropic key through both header conventions.
+ expect(messages.headers.authorization).toBe("Bearer sk-anthropic");
+ expect(messages.headers["x-api-key"]).toBe("sk-anthropic");
+ // Whatever the source, the credential never reaches a URL.
+ for (const call of seen) {
+ expect(call.url).not.toContain("sk-openai");
+ expect(call.url).not.toContain("sk-anthropic");
+ }
+ });
+
+ it("an explicit key wins over the environment for every probe", async () => {
+ const seen: SeenCall[] = [];
+ await detectModelProtocol({
+ baseUrl: BASE,
+ apiKey: "sk-typed",
+ env: { OPENAI_API_KEY: "sk-openai", ANTHROPIC_API_KEY: "sk-anthropic" },
+ fetchImpl: fakeFetch({}, seen),
+ });
+ for (const call of seen) expect(call.headers.authorization).toBe("Bearer sk-typed");
+ expect((seen[1] as SeenCall).headers["x-api-key"]).toBe("sk-typed");
+ });
+
+ it("falls back per protocol, so a half-configured environment still authenticates its own probe", async () => {
+ const seen: SeenCall[] = [];
+ await detectModelProtocol({
+ baseUrl: BASE,
+ env: { ANTHROPIC_API_KEY: "sk-anthropic" },
+ fetchImpl: fakeFetch({}, seen),
+ });
+ const [responses, messages, chat] = seen as [SeenCall, SeenCall, SeenCall];
+ expect(responses.headers.authorization).toBeUndefined();
+ expect(chat.headers.authorization).toBeUndefined();
+ expect(messages.headers["x-api-key"]).toBe("sk-anthropic");
+ });
+
+ it("never echoes an env-sourced key into the reported probes", async () => {
+ const res = await detectModelProtocol({
+ baseUrl: BASE,
+ env: { OPENAI_API_KEY: "sk-openai", ANTHROPIC_API_KEY: "sk-anthropic" },
+ fetchImpl: fakeFetch({ "/v1/responses": { status: 401, body: OPENAI_ERROR } }),
+ });
+ expect(res.detected).toBe("openai-responses");
+ expect(JSON.stringify(res)).not.toContain("sk-openai");
+ expect(JSON.stringify(res)).not.toContain("sk-anthropic");
+ });
+});
+
+// ---------------------------------------------------------------------------
+// Route integration
+// ---------------------------------------------------------------------------
+
+/** Local endpoint serving only the Anthropic Messages path; records each request's path and auth headers. */
+function antOnlyServer(seen: Array<{ path: string; auth?: string; xApiKey?: string }>): Server {
+ return createServer((req: IncomingMessage, res: ServerResponse) => {
+ req.on("data", () => {});
+ req.on("end", () => {
+ seen.push({
+ path: req.url ?? "",
+ ...(req.headers.authorization ? { auth: req.headers.authorization } : {}),
+ ...(typeof req.headers["x-api-key"] === "string"
+ ? { xApiKey: req.headers["x-api-key"] }
+ : {}),
+ });
+ res.setHeader("content-type", "application/json");
+ if (req.url === "/v1/messages") {
+ res.statusCode = 401;
+ res.end(ANTHROPIC_ERROR);
+ } else {
+ res.statusCode = 404;
+ res.end(OPENAI_ERROR);
+ }
+ });
+ });
+}
+
+describe("POST /api/projects/:p/models/detect", () => {
+ let t: TestApp;
+ let api: ReturnType<typeof apiClient>;
+ let projectId: string;
+ const detectUrl = () => `/api/projects/${projectId}/models/detect`;
+
+ beforeEach(async () => {
+ t = await createTestApp();
+ const { cookie } = await provisionUser(t.app, "diana");
+ api = apiClient(t.app, cookie);
+ const created = (await (
+ await api.post("/api/projects", { projectId: "diana-detect", name: "Detect project" })
+ ).json()) as { project: { projectId: string } };
+ projectId = created.project.projectId;
+ });
+ afterEach(async () => {
+ await t.cleanup();
+ });
+
+ it("validates baseUrl: required and absolute http(s)", async () => {
+ expect((await api.post(detectUrl(), {})).status).toBe(400);
+ expect((await api.post(detectUrl(), { baseUrl: "ftp://example.com" })).status).toBe(400);
+ expect((await api.post(detectUrl(), { baseUrl: "not a url" })).status).toBe(400);
+ });
+
+ it("detects ant-messages on a live endpoint (404 on /responses, protocol-shaped 401 on /v1/messages) and never echoes the key", async () => {
+ const seen: Array<{ path: string; auth?: string; xApiKey?: string }> = [];
+ const server = antOnlyServer(seen);
+ await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
+ const port = (server.address() as AddressInfo).port;
+ try {
+ const res = await api.post(detectUrl(), {
+ baseUrl: `http://127.0.0.1:${port}`,
+ apiKey: "sk-local-detect",
+ });
+ expect(res.status).toBe(200);
+ const body = (await res.json()) as ModelProtocolDetectResponse;
+ expect(body.detected).toBe("ant-messages");
+ expect(body.probes.map((p) => [p.clientType, p.outcome])).toEqual([
+ ["openai-responses", "route_missing"],
+ ["ant-messages", "served"],
+ ]);
+ expect(body.probes[1]!.status).toBe(401);
+ // The key travels only in headers, and the response carries no secret.
+ expect(JSON.stringify(body)).not.toContain("sk-local-detect");
+ expect(seen.map((c) => c.path)).toEqual(["/responses", "/v1/messages"]);
+ expect(seen[1]!.xApiKey).toBe("sk-local-detect");
+ expect(seen[1]!.auth).toBe("Bearer sk-local-detect");
+ } finally {
+ await new Promise<void>((resolve) => server.close(() => resolve()));
+ }
+ });
+
+ it("the paired reference falls back to the stored key (probe before the frontend ever sees a plaintext key); clearApiKey suppresses that fallback and leaves only the env var", async () => {
+ const seen: Array<{ path: string; auth?: string; xApiKey?: string }> = [];
+ const server = antOnlyServer(seen);
+ await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
+ const port = (server.address() as AddressInfo).port;
+ try {
+ const put = await api.put(`/api/projects/${projectId}/models`, {
+ models: [
+ {
+ provider: "custom",
+ modelId: "stored-key-model",
+ apiKey: "sk-stored-secret",
+ baseUrl: `http://127.0.0.1:${port}`,
+ },
+ ],
+ });
+ expect(put.status).toBe(200);
+ expect(((await put.json()) as ModelsResponse).models).toHaveLength(1);
+
+ const res = await api.post(detectUrl(), {
+ baseUrl: `http://127.0.0.1:${port}`,
+ provider: "custom",
+ modelId: "stored-key-model",
+ });
+ expect(res.status).toBe(200);
+ expect(((await res.json()) as ModelProtocolDetectResponse).detected).toBe("ant-messages");
+ const messagesProbe = seen.find((c) => c.path === "/v1/messages");
+ expect(messagesProbe?.xApiKey).toBe("sk-stored-secret");
+
+ // clearApiKey drops the STORED key. What backs the probe then is whatever the saved
+ // model itself would run on, i.e. the protocol's env var — so this asserts both
+ // halves, with the environment pinned (the real one must never reach a probe, and
+ // the suite must not depend on what the developer exports).
+ const savedAnthropic = process.env.ANTHROPIC_API_KEY;
+ const savedOpenai = process.env.OPENAI_API_KEY;
+ try {
+ delete process.env.ANTHROPIC_API_KEY;
+ delete process.env.OPENAI_API_KEY;
+ seen.length = 0;
+ const cleared = await api.post(detectUrl(), {
+ baseUrl: `http://127.0.0.1:${port}`,
+ provider: "custom",
+ modelId: "stored-key-model",
+ clearApiKey: true,
+ });
+ expect(cleared.status).toBe(200);
+ // Route-shape 401s still detect the protocol, but no credential was sent.
+ expect(((await cleared.json()) as ModelProtocolDetectResponse).detected).toBe(
+ "ant-messages",
+ );
+ const keyless = seen.find((c) => c.path === "/v1/messages");
+ expect(keyless?.xApiKey).toBeUndefined();
+ expect(keyless?.auth).toBeUndefined();
+
+ // Same request with the env var present: the stored key stays suppressed, and the
+ // environment — not "sk-stored-secret" — is what authenticates the probe.
+ process.env.ANTHROPIC_API_KEY = "sk-env-anthropic";
+ seen.length = 0;
+ const viaEnv = await api.post(detectUrl(), {
+ baseUrl: `http://127.0.0.1:${port}`,
+ provider: "custom",
+ modelId: "stored-key-model",
+ clearApiKey: true,
+ });
+ expect(viaEnv.status).toBe(200);
+ const envProbe = seen.find((c) => c.path === "/v1/messages");
+ expect(envProbe?.xApiKey).toBe("sk-env-anthropic");
+ expect(envProbe?.auth).toBe("Bearer sk-env-anthropic");
+ } finally {
+ if (savedAnthropic === undefined) delete process.env.ANTHROPIC_API_KEY;
+ else process.env.ANTHROPIC_API_KEY = savedAnthropic;
+ if (savedOpenai === undefined) delete process.env.OPENAI_API_KEY;
+ else process.env.OPENAI_API_KEY = savedOpenai;
+ }
+ } finally {
+ await new Promise<void>((resolve) => server.close(() => resolve()));
+ }
+ });
+});
diff --git a/packages/server/test/vision-detect.test.ts b/packages/server/test/vision-detect.test.ts
new file mode 100644
index 00000000..ffcfa03b
--- /dev/null
+++ b/packages/server/test/vision-detect.test.ts
@@ -0,0 +1,96 @@
+/**
+ * Vision probe classification: the three-valued verdict that decides whether the models
+ * dialog turns "supports vision" on, off, or leaves it alone.
+ *
+ * The distinction under test is the whole point of the feature: a model that ANSWERS "I
+ * cannot take images" has told us something definitive (switch it off), while a probe that
+ * failed on auth or the network has told us nothing (leave the user's setting untouched).
+ * Getting that backwards would silently disable vision on a working model.
+ */
+import { describe, expect, it } from "vitest";
+import type { LLMOutcome } from "@prismshadow/penguin-core";
+import {
+ VISION_PROBE_IMAGE,
+ VISION_PROBE_MAX_TOKENS,
+ classifyVisionProbe,
+ classifyVisionProbeError,
+ isImageRejection,
+} from "../src/services/vision-detect.js";
+
+const failed = (errorMessage: string): LLMOutcome => ({ status: "failed", errorMessage });
+
+describe("VISION_PROBE_IMAGE", () => {
+ it("is a tiny inline PNG data URL, so the probe sends the least it can", () => {
+ expect(VISION_PROBE_IMAGE.startsWith("data:image/png;base64,")).toBe(true);
+ const b64 = VISION_PROBE_IMAGE.slice("data:image/png;base64,".length);
+ const bytes = Buffer.from(b64, "base64");
+ // A real PNG signature, and well under 200 bytes.
+ expect([...bytes.subarray(0, 4)]).toEqual([0x89, 0x50, 0x4e, 0x47]);
+ expect(bytes.length).toBeLessThan(200);
+ // Round-trips, i.e. it is valid base64 rather than a truncated paste.
+ expect(bytes.toString("base64")).toBe(b64);
+ });
+
+ it("caps the probe's output so a success cannot run up a bill", () => {
+ expect(VISION_PROBE_MAX_TOKENS).toBeLessThanOrEqual(16);
+ });
+});
+
+describe("classifyVisionProbe", () => {
+ it("treats a completed answer as vision support", () => {
+ expect(classifyVisionProbe({ status: "completed" }, false)).toBe("supported");
+ });
+
+ it("treats streamed content as support even if the run then trips its tiny cap", () => {
+ // Same allowance the connectivity probe makes: the image was plainly accepted, which
+ // is the only question being asked.
+ expect(classifyVisionProbe(failed("max tokens reached"), true)).toBe("supported");
+ });
+
+ it("reads an image-specific rejection as a definitive no", () => {
+ for (const msg of [
+ "Invalid content type 'image_url' for this model",
+ "This model does not support image input",
+ "400 Bad Request: unsupported image content",
+ "The selected model has no vision capability",
+ "modality not supported: image",
+ ]) {
+ expect(classifyVisionProbe(failed(msg), false)).toBe("unsupported");
+ }
+ });
+
+ it("does NOT read an unrelated failure as a no — the setting must survive it", () => {
+ for (const msg of [
+ "401 Unauthorized: invalid api key",
+ "fetch failed: ECONNREFUSED",
+ "request timed out",
+ "429 rate limit exceeded",
+ "Invalid request: missing parameter 'model'",
+ "500 Internal Server Error",
+ ]) {
+ expect(classifyVisionProbe(failed(msg), false)).toBe("failed");
+ }
+ });
+});
+
+describe("classifyVisionProbeError (the SDK can throw before streaming)", () => {
+ it("keeps the same split for thrown errors", () => {
+ expect(classifyVisionProbeError(new Error("image_url is not supported"))).toBe("unsupported");
+ expect(classifyVisionProbeError(new Error("OPENAI_API_KEY is not set"))).toBe("failed");
+ expect(classifyVisionProbeError("some string failure")).toBe("failed");
+ });
+});
+
+describe("isImageRejection", () => {
+ it("matches case-insensitively", () => {
+ expect(isImageRejection("UNSUPPORTED IMAGE")).toBe(true);
+ expect(isImageRejection("Vision")).toBe(true);
+ });
+
+ it("never fires on a bare validation error with no image concept in it", () => {
+ // The asymmetry that matters: a false positive switches vision off on a working model.
+ expect(isImageRejection("invalid request")).toBe(false);
+ expect(isImageRejection("bad request")).toBe(false);
+ expect(isImageRejection("unauthorized")).toBe(false);
+ });
+});
diff --git a/packages/web/src/api/endpoints.ts b/packages/web/src/api/endpoints.ts
index 0b85bcfa..7f7d3ab1 100644
--- a/packages/web/src/api/endpoints.ts
+++ b/packages/web/src/api/endpoints.ts
@@ -44,10 +44,14 @@ import type {
MemoryFilesResponse,
MemoryOverviewResponse,
MessagesResponse,
+ ModelProtocolDetectRequest,
+ ModelProtocolDetectResponse,
ModelsResponse,
ModelsUpdateRequest,
ModelTestRequest,
ModelTestResponse,
+ ModelVisionDetectRequest,
+ ModelVisionDetectResponse,
PasswordChangeRequest,
PrefsResponse,
ProjectCreateRequest,
@@ -202,6 +206,20 @@ export const testModel = (projectId: string, body: ModelTestRequest) =>
body,
});
+/** Protocol auto-detection for a custom base URL: probes openai-responses → ant-messages → openai-chat and returns the first protocol the endpoint serves. */
+export const detectProtocol = (projectId: string, body: ModelProtocolDetectRequest) =>
+ apiFetch<ModelProtocolDetectResponse>(
+ `/api/projects/${encodeURIComponent(projectId)}/models/detect`,
+ { method: "POST", body },
+ );
+
+/** Vision probe: sends one 1x1 image on this model's credential and reports whether it was accepted (a real, billed completion — unlike the protocol probes). */
+export const detectVision = (projectId: string, body: ModelVisionDetectRequest) =>
+ apiFetch<ModelVisionDetectResponse>(
+ `/api/projects/${encodeURIComponent(projectId)}/models/detect-vision`,
+ { method: "POST", body },
+ );
+
// Vault environment variables (Agent-level) -------------------------------------------------------
export const getVault = (projectId: string, agentId: string) =>
diff --git a/packages/web/src/features/models/models-page.tsx b/packages/web/src/features/models/models-page.tsx
index fb8811ca..f7a245b5 100644
--- a/packages/web/src/features/models/models-page.tsx
+++ b/packages/web/src/features/models/models-page.tsx
@@ -29,14 +29,16 @@
* Saving does a PUT full-table replace (models not present are deleted; an empty apiKey
* means keep the existing value); only the owner can edit.
*/
-import { useCallback, useEffect, useMemo, useState } from "react";
+import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import type {
CredentialInfo,
+ ModelProtocolDetectRequest,
ModelRefDto,
ModelsResponse,
ModelsUpdateRequest,
ModelTestRequest,
ModelUpdateEntry,
+ ModelVisionDetectRequest,
} from "@prismshadow/penguin-server/api";
import * as api from "../../api/endpoints";
import { S } from "../../lib/strings";
@@ -73,6 +75,19 @@ import {
import type { ModelProviderInfo } from "@prismshadow/penguin-core/model-catalog";
import { groupModelRows, isFreeModel, sameModelRef, userProviderInfo } from "./model-grouping";
import { protocolPathForModel } from "./protocol-path";
+import { ProtocolSuffixMenu } from "./protocol-suffix";
+import {
+ DEFAULT_CUSTOM_CLIENT_TYPE,
+ detectableBaseUrl,
+ displayWidthCh,
+ envHintClientType,
+ isCustomLikeGroup,
+ isGenericProtocolClientType,
+ needsProtocolDetectOnSave,
+ protocolForPersist,
+ protocolSelectorValue,
+} from "./protocol-types";
+import type { ProtocolClientType } from "./protocol-types";
import {
isGroupExpanded,
loadExpandedProviders,
@@ -161,11 +176,23 @@ export function decimalOnly(v: string): string {
return i === -1 ? cleaned : cleaned.slice(0, i + 1) + cleaned.slice(i + 1).replace(/\./g, "");
}
-/** Moving an existing model to Custom switches it to the generic OpenAI Chat Completions client. */
+/**
+ * Moving an existing model to Custom keeps a generic protocol client type (protocol
+ * detection / the in-field picker manage those) and otherwise switches to the generic
+ * OpenAI Chat Completions client — an unroutable or vendor-pinned type must not leak
+ * into a custom group.
+ */
export function clientTypeAfterProviderChange(provider: string, current: string): string {
- return provider === "custom" ? "openai-chat" : current;
+ if (provider !== "custom") return current;
+ return current.trim() !== "" && isGenericProtocolClientType(current) ? current : "openai-chat";
}
+/**
+ * Who asked for a detection run. Only the failure wording differs: a manual run reports a
+ * failure, a save-triggered one reports the fallback it is proceeding with.
+ */
+type DetectMode = "manual" | "save";
+
/** Local edit state for one model row (string-typed for form use; parsed uniformly on save). */
export interface RowState {
/**
@@ -197,7 +224,12 @@ export interface RowState {
contextWindow: string;
/** Per-model max output tokens ("" = inherit the Agent setting): caps output per request; user-only, never preset by the catalog. */
maxTokens: string;
- /** AgentHub client protocol: defaults for preset models (auto-routed), "openai-chat" for new custom models; kept as-is, not user-editable. */
+ /**
+ * AgentHub client protocol. Empty for preset models (auto-routed from the model id) and
+ * for a NEW custom model, which starts with nothing selected until the user picks from
+ * the base URL field's suffix or a detection run fills it in; gateway groups start on
+ * their preset pin. Never persisted empty for a custom-like entry — see protocolForPersist.
+ */
clientType: string;
cacheRead: string;
cacheWrite: string;
@@ -304,7 +336,8 @@ export function toRow(m: ModelsResponse["models"][number]): RowState {
return row;
}
-function rowToEntry(row: RowState): ModelUpdateEntry {
+/** Row edit state -> wire entry (exported for unit tests): the single funnel into the config PUT. */
+export function rowToEntry(row: RowState): ModelUpdateEntry {
// provider and modelId are always submitted as separate fields ((provider, modelId) is the entry's unique key, no concatenation).
const entry: ModelUpdateEntry = { provider: row.provider, modelId: row.modelId };
// Rename (either provider or model_id changing is a key change): include the original paired reference so the server
@@ -316,7 +349,10 @@ function rowToEntry(row: RowState): ModelUpdateEntry {
if (row.displayName?.trim()) entry.displayName = row.displayName.trim();
const cw = Number(row.contextWindow.trim());
if (row.contextWindow.trim() && Number.isFinite(cw)) entry.contextWindow = cw;
- if (row.clientType.trim()) entry.clientType = row.clientType.trim();
+ // Never persists an empty protocol for a custom-like entry (that entry could not start —
+ // see protocolForPersist); preset / vendor rows keep "" so AgentHub infers from the id.
+ const clientType = protocolForPersist(row.provider, row.clientType);
+ if (clientType) entry.clientType = clientType;
// Supported by default: submit false only when explicitly marked "unsupported" (preset vision models and checked custom models aren't persisted).
if (!row.vision) entry.vision = false;
// Output cap ("" = inherit the Agent setting): submitted only when filled; omitting clears the stored annotation.
@@ -1095,10 +1131,16 @@ function ModelDialog({
provider: addProvider,
modelId: "",
original: null,
- vision: true,
+ // A new custom model claims no vision support until it is detected or switched on by
+ // hand (per maintainer). Vendor and gateway adds keep the old optimistic default:
+ // their ids are catalog-known, so the capability is already established for them.
+ vision: !isCustomLikeGroup(addProvider),
contextWindow: "",
maxTokens: "",
- clientType: vendorAdd ? "" : "openai-chat",
+ // No protocol is preselected for a custom / user-defined group: it is detected from
+ // the endpoint (on demand, or on save while still unset) or picked by hand. Vendor
+ // groups auto-route by model id, and gateways keep their preset Chat Completions pin.
+ clientType: vendorAdd || isCustomLikeGroup(addProvider) ? "" : "openai-chat",
cacheRead: "",
cacheWrite: "",
output: "",
@@ -1118,6 +1160,30 @@ function ModelDialog({
* (Adding a new custom model isn't confirmed: opening the dialog is itself a clear intent.)
*/
const [confirming, setConfirming] = useState<DialogAction | null>(null);
+ /** Protocol detection in progress (custom / user-defined groups only). */
+ const [detecting, setDetecting] = useState(false);
+ /**
+ * Whether the last detection run came back empty, purely to tint the suffix trigger
+ * amber. That tint is the control's own appearance, not a message occupying the form —
+ * the wording of both outcomes lives in a toast. It is deliberately the only trace left:
+ * a hit needs none, because the suffix then shows the protocol it applied.
+ */
+ const [detectFailed, setDetectFailed] = useState(false);
+ /**
+ * Monotonic run counter: each detection captures it and only the newest run may apply
+ * its result — a manual protocol pick or a re-run supersedes anything still in flight.
+ */
+ const detectSeq = useRef(0);
+ /**
+ * The run currently in flight, so a second trigger joins it instead of starting a rival
+ * probe: clicking Detect while the save path is already probing (or vice versa) must not
+ * double-fire, and the save path needs the SAME run's verdict to decide whether to go on.
+ */
+ const detectInFlight = useRef<Promise<string | null> | null>(null);
+ /** Vision probe in progress (its own control, so its own busy state). */
+ const [visionDetecting, setVisionDetecting] = useState(false);
+ /** Single-flight guard for the vision probe: it bills the user, so never twice at once. */
+ const visionInFlight = useRef<Promise<void> | null>(null);
const isNew = row === null;
const preset = row !== null && isPreset(row);
@@ -1167,6 +1233,163 @@ function ModelDialog({
}
};
+ /**
+ * Protocol detection: POST /models/detect probes the base URL for the three generic
+ * protocols (openai-responses → ant-messages → openai-chat, first hit wins) and applies
+ * the result to the form's clientType. Resolves to the detected client type, or null
+ * when nothing matched / the probe failed, so the save path can decide from the same run
+ * whether it may go on.
+ *
+ * No API key is required. The server resolves the probe credential in three layers — the
+ * key typed here, else this entry's stored key, else the environment variable for
+ * whichever protocol each probe speaks (ANTHROPIC_* / OPENAI_*) — and a keyless probe
+ * still identifies a route that answers in a protocol's own error shape. So the button
+ * is always live and a failure explains itself afterwards instead of being pre-empted by
+ * a disabled control.
+ *
+ * Outcomes are announced with a toast, like the connectivity test in this same dialog:
+ * transient and non-blocking, nothing to dismiss before carrying on. The Toaster portals
+ * to document.body at z-[100] against the Modal's z-50, so a toast raised from inside
+ * this dialog renders above it rather than behind or clipped by it.
+ *
+ * Every failure — unreachable, timeout, gateway junk, nothing matched, an invalid URL
+ * that never left the browser, a server error — raises the SAME message naming the two
+ * things the user can act on (API key, base URL). Per maintainer: the finer distinctions
+ * are invisible from the outside, and wording one of them as "the endpoint responded"
+ * read as success. The per-probe outcomes stay in the endpoint's response for debugging.
+ *
+ * A stale run (superseded by a manual pick or a newer run) discards its result instead
+ * of clobbering the form, and stays silent.
+ */
+ const detectOnce = async (mode: DetectMode): Promise<string | null> => {
+ // What an empty result means depends on who asked. A manual run simply failed; a save
+ // run silently resolves to the compatible client and says so, because the save it was
+ // serving is still going through.
+ const failed = () => {
+ setDetectFailed(true);
+ if (mode === "save") toastInfo(S.models.detectFellBack);
+ else toastError(S.models.detectFailedBody);
+ };
+ const baseUrl = form.baseUrl.trim();
+ // An unusable URL is just another failure: the server would 400 it, and the user gets
+ // the same message either way rather than a distinct piece of API error text.
+ if (!detectableBaseUrl(baseUrl)) {
+ failed();
+ return null;
+ }
+ const seq = ++detectSeq.current;
+ setDetecting(true);
+ setDetectFailed(false);
+ try {
+ const body: ModelProtocolDetectRequest = { baseUrl };
+ const key = form.apiKeyInput.trim();
+ if (key) body.apiKey = key;
+ else if (form.clearApiKey) body.clearApiKey = true;
+ const modelId = form.modelId.trim();
+ if (modelId) {
+ body.provider = form.provider;
+ body.modelId = modelId;
+ }
+ const res = await api.detectProtocol(projectId, body);
+ if (seq !== detectSeq.current) return null;
+ if (res.detected) {
+ set({ clientType: res.detected });
+ return res.detected;
+ }
+ failed();
+ return null;
+ } catch {
+ if (seq === detectSeq.current) failed();
+ return null;
+ } finally {
+ if (seq === detectSeq.current) setDetecting(false);
+ }
+ };
+
+ /**
+ * Single-flight wrapper: a second trigger joins the run already in progress rather than
+ * starting a rival probe (the button while the save path is probing, or vice versa).
+ */
+ const runDetect = (mode: DetectMode): Promise<string | null> => {
+ if (detectInFlight.current) return detectInFlight.current;
+ const run = detectOnce(mode).finally(() => {
+ detectInFlight.current = null;
+ });
+ detectInFlight.current = run;
+ return run;
+ };
+
+ /**
+ * The Detect button. Announces BOTH outcomes: the user asked a question and gets an
+ * answer either way. (The save path calls runDetect directly instead — a hit there needs
+ * no announcement of its own, since the save it was serving carries straight on.)
+ */
+ const detectFromButton = async () => {
+ const detected = await runDetect("manual");
+ if (detected === null) return; // detectOnce already raised the failure toast
+ toastSuccess(S.models.detectedProtocol(S.models.protocolNames[detected] ?? detected));
+ };
+
+ /**
+ * Vision probe. Mirrors protocol detection — always clickable, single-flight, toast-only
+ * — with one deliberate difference: it is a REAL completion on the user's credential
+ * (an image request cannot be shaped to cost nothing the way the protocol probes are), so
+ * it only ever runs from this button, never implicitly and never on save.
+ *
+ * Three outcomes, because "the probe failed" and "the model says no" are different facts:
+ * a hit switches vision ON, a definitive image rejection switches it OFF, and a probe
+ * that learned nothing leaves the switch exactly as the user had it.
+ */
+ const detectVisionFromButton = async () => {
+ if (visionInFlight.current) return;
+ const modelId = form.modelId.trim();
+ if (!modelId) {
+ toastError(S.models.detectVisionNeedsId);
+ return;
+ }
+ setVisionDetecting(true);
+ const run = (async () => {
+ try {
+ const body: ModelVisionDetectRequest = { provider: form.provider, modelId };
+ const key = form.apiKeyInput.trim();
+ if (key) body.apiKey = key;
+ else if (form.clearApiKey) body.clearApiKey = true;
+ const bu = form.baseUrl.trim();
+ body.baseUrl = bu ? bu : null;
+ if (form.clientType.trim()) body.clientType = form.clientType.trim();
+ const res = await api.detectVision(projectId, body);
+ if (res.outcome === "supported") {
+ set({ vision: true });
+ toastSuccess(S.models.detectVisionOk);
+ } else if (res.outcome === "unsupported") {
+ set({ vision: false });
+ toastInfo(S.models.detectVisionNo);
+ } else {
+ toastError(S.models.detectFailedBody);
+ }
+ } catch {
+ toastError(S.models.detectFailedBody);
+ } finally {
+ setVisionDetecting(false);
+ }
+ })();
+ visionInFlight.current = run.finally(() => {
+ visionInFlight.current = null;
+ });
+ await visionInFlight.current;
+ };
+
+ /**
+ * Manual protocol override from the in-field picker. Bumping the run counter supersedes
+ * any in-flight detection, so a late result cannot clobber a choice the user just made.
+ */
+ const pickProtocol = (clientType: ProtocolClientType) => {
+ detectSeq.current++;
+ setDetecting(false);
+ setDetectFailed(false);
+ set({ clientType });
+ };
+
/**
* Validate and convert pricing back to USD storage; returns null on validation failure —
* every error is placed below the offending input, which is highlighted red (no more
@@ -1183,11 +1406,29 @@ function ModelDialog({
form.provider === "custom" ||
providerInfo(form.provider) === undefined;
const baseUrlRequired = !preset && openAiLike;
+ // Custom-like groups (custom + user-defined) pick among AgentHub's generic protocol
+ // clients: the base URL field's suffix becomes the protocol picker there, unless the
+ // entry carries a legacy vendor-pinned client_type — that keeps the read-only note below
+ // instead. Gateways stay pinned to their preset protocol (their base URL is fixed too).
+ const customLikeGroup = form.provider === "custom" || providerInfo(form.provider) === undefined;
+ const showProtocolSelector =
+ customLikeGroup && isGenericProtocolClientType(form.clientType) && !preset;
+ // A viewer without edit rights gets the plain grey suffix: the picker would offer writes
+ // the save path rejects anyway.
+ const showProtocolPicker = showProtocolSelector && canEdit;
// Protocol-path suffix shown inside the base URL field (every model, even while the
// field is empty): the path the client appends to the base URL, i.e. the endpoint
// shape a custom URL must serve. Recomputed from the live form so switching the
// group in add mode updates it.
const protocolPath = protocolPathForModel(form.provider, form.clientType);
+ // Which protocol the picker shows as chosen — null while a fresh custom model has none.
+ const protocolChoice = protocolSelectorValue(form.clientType);
+ // In the picker, an unchosen protocol has no path to show: the field would otherwise
+ // display /chat/completions and read as a decision the user never made. The placeholder
+ // takes its place until a pick or a detection lands. (The read-only suffix used by preset
+ // groups keeps showing the real path — those entries genuinely route that way.)
+ const suffixLabel =
+ showProtocolPicker && protocolChoice === null ? S.models.protocolUnset : protocolPath;
const validated = (): RowState | null => {
const modelId = form.modelId.trim();
@@ -1245,9 +1486,30 @@ function ModelDialog({
};
};
- const submit = (action: DialogAction) => {
+ /**
+ * Commit one dialog action. Saving a custom-like entry whose protocol is still unset
+ * detects it FIRST and continues with whatever comes back: asking the endpoint is more
+ * reliable than guessing, so it is worth the round-trip.
+ *
+ * A probe that finds nothing no longer blocks (per maintainer: 默认都走兼容类型). The save
+ * goes through on the compatible client, with a toast saying that is what happened —
+ * detection is an accuracy improvement, not a gate, and refusing to save left the user
+ * stuck on an endpoint that simply cannot be probed. Nothing unstartable is written
+ * either way, because the fallback is a real client.
+ *
+ * The other actions (set-default / set-vision-proxy / remove) do not probe: they are not
+ * the user saying "this model is ready", and rowToEntry's fallback still applies.
+ */
+ const submit = async (action: DialogAction) => {
const next = validated();
- if (next) onSubmit(next, action);
+ if (!next) return;
+ if (needsProtocolDetectOnSave(action, next.provider, next.clientType)) {
+ // Joins a run already started from the Detect button rather than probing twice.
+ const detected = await runDetect("save");
+ onSubmit({ ...next, clientType: detected ?? DEFAULT_CUSTOM_CLIENT_TYPE }, action);
+ return;
+ }
+ onSubmit(next, action);
};
// Provider info for the current group (updates live as the group dropdown
@@ -1256,12 +1518,17 @@ function ModelDialog({
// and self-defined groups have no link).
const dialogProvider = providerInfo(form.provider);
// env fallback resolves live from the current form (uses the same
- // resolveModelEnv as the server's getModels): explicit
- // client_type takes priority, otherwise auto-route by model_id; no
- // fallback if it can't be routed.
+ // resolveModelEnv as the server's getModels): explicit client_type takes
+ // priority, otherwise auto-route by model_id; no fallback if it can't be routed.
+ //
+ // Custom and user-defined groups opt out of the model_id half (per maintainer): typing
+ // `claude-sonnet-5` into a custom group must not quietly imply the Anthropic client and
+ // its ANTHROPIC_* key. Those groups default to the compatible client, which is also what
+ // gets persisted when nothing is picked or detected — so keying the hint off it is what
+ // the entry will actually read after saving.
const liveEnvKey = resolveModelEnv(
form.modelId.trim(),
- form.clientType.trim() || undefined,
+ envHintClientType(form.provider, form.clientType),
)?.envKey;
// First-party provider group (built-in, non-gateway, non-custom): adding
// goes through auto-routing — show a hint when the id can't be routed
@@ -1403,7 +1670,14 @@ function ModelDialog({
<Modal
open
title={
- isNew ? (vendorGroup ? S.models.addTitleVendor : S.models.addTitle) : S.models.editTitle
+ isNew
+ ? vendorGroup
+ ? S.models.addTitleVendor
+ : customLikeGroup
+ ? // Custom / user-defined groups no longer pin one protocol (detection + selector), so the title drops the "(OpenAI protocol)" suffix gateways keep.
+ S.models.addTitleCustom
+ : S.models.addTitle
+ : S.models.editTitle
}
onClose={onClose}
widthClass="sm:max-w-lg"
@@ -1413,11 +1687,15 @@ function ModelDialog({
{canEdit && (
<Button
variant="primary"
+ // Saving may have to probe the endpoint first (protocol still unset), which
+ // is a network round-trip: the label says so and the button locks, matching
+ // the "test connection" convention used inside this dialog.
+ disabled={detecting}
onClick={() => {
// Validate first: if validation fails, the inline field errors show right away without popping the confirm dialog.
if (!validated()) return;
if (isNew || row === null) {
- submit("save");
+ void submit("save");
return;
}
// Nothing changed: report it instead of confirming a no-op write (the
@@ -1436,7 +1714,7 @@ function ModelDialog({
setConfirming("save");
}}
>
- {S.common.confirm}
+ {detecting ? S.models.detecting : S.common.confirm}
</Button>
)}
</>
@@ -1505,7 +1783,9 @@ function ModelDialog({
<p className="text-xs text-gray-500 dark:text-gray-400">
{vendorGroup && dialogProvider
? S.models.vendorProtocolHint(dialogProvider.label)
- : S.models.addProtocolHint}
+ : customLikeGroup
+ ? S.models.addProtocolHintDetect
+ : S.models.addProtocolHint}
</p>
{identityFields}
</>
@@ -1603,37 +1883,115 @@ function ModelDialog({
)}
{/* 2) base URL (required for custom / user-defined groups and explicit openai protocol — see
- baseUrlRequired). The grey in-field suffix shows the protocol path the client appends
- to the base URL — the endpoint shape a custom URL must serve; it renders for every
- model and stays while the field is empty (hints the shape before typing). Reuses the
- unit-adornment idiom of the context window / max tokens fields below; the error text
- sits outside the relative wrapper (see Input.invalid). */}
- <label className="block">
- <FieldLabel required={baseUrlRequired}>{S.models.baseUrl}</FieldLabel>
- <span className="relative block">
+ baseUrlRequired). The in-field suffix at the right edge shows the protocol path the
+ client appends to the base URL — the endpoint shape a custom URL must serve; it
+ renders for every model and stays while the field is empty (hints the shape before
+ typing). Reuses the unit-adornment idiom of the context window / max tokens fields
+ below; the error text sits outside the relative wrapper (see Input.invalid).
+
+ For custom / user-defined groups that suffix IS the protocol SELECTOR (see
+ protocol-suffix.tsx): the path is one-to-one with the three generic protocol
+ clients, so picking one reuses it instead of taking a form row of its own.
+ Elsewhere (preset groups, read-only viewers) it stays the plain grey label it has
+ always been.
+
+ The detect ACTION sits at this field's top-right instead, next to the label — the
+ same idiom as the API key field's "get API key" link above (per maintainer). It is
+ gated on the API key, and a disabled row buried inside the suffix menu could not
+ say so where the user is looking.
+
+ A <div>, not a <label>: the picker is a <button>, and a label may not contain a
+ second labelable element besides its control — the click would fire the button AND
+ re-focus the input. Same shape as the API key block above; the input carries an
+ aria-label so it stays named. */}
+ <div className="block">
+ {showProtocolPicker ? (
+ <span className="mb-1 flex items-baseline justify-between gap-2">
+ <span className="text-xs font-semibold text-gray-600 dark:text-gray-400">
+ {S.models.baseUrl}
+ {baseUrlRequired && (
+ <span className="ml-0.5 text-red-500 dark:text-red-400" aria-hidden>
+ *
+ </span>
+ )}
+ </span>
+ {/* Always live: no API key is needed (the server falls back to the stored key
+ and then to the protocol's env var), and anything that does go wrong is
+ explained in a popup. `detecting` only guards re-entrancy. */}
+ <button
+ type="button"
+ disabled={detecting}
+ onClick={() => void detectFromButton()}
+ title={S.models.detectProtocolHint}
+ className="flex shrink-0 items-center gap-1 text-xs text-brand-600 underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:text-gray-400 disabled:no-underline dark:text-brand-300 dark:disabled:text-gray-500"
+ >
+ {detecting && (
+ <span
+ aria-hidden
+ className="inline-block h-2.5 w-2.5 shrink-0 animate-spin rounded-full border border-current border-t-transparent"
+ />
+ )}
+ {detecting ? S.models.detecting : S.models.detectProtocol}
+ </button>
+ </span>
+ ) : (
+ <FieldLabel required={baseUrlRequired}>{S.models.baseUrl}</FieldLabel>
+ )}
+ <div className="relative">
<Input
size="sm"
+ aria-label={S.models.baseUrl}
required={baseUrlRequired}
value={form.baseUrl}
disabled={!canEdit}
invalid={Boolean(fieldErrors.baseUrl)}
- onChange={(e) => set({ baseUrl: e.target.value })}
+ // Editing the URL retires the previous run's verdict: it described the old
+ // endpoint, and leaving it up would keep asserting a result for a URL that is
+ // no longer in the field.
+ onChange={(e) => {
+ setDetectFailed(false);
+ set({ baseUrl: e.target.value });
+ }}
+ // Detection on leaving the field (custom / user-defined groups): only a
+ // probeable URL the user actually changed in this dialog triggers a run —
+ // typing never fires requests, and a click-through must not rewrite a working
className="font-mono"
// Reserve room so the typed URL never slides under the suffix. Input and
- // suffix share the same monospace size, so the suffix width is exactly its
- // character count in ch (plus the right offset and a small gap).
- style={{ paddingRight: `calc(${protocolPath.length}ch + 1.25rem)` }}
- // The suffix itself is hover-transparent (pointer-events-none), so the
- // explanation rides on the input's title.
+ // suffix share the same monospace size, so the suffix width is its display
+ // width in ch (CJK placeholder glyphs count double), plus the right offset
+ // and — for the interactive version — its padding, gap and chevron. The
+ // picker never changes width between states, so this holds for all of them.
+ style={{
+ paddingRight: `calc(${displayWidthCh(suffixLabel)}ch + ${showProtocolPicker ? "2.25rem" : "1.25rem"})`,
+ }}
+ // The read-only suffix is hover-transparent (pointer-events-none), so the
+ // explanation rides on the input's title; the picker carries its own.
title={S.models.baseUrlSuffixTitle}
placeholder={preset ? S.models.baseUrlHint : "https://…"}
/>
- <span className="pointer-events-none absolute inset-y-0 right-2 flex items-center font-mono text-xs text-gray-400">
- {protocolPath}
- </span>
- </span>
+ {showProtocolPicker ? (
+ <div className="absolute inset-y-0 right-1 flex items-center">
+ <ProtocolSuffixMenu
+ value={protocolChoice}
+ path={suffixLabel}
+ detecting={detecting}
+ tone={detectFailed ? "warn" : null}
+ onPick={pickProtocol}
+ />
+ </div>
+ ) : (
+ <span className="pointer-events-none absolute inset-y-0 right-2 flex items-center font-mono text-xs text-gray-400">
+ {protocolPath}
+ </span>
+ )}
+ </div>
{fieldErrors.baseUrl && <FieldError>{fieldErrors.baseUrl}</FieldError>}
- </label>
+ {/* No detection verdict is rendered here (per maintainer): a result must not take
+ up room in the form. Both outcomes are toasts, and where the protocol ENDED UP
+ is already visible in the suffix above, which is the thing that actually holds
+ it. Nothing conditional remains below the field, so the idle and post-detection
+ layouts are identical — no reserved height, no shift. */}
+ </div>
{/* 3) Context window + max output tokens side by side (one row): the "Token" unit
sits inside each box as a muted right suffix. Placeholders cannot scroll, so at
@@ -1736,9 +2094,12 @@ function ModelDialog({
{!isNew && identityFields}
{/* Legacy entries carrying a client_type other than the standard openai-chat
(historical config): read-only display. Compared canonically so the deprecated
- bare "openai" spelling (pre-0.4.2 configs) is not flagged as legacy either. */}
+ bare "openai" spelling (pre-0.4.2 configs) is not flagged as legacy either, and
+ skipped when the protocol selector above already represents it (generic protocol
+ types in custom-like groups are editable there). */}
{!isNew &&
!preset &&
+ !showProtocolSelector &&
form.clientType &&
canonicalClientType(form.clientType) !== "openai-chat" && (
<p className="text-xs text-gray-400 dark:text-gray-500">
@@ -1753,19 +2114,43 @@ function ModelDialog({
proxy model (describe_image). */}
{!preset && (
<div>
- <label
- className={`inline-flex items-center gap-2 ${canEdit ? "cursor-pointer" : "cursor-not-allowed"}`}
- >
- <span className="text-xs font-semibold text-gray-600 dark:text-gray-400">
- {S.models.vision}
- </span>
- <Switch
- checked={form.vision}
- disabled={!canEdit}
- onChange={(vision) => set({ vision })}
- aria-label={S.models.vision}
- />
- </label>
+ {/* Detect sits inline after the switch — the protocol control's idiom, but next
+ to the setting it fills in rather than at a field's top-right, since this row
+ has no field to hang off. Always clickable, single-flight, toast-only, like
+ protocol detection; it just costs a real (tiny) completion, so it never runs
+ on its own. */}
+ <div className="flex flex-wrap items-center gap-x-3 gap-y-1">
+ <label
+ className={`inline-flex items-center gap-2 ${canEdit ? "cursor-pointer" : "cursor-not-allowed"}`}
+ >
+ <span className="text-xs font-semibold text-gray-600 dark:text-gray-400">
+ {S.models.vision}
+ </span>
+ <Switch
+ checked={form.vision}
+ disabled={!canEdit}
+ onChange={(vision) => set({ vision })}
+ aria-label={S.models.vision}
+ />
+ </label>
+ {canEdit && (
+ <button
+ type="button"
+ disabled={visionDetecting}
+ onClick={() => void detectVisionFromButton()}
+ title={S.models.detectVisionHint}
+ className="flex shrink-0 items-center gap-1 text-xs text-brand-600 underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:text-gray-400 disabled:no-underline dark:text-brand-300 dark:disabled:text-gray-500"
+ >
+ {visionDetecting && (
+ <span
+ aria-hidden
+ className="inline-block h-2.5 w-2.5 shrink-0 animate-spin rounded-full border border-current border-t-transparent"
+ />
+ )}
+ {visionDetecting ? S.models.detectingVision : S.models.detectVision}
+ </button>
+ )}
+ </div>
{!form.vision && (
<p className="mt-1 text-xs text-gray-400 dark:text-gray-500">
{S.models.visionOffProxyHint}
@@ -1785,7 +2170,7 @@ function ModelDialog({
onConfirm={() => {
const action = confirming;
setConfirming(null);
- submit(action);
+ void submit(action);
}}
>
<p className="text-sm text-gray-700 dark:text-gray-300">
diff --git a/packages/web/src/features/models/protocol-path.ts b/packages/web/src/features/models/protocol-path.ts
index 3573a77a..cf41bc8f 100644
--- a/packages/web/src/features/models/protocol-path.ts
+++ b/packages/web/src/features/models/protocol-path.ts
@@ -19,6 +19,9 @@
* `client_type: "openai-chat"` (gateways, custom and user-defined groups; the bare
* "openai" spelling is a deprecated pre-0.4.2 alias) plus the DeepSeek / GLM / Kimi
* direct clients — POST {base}/chat/completions.
+ *
+ * The three generic protocol clients — `openai-responses`, `ant-messages`,
+ * `openai-chat` — are also what the custom-model protocol detection stores.
*/
/**
@@ -33,6 +36,7 @@ export function protocolPathForModel(provider: string, clientType: string): stri
const t = clientType.trim().toLowerCase();
// The generic protocol clients (agenthub 0.4.2): checked before the substring matches
// below — "openai-responses" also contains "openai" but speaks the Responses API.
+ // Order mirrors AutoLLMClient's routing.
if (t.includes("ant-messages")) return "/v1/messages";
if (t.includes("openai-responses")) return "/responses";
// openai-chat / openai-embedding / the deprecated bare "openai" alias.
diff --git a/packages/web/src/features/models/protocol-suffix.tsx b/packages/web/src/features/models/protocol-suffix.tsx
new file mode 100644
index 00000000..90a94dab
--- /dev/null
+++ b/packages/web/src/features/models/protocol-suffix.tsx
@@ -0,0 +1,147 @@
+/**
+ * The protocol control that lives INSIDE the right edge of the config dialog's base URL
+ * field — it is the grey protocol-path suffix that field always had, promoted from a
+ * passive label into the protocol picker itself.
+ *
+ * The suffix already displayed the very thing this control selects: the path the AgentHub
+ * client appends to the base URL (`/responses` / `/v1/messages` / `/chat/completions`),
+ * which is one-to-one with the three generic protocol clients. So protocol selection
+ * reuses that component rather than occupying a form row of their own: the trigger keeps
+ * showing the live path, and clicking it opens a menu of the three protocols (name as the
+ * row title, the appended path as its description) — the manual override.
+ *
+ * Selection only. The detect ACTION lives at the base URL field's top-right, next to its
+ * label (per maintainer: same placement idiom as the API key field's "get API key" link),
+ * because detection is gated on the API key and a disabled menu row cannot carry that
+ * reason where the user is looking. This menu still *reflects* a run in progress — the
+ * trigger spins, and an inconclusive run tints it amber — since the value being determined
+ * is the one it displays.
+ *
+ * Built on the shared Dropdown primitive in portal mode, which is what makes this safe
+ * inside a Modal at phone widths: the panel is mounted on document.body at fixed viewport
+ * coordinates (no ancestor overflow can clip it), clamped to stay on-screen, flipped up
+ * when there is no room below, capped to the room it does have, and wired into the shared
+ * Escape-layer stack plus arrow-key navigation.
+ *
+ * State is rendered without ever changing the trigger's width, so the in-field padding the
+ * base URL input reserves for it never has to move: "detecting" swaps the chevron for a
+ * spinner of the same box, and the warning state (nothing matched / detection failed) is
+ * colour-only. The message itself goes in the base URL field's own text slot below — see
+ * the call site — so no extra row appears in the happy path.
+ */
+import { useState } from "react";
+import { Dropdown } from "../../components/ui/dropdown";
+import { CheckIcon, ChevronDown } from "../../components/ui/icons";
+import { menuRowClass } from "../../components/ui/field";
+import { S } from "../../lib/strings";
+import { protocolPathForModel } from "./protocol-path";
+import { PROTOCOL_CLIENT_TYPES } from "./protocol-types";
+import type { ProtocolClientType } from "./protocol-types";
+
+/** Whether the last detection run left something to warn about (drives the amber trigger). */
+export type ProtocolDetectTone = "ok" | "warn" | null;
+
+export function ProtocolSuffixMenu({
+ value,
+ path,
+ detecting,
+ tone,
+ onPick,
+}: {
+ /**
+ * Protocol the selector currently represents, or null when none has been chosen yet
+ * (a fresh custom model). Null renders the placeholder label and leaves every menu row
+ * unchecked — nothing may look selected that the user did not select.
+ */
+ value: ProtocolClientType | null;
+ /** Trigger text: the live protocol path, or the "pick one" placeholder while unset. */
+ path: string;
+ /** A detection run is in flight (started from the field's top-right button). */
+ detecting: boolean;
+ /** `warn` paints the trigger amber (nothing matched / detection failed); the wording lives below the field. */
+ tone: ProtocolDetectTone;
+ onPick: (clientType: ProtocolClientType) => void;
+}) {
+ const [open, setOpen] = useState(false);
+ // Unset reads as "not selected" everywhere it is announced, not as a default.
+ const name = value === null ? S.models.protocolUnset : (S.models.protocolNames[value] ?? value);
+ return (
+ <Dropdown
+ open={open}
+ setOpen={setOpen}
+ className="flex"
+ // Right-aligned under the field's right edge, where the trigger sits; Dropdown still
+ // clamps the panel into the viewport, so a phone-width dialog keeps every row on screen.
+ menuClass="w-72 max-w-[calc(100vw-2rem)] origin-top-right"
+ portal={{ direction: "down", align: "right" }}
+ button={
+ <button
+ type="button"
+ aria-haspopup="menu"
+ aria-expanded={open}
+ aria-label={`${S.models.protocol}: ${name}`}
+ title={S.models.protocolTriggerTitle(name)}
+ onClick={() => setOpen((v) => !v)}
+ className={
+ "flex items-center gap-1 rounded px-1 py-0.5 font-mono text-xs transition-colors duration-150 " +
+ "hover:bg-gray-100 focus:outline-none focus:ring-2 focus:ring-gray-400/30 dark:hover:bg-gray-800 " +
+ (tone === "warn"
+ ? "text-amber-600 hover:text-amber-700 dark:text-amber-500 dark:hover:text-amber-400"
+ : "text-gray-400 hover:text-gray-700 dark:hover:text-gray-200")
+ }
+ >
+ <span className="truncate">{path}</span>
+ {/* Same 10px box as the chevron it replaces: switching to the busy state must not
+ resize the trigger, or the input's reserved padding would jump mid-detection. */}
+ {detecting ? (
+ <span
+ aria-hidden
+ className="inline-block h-2.5 w-2.5 shrink-0 animate-spin rounded-full border border-current border-t-transparent"
+ />
+ ) : (
+ <ChevronDown size={10} />
+ )}
+ </button>
+ }
+ >
+ <div role="menu">
+ {PROTOCOL_CLIENT_TYPES.map((t) => {
+ const selected = t === value;
+ return (
+ <button
+ key={t}
+ type="button"
+ role="menuitemradio"
+ aria-checked={selected}
+ onClick={() => {
+ setOpen(false);
+ onPick(t);
+ }}
+ className={`block ${menuRowClass} hover:bg-gray-100 dark:hover:bg-gray-800 ${
+ selected ? "bg-gray-100 dark:bg-gray-800" : ""
+ }`}
+ >
+ <span className="flex items-center justify-between gap-2">
+ <span
+ className={`min-w-0 truncate text-xs ${
+ selected
+ ? "font-medium text-gray-900 dark:text-gray-100"
+ : "text-gray-700 dark:text-gray-300"
+ }`}
+ >
+ {S.models.protocolNames[t] ?? t}
+ </span>
+ {selected && <CheckIcon className="text-gray-500 dark:text-gray-400" />}
+ </span>
+ {/* The path this protocol appends: the same string the trigger shows, so the
+ menu explains what the suffix in the field means. */}
+ <span className="mt-0.5 block font-mono text-[11px] text-gray-500 dark:text-gray-500">
+ {protocolPathForModel("custom", t)}
+ </span>
+ </button>
+ );
+ })}
+ </div>
+ </Dropdown>
+ );
+}
diff --git a/packages/web/src/features/models/protocol-types.ts b/packages/web/src/features/models/protocol-types.ts
new file mode 100644
index 00000000..635f6247
--- /dev/null
+++ b/packages/web/src/features/models/protocol-types.ts
@@ -0,0 +1,155 @@
+/**
+ * The generic protocol client family for custom / user-defined model groups, and the pure
+ * helpers the config dialog composes around it. Kept out of models-page.tsx so the in-field
+ * protocol control (protocol-suffix.tsx) can share them without importing the page back.
+ *
+ * The probing itself is server-side (packages/server/src/services/protocol-detect.ts); these
+ * only decide what the dialog shows and what is worth sending there.
+ */
+import { providerInfo } from "@prismshadow/penguin-core/model-catalog";
+
+/**
+ * AgentHub's generic protocol client types, in detection order (custom / user-defined
+ * groups select among these; see the in-field protocol menu and the /models/detect probes).
+ */
+export const PROTOCOL_CLIENT_TYPES = ["openai-responses", "ant-messages", "openai-chat"] as const;
+export type ProtocolClientType = (typeof PROTOCOL_CLIENT_TYPES)[number];
+
+/**
+ * What a custom / user-defined group falls back to whenever its protocol is undetermined:
+ * the compatible client (OpenAI Chat Completions), which is the broadest of the three.
+ *
+ * Per maintainer, this fallback is unconditional for those groups — nothing is ever
+ * inferred from the model id there, and a detection that comes back empty resolves here
+ * instead of blocking the save. Vendor and gateway groups are unaffected: their entries
+ * are auto-routed by a catalog-known id or pinned by the group's preset.
+ */
+export const DEFAULT_CUSTOM_CLIENT_TYPE = "openai-chat";
+
+/**
+ * Whether a stored client_type belongs to the generic protocol family the picker can
+ * represent: the three protocol clients, the bare `openai` alias (legacy default for
+ * custom groups; routes to openai-chat), or empty. Any other explicit type (a legacy
+ * vendor-pinned config like `deepseek-v4`) keeps the read-only note instead — showing
+ * the picker there would silently rewrite it.
+ */
+export function isGenericProtocolClientType(clientType: string): boolean {
+ const t = clientType.trim().toLowerCase();
+ return t === "" || t === "openai" || (PROTOCOL_CLIENT_TYPES as readonly string[]).includes(t);
+}
+
+/**
+ * Picker value for the current clientType, or **null when nothing is selected yet**.
+ *
+ * The null case is load-bearing: a new custom model starts with no protocol, and the
+ * control has to look that way — no checked row in the menu, no path in the field. Before
+ * this returned "openai-chat" for the empty string, which rendered `/chat/completions` in
+ * the field and a checkmark in the menu, i.e. a default the user never chose and could not
+ * tell apart from one they did.
+ *
+ * A stored legacy `openai` still displays as Chat Completions (that IS its routing) without
+ * rewriting the stored value — only an actual selection or a detection hit writes the
+ * new-style client type.
+ */
+export function protocolSelectorValue(clientType: string): ProtocolClientType | null {
+ const t = clientType.trim().toLowerCase();
+ if (t === "") return null;
+ return t === "openai-responses" || t === "ant-messages" ? t : "openai-chat";
+}
+
+/**
+ * Monospace display width in `ch` units, counting wide (CJK) glyphs as two. The base URL
+ * input reserves right padding for whatever the suffix renders; `.length` is exact for the
+ * ASCII protocol paths but halves the reservation for a localized placeholder, which would
+ * let the typed URL slide under it.
+ */
+export function displayWidthCh(text: string): number {
+ let width = 0;
+ for (const ch of text) width += ch.codePointAt(0)! > 0x2e7f ? 2 : 1;
+ return width;
+}
+
+/** A base URL detection can probe: absolute http(s) (mirrors the server-side check; anything else 400s). */
+export function detectableBaseUrl(baseUrl: string): boolean {
+ try {
+ const u = new URL(baseUrl.trim());
+ return u.protocol === "http:" || u.protocol === "https:";
+ } catch {
+ return false;
+ }
+}
+
+/**
+ * Custom-like group: the entry picks its own protocol from the generic trio, rather than
+ * being auto-routed inside a first-party vendor group or pinned by a gateway preset.
+ * `custom` plus every user-defined group (a provider id the catalog does not know).
+ */
+export function isCustomLikeGroup(provider: string): boolean {
+ return provider === "custom" || providerInfo(provider) === undefined;
+}
+
+/**
+ * The `client_type` actually persisted for a row.
+ *
+ * A custom-like entry must never reach the config with an empty protocol: AgentHub's
+ * AutoLLMClient resolves an unmatched client type by THROWING (`"<type> is not
+ * supported"`) rather than falling back, and a custom model id matches none of its
+ * substring rules — so an entry saved with no protocol is a model that cannot start.
+ * The dialog's save path detects the protocol before submitting; this is the last-resort
+ * net for the paths that do not (set-default, set-vision-proxy, remove), where probing
+ * the endpoint would be the wrong thing to do.
+ *
+ * Preset and vendor-group entries are returned untouched: their model ids ARE routable,
+ * so an empty value there correctly means "let AgentHub infer from the id", and the
+ * empty default must not leak into them as a bogus pin.
+ */
+export function protocolForPersist(provider: string, clientType: string): string {
+ const t = clientType.trim();
+ if (t !== "" || !isCustomLikeGroup(provider)) return t;
+ return DEFAULT_CUSTOM_CLIENT_TYPE;
+}
+
+/**
+ * Whether committing this dialog action must detect the protocol before it may proceed:
+ * the user pressed save/add on a custom-like entry that still has no protocol. Detection
+ * is preferred over guessing here because the endpoint is the authority, and because a
+ * wrong guess surfaces much later as a failing session rather than a failing save.
+ *
+ * Deliberately NOT the other actions (set-default / set-vision-proxy / remove): those are
+ * not the user declaring the model ready, and probing an endpoint as a side effect of
+ * "make this the default" would be surprising. Those paths stay safe through
+ * protocolForPersist instead.
+ */
+export function needsProtocolDetectOnSave(
+ action: string,
+ provider: string,
+ clientType: string,
+): boolean {
+ return action === "save" && isCustomLikeGroup(provider) && clientType.trim() === "";
+}
+
+/**
+ * The client type the dialog's API-key env hint should resolve against.
+ *
+ * `resolveModelEnv` normally falls back to routing by model id when no client type is
+ * given — right for a vendor group, wrong for a custom one: typing `claude-sonnet-5` into
+ * a custom group would then claim the entry reads ANTHROPIC_API_KEY, when nothing about
+ * that group routes by id and the entry will in fact be saved on the compatible client.
+ * Returning the default for custom-like groups keeps the hint honest; undefined elsewhere
+ * preserves the id-based routing those groups genuinely use.
+ */
+export function envHintClientType(provider: string, clientType: string): string | undefined {
+ const t = clientType.trim();
+ if (t !== "") return t;
+ return isCustomLikeGroup(provider) ? DEFAULT_CUSTOM_CLIENT_TYPE : undefined;
+}
+
+/*
+ * Failure classification used to live here, splitting "unreachable" from "the endpoint
+ * answered but serves none of the three". Removed deliberately (per maintainer): the
+ * distinction is invisible to the person configuring a model, and phrasing one branch as
+ * "the endpoint responded" read as success to users. Every failure now shows the same
+ * short message naming the two things they can actually act on — the API key and the base
+ * URL. The per-protocol outcomes are still reported by the detect endpoint, so the detail
+ * remains available for debugging in the network response.
+ */
diff --git a/packages/web/src/lib/strings-en.ts b/packages/web/src/lib/strings-en.ts
index a7528952..dda57f8d 100644
--- a/packages/web/src/lib/strings-en.ts
+++ b/packages/web/src/lib/strings-en.ts
@@ -469,7 +469,7 @@ export const en: Strings = {
addGroup: "Add group",
addGroupTitle: "Add group",
addGroupDesc:
- "User-defined groups share Custom semantics: models use the OpenAI Chat Completions protocol (base URL required; an empty API key reads OPENAI_API_KEY). Groups live on model entries — the group appears once its first model is saved.",
+ "User-defined groups share Custom semantics: the protocol is picked manually or detected from the base URL (base URL required; an empty API key reads the OPENAI_* / ANTHROPIC_* env vars per the chosen protocol). Groups live on model entries — the group appears once its first model is saved.",
groupNameLabel: "Group name",
groupNameHint: "Starts with a lowercase letter / digit; may contain - and _",
groupNameInvalid:
@@ -509,7 +509,36 @@ export const en: Strings = {
"Caps output tokens per request; leave empty to inherit the agent setting — lower it for small-context models",
maxTokensInvalid: "Must be a positive integer",
clientTypeLocked: (t: string): string => `Protocol: ${t} (kept as configured; not editable)`,
+ protocol: "Protocol",
+ protocolNames: {
+ "openai-responses": "OpenAI Responses",
+ "ant-messages": "Anthropic Messages",
+ "openai-chat": "OpenAI Chat Completions",
+ } as Record<string, string | undefined>,
+ protocolTriggerTitle: (name: string): string => `Protocol: ${name}. Click to change it.`,
+ /** Suffix placeholder while no protocol is selected — never a protocol name, so nothing looks pre-chosen. */
+ protocolUnset: "Select protocol",
+ detectProtocol: "Detect",
+ detectProtocolHint: "Probe the base URL and apply the protocol it serves",
+ detecting: "Detecting…",
+ /** Success toast; the protocol itself then shows in the base URL field's suffix. */
+ detectedProtocol: (name: string): string => `Detected ${name}; applied`,
+ /** The ONE failure toast: every mode collapses to it, naming only what the user can act on. */
+ detectFailedBody: "Could not detect the protocol. Please check the API key and the base URL.",
+ /** Save-time detection came back empty: the save proceeds on the compatible client. */
+ detectFellBack: "Protocol not detected; saved as OpenAI Chat Completions",
+ addProtocolHintDetect:
+ "Pick the protocol from the base URL field's suffix (OpenAI Responses / Anthropic Messages / OpenAI Chat Completions), or press Detect to probe the endpoint — saving without one detects it first",
+ addTitleCustom: "Add model",
vision: "Vision support",
+ /** Detect action beside the vision switch. */
+ detectVision: "Detect",
+ detectingVision: "Testing…",
+ detectVisionHint:
+ "Send one tiny test image to see whether this model accepts images (uses your API key)",
+ detectVisionNeedsId: "Fill in the model id first, then detect.",
+ detectVisionOk: "This model accepts images; vision turned on",
+ detectVisionNo: "This model does not accept images; vision left off",
visionOffProxyHint: "Images are read via the vision proxy model",
visionBadge: "Vision",
freeBadge: "Free",
@@ -534,7 +563,8 @@ export const en: Strings = {
clearApiKey: "Clear stored API key",
baseUrl: "Custom base URL",
baseUrlHint: "Leave empty to use the provider default",
- baseUrlSuffixTitle: "The client appends the grey protocol path to the base URL",
+ baseUrlSuffixTitle:
+ "The client appends the protocol path shown at the field's right edge to the base URL",
baseUrlRequired: "A base URL is required",
contextWindowDefaultHint: (n: number): string => `Defaults to ${n} if empty`,
confirmDeleteTitle: "Delete model",
diff --git a/packages/web/src/lib/strings.ts b/packages/web/src/lib/strings.ts
index d12dd76b..8e9b1833 100644
--- a/packages/web/src/lib/strings.ts
+++ b/packages/web/src/lib/strings.ts
@@ -441,7 +441,7 @@ export const zh = {
addGroup: "新增分组",
addGroupTitle: "新增分组",
addGroupDesc:
- "自建分组与 Custom 同语义:组内模型走 OpenAI Chat Completions 兼容协议(base URL 必填,API key 留空读取 OPENAI_API_KEY)。分组由模型条目承载,保存首个模型后即出现。",
+ "自建分组与 Custom 同语义:接口协议可手动选择,也可按 base URL 检测(base URL 必填,API key 留空按所选协议读取 OPENAI_* / ANTHROPIC_* 环境变量)。分组由模型条目承载,保存首个模型后即出现。",
groupNameLabel: "分组名",
groupNameHint: "小写字母 / 数字开头,可含 - 与 _",
groupNameInvalid: "分组名只能用小写字母、数字、- 与 _(首字符为字母或数字),长度不超过 32",
@@ -481,8 +481,41 @@ export const zh = {
"按模型限制单次请求的最大输出 Token 数;留空沿用 Agent 设置,小上下文模型建议调低",
maxTokensInvalid: "必须为正整数",
clientTypeLocked: (t: string): string => `协议:${t}(沿用原配置,不可修改)`,
+ /** Protocol selector (custom / user-defined groups): AgentHub's generic protocol clients. Protocol names are proper nouns, identical in both locales. */
+ protocol: "接口协议",
+ protocolNames: {
+ "openai-responses": "OpenAI Responses",
+ "ant-messages": "Anthropic Messages",
+ "openai-chat": "OpenAI Chat Completions",
+ } as Record<string, string | undefined>,
+ /** Hover title on the in-field protocol picker (the base URL field's right-edge suffix). */
+ protocolTriggerTitle: (name: string): string => `接口协议:${name}。点击可更换。`,
+ /** Suffix placeholder while no protocol is selected — 不显示任何协议名,避免看起来已选好。 */
+ protocolUnset: "选择协议",
+ /** Detect button at the base URL field's top-right. */
+ detectProtocol: "检测协议",
+ /** Hover title on the detect button. */
+ detectProtocolHint: "探测 base URL,采用它实际提供的协议",
+ detecting: "检测中…",
+ /** Success toast;协议本身随后显示在 base URL 输入框的后缀处。 */
+ detectedProtocol: (name: string): string => `检测到 ${name} 协议,已应用`,
+ /** The ONE failure toast: 所有失败情形共用,只讲用户能动手改的两件事。 */
+ detectFailedBody: "无法检测接口协议,请检查 API Key 与 base URL。",
+ /** 保存时检测无结果:按兼容协议继续保存。 */
+ detectFellBack: "未检测到协议,已按 OpenAI Chat Completions 保存",
+ /** Add-dialog note for custom / user-defined groups (protocol selectable): replaces the fixed-OpenAI wording. */
+ addProtocolHintDetect:
+ "可在 base URL 输入框右端的后缀处手动选择接口协议(OpenAI Responses / Anthropic Messages / OpenAI Chat Completions),也可点“检测协议”探测端点;未选协议时保存会先自动检测",
+ addTitleCustom: "新增模型",
/** Switch label only — the dialog carries no explanation text for it (per owner). */
vision: "支持视觉",
+ /** Detect action beside the vision switch. */
+ detectVision: "检测",
+ detectingVision: "测试中…",
+ detectVisionHint: "发送一张极小的测试图片,判断该模型是否接受图片输入(会消耗 API Key 额度)",
+ detectVisionNeedsId: "请先填写模型 id,再进行检测。",
+ detectVisionOk: "该模型接受图片输入,已开启视觉",
+ detectVisionNo: "该模型不接受图片输入,视觉保持关闭",
/** Shown only while the vision switch is OFF: images are then read via the configured vision proxy model (describe_image). */
visionOffProxyHint: "使用视觉代理模型读图",
visionBadge: "视觉",
@@ -509,8 +542,8 @@ export const zh = {
clearApiKey: "清除已存 API key",
baseUrl: "自定义 base URL",
baseUrlHint: "留空使用厂商默认地址",
- /** Hover title for the base URL field: explains the grey in-field suffix (the protocol path the client appends to the base URL). */
- baseUrlSuffixTitle: "客户端会在 base URL 后追加右侧灰色协议路径",
+ /** Hover title for the base URL field: explains the in-field suffix (the protocol path the client appends to the base URL); for custom groups that suffix is also the protocol picker. */
+ baseUrlSuffixTitle: "客户端会在 base URL 后追加字段右侧的协议路径",
baseUrlRequired: "必须填写 base URL",
contextWindowDefaultHint: (n: number): string => `留空按 ${n} 计`,
confirmDeleteTitle: "删除模型",
diff --git a/packages/web/test/protocol-detect.test.ts b/packages/web/test/protocol-detect.test.ts
new file mode 100644
index 00000000..70363af5
--- /dev/null
+++ b/packages/web/test/protocol-detect.test.ts
@@ -0,0 +1,271 @@
+/**
+ * Custom-model protocol detection UI logic: the generic protocol client-type family
+ * (selector visibility / value mapping), when saving must detect first, what the failure
+ * popup explains, the guarantee that no entry is persisted without a protocol, and the
+ * protocol-path suffix for the new client types. The probing itself is server-side (see
+ * the server package's protocol-detect tests); these are the pure helpers the dialog
+ * composes.
+ */
+import { describe, expect, it } from "vitest";
+import { resolveModelEnv } from "@prismshadow/penguin-core/model-catalog";
+import { zh as ZH } from "../src/lib/strings";
+import { en as EN } from "../src/lib/strings-en";
+import { clientTypeAfterProviderChange, rowToEntry } from "../src/features/models/models-page";
+import type { RowState } from "../src/features/models/models-page";
+import {
+ PROTOCOL_CLIENT_TYPES,
+ detectableBaseUrl,
+ displayWidthCh,
+ envHintClientType,
+ isCustomLikeGroup,
+ isGenericProtocolClientType,
+ needsProtocolDetectOnSave,
+ protocolForPersist,
+ protocolSelectorValue,
+} from "../src/features/models/protocol-types";
+import { protocolPathForModel } from "../src/features/models/protocol-path";
+
+describe("PROTOCOL_CLIENT_TYPES", () => {
+ it("lists the three generic clients in the required detection order (responses first, chat completions last)", () => {
+ expect(PROTOCOL_CLIENT_TYPES).toEqual(["openai-responses", "ant-messages", "openai-chat"]);
+ });
+});
+
+describe("isGenericProtocolClientType", () => {
+ it("accepts the protocol trio, the bare openai alias, and empty; rejects vendor-pinned types", () => {
+ expect(isGenericProtocolClientType("openai-responses")).toBe(true);
+ expect(isGenericProtocolClientType("ant-messages")).toBe(true);
+ expect(isGenericProtocolClientType("openai-chat")).toBe(true);
+ expect(isGenericProtocolClientType("openai")).toBe(true);
+ expect(isGenericProtocolClientType("")).toBe(true);
+ expect(isGenericProtocolClientType(" OpenAI-Responses ")).toBe(true);
+ expect(isGenericProtocolClientType("deepseek-v4")).toBe(false);
+ expect(isGenericProtocolClientType("claude-4-8")).toBe(false);
+ expect(isGenericProtocolClientType("minimax-m3")).toBe(false);
+ });
+});
+
+describe("protocolSelectorValue", () => {
+ it("maps openai / unknown to openai-chat for display without rewriting the stored value", () => {
+ expect(protocolSelectorValue("openai-responses")).toBe("openai-responses");
+ expect(protocolSelectorValue("ant-messages")).toBe("ant-messages");
+ expect(protocolSelectorValue("openai-chat")).toBe("openai-chat");
+ expect(protocolSelectorValue("openai")).toBe("openai-chat");
+ });
+
+ it("reports NOTHING selected for an unset protocol, so the control cannot imply a default", () => {
+ // A fresh custom model: no checked row in the menu, no path in the field. Returning
+ // "openai-chat" here (as it once did) rendered a choice the user never made.
+ expect(protocolSelectorValue("")).toBeNull();
+ expect(protocolSelectorValue(" ")).toBeNull();
+ });
+});
+
+describe("displayWidthCh (padding reserved for the in-field suffix)", () => {
+ it("counts ASCII as one column, so the protocol paths reserve exactly their length", () => {
+ expect(displayWidthCh("/chat/completions")).toBe(17);
+ expect(displayWidthCh("/responses")).toBe(10);
+ expect(displayWidthCh("Select protocol")).toBe(15);
+ });
+
+ it("counts CJK as two, so a localized placeholder is not under-reserved by half", () => {
+ expect(displayWidthCh("选择协议")).toBe(8);
+ expect(displayWidthCh("")).toBe(0);
+ });
+});
+
+describe("detectableBaseUrl", () => {
+ it("only absolute http(s) URLs are probeable (mirrors the server-side validation)", () => {
+ expect(detectableBaseUrl("https://api.example.com/v1")).toBe(true);
+ expect(detectableBaseUrl(" http://127.0.0.1:8000/v1 ")).toBe(true);
+ expect(detectableBaseUrl("api.example.com/v1")).toBe(false);
+ expect(detectableBaseUrl("ftp://example.com")).toBe(false);
+ expect(detectableBaseUrl("")).toBe(false);
+ expect(detectableBaseUrl("https://")).toBe(false);
+ });
+});
+
+describe("clientTypeAfterProviderChange (protocol family kept on move to Custom)", () => {
+ it("keeps generic protocol client types when moving to Custom", () => {
+ expect(clientTypeAfterProviderChange("custom", "openai-responses")).toBe("openai-responses");
+ expect(clientTypeAfterProviderChange("custom", "ant-messages")).toBe("ant-messages");
+ expect(clientTypeAfterProviderChange("custom", "openai-chat")).toBe("openai-chat");
+ expect(clientTypeAfterProviderChange("custom", "openai")).toBe("openai");
+ });
+
+ it("still pins vendor-specific or empty types to openai-chat when moving to Custom, and never touches other groups", () => {
+ expect(clientTypeAfterProviderChange("custom", "")).toBe("openai-chat");
+ expect(clientTypeAfterProviderChange("custom", "claude-5")).toBe("openai-chat");
+ expect(clientTypeAfterProviderChange("google", "openai")).toBe("openai");
+ expect(clientTypeAfterProviderChange("my-group", "ant-messages")).toBe("ant-messages");
+ });
+});
+
+describe("isCustomLikeGroup", () => {
+ it("covers custom and every unknown (user-defined) group, but no catalog vendor group", () => {
+ expect(isCustomLikeGroup("custom")).toBe(true);
+ expect(isCustomLikeGroup("my-group")).toBe(true);
+ expect(isCustomLikeGroup("openai")).toBe(false);
+ expect(isCustomLikeGroup("anthropic")).toBe(false);
+ });
+});
+
+describe("needsProtocolDetectOnSave (save detects a still-unset protocol first)", () => {
+ it("fires for saving a custom-like entry with no protocol yet", () => {
+ expect(needsProtocolDetectOnSave("save", "custom", "")).toBe(true);
+ expect(needsProtocolDetectOnSave("save", "my-group", " ")).toBe(true);
+ });
+
+ it("does not fire once a protocol is chosen", () => {
+ expect(needsProtocolDetectOnSave("save", "custom", "ant-messages")).toBe(false);
+ expect(needsProtocolDetectOnSave("save", "custom", "openai-chat")).toBe(false);
+ });
+
+ it("never probes for vendor groups, nor for actions other than save", () => {
+ // A vendor group auto-routes by model id; an empty protocol there is correct.
+ expect(needsProtocolDetectOnSave("save", "openai", "")).toBe(false);
+ expect(needsProtocolDetectOnSave("remove", "custom", "")).toBe(false);
+ expect(needsProtocolDetectOnSave("setDefault", "custom", "")).toBe(false);
+ expect(needsProtocolDetectOnSave("setVisionModel", "custom", "")).toBe(false);
+ });
+});
+
+describe("envHintClientType (custom groups never infer a client from the model id)", () => {
+ const envFor = (provider: string, modelId: string, clientType: string) =>
+ resolveModelEnv(modelId, envHintClientType(provider, clientType))?.envKey;
+
+ it("keeps a vendor-looking model id in a custom group on the compatible client's env var", () => {
+ // The bug this pins: `claude-sonnet-5` typed into a custom group used to resolve to
+ // ANTHROPIC_API_KEY, implying an Anthropic client that group never routes to.
+ expect(envFor("custom", "claude-sonnet-5", "")).toBe("OPENAI_API_KEY");
+ expect(envFor("custom", "gpt-5.6", "")).toBe("OPENAI_API_KEY");
+ expect(envFor("my-group", "claude-sonnet-5", "")).toBe("OPENAI_API_KEY");
+ });
+
+ it("still honours a protocol the user actually chose", () => {
+ expect(envFor("custom", "whatever", "ant-messages")).toBe("ANTHROPIC_API_KEY");
+ expect(envFor("custom", "whatever", "openai-responses")).toBe("OPENAI_API_KEY");
+ });
+
+ it("leaves vendor groups routing by model id, which is how they really work", () => {
+ expect(envHintClientType("anthropic", "")).toBeUndefined();
+ expect(envFor("anthropic", "claude-sonnet-5", "")).toBe("ANTHROPIC_API_KEY");
+ expect(envFor("google", "gemini-3.1-pro", "")).toBe("GEMINI_API_KEY");
+ });
+});
+
+describe("protocolForPersist (an empty protocol must never reach the config)", () => {
+ it("falls back to openai-chat for a custom-like entry that still has none", () => {
+ // AutoLLMClient THROWS on an unmatched client type, so persisting "" would save a
+ // model that cannot start.
+ expect(protocolForPersist("custom", "")).toBe("openai-chat");
+ expect(protocolForPersist("my-group", " ")).toBe("openai-chat");
+ });
+
+ it("keeps an explicit protocol exactly as chosen", () => {
+ expect(protocolForPersist("custom", "ant-messages")).toBe("ant-messages");
+ expect(protocolForPersist("custom", "openai-responses")).toBe("openai-responses");
+ expect(protocolForPersist("my-group", " openai-chat ")).toBe("openai-chat");
+ });
+
+ it("leaves preset / vendor groups empty so AgentHub still infers from the model id", () => {
+ expect(protocolForPersist("openai", "")).toBe("");
+ expect(protocolForPersist("anthropic", "")).toBe("");
+ expect(protocolForPersist("google", "")).toBe("");
+ });
+});
+
+describe("rowToEntry (the persistence funnel)", () => {
+ const row = (over: Partial<RowState>): RowState => ({
+ provider: "custom",
+ modelId: "my-model",
+ original: null,
+ vision: true,
+ contextWindow: "",
+ maxTokens: "",
+ clientType: "",
+ cacheRead: "",
+ cacheWrite: "",
+ output: "",
+ baseUrl: "",
+ originalBaseUrl: "",
+ apiKeyInput: "",
+ clearApiKey: false,
+ ...over,
+ });
+
+ it("never writes a custom entry without a protocol, even when the form left it empty", () => {
+ expect(rowToEntry(row({})).clientType).toBe("openai-chat");
+ expect(rowToEntry(row({ provider: "my-group" })).clientType).toBe("openai-chat");
+ });
+
+ it("writes the chosen protocol verbatim", () => {
+ expect(rowToEntry(row({ clientType: "ant-messages" })).clientType).toBe("ant-messages");
+ });
+
+ it("omits clientType for a vendor group so AgentHub keeps inferring it", () => {
+ expect(rowToEntry(row({ provider: "openai", modelId: "gpt-5.6" })).clientType).toBeUndefined();
+ });
+});
+
+describe("detection copy", () => {
+ it("has exactly one user-facing failure message, in both locales", () => {
+ // Every failure mode collapses to this: the maintainer's point is that a user cannot
+ // act on "the endpoint responded but serves no protocol", only on the key and the URL.
+ for (const catalog of [EN.models, ZH.models] as const) {
+ expect(catalog.detectFailedBody).toBeTruthy();
+ // No protocol names to parse, and no "pick one manually" instruction.
+ expect(catalog.detectFailedBody).not.toContain("OpenAI Responses");
+ expect(catalog.detectFailedBody).not.toContain("Anthropic Messages");
+ }
+ // The distinguishing strings are gone, not merely unused.
+ expect("detectNone" in EN.models).toBe(false);
+ expect("detectUnreachable" in EN.models).toBe(false);
+ expect("detectNone" in ZH.models).toBe(false);
+ expect("detectUnreachable" in ZH.models).toBe(false);
+ });
+
+ it("names the unset protocol as a placeholder rather than a protocol", () => {
+ for (const catalog of [EN.models, ZH.models] as const) {
+ expect(catalog.protocolUnset).toBeTruthy();
+ expect(Object.values(catalog.protocolNames)).not.toContain(catalog.protocolUnset);
+ }
+ });
+
+ it("carries no dialog-era copy: both outcomes are toasts, so no popup titles remain", () => {
+ // The verdict used to be a blocking AlertModal (which needed an accessible name) and,
+ // before that, a line rendered under the base URL field. Both are gone: a detection
+ // result is transient and must occupy no space in the form.
+ for (const catalog of [EN.models, ZH.models] as const) {
+ expect("detectOkTitle" in catalog).toBe(false);
+ expect("detectFailedTitle" in catalog).toBe(false);
+ }
+ });
+
+ it("keeps both toast messages to a single short line", () => {
+ for (const catalog of [EN.models, ZH.models] as const) {
+ const success = catalog.detectedProtocol("OpenAI Responses");
+ for (const text of [success, catalog.detectFailedBody]) {
+ expect(text.length).toBeLessThanOrEqual(80);
+ expect(text).not.toContain("\n");
+ }
+ }
+ });
+});
+
+describe("protocolPathForModel (generic protocol client types)", () => {
+ it("maps each protocol client to the path its AgentHub client appends", () => {
+ expect(protocolPathForModel("custom", "openai-responses")).toBe("/responses");
+ expect(protocolPathForModel("custom", "ant-messages")).toBe("/v1/messages");
+ expect(protocolPathForModel("custom", "openai-chat")).toBe("/chat/completions");
+ // The bare alias and user-defined groups behave the same as before.
+ expect(protocolPathForModel("my-group", "openai")).toBe("/chat/completions");
+ });
+
+ it("keeps the legacy explicit-type and group fallbacks intact", () => {
+ expect(protocolPathForModel("custom", "claude-4-8")).toBe("/v1/messages");
+ expect(protocolPathForModel("openai", "")).toBe("/responses");
+ expect(protocolPathForModel("anthropic", "")).toBe("/v1/messages");
+ expect(protocolPathForModel("custom", "")).toBe("/chat/completions");
+ });
+});