Skip to content

A local-first multi-agent app development platform, built on PenguinHarness

A vLLM provider group in the model catalog (#606)

Yaowei Zheng39ce630930e4
diff --git a/changelog/unreleased/2026-09-03-vllm-model-group.md b/changelog/unreleased/2026-09-03-vllm-model-group.md
new file mode 100644
index 00000000..3b2b7a90
--- /dev/null
+++ b/changelog/unreleased/2026-09-03-vllm-model-group.md
@@ -0,0 +1,55 @@
+# A vLLM group in the model catalog, with its protocol pinned by the group
+
+- **Date:** 2026-09-03
+- **Type:** feature
+- **Scope:** `core`, `server`, `web`, `cli`
+
+[中文版](2026-09-03-vllm-model-group.zh.md)
+
+The model catalog gains a **vLLM** group, placed immediately before Custom, holding the eight
+models AgentHub's `openai-chat-vllm-adapter` client carries a per-model thinking switch for:
+`Qwen/Qwen3.8-Flash-Next`, `Qwen/Qwen3.8-27B`, `Qwen/Qwen3.6-35B-A3B`, `Qwen/Qwen3.5-0.8B`,
+`Qwen/Qwen3.5-9B`, `deepseek-ai/DeepSeek-V4-Pro`, `deepseek-ai/DeepSeek-V4-Flash` and
+`deepseek-ai/DeepSeek-V4-Flash-Vision-Exp`.
+
+These are self-hosted, which is what the group's shape follows from. The rows price at
+**zero** — nobody bills per token for a server the user runs, and what it does cost is that
+operator's own hardware, which no catalog rate expresses. Zero here is a real rate, not a
+missing one: the rows show the free badge and contribute 0 to the cost center, the same
+treatment the `:free` gateway rows already get. They carry **no base URL**: every deployment has its own, so the user supplies it as in the Custom
+group. Context windows are each model's native length as documented at `recipes.vllm.ai`
+(262,144 for the Qwen rows, 1,000,000 for the DeepSeek V4 rows); a deployment started with a
+smaller `--max-model-len` serves less, and the entry's context window is editable.
+
+The group's glyph is vLLM's official mark, taken from the project's media kit and flattened
+from its two brand colours to `currentColor` the way Qwen's gradient wordmark already is —
+the provider glyphs are one monochrome family, so a row of them reads as one.
+
+## The protocol is a property of the group
+
+`ModelProviderInfo` gains a `clientType` field: the protocol every entry in a group speaks,
+models the user adds included. vLLM is the only group that declares one, and reading it goes
+through `providerClientType` so the places that decide a saved model's protocol answer as one:
+
+- adding a model to the group preselects `openai-chat-vllm-adapter` rather than the generic
+ `openai-chat`, and the dialog names the protocol instead of offering a picker;
+- moving an existing entry into the group rewrites it to the pin;
+- the last-resort protocol on the save paths that do not probe (set-default, set-vision-proxy,
+ remove) is the pin, and protocol detection is skipped entirely — the group already knows;
+- the API-key env hint resolves against the pin, so `deepseek-ai/DeepSeek-V4-Pro` in this group
+ reports `OPENAI_API_KEY` and not the DeepSeek variable its id would otherwise route to;
+- `penguin config model add --provider vllm` defaults a new entry to the pin.
+
+The pin is load-bearing on the preset rows too: `Qwen/*` matches none of AgentHub's routing
+rules and would be rejected outright, while `deepseek-ai/DeepSeek-V4-*` contains `deepseek-v4`
+and would otherwise reach DeepSeek's first-party client, pointed at a vLLM server.
+
+## The AgentHub dependency moves to 0.4.10
+
+`openai-chat-vllm-adapter` first ships in `@prismshadow/agenthub` 0.4.10 (agenthub
+[#197](https://github.com/Prism-Shadow/agenthub/pull/197),
+[#198](https://github.com/Prism-Shadow/agenthub/pull/198)), so `packages/core` and
+`packages/cli` move from `^0.4.9` to `^0.4.10`. On 0.4.9 a model in this group would have
+failed at request time with AgentHub's `"openai-chat-vllm-adapter is not supported"`, because
+the client type did not exist there. The `minimumReleaseAgeExclude` entry in
+`pnpm-workspace.yaml` moves to the new version, as its comment prescribes.
diff --git a/changelog/unreleased/2026-09-03-vllm-model-group.zh.md b/changelog/unreleased/2026-09-03-vllm-model-group.zh.md
new file mode 100644
index 00000000..c28bc5b6
--- /dev/null
+++ b/changelog/unreleased/2026-09-03-vllm-model-group.zh.md
@@ -0,0 +1,47 @@
+# 模型目录新增 vLLM 分组,协议由分组钉死
+
+- **Date:** 2026-09-03
+- **Type:** feature
+- **Scope:** `core`, `server`, `web`, `cli`
+
+[English](2026-09-03-vllm-model-group.md)
+
+模型目录新增 **vLLM** 分组,位置紧挨在 Custom 之前,收录 AgentHub `openai-chat-vllm-adapter` 客户端为其
+逐模型映射思考开关的八个模型:`Qwen/Qwen3.8-Flash-Next`、`Qwen/Qwen3.8-27B`、
+`Qwen/Qwen3.6-35B-A3B`、`Qwen/Qwen3.5-0.8B`、`Qwen/Qwen3.5-9B`、`deepseek-ai/DeepSeek-V4-Pro`、
+`deepseek-ai/DeepSeek-V4-Flash` 与 `deepseek-ai/DeepSeek-V4-Flash-Vision-Exp`。
+
+它们都是自托管模型,分组的形态由此而来。各行**价格全为 0**——服务由用户自己跑,没有谁按 token 收费,
+真正的开销是运维者自己的硬件,那不是任何牌价能表达的。这里的 0 是真实费率而非缺失:这些行会显示免费
+徽标,并在成本中心计为 0,与既有的 `:free` 网关行同一口径。也**不预置 base URL**:每个部署各有各的地址,
+与 Custom 分组一样由用户填写。上下文窗口取 `recipes.vllm.ai` 记载的各模型原生长度(Qwen 各行
+262,144,DeepSeek V4 各行 1,000,000);以更小的 `--max-model-len` 启动的部署实际供给更少,条目的
+上下文窗口可改。
+
+分组图标取 vLLM 官方标识(来自项目的 media kit),并像既有的 Qwen 渐变字标一样,把它的两种品牌色
+压成 `currentColor`——各家 provider 图标是同一族单色标,一行看过去才是一套。
+
+## 协议是分组的属性
+
+`ModelProviderInfo` 新增 `clientType` 字段:该分组下每个条目所用的协议,用户自行新增的模型同样适用。
+目前只有 vLLM 声明它,读取一律经 `providerClientType`,使各处对「这个模型该落哪个协议」给出同一个答案:
+
+- 在该分组新增模型预选 `openai-chat-vllm-adapter` 而非通用的 `openai-chat`,弹窗直接点名协议,不再给出选择器;
+- 把既有条目移入该分组即改写为这一协议;
+- 不做探测的那几条保存路径(设为默认、设为视觉代理、删除)的兜底协议就是它,协议检测整个跳过——分组已经知道答案;
+- API key 的环境变量提示按它解析,因此该分组下的 `deepseek-ai/DeepSeek-V4-Pro` 报 `OPENAI_API_KEY`,
+ 而不是其 id 本会路由到的 DeepSeek 变量;
+- `penguin config model add --provider vllm` 为新条目缺省落这一协议。
+
+预置行同样依赖这个钉子:`Qwen/*` 匹配不上 AgentHub 的任何路由规则、会被直接拒绝,而
+`deepseek-ai/DeepSeek-V4-*` 含有 `deepseek-v4`,否则会走到 DeepSeek 的一方客户端上——却指着一台
+vLLM 服务。
+
+## AgentHub 依赖升至 0.4.10
+
+`openai-chat-vllm-adapter` 首次随 `@prismshadow/agenthub` 0.4.10 发布(agenthub
+[#197](https://github.com/Prism-Shadow/agenthub/pull/197)、
+[#198](https://github.com/Prism-Shadow/agenthub/pull/198)),因此 `packages/core` 与
+`packages/cli` 的依赖从 `^0.4.9` 升到 `^0.4.10`。在 0.4.9 上,该客户端类型并不存在,本分组下的模型
+会在发起请求时以 AgentHub 的 `"openai-chat-vllm-adapter is not supported"` 失败。
+`pnpm-workspace.yaml` 中的 `minimumReleaseAgeExclude` 条目按其注释的约定移到新版本。
diff --git a/packages/cli/package.json b/packages/cli/package.json
index b55e606b..90d86283 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -36,7 +36,7 @@
"@penguinharness/use-bento-slides": "workspace:*",
"@penguinharness/use-firecrawl": "workspace:*",
"@penguinharness/use-claude-code": "workspace:*",
- "@prismshadow/agenthub": "^0.4.9",
+ "@prismshadow/agenthub": "^0.4.10",
"@prismshadow/penguin-core": "workspace:*",
"@prismshadow/penguin-server": "workspace:*",
"commander": "^13.0.0",
diff --git a/packages/cli/src/commands/config.ts b/packages/cli/src/commands/config.ts
index 02618ae1..2edc1d5e 100644
--- a/packages/cli/src/commands/config.ts
+++ b/packages/cli/src/commands/config.ts
@@ -45,6 +45,7 @@ import {
getModel,
loadAgentVault,
loadProjectConfig,
+ providerClientType,
providerInfo,
removeModel,
removeVaultEntry,
@@ -155,16 +156,19 @@ export function registerConfigCommand(program: Command, t: Messages): void {
const before = await loadProjectConfig(root, opts.projectId);
const existed = getModel(before, ref) !== undefined;
// client_type default rule, only injected for new entries (updating an
- // existing entry never overrides an explicit config): not set for first-party
- // vendor groups (AgentHub auto-routes by upstream id, with env fallback keyed on
- // id); defaults to openai-chat for custom / self-hosted / gateway groups, with the
- // gateway's endpoint base URL pre-filled as well. (An explicit --client-type is
- // passed through; core's addModel normalizes the deprecated bare "openai" alias.)
+ // existing entry never overrides an explicit config): a group that pins a protocol
+ // (vLLM) gets that pin, whatever the id; otherwise not set for first-party vendor
+ // groups (AgentHub auto-routes by upstream id, with env fallback keyed on id), and
+ // openai-chat for custom / user-defined / gateway groups, with the gateway's endpoint
+ // base URL pre-filled as well. (An explicit --client-type is passed through; core's
+ // addModel normalizes the deprecated bare "openai" alias.)
const pInfo = providerInfo(provider);
const openAiDefault =
pInfo === undefined || pInfo.id === "custom" || pInfo.gatewayBaseUrl !== undefined;
+ const groupClientType = providerClientType(provider);
+ const defaultClientType = groupClientType ?? (openAiDefault ? "openai-chat" : undefined);
const clientType: string | undefined =
- opts.clientType ?? (!existed && openAiDefault ? "openai-chat" : undefined);
+ opts.clientType ?? (!existed ? defaultClientType : undefined);
const baseUrl: string | undefined =
opts.baseUrl ?? (!existed ? pInfo?.gatewayBaseUrl : undefined);
// Only collect explicitly given price fields, letting addModel merge them with the existing pricing per-field.
diff --git a/packages/core/package.json b/packages/core/package.json
index 5fa15fe3..1253b5d7 100644
--- a/packages/core/package.json
+++ b/packages/core/package.json
@@ -65,7 +65,7 @@
"@penguinharness/use-bento-slides": "workspace:*",
"@penguinharness/use-firecrawl": "workspace:*",
"@penguinharness/use-claude-code": "workspace:*",
- "@prismshadow/agenthub": "^0.4.9",
+ "@prismshadow/agenthub": "^0.4.10",
"arktype": "^2.2.3",
"smol-toml": "^1.3.0",
"yaml": "^2.5.0"
diff --git a/packages/core/src/state/model-catalog.ts b/packages/core/src/state/model-catalog.ts
index df1260b5..ac362b28 100644
--- a/packages/core/src/state/model-catalog.ts
+++ b/packages/core/src/state/model-catalog.ts
@@ -7,8 +7,8 @@
* 2026-08-21; the TokenDance group: 2026-08-25, its glm-5.3-flash row: 2026-08-26, its
* qwen3.8-flash row: 2026-08-27 and its running promotions plus the hy4-preview rows
* (TokenDance + OpenRouter): 2026-08-28; the GLM-5.3 Flash rows (direct + OpenRouter) and
- * the direct qwen3.8-flash: 2026-08-26; the GPT-6 Astra rows (direct + OpenRouter):
- * 2026-09-09 — per each provider's docs).
+ * the direct qwen3.8-flash: 2026-08-26; the vLLM group: 2026-09-03; the GPT-6 Astra rows
+ * (direct + OpenRouter): 2026-09-09 — per each provider's docs).
* Docs: packages/docs/content/models.{zh,en}.md (site path /docs/models) documents the
* provider groups and credential resolution described here.
*
@@ -32,7 +32,10 @@
* auto-routed by AgentHub and leave client_type unset; the six gateway groups (OpenRouter,
* Fireworks AI, SiliconFlow, TokenDance, Qwen Pay-As-You-Go, Qwen Token Plan) can't be
* auto-routed, so every gateway row **always pins an explicit client_type** and inlines its
- * preset base URL.
+ * preset base URL. The vLLM group pins one too, but as a property of the GROUP rather than
+ * of each row (ModelProviderInfo.clientType, read through providerClientType): a model the
+ * user adds there speaks the same protocol as the presets, and has no preset base URL to
+ * inherit — see the group's own block comment.
* That pin is load-bearing, not decoration: AgentHub's AutoLLMClient matches raw substrings
* against `client_type || model_id` and never looks at base_url, so an unpinned gateway id
* would be routed by its own spelling — `openai/gpt-5.6-sol` would reach the first-party
@@ -42,7 +45,10 @@
* Completions client — the bare "openai" spelling is a deprecated upstream alias, see
* canonicalClientType);
* - `openai-responses` for the OpenRouter `openai/*` rows, whose upstream really is an
- * OpenAI Responses server (see the OpenRouter block comment for why only those rows).
+ * OpenAI Responses server (see the OpenRouter block comment for why only those rows);
+ * - `openai-chat-vllm-adapter` for the vLLM group, which is Chat Completions on the wire
+ * but maps the thinking level onto the served model's own chat template
+ * (VLLM_CLIENT_TYPE).
* The MiniMax M3 preset pins AgentHub's first-party `minimax-m3` protocol and direct API
* endpoint.
*
@@ -93,6 +99,22 @@ export interface ModelProviderInfo {
* every provider whose keys are only obtainable from its console.
*/
oauth?: ModelProviderOAuth;
+ /**
+ * The AgentHub protocol EVERY entry in this group speaks, models the user adds included.
+ *
+ * Set it only where the group itself decides the answer and no other property already
+ * implies it: the gateways derive `openai-chat` from carrying a `gatewayBaseUrl`, and
+ * `custom` / user-defined groups deliberately declare nothing — their whole point is that
+ * the protocol is detected from the endpoint or picked by hand, and a pin here would
+ * take that choice away.
+ *
+ * Where it IS set, it outranks every group-shape guess in the app: the add-model dialog
+ * preselects it, moving an entry into the group rewrites the entry to it, the API-key env
+ * hint resolves against it, and protocol detection is skipped because the group already
+ * knows. Read it through providerClientType rather than reaching for the field, so those
+ * call sites keep answering as one.
+ */
+ clientType?: string;
/**
* The group the product recommends, captioned as such on the models page. It marks the
* GROUP, not a position: a user who drags the group elsewhere keeps the caption with it,
@@ -137,6 +159,15 @@ export interface ModelCatalogEntry {
baseUrl?: string;
}
+/**
+ * AgentHub's client for models served by vLLM's OpenAI-compatible Chat Completions API. It
+ * is Chat Completions on the wire, but a distinct client: it maps the thinking level onto
+ * the `chat_template_kwargs` the SERVED model's chat template reads, which differs per model
+ * family, and AgentHub matches this name by exact equality (before its `openai` substring
+ * branches) so `openai-chat` would silently lose that mapping.
+ */
+const VLLM_CLIENT_TYPE = "openai-chat-vllm-adapter";
+
/** Preset provider endpoints; only OpenAI-compatible gateways expose theirs as gatewayBaseUrl. */
const OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1";
const SILICONFLOW_BASE_URL = "https://api.siliconflow.cn/v1";
@@ -285,6 +316,17 @@ export const MODEL_PROVIDERS: ModelProviderInfo[] = [
"https://platform.qianwenai.com/docs/token-plan/personal/token-plan-personal-overview",
gatewayBaseUrl: QWEN_TOKEN_PLAN_BASE_URL,
},
+ {
+ // Self-hosted: the user runs the server, so there is no console to mint a key at
+ // (apiKeyUrl) and no endpoint to preset (gatewayBaseUrl). modelsUrl points at the recipe
+ // index, which is where the served ids in this group are documented.
+ id: "vllm",
+ label: "vLLM",
+ envKey: "OPENAI_API_KEY",
+ envBaseUrlKey: "OPENAI_BASE_URL",
+ modelsUrl: "https://recipes.vllm.ai/",
+ clientType: VLLM_CLIENT_TYPE,
+ },
{ id: "custom", label: "Custom", envKey: "OPENAI_API_KEY", envBaseUrlKey: "OPENAI_BASE_URL" },
];
@@ -1694,6 +1736,104 @@ export const MODEL_CATALOG: ModelCatalogEntry[] = [
pricing: cny(0.7, 4, 21),
supportsVision: true,
},
+ // -- vLLM (self-hosted: the models AgentHub's openai-chat-vllm-adapter client carries a
+ // per-model thinking switch for, as published at recipes.vllm.ai — read 2026-09-03).
+ //
+ // Every row prices at zero, and omits two other things, all because the user runs the server:
+ // - **zero pricing**. There is no seller charging per token: what the deployment costs is
+ // the operator's own hardware, which no catalog rate could express. Three zero buckets are
+ // the same genuine $0 tier the `:free` gateway rows carry, so these models show the free
+ // badge and contribute 0 to the cost center rather than the "unpriced" mark — which is the
+ // truthful reading of a self-hosted endpoint that bills nobody.
+ // - **no base URL**. Every deployment has its own; the user supplies it, as in `custom`.
+ // - **no auto-routing**. Each row pins openai-chat-vllm-adapter explicitly, and the pin is
+ // load-bearing twice over: `Qwen/*` matches none of AutoLLMClient's substring rules and
+ // would be rejected outright, while `deepseek-ai/DeepSeek-V4-*` contains "deepseek-v4"
+ // and would reach DeepSeek's first-party Responses client — pointed at a vLLM server.
+ //
+ // contextWindow is the recipe's NATIVE length, which is the most a deployment can serve
+ // without reconfiguration; an operator may serve less (`--max-model-len` below the native
+ // limit) or, for the Qwen rows, far more with YaRN rope scaling. The catalog cannot know
+ // which, and it derives the compaction thresholds from this number, so the honest default
+ // is the checkpoint's own figure — an entry without one would be assumed to be 128000.
+ // The DeepSeek rows additionally document `--max-model-len >= 393216` as the floor for
+ // their top reasoning levels, which is well inside the window recorded here.
+ {
+ modelId: "deepseek-ai/DeepSeek-V4-Flash",
+ displayName: "DeepSeek V4 Flash",
+ provider: "vllm",
+ contextWindow: 1000000,
+ pricing: usd(0, 0, 0),
+ supportsVision: false,
+ clientType: VLLM_CLIENT_TYPE,
+ },
+ {
+ // The experimental vision revision: DeepSeek's first multimodal V4, a ViT tower on the
+ // same language backbone. Its recipe verifies a 32K deployment and notes the 1M the
+ // checkpoint advertises was not what was measured; the window below is the checkpoint's,
+ // matching every other DeepSeek V4 row in this catalog.
+ modelId: "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
+ displayName: "DeepSeek V4 Flash Vision Exp",
+ provider: "vllm",
+ contextWindow: 1000000,
+ pricing: usd(0, 0, 0),
+ supportsVision: true,
+ clientType: VLLM_CLIENT_TYPE,
+ },
+ {
+ modelId: "deepseek-ai/DeepSeek-V4-Pro",
+ displayName: "DeepSeek V4 Pro",
+ provider: "vllm",
+ contextWindow: 1000000,
+ pricing: usd(0, 0, 0),
+ supportsVision: false,
+ clientType: VLLM_CLIENT_TYPE,
+ },
+ {
+ modelId: "Qwen/Qwen3.5-0.8B",
+ displayName: "Qwen 3.5 0.8B",
+ provider: "vllm",
+ contextWindow: 262144,
+ pricing: usd(0, 0, 0),
+ supportsVision: true,
+ clientType: VLLM_CLIENT_TYPE,
+ },
+ {
+ modelId: "Qwen/Qwen3.5-9B",
+ displayName: "Qwen 3.5 9B",
+ provider: "vllm",
+ contextWindow: 262144,
+ pricing: usd(0, 0, 0),
+ supportsVision: true,
+ clientType: VLLM_CLIENT_TYPE,
+ },
+ {
+ modelId: "Qwen/Qwen3.6-35B-A3B",
+ displayName: "Qwen 3.6 35B A3B",
+ provider: "vllm",
+ contextWindow: 262144,
+ pricing: usd(0, 0, 0),
+ supportsVision: true,
+ clientType: VLLM_CLIENT_TYPE,
+ },
+ {
+ modelId: "Qwen/Qwen3.8-27B",
+ displayName: "Qwen 3.8 27B",
+ provider: "vllm",
+ contextWindow: 262144,
+ pricing: usd(0, 0, 0),
+ supportsVision: true,
+ clientType: VLLM_CLIENT_TYPE,
+ },
+ {
+ modelId: "Qwen/Qwen3.8-Flash-Next",
+ displayName: "Qwen 3.8 Flash Next",
+ provider: "vllm",
+ contextWindow: 262144,
+ pricing: usd(0, 0, 0),
+ supportsVision: true,
+ clientType: VLLM_CLIENT_TYPE,
+ },
];
/**
@@ -1723,6 +1863,20 @@ export function providerInfo(providerId: string): ModelProviderInfo | undefined
return MODEL_PROVIDERS.find((p) => p.id === providerId);
}
+/**
+ * The protocol a group pins on every one of its entries (ModelProviderInfo.clientType), or
+ * undefined when the group pins none — an unknown id (a user-defined group) included.
+ *
+ * The single entry point for the pin, so the places that decide a saved model's client_type
+ * cannot drift apart: the web add-model dialog's default, moving an entry between groups,
+ * the last-resort protocol on the save paths that do not probe, the env-var hint, and the
+ * CLI's `config model add`. A group without a pin keeps whatever those call sites already
+ * derive from its shape.
+ */
+export function providerClientType(providerId: string): string | undefined {
+ return providerInfo(providerId)?.clientType;
+}
+
/** Env var fallback for a single model (the var names AgentHub's client actually reads when api_key / base_url is blank). */
export interface ModelEnvInfo {
envKey: string;
@@ -1774,8 +1928,11 @@ export function resolveModelEnv(modelId: string, clientType?: string): ModelEnvI
// Order mirrors AutoLLMClient: ant-messages before the openai substring match.
if (t.includes("ant-messages")) return env("ANTHROPIC");
// The generic OpenAI-protocol clients — openai-chat (canonical since agenthub 0.4.2, with
- // bare "openai" as a deprecated alias), openai-responses and openai-embedding — all read
- // the OPENAI_* pair.
+ // bare "openai" as a deprecated alias), openai-responses, openai-embedding, and
+ // openai-chat-vllm-adapter (an openai_chat subclass, so it reads the same pair) — all
+ // read the OPENAI_* pair. AutoLLMClient matches openai-chat-vllm-adapter by exact
+ // equality one branch earlier; the substring lands on the same answer, so the order costs
+ // nothing here.
if (t.includes("openai")) return env("OPENAI");
return undefined;
}
@@ -1857,6 +2014,10 @@ export function fastModeProtocol(
if (t.includes("kimi-k3") || t.includes("kimi-k2.5") || t.includes("kimi-k2.6")) return undefined;
if (t.includes("deepseek-v4")) return undefined;
if (t.includes("ant-messages")) return "anthropic";
+ // openai-chat-vllm-adapter is not carved out: it subclasses openai_chat without touching
+ // fast mode, so it maps the parameter exactly as the substring branch below reports. What
+ // a self-hosted server then does with `service_tier` is the third-party residue named
+ // above.
if (t.includes("openai-responses")) return "openai";
if (t.includes("openai") && t.includes("embedding")) return undefined;
if (t.includes("openai")) return "openai";
@@ -1933,6 +2094,13 @@ export function modelHomepageUrl(provider: string, modelId: string): string | un
? `https://platform.kimi.com/docs/pricing/chat-k${m[1]}${m[2]}`
: providerInfo(provider)?.modelsUrl;
}
+ if (provider === "vllm") {
+ // recipes.vllm.ai has a page per model vLLM published a recipe for, which is exactly what
+ // this group presets; an id the user serves themselves has no page, so it gets the index.
+ return catalogEntryFor(provider, modelId) !== undefined
+ ? `https://recipes.vllm.ai/${modelId}`
+ : providerInfo(provider)?.modelsUrl;
+ }
if (provider === "custom") return undefined;
return providerInfo(provider)?.modelsUrl;
}
diff --git a/packages/core/test/model-catalog.test.ts b/packages/core/test/model-catalog.test.ts
index 9b270b59..5d0b1e3c 100644
--- a/packages/core/test/model-catalog.test.ts
+++ b/packages/core/test/model-catalog.test.ts
@@ -15,6 +15,7 @@ import {
offPeakAt,
DEEPSEEK_OFF_PEAK,
presetModelEntries,
+ providerClientType,
providerInfo,
fastModeProtocol,
resolveModelEnv,
@@ -30,9 +31,10 @@ describe("model-catalog", () => {
const ids = MODEL_CATALOG.map((m) => m.modelId);
expect(MODEL_CATALOG[0]!.provider).toBe("deepseek");
// Group order is hand-curated, interleaving gateways and first-party vendors: TokenDance
- // first (the recommended group), DeepSeek next (the default model's provider) and custom
- // always last. This is the page's DEFAULT only — a Project that has reordered its groups
- // stores every key and keeps its own arrangement (web's model-group-order.ts).
+ // first (the recommended group), DeepSeek next (the default model's provider), vLLM last
+ // among the vendors (self-hosted, so nothing in it runs until the user names a server)
+ // and custom always last. This is the page's DEFAULT only — a Project that has reordered
+ // its groups stores every key and keeps its own arrangement (web's model-group-order.ts).
expect(MODEL_PROVIDERS.map((p) => p.id)).toEqual([
"tokendance",
"deepseek",
@@ -47,6 +49,7 @@ describe("model-catalog", () => {
"minimax",
"qwen-pay-as-you-go",
"qwen-token-plan",
+ "vllm",
"custom",
]);
// Exactly one group is marked recommended, and it is the one that leads the default
@@ -73,12 +76,17 @@ describe("model-catalog", () => {
expect(providerIds.has(m.provider)).toBe(true);
expect(m.provider).not.toBe("custom");
}
- // Except for custom, every provider gives a console link to "get an API key" (shown in the
- // frontend's group header) and a model list / docs link to "get a model id" (shown in the add-model dialog).
+ // Every provider gives a console link to "get an API key" (shown in the frontend's group
+ // header) and a model list / docs link to "get a model id" (shown in the add-model
+ // dialog) — except where there is no such page to link. custom has neither; vLLM has no
+ // console at all (the user runs the server), but its served ids are documented.
for (const p of MODEL_PROVIDERS) {
if (p.id === "custom") {
expect(p.apiKeyUrl).toBeUndefined();
expect(p.modelsUrl).toBeUndefined();
+ } else if (p.id === "vllm") {
+ expect(p.apiKeyUrl).toBeUndefined();
+ expect(p.modelsUrl).toBe("https://recipes.vllm.ai/");
} else {
expect(p.apiKeyUrl).toMatch(/^https:\/\//);
expect(p.modelsUrl).toMatch(/^https:\/\//);
@@ -111,9 +119,11 @@ describe("model-catalog", () => {
it("every entry has valid three-bucket pricing; context_window is a positive integer", () => {
for (const m of MODEL_CATALOG) {
- if (m.modelId.endsWith(":free") || m.modelId === "openrouter/free") {
- // Free-tier gateway model (:free variants and the openrouter/free router): a genuine
- // $0 price (not "unknown"), so costs compute to 0.
+ if (m.provider === "vllm" || m.modelId.endsWith(":free") || m.modelId === "openrouter/free") {
+ // Self-hosted vLLM and the free-tier gateway rows share one treatment: a genuine $0
+ // price (not "unknown"), so costs compute to 0 and the free badge shows. Nobody bills
+ // per token for either — a vLLM deployment costs its operator hardware, which no
+ // catalog rate expresses.
expect(m.pricing, m.modelId).toBeDefined();
expect([m.pricing!.cache_read, m.pricing!.cache_write, m.pricing!.output]).toEqual([
0, 0, 0,
@@ -130,6 +140,77 @@ describe("model-catalog", () => {
}
});
+ it("vLLM (self-hosted): the group pins openai-chat-vllm-adapter, prices at zero and carries no endpoint", () => {
+ const vllm = MODEL_CATALOG.filter((m) => m.provider === "vllm");
+ // Dictionary order by upstream id, case-insensitive (as in siliconflow): deepseek-ai/
+ // before Qwen/, and the flash revision before the vision revision it prefixes.
+ expect(vllm.map((m) => [m.modelId, m.contextWindow, m.supportsVision])).toEqual([
+ ["deepseek-ai/DeepSeek-V4-Flash", 1000000, false],
+ ["deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", 1000000, true],
+ ["deepseek-ai/DeepSeek-V4-Pro", 1000000, false],
+ ["Qwen/Qwen3.5-0.8B", 262144, true],
+ ["Qwen/Qwen3.5-9B", 262144, true],
+ ["Qwen/Qwen3.6-35B-A3B", 262144, true],
+ ["Qwen/Qwen3.8-27B", 262144, true],
+ ["Qwen/Qwen3.8-Flash-Next", 262144, true],
+ ]);
+ for (const m of vllm) {
+ // The pin is load-bearing on every row: Qwen/* matches none of AutoLLMClient's rules
+ // and would be rejected, and deepseek-ai/DeepSeek-V4-* contains "deepseek-v4", which
+ // would reach DeepSeek's first-party Responses client pointed at a vLLM server.
+ expect(m.clientType, m.modelId).toBe("openai-chat-vllm-adapter");
+ // Nobody bills per token and there is no shared endpoint: the user runs the server and
+ // supplies its URL. Zero is a real rate here, not a missing one — see the pricing test.
+ expect([m.pricing?.cache_read, m.pricing?.cache_write, m.pricing?.output], m.modelId).toEqual(
+ [0, 0, 0],
+ );
+ expect(m.pricing!.unit, m.modelId).toBe("usd_per_mtok");
+ expect(m.baseUrl, m.modelId).toBeUndefined();
+ // Chat Completions on the wire, so the credential fallback is the OPENAI_* pair.
+ expect(resolveModelEnv(m.modelId, m.clientType)?.envKey, m.modelId).toBe("OPENAI_API_KEY");
+ }
+ // The upstream id survives verbatim, capitals and all — it is what the vLLM server was
+ // started with, and AgentHub's per-model thinking table lowercases it on its own side.
+ expect(catalogEntryFor("vllm", "Qwen/Qwen3.8-27B")?.displayName).toBe("Qwen 3.8 27B");
+ expect(catalogEntryFor("vllm", "qwen/qwen3.8-27b")).toBeUndefined();
+ // The same upstream ids are resold by SiliconFlow, and the pair lookup keeps the two
+ // apart: same id, different group, different protocol and endpoint.
+ expect(catalogEntryFor("siliconflow", "deepseek-ai/DeepSeek-V4-Pro")?.clientType).toBe(
+ "openai-chat",
+ );
+ // The group pin, read the one way every call site reads it.
+ expect(providerClientType("vllm")).toBe("openai-chat-vllm-adapter");
+ expect(providerInfo("vllm")!.gatewayBaseUrl).toBeUndefined();
+ // No group but vLLM declares one: the gateways derive openai-chat from their preset
+ // endpoint, and custom / user-defined groups leave the protocol to detection.
+ expect(MODEL_PROVIDERS.filter((p) => p.clientType !== undefined).map((p) => p.id)).toEqual([
+ "vllm",
+ ]);
+ expect(providerClientType("custom")).toBeUndefined();
+ expect(providerClientType("my-own-group")).toBeUndefined();
+ // Presets reach a Project with the pin and the zero rate, and without an endpoint: what a
+ // self-hosted deployment bills per token is nothing, and the Project stores that as a rate
+ // rather than as a gap (see the pricing test).
+ const preset = presetModelEntries().filter((e) => e.provider === "vllm");
+ expect(preset).toHaveLength(8);
+ for (const e of preset) {
+ expect(e.client_type, e.model_id).toBe("openai-chat-vllm-adapter");
+ expect(
+ [e.pricing?.cache_read, e.pricing?.cache_write, e.pricing?.output],
+ e.model_id,
+ ).toEqual([0, 0, 0]);
+ expect(e.base_url, e.model_id).toBeUndefined();
+ }
+ // Each preset id has a recipe page; an id the user serves themselves has none, so it
+ // falls back to the recipe index.
+ expect(modelHomepageUrl("vllm", "Qwen/Qwen3.8-27B")).toBe(
+ "https://recipes.vllm.ai/Qwen/Qwen3.8-27B",
+ );
+ expect(modelHomepageUrl("vllm", "my-own-finetune")).toBe("https://recipes.vllm.ai/");
+ // Fast mode rides on the client, and the vLLM client inherits openai_chat's mapping.
+ expect(fastModeProtocol("Qwen/Qwen3.8-27B", "openai-chat-vllm-adapter")).toBe("openai");
+ });
+
it("providerInfo matches by id; unknown ids return undefined", () => {
expect(providerInfo("moonshot")?.envKey).toBe("MOONSHOT_API_KEY");
expect(providerInfo("nonexistent")).toBeUndefined();
@@ -459,6 +540,7 @@ describe("model-catalog", () => {
"tokendance",
"qwen-token-plan",
"qwen-pay-as-you-go",
+ "vllm",
"custom",
]) {
expect(providerInfo(id)!.envKey).toBe("OPENAI_API_KEY");
diff --git a/packages/server/src/services/project-config-service.ts b/packages/server/src/services/project-config-service.ts
index ac3b1b96..daedee76 100644
--- a/packages/server/src/services/project-config-service.ts
+++ b/packages/server/src/services/project-config-service.ts
@@ -102,9 +102,10 @@ export function maskApiKey(key: string): string {
* official endpoint, so consulting the vendor variable (ANTHROPIC_API_KEY, …) is the intended
* configuration and its value may be previewed masked. Excluded — no detection, no preview:
*
- * - gateway groups (they resell through generic OpenAI-protocol clients whose fallback is
- * OPENAI_API_KEY, the *official OpenAI* variable; steering it to a reseller endpoint is
- * exactly the misconfiguration the preview must not encourage) and the custom group;
+ * - gateway groups and any group pinning a protocol, e.g. vLLM (both reach a generic
+ * OpenAI-protocol client whose fallback is OPENAI_API_KEY, the *official OpenAI*
+ * variable; steering it to a reseller or to the user's own server is exactly the
+ * misconfiguration the preview must not encourage), and the custom group;
* - user-defined groups (not in MODEL_PROVIDERS at all);
* - any entry re-pointed away from the official shape: a catalog preset whose client_type or
* base_url differs from the catalog's own values, or an off-catalog vendor-group entry that
@@ -120,7 +121,12 @@ export function envFallbackFirstParty(entry: {
baseUrl: string | undefined;
}): boolean {
const group = providerInfo(entry.provider);
- if (group === undefined || group.id === "custom" || group.gatewayBaseUrl !== undefined) {
+ if (
+ group === undefined ||
+ group.id === "custom" ||
+ group.gatewayBaseUrl !== undefined ||
+ group.clientType !== undefined
+ ) {
return false;
}
const cat = catalogEntryFor(entry.provider, entry.modelId);
diff --git a/packages/web/src/components/ui/provider-logo.tsx b/packages/web/src/components/ui/provider-logo.tsx
index e75aab14..239075da 100644
--- a/packages/web/src/components/ui/provider-logo.tsx
+++ b/packages/web/src/components/ui/provider-logo.tsx
@@ -7,8 +7,10 @@
* MiniMax use each vendor's brand mark (for recognition purposes, not under trademark license;
* Qwen's official gradient wordmark is flattened to currentColor monochrome).
* Z.AI uses a simplified geometric approximation of its branded glyph (not an
- * exact reproduction of the trademark); custom models use a generic cube. All
- * are pure paths, no external image assets.
+ * exact reproduction of the trademark); vLLM uses its official mark from the
+ * project's media kit, its two brand colours flattened to currentColor like
+ * Qwen's gradient; custom models use a generic cube. All are pure paths, no
+ * external image assets.
*
* Vendor ids outside the preset table are user-defined groups: instead of all
* sharing the cube (which made same-named models across groups
@@ -139,6 +141,20 @@ const GLYPHS: Record<string, Glyph> = {
</>
),
},
+ vllm: {
+ // The official mark (vllm-project/media-kit, vLLM-Logo.svg): the solid wedge and the
+ // swoosh beside it, coordinates rounded to 2dp and the viewBox tightened to their bounding
+ // box. Its two brand colours (amber wedge, blue swoosh) collapse into currentColor the way
+ // Qwen's gradient does above — this set is monochrome so a row of provider glyphs reads as
+ // one family. The drop-shadow layer the source draws underneath is dropped with them.
+ viewBox: "13.39 12.21 68.04 70.39",
+ path: (
+ <>
+ <path d="m41.05 27.29l0 55.31l-27.66 -55.31z" />
+ <path d="m41.05 82.6l21.73 0l18.65 -70.39l-25.57 13.46z" />
+ </>
+ ),
+ },
custom: {
stroke: true,
path: (
diff --git a/packages/web/src/features/models/models-page.tsx b/packages/web/src/features/models/models-page.tsx
index 561b1dec..205c3b53 100644
--- a/packages/web/src/features/models/models-page.tsx
+++ b/packages/web/src/features/models/models-page.tsx
@@ -71,6 +71,7 @@ import {
catalogEntryFor,
fastModeProtocol,
modelHomepageUrl,
+ providerClientType,
providerInfo,
resolveModelEnv,
} from "@prismshadow/penguin-core/model-catalog";
@@ -209,12 +210,19 @@ export function decimalOnly(v: string): string {
}
/**
- * Moving an existing model to Custom keeps a generic protocol client type (protocol
- * detection / the in-field picker manage those) and otherwise switches to the generic
- * OpenAI Chat Completions client — an unroutable or vendor-pinned type must not leak
- * into a custom group.
+ * The client type an entry carries after it is moved into `provider`.
+ *
+ * A group that pins a protocol takes it unconditionally — that is what the pin means, and
+ * the entry the user dragged in has to speak what the rest of the group speaks. Moving to
+ * Custom keeps a generic protocol client type (protocol detection / the in-field picker
+ * manage those) and otherwise switches to the generic OpenAI Chat Completions client — an
+ * unroutable or vendor-pinned type must not leak into a custom group. Every other group
+ * keeps the current value: a first-party group auto-routes by id, and a gateway's rows
+ * carry the pin their preset already gave them.
*/
export function clientTypeAfterProviderChange(provider: string, current: string): string {
+ const pinned = providerClientType(provider);
+ if (pinned !== undefined) return pinned;
if (provider !== "custom") return current;
return current.trim() !== "" && isGenericProtocolClientType(current) ? current : "openai-chat";
}
@@ -2159,16 +2167,20 @@ function ModelDialog({
output: usdToInput(row.output, currency),
};
}
- // New model: protocol follows group semantics — a first-party vendor group
- // doesn't persist client_type (AgentHub auto-routes by upstream id, with env fallback
- // resolved live from the id); custom / user-defined groups / gateways use a fixed
- // openai-chat protocol (env fallback OPENAI_*), and gateways additionally pre-fill their
- // endpoint base URL. provider keeps the entry point's original value (a user-defined
- // group must not collapse into custom), stored as a separate field from model_id, with
- // no concatenation on save.
+ // New model: protocol follows group semantics — a group that pins one (vLLM) hands it
+ // to the new entry outright; a first-party vendor group doesn't persist client_type
+ // (AgentHub auto-routes by upstream id, with env fallback resolved live from the id);
+ // custom / user-defined groups / gateways use a fixed openai-chat protocol (env fallback
+ // OPENAI_*), and gateways additionally pre-fill their endpoint base URL. provider keeps
+ // the entry point's original value (a user-defined group must not collapse into custom),
+ // stored as a separate field from model_id, with no concatenation on save.
const info = providerInfo(addProvider);
+ const pinnedClientType = providerClientType(addProvider);
const vendorAdd =
- info !== undefined && info.id !== "custom" && info.gatewayBaseUrl === undefined;
+ info !== undefined &&
+ info.id !== "custom" &&
+ info.gatewayBaseUrl === undefined &&
+ pinnedClientType === undefined;
return {
provider: addProvider,
modelId: "",
@@ -2183,7 +2195,8 @@ function ModelDialog({
// No protocol is preselected for a custom / user-defined group: it is detected from
// the endpoint (on demand, or on save while still unset) or picked by hand. Vendor
// groups auto-route by model id, and gateways keep their preset Chat Completions pin.
- clientType: vendorAdd || isCustomLikeGroup(addProvider) ? "" : "openai-chat",
+ clientType:
+ pinnedClientType ?? (vendorAdd || isCustomLikeGroup(addProvider) ? "" : "openai-chat"),
cacheRead: "",
cacheWrite: "",
output: "",
@@ -2593,14 +2606,18 @@ function ModelDialog({
form.modelId.trim(),
envHintClientType(form.provider, form.clientType),
)?.envKey;
- // First-party provider group (built-in, non-gateway, non-custom): adding
- // goes through auto-routing — show a hint when the id can't be routed
+ // The protocol this group pins on every entry, user-added ones included (vLLM); undefined
+ // for every group that leaves the protocol to auto-routing, a gateway preset, or detection.
+ const pinnedGroupClientType = providerClientType(form.provider);
+ // First-party provider group (built-in, non-gateway, non-custom, no group-level pin):
+ // adding goes through auto-routing — show a hint when the id can't be routed
// (doesn't block saving: the routing table evolves with the AgentHub
// version, so it's judged at runtime).
const vendorGroup =
dialogProvider !== undefined &&
dialogProvider.id !== "custom" &&
- dialogProvider.gatewayBaseUrl === undefined;
+ dialogProvider.gatewayBaseUrl === undefined &&
+ pinnedGroupClientType === undefined;
const autoRouteMiss =
vendorGroup &&
!form.clientType.trim() &&
@@ -2847,18 +2864,21 @@ function ModelDialog({
{/* Adding a model: protocol note first (preset direct-vendor group = only the
vendor's official protocol, named via the group label — the in-field suffix
- on the base URL below says which path; custom / self-defined group / gateway
- = fixed OpenAI protocol), then the identity fields ("get model id / API key"
- links next to the respective inputs; fill in the id to test connectivity —
- verify before saving). */}
+ on the base URL below says which path; a group that pins a protocol = that
+ protocol, named outright, plus its own endpoint; custom / self-defined group /
+ gateway = fixed OpenAI protocol), then the identity fields ("get model id /
+ API key" links next to the respective inputs; fill in the id to test
+ connectivity — verify before saving). */}
{isNew && (
<>
<p className="text-xs text-gray-500 dark:text-gray-400">
{vendorGroup && dialogProvider
? S.models.vendorProtocolHint(dialogProvider.label)
- : customLikeGroup
- ? S.models.addProtocolHintDetect
- : S.models.addProtocolHint}
+ : pinnedGroupClientType !== undefined
+ ? S.models.addProtocolHintPinned(pinnedGroupClientType)
+ : customLikeGroup
+ ? S.models.addProtocolHintDetect
+ : S.models.addProtocolHint}
</p>
{identityFields}
</>
@@ -3172,13 +3192,16 @@ function ModelDialog({
{!isNew && identityFields}
{/* Legacy entries carrying a client_type other than the standard openai-chat
(historical config): read-only display. Compared canonically so the deprecated
- bare "openai" spelling (pre-0.4.2 configs) is not flagged as legacy either, and
+ bare "openai" spelling (pre-0.4.2 configs) is not flagged as legacy either,
skipped when the protocol selector above already represents it (generic protocol
- types in custom-like groups are editable there). */}
+ types in custom-like groups are editable there), and skipped when the value IS
+ the group's own pin — that is this group's normal protocol, not a leftover from
+ an older config, and "kept as configured" would misdescribe it. */}
{!isNew &&
!preset &&
!showProtocolSelector &&
form.clientType &&
+ form.clientType !== pinnedGroupClientType &&
canonicalClientType(form.clientType) !== "openai-chat" && (
<p className="text-xs text-gray-400 dark:text-gray-500">
{S.models.clientTypeLocked(form.clientType)}
diff --git a/packages/web/src/features/models/protocol-path.ts b/packages/web/src/features/models/protocol-path.ts
index f0ebfe8e..53ca9de5 100644
--- a/packages/web/src/features/models/protocol-path.ts
+++ b/packages/web/src/features/models/protocol-path.ts
@@ -20,8 +20,10 @@
* https://api.deepseek.com carries no /v1, and the request path adds none).
* - Every OpenAI Chat Completions compatible client — explicit
* `client_type: "openai-chat"` (gateways, custom and user-defined groups; the bare
- * "openai" spelling is a deprecated pre-0.4.2 alias) plus the GLM / Kimi direct
- * clients — POST {base}/chat/completions.
+ * "openai" spelling is a deprecated pre-0.4.2 alias),
+ * `client_type: "openai-chat-vllm-adapter"` (the vLLM group; an openai-chat subclass that
+ * differs only in the thinking switch it sends), plus the GLM / Kimi direct clients —
+ * POST {base}/chat/completions.
*
* The three generic protocol clients — `openai-responses`, `ant-messages`,
* `openai-chat` — are also what the custom-model protocol detection stores.
@@ -42,7 +44,10 @@ export function protocolPathForModel(provider: string, clientType: string): stri
// Order mirrors AutoLLMClient's routing.
if (t.includes("ant-messages")) return "/v1/messages";
if (t.includes("openai-responses")) return "/responses";
- // openai-chat / openai-embedding / the deprecated bare "openai" alias.
+ // openai-chat / openai-chat-vllm-adapter / openai-embedding / the deprecated bare
+ // "openai" alias. AgentHub places openai-chat-vllm-adapter one branch earlier, matched by
+ // exact equality; here the substring reaches the same path, so the two orderings cannot
+ // disagree.
if (t.includes("openai")) return "/chat/completions";
// Legacy explicit client types (historical config): pin the family like auto-routing would.
if (t.includes("claude")) return "/v1/messages";
diff --git a/packages/web/src/features/models/protocol-types.ts b/packages/web/src/features/models/protocol-types.ts
index 635f6247..372f860b 100644
--- a/packages/web/src/features/models/protocol-types.ts
+++ b/packages/web/src/features/models/protocol-types.ts
@@ -6,7 +6,7 @@
* The probing itself is server-side (packages/server/src/services/protocol-detect.ts); these
* only decide what the dialog shows and what is worth sending there.
*/
-import { providerInfo } from "@prismshadow/penguin-core/model-catalog";
+import { providerClientType, providerInfo } from "@prismshadow/penguin-core/model-catalog";
/**
* AgentHub's generic protocol client types, in detection order (custom / user-defined
@@ -81,8 +81,9 @@ export function detectableBaseUrl(baseUrl: string): boolean {
/**
* Custom-like group: the entry picks its own protocol from the generic trio, rather than
- * being auto-routed inside a first-party vendor group or pinned by a gateway preset.
- * `custom` plus every user-defined group (a provider id the catalog does not know).
+ * being auto-routed inside a first-party vendor group or pinned by a gateway preset or a
+ * group-level pin. `custom` plus every user-defined group (a provider id the catalog does
+ * not know).
*/
export function isCustomLikeGroup(provider: string): boolean {
return provider === "custom" || providerInfo(provider) === undefined;
@@ -99,13 +100,20 @@ export function isCustomLikeGroup(provider: string): boolean {
* net for the paths that do not (set-default, set-vision-proxy, remove), where probing
* the endpoint would be the wrong thing to do.
*
+ * A group that pins a protocol answers the same question one step earlier, and for a
+ * stronger reason: the pin is not a fallback but the group's own semantics, so an entry
+ * that reaches here without one takes it rather than the compatible-client default.
+ *
* Preset and vendor-group entries are returned untouched: their model ids ARE routable,
* so an empty value there correctly means "let AgentHub infer from the id", and the
* empty default must not leak into them as a bogus pin.
*/
export function protocolForPersist(provider: string, clientType: string): string {
const t = clientType.trim();
- if (t !== "" || !isCustomLikeGroup(provider)) return t;
+ if (t !== "") return t;
+ const pinned = providerClientType(provider);
+ if (pinned !== undefined) return pinned;
+ if (!isCustomLikeGroup(provider)) return t;
return DEFAULT_CUSTOM_CLIENT_TYPE;
}
@@ -119,6 +127,9 @@ export function protocolForPersist(provider: string, clientType: string): string
* not the user declaring the model ready, and probing an endpoint as a side effect of
* "make this the default" would be surprising. Those paths stay safe through
* protocolForPersist instead.
+ *
+ * A group that pins a protocol never reaches here either — it is not custom-like, and there
+ * is nothing to probe for when the group has already decided the answer.
*/
export function needsProtocolDetectOnSave(
action: string,
@@ -136,12 +147,17 @@ export function needsProtocolDetectOnSave(
* a custom group would then claim the entry reads ANTHROPIC_API_KEY, when nothing about
* that group routes by id and the entry will in fact be saved on the compatible client.
* Returning the default for custom-like groups keeps the hint honest; undefined elsewhere
- * preserves the id-based routing those groups genuinely use.
+ * preserves the id-based routing those groups genuinely use. A group that pins a protocol
+ * is the same argument again: nothing there routes by id, so the pin is what the entry will
+ * be saved on and what the hint has to resolve against.
*/
export function envHintClientType(provider: string, clientType: string): string | undefined {
const t = clientType.trim();
if (t !== "") return t;
- return isCustomLikeGroup(provider) ? DEFAULT_CUSTOM_CLIENT_TYPE : undefined;
+ return (
+ providerClientType(provider) ??
+ (isCustomLikeGroup(provider) ? DEFAULT_CUSTOM_CLIENT_TYPE : undefined)
+ );
}
/*
diff --git a/packages/web/src/lib/strings-en.ts b/packages/web/src/lib/strings-en.ts
index d11bac8c..7261882f 100644
--- a/packages/web/src/lib/strings-en.ts
+++ b/packages/web/src/lib/strings-en.ts
@@ -739,6 +739,8 @@ export const en: Strings = {
"New models use the OpenAI Chat Completions protocol; set the base URL to a compatible endpoint",
vendorProtocolHint: (vendor: string): string =>
`Only ${vendor}'s official API protocol is supported; use a custom model group for OpenAI-compatible endpoints.`,
+ addProtocolHintPinned: (protocol: string): string =>
+ `Models in this group always use the ${protocol} protocol; set the base URL to your own server`,
autoRouteNone:
"This model ID cannot be routed with the current provider protocol. If it uses an OpenAI-compatible endpoint, move it to Custom.",
useCustomGroup: "Move to Custom",
diff --git a/packages/web/src/lib/strings.ts b/packages/web/src/lib/strings.ts
index 1f823573..5b16e93b 100644
--- a/packages/web/src/lib/strings.ts
+++ b/packages/web/src/lib/strings.ts
@@ -707,6 +707,9 @@ export const zh = {
/** Add-dialog note for preset direct-vendor groups (fed the provider label): states whose protocol the group speaks — the in-field suffix on the base URL shows which path. */
vendorProtocolHint: (vendor: string): string =>
`仅支持 ${vendor} 官方接口协议,OpenAI 兼容接口请使用自定义模型分组`,
+ /** Add-dialog note for a group that pins one protocol on every entry (fed the client type): the protocol is not a choice here, and the endpoint is the user's own. */
+ addProtocolHintPinned: (protocol: string): string =>
+ `本分组的模型固定使用 ${protocol} 协议,base URL 填你自己的服务地址`,
autoRouteNone: "该模型 ID 无法按当前厂商协议识别;若使用 OpenAI 兼容接口,可转为自定义模型。",
useCustomGroup: "转为自定义模型",
addGroup: "新增分组",
diff --git a/packages/web/test/model-grouping.test.ts b/packages/web/test/model-grouping.test.ts
index 2db0dbf9..390d7265 100644
--- a/packages/web/test/model-grouping.test.ts
+++ b/packages/web/test/model-grouping.test.ts
@@ -98,6 +98,7 @@ describe("groupModelRows", () => {
"minimax",
"qwen-pay-as-you-go",
"qwen-token-plan",
+ "vllm",
"custom",
]);
expect(MODEL_PROVIDERS.find((p) => p.id === "siliconflow")!.label).toBe("SiliconFlow");
diff --git a/packages/web/test/protocol-detect.test.ts b/packages/web/test/protocol-detect.test.ts
index 6ed44277..9fd8b410 100644
--- a/packages/web/test/protocol-detect.test.ts
+++ b/packages/web/test/protocol-detect.test.ts
@@ -99,6 +99,17 @@ describe("clientTypeAfterProviderChange (protocol family kept on move to Custom)
expect(clientTypeAfterProviderChange("google", "openai")).toBe("openai");
expect(clientTypeAfterProviderChange("my-group", "ant-messages")).toBe("ant-messages");
});
+
+ it("rewrites to the group's own protocol when one is pinned, whatever the entry carried", () => {
+ // The pin is the group's semantics, not a fallback: an entry dragged into vLLM speaks
+ // what the rest of the group speaks, including one that had already picked a protocol.
+ expect(clientTypeAfterProviderChange("vllm", "")).toBe("openai-chat-vllm-adapter");
+ expect(clientTypeAfterProviderChange("vllm", "openai-chat")).toBe("openai-chat-vllm-adapter");
+ expect(clientTypeAfterProviderChange("vllm", "ant-messages")).toBe("openai-chat-vllm-adapter");
+ expect(clientTypeAfterProviderChange("vllm", "deepseek-v4")).toBe("openai-chat-vllm-adapter");
+ // And moving back out of it does not carry the pin along.
+ expect(clientTypeAfterProviderChange("custom", "openai-chat-vllm-adapter")).toBe("openai-chat");
+ });
});
describe("isCustomLikeGroup", () => {
@@ -107,6 +118,8 @@ describe("isCustomLikeGroup", () => {
expect(isCustomLikeGroup("my-group")).toBe(true);
expect(isCustomLikeGroup("openai")).toBe(false);
expect(isCustomLikeGroup("anthropic")).toBe(false);
+ // A group that pins its protocol picks nothing: the choice is already made for it.
+ expect(isCustomLikeGroup("vllm")).toBe(false);
});
});
@@ -124,6 +137,8 @@ describe("needsProtocolDetectOnSave (save detects a still-unset protocol first)"
it("never probes for vendor groups, nor for actions other than save", () => {
// A vendor group auto-routes by model id; an empty protocol there is correct.
expect(needsProtocolDetectOnSave("save", "openai", "")).toBe(false);
+ // A pinned group has nothing to probe for — the group already knows the answer.
+ expect(needsProtocolDetectOnSave("save", "vllm", "")).toBe(false);
expect(needsProtocolDetectOnSave("remove", "custom", "")).toBe(false);
expect(needsProtocolDetectOnSave("setDefault", "custom", "")).toBe(false);
expect(needsProtocolDetectOnSave("setVisionModel", "custom", "")).toBe(false);
@@ -147,6 +162,13 @@ describe("envHintClientType (custom groups never infer a client from the model i
expect(envFor("custom", "whatever", "openai-responses")).toBe("OPENAI_API_KEY");
});
+ it("resolves a pinned group against its pin rather than the model id", () => {
+ // deepseek-ai/DeepSeek-V4-Pro would otherwise route by id to DEEPSEEK_API_KEY, which is
+ // not what an entry saved on the vLLM client reads.
+ expect(envHintClientType("vllm", "")).toBe("openai-chat-vllm-adapter");
+ expect(envFor("vllm", "deepseek-ai/DeepSeek-V4-Pro", "")).toBe("OPENAI_API_KEY");
+ });
+
it("leaves vendor groups routing by model id, which is how they really work", () => {
expect(envHintClientType("anthropic", "")).toBeUndefined();
expect(envFor("anthropic", "claude-sonnet-5", "")).toBe("ANTHROPIC_API_KEY");
@@ -168,6 +190,13 @@ describe("protocolForPersist (an empty protocol must never reach the config)", (
expect(protocolForPersist("my-group", " openai-chat ")).toBe("openai-chat");
});
+ it("writes a pinned group's protocol rather than leaving it to inference", () => {
+ expect(protocolForPersist("vllm", "")).toBe("openai-chat-vllm-adapter");
+ expect(protocolForPersist("vllm", " ")).toBe("openai-chat-vllm-adapter");
+ // An explicit value still wins — this is the last-resort net, not an override.
+ expect(protocolForPersist("vllm", "openai-chat")).toBe("openai-chat");
+ });
+
it("leaves preset / vendor groups empty so AgentHub still infers from the model id", () => {
expect(protocolForPersist("openai", "")).toBe("");
expect(protocolForPersist("anthropic", "")).toBe("");
diff --git a/packages/web/test/protocol-path.test.ts b/packages/web/test/protocol-path.test.ts
index e5ce92e2..1a1a388a 100644
--- a/packages/web/test/protocol-path.test.ts
+++ b/packages/web/test/protocol-path.test.ts
@@ -55,6 +55,14 @@ describe("protocolPathForModel", () => {
expect(protocolPathForModel("myproxy", "")).toBe("/chat/completions");
});
+ it("openai-chat-vllm-adapter is chat completions, whatever the model id looks like", () => {
+ // It contains "openai" and reaches the same branch, which is the right answer: the vLLM
+ // client subclasses openai-chat and POSTs the same path. The DeepSeek id is the one that
+ // would go wrong if the group implied the path — the vendor's own client uses /responses.
+ expect(protocolPathForModel("vllm", "openai-chat-vllm-adapter")).toBe("/chat/completions");
+ expect(protocolPathForModel("custom", "openai-chat-vllm-adapter")).toBe("/chat/completions");
+ });
+
it("an explicit openai-chat client type wins over vendor-group membership", () => {
expect(protocolPathForModel("anthropic", "openai-chat")).toBe("/chat/completions");
expect(protocolPathForModel("google", "openai-chat")).toBe("/chat/completions");
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 2770f079..691bd88b 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -88,8 +88,8 @@ importers:
specifier: workspace:*
version: link:../../plugins/use-firecrawl
'@prismshadow/agenthub':
- specifier: ^0.4.9
+ specifier: ^0.4.10
'@prismshadow/penguin-core':
specifier: workspace:*
version: link:../core
@@ -167,8 +167,8 @@ importers:
specifier: workspace:*
version: link:../../plugins/use-firecrawl
'@prismshadow/agenthub':
- specifier: ^0.4.9
+ specifier: ^0.4.10
arktype:
specifier: ^2.2.3
version: 2.2.3
@@ -1254,8 +1254,8 @@ packages:
engines: {node: '>=18'}
hasBin: true
- '@prismshadow/[email protected]':
- resolution: {integrity: sha512-Sjdm7wDDMhkKoW0v0ct3LgH6JedpnnGWwJ+NdEw8VjoXunwzLT39qAVKk/CVMizvoTUFEOgWshh6EGDnzODScA==}
+ '@prismshadow/[email protected]':
+ resolution: {integrity: sha512-zrBF/AVCJlLfjTFYRcxl9awksBNVSzkKqseTHZfEDIifg1ng5+a/LtWnABf8u3WcWlntU14JwdJiy1/4tgMqLw==}
'@prismshadow/penguin-cli@file:packages/cli':
resolution: {directory: packages/cli, type: directory}
@@ -5001,7 +5001,7 @@ snapshots:
dependencies:
playwright: 1.61.1
dependencies:
'@anthropic-ai/bedrock-sdk': 0.26.4([email protected])
'@anthropic-ai/sdk': 0.91.1([email protected])
@@ -5016,7 +5016,7 @@ snapshots:
- ws
- zod
dependencies:
'@anthropic-ai/bedrock-sdk': 0.26.4([email protected])
'@anthropic-ai/sdk': 0.91.1([email protected])
@@ -5046,7 +5046,7 @@ snapshots:
'@penguinharness/use-bento-slides': file:plugins/use-bento-slides
'@penguinharness/use-claude-code': file:plugins/use-claude-code
'@penguinharness/use-firecrawl': file:plugins/use-firecrawl
- '@prismshadow/agenthub': 0.4.9([email protected])([email protected])
+ '@prismshadow/agenthub': 0.4.10([email protected])([email protected])
'@prismshadow/penguin-core': file:packages/core([email protected])([email protected])
'@prismshadow/penguin-server': file:packages/server([email protected])
commander: 13.1.0
@@ -5077,7 +5077,7 @@ snapshots:
'@penguinharness/use-bento-slides': file:plugins/use-bento-slides
'@penguinharness/use-claude-code': file:plugins/use-claude-code
'@penguinharness/use-firecrawl': file:plugins/use-firecrawl
- '@prismshadow/agenthub': 0.4.9([email protected])([email protected])
+ '@prismshadow/agenthub': 0.4.10([email protected])([email protected])
arktype: 2.2.3
smol-toml: 1.6.1
yaml: 2.9.0
@@ -5104,7 +5104,7 @@ snapshots:
'@penguinharness/use-bento-slides': file:plugins/use-bento-slides
'@penguinharness/use-claude-code': file:plugins/use-claude-code
'@penguinharness/use-firecrawl': file:plugins/use-firecrawl
- '@prismshadow/agenthub': 0.4.9([email protected])([email protected])([email protected])
+ '@prismshadow/agenthub': 0.4.10([email protected])([email protected])([email protected])
arktype: 2.2.3
smol-toml: 1.6.1
yaml: 2.9.0
diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml
index b790567e..221bef67 100644
--- a/pnpm-workspace.yaml
+++ b/pnpm-workspace.yaml
@@ -37,4 +37,4 @@ allowBuilds:
# Every bump lands inside the age gate, so the entry names the pinned version and moves
# with each bump rather than being removed.
minimumReleaseAgeExclude:
- - "@prismshadow/[email protected]"
+ - "@prismshadow/[email protected]"