A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(agent): your providers are a list, and each says what it can do
yhh8ef59ed8caee
diff --git a/desktop/electron/agent/anthropic.ts b/desktop/electron/agent/anthropic.tsindex f757b5a54..2a507b2c1 100644--- a/desktop/electron/agent/anthropic.ts+++ b/desktop/electron/agent/anthropic.ts@@ -48,6 +48,16 @@ export interface AnthropicOptions {/** Point at an Anthropic-compatible gateway; omit for the API. Its presence* is what drops the request to the core wire format — see the file header. */baseURL?: string;+ /**+ * Whether this endpoint's models can be handed an image.+ *+ * A separate fact from the dialect, though the address once stood in for+ * both. The dialect is about which EXTENSIONS the request may carry; image+ * blocks are in the core format, so a gateway may well take them — and a+ * gateway that does had no way to say so. Omitted falls back to the guess+ * that fact replaced, for a caller with nothing to declare.+ */+ vision?: boolean;/** How far the conversation may grow before the oldest runs are folded away.* The window knows which provider this is, so it is the side that says. */contextWindow?: number;@@ -127,15 +137,16 @@ export function buildStepRequest(opts: {* happened: a model handed a silently dropped screenshot concludes the editor is* broken, while one told the endpoint cannot carry it asks a different way.*- * Exported because the native branch cannot be exercised against a local- * stand-in gateway — pointing at one is exactly what selects the other dialect.+ * Keyed on `vision` rather than on the dialect, which it once shared: image+ * blocks are core format, so a gateway that takes them is an ordinary case and+ * not an exception to be spotted from the address.*/export function toolResultContent(outcome: ToolOutcome,- dialect: Dialect,+ vision: boolean,): Anthropic.Beta.BetaToolResultBlockParam['content'] {if (!outcome.image) return outcome.content;- if (dialect !== 'anthropic') {+ if (!vision) {return `${outcome.content}\n[the screenshot could not be sent: this endpoint does not accept images]`;}return [@@ -179,7 +190,9 @@ class AnthropicSession implements AgentSession {constructor(private readonly client: Anthropic,- private readonly opts: { model: string; effort: string; dialect: Dialect; contextWindow: number },+ private readonly opts: {+ model: string; effort: string; dialect: Dialect; vision: boolean; contextWindow: number;+ },private readonly system: string,private readonly tools: readonly CatalogTool[],) {@@ -226,7 +239,7 @@ class AnthropicSession implements AgentSession {// than quietly handed the text alone: a model that never learns a picture// was attached answers confidently about something it was not shown, and// the person watching has every reason to think it looked.- if (this.opts.dialect !== 'anthropic') {+ if (!this.opts.vision) {const note = images.length === 1? '[the user attached an image; this endpoint cannot receive images, so you have not seen it]': `[the user attached ${images.length} images; this endpoint cannot receive images, so you have not seen them]`;@@ -269,7 +282,7 @@ class AnthropicSession implements AgentSession {content: outcomes.map((o) => ({type: 'tool_result' as const,tool_use_id: o.id,- content: toolResultContent(o, this.opts.dialect),+ content: toolResultContent(o, this.opts.vision),is_error: o.isError,})),});@@ -468,16 +481,20 @@ export function createAnthropicProvider(options: AnthropicOptions): AgentProvideconst effort = options.effort ?? DEFAULT_EFFORT;const baseURL = options.baseURL || undefined;const dialect: Dialect = baseURL ? 'compatible' : 'anthropic';+ // Declared by the provider that was picked. The fallback is the guess this+ // replaced — right for the API, and the safe way to be wrong for a caller+ // that says nothing, since a claimed sight an endpoint lacks costs the turn.+ const vision = options.vision ?? dialect === 'anthropic';const client = new Anthropic({ apiKey: options.apiKey, baseURL });return {id: dialect === 'anthropic' ? 'anthropic' : `compatible:${baseURL ?? ''}`,model,- // The same condition toolResultContent substitutes on — one fact, one place.- acceptsImages: dialect === 'anthropic',+ // The same flag toolResultContent substitutes on — one fact, one place.+ acceptsImages: vision,createSession: ({ system, tools }, memory) => {const session = new AnthropicSession(client,- { model, effort, dialect, contextWindow: options.contextWindow || DEFAULT_CONTEXT_WINDOW },+ { model, effort, dialect, vision, contextWindow: options.contextWindow || DEFAULT_CONTEXT_WINDOW },system,tools,);diff --git a/desktop/electron/agent/openai.ts b/desktop/electron/agent/openai.tsindex c6d4e8e3d..a8d0bd927 100644--- a/desktop/electron/agent/openai.ts+++ b/desktop/electron/agent/openai.ts@@ -58,6 +58,10 @@ export interface OpenAIOptions {* Whether this endpoint's model can see. Declared rather than probed: finding* out costs a call spent on a screenshot the model will be told it cannot* look at, and the answer is a property of the endpoint, not of the editor.+ *+ * The editor always says (src/agent/providers.ts declares it per provider);+ * omitted falls back to the guess that used to be the only answer — the+ * address — for a caller with nothing to declare.*/vision?: boolean;/**diff --git a/desktop/electron/main.ts b/desktop/electron/main.tsindex d02936227..0c221514b 100644--- a/desktop/electron/main.ts+++ b/desktop/electron/main.ts@@ -690,7 +690,7 @@ ipcMain.handle('secret:clear', (_e, id: string) => clearSecret(id));// settings live in its localStorage) and merged, since the two rows fire// separately; read at session creation, so a change lands on the next// conversation rather than under the one being read.-let agentEndpoint: { protocol?: string; baseUrl?: string; model?: string; keyId?: string; contextWindow?: number; effort?: string } = {};+let agentEndpoint: { protocol?: string; baseUrl?: string; model?: string; keyId?: string; contextWindow?: number; effort?: string; vision?: boolean; reasoningEffort?: boolean } = {};/** Contributed agent tools, as the window last reported them. */let agentPluginTools: ContributedTool[] = [];@@ -718,6 +718,11 @@ const agentHost = createAgentHost({model: agentEndpoint.model,effort,contextWindow: agentEndpoint.contextWindow,+ vision: agentEndpoint.vision,+ // An endpoint that has never heard of `reasoning_effort` refuses the+ // whole request over it, so whether to name the depth is the provider's+ // to declare — the same journey as the window and the sight.+ reasoningEffort: agentEndpoint.reasoningEffort,});}return createAnthropicProvider({@@ -726,6 +731,10 @@ const agentHost = createAgentHost({model: agentEndpoint.model,effort,contextWindow: agentEndpoint.contextWindow,+ // What the provider declared, not what its address implies. The window is+ // the side that knows which provider was picked, so it is the side that+ // says — for this the same as for the model and the window size.+ vision: agentEndpoint.vision,});},// Conversations are kept with the project they are about. With none open@@ -761,7 +770,7 @@ ipcMain.handle('agent:resumeConversation', async (_e, id: string) => {ipcMain.handle('agent:deleteConversation', async (_e, id: string) => {if (projectRoot) await deleteConversation(projectRoot, id);});-ipcMain.handle('agent:setEndpoint', (_e, patch: { protocol?: string; baseUrl?: string; model?: string; keyId?: string; contextWindow?: number; effort?: string }) => {+ipcMain.handle('agent:setEndpoint', (_e, patch: { protocol?: string; baseUrl?: string; model?: string; keyId?: string; contextWindow?: number; effort?: string; vision?: boolean; reasoningEffort?: boolean }) => {agentEndpoint = { ...agentEndpoint, ...patch };// `ready` is derived from which key this points at, so it just changed.agentHost.announce();diff --git a/desktop/electron/preload.ts b/desktop/electron/preload.tsindex 568a62963..4d10c3851 100644--- a/desktop/electron/preload.ts+++ b/desktop/electron/preload.ts@@ -139,7 +139,7 @@ const api = {ipcRenderer.invoke('agent:deleteConversation', id),/** Point the NEXT session at an endpoint/model. Merged, so each settings row* can push its own field. */- setEndpoint: (patch: { protocol?: string; baseUrl?: string; model?: string; keyId?: string; contextWindow?: number; effort?: string }): Promise<void> =>+ setEndpoint: (patch: { protocol?: string; baseUrl?: string; model?: string; keyId?: string; contextWindow?: number; effort?: string; vision?: boolean; reasoningEffort?: boolean }): Promise<void> =>ipcRenderer.invoke('agent:setEndpoint', patch),/** Report what loaded plugins contributed for the agent to call. Metadata* only — the handlers stay in this window, reached through the editordiff --git a/desktop/src/agent/providers.ts b/desktop/src/agent/providers.tsindex b31f515b4..40323a594 100644--- a/desktop/src/agent/providers.ts+++ b/desktop/src/agent/providers.ts@@ -56,6 +56,21 @@ export interface AgentProviderDef {* high is what makes a turn fail outright.*/contextWindow?: number;+ /**+ * Whether a rendered frame can reach this endpoint's models at all.+ *+ * DECLARED, never inferred. It was inferred once — "no baseUrl, therefore the+ * vendor's own API, therefore it can see" — which is only a statement about+ * the address, and it made every gateway blind with no way to say otherwise.+ *+ * Per provider rather than per model, for {@link contextWindow}'s reason: a+ * table with a row per model goes stale a model at a time. Absent means NO,+ * deliberately — the two mistakes are not symmetrical. Claiming sight an+ * endpoint lacks spends the turn on a request it refuses; withholding it costs+ * `screenshot`'s coarse text grid instead of a picture, which still answers+ * "did anything draw at all".+ */+ vision?: boolean;/*** The model list is TYPED, not shipped.*@@ -66,8 +81,24 @@ export interface AgentProviderDef {* shipped, including its key row.*/typedModels?: boolean;- /** Endpoint, protocol AND models all come from settings — see- * {@link CUSTOM_PROVIDER}, the one provider we cannot ship anything for. */+ /**+ * Send the reasoning-depth parameter with each request.+ *+ * Only the OpenAI protocol carries it as a named argument (`reasoning_effort`),+ * and an endpoint that has never heard of one refuses the whole request rather+ * than ignoring the field — so for a gateway this has to be a switch, not an+ * assumption. Absent means yes, which is right for every endpoint that+ * implements the parameter and is the behaviour that predates the field.+ */+ reasoningEffort?: boolean;+ /**+ * This def came from the person's own table rather than from this file.+ *+ * Every OTHER field means the same thing whichever way it arrived — that is+ * the point of projecting the table into this registry instead of keeping it+ * beside it. This one exists so a reader that must not offer to edit a shipped+ * provider can tell the two apart.+ */userDefined?: boolean;}@@ -76,8 +107,18 @@ export { DEFAULT_CONTEXT_WINDOW } from '@/settings/agentIds';/** The id a provider's key is stored under (electron/secrets.ts). */export const agentKeyId = (providerId: string): string => `agents.key.${providerId}`;+/**+ * The id the ONE custom provider had, back when there could only be one.+ *+ * Kept because a key is filed under its provider's id: the row that setup+ * becomes on upgrade has to carry this id, or the credential already sealed on+ * the machine belongs to a provider that no longer exists (agent/userProviders.ts).+ */export const CUSTOM_PROVIDER = 'custom';+/** Who owns the providers the person typed — see {@link setUserProviders}. */+export const USER_OWNER = 'user';+/** A provider whose models the user types. Its list is per provider, so two of* them do not share one box. */export const modelsSettingId = (providerId: string): string => `agents.models.${providerId}`;@@ -91,6 +132,7 @@ registry.registerAll('core', [baseUrl: '',models: ['claude-opus-5', 'claude-sonnet-5', 'claude-haiku-4-5'],contextWindow: 200_000,+ vision: true,},{id: 'deepseek',@@ -107,16 +149,11 @@ registry.registerAll('core', [models: [],typedModels: true,contextWindow: 400_000,- },- // The escape hatch. Its endpoint, protocol and model list come from settings,- // because a provider we have not heard of is exactly the one we cannot ship- // a list for — and Chat Completions is what most of those speak.- {- id: CUSTOM_PROVIDER,- label: '',- baseUrl: '',- models: [],- userDefined: true,+ // Its models are typed, so this is a claim about the ENDPOINT: it carries+ // image parts. A text-only model named into that box is the same staleness+ // `contextWindow` accepts, and it degrades the same way — the model is told+ // a picture it cannot see was attached.+ vision: true,},]);@@ -131,6 +168,24 @@ export const providersRevision = (): number => registry.getRevision();export const registerAgentProvider = (def: AgentProviderDef, owner: Owner = 'core'): Disposable =>registry.register(owner, def);+/**+ * Replace the set of providers the person defined themselves.+ *+ * They arrive as a SET rather than one at a time because that is how they are+ * edited — a table, rewritten whole on each keystroke — and because the removals+ * matter as much as the additions: a provider deleted from the table must stop+ * being offered in the picker, and dispose-then-register is the only way to say+ * that without diffing. Everything downstream reads them through the same door+ * as the shipped ones; nothing but this function knows they were typed.+ *+ * Kept here, taking finished defs, so this module stays free of the settings+ * store — the projection from one to the other lives in agent/userProviders.ts.+ */+export function setUserProviders(defs: readonly AgentProviderDef[]): void {+ registry.disposeOwner(USER_OWNER);+ registry.registerAll(USER_OWNER, defs.map((d) => ({ ...d, userDefined: true })));+}+export const AGENT_PROTOCOLS: readonly AgentProtocol[] = ['openai', 'anthropic'];/** What an endpoint speaks, defaulting to the format that predates the field. */diff --git a/desktop/src/agent/userProviders.ts b/desktop/src/agent/userProviders.tsnew file mode 100644index 000000000..308751493--- /dev/null+++ b/desktop/src/agent/userProviders.ts@@ -0,0 +1,187 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file userProviders.ts — the providers a person defined, projected into the+ * same registry the shipped ones live in.+ *+ * There used to be exactly one, spelled out as four global settings+ * (`agents.customProtocol`, `…BaseUrl`, `…Models`, plus one key). That+ * contradicted the rule the shipped providers are built on — the unit of+ * configuration is a PROVIDER, and every one keeps its own key so switching back+ * to the one you used last week costs nothing. A local runner and a company+ * gateway could not both exist: configuring the second overwrote the first,+ * credential included.+ *+ * So they are a LIST now, and each row is a provider like any other: address,+ * protocol, models, its own key, and what it can do. This module is the only+ * place that knows a provider can come from a table — providers.ts stays a+ * registry, and every reader downstream sees one kind of provider.+ */+import { useSettings, dropPersisted } from '@/store/settingsStore';+import {+ type AgentProviderDef, agentKeyId, asProtocol, parseModelList, setUserProviders,+ CUSTOM_PROVIDER,+} from '@/agent/providers';+import { forgetSecret } from '@/store/SecretStore';+import { DEFAULT_CONTEXT_WINDOW } from '@/settings/agentIds';+import { t } from '@/i18n';++export const AGENT_PROVIDERS_SETTING = 'agents.providers';++/**+ * One row of that table, as it is stored.+ *+ * Flat and all-scalar because it IS the edited shape — the settings table writes+ * these fields directly, so a nested or derived form here would be a translation+ * layer between the control and the file. `models` is the raw text the person+ * typed for the same reason: splitting it on save would make the field fight the+ * cursor halfway through a comma.+ */+export interface UserProviderRow {+ /** Stable for the row's life — the key is filed under it. Never re-used. */+ id: string;+ label: string;+ protocol: string;+ baseUrl: string;+ models: string;+ vision: boolean;+ /** Left blank on a new row, and blank means {@link DEFAULT_CONTEXT_WINDOW} —+ * a zero typed into the cell reads the same way. */+ contextWindow?: number;+ reasoningEffort: boolean;+}++const str = (v: unknown): string => (typeof v === 'string' ? v : '');++/**+ * What to call a provider whose name was left blank.+ *+ * The picker groups models under this, so an empty string there is a heading of+ * nothing above a list of models with no way to tell whose they are. The host is+ * the better guess than a generic word: someone running two local models knows+ * their ports.+ */+function fallbackLabel(baseUrl: string): string {+ try {+ return new URL(baseUrl).host || t('agent.picker.custom');+ } catch {+ return t('agent.picker.custom');+ }+}++/**+ * A stored row as the rest of the editor sees it, or null for one that cannot be+ * used yet — a row still being typed is an ordinary state, not an error, and the+ * table says what is missing (see the setting's `rowError`).+ */+export function rowToDef(row: Record<string, unknown>): AgentProviderDef | null {+ const id = str(row.id).trim();+ if (!id) return null;+ const baseUrl = str(row.baseUrl).trim();+ const contextWindow = Number(row.contextWindow);+ return {+ id,+ label: str(row.label).trim() || fallbackLabel(baseUrl),+ protocol: asProtocol(row.protocol),+ baseUrl,+ models: parseModelList(str(row.models)),+ vision: row.vision === true,+ // Absent means yes for a shipped provider, so only an explicit `false` from+ // the table turns it off — a row written before the field existed must not+ // read as "this endpoint refuses the parameter".+ reasoningEffort: row.reasoningEffort !== false,+ contextWindow: contextWindow > 0 ? contextWindow : DEFAULT_CONTEXT_WINDOW,+ userDefined: true,+ };+}++/** The table, as stored. */+export const userProviderRows = (): Record<string, unknown>[] =>+ useSettings.getState().getValue<Record<string, unknown>[]>(AGENT_PROVIDERS_SETTING) ?? [];++/** Project the table into the provider registry. Called on every edit to it. */+export const syncUserProviders = (): void => {+ const defs: AgentProviderDef[] = [];+ for (const row of userProviderRows()) {+ const def = rowToDef(row);+ if (def) defs.push(def);+ }+ setUserProviders(defs);+};++/**+ * A row for a provider being added, with an id no earlier row has held.+ *+ * Monotonic rather than "first free": ids that come back around would inherit+ * whatever is still filed under them, and a key belonging to a provider that was+ * deleted is not one to hand to its replacement. {@link forgetProviderSecret}+ * closes the same hole from the other side.+ */+export function newProviderRow(rows: readonly Record<string, unknown>[]): UserProviderRow {+ let next = 1;+ for (const row of rows) {+ const m = /^custom-(\d+)$/.exec(str(row.id));+ if (m) next = Math.max(next, Number(m[1]) + 1);+ }+ return {+ id: `custom-${next}`,+ label: '',+ // What most gateways and every local runner implement. Getting this wrong is+ // refused rather than degraded, so the likelier answer is the better default.+ protocol: 'openai',+ baseUrl: '',+ models: '',+ vision: false,+ // Left out rather than zeroed: the cell then shows the default it would use+ // as its placeholder, instead of a 0 that reads like a window of nothing.+ reasoningEffort: true,+ };+}++/** The legacy singleton's settings, folded into the table on first run. */+const LEGACY_IDS = [+ 'agents.customProtocol', 'agents.customBaseUrl', 'agents.customModels', 'agents.customVision',+] as const;++/**+ * Carry a pre-table setup across, once.+ *+ * The row takes {@link CUSTOM_PROVIDER} as its id on purpose: a key is filed+ * under its provider's id, and this is the one migration step that cannot be+ * repaired by hand — main never hands a sealed credential back, so an id that+ * did not carry over is a key the person has to find again. The saved model pick+ * (`estella.agent.selection`) names the same id and so keeps resolving too.+ *+ * Runs only while the table has never been written. A person who has since+ * deleted every row means it, and re-adding what they removed on each boot is+ * the worst way for a migration to be wrong.+ */+export function migrateLegacyCustomProvider(): void {+ const { values } = useSettings.getState();+ if (values[AGENT_PROVIDERS_SETTING] !== undefined) return;+ const baseUrl = str(values['agents.customBaseUrl']).trim();+ const models = str(values['agents.customModels']).trim();+ if (!baseUrl && !models) {+ // Nothing was configured, but the legacy ids may still sit in the file as+ // defaults someone toggled and toggled back.+ dropPersisted(LEGACY_IDS);+ return;+ }+ const migrated: UserProviderRow = {+ id: CUSTOM_PROVIDER,+ label: t('agent.picker.custom'),+ protocol: asProtocol(values['agents.customProtocol']),+ baseUrl,+ models,+ vision: values['agents.customVision'] === true,+ reasoningEffort: true,+ };+ useSettings.getState().setValue(AGENT_PROVIDERS_SETTING, [migrated as unknown as Record<string, unknown>]);+ dropPersisted(LEGACY_IDS);+}++/** Let go of the credential a removed provider held — see {@link newProviderRow}. */+export function forgetProviderSecret(row: Record<string, unknown>): void {+ const id = str(row.id).trim();+ if (id) void forgetSecret(agentKeyId(id));+}diff --git a/desktop/src/components/AgentDrawer.tsx b/desktop/src/components/AgentDrawer.tsxindex 2629e1f03..0872b7f27 100644--- a/desktop/src/components/AgentDrawer.tsx+++ b/desktop/src/components/AgentDrawer.tsx@@ -26,7 +26,7 @@import { memo, useCallback, useEffect, useLayoutEffect, useRef, useState, useSyncExternalStore } from 'react';import {X, Plus, PanelRight, ArrowUp, Square, ChevronRight, ChevronDown, Check, TriangleAlert,- Loader, Copy, Pencil, Eye, KeyRound, Boxes, Stethoscope, Image as ImageIcon, RotateCcw,+ Loader, Copy, Pencil, Eye, KeyRound, Boxes, Stethoscope, Image as ImageIcon, ImageOff, RotateCcw,File as FileIcon, ArrowRight, History as HistoryIcon, Trash2, FoldVertical,} from 'lucide-react';import { useEditorStore } from '@/store/editorStore';@@ -34,13 +34,13 @@ import {useAgent, agentEffort, sendAgentMessage, stopAgentTurn, confirmAgentCall, startNewConversation,peekEntities, entitiesInInput, effectiveSelection, selectAgentModel, retryAgentTurn, setAgentDraft,RESUMABLE, openAgentHistory, closeAgentHistory, resumeConversation, forgetConversation,- addAgentAttachments, removeAgentAttachment, latestContext,+ addAgentAttachments, removeAgentAttachment, latestContext, agentAcceptsImages, resolveProvider,type AgentTurn, type AgentEntry, type AgentToolEntry, type AgentProseEntry,} from '@/store/AgentStore';import { COMPACT_AT, shouldCompact } from '@/settings/agentIds';import type { ConfirmAnswer } from '../../electron/agent/types';import {- agentProviders, agentProvider, agentKeyId, parseModelList, subscribeProviders, providersRevision,+ agentProviders, agentKeyId, subscribeProviders, providersRevision,} from '@/agent/providers';import { secretStatus, subscribeSecrets, secretRevision } from '@/store/SecretStore';import { useSettings } from '@/store/settingsStore';@@ -952,7 +952,11 @@ function ModelPicker() {{open && (<div className="ag-picker-menu">{providers.map((p) => {- const def = resolveProviderModels(p.id);+ // Its models, with the typed list filled in for the one vendor whose+ // names are not shippable. A provider the person defined arrives+ // whole — nothing here can tell it from a built-in, which is the+ // point of projecting the table into the same registry.+ const def = resolveProvider(p.id) ?? p;const keyed = secretStatus(agentKeyId(p.id))?.configured === true;if (def.models.length === 0 && !def.label) return null;return (@@ -992,19 +996,6 @@ function ModelPicker() {const openAgentSettings = () => useEditorStore.getState().openSettings('agents');-/** The custom provider's list comes from settings; the rest ship theirs. */-function resolveProviderModels(id: string): { label: string; models: readonly string[] } {- const def = agentProvider(id);- if (!def) return { label: '', models: [] };- if (def.userDefined) {- return {- label: t('agent.picker.custom'),- models: parseModelList(String(useSettings.getState().getValue('agents.customModels') ?? '')),- };- }- return { label: def.label, models: def.models };-}-/** One thing `@` can name: something in the scene, or something on disk. */interface Mention {key: string;@@ -1185,6 +1176,14 @@ function Compose({ autoFocus }: { autoFocus?: boolean }) {)}{/* What is going along with the message, as the picture itself: a row offilenames would be a list of things you cannot check you picked. */}+ {/* An endpoint that cannot carry them still takes the turn — the model is+ told a picture was attached rather than handed the text alone — but+ that is a thing to learn before pressing send, not from the reply. */}+ {attachments.length > 0 && !agentAcceptsImages() && (+ <button type="button" className="ag-att-blind" onClick={openAgentSettings}>+ <ImageOff size={12} strokeWidth={1.8} />{t('agent.attach.blind')}+ </button>+ )}{attachments.length > 0 && (<div className="ag-atts">{attachments.map((a) => (diff --git a/desktop/src/contrib/ContributionRegistry.ts b/desktop/src/contrib/ContributionRegistry.tsindex 881cdfa43..a34970f55 100644--- a/desktop/src/contrib/ContributionRegistry.ts+++ b/desktop/src/contrib/ContributionRegistry.ts@@ -19,8 +19,16 @@* are recorded in {@link ContributionRegistry.conflicts} for the Plugins panel.*/-/** Who owns a contribution. `'core'` is the editor itself. */-export type Owner = 'core' | `plugin:${string}`;+/**+ * Who owns a contribution. `'core'` is the editor itself.+ *+ * `'user'` is the person's own configuration — a set they edit in Settings that+ * is projected into a registry rather than kept beside it (agent providers are+ * the first). It disposes and re-registers as a group on every edit, which is+ * the same operation a plugin reload performs, so it belongs here rather than in+ * a parallel list that every reader would then have to remember to merge.+ */+export type Owner = 'core' | 'user' | `plugin:${string}`;export interface Disposable {dispose(): void;diff --git a/desktop/src/i18n/messages/agent.ts b/desktop/src/i18n/messages/agent.tsindex 8a3fa782a..65d101335 100644--- a/desktop/src/i18n/messages/agent.ts+++ b/desktop/src/i18n/messages/agent.ts@@ -32,6 +32,10 @@ export const agentMessages = defineMessages({// — Images the person attaches —'agent.attach.remove': { en: 'Remove', zh: '移除' },'agent.attach.hint': { en: 'drop or paste an image', zh: '拖入或粘贴图片' },+ 'agent.attach.blind': {+ en: 'This endpoint does not accept images — it will be told one was attached, not shown it. Settings › AI Agents.',+ zh: '当前接口不接受图片——只会告诉模型「有人附了一张图」,但它看不到。可在「设置 › AI 代理」里调整。',+ },// — Conversations this project has had —'agent.history.open': { en: 'Earlier conversations', zh: '历史对话' },diff --git a/desktop/src/i18n/messages/settings.ts b/desktop/src/i18n/messages/settings.tsindex b39cfb24a..6b2f307ec 100644--- a/desktop/src/i18n/messages/settings.ts+++ b/desktop/src/i18n/messages/settings.ts@@ -100,6 +100,7 @@ export const settingsMessages = defineMessages({'set.secret.stored': { en: 'Stored', zh: '已配置' },'set.secret.store': { en: 'Store it', zh: '保存' },'set.secret.forget': { en: 'Forget it', zh: '清除' },+ 'set.secret.needed': { en: 'Add key', zh: '添加密钥' },'set.secret.noKeychain': {en: 'This machine has no keychain to encrypt with, so nothing can be stored here. On Linux, install GNOME Keyring or KWallet, or start the editor with --password-store=gnome-libsecret.',zh: '这台机器没有可用于加密的密钥库,因此无法保存。Linux 上请安装 GNOME Keyring 或 KWallet,或以 --password-store=gnome-libsecret 启动编辑器。',@@ -119,7 +120,7 @@ export const settingsMessages = defineMessages({en: 'Held by the system keychain — never written to the project or the settings file, and never handed back to this window. Every provider keeps its own, so switching back to one costs nothing.',zh: '存在系统钥匙串里——不写进项目、不写进设置文件,也不会交回给这个窗口。每个提供方各存各的,切回来不用重输。',},- 'set.group.customProvider': { en: 'Custom Provider', zh: '自定义提供方' },+ 'set.group.customProvider': { en: 'Your providers', zh: '你的提供方' },'set.agents.effort': { en: 'Reasoning depth', zh: '思考深度' },'set.agents.effort.desc': {en: 'How hard the model is asked to think before it acts. `xhigh` is what agentic work is for and is the default; drop it when a turn costs more or takes longer than the task deserved. Applies to the next conversation.',@@ -130,24 +131,37 @@ export const settingsMessages = defineMessages({en: 'Which models to offer in the composer, comma- or newline-separated. Typed rather than shipped: a list here would be right until the vendor\'s next release, and a name that no longer exists is not refused — the endpoint serves something smaller instead, for the rest of the session.',zh: '在输入框旁边可选的型号,用逗号或换行分隔。要自己填而不是内置:内置的列表只能正确到厂商下次发布为止,而写了个已经不存在的名字并不会被拒绝——接口会改用一个更小的模型顶上,并且一整个会话都是如此。',},- 'set.agents.customProtocol': { en: 'Protocol', zh: '协议' },- 'set.agents.customProtocol.desc': {- en: 'Which API this endpoint speaks. The two formats differ in every message they send, so a wrong pick is refused rather than degraded. Chat Completions is what most gateways and every local runner implement.',- zh: '这个接口讲哪种 API。两种格式发出的每一条消息都不一样,所以选错会被直接拒绝,而不是降级运行。绝大多数网关和所有本地运行器讲的都是 Chat Completions。',- },'set.agents.protocol.openai': { en: 'OpenAI (Chat Completions)', zh: 'OpenAI(Chat Completions)' },'set.agents.protocol.anthropic': { en: 'Anthropic (Messages)', zh: 'Anthropic(Messages)' },- 'set.agents.customBaseUrl': { en: 'Endpoint', zh: 'API 地址' },- 'set.agents.customBaseUrl.desc': {- en: 'The address of an endpoint speaking the protocol above. Only needed for a provider not in the list — the built-in ones already know theirs. On the Anthropic protocol the agent then sends only the core Messages format, without extended thinking, prompt caching or automatic fallbacks.',- zh: '讲上面那个协议的接口地址。只有列表里没有的提供方才需要填——内置的那些自己知道地址。走 Anthropic 协议时,Agent 只发送基础 Messages 格式,不带扩展思考、提示缓存和自动回退。',- },- 'set.agents.customModels': { en: 'Models', zh: '模型' },- 'set.agents.customModels.desc': {- en: 'The model names this endpoint offers, comma- or newline-separated. They become choices in the composer.',- zh: '这个接口提供的模型名,用逗号或换行分隔。它们会出现在输入框的模型选择器里。',+ // In a table cell the long form is truncated to something no shorter and far+ // less readable; which format each name means is the row's description.+ 'set.agents.protocolShort.openai': { en: 'OpenAI', zh: 'OpenAI' },+ 'set.agents.protocolShort.anthropic': { en: 'Anthropic', zh: 'Anthropic' },+ 'set.agents.providers': { en: 'Endpoints', zh: '接口' },+ 'set.agents.providers.desc': {+ en: 'Endpoints of your own — a local runner, a company gateway, a vendor not in the list above. Each keeps its own key and appears in the composer under its own name. The expander holds what the endpoint can DO: what it cannot be told, the agent has to work around.',+ zh: '你自己的接口——本地运行器、公司网关,或者上面列表里没有的厂商。每一条各存各的密钥,并以自己的名字出现在输入框的模型选择器里。展开项里是这个接口「能做什么」:没告诉它的部分,Agent 只能绕着走。',+ },+ 'set.agents.providers.empty': {+ en: 'None yet. The built-in providers above cover the major vendors; add one here for anything else that speaks either protocol.',+ zh: '还没有。上面的内置提供方覆盖了主要厂商;讲这两种协议中任意一种的其它接口,在这里添加。',+ },+ 'set.agents.addProvider': { en: 'Add a provider', zh: '添加提供方' },+ 'set.agents.col.label': { en: 'Name', zh: '名称' },+ 'set.agents.col.protocol': { en: 'Protocol', zh: '协议' },+ 'set.agents.col.baseUrl': { en: 'Endpoint', zh: 'API 地址' },+ 'set.agents.col.models': { en: 'Models', zh: '模型' },+ 'set.agents.col.key': { en: 'Key', zh: '密钥' },+ 'set.agents.capabilities': { en: 'What it can do', zh: '它能做什么' },+ 'set.agents.col.contextWindow': { en: 'Context window', zh: '上下文窗口' },+ 'set.agents.col.reasoningEffort': { en: 'Takes a reasoning-depth argument', zh: '接受思考深度参数' },+ 'set.agents.err.baseUrl': { en: 'This provider has no endpoint address.', zh: '这个提供方还没有填接口地址。' },+ 'set.agents.err.models': { en: 'No models, so nothing to pick in the composer.', zh: '没有填模型,输入框里就没有可选项。' },+ 'set.agents.customVision': { en: 'Accepts images', zh: '接受图片' },+ 'set.agents.customVision.desc': {+ en: 'Turn on only if this endpoint\'s models can see. The agent takes screenshots to check its own work; where they cannot be sent it asks for the same frame as a coarse text grid instead, and says so in the transcript. Claiming sight an endpoint lacks costs a whole turn to a refused request, which is why this starts off.',+ zh: '只有这个接口的模型确实能看图时才打开。Agent 会截图来检查自己干的活;发不出去图片时,它会改成把同一帧要成粗颗粒的文字色块网格,并在对话里说明。给一个其实看不了图的接口打开它,代价是整轮请求被拒——所以默认是关的。',},- 'set.agents.customKey': { en: 'Custom API key', zh: '自定义提供方密钥' },'set.agents.mcpEnabled': { en: 'Allow AI agents to connect', zh: '允许 AI 代理连接' },'set.agents.mcpEnabled.desc': {en: 'Serve this editor over MCP on a local-only port, so an agent set up with --attach can drive the project you have open. Off means nothing is listening.',diff --git a/desktop/src/settings/editorSettings.ts b/desktop/src/settings/editorSettings.tsindex db9247e9b..b98e7a7ef 100644--- a/desktop/src/settings/editorSettings.ts+++ b/desktop/src/settings/editorSettings.ts@@ -20,10 +20,15 @@ import { Toasts } from '@/store/Toasts';import { mcpStatus, setMcpEnabled, subscribeMcp } from '@/store/McpStore';import { secretStatusLine, subscribeSecrets } from '@/store/SecretStore';import { syncAgentEndpoint } from '@/store/AgentStore';+import { useSettings } from '@/store/settingsStore';import {- agentProviders, agentKeyId, modelsSettingId, AGENT_PROTOCOLS, CUSTOM_PROVIDER,+ agentProviders, agentKeyId, modelsSettingId, AGENT_PROTOCOLS, parseModelList,} from '@/agent/providers';-import { AGENT_EFFORTS, DEFAULT_EFFORT } from '@/settings/agentIds';+import {+ AGENT_PROVIDERS_SETTING, syncUserProviders, newProviderRow, forgetProviderSecret,+ migrateLegacyCustomProvider,+} from '@/agent/userProviders';+import { AGENT_EFFORTS, DEFAULT_EFFORT, DEFAULT_CONTEXT_WINDOW } from '@/settings/agentIds';import { applyUiZoom, UI_SCALE_SETTING, ZOOM_DEFAULT, ZOOM_MIN, ZOOM_MAX } from '@/layout/uiZoom';import { t, type MsgKey, editorLocale, systemDefaultLocale, EDITOR_LOCALES, LANGUAGE_SETTING_ID } from '@/i18n';@@ -290,62 +295,68 @@ settingsRegistry.register({effect: () => syncAgentEndpoint(),});-// The escape hatch, for a provider we have not heard of — which is exactly the-// one we cannot ship an endpoint or a model list for.+// Providers we have not heard of — which are exactly the ones we cannot ship an+// address, a model list, or a word about what they accept.//-// Which PROTOCOL it speaks has to be asked, not guessed: the two formats differ-// in every message they send, so getting it wrong is a 400 about a field the-// person never wrote. Chat Completions is offered first because it is what most-// endpoints — and every local runner — implement.-settingsRegistry.register({- id: 'agents.customProtocol',- type: 'enum',- scope: 'editor',- section: 'agents',- group: t('set.group.customProvider'),- label: t('set.agents.customProtocol'),- description: t('set.agents.customProtocol.desc'),- default: 'openai',- options: AGENT_PROTOCOLS.map((value) => ({ value, label: t(`set.agents.protocol.${value}` as MsgKey) })),- effect: () => syncAgentEndpoint(),-});--settingsRegistry.register({- id: 'agents.customBaseUrl',- type: 'string',- scope: 'editor',- section: 'agents',- group: t('set.group.customProvider'),- label: t('set.agents.customBaseUrl'),- description: t('set.agents.customBaseUrl.desc'),- placeholder: 'https://example.com/v1',- default: '',- effect: () => syncAgentEndpoint(),-});-+// A TABLE, not a set of fields, because the unit of configuration is a provider+// and a person has more than one: a local runner and a company gateway are not+// two states of one setting. Each row keeps its own key, for the same reason the+// shipped rows above do.+//+// The main columns say where it is and what it speaks; the expander says what it+// can DO. That split is the point — an endpoint that could only be given an+// address had no way to tell the agent it accepts screenshots, and the agent+// spent every session working blind without either side deciding that.settingsRegistry.register({- id: 'agents.customModels',- type: 'string',+ id: AGENT_PROVIDERS_SETTING,+ type: 'objectList',scope: 'editor',section: 'agents',group: t('set.group.customProvider'),- label: t('set.agents.customModels'),- description: t('set.agents.customModels.desc'),- placeholder: 'model-a, model-b',- default: '',- effect: () => syncAgentEndpoint(),+ label: t('set.agents.providers'),+ description: t('set.agents.providers.desc'),+ layout: 'block',+ default: [],+ columns: [+ { key: 'label', label: t('set.agents.col.label'), type: 'text', width: '0.9fr', placeholder: 'Local llama' },+ {+ key: 'protocol',+ label: t('set.agents.col.protocol'),+ type: 'enum',+ width: '108px',+ options: AGENT_PROTOCOLS.map((value) => ({ value, label: t(`set.agents.protocolShort.${value}` as MsgKey) })),+ },+ { key: 'baseUrl', label: t('set.agents.col.baseUrl'), type: 'text', width: '1.6fr', placeholder: 'http://localhost:11434/v1' },+ { key: 'models', label: t('set.agents.col.models'), type: 'text', width: '1.1fr', placeholder: 'model-a, model-b' },+ { key: 'apiKey', label: t('set.agents.col.key'), type: 'secret', width: '96px', secretId: (row) => agentKeyId(String(row.id ?? '')) },+ ],+ detailColumns: [+ { key: 'vision', label: t('set.agents.customVision'), type: 'boolean' },+ { key: 'contextWindow', label: t('set.agents.col.contextWindow'), type: 'number', width: '104px', min: 0, placeholder: String(DEFAULT_CONTEXT_WINDOW) },+ { key: 'reasoningEffort', label: t('set.agents.col.reasoningEffort'), type: 'boolean' },+ ],+ detailLabel: t('set.agents.capabilities'),+ addLabel: t('set.agents.addProvider'),+ emptyHint: t('set.agents.providers.empty'),+ newRow: () => newProviderRow(+ useSettings.getState().getValue<Record<string, unknown>[]>(AGENT_PROVIDERS_SETTING) ?? [],+ ) as unknown as Record<string, unknown>,+ rowError: (row) => {+ if (!String(row.baseUrl ?? '').trim()) return t('set.agents.err.baseUrl');+ if (parseModelList(String(row.models ?? '')).length === 0) return t('set.agents.err.models');+ return null;+ },+ // A key outlives the row it belonged to otherwise: sealed on the machine,+ // unreachable from any UI, and waiting to be inherited by a later provider.+ onRowRemoved: forgetProviderSecret,+ effect: () => {+ syncUserProviders();+ syncAgentEndpoint();+ },});-settingsRegistry.register({- id: agentKeyId(CUSTOM_PROVIDER),- type: 'secret',- scope: 'editor',- section: 'agents',- group: t('set.group.customProvider'),- label: t('set.agents.customKey'),- default: false,- status: { read: () => secretStatusLine(agentKeyId(CUSTOM_PROVIDER)), subscribe: subscribeSecrets },-});+// Fold a pre-table setup into the table, before anything reads either.+migrateLegacyCustomProvider();// ── External agents (main-owned endpoint, driven from here) ─────────────────// An external agent reaches this editor through the MCP endpoint main can expose,diff --git a/desktop/src/store/AgentStore.ts b/desktop/src/store/AgentStore.tsindex 09352e73c..b1a8e54b4 100644--- a/desktop/src/store/AgentStore.ts+++ b/desktop/src/store/AgentStore.ts@@ -20,9 +20,8 @@ import { create } from 'zustand';import type { AgentStatus, AgentMessage } from '../../electron/agent/host';import type { AgentEvent, ConfirmAnswer, ConfirmReason, ConfirmRequest } from '../../electron/agent/types';import {- agentProviders, agentProvider, agentKeyId, parseModelList, protocolOf, asProtocol,- modelsSettingId, CUSTOM_PROVIDER,- DEFAULT_CONTEXT_WINDOW,+ agentProviders, agentProvider, agentKeyId, parseModelList, protocolOf,+ modelsSettingId, DEFAULT_CONTEXT_WINDOW, type AgentProviderDef,} from '@/agent/providers';import { refreshSecret, secretStatus, subscribeSecrets } from '@/store/SecretStore';import { useSettings } from '@/store/settingsStore';@@ -276,26 +275,34 @@ function loadSelection(): AgentSelection | null {}}-/** The custom provider's endpoint and models come from settings — it is the one- * we cannot ship a list for. */-function resolveProvider(id: string) {+/**+ * A provider, with the one part of it that cannot be shipped filled in.+ *+ * Providers the person typed need nothing here — they are whole defs by the time+ * they reach the registry (agent/userProviders.ts), which is the point of+ * projecting them into it. What is left is the shipped vendor whose ADDRESS+ * holds still while its model names do not.+ */+export function resolveProvider(id: string): AgentProviderDef | undefined {const def = agentProvider(id);- if (!def) return undefined;- const settings = useSettings.getState();- // A shipped provider whose model NAMES are not shippable — everything else- // about it still is, including its endpoint and its key row.- if (def.typedModels) {- return { ...def, models: parseModelList(String(settings.getValue(modelsSettingId(id)) ?? '')) };- }- if (id !== CUSTOM_PROVIDER) return def;- return {- ...def,- protocol: asProtocol(settings.getValue('agents.customProtocol')),- baseUrl: String(settings.getValue('agents.customBaseUrl') ?? ''),- models: parseModelList(String(settings.getValue('agents.customModels') ?? '')),- };+ if (!def?.typedModels) return def;+ const typed = String(useSettings.getState().getValue(modelsSettingId(id)) ?? '');+ return { ...def, models: parseModelList(typed) };}+/**+ * Whether the picked endpoint can be handed a picture.+ *+ * The composer takes attachments whatever is selected, and an endpoint that+ * cannot carry them is not a failure — the turn goes out with the text and the+ * model is told what it was not shown. But that is a thing to learn BEFORE+ * sending, so this is read where the attachment is made too.+ */+export const agentAcceptsImages = (): boolean => {+ const selection = effectiveSelection();+ return selection ? resolveProvider(selection.providerId)?.vision === true : false;+};+/** The configured reasoning depth, narrowed. */export const agentEffort = (): AgentEffort =>asEffort(useSettings.getState().getValue('agents.effort'));@@ -335,6 +342,14 @@ export function syncAgentEndpoint(): void {// How far the conversation may grow before it is compacted. Travels with the// endpoint because it is the same piece of knowledge: which provider this is.contextWindow: def?.contextWindow ?? DEFAULT_CONTEXT_WINDOW,+ // Whether a screenshot can reach it. Same journey as the window and for the+ // same reason: both are facts about which provider this is, and both were+ // once guessed from the address instead of being told.+ vision: def?.vision === true,+ // Whether the depth below may be NAMED in the request. Its own fact because+ // an endpoint that has never heard of the argument refuses the call rather+ // than ignoring it — losing the turn over a field nobody asked for.+ reasoningEffort: def?.reasoningEffort !== false,// How hard the model is asked to think. A setting rather than part of the// model pick: the same model is worth running at different depths, and the// depth is the one thing a person adjusts because a turn cost too much ordiff --git a/desktop/src/theme/agent.css b/desktop/src/theme/agent.cssindex 98c201585..ef592550b 100644--- a/desktop/src/theme/agent.css+++ b/desktop/src/theme/agent.css@@ -1069,6 +1069,21 @@transition: color var(--t-fast), background var(--t-fast);}.ag-att-x:hover { color: var(--text-hi); background: var(--pressed); }+/* The picture will not reach the model. Said before send, next to the thumbnail+ it is about, and it leads to the setting that decides it. */+.ag-att-blind {+ display: flex; align-items: flex-start; gap: 6px;+ margin: 0 0 6px;+ padding: 5px 8px;+ border-radius: var(--r-sm);+ background: color-mix(in srgb, var(--warn) 12%, transparent);+ color: var(--text-dim);+ font: var(--fs-2xs) var(--font-ui);+ text-align: left;+ line-height: 1.45;+}+.ag-att-blind:hover { color: var(--text-hi); }+.ag-att-blind svg { color: var(--warn); flex: none; margin-top: 1px; }/* A drag is over the composer: the whole box says so, since the whole box takesit — aiming at a one-line textarea is a precision task nobody should be set. */.ag-compose.dropping .ag-cbox { border-color: var(--acc); background: var(--hover); }diff --git a/desktop/tests/agent-dialect.test.ts b/desktop/tests/agent-dialect.test.tsindex c719fa23f..accc5ddc4 100644--- a/desktop/tests/agent-dialect.test.ts+++ b/desktop/tests/agent-dialect.test.ts@@ -143,8 +143,8 @@ describe('a tool result carrying a rendered frame', () => {image: { data: 'BASE64PNG', mediaType: 'image/png' },};- it('sends the image itself to Anthropic', () => {- const content = toolResultContent(shot, 'anthropic');+ it('sends the image itself where the endpoint takes one', () => {+ const content = toolResultContent(shot, true);expect(Array.isArray(content)).toBe(true);expect((content as { type: string }[])[0]).toMatchObject({type: 'image',@@ -152,16 +152,48 @@ describe('a tool result carrying a rendered frame', () => {});});- it('tells a gateway why there is no image rather than dropping it in silence', () => {- const content = toolResultContent(shot, 'compatible');+ it('tells a blind endpoint why there is no image rather than dropping it in silence', () => {+ const content = toolResultContent(shot, false);expect(typeof content).toBe('string');expect(content).toContain('does not accept images');});- it('leaves an ordinary result a plain string on both', () => {+ it('leaves an ordinary result a plain string either way', () => {const plain = { id: 'c2', content: '7 created', isError: false };- expect(toolResultContent(plain, 'anthropic')).toBe('7 created');- expect(toolResultContent(plain, 'compatible')).toBe('7 created');+ expect(toolResultContent(plain, true)).toBe('7 created');+ expect(toolResultContent(plain, false)).toBe('7 created');+ });+});++/**+ * Sight is DECLARED, not read off the address.+ *+ * It was read off the address once, and that made every gateway blind with no+ * way to say otherwise — including ones whose models see perfectly well, since+ * image blocks are core format and not one of the extensions the dialect gates.+ */+describe('whether the endpoint is told to be blind', () => {+ const provider = (over = {}) => createAnthropicProvider({ apiKey: 'k', ...over });+ const gateway = 'https://gateway.example/anthropic';++ it('takes the provider at its word, over what the address implies', () => {+ expect(provider({ baseURL: gateway, vision: true }).acceptsImages).toBe(true);+ expect(provider({ vision: false }).acceptsImages).toBe(false);+ });++ it('falls back to the address for a caller that declares nothing', () => {+ expect(provider().acceptsImages).toBe(true);+ expect(provider({ baseURL: gateway }).acceptsImages).toBe(false);+ });++ // The dialect still gates the EXTENSIONS — a gateway that sees is not a+ // gateway that took adaptive thinking, caching and server-side fallbacks.+ it('does not buy the extensions along with the sight', () => {+ const session = provider({ baseURL: gateway, vision: true })+ .createSession({ system: 's', tools: TOOLS }) as unknown as {+ opts: { dialect: string; vision: boolean };+ };+ expect([session.opts.dialect, session.opts.vision]).toEqual(['compatible', true]);});});@@ -224,8 +256,8 @@ describe('what a failed request says', () => {describe('an image on the person\'s turn', () => {const shot = [{ mediaType: 'image/png', data: 'AAAA' }];/** The session's messages — what the request would carry. */- const messagesOf = (baseURL?: string, images = shot, text = 'like this') => {- const provider = createAnthropicProvider({ apiKey: 'k', baseURL });+ const messagesOf = (baseURL?: string, images = shot, text = 'like this', vision?: boolean) => {+ const provider = createAnthropicProvider({ apiKey: 'k', baseURL, vision });const session = provider.createSession({ system: 's', tools: TOOLS });session.pushUser(text, images);return (session.serialize() as { messages: Array<{ role: string; content: unknown }> }).messages;@@ -255,6 +287,13 @@ describe('an image on the person\'s turn', () => {expect(msg.content as string).toMatch(/cannot receive images|have not seen/);});+ // And a gateway that DOES take them gets the picture, because what decides+ // this is what the provider declared and not where the endpoint lives.+ it('rides the turn on a gateway that declared it can see', () => {+ const [msg] = messagesOf('https://gateway.example/anthropic', shot, 'like this', true);+ expect((msg.content as Array<{ type: string }>).map((b) => b.type)).toEqual(['image', 'text']);+ });+it('counts them when there are several', () => {const two = [shot[0], { mediaType: 'image/jpeg', data: 'BBBB' }];const [msg] = messagesOf('https://gateway.example/anthropic', two);diff --git a/desktop/tests/agent-openai.test.ts b/desktop/tests/agent-openai.test.tsindex bf9a347fa..827984a1d 100644--- a/desktop/tests/agent-openai.test.ts+++ b/desktop/tests/agent-openai.test.ts@@ -121,12 +121,14 @@ describe('the provider', () => {expect(provider({ baseURL: 'http://localhost:11434/v1' }).id).toBe('openai:http://localhost:11434/v1');});- // A gateway we have never heard of may not take images, and finding out costs- // a call spent on a screenshot the model is then told it cannot look at.- it('assumes a gateway cannot see, and OpenAI itself can', () => {+ // Declared by whoever picked the provider (src/agent/providers.ts). The+ // address is only the fallback for a caller that says nothing — reading it as+ // the answer is what left every gateway blind with no way to say otherwise.+ it('takes the caller at its word, and falls back to the address', () => {+ expect(provider({ baseURL: 'http://x/v1', vision: true }).acceptsImages).toBe(true);+ expect(provider({ vision: false }).acceptsImages).toBe(false);expect(provider().acceptsImages).toBe(true);expect(provider({ baseURL: 'http://localhost:11434/v1' }).acceptsImages).toBe(false);- expect(provider({ baseURL: 'http://x/v1', vision: true }).acceptsImages).toBe(true);});/**diff --git a/desktop/tests/agent-providers.test.ts b/desktop/tests/agent-providers.test.tsindex eae08986a..cd1776b96 100644--- a/desktop/tests/agent-providers.test.ts+++ b/desktop/tests/agent-providers.test.ts@@ -10,6 +10,7 @@import { describe, it, expect } from 'vitest';import {agentProviders, agentProvider, agentKeyId, parseModelList, protocolOf, asProtocol, CUSTOM_PROVIDER,+ setUserProviders,} from '@/agent/providers';describe('the shipped providers', () => {@@ -41,14 +42,55 @@ describe('the shipped providers', () => {expect(protocolOf(openai)).toBe('openai');expect(openai?.typedModels).toBe(true);expect(openai?.models).toEqual([]);- // Still a shipped provider: it keeps its own key row, unlike the custom slot.+ // Still a shipped provider: it keeps its own key row, unlike a typed one.expect(openai?.userDefined).toBeUndefined();});- it('ships the custom slot empty — it is the one we cannot know', () => {- const custom = agentProvider(CUSTOM_PROVIDER);- expect(custom?.userDefined).toBe(true);- expect(custom?.models).toEqual([]);+ // There is no reserved slot for "the custom one" any more. There cannot be:+ // the person has as many as they have endpoints, and a single slot meant+ // configuring the second overwrote the first, credential included.+ it('ships no placeholder for the providers a person types', () => {+ expect(agentProviders().every((p) => !p.userDefined)).toBe(true);+ expect(agentProvider(CUSTOM_PROVIDER)).toBeUndefined();+ });+});++/**+ * The typed providers arrive as a SET, because that is how they are edited — a+ * table rewritten whole — and because the removals matter as much as the+ * additions: one deleted from the table must stop being offered.+ */+describe('the providers a person defined', () => {+ const def = (id: string, over = {}) => ({ id, label: id, baseUrl: `http://${id}/v1`, models: ['m'], ...over });++ it('joins the shipped ones, marked as theirs, and leaves when withdrawn', () => {+ setUserProviders([def('mine-a'), def('mine-b')]);+ expect(agentProvider('mine-a')?.userDefined).toBe(true);+ expect(agentProviders().map((p) => p.id)).toEqual(+ expect.arrayContaining(['anthropic', 'deepseek', 'openai', 'mine-a', 'mine-b']),+ );++ setUserProviders([def('mine-a')]);+ expect(agentProvider('mine-b')).toBeUndefined();+ expect(agentProvider('mine-a')).toBeDefined();++ setUserProviders([]);+ expect(agentProviders().every((p) => !p.userDefined)).toBe(true);+ });++ // Shipped first: the picker reads top to bottom, and the vendor rows are what+ // a person who has typed nothing is looking for.+ it('lists after the shipped ones', () => {+ setUserProviders([def('mine-a')]);+ const ids = agentProviders().map((p) => p.id);+ expect(ids.indexOf('mine-a')).toBeGreaterThan(ids.indexOf('openai'));+ setUserProviders([]);+ });++ it('cannot take an id the editor already ships', () => {+ setUserProviders([def('anthropic', { label: 'Impostor' })]);+ expect(agentProvider('anthropic')?.label).toBe('Anthropic');+ setUserProviders([]);});});diff --git a/desktop/tests/agent-vision.test.ts b/desktop/tests/agent-vision.test.tsnew file mode 100644index 000000000..8541cc033--- /dev/null+++ b/desktop/tests/agent-vision.test.ts@@ -0,0 +1,245 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file What an endpoint can DO, from the row that declares it to the endpoint+ * main builds.+ *+ * Sight used to be read off the address — no baseUrl, therefore the+ * vendor's own API, therefore it can see — which is a statement about+ * WHERE an endpoint is and not about what it accepts. Every gateway was+ * blind with no way to say otherwise, and the whole consequence was+ * silent: the agent was told each turn that screenshots could not reach+ * it, asked for `format: 'grid'` instead, and nobody could tell that+ * apart from a preference.+ *+ * The wire halves live with their protocols (agent-dialect, agent-openai);+ * what is checked here is that the facts travel.+ */+import { describe, it, expect, beforeEach, afterEach } from 'vitest';+import '@/settings';+import { useSettings } from '@/store/settingsStore';+import { useAgent, syncAgentEndpoint, agentAcceptsImages } from '@/store/AgentStore';+import { agentProvider, CUSTOM_PROVIDER } from '@/agent/providers';+import {+ AGENT_PROVIDERS_SETTING, syncUserProviders, type UserProviderRow,+} from '@/agent/userProviders';++interface Endpoint {+ vision?: boolean; baseUrl?: string; protocol?: string; contextWindow?: number;+ reasoningEffort?: boolean;+}++let sent: Endpoint[];++/** A row as the settings table stores one. */+const row = (over: Partial<UserProviderRow> = {}): Record<string, unknown> => ({+ id: 'custom-1',+ label: 'Local llama',+ protocol: 'openai',+ baseUrl: 'http://localhost:11434/v1',+ models: 'llama-4',+ vision: false,+ contextWindow: 0,+ reasoningEffort: true,+ ...over,+} as unknown as Record<string, unknown>);++/** Put rows in the table and select the first — what picking one in the composer does. */+const configure = (...rows: Record<string, unknown>[]) => {+ useSettings.getState().setValue(AGENT_PROVIDERS_SETTING, rows);+ syncUserProviders();+ const first = rows[0];+ useAgent.setState({+ selection: { providerId: String(first.id), model: String(first.models).split(',')[0].trim() },+ });+};++beforeEach(() => {+ sent = [];+ (globalThis as { window?: unknown }).window = {+ estella: { agent: { setEndpoint: async (patch: Endpoint) => { sent.push(patch); } } },+ };+ useAgent.setState({ selection: null });+ useSettings.getState().setValue(AGENT_PROVIDERS_SETTING, []);+ syncUserProviders();+});++afterEach(() => {+ useSettings.getState().setValue(AGENT_PROVIDERS_SETTING, []);+ syncUserProviders();+ delete (globalThis as { window?: unknown }).window;+});++describe('what a shipped provider declares', () => {+ it('says so where it is true, rather than leaving it to the address', () => {+ expect(agentProvider('anthropic')?.vision).toBe(true);+ expect(agentProvider('openai')?.vision).toBe(true);+ });++ // The two mistakes are not symmetrical: claiming sight an endpoint lacks+ // spends the turn on a refused request, withholding it costs a text grid.+ it('withholds it from a provider we have not confirmed', () => {+ expect(agentProvider('deepseek')?.vision).toBeUndefined();+ });+});++describe('an endpoint the person described', () => {+ it('is blind until they say otherwise', () => {+ configure(row());+ expect(agentAcceptsImages()).toBe(false);+ });++ it('can be declared sighted, and the endpoint hears about it', () => {+ configure(row({ vision: true }));+ syncAgentEndpoint();+ expect(agentAcceptsImages()).toBe(true);+ expect(sent.at(-1)).toMatchObject({ vision: true, baseUrl: 'http://localhost:11434/v1' });+ });++ // A gateway address is not an answer to this question — that reading is the+ // bug. What matters is that the two facts travel separately.+ it('carries the declaration alongside the address, not derived from it', () => {+ configure(row({ vision: true }));+ sent.length = 0;+ syncAgentEndpoint();+ expect(sent).toHaveLength(1);+ expect([sent[0].baseUrl, sent[0].vision]).toEqual(['http://localhost:11434/v1', true]);+ });++ // Compaction at three quarters of a window that is not this endpoint's folds+ // a conversation that had room, or fails one that did not.+ it('says how much room it has, instead of being assumed small', () => {+ configure(row({ contextWindow: 1_000_000 }));+ syncAgentEndpoint();+ expect(sent.at(-1)?.contextWindow).toBe(1_000_000);+ });++ // An endpoint that has never heard of the argument refuses the whole call.+ it('can refuse the reasoning-depth argument without losing every turn', () => {+ configure(row({ reasoningEffort: false }));+ syncAgentEndpoint();+ expect(sent.at(-1)?.reasoningEffort).toBe(false);+ });+});++describe('a provider that ships its own answer', () => {+ it('travels without the person configuring anything', () => {+ useAgent.setState({ selection: { providerId: 'anthropic', model: 'claude-opus-5' } });+ syncAgentEndpoint();+ expect(sent.at(-1)).toMatchObject({ vision: true, reasoningEffort: true });+ expect(agentAcceptsImages()).toBe(true);+ });++ it('and a shipped gateway that has not claimed sight stays blind', () => {+ useAgent.setState({ selection: { providerId: 'deepseek', model: 'deepseek-v4-pro' } });+ syncAgentEndpoint();+ expect(sent.at(-1)?.vision).toBe(false);+ expect(agentAcceptsImages()).toBe(false);+ });+});++/**+ * The reason the table exists: a person has more than one of these, and the+ * shape it replaced could only hold one — configuring a second overwrote the+ * first, credential included.+ */+describe('more than one of them', () => {+ it('keeps both, each with its own key id and its own capabilities', () => {+ configure(+ row({ id: 'custom-1', label: 'Local llama', vision: false }),+ row({ id: 'custom-2', label: 'Work gateway', baseUrl: 'https://gw.corp/v1', models: 'gpt-x', vision: true }),+ );+ const a = agentProvider('custom-1');+ const b = agentProvider('custom-2');+ expect([a?.label, a?.vision, a?.baseUrl]).toEqual(['Local llama', false, 'http://localhost:11434/v1']);+ expect([b?.label, b?.vision, b?.baseUrl]).toEqual(['Work gateway', true, 'https://gw.corp/v1']);+ });++ it('stops offering one that was deleted', () => {+ configure(row({ id: 'custom-1' }), row({ id: 'custom-2' }));+ expect(agentProvider('custom-2')).toBeDefined();+ useSettings.getState().setValue(AGENT_PROVIDERS_SETTING, [row({ id: 'custom-1' })]);+ syncUserProviders();+ expect(agentProvider('custom-2')).toBeUndefined();+ expect(agentProvider('custom-1')).toBeDefined();+ });++ // The registry is the one door: a shipped provider and a typed one are the+ // same kind of thing to everything downstream of it.+ it('does not let a typed row displace a shipped provider', () => {+ configure(row({ id: 'anthropic', label: 'Impostor', baseUrl: 'http://evil/v1' }));+ expect(agentProvider('anthropic')?.label).toBe('Anthropic');+ });+});++/**+ * A key is filed under its provider's id and lives outside the list, so the two+ * ends of a row's life both have to account for it: a deleted row's credential+ * must go, and a new row must never be handed an id that could still have one.+ */+describe('the credential a row owns', () => {+ it('goes when the row does, under the id it was filed against', async () => {+ const cleared: string[] = [];+ (globalThis as { window?: unknown }).window = {+ estella: { secrets: { clear: async (id: string) => { cleared.push(id); return undefined; } } },+ };+ const { forgetProviderSecret } = await import('@/agent/userProviders');+ forgetProviderSecret(row({ id: 'custom-2' }));+ await Promise.resolve();+ expect(cleared).toEqual(['agents.key.custom-2']);+ });++ // An id that came back around would inherit whatever is still sealed under it.+ it('never re-uses an id, even after the rows between are gone', async () => {+ const { newProviderRow } = await import('@/agent/userProviders');+ expect(newProviderRow([]).id).toBe('custom-1');+ expect(newProviderRow([row({ id: 'custom-1' }), row({ id: 'custom-2' })]).id).toBe('custom-3');+ // custom-2 was deleted; the next row must not step into its place.+ expect(newProviderRow([row({ id: 'custom-3' })]).id).toBe('custom-4');+ // The migrated singleton carries a non-numbered id and must not confuse it.+ expect(newProviderRow([row({ id: CUSTOM_PROVIDER })]).id).toBe('custom-1');+ });+});++/**+ * A key is filed under its provider's id, and main never hands a sealed one+ * back — so an id that does not carry over is a credential the person has to+ * find again. This is the one migration step that cannot be repaired by hand.+ */+describe('a setup made before the table existed', () => {+ it('keeps the id its key is filed under', async () => {+ const { migrateLegacyCustomProvider } = await import('@/agent/userProviders');+ useSettings.setState({+ values: {+ 'agents.customProtocol': 'anthropic',+ 'agents.customBaseUrl': 'https://gateway.example/anthropic',+ 'agents.customModels': 'some-model',+ 'agents.customVision': true,+ },+ });+ migrateLegacyCustomProvider();+ syncUserProviders();++ const def = agentProvider(CUSTOM_PROVIDER);+ expect(def).toBeDefined();+ expect([def?.protocol, def?.baseUrl, def?.vision]).toEqual([+ 'anthropic', 'https://gateway.example/anthropic', true,+ ]);+ expect(def?.models).toEqual(['some-model']);+ // And the shape it replaced is gone from the file rather than left to be+ // puzzled over by whoever opens it next.+ expect(useSettings.getState().values['agents.customBaseUrl']).toBeUndefined();+ });++ it('leaves a table the person has already emptied alone', async () => {+ const { migrateLegacyCustomProvider } = await import('@/agent/userProviders');+ useSettings.setState({+ values: {+ [AGENT_PROVIDERS_SETTING]: [],+ 'agents.customBaseUrl': 'https://gateway.example/anthropic',+ },+ });+ migrateLegacyCustomProvider();+ expect(useSettings.getState().getValue(AGENT_PROVIDERS_SETTING)).toEqual([]);+ });+});diff --git a/docs/astro/src/content/docs/extending/built-in-agent.mdx b/docs/astro/src/content/docs/extending/built-in-agent.mdxindex 5fb9d3e01..15beadf22 100644--- a/docs/astro/src/content/docs/extending/built-in-agent.mdx+++ b/docs/astro/src/content/docs/extending/built-in-agent.mdx@@ -33,8 +33,10 @@ and configure one provider — that is the whole setup.name that no longer exists is not refused — the endpoint quietly serves somethingsmaller instead, for the rest of the session.-3. Or fill in the **Custom Provider** — any endpoint at all. Pick which **protocol**- it speaks, then give its address and its model names.+3. Or add your own under **Your providers** — any endpoint at all, as many as you+ have. Each row is a provider like the shipped ones: a name, which **protocol**+ it speaks, its address, its model names, and its own key. Expand a row to say+ what that endpoint can do.4. Close Settings. The model picker sits next to the box you type in — it is aper-message choice, not a preference buried in a dialog.@@ -56,6 +58,24 @@ settings file, and never handed back to the editor window — only to the clienthas to send it.</Aside>+## What an endpoint can do++The shipped providers already say. For one of your own the editor has no way to+find out, and guessing is expensive in both directions — so a row's expander is+where you tell it. What is left unsaid is worked around, not discovered.++- **Accepts images** — the agent takes screenshots to check its own work. Where a+ picture cannot be sent it asks for the same frame as a coarse colour grid in+ text instead, and says so in the transcript. Off by default: claiming sight an+ endpoint lacks costs a whole turn to a refused request, while withholding it+ costs detail the agent can mostly do without.+- **Context window** — how far a conversation may grow before its oldest runs are+ folded away. Blank assumes a conservative window, which folds a conversation+ that still had room; too generous fails the turn outright, after the wait.+- **Takes a reasoning-depth argument** — whether the request may name the depth+ set above. An endpoint that has never heard of the argument refuses the whole+ call rather than ignoring it, so turn this off for one that does.+## Ask it somethingOpen the command palette (`Ctrl/Cmd+Shift+P`) and run **Agent: Open**. Or skip that:diff --git a/docs/astro/src/content/docs/zh-cn/extending/built-in-agent.mdx b/docs/astro/src/content/docs/zh-cn/extending/built-in-agent.mdxindex 62d8adfd4..abddd4093 100644--- a/docs/astro/src/content/docs/zh-cn/extending/built-in-agent.mdx+++ b/docs/astro/src/content/docs/zh-cn/extending/built-in-agent.mdx@@ -31,8 +31,9 @@ import { Aside, Steps } from '@astrojs/starlight/components';厂商下次发布为止,而写了个已经不存在的名字并不会被拒绝——接口会悄悄改用一个更小的模型顶上,并且一整个会话都是如此。-3. 或者填**自定义提供方**——任何接口都行。先选它讲哪种**协议**,- 再填地址和模型名。+3. 或者在**你的提供方**里添加自己的——任何接口都行,有几个加几个。每一行+ 都和内置的提供方一样:名称、讲哪种**协议**、地址、模型名,以及它自己的密钥。+ 展开一行还能说明这个接口**能做什么**。4. 关掉设置。模型选择器就在你打字的那个框旁边——它是**逐条消息**的选择,而不是埋在对话框里的偏好设置。@@ -52,6 +53,21 @@ import { Aside, Steps } from '@astrojs/starlight/components';只交给那个必须发送它的客户端。</Aside>+## 接口能做什么++内置的提供方已经自己写好了。你自己加的那些,编辑器没有办法探出来,而猜错的+两个方向代价都不小——所以一行的展开项就是你告诉它的地方。没说的部分不会被+发现,只会被绕开。++- **接受图片**——Agent 会截图来检查自己干的活。发不出去图片时,它会改成把+ 同一帧要成粗颗粒的文字色块网格,并在对话里说明。默认关闭:给一个其实看不了+ 图的接口打开它,代价是整轮请求被拒;而不打开,只是少了些它多半能不依赖的细节。+- **上下文窗口**——一段对话能长到多大,才开始把最早的几轮折叠掉。留空按一个+ 保守值算,会把本来还有余量的对话提前折叠;写得太大则是整轮请求直接失败,+ 而且是在你等完之后。+- **接受思考深度参数**——请求里可不可以带上上面设的那个深度。没听说过这个参数+ 的接口会拒掉整次调用,而不是忽略它——遇到这种就把它关掉。+## 问它点什么打开命令面板(`Ctrl/Cmd+Shift+P`)执行 **Agent:打开**。或者干脆跳过这一步: