🤖 Open-source, local-first alternative to Grok Bot, OpenAI dots and Claude Duet. Your own AI bots on any model: GPT, Claude, Gemini, Grok, DeepSeek, Kimi, GLM and 70+ more through one Modelflare key. Reach them in the app, WeChat or Feishu; they remember you, work in the background, run commands and use your computer. Pay per use.
Python ai-agentai-assistantchatbotclaudecomputer-usedeepseekdesktop-appdshelectronfeishugeminigptgrokllmlocal-firstmodelflaremulti-modelopenai-compatiblepersonal-ai-agentwechat
feat(media): 生图生视频先确认模型,按 Modelflare 文档实时传参
dyk982505d7cbf2b3
diff --git a/README.md b/README.mdindex 8d21772..bb6b20d 100644--- a/README.md+++ b/README.md@@ -80,7 +80,9 @@ docker exec amb cat /data/ui-token # 打开 http://<主机>:3600/?token=<令- 余额:侧栏左下角和 设置 → Modelflare 显示账户钱包余额(只用 API Key 时读 `/v1/dashboard/billing/credit_grants`);「充值」打开 Modelflare 网页,回到窗口时自动刷新。余额低于提醒线(默认 $1,可改或关闭)时在应用里提醒一次,也可以设置让机器人发到你的微信 / 飞书。- 登录后还能在 设置 → Modelflare 查看按天、按模型的用量,Modelflare 发来的通知,以及复制邀请链接。- 机器人预算(需要登录):机器人设置 → 预算,开启后这个机器人用自己的 Key(内置网关里单独的 provider),最多花设定的美元数;关闭后回到共用的 Key。-- 生图 / 生视频:代理模式下机器人有 `generate_image`、`generate_video` 工具,由本程序带着该机器人用的 Key 调 Modelflare,结果存进机器人工作区的 `media/`,再交付到聊天。+- 生图 / 生视频:代理模式下机器人有 `media_models`(可用模型、默认模型、参数范围、参考价)、`media_docs`(从 docs.modelflare.dev 实时读该模型的请求格式)、`generate_image`、`generate_video` 四个工具,并自带一份 media 技能说明。各模型的参数格式不同,机器人按文档写请求体,工作区里的参考图写成 `workspace:路径` 即可;Key 始终只在本程序里,由本程序带着该机器人用的 Key 调 Modelflare,结果存进机器人工作区的 `media/`,再交付到聊天。+- 生成前确认:每次生成图片或视频之前都会在聊天里问你这次用哪个模型(附提示词、参数和参考价),可以确认、换成别的模型或取消;微信回复编号或模型名,飞书 / 网页点选。10 分钟内没回答就不生成。+- 默认图片 / 视频模型:首次设置时和默认文本模型一起选,之后在 设置 → Modelflare 修改;选「自动」时用推荐顺序里第一个不需要参考素材的模型。- 模型选择器里显示上下文长度、视觉 / 工具 / 推理能力,以及每百万 tokens 输入 / 输出的起步参考价(来自公开价格目录,以账单为准)。- 调用归因:经 Modelflare 的请求都带 `X-Modelflare-Skill: anymodel-bot/<版本>`,链接都带 `utm_source=anymodelbot`。- 线路:默认国际线路 `https://modelflare.dev/v1`;中国大陆可在「高级」或 设置 → Modelflare → 线路 选「中国大陆优化」(`https://origin.modelflare.dev/v1`)。线路由你选定,不自动切换。diff --git a/dsh-runtime/plugins/amb-media.mjs b/dsh-runtime/plugins/amb-media.mjsindex b534b9f..b387e91 100644--- a/dsh-runtime/plugins/amb-media.mjs+++ b/dsh-runtime/plugins/amb-media.mjs@@ -1,9 +1,10 @@// anymodel bot image / video generation tools (MIT, part of anymodel bot).//-// Registers generate_image and generate_video. The agent never holds a Modelflare key: each call goes-// to the host over the loopback bridge (AMB_BRIDGE_URL / AMB_BRIDGE_TOKEN), which calls Modelflare with-// this bot's key and saves the result under media/ in the workspace. The agent then delivers the file-// with the present tool.+// Registers media_models, media_docs, generate_image and generate_video. The agent never holds a+// Modelflare key: each call goes to the host over the loopback bridge (AMB_BRIDGE_URL / AMB_BRIDGE_TOKEN),+// which asks the user to confirm the model, calls Modelflare with this bot's key and saves the result+// under media/ in the workspace. The agent then delivers the file with the present tool. Every model has+// its own request format, so the agent reads the model's live docs with media_docs and passes the body.import { defineTool } from '@deepseek-ai/dsh-tools'export const name = 'amb-media'@@ -33,13 +34,41 @@ export function apply(ctx) {return { text: String(r.text ?? '') }}+ const BODY = {+ type: 'object',+ additionalProperties: true,+ description: "Request body exactly as the model's media_docs page shows it, without model. Write local files as \"workspace:<path>\" (sent as a data URL) or \"workspace-base64:<path>\" (raw Base64, e.g. Gemini inline_data.data). Omit for a plain text-only request.",+ }++ ctx.tools.register(defineTool({+ name: 'media_models',+ description: 'List the image and video models this bot may use on Modelflare, with the default model, parameter ranges (sizes, ratios, resolutions, seconds, reference media) and reference prices. Free; call it before choosing a model.',+ parameters: {+ kind: { type: 'string', enum: ['image', 'video'], description: 'Only this kind. Omit for both.' },+ },+ output: TEXT_OUT,+ execute: (args, exec) => run('models', args, exec),+ }))++ ctx.tools.register(defineTool({+ name: 'media_docs',+ description: "Read the live documentation page of one image or video model from docs.modelflare.dev: endpoint, exact request body, parameter table, reference-media fields and limits. Every model has its own format; read it before passing a body to generate_image or generate_video.",+ parameters: {+ model: { type: 'string', required: true, description: 'Exact model id, e.g. kling-3.0-turbo.' },+ },+ output: TEXT_OUT,+ execute: (args, exec) => run('docs', args, exec),+ }))+ctx.tools.register(defineTool({name: 'generate_image',- description: 'Generate an image from a text description with an image model on Modelflare. The image is saved in the workspace under media/; deliver it with the present tool when the user should receive it. Write the prompt in detail (subject, style, composition, text in the image).',+ description: 'Generate or edit an image with an image model on Modelflare. Before anything is generated the user is asked to confirm the model; they may pick another one or cancel. The image is saved in the workspace under media/; deliver it with the present tool when the user should receive it. Write the prompt in detail (subject, style, composition, text in the image).',parameters: {- prompt: { type: 'string', required: true, description: 'What the image should show, in detail.' },- model: { type: 'string', description: 'Image model id. Omit to use the first one this bot may use.' },- size: { type: 'string', description: 'Optional size such as 1024x1024, for models on the Images API.' },+ prompt: { type: 'string', required: true, description: 'What the image should show, in detail. Shown to the user in the confirmation; keep it in sync with the prompt inside body.' },+ model: { type: 'string', description: 'Image model id. Omit to use the default model.' },+ body: BODY,+ endpoint: { type: 'string', enum: ['generations', 'edits'], description: 'For OpenAI Images models only: "edits" when the docs say reference images go to /v1/images/edits. Default generations.' },+ size: { type: 'string', description: 'Optional size such as 1024x1024 for a plain request without body, Images API models only.' },},output: TEXT_OUT,execute: (args, exec) => run('image', args, exec),@@ -47,12 +76,13 @@ export function apply(ctx) {ctx.tools.register(defineTool({name: 'generate_video',- description: 'Generate a short video from a text description with a video model on Modelflare. It can take several minutes. The video is saved in the workspace under media/; deliver it with the present tool. Only use it when the user asks for a video: video generation costs much more than images.',+ description: 'Generate a short video with a video model on Modelflare. Before anything is generated the user is asked to confirm the model; they may pick another one or cancel. It can take several minutes. The video is saved in the workspace under media/; deliver it with the present tool. Only use it when the user asks for a video: video generation costs much more than images.',parameters: {- prompt: { type: 'string', required: true, description: 'What the video should show, in detail.' },- model: { type: 'string', description: 'Video model id. Omit to use the first one this bot may use.' },- seconds: { type: 'number', description: 'Optional length in seconds, if the model supports it.' },- size: { type: 'string', description: 'Optional size such as 1280x720, if the model supports it.' },+ prompt: { type: 'string', required: true, description: 'What the video should show, in detail. Shown to the user in the confirmation; keep it in sync with the prompt inside body.' },+ model: { type: 'string', description: 'Video model id. Omit to use the default model.' },+ body: BODY,+ seconds: { type: 'number', description: 'Optional length in seconds for a plain request without body.' },+ size: { type: 'string', description: 'Optional size such as 1280x720 for a plain request without body.' },},output: TEXT_OUT,execute: (args, exec) => run('video', args, exec),diff --git a/src/agent/host.ts b/src/agent/host.tsindex fbdced0..e457c66 100644--- a/src/agent/host.ts+++ b/src/agent/host.ts@@ -135,6 +135,8 @@ export interface AgentHostOptions {turnTimeoutMs?: number;/** 需要让用户看到的提示(界面通知) */onNotice?: (text: string) => void;+ /** 能生图 / 生视频(已连接 Modelflare):写入 media 技能说明 */+ media?: boolean;}export type HostState = "stopped" | "starting" | "running" | "backoff" | "error";@@ -315,6 +317,7 @@ export class AgentHost {const lim = MODEL_LIMITS.find(([re]) => re.test(model)) ?? [/./, 128_000, 16_000];const plugin = path.join(this.o.runtimeDir, "plugins", "amb-bridge.mjs");const memory = path.join(this.o.runtimeDir, "plugins", "amb-memory.mjs");+ this.writeMediaSkill();const text = ["# 由 anymodel bot 每次启动生成;请勿手改。","- id: llm-pi-ai",@@ -341,7 +344,7 @@ export class AgentHost {` name: ${yamlStr(path.join(this.o.runtimeDir, "plugins", "amb-permission.mjs"))}`," - id: amb-memory",` name: ${yamlStr(memory)}`,- // 生图 / 生视频:宿主带着本机器人的 Key 调 Modelflare,文件存进工作区 media/+ // 生图 / 生视频:宿主带着本机器人的 Key 调 Modelflare,文件存进工作区 media/;技能说明在 skills/media" - id: amb-media",` name: ${yamlStr(path.join(this.o.runtimeDir, "plugins", "amb-media.mjs"))}`,// sdk 配置默认不带这两个工具:ask_user_question 走 user-questions → 插件 → 审批中心;@@ -358,6 +361,20 @@ export class AgentHost {return f;}+ /** media 技能:怎么挑模型、读文档、写请求体、交付结果。没有连接 Modelflare 时删掉。 */+ private writeMediaSkill(): void {+ const dir = path.join(this.home, "skills", "media");+ try {+ if (!this.o.media) fs.rmSync(dir, { recursive: true, force: true });+ else {+ fs.mkdirSync(dir, { recursive: true });+ fs.writeFileSync(path.join(dir, "SKILL.md"), MEDIA_SKILL);+ }+ } catch (e) {+ log.warn("media skill setup failed", { err: String(e) });+ }+ }+/** 当前机器上本机器人所选后端的可用性(不看本机器人的开关)。 */computerUseStatus(): CuResolved {return resolveComputerUse(this.settings().computerUse.backend, this.o.runtimeDir);@@ -868,3 +885,30 @@ Tools are named \`mcp__computer-use__<tool>\`. Actions run automatically; no app- On Linux, reading window contents needs AT-SPI; on Wayland only the foreground window can be driven. On macOS the app needs Accessibility and Screen Recording permission; if \`check_permissions\` reports them missing, tell the user how to grant them.`;++/** 生图 / 生视频的技能说明(由 anymodel bot 编写;不含 Key、不让代理自己调接口,参数格式以 Modelflare 文档为准)。 */+const MEDIA_SKILL = `---+name: media+description: Generate or edit images and generate videos with Modelflare image and video models through the media_models, media_docs, generate_image and generate_video tools. Use when the user asks to draw, create, edit or restyle a picture, animate an image, or make a video.+---++# Images and videos (Modelflare)++You have no Modelflare key and must not call the API yourself (no curl, no scripts): the tools call it with this bot's key. Every generation costs the user money, so the user confirms the model before anything is generated.++## Steps+1. \`media_models\` (kind \`image\` or \`video\`) lists the models this bot may use, the default one, their parameter ranges and reference prices. Use the default unless the user named a model or the request needs something the default cannot do (a reference image, a ratio or length it does not support).+2. Plain text-to-image or text-to-video with default settings: call \`generate_image\` / \`generate_video\` with a detailed \`prompt\` only.+3. Anything else (ratio, resolution, size, length, quality, audio, reference images, first/last frames, editing an image): call \`media_docs\` for the chosen model, then pass \`body\` in exactly the format its page shows, without \`model\`. Every family is different (for example Seedance uses \`content[]\`, Kling \`contents\` and \`settings\`, Wan \`input\` and \`parameters\`, Gemini \`contents[].parts[]\` with \`generationConfig\`), so never reuse field names from another model's page. For OpenAI Images models whose page sends references to \`/v1/images/edits\`, also pass \`endpoint: "edits"\`. Keep \`prompt\` the same as the prompt inside \`body\`: the user sees \`prompt\` in the confirmation.+4. Reference media: a file in the workspace is written as \`"workspace:media/x.png"\` (sent as a data URL) or \`"workspace-base64:media/x.png"\` (raw Base64, e.g. Gemini \`inline_data.data\`), up to 20 MB each. A public https URL can be used as is. Models marked 必须带参考素材 (image-to-video, reference-to-video) only work with references; many models accept reference videos only as public URLs, see the docs. Images the user sent in chat are not in the workspace: if you have no file or link for a reference, ask the user for one.+5. The tool result says what happened:+ - Generated: the file is under \`media/\`; deliver it with \`present\`.+ - The user picked another model while you had passed a \`body\`: read that model's page with \`media_docs\`, rewrite \`body\` for it and call again with \`model\` set to it. That call is not confirmed again.+ - The user answered something else: adjust as they said and call again (they will be asked again).+ - Cancelled or not confirmed in time: nothing was generated or charged; do not retry on your own.+6. Videos take minutes and cost much more than images: only make a video when the user asks for one, and do not generate extra variants unless asked.++## Errors+- 这把 Key 不能用这个模型 (model_not_found): this account cannot use that model; pick another one from \`media_models\`.+- Any other 4xx names the problem: fix the body according to the model's page instead of sending the same request again. If the balance is empty, tell the user to top up on Modelflare.+`;diff --git a/src/agent/manager.ts b/src/agent/manager.tsindex a6d373a..4bfb2f0 100644--- a/src/agent/manager.ts+++ b/src/agent/manager.ts@@ -4,8 +4,8 @@ import { logger } from "../log.js";import { AgentBridge } from "./bridge.js";import path from "node:path";import { AgentHost, callDetail, type TurnObserver, type TurnUsage } from "./host.js";-import type { MediaService } from "./media.js";-import { ApprovalCenter, type QuestionItem } from "./approvals.js";+import type { MediaModel, MediaPlan, MediaService } from "./media.js";+import { APPROVAL_TTL_MS, ApprovalCenter, type QuestionItem } from "./approvals.js";import { MemoryError, MemoryStore, type Scope } from "./memory.js";import { DEFAULT_SETTINGS, type BotSettings } from "./settings.js";import {@@ -41,6 +41,9 @@ export interface AgentManagerOptions {onNotice?: (text: string) => void;}+/** 生成确认里"取消"那一项 */+const CANCEL = "取消";+function deferred<T>() {let resolve!: (v: T) => void;const promise = new Promise<T>((r) => (resolve = r));@@ -67,6 +70,8 @@ export class AgentManager {private compactDone = new Map<string, (v: unknown) => void>();private judgedAt = new Map<string, number>();private reviewTimers = new Map<string, NodeJS.Timeout>();+ /** 会话 → 用户在生成确认里换了模型、代理要按新模型文档重写请求:重写后的那一次免确认 */+ private mediaGrants = new Map<string, { kind: string; model: string; at: number }>();/** 最近一次管家/整理结果(状态页展示) */readonly stewardLog: Array<{ at: number; conversationId: string; action: string; detail: string }> = [];@@ -84,6 +89,11 @@ export class AgentManager {this.memory = new MemoryStore(path.join(o.dataDir, "memory"));}+ /** 生图 / 生视频服务;未接内置网关时没有 */+ get media(): MediaService | undefined {+ return this.o.media;+ }+/** 是否可用(已启用且找到 DSH 运行时)。 */get available(): boolean {return this.o.enabled && !!this.o.runtimeDir;@@ -106,6 +116,7 @@ export class AgentManager {turnTimeoutMs: this.o.turnTimeoutMs,settings: () => this.settings(botId),onNotice: this.o.onNotice,+ media: !!this.o.media,});this.hosts.set(botId, h);}@@ -338,20 +349,90 @@ export class AgentManager {return res;}- /** 插件 → 宿主:remember / forget / recall。 */- /** 生图 / 生视频:只在该机器人有回合进行中时受理(工具调用只会发生在回合里) */+ /**+ * 生图 / 生视频:只在该机器人有回合进行中时受理(工具调用只会发生在回合里)。+ * models / docs 直接回答;image / video 每次都先向用户确认这次用哪个模型,确认后才花钱。+ */private async mediaOp(b: any): Promise<{ text?: string; error?: string }> {const botId = String(b?.botId ?? "");if (!this.hosts.has(botId) || !this.activeConvs.get(botId)?.size) return { error: "unknown bot" };- if (!this.o.media) return { error: "生成图片和视频需要连接 Modelflare" };+ const media = this.o.media;+ if (!media) return { error: "生成图片和视频需要连接 Modelflare" };+ const op = String(b?.op ?? "");+ const args = (b?.args ?? {}) as Record<string, unknown>;try {- return { text: await this.o.media.run(botId, String(b?.op ?? ""), (b?.args ?? {}) as Record<string, unknown>) };+ if (op === "models") return { text: await media.listText(botId, args.kind) };+ if (op === "docs") return { text: await media.docs(args.model) };+ const plan = await media.prepare(botId, op, args);+ const sessionId = String(b?.sessionId ?? "");+ const conv = this.resolveConversation(botId, sessionId);+ if (!conv) return { error: "找不到这次对话,没法向用户确认,没有生成" };+ const tool = plan.kind === "image" ? "generate_image" : "generate_video";+ let model = plan.model;+ const grant = this.mediaGrants.get(conv);+ if (grant?.kind === plan.kind && grant.model === model.id && Date.now() - grant.at < (this.o.approvalTtlMs ?? APPROVAL_TTL_MS)) {+ this.mediaGrants.delete(conv);+ } else {+ const r = await this.confirmMedia(botId, conv, sessionId, plan, tool);+ if (r.cancel) return { text: "用户取消了这次生成(或没有及时确认),没有生成文件,也没有扣费。不要自行重试;需要的话问用户想怎么改。" };+ if (r.reply !== undefined) return { text: `用户没有确认,而是回复:「${r.reply}」。按用户的意思调整后再调用 ${tool}(会再次确认)。` };+ if (r.model && r.model.id !== model.id) {+ if (plan.custom) {+ // 请求体是按原模型的文档写的,换了模型就得按新模型的文档重写;重写后的那一次不再确认+ this.mediaGrants.set(conv, { kind: plan.kind, model: r.model.id, at: Date.now() });+ return { text: `用户选择改用 ${r.model.id}。各模型的请求格式不同:先用 media_docs 查 ${r.model.id} 的文档,按它重写 body,再调用 ${tool}(model 填 ${r.model.id})。这一次不会再要求确认。` };+ }+ model = r.model;+ }+ }+ return { text: await media.execute(botId, model, args) };} catch (e) {- log.warn("media tool failed", { bot: botId, message: (e as Error).message.slice(0, 300) });+ log.warn("media tool failed", { bot: botId, op, message: (e as Error).message.slice(0, 300) });return { error: (e as Error).message };}}+ /** 生成前的确认:问用户这次用哪个模型(本次的、默认的和几个推荐的,或回复别的模型名),等到回答、取消或过期 */+ private async confirmMedia(botId: string, conv: string, sessionId: string, plan: MediaPlan, tool: string): Promise<{ model?: MediaModel; cancel?: boolean; reply?: string }> {+ const name = plan.kind === "image" ? "图片" : "视频";+ const opt = (m: MediaModel, tag = "") => ({ label: m.id, description: [tag, m.price].filter(Boolean).join(" · ") || undefined });+ const q: QuestionItem = {+ id: "model",+ header: `生成${name}`,+ question: `这次用 ${plan.model.id} 生成${name}吗?`,+ detail: `${plan.summary}\n也可以直接回复其他模型名。`,+ options: [opt(plan.model, plan.model === plan.default ? "默认" : "机器人选的"), ...plan.candidates.map((m) => opt(m, m === plan.default ? "默认" : "")), { label: CANCEL, description: "这次不生成" }],+ };+ const row = this.approvals.create({ kind: "question", botId, conversationId: conv, sessionId, tool, detail: q.question, reason: "", payload: JSON.stringify([q]) });+ const deadline = Date.now() + (this.o.approvalTtlMs ?? APPROVAL_TTL_MS) + 10_000;+ let cur = row;+ while (cur.status === "pending") {+ if (Date.now() > deadline) {+ this.approvals.cancel(row.id, "timeout");+ return { cancel: true };+ }+ const next = await this.approvals.wait(row.id, 60_000);+ if (!next) return { cancel: true };+ cur = next;+ }+ if (cur.status !== "answered") return { cancel: true };+ let a: { selected?: string[]; custom?: string } | undefined;+ try {+ a = (JSON.parse(cur.result) as { answers?: Array<{ selected?: string[]; custom?: string }> }).answers?.[0];+ } catch {+ a = undefined;+ }+ const pick = (a?.selected?.[0] ?? "").trim();+ const text = (a?.custom ?? "").trim();+ if (pick === CANCEL || /^(取消|不要?|不用|算了|no?|cancel)$/i.test(text)) return { cancel: true };+ if (!pick && /^(好的?|行|可以|确认|确定|是的?|对|嗯|ok(ay)?|y(es)?|生成吧?|就用它)$/i.test(text)) return { model: plan.model };+ const want = (pick || text).replace(/^(用|换成|改用)\s*/, "");+ const hit = plan.all.find((m) => m.id.toLowerCase() === want.toLowerCase());+ if (hit) return { model: hit };+ return text ? { reply: text } : { cancel: true };+ }++ /** 插件 → 宿主:remember / forget / recall。 */private async memoryOp(b: any): Promise<{ text?: string; error?: string }> {const botId = String(b?.botId ?? "");if (!this.hosts.has(botId)) return { error: "unknown bot" };@@ -423,6 +504,7 @@ export class AgentManager {resetConversation(conversationId: string): void {this.o.store.clearAgentSession(conversationId);this.o.store.setKV(`seed:${conversationId}`, "");+ this.mediaGrants.delete(conversationId);for (const r of this.approvals.pending(conversationId)) this.approvals.cancel(r.id, "reset");}diff --git a/src/agent/media.ts b/src/agent/media.tsindex 495943d..30b21c3 100644--- a/src/agent/media.ts+++ b/src/agent/media.ts@@ -1,9 +1,12 @@/**- * 生图 / 生视频:代理的 generate_image / generate_video 工具经内部 bridge 调到这里,由宿主带着- * 该机器人用的 Key 直连 Modelflare(代理子进程拿不到任何密钥),结果存进机器人工作区的 media/,- * 再由代理用 present 交付到聊天。- * - 模型取自 GET /v1/models?scope=authorized&kind=image|video(这把 Key 真正能用的)。- * - 图片:Gemini 图片模型走 /v1/models/{id}:generateContent,其余走 /v1/images/generations。+ * 生图 / 生视频:代理的 media_models / media_docs / generate_image / generate_video 工具经内部 bridge 调到这里,+ * 由宿主带着该机器人用的 Key 直连 Modelflare(代理子进程拿不到任何密钥),结果存进机器人工作区的 media/,+ * 再由代理用 present 交付到聊天。每次生成前都由 AgentManager 向用户确认这次用哪个模型(manager.ts mediaOp)。+ * - 模型取自 GET /v1/models?scope=authorized&kind=image|video(这把 Key 真正能用的);参数范围、是否必须带参考素材、+ * 参考价、推荐顺序取自公开价格目录 /api/pricing。+ * - 每个模型的请求格式都不一样,以 docs.modelflare.dev 上该模型的文档为准:代理先用 media_docs 读文档,按文档写+ * 请求体(body);宿主只填 model、把 "workspace:路径" 换成文件内容、发请求、保存结果。不带 body 时发最简单的文字请求。+ * - 图片:Gemini 图片模型走 /v1/models/{id}:generateContent,其余走 /v1/images/generations(改图走 /v1/images/edits)。* - 视频:POST /v1/videos 建任务 → 轮询 → GET /v1/videos/{id}/content 下载。*/import fs from "node:fs";@@ -12,6 +15,8 @@ import { logger } from "../log.js";const log = logger("agent.media");+export type MediaKind = "image" | "video";+export interface MediaOptions {/** Modelflare /v1 基址(随线路) */baseUrl: () => string;@@ -20,37 +25,192 @@ export interface MediaOptions {/** 额外请求头(调用归因) */headers: () => Record<string, string>;workspace: (botId: string) => string;+ /** 设置里选的默认图片 / 视频模型;空 = 按推荐顺序的第一个 */+ defaults?: () => { image: string; video: string };+ /** 公开价格目录(参数范围、参考价、推荐顺序);取不到时只是少了这些信息 */+ pricingUrl?: () => string;+ /** 文档全文(每个模型一节) */+ docsUrl?: string;fetchImpl?: typeof fetch;/** 视频轮询间隔与上限(测试用) */pollMs?: number;videoTimeoutMs?: number;}-interface MediaModel {+export interface MediaModel {id: string;- kind?: string;- image_route?: string;+ kind: MediaKind;+ /** images = OpenAI Images API;gemini = generateContent;videos = /v1/videos */+ route: "images" | "gemini" | "videos";+ /** 价格目录里的 media_options.image / media_options.video */+ options?: Record<string, unknown>;+ /** 必须带参考素材(图生视频之类),只有文字时不能用 */+ requiresReference: boolean;+ /** 一句参考价,如 "$0.03/张"、"$0.09/秒(720p)" */+ price: string;+ priority: number;+}++/** 一次生成要确认的内容 */+export interface MediaPlan {+ kind: MediaKind;+ /** 这次要用的模型(代理指定的,或默认) */+ model: MediaModel;+ /** 默认模型(设置里选的,或推荐的) */+ default: MediaModel;+ /** 确认时一并列出的其他选择 */+ candidates: MediaModel[];+ /** 这把 Key 能用的全部同类模型 */+ all: MediaModel[];+ /** 代理按文档写了自己的请求体:换模型后需要按新模型的文档重写 */+ custom: boolean;+ /** 给用户看的说明:提示词摘要、参数、参考素材 */+ summary: string;+}++interface CatalogEntry {+ model_name: string;+ display_priority?: number;+ quota_type?: number;+ model_price?: number;+ billing_unit?: string;+ enable_groups?: string[];+ video_resolution_price?: Record<string, number>;+ media_options?: { image?: Record<string, unknown>; video?: Record<string, unknown> };+}++interface Catalog {+ data?: CatalogEntry[];+ group_ratio?: Record<string, number>;+ usable_group?: Record<string, unknown>;}const MAX_BYTES = 200 * 1024 * 1024;+/** 参考素材按 data URL / Base64 嵌进请求,单个文件上限 */+const MAX_REFERENCE_BYTES = 20 * 1024 * 1024;+const CACHE_MS = 3600_000;+const DOCS_URL = "https://docs.modelflare.dev";const EXT: Record<string, string> = { "image/png": "png", "image/jpeg": "jpg", "image/webp": "webp", "image/gif": "gif", "video/mp4": "mp4", "video/webm": "webm" };+const MIME: Record<string, string> = {+ ".png": "image/png",+ ".jpg": "image/jpeg",+ ".jpeg": "image/jpeg",+ ".webp": "image/webp",+ ".gif": "image/gif",+ ".bmp": "image/bmp",+ ".mp4": "video/mp4",+ ".mov": "video/quicktime",+ ".webm": "video/webm",+ ".mp3": "audio/mpeg",+ ".wav": "audio/wav",+};+/** 请求体里的 "workspace:media/a.png"(data URL)或 "workspace-base64:media/a.png"(纯 Base64) */+const FILE_REF = /^workspace(-base64)?:(.+)$/;export class MediaError extends Error {}+const num = (n: number) => String(+n.toFixed(n < 0.1 ? 4 : 3));++/** 推荐顺序:价格目录的 display_priority 小的在前,必须带参考素材的放后面,其余按名字 */+export function sortModels(list: MediaModel[]): MediaModel[] {+ return [...list].sort((a, b) => a.priority - b.priority || Number(a.requiresReference) - Number(b.requiresReference) || a.id.localeCompare(b.id, "en", { sensitivity: "base" }));+}++/** 设置里的默认模型还能用就用它,否则推荐顺序里第一个不需要参考素材的 */+export function pickDefault(list: MediaModel[], configured: string): MediaModel | undefined {+ return list.find((m) => m.id === configured) ?? list.find((m) => !m.requiresReference) ?? list[0];+}++/** 一句参考价;复杂计费(按 token)只说明计费方式 */+export function priceNote(e: CatalogEntry | undefined, catalog: Catalog): string {+ if (!e) return "";+ const ratios = (e.enable_groups ?? [])+ .filter((g) => !catalog.usable_group || g in catalog.usable_group)+ .map((g) => catalog.group_ratio?.[g])+ .filter((r): r is number => typeof r === "number" && Number.isFinite(r) && r >= 0);+ const factor = ratios.length ? Math.min(...ratios) : 1;+ const perSecond = Object.entries(e.video_resolution_price ?? {}).filter(([, p]) => typeof p === "number" && p > 0);+ if (perSecond.length) {+ const want = String(e.media_options?.video?.default_resolution ?? "");+ const [res, p] = perSecond.find(([r]) => r === want) ?? perSecond.sort((a, b) => a[1] - b[1])[0];+ return `$${num(p * factor)}/秒(${res})`;+ }+ if (e.quota_type === 1 && typeof e.model_price === "number" && e.model_price > 0) return `$${num(e.model_price * factor)}/${e.billing_unit === "per_call" ? "次" : "张"}`;+ if (e.billing_unit === "token") return "按 token 计费";+ return "";+}++/** 一行参数范围:尺寸、比例、分辨率、时长、参考素材 */+export function optionsNote(m: MediaModel): string {+ const o = m.options ?? {};+ const list = (k: string) => (Array.isArray(o[k]) ? (o[k] as unknown[]).map(String) : []);+ const parts: string[] = [];+ if (list("sizes").length) parts.push(`尺寸 ${list("sizes").join("/")}`);+ if (list("resolutions").length) parts.push(`分辨率 ${list("resolutions").join("/")}`);+ if (list("ratios").length) parts.push(`比例 ${list("ratios").join("/")}`);+ const secs = list("seconds").map(Number).filter((n) => n > 0);+ if (secs.length) parts.push(`时长 ${Math.min(...secs)}–${Math.max(...secs)} 秒`);+ const refs = list("reference_media");+ if (refs.length) parts.push(`参考素材 ${refs.join("/")}`);+ else if (o.reference_images === true || Number(o.max_references) > 0) parts.push(`参考图${Number(o.max_references) > 0 ? ` 最多 ${o.max_references} 张` : ""}`);+ if (m.requiresReference) parts.push("必须带参考素材");+ return parts.join(" · ");+}++/** 从请求体里挑出给用户看的参数(分辨率、比例、时长……),并数一数参考素材 */+export function describeBody(body: Record<string, unknown> | undefined): string {+ if (!body) return "";+ const keys: Record<string, string> = { resolution: "", aspect_ratio: "", ratio: "", size: "", imageSize: "", aspectRatio: "", seconds: "秒", duration: "秒", quality: "" };+ const seen = new Map<string, string>();+ let refs = 0;+ const walk = (v: unknown, depth: number) => {+ if (typeof v === "string") {+ if (FILE_REF.test(v) || /^https?:\/\//i.test(v) || v.startsWith("data:")) refs++;+ return;+ }+ if (!v || typeof v !== "object" || depth > 6) return;+ for (const [k, x] of Object.entries(v as Record<string, unknown>)) {+ if (k in keys && (typeof x === "string" || typeof x === "number")) {+ // 时长两种写法只显示一个+ const label = k === "duration" ? "seconds" : k === "aspectRatio" || k === "aspect_ratio" ? "ratio" : k === "imageSize" ? "size" : k;+ if (!seen.has(label)) seen.set(label, `${x}${keys[k] && Number(x) > 0 ? ` ${keys[k]}` : ""}`);+ } else walk(x, depth + 1);+ }+ };+ walk(body, 0);+ const parts = [...seen.values()];+ if (refs) parts.push(`${refs} 个参考素材`);+ return parts.join(" · ");+}++/** 从文档全文里取出某个模型的那一节(每节以 "# 标题" 开头、下一行是 "Source: <页面地址>") */+export function docSection(full: string, docsUrl: string, kind: MediaKind, model: string): string {+ const slug = model.toLowerCase().replace(/\./g, "-");+ const marker = `\nSource: ${docsUrl}/models/${kind}/${slug}/\n`;+ const at = full.indexOf(marker);+ if (at < 0) return "";+ const start = full.lastIndexOf("\n# ", at);+ const next = full.indexOf("\n---\n\n# ", at + marker.length);+ return full.slice(start < 0 ? at : start + 1, next < 0 ? undefined : next).trim();+}+export class MediaService {+ private catalogCache: { at: number; data: Catalog } | undefined;+ private docsCache: { at: number; text: string } | undefined;+constructor(private o: MediaOptions) {}private get fetch(): typeof fetch {return this.o.fetchImpl ?? fetch;}+ private get docsUrl(): string {+ return (this.o.docsUrl ?? DOCS_URL).replace(/\/+$/, "");+ }- /** bridge 入口:返回给模型的一段文字 */+ /** 直接生成(不经确认,测试与内部用):解析模型后执行 */async run(botId: string, op: string, args: Record<string, unknown>): Promise<string> {- const prompt = String(args.prompt ?? "").trim();- if (!prompt) throw new MediaError("缺少 prompt");- if (op === "image") return this.image(botId, prompt, args);- if (op === "video") return this.video(botId, prompt, args);- throw new MediaError(`未知操作 ${op}`);+ const plan = await this.prepare(botId, op, args);+ return this.execute(botId, plan.model, args);}private key(botId: string): string {@@ -59,10 +219,10 @@ export class MediaService {return key;}- private async call(botId: string, method: string, p: string, body?: unknown, timeoutMs = 300_000): Promise<Response> {+ private async call(key: string, method: string, p: string, body?: unknown, timeoutMs = 300_000): Promise<Response> {const res = await this.fetch(`${this.o.baseUrl()}${p}`, {method,- headers: { Authorization: `Bearer ${this.key(botId)}`, ...this.o.headers(), ...(body !== undefined ? { "content-type": "application/json" } : {}) },+ headers: { Authorization: `Bearer ${key}`, ...this.o.headers(), ...(body !== undefined ? { "content-type": "application/json" } : {}) },body: body !== undefined ? JSON.stringify(body) : undefined,signal: AbortSignal.timeout(timeoutMs),});@@ -74,24 +234,150 @@ export class MediaService {return res;}- private async models(botId: string, kind: "image" | "video"): Promise<MediaModel[]> {- const res = await this.call(botId, "GET", `/models?scope=authorized&kind=${kind}`, undefined, 20_000);- const body = (await res.json()) as { data?: MediaModel[] };- return (body.data ?? []).filter((m) => m.id && (!m.kind || m.kind === kind));+ private async catalog(): Promise<Catalog> {+ if (this.catalogCache && Date.now() - this.catalogCache.at < CACHE_MS) return this.catalogCache.data;+ let data: Catalog = {};+ const url = this.o.pricingUrl?.();+ if (url) {+ try {+ const res = await this.fetch(url, { headers: { Accept: "application/json" }, signal: AbortSignal.timeout(10_000) });+ if (res.ok) data = (await res.json()) as Catalog;+ } catch (e) {+ log.warn("pricing catalog unavailable", { message: (e as Error).message.slice(0, 200) });+ }+ }+ this.catalogCache = { at: Date.now(), data };+ return data;+ }++ /** 这把 Key 能用的图片 / 视频模型,按推荐顺序 */+ async available(key: string, kind: MediaKind): Promise<MediaModel[]> {+ const [res, catalog] = await Promise.all([this.call(key, "GET", `/models?scope=authorized&kind=${kind}`, undefined, 20_000), this.catalog()]);+ const body = (await res.json()) as { data?: Array<{ id?: string; kind?: string; image_route?: string }> };+ const entries = new Map((catalog.data ?? []).map((e) => [e.model_name, e]));+ const out: MediaModel[] = [];+ for (const m of body.data ?? []) {+ if (!m.id || (m.kind && m.kind !== kind)) continue;+ const route = kind === "video" ? "videos" : m.image_route === "gemini-generate-content" ? "gemini" : m.image_route === "images" ? "images" : undefined;+ if (!route) continue;+ const e = entries.get(m.id);+ const options = e?.media_options?.[kind];+ out.push({+ id: m.id,+ kind,+ route,+ options,+ requiresReference: options?.requires_reference === true,+ price: priceNote(e, catalog),+ priority: typeof e?.display_priority === "number" ? e.display_priority : 1e9,+ });+ }+ return sortModels(out);+ }++ async models(botId: string, kind: MediaKind): Promise<MediaModel[]> {+ return this.available(this.key(botId), kind);+ }++ defaultId(kind: MediaKind): string {+ return this.o.defaults?.()[kind] ?? "";+ }++ /** media_models 工具:列出可用模型、默认模型、参数范围与参考价 */+ async listText(botId: string, kind: unknown): Promise<string> {+ const kinds: MediaKind[] = kind === "image" || kind === "video" ? [kind] : ["image", "video"];+ const blocks: string[] = [];+ for (const k of kinds) {+ const list = await this.models(botId, k);+ const name = k === "image" ? "图片" : "视频";+ if (!list.length) {+ blocks.push(`这把 Key 没有可用的${name}模型。`);+ continue;+ }+ const def = pickDefault(list, this.defaultId(k));+ const lines = list.map((m) => `- ${m.id}${m === def ? "(默认)" : ""}${[m.price, optionsNote(m)].filter(Boolean).map((s) => ` — ${s}`).join("")}`);+ blocks.push(`可用的${name}模型(按推荐顺序):\n${lines.join("\n")}`);+ }+ blocks.push("生成前会由程序向用户确认这次用哪个模型。要用参考图、比例、分辨率、时长等参数时,先用 media_docs 读该模型的文档,按文档写 body。");+ return blocks.join("\n\n");+ }++ /** media_docs 工具:该模型在 docs.modelflare.dev 上的那一页(Markdown) */+ async docs(model: unknown): Promise<string> {+ const id = String(model ?? "").trim();+ if (!id) throw new MediaError("缺少 model");+ if (!this.docsCache || Date.now() - this.docsCache.at > CACHE_MS) {+ const res = await this.fetch(`${this.docsUrl}/llms-full.txt`, { signal: AbortSignal.timeout(30_000) }).catch((e: Error) => {+ throw new MediaError(`读取文档失败:${e.message}`);+ });+ if (!res.ok) throw new MediaError(`读取文档失败(HTTP ${res.status})`);+ this.docsCache = { at: Date.now(), text: await res.text() };+ }+ const text = docSection(this.docsCache.text, this.docsUrl, "image", id) || docSection(this.docsCache.text, this.docsUrl, "video", id);+ if (!text) throw new MediaError(`文档里没有 ${id} 的页面,模型列表见 ${this.docsUrl}/models/`);+ const cap = 24_000;+ return `${text.length > cap ? `${text.slice(0, cap)}\n…(已截断)` : text}\n\n---\n按上面的请求格式写 generate_* 工具的 body(不用写 model,也不要自己用 curl 调接口:你没有 Key,程序会代发)。本地文件写成 "workspace:相对路径"(换成 data URL)或 "workspace-base64:相对路径"(换成纯 Base64,例如 Gemini 的 inline_data.data)。`;+ }++ /** 解析这次要用的模型与确认时列出的其他选择 */+ async prepare(botId: string, op: string, args: Record<string, unknown>): Promise<MediaPlan> {+ if (op !== "image" && op !== "video") throw new MediaError(`未知操作 ${op}`);+ const kind: MediaKind = op;+ const prompt = String(args.prompt ?? "").trim();+ if (!prompt) throw new MediaError("缺少 prompt");+ const body = parseBody(args.body);+ const all = await this.models(botId, kind);+ if (!all.length) throw new MediaError(kind === "image" ? "这把 Key 没有可用的图片模型" : "这把 Key 没有可用的视频模型");+ const name = String(args.model ?? "").trim();+ const def = pickDefault(all, this.defaultId(kind))!;+ const model = name ? all.find((m) => m.id === name) : def;+ if (!model) throw new MediaError(`没有这个模型:${name}。可用:${all.map((m) => m.id).join("、")}`);+ const hasRefs = /"workspace(-base64)?:|"https?:\/\/|"data:/.test(JSON.stringify(body ?? {}));+ const candidates = [def, ...all].filter((m, i, a) => m !== model && a.indexOf(m) === i && (hasRefs || !m.requiresReference)).slice(0, 3);+ const params = describeBody(body) || [typeof args.size === "string" ? args.size : "", Number(args.seconds) > 0 ? `${args.seconds} 秒` : ""].filter(Boolean).join(" · ");+ const summary = [`提示词:${prompt.length > 120 ? `${prompt.slice(0, 120)}…` : prompt}`, params && `参数:${params}`, model.price && `参考价:${model.price}`].filter(Boolean).join("\n");+ return { kind, model, default: def, candidates, all, custom: !!body, summary };+ }++ /** 用确定下来的模型生成,返回给模型的一段文字 */+ async execute(botId: string, model: MediaModel, args: Record<string, unknown>): Promise<string> {+ const prompt = String(args.prompt ?? "").trim();+ const body = parseBody(args.body);+ const resolved = body ? (this.inline(botId, body) as Record<string, unknown>) : undefined;+ return model.kind === "image" ? this.image(botId, model, prompt, resolved, args) : this.video(botId, model, prompt, resolved, args);}- private async pick(botId: string, kind: "image" | "video", wanted: unknown): Promise<MediaModel> {- const list = await this.models(botId, kind);- const usable = kind === "image" ? list.filter((m) => m.image_route === "images" || m.image_route === "gemini-generate-content") : list;- if (!usable.length) throw new MediaError(kind === "image" ? "这把 Key 没有可用的图片模型" : "这把 Key 没有可用的视频模型");- const name = String(wanted ?? "").trim();- if (!name) return usable[0];- const hit = usable.find((m) => m.id === name);- if (!hit) throw new MediaError(`没有这个模型:${name}。可用:${usable.map((m) => m.id).join("、")}`);- return hit;+ /** 把请求体里的 "workspace:路径" 换成 data URL、"workspace-base64:路径" 换成纯 Base64;只认工作区里的文件 */+ private inline(botId: string, v: unknown): unknown {+ if (typeof v === "string") {+ const m = FILE_REF.exec(v);+ if (!m) return v;+ const file = this.workspaceFile(botId, m[2].trim());+ const data = fs.readFileSync(file);+ const mime = MIME[path.extname(file).toLowerCase()] ?? "application/octet-stream";+ return m[1] ? data.toString("base64") : `data:${mime};base64,${data.toString("base64")}`;+ }+ if (Array.isArray(v)) return v.map((x) => this.inline(botId, x));+ if (v && typeof v === "object") return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, this.inline(botId, x)]));+ return v;}- private save(botId: string, kind: "image" | "video", data: Buffer, mime: string): string {+ private workspaceFile(botId: string, rel: string): string {+ const root = fs.realpathSync(this.o.workspace(botId));+ let real: string;+ try {+ real = fs.realpathSync(path.resolve(root, rel));+ } catch {+ throw new MediaError(`工作区里没有这个文件:${rel}`);+ }+ if (real !== root && !real.startsWith(root + path.sep)) throw new MediaError(`只能引用工作区里的文件:${rel}`);+ const st = fs.statSync(real);+ if (!st.isFile()) throw new MediaError(`不是文件:${rel}`);+ if (st.size > MAX_REFERENCE_BYTES) throw new MediaError(`参考素材太大(${rel},上限 20 MB)`);+ return real;+ }++ private save(botId: string, kind: MediaKind, data: Buffer, mime: string): string {const dir = path.join(this.o.workspace(botId), "media");fs.mkdirSync(dir, { recursive: true });const stamp = new Date().toISOString().replace(/[-:]/g, "").replace(/\..+$/, "").replace("T", "-");@@ -118,22 +404,24 @@ export class MediaService {return { data, mime: res.headers.get("content-type") || "" };}- private async image(botId: string, prompt: string, args: Record<string, unknown>): Promise<string> {- const model = await this.pick(botId, "image", args.model);+ private async image(botId: string, model: MediaModel, prompt: string, body: Record<string, unknown> | undefined, args: Record<string, unknown>): Promise<string> {+ const key = this.key(botId);const files: string[] = [];- if (model.image_route === "gemini-generate-content") {- const res = await this.call(botId, "POST", `/models/${encodeURIComponent(model.id)}:generateContent`, { contents: [{ role: "user", parts: [{ text: prompt }] }] });- const body = (await res.json()) as { candidates?: Array<{ content?: { parts?: Array<{ inlineData?: { mimeType?: string; data?: string }; inline_data?: { mime_type?: string; data?: string } }> } }> };- for (const c of body.candidates ?? [])+ if (model.route === "gemini") {+ const req = body ? withoutModel(body) : { contents: [{ role: "user", parts: [{ text: prompt }] }] };+ const res = await this.call(key, "POST", `/models/${encodeURIComponent(model.id)}:generateContent`, req);+ const out = (await res.json()) as { candidates?: Array<{ content?: { parts?: Array<{ inlineData?: { mimeType?: string; data?: string }; inline_data?: { mime_type?: string; data?: string } }> } }> };+ for (const c of out.candidates ?? [])for (const p of c.content?.parts ?? []) {const data = p.inlineData?.data ?? p.inline_data?.data;if (data) files.push(this.save(botId, "image", Buffer.from(data, "base64"), p.inlineData?.mimeType ?? p.inline_data?.mime_type ?? "image/png"));}} else {const size = typeof args.size === "string" && /^\d{2,5}x\d{2,5}$/.test(args.size) ? args.size : undefined;- const res = await this.call(botId, "POST", "/images/generations", { model: model.id, prompt, ...(size ? { size } : {}) });- const body = (await res.json()) as { data?: Array<{ b64_json?: string; url?: string }> };- for (const item of body.data ?? []) {+ const req = body ? { ...body, model: model.id } : { model: model.id, prompt, ...(size ? { size } : {}) };+ const res = await this.call(key, "POST", args.endpoint === "edits" ? "/images/edits" : "/images/generations", req);+ const out = (await res.json()) as { data?: Array<{ b64_json?: string; url?: string }> };+ for (const item of out.data ?? []) {if (item.b64_json) files.push(this.save(botId, "image", Buffer.from(item.b64_json, "base64"), "image/png"));else if (item.url) {const d = await this.download(item.url);@@ -146,25 +434,52 @@ export class MediaService {return `已用 ${model.id} 生成 ${files.length} 张图片,保存在工作区:${files.join("、")}。要发给用户,请用 present 工具交付这些文件。`;}- private async video(botId: string, prompt: string, args: Record<string, unknown>): Promise<string> {- const model = await this.pick(botId, "video", args.model);+ private async video(botId: string, model: MediaModel, prompt: string, body: Record<string, unknown> | undefined, args: Record<string, unknown>): Promise<string> {+ const key = this.key(botId);const seconds = Number(args.seconds);const size = typeof args.size === "string" && /^\d{2,5}x\d{2,5}$/.test(args.size) ? args.size : undefined;- const created = (await (await this.call(botId, "POST", "/videos", { model: model.id, prompt, ...(Number.isFinite(seconds) && seconds > 0 ? { seconds: String(Math.round(seconds)) } : {}), ...(size ? { size } : {}) })).json()) as { id?: string; task_id?: string };+ const req = body+ ? { ...body, model: model.id }+ : { model: model.id, prompt, ...(Number.isFinite(seconds) && seconds > 0 ? { seconds: String(Math.round(seconds)) } : {}), ...(size ? { size } : {}) };+ const created = (await (await this.call(key, "POST", "/videos", req)).json()) as { id?: string; task_id?: string };const id = created.id || created.task_id;if (!id) throw new MediaError("Modelflare 没有返回视频任务编号");const deadline = Date.now() + (this.o.videoTimeoutMs ?? 20 * 60_000);for (;;) {- const t = (await (await this.call(botId, "GET", `/videos/${encodeURIComponent(id)}`, undefined, 30_000)).json()) as { status?: string; error?: { message?: string } | null };+ const t = (await (await this.call(key, "GET", `/videos/${encodeURIComponent(id)}`, undefined, 30_000)).json()) as {+ status?: string;+ error?: { message?: string } | null;+ error_message?: string;+ };if (t.status === "completed") break;- if (t.status === "failed") throw new MediaError(`视频生成失败:${t.error?.message || "未知原因"}`);+ if (t.status === "failed") throw new MediaError(`视频生成失败:${t.error?.message || t.error_message || "未知原因"}`);if (Date.now() > deadline) throw new MediaError(`视频任务 ${id} 超时还没完成,可以稍后再试`);await new Promise((r) => setTimeout(r, this.o.pollMs ?? 5000));}- const content = await this.call(botId, "GET", `/videos/${encodeURIComponent(id)}/content`, undefined, 600_000);+ const content = await this.call(key, "GET", `/videos/${encodeURIComponent(id)}/content`, undefined, 600_000);const d = await this.readBody(content);const file = this.save(botId, "video", d.data, d.mime || "video/mp4");log.info("video generated", { bot: botId, model: model.id });return `已用 ${model.id} 生成视频,保存在工作区:${file}。要发给用户,请用 present 工具交付这个文件。`;}}++/** body 可以是对象,也可以是 JSON 文本(有的模型会把它写成字符串) */+function parseBody(v: unknown): Record<string, unknown> | undefined {+ if (v === undefined || v === null || v === "") return undefined;+ let o = v;+ if (typeof v === "string") {+ try {+ o = JSON.parse(v);+ } catch {+ throw new MediaError("body 不是合法的 JSON");+ }+ }+ if (!o || typeof o !== "object" || Array.isArray(o)) throw new MediaError("body 必须是 JSON 对象");+ return o as Record<string, unknown>;+}++function withoutModel(body: Record<string, unknown>): Record<string, unknown> {+ const { model: _model, ...rest } = body;+ return rest;+}diff --git a/src/app.ts b/src/app.tsindex fc2b895..1f357ad 100644--- a/src/app.ts+++ b/src/app.ts@@ -35,7 +35,7 @@ import {import { AccountManager } from "./account/manager.js";import { AccountVault, type SecretBox } from "./account/vault.js";import { loadModelInfo, type ModelInfo } from "./account/model-info.js";-import { MediaService } from "./agent/media.js";+import { MediaService, pickDefault } from "./agent/media.js";import type { BotSettings } from "./agent/settings.js";const log = logger("app");@@ -169,6 +169,8 @@ export class App {keyFor: (botId) => this.botKey(botId),headers: () => this.requestHeaders(),workspace: (botId) => path.join(config.dataDir, "workspaces", botId),+ defaults: () => ({ image: this.config.media.imageModel, video: this.config.media.videoModel }),+ pricingUrl: () => `${modelflareIssuer()}/api/pricing`,}): undefined,onNotice: (text) => this.notice("warn", text),@@ -445,6 +447,30 @@ export class App {saveConfigFile(this.config.dataDir, (f) => (f.llm = { ...(f.llm ?? {}), defaultModel: model }));}+ /** 默认的图片 / 视频模型;空字符串 = 自动(推荐顺序的第一个) */+ setMediaModels(patch: { imageModel?: string; videoModel?: string }): void {+ Object.assign(this.config.media, patch);+ saveConfigFile(this.config.dataDir, (f) => (f.media = { ...(f.media ?? {}), ...patch }));+ }++ /** 设置页与首次设置用:共用 Key 能用的图片 / 视频模型(按推荐顺序)与自动时会选的那个 */+ async mediaModels(): Promise<{ image: string[]; video: string[]; recommended: { image: string; video: string }; error?: string }> {+ const media = this.agent.media;+ const key = this.magpie?.providerKey(this.magpie.modelflareProviderId ?? MODELFLARE_PROVIDER) ?? "";+ const empty = { image: [], video: [], recommended: { image: "", video: "" } };+ if (!media || !key) return empty;+ try {+ const [image, video] = await Promise.all([media.available(key, "image"), media.available(key, "video")]);+ return {+ image: image.map((m) => m.id),+ video: video.map((m) => m.id),+ recommended: { image: pickDefault(image, "")?.id ?? "", video: pickDefault(video, "")?.id ?? "" },+ };+ } catch (e) {+ return { ...empty, error: (e as Error).message };+ }+ }+private seedDefaultBot(): void {if (this.store.listBots().length) return;const bot = this.store.createBot({diff --git a/src/config.ts b/src/config.tsindex 719cfe2..958d3ef 100644--- a/src/config.ts+++ b/src/config.ts@@ -32,6 +32,8 @@ export interface AppConfig {externalApiKey: string;};llm: { defaultModel: string; contextMessages: number; contextChars: number };+ /** 默认的图片 / 视频模型(生成前确认时默认选中);空 = 按推荐顺序的第一个 */+ media: { imageModel: string; videoModel: string };feishu: { throttleMs: number };weixin: { apiBaseUrl: string };/** 代理宿主:dsh = DeepSeek Harness(工具/沙箱/审批);off = 纯对话 */@@ -87,6 +89,10 @@ export function loadConfig(overrides: { dataDir?: string; env?: NodeJS.ProcessEncontextMessages: num(env.AMB_CONTEXT_MESSAGES, file.llm?.contextMessages ?? 40),contextChars: num(env.AMB_CONTEXT_CHARS, file.llm?.contextChars ?? 24000),},+ media: {+ imageModel: typeof file.media?.imageModel === "string" ? file.media.imageModel : "",+ videoModel: typeof file.media?.videoModel === "string" ? file.media.videoModel : "",+ },feishu: { throttleMs: num(env.AMB_FEISHU_THROTTLE_MS, file.feishu?.throttleMs ?? 400) },weixin: { apiBaseUrl: env.AMB_WEIXIN_API_BASE || file.weixin?.apiBaseUrl || "https://ilinkai.weixin.qq.com" },agent: {diff --git a/src/web/server.ts b/src/web/server.tsindex c59c8e5..0136465 100644--- a/src/web/server.ts+++ b/src/web/server.ts@@ -252,6 +252,7 @@ export async function startWebServer(app: App, opts: { port?: number; host?: strawait app.setModelflareLine(normalizeLine(b.line));return send(res, 200, app.modelflareStatus());}+ if (p === "/media/models" && m === "GET") return send(res, 200, await app.mediaModels());if (p === "/models/info" && m === "GET") {if (!app.magpie) return send(res, 200, { models: {} });return send(res, 200, { models: await app.modelInfo(url.searchParams.get("refresh") === "1") });@@ -308,6 +309,12 @@ export async function startWebServer(app: App, opts: { port?: number; host?: strif (m === "PUT") {const b = await readJson(req);if (b.defaultModel !== undefined) app.setDefaultModel(String(b.defaultModel));+ if (b.imageModel !== undefined || b.videoModel !== undefined) {+ app.setMediaModels({+ ...(b.imageModel !== undefined ? { imageModel: String(b.imageModel) } : {}),+ ...(b.videoModel !== undefined ? { videoModel: String(b.videoModel) } : {}),+ });+ }if (b.closeToTray !== undefined) {cfg.ui.closeToTray = !!b.closeToTray;saveConfigFile(cfg.dataDir, (f) => (f.ui = { ...(f.ui ?? {}), closeToTray: cfg.ui.closeToTray }));@@ -326,6 +333,8 @@ export async function startWebServer(app: App, opts: { port?: number; host?: str} else if (m !== "GET") throw new HttpError(405, "method not allowed");return send(res, 200, {defaultModel: cfg.llm.defaultModel,+ imageModel: cfg.media.imageModel,+ videoModel: cfg.media.videoModel,closeToTray: cfg.ui.closeToTray,zoom: cfg.ui.zoom,autostart: app.desktop ? app.desktop.autostart() : { supported: false, on: false },diff --git a/test/account.test.ts b/test/account.test.tsindex 8f3a6ac..5b0bd8d 100644--- a/test/account.test.ts+++ b/test/account.test.ts@@ -12,7 +12,7 @@ import { discover, startLogin, pkcePair } from "../src/account/oauth.js";import { AccountManager, summarizeUsage, type AccountHooks } from "../src/account/manager.js";import { AccountVault } from "../src/account/vault.js";import { referencePrice } from "../src/account/model-info.js";-import { MediaService } from "../src/agent/media.js";+import { MediaService, docSection } from "../src/agent/media.js";import { AgentManager } from "../src/agent/manager.js";import { locateDshRuntime } from "../src/agent/host.js";import { findAppRoot } from "../src/magpie/supervisor.js";@@ -291,6 +291,102 @@ describe("生图 / 生视频", () => {await srv.close();}});++ it("按文档写的请求体原样发出:填上 model,工作区文件换成 data URL / Base64,工作区外的文件不让引用", async () => {+ const bodies: Array<{ path: string; body: any }> = [];+ const srv = await listen(async (req, res) => {+ const u = new URL(req.url!, "http://x");+ const json = (o: unknown) => res.writeHead(200, { "content-type": "application/json" }).end(JSON.stringify(o));+ if (u.pathname === "/v1/models") {+ return json({ data: u.searchParams.get("kind") === "image" ? [{ id: "gemini-img", kind: "image", image_route: "gemini-generate-content" }, { id: "gpt-image", kind: "image", image_route: "images" }] : [{ id: "kling-x", kind: "video" }] });+ }+ if (req.method !== "POST") return res.writeHead(404).end();+ bodies.push({ path: u.pathname, body: await readJson(req) });+ if (u.pathname.endsWith(":generateContent")) return json({ candidates: [{ content: { parts: [{ inlineData: { mimeType: "image/png", data: png.toString("base64") } }] } }] });+ if (u.pathname === "/v1/images/edits") return json({ data: [{ b64_json: png.toString("base64") }] });+ if (u.pathname === "/v1/videos") return json({ id: "t1" });+ res.writeHead(404).end();+ });+ const ws = fs.mkdtempSync(path.join(os.tmpdir(), "amb-media-"));+ fs.mkdirSync(path.join(ws, "media"));+ fs.writeFileSync(path.join(ws, "media", "ref.png"), png);+ try {+ const media = new MediaService({ baseUrl: () => srv.url + "/v1", keyFor: () => "sk", headers: () => ({}), workspace: () => ws, pollMs: 5, videoTimeoutMs: 1 });+ await media.run("b1", "image", {+ prompt: "换成水彩",+ model: "gemini-img",+ body: { model: "ignored", contents: [{ role: "user", parts: [{ text: "换成水彩" }, { inline_data: { mime_type: "image/png", data: "workspace-base64:media/ref.png" } }] }] },+ });+ expect(bodies[0].path).toBe("/v1/models/gemini-img:generateContent");+ expect(bodies[0].body).toEqual({ contents: [{ role: "user", parts: [{ text: "换成水彩" }, { inline_data: { mime_type: "image/png", data: png.toString("base64") } }] }] });++ await media.run("b1", "image", { prompt: "加一顶帽子", model: "gpt-image", endpoint: "edits", body: JSON.stringify({ prompt: "加一顶帽子", images: [{ image_url: "workspace:media/ref.png" }] }) });+ expect(bodies[1]).toEqual({ path: "/v1/images/edits", body: { prompt: "加一顶帽子", images: [{ image_url: `data:image/png;base64,${png.toString("base64")}` }], model: "gpt-image" } });++ // 视频:body 原样带上 model 发出(这个 mock 不回答轮询,只看建任务的请求)+ await expect(media.run("b1", "video", { prompt: "猫", body: { contents: [{ type: "prompt", text: "猫" }], settings: { duration: 5 } } })).rejects.toThrow();+ expect(bodies[2]).toEqual({ path: "/v1/videos", body: { contents: [{ type: "prompt", text: "猫" }], settings: { duration: 5 }, model: "kling-x" } });++ await expect(media.run("b1", "image", { prompt: "x", model: "gemini-img", body: { a: "workspace:../../etc/passwd" } })).rejects.toThrow(/工作区/);+ await expect(media.run("b1", "image", { prompt: "x", model: "gemini-img", body: "{oops" })).rejects.toThrow(/JSON/);+ expect(bodies).toHaveLength(3);+ } finally {+ await srv.close();+ }+ });++ it("默认模型:设置里选的优先;没选时按价格目录的推荐顺序,跳过必须带参考素材的;确认时附参考价", async () => {+ const srv = await listen(async (req, res) => {+ const u = new URL(req.url!, "http://x");+ const json = (o: unknown) => res.writeHead(200, { "content-type": "application/json" }).end(JSON.stringify(o));+ if (u.pathname === "/v1/models") return json({ data: [{ id: "a-i2v", kind: "video" }, { id: "b-t2v", kind: "video" }, { id: "c-pro", kind: "video" }] });+ if (u.pathname === "/api/pricing") {+ return json({+ group_ratio: { Media: 1 },+ usable_group: { Media: "" },+ data: [+ { model_name: "a-i2v", enable_groups: ["Media"], media_options: { video: { requires_reference: true, resolutions: ["720p"] } } },+ { model_name: "b-t2v", enable_groups: ["Media"], video_resolution_price: { "720p": 0.05, "1080p": 0.1 }, media_options: { video: { default_resolution: "720p", seconds: [-1, 4, 10] } } },+ { model_name: "c-pro", enable_groups: ["Media"], display_priority: 5 },+ ],+ });+ }+ res.writeHead(404).end();+ });+ const defaults = { image: "", video: "" };+ try {+ const media = new MediaService({ baseUrl: () => srv.url + "/v1", keyFor: () => "sk", headers: () => ({}), workspace: () => os.tmpdir(), defaults: () => defaults, pricingUrl: () => srv.url + "/api/pricing" });+ const list = await media.models("b1", "video");+ expect(list.map((m) => m.id)).toEqual(["c-pro", "b-t2v", "a-i2v"]);+ expect(list[1].price).toBe("$0.05/秒(720p)");+ let plan = await media.prepare("b1", "video", { prompt: "海浪" });+ expect(plan.model.id).toBe("c-pro");+ expect(plan.candidates.map((m) => m.id)).toEqual(["b-t2v"]);+ defaults.video = "b-t2v";+ plan = await media.prepare("b1", "video", { prompt: "海浪", body: { duration: 10, ratio: "16:9", content: [{ image_url: { url: "workspace:media/x.png" } }] } });+ expect(plan.model.id).toBe("b-t2v");+ expect(plan.custom).toBe(true);+ // 带了参考素材,必须带参考素材的模型也列进可选+ expect(plan.candidates.map((m) => m.id)).toEqual(["c-pro", "a-i2v"]);+ expect(plan.summary).toContain("10 秒");+ expect(plan.summary).toContain("1 个参考素材");+ expect(plan.summary).toContain("参考价:$0.05/秒(720p)");+ expect(await media.listText("b1", "video")).toMatch(/- b-t2v(默认) — \$0\.05\/秒(720p) — 时长 4–10 秒/);+ } finally {+ await srv.close();+ }+ });++ it("文档:从全文里取出该模型那一节", () => {+ const full = [+ "# Overview\n\nSource: https://docs.x/\n\nhello",+ "# kling-3.0 API\n\nSource: https://docs.x/models/video/kling-3-0/\n\nbody A",+ "# kling-3.0-turbo API\n\nSource: https://docs.x/models/video/kling-3-0-turbo/\n\nbody B\n\n## Params\n\n| a |",+ ].join("\n\n---\n\n");+ expect(docSection(full, "https://docs.x", "video", "kling-3.0")).toBe("# kling-3.0 API\n\nSource: https://docs.x/models/video/kling-3-0/\n\nbody A");+ expect(docSection(full, "https://docs.x", "video", "kling-3.0-turbo")).toMatch(/^# kling-3\.0-turbo API[\s\S]*\| a \|$/);+ expect(docSection(full, "https://docs.x", "image", "kling-3.0")).toBe("");+ });});describe("amb-media 插件与 bridge", () => {@@ -308,15 +404,77 @@ describe("amb-media 插件与 bridge", () => {const plugin = await import(path.join(runtime!, "plugins", "amb-media.mjs"));const tools: any[] = [];plugin.apply({ tools: { register: (t: any) => tools.push(t) } });- expect(tools.map((t) => t.name)).toEqual(["generate_image", "generate_video"]);- expect(await tools[0].execute({ prompt: "猫" }, { agent: { session: { id: "s1" } }, signal: AbortSignal.timeout(5000) })).toEqual({ text: "ok" });- expect(seen).toEqual([{ url: "/dsh/media", auth: "Bearer tok", body: { botId: "b1", sessionId: "s1", op: "image", args: { prompt: "猫" } } }]);+ expect(tools.map((t) => t.name)).toEqual(["media_models", "media_docs", "generate_image", "generate_video"]);+ const image = tools.find((t) => t.name === "generate_image");+ const body = { contents: [{ parts: [{ text: "猫" }, { inline_data: { mime_type: "image/png", data: "workspace-base64:media/a.png" } }] }] };+ expect(await image.execute({ prompt: "猫", body }, { agent: { session: { id: "s1" } }, signal: AbortSignal.timeout(5000) })).toEqual({ text: "ok" });+ expect(seen).toEqual([{ url: "/dsh/media", auth: "Bearer tok", body: { botId: "b1", sessionId: "s1", op: "image", args: { prompt: "猫", body } } }]);} finally {process.env = env;await bridge.close();}});+ it("每次生成前都向用户确认模型:确认、换模型、按新模型文档重写后免确认、取消、回复别的话", async () => {+ const store = new Store(":memory:");+ const M1 = { id: "m1", kind: "image", price: "$0.03/张" };+ const M2 = { id: "m2", kind: "image", price: "" };+ const ran: string[] = [];+ const media = {+ prepare: async (_b: string, op: string, args: any) => ({ kind: op, model: args.model === "m2" ? M2 : M1, default: M1, candidates: [M2], all: [M1, M2], custom: !!args.body, summary: "提示词:猫" }),+ execute: async (_b: string, model: { id: string }) => (ran.push(model.id), `done ${model.id}`),+ } as unknown as MediaService;+ const mgr = new AgentManager({ store, dataDir: fs.mkdtempSync(path.join(os.tmpdir(), "amb-mgr-")), runtimeDir: undefined, upstream: () => ({ baseUrl: "", apiKey: "" }), enabled: false, media });+ // 模拟机器人 b1 在会话 c1 里有回合进行中+ (mgr as any).hosts.set("b1", {});+ (mgr as any).activeConvs.set("b1", new Set(["c1"]));+ await mgr.bridge.start();+ const call = (args: Record<string, unknown>) =>+ fetch(`${mgr.bridge.url}/dsh/media`, {+ method: "POST",+ headers: { authorization: `Bearer ${mgr.bridge.token}`, "content-type": "application/json" },+ body: JSON.stringify({ botId: "b1", sessionId: "s1", op: "image", args }),+ }).then((r) => r.json());+ const answer = async (a: { selected?: string[]; custom?: string }) => {+ for (let i = 0; i < 100; i++) {+ const q = mgr.approvals.pending("c1", "question")[0];+ if (q) {+ const [item] = JSON.parse(q.payload);+ mgr.approvals.answer(q.id, { answers: [{ id: "model", selected: a.selected ?? [], ...(a.custom ? { custom: a.custom } : {}) }] }, "test");+ return item;+ }+ await new Promise((r) => setTimeout(r, 10));+ }+ throw new Error("没有等到确认");+ };+ try {+ let [r, q] = await Promise.all([call({ prompt: "猫" }), answer({ selected: ["m1"] })]);+ expect(q.question).toBe("这次用 m1 生成图片吗?");+ expect(q.options.map((o: any) => o.label)).toEqual(["m1", "m2", "取消"]);+ expect(q.options[0].description).toBe("默认 · $0.03/张");+ expect(r).toEqual({ text: "done m1" });++ [r] = await Promise.all([call({ prompt: "猫" }), answer({ selected: ["m2"] })]);+ expect(r).toEqual({ text: "done m2" });++ // 请求体是按 m1 的文档写的:换成 m2 时先让代理按 m2 的文档重写,重写后的那一次不再确认+ [r] = await Promise.all([call({ prompt: "猫", body: { a: 1 } }), answer({ custom: "用 M2" })]);+ expect(r.text).toMatch(/media_docs 查 m2/);+ expect(await call({ prompt: "猫", model: "m2", body: { b: 1 } })).toEqual({ text: "done m2" });++ [r] = await Promise.all([call({ prompt: "猫" }), answer({ selected: ["取消"] })]);+ expect(r.text).toMatch(/取消/);+ [r] = await Promise.all([call({ prompt: "猫" }), answer({ custom: "好的" })]);+ expect(r).toEqual({ text: "done m1" });+ [r] = await Promise.all([call({ prompt: "猫" }), answer({ custom: "换成竖屏的" })]);+ expect(r.text).toMatch(/「换成竖屏的」/);+ expect(ran).toEqual(["m1", "m2", "m2", "m1"]);+ } finally {+ await mgr.bridge.stop();+ store.close();+ }+ });+it("bridge 只在该机器人有回合进行中时受理生图(别的机器人不能借它的 Key)", async () => {const store = new Store(":memory:");const calls: string[] = [];diff --git a/ui/components/model-picker.js b/ui/components/model-picker.jsindex 39f0dbb..7b1c804 100644--- a/ui/components/model-picker.js+++ b/ui/components/model-picker.js@@ -4,7 +4,7 @@ import { useEffect, useMemo, useRef, useState } from "preact/hooks";import { html, Popover, Icon } from "./ui.js";import { api } from "../lib/api.js";import { createStore, useStore } from "../lib/store.js";-import { groupModels, modelFamily, normModel, sameModel } from "../lib/models.js";+import { groupModels, modelFamily, normModel, sameModel, shortModel } from "../lib/models.js";export { groupModels, normModel, sameModel, shortModel } from "../lib/models.js";@@ -154,3 +154,17 @@ export function ModelPicker({ anchor, models, error, current, onPick, onClose, r${foot && html`<div class="foot"><button type="button" disabled=${foot.disabled} onClick=${() => { onClose(); foot.onClick(); }}>${foot.label}</button></div>`}<//>`;}++/**+ * 菜单按钮 + 模型选择器(设置页、首次设置用)。value 为空时按钮显示 resetLabel;+ * 给了 resetLabel 时列表顶部有一项"恢复默认",选它 onChange("")。+ */+export function PickModel({ value, models, modelsError, reloadModels, onChange, resetLabel, align = "right", label = "选择模型" }) {+ const [open, setOpen] = useState(false);+ const ref = useRef(null);+ return html`<button type="button" class="pick" ref=${ref} aria-haspopup="listbox" aria-expanded=${open} aria-label=${label} onClick=${() => setOpen((v) => !v)}>+ <span class="v">${value ? shortModel(value) : resetLabel || label}</span><${Icon} name="chevDown" />+ </button>+ ${open && html`<${ModelPicker} anchor=${ref.current} align=${align} models=${models} error=${modelsError} onRetry=${reloadModels} current=${value}+ onPick=${onChange} onClose=${() => setOpen(false)} reset=${resetLabel ? { label: resetLabel, onSelect: () => onChange("") } : undefined} />`}`;+}diff --git a/ui/css/settings.css b/ui/css/settings.cssindex 9102bb7..6ef863a 100644--- a/ui/css/settings.css+++ b/ui/css/settings.css@@ -140,6 +140,8 @@.onboard details summary:hover { color: var(--fg); }/* "高级"里的行直接排在卡片里,左右贴着卡片的内容边,不再套一块玻璃 */.onboard details .row { --row-pl: 0; --row-pr: 0; margin-top: var(--space-1); }+.onboard .media-pick { display: grid; gap: var(--space-1); }+.onboard .media-pick .row { --row-pl: 0; --row-pr: 0; }.onboard .connected { display: flex; align-items: center; gap: var(--space-2); font-size: var(--fs-lg); font-weight: var(--fw-semibold); }.login-cta { display: grid; gap: var(--space-2-5); }.login-wait { display: flex; align-items: center; justify-content: center; gap: var(--space-2); font-size: var(--fs-md); color: var(--fg-2); }diff --git a/ui/screens/onboarding.js b/ui/screens/onboarding.jsindex 73ee74c..304dad2 100644--- a/ui/screens/onboarding.js+++ b/ui/screens/onboarding.js@@ -4,7 +4,7 @@import { useEffect, useMemo, useRef, useState } from "preact/hooks";import { html, Icon, Notice, Seg, Spinner, CopyButton } from "../components/ui.js";import { BotAvatar, MAIN_CHAR } from "../components/avatar.js";-import { FamilyHead, groupModels, useFamilies } from "../components/model-picker.js";+import { FamilyHead, PickModel, groupModels, useFamilies } from "../components/model-picker.js";import { api } from "../lib/api.js";import { MF_CONSOLE, MF_DOCS, MF_KEYS } from "../lib/links.js";@@ -173,10 +173,27 @@ function ModelStep({ models, initial, onDone }) {const groups = useMemo(() => groupModels(models, q), [models, q]);const { isOpen, toggle } = useFamilies(groups, cur);const folding = !q.trim() && groups.length > 1;+ // 默认的图片 / 视频模型:之前选过且还能用就沿用,否则用推荐的+ const [media, setMedia] = useState(null);+ const [img, setImg] = useState("");+ const [vid, setVid] = useState("");+ useEffect(() => {+ Promise.all([api("/media/models"), api("/settings").catch(() => ({}))])+ .then(([d, st]) => {+ setMedia(d);+ setImg(d.image.includes(st.imageModel) ? st.imageModel : d.recommended.image || "");+ setVid(d.video.includes(st.videoModel) ? st.videoModel : d.recommended.video || "");+ })+ .catch(() => setMedia({ image: [], video: [] }));+ }, []);+ const hasMedia = media && (media.image.length > 0 || media.video.length > 0);const start = async () => {setBusy(true);try {- await api("/settings", { method: "PUT", body: { defaultModel: cur } });+ await api("/settings", {+ method: "PUT",+ body: { defaultModel: cur, ...(media && media.image.length ? { imageModel: img } : {}), ...(media && media.video.length ? { videoModel: vid } : {}) },+ });onDone();} catch (e) {setErr(e.message);@@ -199,7 +216,18 @@ function ModelStep({ models, initial, onDone }) {return html`<button type="button" class="fam" aria-expanded=${open} onClick=${() => toggle(g)}><${FamilyHead} name=${g} count=${ms.length} current=${hit?.[1]} /></button>${open && rows}`;})}${!groups.length && html`<p class="hint">没有匹配的模型</p>`}- </div>`+ </div>+ ${hasMedia && html`<div class="media-pick">+ <p class="hint">生成图片和视频时默认用的模型。每次生成前都会先问你,也可以当场换。</p>+ ${media.image.length > 0 && html`<div class="row compact">+ <div class="who"><div class="name">图片</div></div>+ <${PickModel} value=${img} models=${media.image} onChange=${setImg} label="默认图片模型" />+ </div>`}+ ${media.video.length > 0 && html`<div class="row compact">+ <div class="who"><div class="name">视频</div></div>+ <${PickModel} value=${vid} models=${media.video} onChange=${setVid} label="默认视频模型" />+ </div>`}+ </div>`}`: html`<${Notice} tone="amber" action=${html`<a class="btn" href=${MF_CONSOLE} target="_blank" rel="noopener noreferrer">去控制台 ↗</a>`}>这个 Key 所在的分组暂无可用模型。可以在控制台给分组开放模型后再回来刷新。<//>`}${err && html`<div class="err-text" role="alert">${err}</div>`}<button type="button" class="btn primary lg block" disabled=${busy} onClick=${start}>${busy ? html`<${Spinner} /> 正在保存…` : models.length ? "开始对话" : "仍然开始"}</button>diff --git a/ui/screens/settings.js b/ui/screens/settings.jsindex 0123ee8..9ab311b 100644--- a/ui/screens/settings.js+++ b/ui/screens/settings.js@@ -3,7 +3,7 @@import { useEffect, useMemo, useRef, useState } from "preact/hooks";import { html, Icon, Seg, Switch, Sheet, CopyButton, Skeleton } from "../components/ui.js";import { AccountButton } from "./sidebar.js";-import { ModelPicker, shortModel } from "../components/model-picker.js";+import { PickModel, shortModel } from "../components/model-picker.js";import { KeyForm, LINE_OPTIONS } from "./onboarding.js";import { LoginSheet, NotificationsSheet, UsageSheet } from "./account.js";import { api } from "../lib/api.js";@@ -44,16 +44,6 @@ function Row({ id, name, sub, title, children, flash, off }) {</div>`;}-function PickModel({ value, models, modelsError, reloadModels, onChange, resetLabel }) {- const [open, setOpen] = useState(false);- const ref = useRef(null);- return html`<button type="button" class="pick" ref=${ref} aria-haspopup="listbox" aria-expanded=${open} onClick=${() => setOpen((v) => !v)}>- <span class="v">${value ? shortModel(value) : resetLabel}</span><${Icon} name="chevDown" />- </button>- ${open && html`<${ModelPicker} anchor=${ref.current} align="right" models=${models} error=${modelsError} onRetry=${reloadModels} current=${value}- onPick=${onChange} onClose=${() => setOpen(false)} reset=${{ label: resetLabel, onSelect: () => onChange("") }} />`}`;-}-function NoticesSheet({ onClose }) {const [d, setD] = useState(null);useEffect(() => {@@ -99,9 +89,18 @@ export function SettingsScreen({ tab, query: q, onQuery: setQ, onTab, focusRow,const [flash, setFlash] = useState("");const mode = useStore(themeMode);+ // 图片 / 视频模型(共用 Key 能用的,按推荐顺序):连上 Modelflare 后取+ const [media, setMedia] = useState(null);+ const loadMedia = () =>+ api("/media/models")+ .then(setMedia)+ .catch((e) => setMedia({ image: [], video: [], recommended: {}, error: e.message }));useEffect(() => {api("/settings").then(setS).catch(failed);}, []);+ useEffect(() => {+ if (mf && mf.connected) loadMedia();+ }, [mf && mf.connected]);useEffect(() => {if (bots && bots[0]) api(`/bots/${encodeURIComponent(bots[0].id)}/settings`).then((d) => setCu(d.computerUse)).catch(() => {});}, [bots && bots[0] && bots[0].id]);@@ -229,6 +228,14 @@ export function SettingsScreen({ tab, query: q, onQuery: setQ, onTab, focusRow,{ tab: "modelflare", id: "default-model", name: "默认模型", sub: "机器人没有单独指定时使用", keys: "模型 model",ctl: html`<${PickModel} value=${s.defaultModel} models=${models} modelsError=${modelsError} reloadModels=${reloadModels} resetLabel="自动(列表第一个)"onChange=${(m) => put({ defaultModel: m }, { defaultModel: m })} />` },+ managed && mf.connected && { tab: "modelflare", id: "image-model", name: "默认图片模型", sub: "机器人生成图片时默认选它,每次生成前都会让你确认", keys: "图片 生图 画图 image 模型",+ ctl: html`<${PickModel} value=${s.imageModel} models=${media ? media.image : null} modelsError=${media && media.error} reloadModels=${loadMedia}+ resetLabel=${media && media.recommended.image ? `自动(${shortModel(media.recommended.image)})` : "自动"} label="默认图片模型"+ onChange=${(m) => put({ imageModel: m }, { imageModel: m })} />` },+ managed && mf.connected && { tab: "modelflare", id: "video-model", name: "默认视频模型", sub: "机器人生成视频时默认选它,每次生成前都会让你确认", keys: "视频 生视频 video 模型",+ ctl: html`<${PickModel} value=${s.videoModel} models=${media ? media.video : null} modelsError=${media && media.error} reloadModels=${loadMedia}+ resetLabel=${media && media.recommended.video ? `自动(${shortModel(media.recommended.video)})` : "自动"} label="默认视频模型"+ onChange=${(m) => put({ videoModel: m }, { videoModel: m })} />` },{ tab: "modelflare", id: "models", name: "可用模型", sub: modelsError ? `获取失败:${modelsError}` : "这个 Key 所在分组开放的模型", keys: "列表 刷新",ctl: html`<span class="val num">${models ? `${models.length} 个` : "…"}</span><button type="button" class="text" onClick=${() => reloadModels().then(() => saved())}>刷新</button>` },{ tab: "modelflare", id: "console", name: "Modelflare 控制台", sub: "账单、发票与 API Key 管理", keys: "控制台 账单 console",