Skip to content

Spot it. Save it. Free, local-first Chrome extension for images, videos and prompts. Explore 2,000+ curated cases and connect your library to AI Agents via MCP. 提示词导演

接入智谱 GLM 并锁定本地视频与创作台链路

MigenAI70865d32ecba
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2f47fca..b29e8ac 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,12 @@
# Changelog
+## 1.19.4
+
+- 新增智谱 GLM 服务连接和按任务模型分配,动态读取账号模型目录;GLM-5.3-Flash 可用于文字标签、Skill 提炼、创作规划、图片分析与视频分析,未确认能力的模型不会被静默代替。
+- 案例库本地视频可主动发送给 GLM-5.3-Flash 分析:发送前明确展示本地文件范围并使用原始 Base64;无法编码的格式或只有社交平台播放页时,会在付费请求前停止并说明原因,未公开的服务端大小边界不会被客户端擅自写死。
+- 创作台可选择已分配的智谱规划模型,并按具体模型分别发送 Data URL 或原始 Base64 参考图;不再展示模型没有提供的思考开关,也不会在刷新、失败或媒体生成阶段切换到其他规划服务。
+- AI 服务错误统一显示实际服务名称并脱敏密钥;本地扩展实验室会阻止未声明的外网请求,验证固定扩展身份、案例库、采集器、创作台、升级与可恢复状态,可长期用于发布前自动化验收。
+
## 1.19.3
- ZIP 分享包、完整资料夹备份与手动跨设备同步共用同一套媒体格式、案例身份、项目关系和清理规则,覆盖音频、动图、现代图片、字幕、视频与文档等可携带资源。
diff --git a/README.md b/README.md
index fc415a8..f44d346 100644
--- a/README.md
+++ b/README.md
@@ -13,7 +13,7 @@ PromptDirector 是一个免费、开源、本地优先的 Chrome 扩展。它把
- 浏览经过审核的精选案例,并把需要的内容保存进自己的本地资料库;
- 把案例提炼成可复用的创作 Skill,在创作台组合案例、Skill 和临时附件;
- 生成可编辑的图片或视频提示词,也可以连接自己的服务直接生成媒体;
-- 使用可选的文字分析或画面分析,让资料更容易检索;
+- 使用可选的文字、画面或视频分析,让资料更容易检索;案例库中的本地视频也可在明确确认后发送给已配置且支持视频输入的模型;
- 为选定案例生成分享 ZIP;导入时逐项隔离坏文件,单个不兼容媒体不会阻断其他健康案例;
- 创建可换电脑恢复的完整资料夹备份(包含回收站,并在备份时复制可读取的本机链接原件),可选择安全合并或带回退点的精确恢复,也可通过加密同步文件夹在自己的设备间恢复资料。
@@ -21,7 +21,7 @@ PromptDirector 是一个免费、开源、本地优先的 Chrome 扩展。它把
资料、媒体、标签和设置默认保存在当前浏览器。不会创建账号、投放广告或收集使用统计。
-文字分析、提示词创作和画面分析均由用户主动开启并使用自己的服务密钥。发送范围会在界面中说明;未选择的资料、整库内容和密钥不会随请求发送。详见[隐私政策](store/PRIVACY_POLICY.md)。
+文字分析、提示词创作、画面分析和视频分析均由用户主动开启并使用自己的服务密钥。发送范围会在界面中说明;未选择的资料、整库内容和密钥不会随请求发送。详见[隐私政策](store/PRIVACY_POLICY.md)。
## 安装
diff --git a/ai-model-capabilities.js b/ai-model-capabilities.js
index a94fa3e..702f8b8 100644
--- a/ai-model-capabilities.js
+++ b/ai-model-capabilities.js
@@ -19,6 +19,19 @@ const DEEPSEEK_CONCURRENCY_SOURCE = Object.freeze({
reviewedAt: "2026-08-24"
});
+const ZHIPU_GLM_46V_SOURCE = Object.freeze({
+ authority: "智谱 AI 开放平台",
+ document: "GLM-4.6V",
+ url: "https://docs.bigmodel.cn/cn/guide/models/vlm/glm-4.6v",
+ reviewedAt: "2026-08-28"
+});
+
+const ZHIPU_GLM_53_FLASH_SOURCE = Object.freeze({
+ authority: "智谱 AI 开放平台",
+ document: "GLM-5.3-Flash",
+ url: "https://docs.bigmodel.cn/cn/guide/models/vlm/glm-5.3-flash",
+ reviewedAt: "2026-08-29"
+});
const STANDARD_ASPECT_RATIOS = Object.freeze([
"1:1", "2:3", "3:2", "3:4", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"
]);
@@ -58,6 +71,39 @@ export const AI_MODEL_CAPABILITIES = Object.freeze([
concurrencyLimit: Object.freeze({ value: 2500, source: DEEPSEEK_CONCURRENCY_SOURCE }),
source: DEEPSEEK_VISION_SOURCE
}),
+ Object.freeze({
+ id: "glm-5.3-flash",
+ providerId: "zhipu",
+ protocol: "chat_completions",
+ tasks: Object.freeze(["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"]),
+ inputModalities: Object.freeze(["text", "image", "video", "file"]),
+ outputModalities: Object.freeze(["text"]),
+ supportedParameters: Object.freeze(["response_format"]),
+ supportedResolutions: Object.freeze([]),
+ supportedAspectRatios: Object.freeze([]),
+ contextLength: 1_000_000,
+ structuredOutput: "json_object",
+ structuredOutputTokenBudget: 4096,
+ mediaInput: Object.freeze({
+ imageBase64: "data_url",
+ localVideo: "base64",
+ preferPublicVideoUrl: true,
+ publicVideoUrl: "direct"
+ }),
+ source: ZHIPU_GLM_53_FLASH_SOURCE
+ }),
+ Object.freeze({
+ id: "glm-4.6v",
+ providerId: "zhipu",
+ protocol: "chat_completions",
+ tasks: Object.freeze(["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"]),
+ inputModalities: Object.freeze(["text", "image", "video", "file"]),
+ outputModalities: Object.freeze(["text"]),
+ supportedParameters: Object.freeze([]),
+ supportedResolutions: Object.freeze([]),
+ supportedAspectRatios: Object.freeze([]),
+ source: ZHIPU_GLM_46V_SOURCE
+ }),
googleImageModel({
id: "gemini-3.1-flash-lite-image",
supportedParameters: ["response_format", "aspect_ratio", "image_size"],
@@ -117,3 +163,10 @@ export function getAiModelCapability(providerIdValue, modelIdValue) {
const capability = MODEL_CAPABILITY_INDEX.get(`${providerId}:${modelId}`);
return capability ? structuredClone(capability) : null;
}
+
+export function listAiModelCapabilities(providerIdValue) {
+ const providerId = String(providerIdValue ?? "").trim();
+ return AI_MODEL_CAPABILITIES
+ .filter((item) => item.providerId === providerId)
+ .map((item) => structuredClone(item));
+}
diff --git a/ai-provider-module.js b/ai-provider-module.js
index 7ec7595..ae346e6 100644
--- a/ai-provider-module.js
+++ b/ai-provider-module.js
@@ -1,5 +1,5 @@
import { boundedMediaBlobFromResponse } from "./bounded-media.js";
-import { getAiModelCapability } from "./ai-model-capabilities.js";
+import { getAiModelCapability, listAiModelCapabilities } from "./ai-model-capabilities.js";
import { getAiProviderPreset } from "./ai-provider-presets.js";
const TEXT_TASKS = Object.freeze(["textTags", "skillExtraction", "creativePlanning"]);
@@ -53,12 +53,15 @@ export function createAiProviderModule(options = {}) {
}) : {
providerId: profile.id,
discoveredAt: new Date().toISOString(),
- models: mergeModels(discovery.models, configuredModels(profile).map((model) => ({
- ...model,
- status: Array.isArray(discovery.models) && !discovery.models.some((item) => item.id === model.id) && !profile.id.startsWith("custom")
- ? "unavailable"
- : model.status
- }))),
+ models: mergeModels(discovery.models, configuredModels(profile).map((model) => {
+ const discovered = discovery.models.find((item) => item.id === model.id);
+ return {
+ ...model,
+ status: discovered?.status ?? (Array.isArray(discovery.models) && !profile.id.startsWith("custom")
+ ? "unavailable"
+ : model.status)
+ };
+ })),
cache: discovery.cache,
source: discovery.source
};
@@ -71,12 +74,15 @@ export function createAiProviderModule(options = {}) {
const modelId = clean(modelIdValue);
if (!modelId) throw new Error("缺少要校验的模型名称");
const result = await discoverModels({ ...profileValue, models: {} }, { etag: "" });
- const providerModels = result.models.filter((model) => model.source !== "user_configuration");
+ const providerModels = result.models.filter((model) =>
+ model.status !== "unavailable" && model.source.includes("provider_models")
+ );
+ const available = providerModels.some((model) => model.id === modelId);
return {
providerId: result.providerId,
modelId,
- available: providerModels.some((model) => model.id === modelId),
- verification: "catalog_visible",
+ available,
+ verification: available ? "catalog_visible" : "not_catalog_visible",
executionVerified: false,
visibleModelIds: providerModels.map((model) => model.id)
};
@@ -97,9 +103,25 @@ async function discoverProviderModels(fetchImpl, profile, cache) {
const result = await adapter(fetchImpl, profile, cache);
if (result.notModified) return result;
const allowedTasks = preset ? new Set(preset.capabilities) : null;
+ const officialModels = preset?.catalogCompleteness === "partial"
+ ? listAiModelCapabilities(profile.id).map((capability) => modelDescriptor({ id: capability.id }, {
+ status: "unverified",
+ confidence: "declared",
+ tasks: capability.tasks,
+ inputModalities: capability.inputModalities,
+ outputModalities: capability.outputModalities,
+ supportedParameters: capability.supportedParameters,
+ supportedResolutions: capability.supportedResolutions,
+ supportedAspectRatios: capability.supportedAspectRatios,
+ referenceImages: capability.referenceImages,
+ concurrencyLimit: capability.concurrencyLimit,
+ contextLength: capability.contextLength,
+ source: "official_capabilities"
+ }))
+ : [];
return {
...result,
- models: result.models.map((model) => ({
+ models: mergeModels(result.models, officialModels).map((model) => ({
...model,
tasks: allowedTasks ? model.tasks.filter((task) => allowedTasks.has(task)) : model.tasks
}))
@@ -274,7 +296,7 @@ function modelDescriptor(item = {}, options = {}) {
const descriptor = {
id,
name: clean(item.displayName ?? item.display_name ?? item.name ?? item.id) || id,
- status: clean(item.status) || "available",
+ status: clean(options.status ?? item.status) || "available",
confidence: options.confidence ?? "manual_unverified",
source: options.source ?? "unknown",
tasks: uniqueTaskIds(options.tasks),
@@ -285,7 +307,7 @@ function modelDescriptor(item = {}, options = {}) {
: stringArray(item.supported_parameters ?? item.supportedParameters ?? item.allowed_passthrough_parameters)),
parameterDescriptors: options.parameterDescriptors ?? parameterDescriptors,
supportedMethods: stringArray(options.supportedMethods),
- contextLength: positiveNumber(item.context_length ?? item.contextLength),
+ contextLength: positiveNumber(options.contextLength ?? item.context_length ?? item.contextLength),
pricing: cloneObject(item.pricing ?? item.pricing_skus),
supportedResolutions: options.supportedResolutions ?? stringArray(item.supported_resolutions ?? item.supportedResolutions ?? parameterDescriptors?.resolution?.values),
supportedAspectRatios: options.supportedAspectRatios ?? stringArray(item.supported_aspect_ratios ?? item.supportedAspectRatios ?? parameterDescriptors?.aspect_ratio?.values),
@@ -338,8 +360,9 @@ function mergeModels(...collections) {
merged.set(collection.id, {
...before,
...collection,
+ status: mergedModelStatus(before, collection),
confidence: strongerConfidence(before.confidence, collection.confidence),
- source: collection.confidence === "manual_unverified" && before.confidence !== "manual_unverified" ? before.source : collection.source,
+ source: mergedModelSource(before, collection),
name: metadataPrimary.name || metadataSecondary.name,
tasks: uniqueTaskIds([...(before.tasks ?? []), ...(collection.tasks ?? [])]),
configuredTasks: uniqueTaskIds([...(before.configuredTasks ?? []), ...(collection.configuredTasks ?? [])]),
@@ -358,6 +381,23 @@ function mergeModels(...collections) {
return [...merged.values()];
}
+function mergedModelStatus(before, collection) {
+ const providerVisible = [before, collection].some((item) =>
+ item.status === "available" && item.source?.includes("provider_models")
+ );
+ return providerVisible ? "available" : collection.status ?? before.status;
+}
+
+function mergedModelSource(before, collection) {
+ const sources = [before.source, collection.source].filter(Boolean);
+ if (sources.some((source) => source.includes("provider_models")) && sources.includes("official_capabilities")) {
+ return "provider_models+official_capabilities";
+ }
+ return collection.confidence === "manual_unverified" && before.confidence !== "manual_unverified"
+ ? before.source
+ : collection.source;
+}
+
function strongerConfidence(left, right) {
return confidenceRank(right) > confidenceRank(left) ? right : left;
}
diff --git a/ai-provider-presets.js b/ai-provider-presets.js
index 7713e49..5cb1710 100644
--- a/ai-provider-presets.js
+++ b/ai-provider-presets.js
@@ -5,7 +5,8 @@ function preset(value) {
...value,
discovery: Object.freeze({ ...value.discovery }),
capabilities: Object.freeze([...value.capabilities]),
- catalogRequiredTasks: Object.freeze([...(value.catalogRequiredTasks ?? [])])
+ catalogRequiredTasks: Object.freeze([...(value.catalogRequiredTasks ?? [])]),
+ ...(value.mediaInput ? { mediaInput: Object.freeze({ ...value.mediaInput }) } : {})
});
}
@@ -20,6 +21,23 @@ export const AI_PROVIDER_PRESETS = Object.freeze([
capabilities: [...TEXT_CAPABILITIES, "imageAnalysis"],
structuredOutput: "json_object"
}),
+ preset({
+ id: "zhipu",
+ label: "智谱 GLM",
+ category: "official",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ protocol: "chat_completions",
+ discovery: { adapter: "identity" },
+ catalogCompleteness: "partial",
+ capabilities: [...TEXT_CAPABILITIES, "imageAnalysis", "videoAnalysis"],
+ structuredOutput: "prompt_only",
+ mediaInput: {
+ imageBase64: "raw",
+ localVideo: "unsupported",
+ preferPublicVideoUrl: true,
+ publicVideoUrl: "direct"
+ }
+ }),
preset({
id: "openai",
label: "OpenAI",
diff --git a/ai-provider-registry.js b/ai-provider-registry.js
index 33684d5..785e6fe 100644
--- a/ai-provider-registry.js
+++ b/ai-provider-registry.js
@@ -83,6 +83,7 @@ export function publicAiProviderRegistry(value = {}) {
endpoint: profile.endpoint,
protocol: profile.protocol,
structuredOutput: profile.structuredOutput,
+ mediaInput: structuredClone(profile.mediaInput),
models: structuredClone(profile.models),
capabilities: [...profile.capabilities],
catalogRequiredTasks: [...profile.catalogRequiredTasks],
@@ -225,6 +226,7 @@ function provider(id, value = {}) {
endpoint: clean(value?.endpoint) || preset?.endpoint || "",
protocol: clean(value?.protocol) || preset?.protocol || "native",
structuredOutput: normalizeStructuredOutput(preset?.structuredOutput ?? value?.structuredOutput),
+ mediaInput: normalizeMediaInput(preset?.mediaInput ?? value?.mediaInput),
apiKey: clean(value?.apiKey),
credentialConfigured: Object.hasOwn(value ?? {}, "apiKey")
? Boolean(clean(value?.apiKey) || isLoopback(value?.endpoint))
@@ -358,9 +360,19 @@ function modelAssignmentEvidence(taskId, model = {}, profile = null) {
}
function normalizeStructuredOutput(value) {
+ if (value === "prompt_only") return "prompt_only";
return value === "json_object" ? "json_object" : "json_schema";
}
+function normalizeMediaInput(value = {}) {
+ return {
+ imageBase64: value?.imageBase64 === "raw" ? "raw" : "data_url",
+ localVideo: ["unsupported", "base64"].includes(value?.localVideo) ? value.localVideo : "data_url",
+ preferPublicVideoUrl: value?.preferPublicVideoUrl === true,
+ publicVideoUrl: value?.publicVideoUrl === "direct" ? "direct" : "any_https"
+ };
+}
+
function taskName(taskId) {
return AI_ASSIGNMENT_TASKS.find((task) => task.id === taskId)?.label || taskId;
}
diff --git a/ai-runtime.js b/ai-runtime.js
index 9496137..dfc06a7 100644
--- a/ai-runtime.js
+++ b/ai-runtime.js
@@ -1,4 +1,5 @@
import { normalizeAiSettings } from "./deepseek.js";
+import { getAiModelCapability } from "./ai-model-capabilities.js";
import { normalizeVisionSettings } from "./vision.js";
import {
normalizeAiProviderRegistry,
@@ -50,6 +51,7 @@ export function projectAiRuntime(configurationValue = {}) {
const configuration = normalizeConfiguration(configurationValue);
const { registry, assignments, preferences } = configuration;
const providers = registry.providers;
+ const providerProfiles = providerProfilesWithAssignments(providers, assignments);
const planning = settingsForTextAssignment(assignments.creativePlanning, providers, preferences, false);
const imageAnalysis = assignments.imageAnalysis;
const imageAnalysisProfile = providers[imageAnalysis.providerId] ?? providers["custom-media"];
@@ -74,6 +76,7 @@ export function projectAiRuntime(configurationValue = {}) {
compatible: {
protocol: imageAnalysisProfile?.protocol === "responses" ? "responses" : "chat_completions",
structuredOutput: imageAnalysisProfile?.structuredOutput,
+ imageBase64: imageAnalysisProfile?.mediaInput?.imageBase64,
endpoint: clean(imageAnalysis?.providerId) ? chatEndpoint(imageAnalysisProfile) : "",
model: !imageAnalysisUsesOpenAi ? assignments.imageAnalysis.model : "",
apiKey: !imageAnalysisUsesOpenAi ? usableKey(imageAnalysisProfile) : "",
@@ -112,7 +115,7 @@ export function projectAiRuntime(configurationValue = {}) {
aiSettings: planning,
visionSettings: {
...vision,
- providerProfiles: providers,
+ providerProfiles,
xai: xaiRuntime
},
aiServiceProfiles: {
@@ -136,6 +139,20 @@ export function projectAiRuntime(configurationValue = {}) {
};
}
+function providerProfilesWithAssignments(providers = {}, assignments = {}) {
+ return Object.fromEntries(Object.entries(providers).map(([providerId, profile]) => [providerId, {
+ ...profile,
+ models: {
+ ...profile.models,
+ ...Object.fromEntries(Object.entries(assignments).flatMap(([taskId, assignment]) =>
+ assignment?.providerId === providerId && clean(assignment.model)
+ ? [[taskId, clean(assignment.model)]]
+ : []
+ ))
+ }
+ }]));
+}
+
export function resolveTextTaskSettings(taskId, configurationValue = {}, options = {}) {
if (!TEXT_TASKS.has(taskId)) throw new Error("这不是文字任务");
const configuration = normalizeConfiguration(configurationValue);
@@ -168,6 +185,7 @@ export function resolveVisionTaskSettings(taskId, configurationValue = {}, optio
}
const profile = configuration.registry.providers[resolved.providerId];
if (options.requireConfigured !== false) requireConsent(profile, resolved.provider);
+ const modelRuntime = modelRuntimeSettings(profile, resolved.model);
const openai = resolved.providerId === "openai";
const endpoint = chatEndpoint(profile);
const settings = normalizeVisionSettings({
@@ -182,7 +200,8 @@ export function resolveVisionTaskSettings(taskId, configurationValue = {}, optio
},
compatible: {
protocol: profile.protocol === "responses" ? "responses" : "chat_completions",
- structuredOutput: profile.structuredOutput,
+ structuredOutput: modelRuntime.structuredOutput,
+ imageBase64: modelRuntime.mediaInput.imageBase64,
endpoint,
model: resolved.model,
apiKey: openai ? "" : profile.apiKey,
@@ -209,12 +228,16 @@ export function resolveVideoAnalysisTask(configurationValue = {}) {
const resolved = resolveAiProviderAssignment("videoAnalysis", configuration.registry, configuration.assignments);
const profile = configuration.registry.providers[resolved.providerId];
requireConsent(profile, resolved.provider);
+ const modelRuntime = modelRuntimeSettings(profile, resolved.model);
return {
...resolved,
providerLabel: resolved.provider,
protocol: profile.protocol,
apiKey: profile.apiKey,
- endpoint: profile.endpoint
+ endpoint: profile.endpoint,
+ ...(["unsupported", "base64"].includes(modelRuntime.mediaInput.localVideo) ? { localVideo: modelRuntime.mediaInput.localVideo } : {}),
+ ...(modelRuntime.mediaInput.preferPublicVideoUrl === true ? { preferPublicVideoUrl: true } : {}),
+ ...(modelRuntime.mediaInput.publicVideoUrl === "direct" ? { publicVideoUrl: "direct" } : {})
};
}
@@ -234,6 +257,7 @@ function settingsForTextAssignment(assignmentValue, providers, preferences, stri
const profile = providers[providerId];
if (strict) requireConsent(profile, profile?.label || providerId);
const model = clean(assignmentValue?.model);
+ const modelRuntime = modelRuntimeSettings(profile, model);
const deepseek = providerId === "deepseek";
const settings = normalizeAiSettings({
activeProvider: deepseek ? "deepseek" : "compatible",
@@ -242,9 +266,12 @@ function settingsForTextAssignment(assignmentValue, providers, preferences, stri
consent: strict ? true : profile?.consent === true,
analysisInstructionsByLocale: preferences.textInstructionsByLocale,
compatible: deepseek ? undefined : {
+ label: profile?.label,
endpoint: chatEndpoint(profile),
model,
- apiKey: profile?.apiKey
+ apiKey: profile?.apiKey,
+ structuredOutput: modelRuntime.structuredOutput,
+ structuredOutputTokenBudget: modelRuntime.structuredOutputTokenBudget
}
});
settings.analysisModel = model;
@@ -252,6 +279,18 @@ function settingsForTextAssignment(assignmentValue, providers, preferences, stri
return settings;
}
+function modelRuntimeSettings(profile = {}, modelValue = "") {
+ const capability = getAiModelCapability(profile?.id, modelValue);
+ return {
+ structuredOutput: capability?.structuredOutput ?? profile?.structuredOutput,
+ structuredOutputTokenBudget: capability?.structuredOutputTokenBudget,
+ mediaInput: {
+ ...(profile?.mediaInput ?? {}),
+ ...(capability?.mediaInput ?? {})
+ }
+ };
+}
+
function normalizeConfiguration(value = {}) {
const registry = normalizeAiProviderRegistry(value.registry ?? value.aiProviderRegistry);
return {
diff --git a/background.js b/background.js
index 0c0bf0e..9b80a46 100644
--- a/background.js
+++ b/background.js
@@ -5160,7 +5160,12 @@ async function analyzeEntryVideo(message) {
mode: message.mode,
customQuestion: message.customQuestion,
videoBlob,
- youtubeUrl: sourceUrl,
+ videoUrl: sourceUrl,
+ referenceProvider: asset.reference?.provider,
+ referencePlaybackMode: asset.reference?.playbackMode,
+ localVideo: route.localVideo,
+ preferPublicVideoUrl: route.preferPublicVideoUrl,
+ publicVideoUrl: route.publicVideoUrl,
onStage: (phase) => chrome.runtime.sendMessage({
type: "VIDEO_ANALYSIS_CHANGED", entryId: entry.id, assetId: asset.id, phase,
provider: route.provider, model: route.model
diff --git a/composer-page.js b/composer-page.js
index 8aea53d..625bcfc 100644
--- a/composer-page.js
+++ b/composer-page.js
@@ -4,6 +4,7 @@ import {
clearComposerFailure,
COMPOSER_INPUT_MAX_CHARACTERS,
composerInputUsage,
+ composerProfileForTaskAssignment,
createComposerSession,
createReferenceSnapshots,
imageReferenceModeAvailability,
@@ -410,8 +411,8 @@ async function createNewSession(referenceIds = [], focus = true) {
targetPlatform: composerSettings.lastTargetPlatform,
outputLanguage: composerSettings.outputLanguage,
routeMode: "auto",
- aiProfile: composerProfileForAssignment(composerAiTaskAssignments.creativePlanning, composerSettings.lastAiProfile),
- generationAiProfile: composerProfileForAssignment(
+ aiProfile: composerProfileForTaskAssignment(composerAiTaskAssignments.creativePlanning, composerSettings.lastAiProfile),
+ generationAiProfile: composerProfileForTaskAssignment(
composerAiTaskAssignments[targetType === "video" ? "videoGeneration" : "imageGeneration"],
composerSettings.lastAiProfile
),
@@ -1569,20 +1570,26 @@ function renderComposerAiProfile() {
function renderGenerationModelChoices(selectedProfile) {
const generationMode = ["create_image", "create_video"].includes(composerSession?.outputMode);
- for (const button of [elements.composerModelFlash, elements.composerModelPro, elements.composerModelOpenai, elements.composerModelCompatible, elements.composerModelXai]) {
- button.hidden = generationMode;
- }
- elements.composerModelDynamic.hidden = !generationMode;
- if (!generationMode) return elements.composerModelDynamic.replaceChildren();
- const videoTask = composerSession.outputMode === "create_video";
- const taskId = videoTask ? "videoGeneration" : "imageGeneration";
+ const videoTask = generationMode && composerSession.outputMode === "create_video";
+ const taskId = generationMode ? (videoTask ? "videoGeneration" : "imageGeneration") : "creativePlanning";
const choices = Object.values(composerVisionSettings.providerProfiles ?? {}).flatMap((provider) => {
const model = provider.models?.[taskId];
if (!provider.credentialConfigured || !provider.consent || !provider.capabilities?.includes(taskId) || !model) return [];
- const candidate = composerProfileForAssignment({ providerId: provider.id, model }, selectedProfile);
+ const candidate = composerProfileForTaskAssignment({ providerId: provider.id, model }, selectedProfile);
+ if (!generationMode) {
+ const service = composerServiceCatalog(composerAiSettings, composerVisionSettings)
+ .find((item) => item.serviceId === candidate.serviceId && item.model === candidate.model);
+ return service?.planning ? [{ provider, candidate }] : [];
+ }
const capability = composerServiceCapabilities(candidate, composerVisionSettings)[videoTask ? "video" : "image"];
return capability?.generate ? [{ provider, candidate }] : [];
});
+ const staticFallback = !generationMode && choices.length === 0;
+ for (const button of [elements.composerModelFlash, elements.composerModelPro, elements.composerModelOpenai, elements.composerModelCompatible, elements.composerModelXai]) {
+ button.hidden = !staticFallback;
+ }
+ elements.composerModelDynamic.hidden = staticFallback;
+ if (staticFallback) return elements.composerModelDynamic.replaceChildren();
elements.composerModelDynamic.replaceChildren(...choices.map(({ provider, candidate }) => {
const button = document.createElement("button");
button.type = "button";
@@ -1594,7 +1601,11 @@ function renderGenerationModelChoices(selectedProfile) {
button.addEventListener("click", () => safely(() => updateComposerAiProfile(candidate))());
return button;
}));
- if (!choices.length) elements.composerModelDynamic.append(rawTextEl("p", "composer-model-empty", t("没有已连接且支持当前生成任务的模型")));
+ if (!choices.length) elements.composerModelDynamic.append(rawTextEl(
+ "p",
+ "composer-model-empty",
+ t(generationMode ? "没有已连接且支持当前生成任务的模型" : "没有已连接且支持创作规划的模型")
+ ));
}
function renderImageGenerationSettings() {
@@ -1758,7 +1769,7 @@ function generationRouteProfile(videoTask) {
? composerVideoAvailability(current, composerVisionSettings, composerSession)
: composerImageAvailability(current, composerVisionSettings, composerSession);
if (currentAvailability.available) return current;
- return composerProfileForAssignment(
+ return composerProfileForTaskAssignment(
composerAiTaskAssignments[videoTask ? "videoGeneration" : "imageGeneration"],
current
);
@@ -1770,17 +1781,6 @@ function activeComposerProfile() {
: composerSession?.aiProfile);
}
-function composerProfileForAssignment(assignment, fallback) {
- const providerId = String(assignment?.providerId ?? "");
- const serviceId = providerId === "deepseek" ? "deepseek"
- : providerId === "openai" ? "openai"
- : providerId === "xai" ? "xai"
- : providerId.startsWith("custom") ? "compatible"
- : ["kimi", "gemini", "openrouter", "minimax", "volcengine"].includes(providerId) ? providerId : "";
- if (!serviceId) return normalizeComposerAiProfile(fallback);
- return normalizeComposerAiProfile({ serviceId, model: assignment?.model, thinking: fallback?.thinking === true });
-}
-
async function updateImageGenerationParameters() {
if (!composerSession || activeOperation) return;
const videoTask = composerSession.targetType === "video";
diff --git a/composer-service.js b/composer-service.js
index d09ad7c..188ac32 100644
--- a/composer-service.js
+++ b/composer-service.js
@@ -18,6 +18,7 @@ import { normalizeVisionSettings, OPENAI_RESPONSES_ENDPOINT, OPENAI_VIDEOS_ENDPO
import { PORTABLE_LIBRARY_LIMITS } from "./resource-limits.js";
import { boundedMediaBlobFromResponse, fetchBoundedMedia } from "./bounded-media.js";
import { createAiProviderModule } from "./ai-provider-module.js";
+import { getAiModelCapability } from "./ai-model-capabilities.js";
import {
compatibleImageSizesFor,
compatibleProviderPresetForEndpoint
@@ -67,29 +68,24 @@ export function composerServiceCatalog(aiSettingsValue = {}, visionSettingsValue
const vision = normalizeVisionSettings(visionSettingsValue);
const compatibleLabel = serviceLabelForEndpoint(vision.compatible.endpoint);
const xai = normalizeXaiComposerSettings(visionSettingsValue?.xai);
+ const deepseekProfile = visionSettingsValue?.providerProfiles?.deepseek;
+ const deepseekModels = [...new Set([
+ "deepseek-v4-flash", "deepseek-v4-pro", deepseekProfile?.models?.creativePlanning
+ ].map((value) => String(value ?? "").trim()).filter(Boolean))];
const catalog = [
- {
+ ...deepseekModels.map((model) => ({
serviceId: "deepseek",
- model: "deepseek-v4-flash",
- label: "DeepSeek Flash",
- shortLabel: "Flash",
+ model,
+ label: model === "deepseek-v4-flash" ? "DeepSeek Flash"
+ : model === "deepseek-v4-pro" ? "DeepSeek Pro" : `DeepSeek · ${model}`,
+ shortLabel: model === "deepseek-v4-flash" ? "Flash" : model === "deepseek-v4-pro" ? "Pro" : "DeepSeek",
configured: Boolean(ai.apiKey && ai.consent),
- vision: false,
- reasoning: true,
+ vision: providerModelSupports(deepseekProfile, model, "imageAnalysis"),
+ planning: model === deepseekProfile?.models?.creativePlanning || providerModelSupports(deepseekProfile, model, "creativePlanning"),
+ reasoning: model === deepseekProfile?.models?.creativePlanning || providerModelSupports(deepseekProfile, model, "creativePlanning"),
imageGeneration: false,
videoGeneration: false
- },
- {
- serviceId: "deepseek",
- model: "deepseek-v4-pro",
- label: "DeepSeek Pro",
- shortLabel: "Pro",
- configured: Boolean(ai.apiKey && ai.consent),
- vision: false,
- reasoning: true,
- imageGeneration: false,
- videoGeneration: false
- },
+ })),
{
serviceId: "openai",
model: vision.openai.model,
@@ -97,6 +93,7 @@ export function composerServiceCatalog(aiSettingsValue = {}, visionSettingsValue
shortLabel: "OpenAI",
configured: Boolean(vision.openai.apiKey && vision.consent && vision.openai.model),
vision: true,
+ planning: true,
reasoning: true,
imageGeneration: Boolean(vision.openai.apiKey && vision.consent && vision.openai.model),
videoGeneration: openAiVideoConfigured(vision)
@@ -108,6 +105,7 @@ export function composerServiceCatalog(aiSettingsValue = {}, visionSettingsValue
shortLabel: compatibleLabel,
configured: compatibleConfigured(vision),
vision: true,
+ planning: true,
reasoning: compatibleReasoningSupported(vision.compatible),
imageGeneration: compatibleImageConfigured(vision.compatible),
videoGeneration: false
@@ -119,14 +117,17 @@ export function composerServiceCatalog(aiSettingsValue = {}, visionSettingsValue
shortLabel: "xAI",
configured: Boolean(xai.mediaConsent && xai.apiKey && xai.textModel),
vision: true,
+ planning: true,
reasoning: false,
imageGeneration: Boolean(xai.mediaConsent && xai.apiKey && xai.textModel && xai.imageModel),
videoGeneration: Boolean(xai.mediaConsent && xai.apiKey && xai.textModel && xai.videoModel)
}
];
- for (const providerId of ["kimi", "gemini", "openrouter", "minimax", "volcengine"]) {
- const profile = visionSettingsValue?.providerProfiles?.[providerId];
- if (!profile) continue;
+ const specialProviderIds = new Set(["deepseek", "openai", "xai", "custom-media"]);
+ for (const profile of Object.values(visionSettingsValue?.providerProfiles ?? {})) {
+ const providerId = String(profile?.id ?? "").trim();
+ if (!providerId || specialProviderIds.has(providerId)) continue;
+ if (!profile.capabilities?.some((taskId) => ["creativePlanning", "imageGeneration", "videoGeneration"].includes(taskId))) continue;
const models = [...new Set([profile.models?.creativePlanning, profile.models?.imageGeneration, profile.models?.videoGeneration]
.map((value) => String(value ?? "").trim()).filter(Boolean))];
for (const model of models.length ? models : [""]) {
@@ -137,7 +138,8 @@ export function composerServiceCatalog(aiSettingsValue = {}, visionSettingsValue
shortLabel: profile.label || providerId,
configured: Boolean(profile.consent && profile.apiKey && model),
vision: providerModelSupports(profile, model, "imageAnalysis"),
- reasoning: providerModelSupports(profile, model, "creativePlanning"),
+ planning: model === profile.models?.creativePlanning || providerModelSupports(profile, model, "creativePlanning"),
+ reasoning: false,
imageGeneration: Boolean(profile.consent && profile.apiKey && providerModelSupports(profile, model, "imageGeneration")),
videoGeneration: Boolean(profile.consent && profile.apiKey && providerModelSupports(profile, model, "videoGeneration"))
});
@@ -179,8 +181,9 @@ export function composerServiceCapabilities(profileValue, visionSettingsValue =
video: xaiVideoCapability(configured, xai)
};
}
- if (["kimi", "gemini", "openrouter", "minimax", "volcengine"].includes(profile.serviceId)) {
- const provider = visionSettingsValue?.providerProfiles?.[profile.serviceId];
+ const registryProvider = visionSettingsValue?.providerProfiles?.[profile.serviceId];
+ if (registryProvider && !["openai", "xai", "custom-media"].includes(profile.serviceId)) {
+ const provider = registryProvider;
const model = profile.model;
const descriptor = providerModelDescriptor(provider, model);
const configured = Boolean(provider?.consent && provider?.apiKey && model && descriptor?.tasks?.includes("videoGeneration"));
@@ -475,6 +478,7 @@ export async function executeComposerTurnWithService(input, settingsValue, prepa
async function generateVideoTurn(input, service, settingsValue, preparedImages, options) {
const visionSettingsValue = settingsValue.vision;
+ const generationProfile = input.session?.generationAiProfile ?? input.session?.aiProfile;
if (["gemini", "openrouter", "minimax", "volcengine"].includes(service.serviceId)) {
return generateProviderVideoTurn(input, service, settingsValue, preparedImages, options);
}
@@ -484,7 +488,7 @@ async function generateVideoTurn(input, service, settingsValue, preparedImages,
const remote = normalizeRemoteVideo(options.remoteVideo, "openai");
const state = remote
? { parameters: remote.requestParameters, issues: [] }
- : normalizeVideoGenerationRequest(input.session?.aiProfile, visionSettingsValue, input.session?.generationParameters);
+ : normalizeVideoGenerationRequest(generationProfile, visionSettingsValue, input.session?.generationParameters);
if (state.issues.length) throw new ComposerServiceError(state.issues.join(";"), 422, { retryable: false });
if (preparedImages.length > 1) throw new ComposerServiceError("当前视频服务最多接收 1 张首帧参考图", 422, { retryable: false });
const settings = normalizeVisionSettings(visionSettingsValue);
@@ -536,7 +540,7 @@ async function generateXaiVideoTurn(input, service, visionSettingsValue, prepare
const remote = normalizeRemoteVideo(options.remoteVideo, "xai");
const state = remote
? { parameters: remote.requestParameters, issues: [] }
- : normalizeVideoGenerationRequest(input.session?.aiProfile, visionSettingsValue, input.session?.generationParameters);
+ : normalizeVideoGenerationRequest(input.session?.generationAiProfile ?? input.session?.aiProfile, visionSettingsValue, input.session?.generationParameters);
if (state.issues.length) throw new ComposerServiceError(state.issues.join(";"), 422, { retryable: false });
if (preparedImages.length) {
throw new ComposerServiceError("当前已验证的 xAI 视频生成接口未启用首帧参考图,请移除图片参考后再生成", 422, { retryable: false });
@@ -588,7 +592,8 @@ async function generateProviderVideoTurn(input, service, settingsValue, prepared
if (!profile?.apiKey || !profile?.consent) {
throw new ComposerServiceError(`${profile?.label || service.serviceId} 还缺少 API Key 或媒体发送授权`, 422, { retryable: false });
}
- const capability = composerServiceCapabilities(input.session?.aiProfile, settingsValue.vision).video;
+ const generationProfile = input.session?.generationAiProfile ?? input.session?.aiProfile;
+ const capability = composerServiceCapabilities(generationProfile, settingsValue.vision).video;
const referenceCapability = capability?.inputs?.referenceImages ?? { supported: null, maxItems: null };
if (preparedImages.length && referenceCapability.supported === false) {
throw new ComposerServiceError("当前视频模型不接收原图", 422, { retryable: false });
@@ -599,7 +604,7 @@ async function generateProviderVideoTurn(input, service, settingsValue, prepared
const remote = normalizeRemoteVideo(options.remoteVideo, service.serviceId);
const state = remote
? { parameters: remote.requestParameters, issues: [] }
- : normalizeVideoGenerationRequest(input.session?.aiProfile, settingsValue.vision, input.session?.generationParameters);
+ : normalizeVideoGenerationRequest(generationProfile, settingsValue.vision, input.session?.generationParameters);
if (state.issues.length) throw new ComposerServiceError(state.issues.join(";"), 422, { retryable: false });
const module = createAiProviderModule({
fetchImpl: options.fetchImpl ?? fetch,
@@ -611,10 +616,12 @@ async function generateProviderVideoTurn(input, service, settingsValue, prepared
let job = remote?.job || null;
let promptResult = null;
if (!job) {
- promptResult = await executeDeepSeekTurn({
- ...input,
- session: { ...input.session, outputMode: "text_prompt" }
- }, settingsValue.ai, { ...options, stream: false });
+ promptResult = await assembleImagePrompt(input, preparedImages, {
+ ...options,
+ stream: false,
+ visionSettings: settingsValue.vision,
+ aiSettings: settingsValue.ai
+ });
if (promptResult.kind !== "prompt") throw new ComposerServiceError("视频任务没有生成可提交的最终提示词", 422, { retryable: false });
finalPrompt = promptResult.finalPrompt;
options.onPhase?.("generation");
@@ -697,7 +704,7 @@ async function pollXaiVideo(remoteId, apiKey, options) {
async function videoJsonRequest(url, apiKey, method, body, options, label = "OpenAI 视频服务") {
const response = await videoRequest(url, apiKey, method, body, options, label);
const payload = await response.json().catch(() => ({}));
- if (!response.ok) throw responseError(label, response.status, payload);
+ if (!response.ok) throw responseError(label, response.status, payload, { secrets: [apiKey] });
return payload;
}
@@ -711,7 +718,7 @@ async function videoBlobRequest(url, apiKey, options, label = "OpenAI 视频服
retryable: true
});
}
- throw responseError(label, response.status, payload);
+ throw responseError(label, response.status, payload, { secrets: [apiKey] });
}
try {
return await boundedMediaBlobFromResponse(response, {
@@ -1032,7 +1039,7 @@ async function generateOpenRouterImageTurn(input, service, preparedImages, refer
};
const response = await requestRaw(service.imageGeneration.endpoint, service.apiKey, body, options, IMAGE_REQUEST_TIMEOUT_MS, "OpenRouter 图片服务");
const payload = await response.json().catch(() => ({}));
- if (!response.ok) throw responseError("OpenRouter 图片服务", response.status, payload);
+ if (!response.ok) throw responseError("OpenRouter 图片服务", response.status, payload, { secrets: [service.apiKey] });
const images = (Array.isArray(payload.data) ? payload.data : []).flatMap((item) => {
const encoded = String(item?.b64_json ?? "").trim();
if (!encoded) return [];
@@ -1128,7 +1135,7 @@ async function requestStructured(service, instructions, content, options = {}) {
return parseResponsesPayload(await requestJson(service, body, options), service);
}
const body = chatBody(service, instructions, jsonContent, false);
- body.response_format = { type: "json_object" };
+ if (service.structuredOutput !== "prompt_only") body.response_format = { type: "json_object" };
return parseChatPayload(await requestJson(service, body, options), service);
}
@@ -1147,12 +1154,12 @@ async function requestText(service, instructions, content, options = {}) {
const contentType = String(response.headers?.get?.("content-type") ?? "");
if (!contentType.includes("text/event-stream")) {
const payload = await response.json().catch(() => ({}));
- if (!response.ok) throw responseError(service.label, response.status, payload);
+ if (!response.ok) throw responseError(service.label, response.status, payload, { secrets: [service.apiKey] });
const result = service.protocol === "responses" ? parseResponsesPayload(payload, service) : parseChatPayload(payload, service);
options.onDelta?.(result.content, result.content);
return result;
}
- if (!response.ok) throw responseError(service.label, response.status, await response.json().catch(() => ({})));
+ if (!response.ok) throw responseError(service.label, response.status, await response.json().catch(() => ({})), { secrets: [service.apiKey] });
return service.protocol === "responses"
? readResponsesSse(response, service, options.onDelta)
: readChatSse(response, service, options.onDelta);
@@ -1218,7 +1225,8 @@ async function requestImagesEndpoint(service, prompt, referenceImages, requestPa
const response = await requestRaw(image.endpoint, image.apiKey, body, options, IMAGE_REQUEST_TIMEOUT_MS, service.label);
const payload = await response.json().catch(() => ({}));
if (!response.ok) throw responseError(service.label, response.status, payload, {
- credentialHint: credentialHint(image.apiKey)
+ credentialHint: credentialHint(image.apiKey),
+ secrets: [image.apiKey]
});
const values = Array.isArray(payload?.data) ? payload.data : [];
const images = [];
@@ -1266,7 +1274,8 @@ async function requestImageEdits(service, prompt, referenceImages, requestParame
async function parseImagesEndpointResponse(service, response, options) {
const payload = await response.json().catch(() => ({}));
if (!response.ok) throw responseError(service.label, response.status, payload, {
- credentialHint: credentialHint(service.imageGeneration?.apiKey)
+ credentialHint: credentialHint(service.imageGeneration?.apiKey),
+ secrets: [service.imageGeneration?.apiKey]
});
const values = Array.isArray(payload?.data) ? payload.data : [];
const images = [];
@@ -1303,16 +1312,24 @@ function chatBody(service, instructions, content, stream) {
messages: [
{ role: "system", content: instructions },
{ role: "user", content: content.map((item) => item.type === "image"
- ? { type: "image_url", image_url: { url: item.dataUrl, detail: item.detail } }
+ ? chatImagePart(service, item)
: { type: "text", text: item.text }) }
]
};
}
+function chatImagePart(service, item) {
+ if (service.mediaInput?.imageBase64 === "raw") {
+ const raw = String(item.dataUrl ?? "").split(",", 2)[1] || "";
+ return { type: "image_url", image_url: { url: raw } };
+ }
+ return { type: "image_url", image_url: { url: item.dataUrl, detail: item.detail } };
+}
+
async function requestJson(service, body, options = {}, timeoutMs = REQUEST_TIMEOUT_MS) {
const response = await requestRaw(service.endpoint, service.apiKey, body, options, timeoutMs, service.label);
const payload = await response.json().catch(() => ({}));
- if (!response.ok) throw responseError(service.label, response.status, payload);
+ if (!response.ok) throw responseError(service.label, response.status, payload, { secrets: [service.apiKey] });
return payload;
}
@@ -1379,12 +1396,14 @@ function requireVisualService(profileValue, visionSettingsValue, action) {
}
};
}
- if (["kimi", "gemini", "openrouter", "minimax", "volcengine"].includes(profile.serviceId)) {
- const provider = visionSettingsValue?.providerProfiles?.[profile.serviceId];
+ const registryProvider = visionSettingsValue?.providerProfiles?.[profile.serviceId];
+ if (registryProvider && !["openai", "xai", "custom-media"].includes(profile.serviceId)) {
+ const provider = registryProvider;
const model = profile.model;
+ const modelCapability = getAiModelCapability(profile.serviceId, model);
if (!provider?.apiKey || !model) throw new ComposerServiceError(`请先完成 ${provider?.label || profile.serviceId} 的 API Key 和所选模型配置`, 422, { retryable: false });
if (!provider.consent) throw new ComposerServiceError(`请先确认:主动${action}时会把本轮文字与所选图片发送到 ${provider.label || profile.serviceId}`, 422, { retryable: false });
- const planning = providerModelSupports(provider, model, "creativePlanning");
+ const planning = model === provider.models?.creativePlanning || providerModelSupports(provider, model, "creativePlanning");
const chatCompatible = ["openrouter", "gemini"].includes(profile.serviceId);
const endpoint = profile.serviceId === "openrouter"
? `${String(provider.endpoint).replace(/\/$/, "")}/chat/completions`
@@ -1400,6 +1419,8 @@ function requireVisualService(profileValue, visionSettingsValue, action) {
model,
planning,
reasoningEffort: "",
+ structuredOutput: modelCapability?.structuredOutput ?? provider.structuredOutput,
+ mediaInput: { ...(provider.mediaInput ?? {}), ...(modelCapability?.mediaInput ?? {}) },
provider,
imageGeneration: profile.serviceId === "openrouter" ? {
protocol: "openrouter_images",
@@ -1739,7 +1760,7 @@ async function uploadOpenAiImage(service, dataUrl, stem, options) {
body.append("file", blob, `${stem}.${imageExtension(blob.type) || "png"}`);
const response = await requestOpenAiFile(service, OPENAI_FILES_ENDPOINT, "POST", body, options);
const payload = await response.json().catch(() => ({}));
- if (!response.ok) throw responseError(service.label, response.status, payload);
+ if (!response.ok) throw responseError(service.label, response.status, payload, { secrets: [service.apiKey] });
const fileId = String(payload?.id ?? "").trim();
if (!fileId) throw new ComposerServiceError("OpenAI 没有返回临时图片文件 ID", 503);
return fileId;
diff --git a/composer.js b/composer.js
index 916cb5b..fed1f13 100644
--- a/composer.js
+++ b/composer.js
@@ -10,6 +10,7 @@ import {
referenceHasUsableLegacyDescription
} from "./reference-readiness.js";
import { validReconstructionPrompt } from "./image-prompt.js";
+import { AI_PROVIDER_PRESETS } from "./ai-provider-presets.js";
import {
COMPOSER_AGENT_VERSION,
AGENT_ROUTES,
@@ -30,7 +31,10 @@ import {
export const COMPOSER_METHOD_VERSION = COMPOSER_AGENT_VERSION;
export const COMPOSER_AI_MODELS = Object.freeze(["deepseek-v4-flash", "deepseek-v4-pro"]);
export const COMPOSER_SERVICE_IDS = Object.freeze([
- "deepseek", "openai", "compatible", "xai", "kimi", "gemini", "openrouter", "minimax", "volcengine"
+ ...AI_PROVIDER_PRESETS
+ .filter((preset) => preset.capabilities.some((taskId) => ["creativePlanning", "imageGeneration", "videoGeneration"].includes(taskId)))
+ .map((preset) => preset.id),
+ "compatible"
]);
export const DEFAULT_COMPOSER_AI_PROFILE = Object.freeze({ serviceId: "deepseek", model: "deepseek-v4-flash", thinking: false });
export const COMPOSER_INPUT_MAX_CHARACTERS = 750_000;
@@ -50,17 +54,32 @@ export function normalizeComposerSettings(value = {}) {
}
export function normalizeComposerAiProfile(value = {}) {
- const serviceId = COMPOSER_SERVICE_IDS.includes(value?.serviceId) ? value.serviceId : "deepseek";
+ const requestedServiceId = String(value?.serviceId ?? "").trim();
+ const serviceId = requestedServiceId && COMPOSER_SERVICE_IDS.includes(requestedServiceId)
+ ? requestedServiceId
+ : "deepseek";
const requestedModel = String(value?.model ?? "").trim();
return {
serviceId,
- model: serviceId === "deepseek"
- ? COMPOSER_AI_MODELS.includes(requestedModel) ? requestedModel : DEFAULT_COMPOSER_AI_PROFILE.model
- : requestedModel,
+ model: requestedServiceId && !COMPOSER_SERVICE_IDS.includes(requestedServiceId)
+ ? DEFAULT_COMPOSER_AI_PROFILE.model
+ : requestedModel || (serviceId === "deepseek" ? DEFAULT_COMPOSER_AI_PROFILE.model : ""),
thinking: value?.thinking === true
};
}
+export function composerProfileForTaskAssignment(assignmentValue = {}, fallbackValue = {}) {
+ const providerId = String(assignmentValue?.providerId ?? "").trim();
+ if (!providerId) return normalizeComposerAiProfile(fallbackValue);
+ const serviceId = providerId === "custom-media" ? "compatible" : providerId;
+ if (!COMPOSER_SERVICE_IDS.includes(serviceId)) return normalizeComposerAiProfile(fallbackValue);
+ return normalizeComposerAiProfile({
+ serviceId,
+ model: assignmentValue?.model,
+ thinking: fallbackValue?.thinking === true
+ });
+}
+
export function updateComposerMethod(settingsValue, { targetType, locale, text }) {
return updateComposerTaskMethod(settingsValue, { taskKey: `compose.${targetType === "video" ? "video" : "image"}`, text, locale });
}
diff --git a/deepseek.js b/deepseek.js
index 299ffa3..e76146f 100644
--- a/deepseek.js
+++ b/deepseek.js
@@ -54,7 +54,7 @@ export function deepSeekErrorDetails(error) {
if (error instanceof TypeError && /fetch|network|load/i.test(String(error.message ?? ""))) {
return { kind: "network", message: "网络连接失败,请检查服务地址、权限或网络后重试", retryable: true };
}
- return { kind: "unknown", message: error?.message || "DeepSeek 处理失败", retryable: false };
+ return { kind: "unknown", message: error?.message || "AI 服务处理失败", retryable: false };
}
export function normalizeAiSettings(value = {}) {
@@ -65,9 +65,14 @@ export function normalizeAiSettings(value = {}) {
analysisModel: String(value.analysisModel ?? "").trim() || DEFAULT_ANALYSIS_MODEL,
analysisInstructionsByLocale: normalizeAnalysisInstructions(value.analysisInstructionsByLocale),
compatible: {
+ ...(String(value.compatible?.label ?? "").trim() ? { label: String(value.compatible.label).trim() } : {}),
endpoint: String(value.compatible?.endpoint ?? "").trim(),
model: String(value.compatible?.model ?? "").trim(),
- apiKey: String(value.compatible?.apiKey ?? "").trim()
+ apiKey: String(value.compatible?.apiKey ?? "").trim(),
+ structuredOutput: value.compatible?.structuredOutput === "prompt_only" ? "prompt_only" : "json_object",
+ ...(positiveInteger(value.compatible?.structuredOutputTokenBudget)
+ ? { structuredOutputTokenBudget: positiveInteger(value.compatible.structuredOutputTokenBudget) }
+ : {})
}
};
}
@@ -90,9 +95,13 @@ export function mergeAiSettings(currentValue = {}, incomingValue = {}) {
en: incoming.analysisInstructionsByLocale?.en ?? current.analysisInstructionsByLocale.en
},
compatible: {
+ label: incoming.compatible?.label ?? current.compatible.label,
endpoint: nextEndpoint,
model: incoming.compatible?.model ?? current.compatible.model,
- apiKey: incomingCompatibleKey || (credentialReset ? "" : current.compatible.apiKey)
+ apiKey: incomingCompatibleKey || (credentialReset ? "" : current.compatible.apiKey),
+ structuredOutput: incoming.compatible?.structuredOutput ?? current.compatible.structuredOutput,
+ structuredOutputTokenBudget: incoming.compatible?.structuredOutputTokenBudget
+ ?? current.compatible.structuredOutputTokenBudget
}
});
if (incoming.clearApiKey) {
@@ -113,6 +122,7 @@ export function publicAiSettings(value = {}) {
consent: settings.consent,
analysisInstructionsByLocale: structuredClone(settings.analysisInstructionsByLocale),
compatible: {
+ ...(settings.compatible.label ? { label: settings.compatible.label } : {}),
endpoint: settings.compatible.endpoint,
model: settings.compatible.model,
configured: Boolean(settings.compatible.apiKey || isLoopbackEndpoint(settings.compatible.endpoint))
@@ -131,9 +141,10 @@ export async function analyzeTextWithDeepSeek(entry, catalogValue, settingsValue
export async function analyzeTextDetailedWithDeepSeek(entry, catalogValue, settingsValue, fetchImpl = fetch, requestOptions = {}) {
const settings = requireAiSettings(settingsValue, "分析");
+ const providerLabel = aiProvider(settings).label;
const outputLocale = settingsValue?.outputLocale === "en" ? "en" : "zh-CN";
const input = requestOptions.analysisInput || canonicalTextAnalysisInput(entry);
- if (!input.text) throw new Error("这条案例没有文字,DeepSeek 文字分析会跳过");
+ if (!input.text) throw new Error(`这条案例没有文字,${providerLabel} 文字分析会跳过`);
const systemMessages = [
{ role: "system", content: analysisSystemInstruction(outputLocale) },
{ role: "system", content: analysisTaxonomyPrompt(catalogValue, outputLocale) },
@@ -150,14 +161,15 @@ export async function analyzeTextDetailedWithDeepSeek(entry, catalogValue, setti
const requestStartedAt = Date.now();
emitAnalysisDiagnostic(requestOptions, "request_started", { attempt });
try {
- result = await requestDeepSeek({
+ result = await requestDeepSeek(structuredRequestBody({
model: settings.analysisModel,
- response_format: { type: "json_object" },
thinking: { type: "disabled" },
temperature: 0.1,
- max_tokens: 1000,
+ max_tokens: settings.activeProvider === "compatible"
+ ? settings.compatible.structuredOutputTokenBudget || 1000
+ : 1000,
messages
- }, settings, {
+ }, settings), settings, {
fetchImpl,
signal: requestOptions.signal,
timeoutMs: outputAttempt
@@ -180,7 +192,7 @@ export async function analyzeTextDetailedWithDeepSeek(entry, catalogValue, setti
}
usage = addNormalizedUsage(usage, result.usage);
try {
- const parsed = parseJsonObject(result.content, "DeepSeek 返回格式不正确,本次没有写入任何标签");
+ const parsed = parseJsonObject(result.content, `${providerLabel} 返回格式不正确,本次没有写入任何标签`);
const tagCount = Array.isArray(parsed.tags) ? parsed.tags.length : null;
emitAnalysisDiagnostic(requestOptions, "response_received", {
attempt,
@@ -213,9 +225,8 @@ export async function analyzeTextDetailedWithDeepSeek(entry, catalogValue, setti
export async function summarizeVisualSetWithAi(input, settingsValue, options = {}) {
const settings = requireAiSettings(settingsValue, "总结整组图片");
- const result = await requestDeepSeek({
+ const result = await requestDeepSeek(structuredRequestBody({
model: settings.analysisModel,
- response_format: { type: "json_object" },
thinking: { type: "disabled" },
temperature: 0.1,
max_tokens: 6000,
@@ -223,7 +234,7 @@ export async function summarizeVisualSetWithAi(input, settingsValue, options = {
{ role: "system", content: String(options.instruction ?? "").trim() },
{ role: "user", content: JSON.stringify(input) }
]
- }, settings, {
+ }, settings), settings, {
fetchImpl: options.fetchImpl ?? fetch,
signal: options.signal,
timeoutMs: options.timeoutMs,
@@ -270,9 +281,8 @@ export async function analysisProfileFingerprint(settingsValue = {}, outputLocal
export async function organizeDetailTagsWithDeepSeek(chunk, settingsValue, fetchImpl = fetch) {
const settings = requireAiSettings(settingsValue, "整理三级标签");
- const result = await requestDeepSeek({
+ const result = await requestDeepSeek(structuredRequestBody({
model: settings.analysisModel,
- response_format: { type: "json_object" },
thinking: { type: "disabled" },
temperature: 0,
max_tokens: 4000,
@@ -283,7 +293,7 @@ export async function organizeDetailTagsWithDeepSeek(chunk, settingsValue, fetch
},
{ role: "user", content: JSON.stringify(chunk) }
]
- }, settings, { fetchImpl, timeoutMessage: "AI 整理超时,正式标签库没有改变" });
+ }, settings), settings, { fetchImpl, timeoutMessage: "AI 整理超时,正式标签库没有改变" });
const parsed = parseJsonObject(result.content, "AI 整理结果格式无效,正式标签库没有改变");
return {
mappings: validateDetailOrganizationResponse(parsed, chunk),
@@ -294,12 +304,12 @@ export async function organizeDetailTagsWithDeepSeek(chunk, settingsValue, fetch
export async function planComposerTurn(input, settingsValue, options = {}) {
const settings = requireAiSettings(settingsValue, "生成");
+ const providerLabel = aiProvider(settings).label;
const profile = normalizeComposerAiProfile(input.session?.aiProfile);
assertComposerInputBudget(input.session, input.userMessage, input.composerSettings);
const request = plannerRequestPayload(input.session, input.userMessage, input.composerSettings);
- const body = withComposerProfile({
+ const body = withComposerProfile(structuredRequestBody({
model: profile.model,
- response_format: { type: "json_object" },
messages: [
{ role: "system", content: compileAgentPlanningPrompt({
settings: input.composerSettings,
@@ -310,15 +320,15 @@ export async function planComposerTurn(input, settingsValue, options = {}) {
}) },
{ role: "user", content: JSON.stringify(request) }
]
- }, profile);
+ }, settings), profile);
assertComposerRequestBudget(body.messages);
const result = await requestDeepSeek(body, settings, {
fetchImpl: options.fetchImpl ?? fetch,
signal: options.signal,
timeoutMs: options.timeoutMs,
- timeoutMessage: "DeepSeek 规划超时,本次没有开始生成"
+ timeoutMessage: `${providerLabel} 规划超时,本次没有开始生成`
});
- const parsed = parseJsonObject(result.content, "DeepSeek 没有返回有效的 Agent 计划");
+ const parsed = parseJsonObject(result.content, `${providerLabel} 没有返回有效的 Agent 计划`);
const latestUserMessage = [...request.messages].reverse().find((item) => item.role === "user")?.content ?? "";
const planned = normalizePlannerResult(parsed, {
route: request.routeMode === "auto" ? "compose" : request.routeMode,
@@ -337,6 +347,7 @@ export async function planComposerTurn(input, settingsValue, options = {}) {
export async function streamComposedPrompt(input, settingsValue, options = {}) {
const settings = requireAiSettings(settingsValue, "生成");
+ const provider = aiProvider(settings);
const profile = normalizeComposerAiProfile(input.session?.aiProfile);
assertComposerInputBudget(input.session, input.userMessage, input.composerSettings);
const request = plannerRequestPayload(input.session, input.userMessage, input.composerSettings);
@@ -370,9 +381,9 @@ export async function streamComposedPrompt(input, settingsValue, options = {}) {
let streamed;
try {
const response = await fetchDeepSeekStream(body, settings, options.fetchImpl ?? fetch, requestController.signal);
- streamed = await readDeepSeekSse(response, options.onDelta);
+ streamed = await readDeepSeekSse(response, options.onDelta, provider.label, provider.apiKey);
} catch (error) {
- if (timedOut) throw new DeepSeekApiError("DeepSeek 流式输出超时,本次结果未保存", 408);
+ if (timedOut) throw new DeepSeekApiError(`${provider.label} 流式输出超时,本次结果未保存`, 408);
throw error;
} finally {
clearTimeout(timeoutId);
@@ -428,6 +439,7 @@ export async function executeAgentTurn(input, settingsValue, options = {}) {
}
async function streamAgentText({ settings, profile, systemInstruction, executionRequest, options }) {
+ const provider = aiProvider(settings);
const body = withComposerProfile({
model: profile.model,
stream: true,
@@ -449,11 +461,11 @@ async function streamAgentText({ settings, profile, systemInstruction, execution
const timeoutId = timeoutMs === null ? null : setTimeout(() => { timedOut = true; controller.abort(); }, timeoutMs);
try {
const response = await fetchDeepSeekStream(body, settings, options.fetchImpl ?? fetch, controller.signal);
- const result = await readDeepSeekSse(response, options.onDelta);
- if (!String(result.content ?? "").trim()) throw new DeepSeekApiError("DeepSeek 没有返回完整内容", 422);
+ const result = await readDeepSeekSse(response, options.onDelta, provider.label, provider.apiKey);
+ if (!String(result.content ?? "").trim()) throw new DeepSeekApiError(`${provider.label} 没有返回完整内容`, 422);
return result;
} catch (error) {
- if (timedOut) throw new DeepSeekApiError("DeepSeek 流式输出超时,本次结果未保存", 408);
+ if (timedOut) throw new DeepSeekApiError(`${provider.label} 流式输出超时,本次结果未保存`, 408);
throw error;
} finally {
if (timeoutId !== null) clearTimeout(timeoutId);
@@ -461,8 +473,8 @@ async function streamAgentText({ settings, profile, systemInstruction, execution
}
}
-export async function readDeepSeekSse(response, onDelta = () => undefined) {
- if (!response?.body?.getReader) throw new DeepSeekApiError("DeepSeek 没有返回流式内容", 503);
+export async function readDeepSeekSse(response, onDelta = () => undefined, providerLabel = "DeepSeek", apiKey = "") {
+ if (!response?.body?.getReader) throw new DeepSeekApiError(`${providerLabel} 没有返回流式内容`, 503);
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
@@ -477,8 +489,8 @@ export async function readDeepSeekSse(response, onDelta = () => undefined) {
if (!data) return;
if (data === "[DONE]") { done = true; return; }
let event;
- try { event = JSON.parse(data); } catch { throw new DeepSeekApiError("DeepSeek 流式结果格式错误,本次结果未保存", 422); }
- if (event?.error) throw new DeepSeekApiError(`DeepSeek 流式生成失败:${String(event.error.message ?? "服务返回错误")}`, Number(event.error.status) || 503);
+ try { event = JSON.parse(data); } catch { throw new DeepSeekApiError(`${providerLabel} 流式结果格式错误,本次结果未保存`, 422); }
+ if (event?.error) throw new DeepSeekApiError(`${providerLabel} 流式生成失败:${redactSecret(String(event.error.message ?? "服务返回错误"), apiKey)}`, Number(event.error.status) || 503);
model = String(event.model ?? model);
if (event.usage) usage = normalizeUsage(event.usage);
const choice = event.choices?.[0];
@@ -502,10 +514,10 @@ export async function readDeepSeekSse(response, onDelta = () => undefined) {
} finally {
reader.releaseLock?.();
}
- if (finishReason === "length") throw new DeepSeekApiError("DeepSeek 输出被截断,本次结果未保存", 422);
- if (finishReason === "content_filter") throw new DeepSeekApiError("DeepSeek 未能返回此内容,本次结果未保存", 422);
- if (!done || finishReason !== "stop") throw new DeepSeekApiError("DeepSeek 流式输出意外中断,本次结果未保存", 503);
- if (!content.trim()) throw new DeepSeekApiError("DeepSeek 没有返回完整提示词", 503);
+ if (finishReason === "length") throw new DeepSeekApiError(`${providerLabel} 输出被截断,本次结果未保存`, 422);
+ if (finishReason === "content_filter") throw new DeepSeekApiError(`${providerLabel} 未能返回此内容,本次结果未保存`, 422);
+ if (!done || finishReason !== "stop") throw new DeepSeekApiError(`${providerLabel} 流式输出意外中断,本次结果未保存`, 503);
+ if (!content.trim()) throw new DeepSeekApiError(`${providerLabel} 没有返回完整提示词`, 503);
return { content, usage, model, finishReason };
}
@@ -524,6 +536,12 @@ function requireAiSettings(value, action) {
return settings;
}
+function structuredRequestBody(body, settings) {
+ return settings.activeProvider === "compatible" && settings.compatible.structuredOutput === "prompt_only"
+ ? body
+ : { ...body, response_format: { type: "json_object" } };
+}
+
async function requestDeepSeek(body, settings, options = {}) {
const provider = aiProvider(settings);
const requestModel = settings.activeProvider === "compatible" ? provider.model : body.model || provider.model;
@@ -543,10 +561,13 @@ async function requestDeepSeek(body, settings, options = {}) {
let response;
let payload = {};
try {
+ const requestBody = settings.activeProvider === "compatible"
+ ? Object.fromEntries(Object.entries(body).filter(([key]) => key !== "thinking"))
+ : body;
response = await fetchImpl(provider.endpoint, {
method: "POST",
headers: { "Content-Type": "application/json", ...(provider.apiKey ? { Authorization: `Bearer ${provider.apiKey}` } : {}) },
- body: JSON.stringify({ ...body, model: requestModel }),
+ body: JSON.stringify({ ...requestBody, model: requestModel }),
signal: requestController.signal
});
try {
@@ -556,24 +577,24 @@ async function requestDeepSeek(body, settings, options = {}) {
payload = {};
}
} catch (error) {
- if (timedOut) throw new DeepSeekApiError(options.timeoutMessage || "DeepSeek 请求超时,本次没有保存", 408);
+ if (timedOut) throw new DeepSeekApiError(options.timeoutMessage || `${provider.label} 请求超时,本次没有保存`, 408);
if (error?.name === "AbortError") throw error;
- throw new DeepSeekApiError("无法连接 DeepSeek,请检查网络后重试", 0, { cause: error });
+ throw new DeepSeekApiError(`无法连接 ${provider.label},请检查网络后重试`, 0, { cause: error });
} finally {
clearTimeout(timeoutId);
options.signal?.removeEventListener("abort", onExternalAbort);
}
if (!response.ok) {
- throw new DeepSeekApiError(apiError(payload, response.status), response.status, {
+ throw new DeepSeekApiError(apiError(payload, response.status, provider), response.status, {
retryAfterMs: retryAfterMilliseconds(response.headers?.get?.("retry-after"))
});
}
const choice = payload?.choices?.[0];
const finishReason = String(choice?.finish_reason ?? "");
const content = choice?.message?.content;
- if (!content) throw new DeepSeekApiError("DeepSeek 没有返回可用结果", 503);
- if (!options.allowPartialContent && finishReason === "length") throw new DeepSeekApiError("DeepSeek 输出被截断,本次结果未应用", 422);
- if (!options.allowPartialContent && finishReason === "content_filter") throw new DeepSeekApiError("DeepSeek 未能返回此内容,本次结果未应用", 422);
+ if (!content) throw new DeepSeekApiError(`${provider.label} 没有返回可用结果`, 503);
+ if (!options.allowPartialContent && finishReason === "length") throw new DeepSeekApiError(`${provider.label} 输出被截断,本次结果未应用`, 422);
+ if (!options.allowPartialContent && finishReason === "content_filter") throw new DeepSeekApiError(`${provider.label} 未能返回此内容,本次结果未应用`, 422);
return {
content,
finishReason,
@@ -595,11 +616,11 @@ async function fetchDeepSeekStream(body, settings, fetchImpl, signal) {
});
} catch (error) {
if (error?.name === "AbortError") throw error;
- throw new DeepSeekApiError("无法连接 DeepSeek,请检查网络后重试", 0, { cause: error });
+ throw new DeepSeekApiError(`无法连接 ${provider.label},请检查网络后重试`, 0, { cause: error });
}
if (!response.ok) {
const payload = await response.json().catch(() => ({}));
- throw new DeepSeekApiError(apiError(payload, response.status), response.status);
+ throw new DeepSeekApiError(apiError(payload, response.status, provider), response.status);
}
return response;
}
@@ -608,9 +629,9 @@ function aiProvider(settingsValue) {
const settings = normalizeAiSettings(settingsValue);
if (settings.activeProvider === "compatible") {
const details = compatibleEndpoint(settings.compatible.endpoint);
- return { endpoint: details.url, apiKey: settings.compatible.apiKey, model: settings.compatible.model, loopback: details.loopback };
+ return { endpoint: details.url, apiKey: settings.compatible.apiKey, model: settings.compatible.model, loopback: details.loopback, label: settings.compatible.label || "所选 AI 服务" };
}
- return { endpoint: DEEPSEEK_ENDPOINT, apiKey: settings.apiKey, model: settings.analysisModel, loopback: false };
+ return { endpoint: DEEPSEEK_ENDPOINT, apiKey: settings.apiKey, model: settings.analysisModel, loopback: false, label: "DeepSeek" };
}
function compatibleEndpoint(value) {
@@ -733,6 +754,11 @@ function finite(value) {
return Number.isFinite(number) && number > 0 ? Math.round(number) : 0;
}
+function positiveInteger(value) {
+ const number = Number(value);
+ return Number.isInteger(number) && number > 0 ? number : 0;
+}
+
function retryAfterMilliseconds(value) {
const seconds = Number(value);
if (Number.isFinite(seconds) && seconds >= 0) return seconds * 1000;
@@ -740,11 +766,17 @@ function retryAfterMilliseconds(value) {
return Number.isFinite(date) ? Math.max(0, date - Date.now()) : 0;
}
-function apiError(payload, status) {
- const detail = String(payload?.error?.message ?? "").trim();
- if (status === 401) return "DeepSeek API Key 无效或已失效";
- if (status === 402) return "DeepSeek 余额不足,请充值后继续";
- if (status === 429) return "DeepSeek 请求过于频繁,请稍后重试";
- if (status === 500 || status === 503) return "DeepSeek 服务暂时不可用,请稍后重试";
- return `DeepSeek 分析失败${detail ? `:${detail}` : `(HTTP ${status})`}`;
+function apiError(payload, status, provider = {}) {
+ const label = provider.label || "AI 服务";
+ const detail = redactSecret(String(payload?.error?.message ?? "").trim(), provider.apiKey);
+ if (status === 401) return `${label} API Key 无效或已失效`;
+ if (status === 402) return `${label} 余额不足,请充值后继续`;
+ if (status === 429) return `${label} 请求过于频繁,请稍后重试`;
+ if (status === 500 || status === 503) return `${label} 服务暂时不可用,请稍后重试`;
+ return `${label} 分析失败${detail ? `:${detail}` : `(HTTP ${status})`}`;
+}
+
+function redactSecret(value, secretValue) {
+ const secret = String(secretValue ?? "").trim();
+ return secret ? String(value ?? "").split(secret).join("[已隐藏 API Key]") : String(value ?? "");
}
diff --git a/docs/KNOWN_LIMITATIONS.md b/docs/KNOWN_LIMITATIONS.md
index 81f46dc..d40a3cb 100644
--- a/docs/KNOWN_LIMITATIONS.md
+++ b/docs/KNOWN_LIMITATIONS.md
@@ -5,3 +5,9 @@
PromptDirector 目前依赖 Chrome 的原生媒体解码能力。遇到浏览器无法解码、但系统播放器可以播放的 MPEG-4 文件时,案例仍会保留原文件和来源信息,并提供使用系统播放器打开的路径;当前版本不在扩展内置软件解码器。
后续如实施软件解码,至少需要同时满足以下验收条件:不改变原文件、不会把解码失败误报为导入成功、能中止长时间任务、限制内存与临时存储占用,并在发布包体积与浏览器安全策略检查通过后再启用。
+
+## GLM-5.3-Flash 本地视频直传边界
+
+案例库中的本地视频可以在用户确认后,以 Base64 发送给 GLM-5.3-Flash 做视频分析。智谱当前公开的 [GLM-5.3-Flash 模型页](https://docs.bigmodel.cn/cn/guide/models/vlm/glm-5.3-flash)和 [Chat Completions 接口页](https://docs.bigmodel.cn/api-reference/%E6%A8%A1%E5%9E%8B-api/%E5%AF%B9%E8%AF%9D%E8%A1%A5%E5%85%A8)没有声明直连视频的文件大小上限,因此 PromptDirector 不擅自设置 8 MB 等未公开限制。
+
+当前版本不会自动压缩、截断、抽帧或改用其他模型,因为这些降级会改变分析对象或产生额外费用。若服务端拒绝文件大小、格式或内容,请求会显示所选服务返回的错误且不会自动重试。只有公网视频文件直链可走 URL 分析;YouTube、Bilibili、抖音、X 或 Vimeo 的播放页面不等同于视频文件直链。
diff --git a/i18n.js b/i18n.js
index a59a286..fd1d087 100644
--- a/i18n.js
+++ b/i18n.js
@@ -1930,6 +1930,7 @@ const EN = Object.freeze({
"自定义兼容服务": "Custom compatible services",
"自定义兼容服务(文字)": "Custom compatible service (Text)",
"自定义兼容服务(图片与生成)": "Custom compatible service (Images and generation)",
+ "智谱 GLM": "Zhipu GLM",
"火山引擎": "Volcengine",
"Skill 提炼": "Skill extraction",
"创作规划": "Creative planning",
@@ -1952,6 +1953,9 @@ const EN = Object.freeze({
"模型目录已读取 · {count} 个模型 · 尚未执行模型调用验证{unavailable}": "Model catalog loaded · {count} models · model execution not yet verified{unavailable}",
" · {count} 个下架或当前不可用": " · {count} retired or currently unavailable",
"(已下架或当前不可用)": " (retired or currently unavailable)",
+ " · {count} 个来自官方能力说明,账号可用性待实测": " · {count} from official capability docs; account access needs a live test",
+ "(官方声明支持,账号可用性待实测)": " (officially documented; account access needs a live test)",
+ "官方声明该模型支持此能力,但账号目录未列出;以真实执行结果为准": "Official documentation supports this capability, but the account catalog does not list the model; rely on a live execution result.",
"(当前目录不可用)": " (not available in the current catalog)",
"尚无已连接且模型目录明确支持“{task}”的服务,请先连接服务并刷新模型": "No connected service has a catalog model explicitly supporting {task}. Connect a service and refresh its models first.",
"尚无已连接且可用于“{task}”的模型,请先连接服务并刷新模型目录": "No connected model can be assigned to {task}. Connect a service and refresh its model catalog first.",
diff --git a/library.js b/library.js
index d0e58e4..fbedd5e 100644
--- a/library.js
+++ b/library.js
@@ -63,6 +63,8 @@ import {
organizeDetailTagsWithDeepSeek
} from "./deepseek.js";
import { runScheduledAnalysisWithRetries } from "./analysis-scheduler.js";
+import { chatCompletionsVideoSourcePlan } from "./video-analysis.js";
+import { getAiModelCapability } from "./ai-model-capabilities.js";
import {
COMPOSER_METHOD_VERSION,
DEFAULT_TASK_METHODS,
@@ -5002,10 +5004,37 @@ async function startVideoAnalysis(entry, asset) {
showFeedback("请先允许所选 AI 服务接收本次明确提交的资料", true);
return;
}
- const sourceKind = asset.storageMode === "managed" ? "本地视频文件(将上传)" : asset.reference?.provider === "youtube" ? "公共 YouTube URL" : "当前社媒引用链接";
+ const sourceUrl = asset.reference?.url || asset.sourceUrl;
+ const modelMediaInput = getAiModelCapability(provider.id, model)?.mediaInput ?? {};
+ const mediaInput = { ...(provider.mediaInput ?? {}), ...modelMediaInput };
+ let sourceKind = asset.storageMode === "managed"
+ ? "本地视频文件(将上传)"
+ : asset.reference?.provider === "youtube" ? "公共 YouTube URL" : "当前视频链接";
+ if (provider.protocol === "chat_completions") {
+ try {
+ const sourcePlan = chatCompletionsVideoSourcePlan({
+ providerLabel: provider.label,
+ videoUrl: sourceUrl,
+ hasLocalVideo: asset.storageMode === "managed",
+ videoByteSize: asset.byteSize,
+ referenceProvider: asset.reference?.provider,
+ referencePlaybackMode: asset.reference?.playbackMode,
+ localVideo: mediaInput.localVideo,
+ videoMimeType: asset.mimeType,
+ preferPublicVideoUrl: mediaInput.preferPublicVideoUrl,
+ publicVideoUrl: mediaInput.publicVideoUrl
+ });
+ sourceKind = sourcePlan.sourceKind === "local-video"
+ ? "本地视频文件(将编码发送)"
+ : mediaInput.publicVideoUrl === "direct" ? "公网视频文件直链" : "公共视频 URL";
+ } catch (error) {
+ showFeedback(error.message || "所选模型不能分析这个视频来源", true);
+ return;
+ }
+ }
await showAppDialog({
title: "分析视频",
- description: `来源:${sourceKind} · 服务:${provider.label} · 模型:${model}。将发送视频或公开 YouTube URL 与本次问题;费用由服务商账户产生。`,
+ description: `来源:${sourceKind} · 服务:${provider.label} · 模型:${model}。将发送上述视频来源与本次问题;费用由服务商账户产生。`,
fields: [
{ id: "mode", label: "分析方式", type: "select", value: "creative-breakdown", options: [
{ value: "creative-breakdown", label: "创意拆解" },
@@ -7965,6 +7994,7 @@ function providerConnectionLabel(profile = {}) {
function providerCatalogLabel(profile = {}) {
const models = profile.discoveredModels ?? [];
const unavailable = models.filter((model) => model.status === "unavailable").length;
+ const unverified = models.filter((model) => model.status === "unverified").length;
if (profile.discovery?.error) {
return t("模型目录读取失败:{error};保留 {count} 个模型{unavailable}", {
error: translateUiMessage(profile.discovery.error),
@@ -7975,7 +8005,7 @@ function providerCatalogLabel(profile = {}) {
if (!profile.discovery?.discoveredAt) return t("模型目录未读取");
return t("模型目录已读取 · {count} 个模型 · 尚未执行模型调用验证{unavailable}", {
count: models.length,
- unavailable: unavailable ? t(" · {count} 个下架或当前不可用", { count: unavailable }) : ""
+ unavailable: `${unavailable ? t(" · {count} 个下架或当前不可用", { count: unavailable }) : ""}${unverified ? t(" · {count} 个来自官方能力说明,账号可用性待实测", { count: unverified }) : ""}`
});
}
@@ -7984,6 +8014,7 @@ function assignedModelState(profile, modelValue) {
if (!profile || !model) return "";
const discovered = (profile.discoveredModels ?? []).find((item) => item.id === model);
if (discovered?.status === "unavailable") return t("(已下架或当前不可用)");
+ if (discovered?.status === "unverified") return t("(官方声明支持,账号可用性待实测)");
if (profile.discovery?.discoveredAt && !discovered) return t("(当前目录不可用)");
return "";
}
@@ -8404,6 +8435,7 @@ function modelAssignmentHelp(profile, taskId, selectedValue) {
const selected = String(selectedValue ?? "").trim();
if (!selected) return "";
const model = availableAiModelsForTask(taskId, profile).find((item) => item.id === selected);
+ if (model?.status === "unverified") return t("官方声明该模型支持此能力,但账号目录未列出;以真实执行结果为准");
if (!model || ["declared", "protocol_inferred"].includes(model.assignmentEvidence)) return "";
return t("当前模型未声明这项能力;是否可用以真实执行结果为准");
}
diff --git a/manifest.json b/manifest.json
index 3b2833f..bc7b6fa 100644
--- a/manifest.json
+++ b/manifest.json
@@ -1,7 +1,7 @@
{
"manifest_version": 3,
"name": "__MSG_extensionName__",
- "version": "1.19.3",
+ "version": "1.19.4",
"key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq2m8W9ACXJTkJqPAd36sPrtzonmHhOYU34uS7c+D7DXX6ggWNMwB4xZapGFJ4ScBQJv71PeG9KfsEnhcXFZ9RoX/X3wY+57evL19hf6woNX1xxhKOe1rtx5QmzWCEYIr99TzW50k/N9U6tS9Ri+2DHKYA85ssXTC8B2yRQ5zUXhsAWZe/HDU18UFuvUKXYt3ZdIUi31c2vIlgFFBfsyKbIUMny9LlxL/smb4vJzU/+A3LwJDjjvOtS/lmQ2vHoQGjHJItO18bYI6h67fIEvmWk+HJKtO08Ca+5RLS8TYhI+K7l7S0RyDVNujyVWtvhAULot1H0BvteYL87fKu3SNywIDAQAB",
"homepage_url": "https://github.com/wchao6891/PromptDirector",
"default_locale": "zh_CN",
diff --git a/package-lock.json b/package-lock.json
index 52942d5..29a3b86 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "prompt-case-collector",
- "version": "1.19.3",
+ "version": "1.19.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "prompt-case-collector",
- "version": "1.19.3",
+ "version": "1.19.4",
"license": "Apache-2.0",
"dependencies": {
"@jonahschulte/rtf-toolkit": "0.5.0",
diff --git a/package.json b/package.json
index b572082..bddba9f 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "prompt-case-collector",
- "version": "1.19.3",
+ "version": "1.19.4",
"private": false,
"license": "Apache-2.0",
"repository": "https://github.com/wchao6891/PromptDirector",
@@ -22,6 +22,8 @@
"package": "npm run build:brand && npm run build:documents && npm run build:icons && node tools/package-extension.mjs",
"package:release": "npm run build:brand && npm run build:documents && npm run build:icons && node tools/package-extension.mjs --release",
"test:e2e": "python3 test/run_e2e.py",
+ "test:local-extension": "npm run package && node tools/local-extension-lab.mjs",
+ "test:live:zhipu": "node tools/zhipu-live-smoke.mjs",
"test:upgrade": "node tools/run-release-upgrade-e2e.mjs",
"test:smart-picker": "python3 test/smart_visual_picker_e2e.py",
"verify:source": "npm run check:public && npm run check && npm run check:brand && npm run check:documents && npm run check:pdfjs && npm run check:store && npm run check:compat && npm test",
diff --git a/store/CHROME_WEB_STORE_SUBMISSION.md b/store/CHROME_WEB_STORE_SUBMISSION.md
index f45218a..e17e53f 100644
--- a/store/CHROME_WEB_STORE_SUBMISSION.md
+++ b/store/CHROME_WEB_STORE_SUBMISSION.md
@@ -6,9 +6,9 @@ This file is the source-of-truth worksheet for the current PromptDirector manife
- Chrome Web Store item public key: stored in the source `manifest.json` for local identity verification only; `package:release` removes `key` from the upload manifest.
- Expected extension ID: `iahakaahijddcjjldidbclicedibgpjm`.
-- Current release candidate: `dist/PromptDirector-1.19.3.zip`.
-- Candidate size: `8,822,765` bytes.
-- Candidate SHA-256: `6c3e8767b9fe206950969c9e23b46a9588ecb586b927727b8f23d90a83639b25`.
+- Current release candidate: `dist/PromptDirector-1.19.4.zip`.
+- Candidate size: `8,839,874` bytes.
+- Candidate SHA-256: `5bfc4a2b230811c8d698df6208aff887afdb257f7036d9d4affe75de7fbbcf09`.
- Generate a candidate only with `npm run package:release`; the command verifies the source public key against the expected identity, then removes `key` from the Web Store upload manifest.
## Store listing
diff --git a/test/LOCAL_EXTENSION_LAB.md b/test/LOCAL_EXTENSION_LAB.md
new file mode 100644
index 0000000..bbd2d92
--- /dev/null
+++ b/test/LOCAL_EXTENSION_LAB.md
@@ -0,0 +1,38 @@
+# PromptDirector 本地插件实验室
+
+本模块把正式打包出的固定 ID 开发插件解压后,加载到 Chrome for Testing 的全新隔离资料中,再运行现有完整浏览器回归。它不会读取或修改用户日常 Chrome 资料,也不会使用真实 API Key。
+
+## 运行
+
+```bash
+npm run test:local-extension
+```
+
+Chrome for Testing 是 Chrome 官方为自动化提供的可复现浏览器。正式品牌版 Chrome 已移除命令行侧载扩展所需的开关,因此本模块不会把无法稳定复现的用户 Chrome 环境伪装成自动化能力。
+
+## 每次运行会证明什么
+
+- 测试对象来自 `npm run package` 生成的 `FIXED-ID-DEV` 插件包,不直接加载源码目录。
+- Manifest 公钥、扩展 ID、版本、Service Worker 和三个核心页面来自同一份运行文件。
+- 测试浏览器、用户代理和每次运行的结果写入收据,可追溯当时使用的自动化运行时。
+- 每条浏览器场景使用全新隔离资料,避免历史状态让结果随机通过或失败。
+- 实验室模式默认阻断所有未被场景显式 mock 的外部 HTTPS 请求;漏写 mock 会直接失败,不会误连真实服务或消耗真实额度。
+- 收据强制要求模型注册、文字与图片分析、创作台生成、视频生成、异步任务恢复,以及智谱 GLM 五项分析任务与创作台规划链路存在且通过;测试清单若误删关键场景,整次实验室运行会失败。
+- 现有本地模型模拟服务会继续覆盖成功、失败、异步轮询和恢复,不消耗真实模型额度。
+- 运行结束生成机器可读收据、完整日志和核心页面截图。
+
+证据默认保存在:
+
+```text
+dist/local-extension-lab/runs/<运行时间>/
+```
+
+`dist/local-extension-lab/latest.json` 指向最近一次结果。整个目录属于可重建产物,不进入 Git。
+
+## 它不能证明什么
+
+- 用户当前安装的 Chrome 插件包和本地实验室完全相同。
+- 智谱、OpenAI、Gemini 等真实账号当前可用、计费正确或模型质量达标。
+- GitHub Release、Chrome Web Store 上传、审核或线上分发状态。
+
+这些证据必须继续单独记录;本地实验室通过时只能称为“打包插件在 Chrome for Testing 自动验收通过”。
diff --git a/test/ai-provider-module.test.js b/test/ai-provider-module.test.js
index dc936aa..bc17982 100644
--- a/test/ai-provider-module.test.js
+++ b/test/ai-provider-module.test.js
@@ -40,6 +40,76 @@ test("DeepSeek identity-only discovery keeps models task-neutral and never guess
assert.equal(result.cache.etag, '"models-v1"');
});
+test("Zhipu discovery uses official GLM-4.6V capability facts while leaving unknown account models unverified", async () => {
+ const calls = [];
+ const module = createAiProviderModule({ fetchImpl: async (url, options) => {
+ calls.push({ url, options });
+ return jsonResponse({ data: [{ id: "glm-4.6v" }, { id: "future-glm-video-model" }] });
+ } });
+
+ const result = await module.discoverModels({
+ id: "zhipu",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ apiKey: "zhipu-secret",
+ protocol: "chat_completions"
+ });
+
+ assert.equal(calls[0].url, "https://open.bigmodel.cn/api/paas/v4/models");
+ assert.equal(calls[0].options.headers.Authorization, "Bearer zhipu-secret");
+ assert.equal(result.models[0].confidence, "declared");
+ assert.deepEqual(result.models[0].tasks, [
+ "textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"
+ ]);
+ assert.deepEqual(result.models[0].inputModalities, ["text", "image", "video", "file"]);
+ assert.deepEqual(result.models[0].supportedParameters, []);
+ assert.equal(result.models[1].confidence, "manual_unverified");
+ assert.deepEqual(result.models[1].tasks, []);
+ assert.equal(result.models[0].source, "provider_models+official_capabilities");
+
+ const official = getAiModelCapability("zhipu", "glm-4.6v");
+ assert.equal(official.source.url, "https://docs.bigmodel.cn/cn/guide/models/vlm/glm-4.6v");
+});
+
+test("Zhipu discovery classifies catalog-visible GLM-5.3-Flash as native multimodal", async () => {
+ const module = createAiProviderModule({ fetchImpl: async () => jsonResponse({ data: [{ id: "glm-5.3-flash" }] }) });
+ const result = await module.discoverModels({
+ id: "zhipu",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ apiKey: "zhipu-secret",
+ protocol: "chat_completions"
+ });
+ const model = result.models.find((item) => item.id === "glm-5.3-flash");
+ assert.equal(model.status, "available");
+ assert.equal(model.source, "provider_models+official_capabilities");
+ assert.equal(model.contextLength, 1_000_000);
+ assert.deepEqual(model.tasks, [
+ "textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"
+ ]);
+});
+
+test("Zhipu supplements a documented model omitted by its partial account catalog without claiming catalog access", async () => {
+ const module = createAiProviderModule({ fetchImpl: async () => jsonResponse({ data: [{ id: "glm-5" }] }) });
+ const profile = {
+ id: "zhipu",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ apiKey: "zhipu-secret",
+ protocol: "chat_completions"
+ };
+
+ const discovery = await module.discoverModels(profile);
+ const model = discovery.models.find((item) => item.id === "glm-4.6v");
+ assert.equal(model.status, "unverified");
+ assert.equal(model.source, "official_capabilities");
+ assert.deepEqual(model.tasks, [
+ "textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"
+ ]);
+
+ const verification = await module.verifyModelAccess(profile, "glm-4.6v");
+ assert.equal(verification.available, false);
+ assert.equal(verification.verification, "not_catalog_visible");
+ assert.deepEqual(verification.visibleModelIds, ["glm-5"]);
+});
+
test("identity discovery automatically classifies standard declared modalities without using model names", async () => {
const module = createAiProviderModule({ fetchImpl: async () => jsonResponse({ data: [
{
diff --git a/test/ai-provider-registry.test.js b/test/ai-provider-registry.test.js
index 446af4b..67f1d6a 100644
--- a/test/ai-provider-registry.test.js
+++ b/test/ai-provider-registry.test.js
@@ -54,7 +54,7 @@ function configuredAssignments() {
test("provider presets are the single model-free source for official and aggregator services", () => {
assert.deepEqual(AI_PROVIDER_PRESETS.map((preset) => preset.id), [
- "deepseek", "openai", "gemini", "xai", "kimi", "minimax", "volcengine", "openrouter",
+ "deepseek", "zhipu", "openai", "gemini", "xai", "kimi", "minimax", "volcengine", "openrouter",
"custom-text", "custom-media"
]);
assert.equal(AI_PROVIDER_PRESETS.every((preset) =>
@@ -66,6 +66,24 @@ test("provider presets are the single model-free source for official and aggrega
assert.equal(getAiProviderPreset("gemini").capabilities.includes("imageGeneration"), true);
assert.equal(getAiProviderPreset("deepseek").capabilities.includes("imageAnalysis"), true);
assert.equal(getAiProviderPreset("deepseek").structuredOutput, "json_object");
+ assert.deepEqual(getAiProviderPreset("zhipu"), {
+ id: "zhipu",
+ label: "智谱 GLM",
+ category: "official",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ protocol: "chat_completions",
+ discovery: { adapter: "identity" },
+ catalogCompleteness: "partial",
+ capabilities: ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"],
+ catalogRequiredTasks: [],
+ structuredOutput: "prompt_only",
+ mediaInput: {
+ imageBase64: "raw",
+ localVideo: "unsupported",
+ preferPublicVideoUrl: true,
+ publicVideoUrl: "direct"
+ }
+ });
assert.deepEqual(getAiProviderPreset("gemini").catalogRequiredTasks, ["imageGeneration"]);
assert.equal(getAiProviderPreset("openrouter").category, "aggregator");
assert.equal(getAiProviderPreset("missing"), null);
diff --git a/test/ai-runtime.test.js b/test/ai-runtime.test.js
index 839bc3c..a402c54 100644
--- a/test/ai-runtime.test.js
+++ b/test/ai-runtime.test.js
@@ -116,6 +116,81 @@ test("DeepSeek image analysis projects the selected account model through its co
assert.equal(projected.compatible.structuredOutput, "json_object");
});
+test("Zhipu runtime preserves its official raw-image and public-video input contract", () => {
+ const configuration = aiConfigurationFromStorage({
+ aiProviderRegistry: { providers: {
+ zhipu: {
+ apiKey: "zhipu-key",
+ consent: true,
+ models: { imageAnalysis: "glm-4.6v", videoAnalysis: "glm-4.6v" },
+ discoveredModels: [{
+ id: "glm-4.6v", status: "available", confidence: "declared",
+ source: "provider_models+official_capabilities",
+ tasks: ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"]
+ }]
+ }
+ } },
+ aiTaskAssignments: {
+ imageAnalysis: { providerId: "zhipu", model: "glm-4.6v" },
+ videoAnalysis: { providerId: "zhipu", model: "glm-4.6v" }
+ }
+ });
+
+ const vision = resolveVisionTaskSettings("imageAnalysis", configuration);
+ assert.equal(vision.compatible.imageBase64, "raw");
+ assert.equal(vision.compatible.structuredOutput, "prompt_only");
+ assert.equal(projectAiRuntime(configuration).visionSettings.providerProfiles.zhipu.models.imageAnalysis, "glm-4.6v");
+ assert.equal(projectAiRuntime(configuration).visionSettings.providerProfiles.zhipu.models.videoAnalysis, "glm-4.6v");
+ assert.deepEqual(resolveVideoAnalysisTask(configuration), {
+ taskId: "videoAnalysis",
+ providerId: "zhipu",
+ provider: "智谱 GLM",
+ providerLabel: "智谱 GLM",
+ model: "glm-4.6v",
+ protocol: "chat_completions",
+ apiKey: "zhipu-key",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ localVideo: "unsupported",
+ preferPublicVideoUrl: true,
+ publicVideoUrl: "direct"
+ });
+});
+
+test("GLM-5.3-Flash overrides older provider defaults with its model-specific multimodal contract", () => {
+ const configuration = aiConfigurationFromStorage({
+ aiProviderRegistry: { providers: {
+ zhipu: {
+ apiKey: "zhipu-key", consent: true,
+ models: {
+ textTags: "glm-5.3-flash", creativePlanning: "glm-5.3-flash",
+ imageAnalysis: "glm-5.3-flash", videoAnalysis: "glm-5.3-flash"
+ },
+ discoveredModels: [{
+ id: "glm-5.3-flash", status: "available", confidence: "declared",
+ source: "provider_models+official_capabilities",
+ tasks: ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"]
+ }]
+ }
+ } },
+ aiTaskAssignments: {
+ textTags: { providerId: "zhipu", model: "glm-5.3-flash" },
+ creativePlanning: { providerId: "zhipu", model: "glm-5.3-flash" },
+ imageAnalysis: { providerId: "zhipu", model: "glm-5.3-flash" },
+ videoAnalysis: { providerId: "zhipu", model: "glm-5.3-flash" }
+ }
+ });
+
+ const text = resolveTextTaskSettings("textTags", configuration);
+ assert.equal(text.compatible.structuredOutput, "json_object");
+ assert.equal(text.compatible.structuredOutputTokenBudget, 4096);
+ const vision = resolveVisionTaskSettings("imageAnalysis", configuration);
+ assert.equal(vision.compatible.structuredOutput, "json_object");
+ assert.equal(vision.compatible.imageBase64, "data_url");
+ const video = resolveVideoAnalysisTask(configuration);
+ assert.equal(video.preferPublicVideoUrl, true);
+ assert.equal(video.localVideo, "base64");
+});
+
test("changing the image-analysis assignment does not erase generation services", () => {
const configuration = aiConfigurationFromStorage({
aiProviderRegistry: { providers: {
@@ -160,7 +235,7 @@ test("new-install runtime previews remain unassigned instead of inventing provid
assert.equal(runtime.aiSettings.activeProvider, "compatible");
assert.equal(runtime.aiSettings.analysisModel, "");
- assert.deepEqual(runtime.aiSettings.compatible, { endpoint: "", model: "", apiKey: "" });
+ assert.deepEqual(runtime.aiSettings.compatible, { endpoint: "", model: "", apiKey: "", structuredOutput: "json_object" });
assert.equal(resolveTextTaskSettings("textTags", configuration, { requireConfigured: false }).analysisModel, "");
assert.throws(() => resolveTextTaskSettings("textTags", configuration), /尚未分配/);
assert.throws(() => resolveVideoAnalysisTask(configuration), /尚未分配/);
diff --git a/test/ai_provider_registry_e2e.py b/test/ai_provider_registry_e2e.py
index f28e78e..fa4a59c 100644
--- a/test/ai_provider_registry_e2e.py
+++ b/test/ai_provider_registry_e2e.py
@@ -289,7 +289,7 @@ def assert_new_install_defaults() -> None:
with extension_session("prompt-director-ai-registry-new-") as run:
library = open_ai_settings(run)
- expect(library.locator("#ai-provider-list .ai-provider-row")).to_have_count(10)
+ expect(library.locator("#ai-provider-list .ai-provider-row")).to_have_count(11)
expect(library.locator('#ai-provider-list [data-provider-category="official"]')).to_contain_text("官方服务")
expect(library.locator('#ai-provider-list [data-provider-category="aggregator"]')).to_contain_text("聚合平台")
expect(library.locator('#ai-provider-list [data-provider-category="custom"]')).to_contain_text("自定义兼容服务")
@@ -321,7 +321,7 @@ def assert_new_install_defaults() -> None:
assert execution_state["ok"] is True, execution_state
assert execution_state["aiSettings"]["analysisModel"] == "", execution_state
assert execution_state["aiSettings"]["compatible"] == {
- "endpoint": "", "model": "", "apiKey": ""
+ "endpoint": "", "model": "", "apiKey": "", "structuredOutput": "json_object"
}, execution_state
library.evaluate(
@@ -500,7 +500,7 @@ def main() -> None:
})
library = open_ai_settings(run)
- expect(library.locator("#ai-provider-list .ai-provider-row")).to_have_count(10)
+ expect(library.locator("#ai-provider-list .ai-provider-row")).to_have_count(11)
expect(library.locator("#ai-assignment-list .ai-assignment-row")).to_have_count(7)
expect(library.locator('[data-ai-routing-panel="tasks"]')).to_be_visible()
expect(library.locator('[data-ai-routing-panel="providers"]')).to_be_hidden()
@@ -524,22 +524,26 @@ def main() -> None:
expect(library.locator("#ai-settings-status")).to_contain_text("规则保存在本机")
assignments_before_rules = library.evaluate("() => chrome.storage.local.get('aiTaskAssignments').then(value => value.aiTaskAssignments)")
library.locator("#analysis-instructions-zh").fill("只保存统一 Registry 的文字分析规则")
- library.locator("#ai-settings-form").get_by_role("button", name="保存分析规则").click()
+ text_save = library.locator("#ai-settings-form").get_by_role("button", name="保存分析规则")
+ text_save.click()
library.wait_for_function(
"""expected => chrome.storage.local.get('aiPreferences').then(value =>
value.aiPreferences?.textInstructionsByLocale?.['zh-CN'] === expected)""",
arg="只保存统一 Registry 的文字分析规则",
)
+ expect(text_save).to_be_enabled()
library.locator('[data-analysis-kind="vision"]').click()
expect(library.locator("#vision-instructions-zh")).to_be_visible()
expect(library.locator("#vision-settings-status")).to_contain_text("规则保存在本机")
library.locator("#vision-instructions-zh").fill("只保存统一 Registry 的图片分析规则")
- library.locator("#vision-settings-form").get_by_role("button", name="保存分析规则").click()
+ vision_save = library.locator("#vision-settings-form").get_by_role("button", name="保存分析规则")
+ vision_save.click()
library.wait_for_function(
"""expected => chrome.storage.local.get('aiPreferences').then(value =>
value.aiPreferences?.visionInstructionsByLocale?.['zh-CN'] === expected)""",
arg="只保存统一 Registry 的图片分析规则",
)
+ expect(vision_save).to_be_enabled()
saved_preferences = library.evaluate("() => chrome.storage.local.get(['aiPreferences', 'aiTaskAssignments'])")
assert saved_preferences["aiPreferences"]["textInstructionsByLocale"]["zh-CN"] == "只保存统一 Registry 的文字分析规则"
assert saved_preferences["aiPreferences"]["visionInstructionsByLocale"]["zh-CN"] == "只保存统一 Registry 的图片分析规则"
@@ -565,7 +569,8 @@ def main() -> None:
dialog.locator("#promptdirector-app-dialog-providerEditor").select_option("openrouter")
expect(dialog.locator("#promptdirector-app-dialog-providerEditor")).to_have_value("openrouter")
expect(dialog.locator("#promptdirector-app-dialog-provider_openrouter_apiKey")).to_be_visible()
- expect(dialog.locator("#promptdirector-app-dialog-providerEditor option")).to_have_count(10)
+ expect(dialog.locator("#promptdirector-app-dialog-providerEditor option")).to_have_count(11)
+ expect(dialog.locator("#promptdirector-app-dialog-providerEditor option", has_text="官方服务 · 智谱 GLM")).to_have_count(1)
expect(dialog.locator("#promptdirector-app-dialog-providerEditor option", has_text="官方服务 · Kimi")).to_have_count(1)
expect(dialog.locator("#promptdirector-app-dialog-providerEditor option", has_text="聚合平台 · OpenRouter")).to_have_count(1)
expect(dialog.locator("#promptdirector-app-dialog-providerEditor option", has_text="自定义兼容服务 · 自定义兼容服务(图片与生成)")).to_have_count(1)
@@ -769,7 +774,7 @@ def main() -> None:
assert composer_parameters["aspectRatio"] == "16:9", composer_parameters
assert composer_parameters["imageSize"] == "2K", composer_parameters
assert gemini_interaction_calls == 0
- print({"providersShown": 10, "connectedProviders": 5, "taskAssignments": 7, "newInstallUnassigned": True, "nanoBananaCatalogAssigned": True, "nanoBananaComposerParameters": True, "paidCalls": 0, "unifiedPreferencesSaved": True, "credentialsExposed": False})
+ print({"providersShown": 11, "connectedProviders": 5, "taskAssignments": 7, "newInstallUnassigned": True, "nanoBananaCatalogAssigned": True, "nanoBananaComposerParameters": True, "paidCalls": 0, "unifiedPreferencesSaved": True, "credentialsExposed": False})
if __name__ == "__main__":
diff --git a/test/composer-page.test.js b/test/composer-page.test.js
index 27e3a44..e5c43c0 100644
--- a/test/composer-page.test.js
+++ b/test/composer-page.test.js
@@ -38,6 +38,7 @@ test("composer combines cases and Skills in one reference workspace while keepin
assert.match(composerHtml, />全程只用案例\/分析文字(不读图,最省 token)</);
assert.match(composerHtml, /id="composer-model-openai"/);
assert.match(composerHtml, /id="composer-model-compatible"/);
+ assert.match(composerHtml, /id="composer-model-dynamic"/);
assert.doesNotMatch(libraryHtml, /id="vision-image-protocol"|id="vision-image-endpoint"|id="vision-image-api-key"/);
assert.match(libraryJs, /provider_\$\{key\}_imageProtocol/);
assert.match(libraryJs, /能力声明,不是本轮输出值/);
@@ -51,6 +52,18 @@ test("composer combines cases and Skills in one reference workspace while keepin
assert.match(libraryJs, /navigateWithinPromptDirector\(url\)/);
});
+test("composer model menu renders the assigned planning provider dynamically instead of hardcoding vendor ids", async () => {
+ const composerJs = await readFile(new URL("../composer-page.js", import.meta.url), "utf8");
+ const renderer = composerJs.slice(
+ composerJs.indexOf("function renderGenerationModelChoices"),
+ composerJs.indexOf("function renderImageGenerationSettings")
+ );
+ assert.match(renderer, /creativePlanning/);
+ assert.match(renderer, /provider\.capabilities\?\.includes\(taskId\)/);
+ assert.match(renderer, /composerProfileForTaskAssignment/);
+ assert.doesNotMatch(renderer, /\["kimi",\s*"gemini"|providerId === "zhipu"/);
+});
+
test("composer reference selection is visual while Skill management stays on its own page", async () => {
const [composerJs, composerCss, skillsJs, i18nJs] = await Promise.all([
readFile(new URL("../composer-page.js", import.meta.url), "utf8"),
diff --git a/test/composer-service.test.js b/test/composer-service.test.js
index fd016ab..636054d 100644
--- a/test/composer-service.test.js
+++ b/test/composer-service.test.js
@@ -209,6 +209,170 @@ test("Kimi creative planning uses the assigned model without falling back to ano
assert.equal(planned.instruction, "保留核心冲突");
});
+test("Zhipu creative planning is selectable from the registry and calls only its assigned GLM model", async () => {
+ const requests = [];
+ const zhipuSettings = visualSettings({
+ providerProfiles: {
+ zhipu: {
+ id: "zhipu",
+ label: "智谱 GLM",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ protocol: "chat_completions",
+ structuredOutput: "prompt_only",
+ mediaInput: { imageBase64: "raw", localVideo: "unsupported", preferPublicVideoUrl: true, publicVideoUrl: "direct" },
+ apiKey: "zhipu-secret",
+ consent: true,
+ capabilities: ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"],
+ models: { creativePlanning: "glm-4.6v" },
+ discoveredModels: [{
+ id: "glm-4.6v",
+ tasks: ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"]
+ }]
+ }
+ }
+ });
+ const catalog = composerServiceCatalog({}, zhipuSettings.vision);
+ assert.equal(catalog.find((item) => item.serviceId === "zhipu" && item.model === "glm-4.6v")?.planning, true);
+ assert.equal(catalog.find((item) => item.serviceId === "zhipu" && item.model === "glm-4.6v")?.reasoning, false);
+ const session = createComposerSession({
+ aiProfile: { serviceId: "zhipu", model: "glm-4.6v" },
+ messages: [{ role: "user", type: "request", content: "整理为广告创作方向" }]
+ });
+
+ const planned = await planComposerTurnWithService({
+ session, userMessage: "", composerSettings: settings
+ }, zhipuSettings, {
+ fetchImpl: async (url, options) => {
+ requests.push({ url, headers: options.headers, body: JSON.parse(options.body) });
+ return response({
+ model: "glm-4.6v",
+ choices: [{ message: { content: JSON.stringify({ route: "compose", status: "ready", instruction: "强化前三秒冲突" }) } }]
+ });
+ }
+ });
+
+ assert.equal(requests.length, 1);
+ assert.equal(requests[0].url, "https://open.bigmodel.cn/api/paas/v4/chat/completions");
+ assert.equal(requests[0].headers.Authorization, "Bearer zhipu-secret");
+ assert.equal(requests[0].body.model, "glm-4.6v");
+ assert.equal(Object.hasOwn(requests[0].body, "response_format"), false);
+ assert.equal(planned.instruction, "强化前三秒冲突");
+
+ const imageSession = referenceSession("compatible");
+ imageSession.aiProfile = { serviceId: "zhipu", model: "glm-4.6v", thinking: true };
+ let imageRequest;
+ await executeComposerTurnWithService({
+ session: imageSession,
+ userMessage: "",
+ composerSettings: settings,
+ route: "compose",
+ instruction: "保持参考图构图"
+ }, zhipuSettings, preparedImages, {
+ stream: false,
+ fetchImpl: async (_url, options) => {
+ imageRequest = JSON.parse(options.body);
+ return response({ model: "glm-4.6v", choices: [{ message: { content: "保持三人关系与长焦层次" }, finish_reason: "stop" }] });
+ }
+ });
+ const imageParts = imageRequest.messages[1].content.filter((item) => item.type === "image_url");
+ assert.equal(imageParts.length, 3);
+ assert.deepEqual(imageParts[0], { type: "image_url", image_url: { url: "AAAA" } });
+ assert.equal(Object.hasOwn(imageRequest, "thinking"), false);
+ assert.equal(Object.hasOwn(imageRequest, "reasoning"), false);
+
+ const flashSettings = visualSettings({
+ providerProfiles: {
+ zhipu: {
+ ...zhipuSettings.vision.providerProfiles.zhipu,
+ models: { creativePlanning: "glm-5.3-flash" },
+ discoveredModels: [{
+ id: "glm-5.3-flash",
+ tasks: ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"]
+ }]
+ }
+ }
+ });
+ const flashSession = referenceSession("compatible");
+ flashSession.aiProfile = { serviceId: "zhipu", model: "glm-5.3-flash" };
+ let flashRequest;
+ await executeComposerTurnWithService({
+ session: flashSession,
+ userMessage: "",
+ composerSettings: settings,
+ route: "compose",
+ instruction: "分析参考图后给出构图方案"
+ }, flashSettings, preparedImages, {
+ stream: false,
+ fetchImpl: async (_url, options) => {
+ flashRequest = JSON.parse(options.body);
+ return response({ model: "glm-5.3-flash", choices: [{ message: { content: "突出主体轮廓" }, finish_reason: "stop" }] });
+ }
+ });
+ const flashImage = flashRequest.messages[1].content.find((item) => item.type === "image_url");
+ assert.deepEqual(flashImage, {
+ type: "image_url",
+ image_url: { url: "data:image/png;base64,AAAA", detail: "high" }
+ });
+});
+
+test("video generation keeps Zhipu planning separate from the assigned generation provider", async () => {
+ const settingsValue = visualSettings({
+ providerProfiles: {
+ zhipu: {
+ id: "zhipu", label: "智谱 GLM",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ protocol: "chat_completions", apiKey: "zhipu-secret", consent: true,
+ capabilities: ["creativePlanning"],
+ models: { creativePlanning: "glm-4.6v" },
+ discoveredModels: [{ id: "glm-4.6v", tasks: ["creativePlanning"] }]
+ },
+ minimax: {
+ id: "minimax", label: "MiniMax",
+ endpoint: "https://api.minimaxi.com/v1",
+ protocol: "minimax_videos", apiKey: "minimax-secret", consent: true,
+ capabilities: ["videoGeneration"],
+ models: { videoGeneration: "hailuo-account-model" },
+ discoveredModels: [{
+ id: "hailuo-account-model", tasks: ["videoGeneration"],
+ inputModalities: ["text"], outputModalities: ["video"],
+ supportedResolutions: ["1080P"]
+ }]
+ }
+ }
+ });
+ const session = createComposerSession({
+ targetType: "video",
+ outputMode: "create_video",
+ aiProfile: { serviceId: "zhipu", model: "glm-4.6v" },
+ generationAiProfile: { serviceId: "minimax", model: "hailuo-account-model" },
+ generationParameters: { size: "1080P" },
+ messages: [{ role: "user", type: "request", content: "生成品牌片头" }]
+ });
+ const calls = [];
+ const result = await executeComposerTurnWithService({
+ session, userMessage: "", composerSettings: settings, route: "compose", instruction: "标志从黑场聚合"
+ }, settingsValue, [], {
+ stream: false,
+ pollIntervalMs: 0,
+ fetchImpl: async (url, options = {}) => {
+ calls.push({ url, body: options.body });
+ if (url === "https://open.bigmodel.cn/api/paas/v4/chat/completions") {
+ return response({ model: "glm-4.6v", choices: [{ message: { content: "黑场中标志快速聚合,镜头推进。" } }] });
+ }
+ if (url.endsWith("/video_generation") && options.method === "POST") return response({ task_id: "minimax-task" });
+ if (url.includes("/query/video_generation")) return response({ status: "Success", file_id: "minimax-file" });
+ if (url.includes("/files/retrieve")) return response({ file: { download_url: "https://cdn.example/result.mp4" } });
+ if (url === "https://cdn.example/result.mp4") return mediaResponse("video");
+ throw new Error(`unexpected ${url}`);
+ }
+ });
+
+ assert.equal(calls[0].url, "https://open.bigmodel.cn/api/paas/v4/chat/completions");
+ assert.match(String(calls.find((call) => call.url.endsWith("/video_generation"))?.body), /黑场中标志快速聚合/);
+ assert.equal(result.serviceId, "minimax");
+ assert.equal(result.requestModel, "hailuo-account-model");
+});
+
test("an assigned provider without creative-planning capability fails instead of using DeepSeek", async () => {
const providerSettings = visualSettings({
providerProfiles: {
diff --git a/test/composer.test.js b/test/composer.test.js
index b2e6d3a..8dc9152 100644
--- a/test/composer.test.js
+++ b/test/composer.test.js
@@ -12,6 +12,7 @@ import {
appendPromptVersion,
clearComposerFailure,
composerInputUsage,
+ composerProfileForTaskAssignment,
createComposerSession,
createReferenceSnapshots,
imageReferenceModeAvailability,
@@ -60,13 +61,22 @@ test("composer agent keeps one editable method per task without a harness contra
assert.equal(resetComposerAgentInstruction(customAgent).agentInstruction.text, DEFAULT_AGENT_INSTRUCTION);
});
-test("composer AI profile keeps DeepSeek choices strict and accepts configured visual models", () => {
+test("composer AI profile preserves dynamically assigned planning models across providers", () => {
assert.deepEqual(normalizeComposerAiProfile(), DEFAULT_COMPOSER_AI_PROFILE);
assert.deepEqual(normalizeComposerAiProfile({ model: "deepseek-v4-pro", thinking: true }), { serviceId: "deepseek", model: "deepseek-v4-pro", thinking: true });
- assert.deepEqual(normalizeComposerAiProfile({ model: "custom", thinking: "yes" }), DEFAULT_COMPOSER_AI_PROFILE);
+ assert.deepEqual(normalizeComposerAiProfile({ model: "account-deepseek-planner", thinking: "yes" }), { serviceId: "deepseek", model: "account-deepseek-planner", thinking: false });
assert.deepEqual(normalizeComposerAiProfile({ serviceId: "openai", model: "gpt-5-mini", thinking: true }), { serviceId: "openai", model: "gpt-5-mini", thinking: true });
assert.deepEqual(normalizeComposerAiProfile({ serviceId: "compatible", model: "vision-pro", thinking: true }), { serviceId: "compatible", model: "vision-pro", thinking: true });
assert.deepEqual(normalizeComposerAiProfile({ serviceId: "kimi", model: "account-planning-model" }), { serviceId: "kimi", model: "account-planning-model", thinking: false });
+ assert.deepEqual(normalizeComposerAiProfile({ serviceId: "zhipu", model: "glm-4.6v" }), { serviceId: "zhipu", model: "glm-4.6v", thinking: false });
+ assert.deepEqual(composerProfileForTaskAssignment(
+ { providerId: "zhipu", model: "glm-4.6v" },
+ { serviceId: "deepseek", model: "account-deepseek-planner", thinking: true }
+ ), { serviceId: "zhipu", model: "glm-4.6v", thinking: true });
+ assert.deepEqual(composerProfileForTaskAssignment(
+ { providerId: "custom-text", model: "local-planner" },
+ DEFAULT_COMPOSER_AI_PROFILE
+ ), { serviceId: "custom-text", model: "local-planner", thinking: false });
assert.deepEqual(createComposerSession({ generationParameters: { size: "1536x1024", quality: "high", aspectRatio: "16:9", imageSize: "2K", secret: "drop" } }).generationParameters, {
size: "1536x1024",
quality: "high",
diff --git a/test/composer_generation_e2e.py b/test/composer_generation_e2e.py
index 691db5e..15394e8 100644
--- a/test/composer_generation_e2e.py
+++ b/test/composer_generation_e2e.py
@@ -469,7 +469,7 @@ def main() -> None:
composer.locator(".composer-case-option", has_text="即梦角色").locator("input").check()
composer.locator("#composer-reference-apply").click()
composer.locator("#composer-model-trigger").click()
- composer.locator("#composer-model-openai").click()
+ composer.locator("#composer-model-dynamic button", has_text="OpenAI").click()
expect(composer.locator("#composer-model-label")).to_have_text("OpenAI")
composer.locator("#composer-instruction").fill("参考1保持三人构图和中间女性聚焦,参考2只负责画面风格")
composer.locator("#composer-action").click()
@@ -489,6 +489,15 @@ def main() -> None:
type: 'UPDATE_AI_PROVIDER_CONFIGURATION',
registry: {
providers: {
+ 'custom-text': {
+ endpoint: `${origin}/v1/responses`,
+ protocol: 'responses',
+ apiKey: 'controlled-chat-test-key',
+ consent: true,
+ models: {
+ creativePlanning: 'local-vision-test'
+ }
+ },
'custom-media': {
endpoint: `${origin}/v1/responses`,
protocol: 'responses',
@@ -511,6 +520,7 @@ def main() -> None:
},
assignments: {
...aiTaskAssignments,
+ creativePlanning: {providerId: 'custom-text', model: 'local-vision-test'},
imageAnalysis: {providerId: 'custom-media', model: 'local-vision-test'},
imageGeneration: {providerId: 'custom-media', model: 'local-image-test'}
}
@@ -518,9 +528,11 @@ def main() -> None:
}""",
local_origin,
)
+ composer.reload()
+ expect(composer.locator("#composer-model-trigger")).to_be_visible()
composer.locator("#composer-model-trigger").click()
- composer.locator("#composer-model-compatible").click()
- expect(composer.locator("#composer-model-label")).to_have_text("兼容服务")
+ composer.locator("#composer-model-dynamic button", has_text="自定义兼容服务(文字)").click()
+ expect(composer.locator("#composer-model-label")).to_have_text("自定义兼容服务(文字)")
CreativeServiceHandler.requests.clear()
CreativeServiceHandler.release_image.clear()
composer.locator("#composer-options summary").click()
diff --git a/test/deepseek-analysis-input.test.js b/test/deepseek-analysis-input.test.js
index 9da200c..affb2e1 100644
--- a/test/deepseek-analysis-input.test.js
+++ b/test/deepseek-analysis-input.test.js
@@ -33,3 +33,91 @@ test("detail text analysis sends the canonical primary-image prompt to DeepSeek"
assert.match(userMessage.content, /当前图片提示词/);
assert.doesNotMatch(userMessage.content, /共享提示词/);
});
+
+test("prompt-only compatible text analysis relies on the JSON instruction without sending response_format", async () => {
+ let request;
+ const result = await analyzeTextDetailedWithDeepSeek({
+ id: "entry", title: "广告案例", text: "主体在前三秒出现"
+ }, createFixedFacetCatalog(), {
+ activeProvider: "compatible",
+ consent: true,
+ analysisModel: "glm-4.6v",
+ compatible: {
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ apiKey: "zhipu-secret",
+ model: "glm-4.6v",
+ structuredOutput: "prompt_only"
+ }
+ }, async (_url, options) => {
+ request = JSON.parse(options.body);
+ return {
+ ok: true,
+ json: async () => ({
+ model: "glm-4.6v",
+ choices: [{ finish_reason: "stop", message: { content: JSON.stringify({ tags: [{ g: "subject.character", t: "角色" }] }) } }]
+ })
+ };
+ });
+
+ assert.equal(result.model, "glm-4.6v");
+ assert.equal(Object.hasOwn(request, "response_format"), false);
+ assert.equal(Object.hasOwn(request, "thinking"), false);
+ assert.match(request.messages[0].content, /JSON/);
+});
+
+test("compatible forced-thinking models can supply a model-specific structured output budget", async () => {
+ let request;
+ await analyzeTextDetailedWithDeepSeek({
+ id: "entry", title: "广告案例", text: "主体在前三秒出现"
+ }, createFixedFacetCatalog(), {
+ activeProvider: "compatible",
+ consent: true,
+ analysisModel: "glm-5.3-flash",
+ compatible: {
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ apiKey: "zhipu-secret",
+ model: "glm-5.3-flash",
+ structuredOutput: "json_object",
+ structuredOutputTokenBudget: 4096
+ }
+ }, async (_url, options) => {
+ request = JSON.parse(options.body);
+ return {
+ ok: true,
+ json: async () => ({
+ model: "glm-5.3-flash",
+ choices: [{ finish_reason: "stop", message: { content: JSON.stringify({ tags: [{ g: "subject.character", t: "角色" }] }) } }]
+ })
+ };
+ });
+
+ assert.equal(request.max_tokens, 4096);
+ assert.equal(Object.hasOwn(request, "thinking"), false);
+ assert.deepEqual(request.response_format, { type: "json_object" });
+});
+
+test("compatible analysis errors name the selected provider and redact its API key", async () => {
+ await assert.rejects(() => analyzeTextDetailedWithDeepSeek({
+ id: "entry", title: "广告案例", text: "主体在前三秒出现"
+ }, createFixedFacetCatalog(), {
+ activeProvider: "compatible",
+ consent: true,
+ compatible: {
+ label: "智谱 GLM",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ apiKey: "never-print-this-zhipu-key",
+ model: "glm-5.3-flash",
+ structuredOutput: "json_object"
+ }
+ }, async () => ({
+ ok: false,
+ status: 400,
+ headers: new Headers(),
+ json: async () => ({ error: { message: "invalid never-print-this-zhipu-key" } })
+ })), (error) => {
+ assert.match(error.message, /智谱 GLM 分析失败/);
+ assert.doesNotMatch(error.message, /DeepSeek/);
+ assert.doesNotMatch(error.message, /never-print-this-zhipu-key/);
+ return true;
+ });
+});
diff --git a/test/e2e_support.py b/test/e2e_support.py
index 88ccb2f..762864f 100644
--- a/test/e2e_support.py
+++ b/test/e2e_support.py
@@ -133,6 +133,8 @@ def launch_context(
assert not bootstrap_errors, f"扩展首次初始化出现运行时错误:{bootstrap_errors}"
finally:
bootstrap.close()
+ if os.environ.get("PROMPTDIRECTOR_E2E_BLOCK_EXTERNAL_NETWORK") == "1":
+ context.route("https://**/*", lambda route: route.abort("blockedbyclient"))
return context
diff --git a/test/english_interaction_states_e2e.py b/test/english_interaction_states_e2e.py
index 4a2fcaf..10a1d2f 100644
--- a/test/english_interaction_states_e2e.py
+++ b/test/english_interaction_states_e2e.py
@@ -22,7 +22,7 @@ def assert_panel_contains_children(panel, state: str) -> None:
const outside = [...node.querySelectorAll('*')]
.filter(child => {
const style = getComputedStyle(child);
- if (style.display === 'none' || style.visibility === 'hidden') return false;
+ if (style.display === 'none' || style.visibility === 'hidden' || child.getClientRects().length === 0) return false;
const rect = child.getBoundingClientRect();
return rect.left < panelRect.left - 1 || rect.right > panelRect.right + 1
|| rect.top < panelRect.top - 1 || rect.bottom > panelRect.bottom + 1;
diff --git a/test/fixtures/zhipu-local-video-smoke.mp4 b/test/fixtures/zhipu-local-video-smoke.mp4
new file mode 100644
index 0000000..1dbdcdb
Binary files /dev/null and b/test/fixtures/zhipu-local-video-smoke.mp4 differ
diff --git a/test/local-extension-lab.test.js b/test/local-extension-lab.test.js
new file mode 100644
index 0000000..941a46e
--- /dev/null
+++ b/test/local-extension-lab.test.js
@@ -0,0 +1,83 @@
+import test from "node:test";
+import assert from "node:assert/strict";
+
+import {
+ buildLabReceipt,
+ hashRuntimeFiles,
+ parseE2eOutput,
+ REQUIRED_MODEL_CHAIN_SCRIPTS
+} from "../tools/local-extension-lab.mjs";
+
+test("local extension lab hashes packaged files independent of archive order", async () => {
+ const left = new Map([
+ ["manifest.json", new Blob(["manifest"])],
+ ["background.js", new Blob(["background"])]
+ ]);
+ const right = new Map([...left.entries()].reverse());
+ assert.equal(await hashRuntimeFiles(left), await hashRuntimeFiles(right));
+});
+
+test("local extension lab parses every browser scenario instead of stopping at the first failure", () => {
+ const summary = parseE2eOutput([
+ "[PASS] first_e2e.py (1.0s)",
+ "[FAIL] second_e2e.py (2.0s)",
+ "[PASS] third_e2e.py (3.0s)"
+ ].join("\n"));
+ assert.deepEqual({ total: summary.total, passed: summary.passed, failed: summary.failed }, {
+ total: 3, passed: 2, failed: 1
+ });
+});
+
+test("local extension receipt requires model analysis and Composer generation chains", () => {
+ const modelChainOutput = REQUIRED_MODEL_CHAIN_SCRIPTS
+ .map((script) => `[PASS] ${script} (1.0s)`)
+ .join("\n");
+ const receipt = buildLabReceipt({
+ startedAt: "2026-08-28T00:00:00.000Z",
+ finishedAt: "2026-08-28T00:01:00.000Z",
+ manifest: { version: "1.19.3" },
+ archive: { name: "PromptDirector-1.19.3-FIXED-ID-DEV.zip", sha256: "archive", fileCount: 10 },
+ runtimeHash: "runtime",
+ extensionId: "fixed-extension-id",
+ channel: "chromium",
+ preflight: { code: 0, output: "ok" },
+ preflightFacts: {
+ extensionId: "fixed-extension-id",
+ manifestVersion: "1.19.3",
+ serviceWorker: "chrome-extension://fixed-extension-id/background.js",
+ pages: [
+ "chrome-extension://fixed-extension-id/collector.html",
+ "chrome-extension://fixed-extension-id/library.html",
+ "chrome-extension://fixed-extension-id/composer.html"
+ ],
+ userAgent: "Chrome for Testing",
+ screenshots: ["collector.png", "library.png", "composer.png"]
+ },
+ e2e: { code: 0, output: modelChainOutput },
+ source: { commit: "abc", dirty: false }
+ });
+ assert.equal(receipt.status, "passed");
+ assert.equal(receipt.checks.modelChainPassed, true);
+ assert.equal(receipt.proofBoundary.localPackagedChromeForTesting, true);
+ assert.equal(receipt.proofBoundary.userInstalledChrome, false);
+ assert.equal(receipt.proofBoundary.liveProvider, false);
+ assert.equal(receipt.proofBoundary.chromeWebStore, false);
+});
+
+test("local extension receipt fails when a required model-chain scenario disappears", () => {
+ const receipt = buildLabReceipt({
+ startedAt: "2026-08-28T00:00:00.000Z",
+ finishedAt: "2026-08-28T00:01:00.000Z",
+ manifest: { version: "1.19.3" },
+ archive: { name: "extension.zip", sha256: "archive", fileCount: 10 },
+ runtimeHash: "runtime",
+ extensionId: "fixed-extension-id",
+ channel: "chromium",
+ preflight: { code: 0, output: "ok" },
+ preflightFacts: null,
+ e2e: { code: 0, output: "[PASS] text_and_image_analysis_e2e.py (1.0s)" },
+ source: { commit: "abc", dirty: false }
+ });
+ assert.equal(receipt.status, "failed");
+ assert.ok(receipt.checks.modelChain.missing.includes("composer_video_generation_e2e.py"));
+});
diff --git a/test/local_extension_lab_e2e.py b/test/local_extension_lab_e2e.py
new file mode 100644
index 0000000..defb1c6
--- /dev/null
+++ b/test/local_extension_lab_e2e.py
@@ -0,0 +1,62 @@
+from __future__ import annotations
+
+import json
+import os
+from pathlib import Path
+
+from e2e_support import EXTENSION_DIR, extension_session
+
+
+def main() -> None:
+ evidence_dir = Path(os.environ["PROMPTDIRECTOR_LAB_EVIDENCE_DIR"]).resolve()
+ expected_id = os.environ["PROMPTDIRECTOR_E2E_EXPECTED_EXTENSION_ID"].strip()
+ expected_version = os.environ["PROMPTDIRECTOR_E2E_EXPECTED_VERSION"].strip()
+ assert EXTENSION_DIR == Path(os.environ["PROMPTDIRECTOR_E2E_EXTENSION_DIR"]).resolve()
+ assert (EXTENSION_DIR / "manifest.json").exists(), EXTENSION_DIR
+
+ evidence_dir.mkdir(parents=True, exist_ok=True)
+ with extension_session("promptdirector-local-lab-", extension_dir=EXTENSION_DIR) as run:
+ collector = run.open_page("collector.html")
+ facts = collector.evaluate(
+ """() => ({
+ extensionId: chrome.runtime.id,
+ manifest: chrome.runtime.getManifest(),
+ location: location.href,
+ userAgent: navigator.userAgent
+ })"""
+ )
+ assert run.extension_id == expected_id, facts
+ assert facts["extensionId"] == expected_id, facts
+ assert facts["manifest"]["version"] == expected_version, facts
+ assert facts["manifest"].get("key"), "本地实验室必须加载固定身份插件包"
+ collector.screenshot(path=evidence_dir / "collector.png")
+
+ library = run.open_page("library.html")
+ library.locator("body").wait_for(state="visible")
+ library.screenshot(path=evidence_dir / "library.png")
+
+ composer = run.open_page("composer.html")
+ composer.locator("body").wait_for(state="visible")
+ composer.screenshot(path=evidence_dir / "composer.png")
+
+ report = {
+ "schema": "promptdirector-local-extension-preflight",
+ "version": 1,
+ "extensionDirectory": str(EXTENSION_DIR),
+ "extensionId": facts["extensionId"],
+ "manifestVersion": facts["manifest"]["version"],
+ "userAgent": facts["userAgent"],
+ "serviceWorker": f"chrome-extension://{run.extension_id}/background.js",
+ "pages": [collector.url, library.url, composer.url],
+ "screenshots": ["collector.png", "library.png", "composer.png"],
+ "pageErrors": run.page_errors,
+ }
+ (evidence_dir / "preflight.json").write_text(
+ json.dumps(report, ensure_ascii=False, indent=2) + "\n",
+ encoding="utf-8",
+ )
+ print(json.dumps(report, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/test/run_e2e.py b/test/run_e2e.py
index a9bb40f..0a47fe4 100644
--- a/test/run_e2e.py
+++ b/test/run_e2e.py
@@ -41,6 +41,7 @@ SCRIPTS = [
"page_capture_e2e.py",
"jimeng_capture_e2e.py",
"ai_provider_registry_e2e.py",
+ "zhipu_glm_analysis_e2e.py",
"creative_skills_e2e.py",
"skill_source_density_e2e.py",
"manager_regression_e2e.py",
diff --git a/test/sharing_and_data_safety_e2e.py b/test/sharing_and_data_safety_e2e.py
index 3c8206d..8762d4f 100644
--- a/test/sharing_and_data_safety_e2e.py
+++ b/test/sharing_and_data_safety_e2e.py
@@ -449,10 +449,14 @@ def main() -> None:
await chrome.storage.local.set({
entries: [...stored.entries, localOnly],
settings: {...stored.settings, libraryTitle: '本机精确恢复前'},
- aiProviderRegistry,
syncSettings: {sentinel: 'keep-local-sync'},
uiPreferences: {sentinel: 'keep-local-ui'}
});
+ const response = await chrome.runtime.sendMessage({
+ type: 'UPDATE_AI_PROVIDER_CONFIGURATION',
+ registry: aiProviderRegistry
+ });
+ if (!response?.ok) throw new Error(response?.message || 'AI 配置夹具写入失败');
window.__originalStorageEstimate = navigator.storage.estimate.bind(navigator.storage);
Object.defineProperty(navigator.storage, 'estimate', {
value: async () => ({quota: 1, usage: 1}),
@@ -460,6 +464,10 @@ def main() -> None:
});
}"""
)
+ library.wait_for_function(
+ """() => chrome.storage.local.get('aiProviderRegistry').then(stored =>
+ stored.aiProviderRegistry?.providers?.deepseek?.apiKey === 'fixture-private-key')"""
+ )
library.locator("#restore-folder-backup").click()
capacity_dialog = library.locator("#promptdirector-app-dialog")
expect(capacity_dialog).to_be_visible(timeout=15_000)
diff --git a/test/video-analysis.test.js b/test/video-analysis.test.js
index d6b20cc..9df3f83 100644
--- a/test/video-analysis.test.js
+++ b/test/video-analysis.test.js
@@ -4,6 +4,7 @@ import {
analyzeVideoWithChatCompletions,
analyzeVideoWithGemini,
analyzeVideoWithOpenRouter,
+ chatCompletionsVideoSourcePlan,
publicYouTubeUrl,
requireVideoAnalysisConfirmation,
videoAnalysisPrompt
@@ -107,6 +108,109 @@ test("Kimi uses its configured Chat Completions endpoint and exposes provider-sp
assert.equal(result.usage.totalTokens, 14);
});
+test("GLM-4.6V prefers its documented public URL and rejects undocumented local-only input", async () => {
+ let body;
+ const result = await analyzeVideoWithChatCompletions({
+ apiKey: "zhipu-key",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ providerLabel: "智谱 GLM",
+ model: "glm-4.6v",
+ mode: "content-summary",
+ preferPublicVideoUrl: true,
+ localVideo: "unsupported",
+ publicVideoUrl: "direct",
+ videoBlob: new Blob([new Uint8Array([1, 2, 3])], { type: "video/mp4" }),
+ youtubeUrl: "https://assets.example/video.mp4"
+ }, {
+ fetchImpl: async (_url, options) => {
+ body = JSON.parse(options.body);
+ return new Response(JSON.stringify({
+ choices: [{ message: { content: "00:01 开场" } }]
+ }), { status: 200, headers: { "content-type": "application/json" } });
+ }
+ });
+
+ assert.equal(body.messages[0].content[1].video_url.url, "https://assets.example/video.mp4");
+ assert.equal(result.sourceKind, "public-video-url");
+ await assert.rejects(() => analyzeVideoWithChatCompletions({
+ apiKey: "zhipu-key",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ providerLabel: "智谱 GLM",
+ model: "glm-4.6v",
+ mode: "content-summary",
+ preferPublicVideoUrl: true,
+ localVideo: "unsupported",
+ publicVideoUrl: "direct",
+ videoBlob: new Blob([new Uint8Array([1, 2, 3])], { type: "video/mp4" })
+ }), /只确认了公网 HTTPS 视频文件直链/);
+});
+
+test("documented direct-video routes reject social playback pages before a paid request", async () => {
+ let fetchCalls = 0;
+ await assert.rejects(() => analyzeVideoWithChatCompletions({
+ apiKey: "zhipu-key",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ providerLabel: "智谱 GLM",
+ model: "glm-5.3-flash",
+ mode: "content-summary",
+ videoUrl: "https://www.youtube.com/watch?v=abc",
+ referenceProvider: "youtube",
+ referencePlaybackMode: "embed",
+ preferPublicVideoUrl: true,
+ publicVideoUrl: "direct"
+ }, { fetchImpl: async () => { fetchCalls += 1; } }), /只确认了公网视频文件直链/);
+ assert.equal(fetchCalls, 0);
+});
+
+test("video source preflight exposes unsupported local input before confirmation", () => {
+ assert.throws(() => chatCompletionsVideoSourcePlan({
+ providerLabel: "智谱 GLM",
+ hasLocalVideo: true,
+ localVideo: "unsupported",
+ publicVideoUrl: "direct"
+ }), /不能直接发送本地视频/);
+});
+
+test("GLM-5.3-Flash sends an eligible local video as raw Base64", async () => {
+ let body;
+ const result = await analyzeVideoWithChatCompletions({
+ apiKey: "zhipu-key",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ providerLabel: "智谱 GLM",
+ model: "glm-5.3-flash",
+ mode: "content-summary",
+ localVideo: "base64",
+ videoBlob: new Blob([new Uint8Array([1, 2, 3])], { type: "video/mp4" })
+ }, { fetchImpl: async (_url, options) => {
+ body = JSON.parse(options.body);
+ return new Response(JSON.stringify({ choices: [{ message: { content: "00:01 开场" } }] }), {
+ status: 200, headers: { "content-type": "application/json" }
+ });
+ } });
+ assert.equal(body.messages[0].content[1].video_url.url, "AQID");
+ assert.equal(result.sourceKind, "local-video");
+});
+
+test("GLM local video preflight does not invent an unpublished 8 MB API limit", () => {
+ assert.deepEqual(chatCompletionsVideoSourcePlan({
+ providerLabel: "智谱 GLM",
+ hasLocalVideo: true,
+ videoByteSize: 9 * 1024 * 1024,
+ videoMimeType: "video/mp4",
+ localVideo: "base64",
+ publicVideoUrl: "direct"
+ }), { videoUrl: "", sourceKind: "local-video" });
+});
+
+test("local formats the Chat Completions adapter cannot encode are blocked before a paid request", () => {
+ assert.throws(() => chatCompletionsVideoSourcePlan({
+ providerLabel: "智谱 GLM",
+ hasLocalVideo: true,
+ videoMimeType: "video/unknown",
+ localVideo: "base64"
+ }), /不能编码发送/);
+});
+
test("generic Chat Completions video errors name the selected provider without exposing its key", async () => {
await assert.rejects(analyzeVideoWithChatCompletions({
apiKey: "never-print-this-key",
diff --git a/test/vision.test.js b/test/vision.test.js
index 780dc5f..6fd6cd8 100644
--- a/test/vision.test.js
+++ b/test/vision.test.js
@@ -436,6 +436,37 @@ test("compatible json_object vision requests keep image input and local reconstr
})), /至少一个有效检索标签/);
});
+test("Zhipu-compatible vision sends raw image Base64 without OpenAI-only detail fields", async () => {
+ let body;
+ await analyzeImageWithVision({
+ imageDataUrl: "data:image/png;base64,AAAA",
+ catalog: createDefaultFacetCatalog(),
+ locale: "zh-CN",
+ settings: {
+ activeProvider: "compatible", consent: true,
+ compatible: {
+ protocol: "chat_completions",
+ structuredOutput: "prompt_only",
+ imageBase64: "raw",
+ endpoint: "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+ apiKey: "zhipu-secret",
+ model: "glm-4.6v"
+ }
+ }
+ }, async (_url, options) => {
+ body = JSON.parse(options.body);
+ return {
+ ok: true,
+ json: async () => ({ choices: [{ message: { content: JSON.stringify(completeVisionResult()) } }] })
+ };
+ });
+
+ const image = body.messages[0].content.find((item) => item.type === "image_url");
+ assert.deepEqual(image, { type: "image_url", image_url: { url: "AAAA" } });
+ assert.equal(Object.hasOwn(body, "response_format"), false);
+ assert.match(body.messages[0].content.find((item) => item.type === "text").text, /reconstructionPrompt/);
+});
+
test("compatible empty vision results get exactly one structured-output correction request", async () => {
const input = {
imageDataUrl: "data:image/png;base64,AAAA",
diff --git a/test/zhipu_glm_analysis_e2e.py b/test/zhipu_glm_analysis_e2e.py
new file mode 100644
index 0000000..257d8f9
--- /dev/null
+++ b/test/zhipu_glm_analysis_e2e.py
@@ -0,0 +1,254 @@
+from __future__ import annotations
+
+import json
+
+from playwright.sync_api import expect
+
+from e2e_support import base_entry, extension_session
+
+
+PNG = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII="
+ANALYSIS_TASKS = ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"]
+
+
+def main() -> None:
+ image_entry = base_entry("zhipu-image-entry", "GLM 图片分析", "测试图片", "content:reference")
+ image_entry["mediaAssets"] = [{
+ "id": "zhipu-image",
+ "kind": "image",
+ "usage": "content",
+ "storageMode": "managed",
+ "mimeType": "image/png",
+ "width": 1,
+ "height": 1,
+ "byteSize": 68,
+ }]
+ image_entry["primaryMediaId"] = "zhipu-image"
+ video_entry = base_entry("zhipu-video-entry", "GLM 视频分析", "测试视频", "content:reference")
+ video_entry["mediaAssets"] = [{
+ "id": "zhipu-video",
+ "kind": "video",
+ "usage": "content",
+ "storageMode": "reference",
+ "mimeType": "video/mp4",
+ "reference": {
+ "url": "https://assets.example/zhipu-video.mp4",
+ "provider": "fixture",
+ "playbackMode": "external",
+ },
+ }]
+ video_entry["primaryMediaId"] = "zhipu-video"
+ local_video_entry = base_entry("zhipu-local-video-entry", "GLM 本地视频边界", "本地测试视频", "content:reference")
+ local_video_entry["mediaAssets"] = [{
+ "id": "zhipu-local-video",
+ "kind": "video",
+ "usage": "content",
+ "storageMode": "managed",
+ "mimeType": "video/mp4",
+ "byteSize": 12,
+ }]
+ local_video_entry["primaryMediaId"] = "zhipu-local-video"
+
+ with extension_session("prompt-director-zhipu-glm-", viewport={"width": 1280, "height": 900}) as run:
+ requests: list[dict] = []
+
+ def mock_zhipu(route) -> None:
+ if route.request.method == "GET" and route.request.url == "https://open.bigmodel.cn/api/paas/v4/models":
+ route.fulfill(
+ status=200,
+ content_type="application/json",
+ body='{"data":[{"id":"glm-5.3-flash"}]}',
+ )
+ return
+ payload = route.request.post_data_json
+ requests.append({
+ "url": route.request.url,
+ "authorization": route.request.headers.get("authorization", ""),
+ "payload": payload,
+ })
+ content = payload.get("messages", [{}])[-1].get("content", [])
+ if isinstance(content, list) and any(item.get("type") == "image_url" for item in content):
+ result = {
+ "reconstructionPrompt": "深色背景中的中央主体,保持居中构图和清晰轮廓。",
+ "tags": [{"g": "camera.composition", "t": "居中构图"}],
+ }
+ route.fulfill(
+ status=200,
+ content_type="application/json",
+ body=json.dumps({
+ "model": "glm-5.3-flash",
+ "choices": [{"message": {"content": json.dumps(result, ensure_ascii=False)}}],
+ "usage": {"prompt_tokens": 10, "completion_tokens": 8, "total_tokens": 18},
+ }, ensure_ascii=False),
+ )
+ return
+ if isinstance(content, list) and any(item.get("type") == "video_url" for item in content):
+ route.fulfill(
+ status=200,
+ content_type="application/json",
+ body=json.dumps({
+ "model": "glm-5.3-flash",
+ "choices": [{"message": {"content": "00:01 黑场出现主体,00:03 镜头推进。"}}],
+ "usage": {"prompt_tokens": 12, "completion_tokens": 6, "total_tokens": 18},
+ }, ensure_ascii=False),
+ )
+ return
+ if payload.get("stream") is True:
+ body = (
+ f'data: {json.dumps({"model": "glm-5.3-flash", "choices": [{"delta": {"content": "黑场中主体快速出现,镜头稳定推进。"}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 16, "completion_tokens": 7, "total_tokens": 23}}, ensure_ascii=False)}\n\n'
+ "data: [DONE]\n\n"
+ )
+ route.fulfill(status=200, content_type="text/event-stream", body=body)
+ return
+ route.fulfill(
+ status=200,
+ content_type="application/json",
+ body=json.dumps({
+ "model": "glm-5.3-flash",
+ "choices": [{"message": {"content": json.dumps({
+ "route": "compose",
+ "status": "ready",
+ "suggestedTitle": "GLM 创作链路",
+ "instruction": "把主体出现设计成前三秒钩子。",
+ "question": None,
+ "librarySearch": None,
+ }, ensure_ascii=False)}}],
+ "usage": {"prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30},
+ }, ensure_ascii=False),
+ )
+
+ run.context.route("https://open.bigmodel.cn/**", mock_zhipu)
+ setup = run.open_page("collector.html")
+ run.seed_storage(setup, {
+ "schemaVersion": 24,
+ "entries": [image_entry, video_entry, local_video_entry],
+ "uiPreferences": {"locale": "zh-CN", "theme": "light", "motion": "none"},
+ })
+ configured = setup.evaluate(
+ """async (tasks) => {
+ const connection = await chrome.runtime.sendMessage({
+ type: 'UPDATE_AI_PROVIDER_CONFIGURATION',
+ registry: {providers: {zhipu: {apiKey: 'zhipu-e2e-secret', consent: true}}}
+ });
+ if (!connection?.ok) throw new Error(connection?.message || '连接保存失败');
+ const catalog = await chrome.runtime.sendMessage({type: 'DISCOVER_AI_PROVIDER_MODELS', providerId: 'zhipu', force: true});
+ if (!catalog?.ok) throw new Error(catalog?.message || '模型目录读取失败');
+ const stored = await chrome.storage.local.get('aiTaskAssignments');
+ const assignments = {...stored.aiTaskAssignments};
+ for (const taskId of tasks) assignments[taskId] = {providerId: 'zhipu', model: 'glm-5.3-flash'};
+ const saved = await chrome.runtime.sendMessage({type: 'UPDATE_AI_PROVIDER_CONFIGURATION', assignments});
+ if (!saved?.ok) throw new Error(saved?.message || '任务分配失败');
+ const runtimes = {};
+ for (const taskId of tasks) {
+ runtimes[taskId] = await chrome.runtime.sendMessage({type: 'GET_AI_TASK_RUNTIME', taskId});
+ }
+ return {catalog, saved, runtimes};
+ }""",
+ ANALYSIS_TASKS,
+ )
+ assert configured["catalog"]["message"] == "智谱 GLM 已发现 2 个模型", configured
+ catalog_models = configured["catalog"]["aiProviderRegistry"]["providers"]["zhipu"]["discoveredModels"]
+ by_model = {model["id"]: model for model in catalog_models}
+ assert by_model["glm-5.3-flash"]["status"] == "available", by_model
+ assert by_model["glm-5.3-flash"]["source"] == "provider_models+official_capabilities", by_model
+ assert by_model["glm-4.6v"]["status"] == "unverified", by_model
+ for task_id in ANALYSIS_TASKS:
+ runtime = configured["runtimes"][task_id]
+ assert runtime["assignment"]["providerId"] == "zhipu", runtime
+ assert runtime["assignment"]["model"] == "glm-5.3-flash", runtime
+
+ setup.evaluate(
+ f"""async () => {{
+ const {{saveMediaBlob}} = await import(chrome.runtime.getURL('media-store.js'));
+ const bytes = Uint8Array.from(atob('{PNG}'), value => value.charCodeAt(0));
+ await saveMediaBlob('zhipu-image', new Blob([bytes], {{type: 'image/png'}}), {{checkCapacity: false}});
+ await saveMediaBlob('zhipu-local-video', new Blob([Uint8Array.from([0,0,0,12,102,116,121,112,105,115,111,109])], {{type: 'video/mp4'}}), {{checkCapacity: false}});
+ }}"""
+ )
+
+ image_result = setup.evaluate(
+ """() => chrome.runtime.sendMessage({
+ type: 'ANALYZE_ENTRY_IMAGE', entryId: 'zhipu-image-entry', visualId: 'zhipu-image', outputLocale: 'zh-CN'
+ })"""
+ )
+ assert image_result["ok"] is True, image_result
+ image_request = next(item["payload"] for item in requests if any(
+ part.get("type") == "image_url"
+ for part in item["payload"].get("messages", [{}])[-1].get("content", [])
+ ))
+ image_part = next(part for part in image_request["messages"][-1]["content"] if part["type"] == "image_url")
+ assert image_part == {
+ "type": "image_url",
+ "image_url": {"url": f"data:image/png;base64,{PNG}", "detail": "high"},
+ }, image_part
+ assert image_request["response_format"] == {"type": "json_object"}, image_request
+
+ video_result = setup.evaluate(
+ """() => chrome.runtime.sendMessage({
+ type: 'ANALYZE_ENTRY_VIDEO', entryId: 'zhipu-video-entry', assetId: 'zhipu-video',
+ mode: 'content-summary', customQuestion: '', singleConfirmation: true
+ })"""
+ )
+ assert video_result["ok"] is True, video_result
+ video_request = next(item["payload"] for item in requests if any(
+ part.get("type") == "video_url"
+ for part in item["payload"].get("messages", [{}])[-1].get("content", [])
+ ))
+ video_part = next(part for part in video_request["messages"][-1]["content"] if part["type"] == "video_url")
+ assert video_part["video_url"]["url"] == "https://assets.example/zhipu-video.mp4", video_part
+
+ requests_before_local_preflight = len(requests)
+ library = run.open_page("library.html", wait_until="networkidle")
+ library.evaluate("() => { chrome.permissions.request = async () => true; }")
+ library.locator('.case-card[data-entry-id="zhipu-local-video-entry"]').click()
+ library.get_by_role("button", name="分析视频", exact=True).click()
+ expect(library.locator("#promptdirector-app-dialog")).to_be_visible()
+ expect(library.locator("#promptdirector-app-dialog")).to_contain_text("本地视频文件(将编码发送)")
+ library.get_by_role("button", name="开始分析", exact=True).click()
+ expect(library.locator(".video-analysis-record")).to_contain_text("00:01 黑场出现主体")
+ assert len(requests) == requests_before_local_preflight + 1, requests
+ local_video_request = requests[-1]["payload"]
+ local_video_part = next(part for part in local_video_request["messages"][-1]["content"] if part["type"] == "video_url")
+ assert local_video_part["video_url"]["url"] == "AAAADGZ0eXBpc29t", local_video_part
+
+ composer = run.open_page("composer.html")
+ composer.locator("#composer-model-trigger").click()
+ dynamic_choice = composer.locator("#composer-model-dynamic button", has_text="glm-5.3-flash")
+ if dynamic_choice.count() == 0:
+ raise AssertionError({
+ "dynamicHtml": composer.locator("#composer-model-dynamic").inner_html(),
+ "menuHtml": composer.locator("#composer-model-menu").inner_html(),
+ "runtime": composer.evaluate("() => chrome.runtime.sendMessage({type: 'GET_COMPOSER_AI_RUNTIME'})"),
+ "pageErrors": run.page_errors,
+ })
+ expect(dynamic_choice).to_be_visible()
+ expect(dynamic_choice).to_contain_text("glm-5.3-flash")
+ dynamic_choice.click()
+ expect(composer.locator("#composer-model-label")).to_contain_text("智谱 GLM")
+ composer.locator("#composer-instruction").fill("为这个主体设计一个前三秒广告钩子")
+ composer.locator("#composer-action").click()
+ expect(composer.locator(".composer-message.prompt .composer-message-text")).to_contain_text("镜头稳定推进")
+ session_id = composer.evaluate("() => new URL(location.href).searchParams.get('session')")
+ composer.reload(wait_until="networkidle")
+ expect(composer.locator("#composer-model-label")).to_contain_text("智谱 GLM")
+ expect(composer.locator(".composer-message.prompt .composer-message-text")).to_contain_text("镜头稳定推进")
+ assert composer.evaluate("() => new URL(location.href).searchParams.get('session')") == session_id
+
+ assert all(item["authorization"] == "Bearer zhipu-e2e-secret" for item in requests), requests
+ stored = composer.evaluate("() => chrome.storage.local.get(['aiProviderRegistry', 'aiTaskAssignments', 'composerSessions'])")
+ assert stored["aiProviderRegistry"]["providers"]["zhipu"]["apiKey"] == "zhipu-e2e-secret"
+ assert all(stored["aiTaskAssignments"][task]["providerId"] == "zhipu" for task in ANALYSIS_TASKS)
+ assert stored["composerSessions"], stored
+ print({
+ "zhipuCatalog": True,
+ "analysisAssignments": len(ANALYSIS_TASKS),
+ "dataUrlImage": True,
+ "publicVideoUrl": True,
+ "composerGlmPlanning": True,
+ "refreshRecovery": True,
+ "paidRequests": 0,
+ })
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/local-extension-lab.mjs b/tools/local-extension-lab.mjs
new file mode 100644
index 0000000..84eeb53
--- /dev/null
+++ b/tools/local-extension-lab.mjs
@@ -0,0 +1,249 @@
+import { createHash } from "node:crypto";
+import { spawn } from "node:child_process";
+import { mkdir, readFile, rename, rm, writeFile } from "node:fs/promises";
+import { dirname, join, resolve, sep } from "node:path";
+import { fileURLToPath } from "node:url";
+
+import { extensionArchiveName, extensionIdFromPublicKey } from "./release-identity.mjs";
+import { readZipBlob } from "../zip.js";
+
+const projectRoot = fileURLToPath(new URL("../", import.meta.url));
+export const REQUIRED_MODEL_CHAIN_SCRIPTS = [
+ "text_and_image_analysis_e2e.py",
+ "composer_generation_e2e.py",
+ "composer_video_generation_e2e.py",
+ "creative_job_recovery_e2e.py",
+ "ai_provider_registry_e2e.py",
+ "zhipu_glm_analysis_e2e.py"
+];
+
+export function parseE2eOutput(value = "") {
+ const scripts = [];
+ for (const match of String(value).matchAll(/^\[(PASS|FAIL)\]\s+([^\s]+)\s+\(([^)]+)\)$/gmu)) {
+ scripts.push({ status: match[1] === "PASS" ? "passed" : "failed", script: match[2], duration: match[3] });
+ }
+ return {
+ total: scripts.length,
+ passed: scripts.filter((item) => item.status === "passed").length,
+ failed: scripts.filter((item) => item.status === "failed").length,
+ scripts
+ };
+}
+
+export async function hashRuntimeFiles(files) {
+ const digest = createHash("sha256");
+ for (const [name, blob] of [...files.entries()].sort(([left], [right]) => left.localeCompare(right))) {
+ digest.update(`${name.length}:${name}:`);
+ digest.update(new Uint8Array(await blob.arrayBuffer()));
+ }
+ return digest.digest("hex");
+}
+
+export function buildLabReceipt({
+ startedAt,
+ finishedAt,
+ manifest,
+ archive,
+ runtimeHash,
+ extensionId,
+ channel,
+ preflight,
+ preflightFacts,
+ e2e,
+ source
+}) {
+ const e2eSummary = parseE2eOutput(e2e.output);
+ const preflightFactsValid = preflight.code === 0
+ && preflightFacts?.extensionId === extensionId
+ && preflightFacts?.manifestVersion === String(manifest.version ?? "")
+ && String(preflightFacts?.serviceWorker ?? "") === `chrome-extension://${extensionId}/background.js`
+ && JSON.stringify(preflightFacts?.pages) === JSON.stringify([
+ `chrome-extension://${extensionId}/collector.html`,
+ `chrome-extension://${extensionId}/library.html`,
+ `chrome-extension://${extensionId}/composer.html`
+ ])
+ && JSON.stringify(preflightFacts?.screenshots) === JSON.stringify([
+ "collector.png", "library.png", "composer.png"
+ ]);
+ const scriptStatus = new Map(e2eSummary.scripts.map((item) => [item.script, item.status]));
+ const missingModelChainScripts = REQUIRED_MODEL_CHAIN_SCRIPTS.filter((script) => !scriptStatus.has(script));
+ const failedModelChainScripts = REQUIRED_MODEL_CHAIN_SCRIPTS.filter((script) => scriptStatus.get(script) === "failed");
+ const modelChainPassed = missingModelChainScripts.length === 0 && failedModelChainScripts.length === 0;
+ const passed = preflightFactsValid
+ && e2e.code === 0
+ && e2eSummary.failed === 0
+ && e2eSummary.total > 0
+ && modelChainPassed;
+ return {
+ schema: "promptdirector-local-extension-lab",
+ version: 1,
+ startedAt,
+ finishedAt,
+ status: passed ? "passed" : "failed",
+ source,
+ extension: {
+ version: String(manifest.version ?? ""),
+ id: extensionId,
+ archive: archive.name,
+ archiveSha256: archive.sha256,
+ runtimeSha256: runtimeHash,
+ runtimeFiles: archive.fileCount
+ },
+ browser: {
+ channel,
+ userAgent: String(preflightFacts?.userAgent ?? ""),
+ profile: "fresh-isolated-profile-per-scenario",
+ runtime: "chrome-for-testing-with-packaged-fixed-id-extension"
+ },
+ checks: {
+ packagedRuntimeLoaded: preflightFactsValid,
+ fullBrowserSuitePassed: e2e.code === 0 && e2eSummary.failed === 0 && e2eSummary.total > 0,
+ modelChainPassed,
+ modelChain: {
+ required: REQUIRED_MODEL_CHAIN_SCRIPTS,
+ missing: missingModelChainScripts,
+ failed: failedModelChainScripts
+ },
+ e2e: e2eSummary
+ },
+ proofBoundary: {
+ localPackagedChromeForTesting: passed,
+ userInstalledChrome: false,
+ liveProvider: false,
+ chromeWebStore: false
+ },
+ artifacts: {
+ preflightLog: "preflight.log",
+ e2eLog: "e2e.log",
+ preflightFacts: "preflight.json",
+ screenshots: Array.isArray(preflightFacts?.screenshots) ? preflightFacts.screenshots : []
+ }
+ };
+}
+
+async function main() {
+ const startedAt = new Date().toISOString();
+ const manifest = JSON.parse(await readFile(join(projectRoot, "manifest.json"), "utf8"));
+ const archiveName = extensionArchiveName(manifest);
+ const archivePath = join(projectRoot, "dist", archiveName);
+ const archiveBytes = await readFile(archivePath);
+ const archiveSha256 = createHash("sha256").update(archiveBytes).digest("hex");
+ const files = await readZipBlob(new Blob([archiveBytes]));
+ const runtimeHash = await hashRuntimeFiles(files);
+ const extensionId = extensionIdFromPublicKey(manifest);
+ const labRoot = join(projectRoot, "dist", "local-extension-lab");
+ const extensionDir = join(labRoot, "extension");
+ const stagingDir = join(labRoot, `extension.staging-${process.pid}`);
+ const runName = startedAt.replaceAll(":", "-").replaceAll(".", "-");
+ const runDir = join(labRoot, "runs", runName);
+ const channel = cleanChannel(process.env.PROMPTDIRECTOR_LAB_CHANNEL);
+
+ await extractRuntime(files, stagingDir, extensionDir);
+ await mkdir(runDir, { recursive: true });
+
+ const environment = {
+ ...process.env,
+ PROMPTDIRECTOR_E2E_EXTENSION_DIR: extensionDir,
+ PROMPTDIRECTOR_E2E_CHANNEL: channel,
+ PROMPTDIRECTOR_E2E_EXPECTED_EXTENSION_ID: extensionId,
+ PROMPTDIRECTOR_E2E_EXPECTED_VERSION: String(manifest.version ?? ""),
+ PROMPTDIRECTOR_LAB_EVIDENCE_DIR: runDir,
+ PROMPTDIRECTOR_E2E_BLOCK_EXTERNAL_NETWORK: "1"
+ };
+ const preflight = await runCaptured("python3", ["test/local_extension_lab_e2e.py"], environment);
+ await writeFile(join(runDir, "preflight.log"), preflight.output);
+ const preflightFacts = await readJsonIfExists(join(runDir, "preflight.json"));
+ const e2e = preflight.code === 0
+ ? await runCaptured("python3", ["test/run_e2e.py"], environment)
+ : { code: 1, output: "未运行:本地插件预检失败。\n" };
+ await writeFile(join(runDir, "e2e.log"), e2e.output);
+
+ const source = await sourceFacts();
+ const receipt = buildLabReceipt({
+ startedAt,
+ finishedAt: new Date().toISOString(),
+ manifest,
+ archive: { name: archiveName, sha256: archiveSha256, fileCount: files.size },
+ runtimeHash,
+ extensionId,
+ channel,
+ preflight,
+ preflightFacts,
+ e2e,
+ source
+ });
+ await writeFile(join(runDir, "receipt.json"), `${JSON.stringify(receipt, null, 2)}\n`);
+ await writeFile(join(labRoot, "latest.json"), `${JSON.stringify({ ...receipt, runDirectory: runDir }, null, 2)}\n`);
+ process.stdout.write(`\n本地插件实验室:${receipt.status === "passed" ? "通过" : "失败"}\n证据:${runDir}\n`);
+ if (receipt.status !== "passed") process.exitCode = 1;
+}
+
+async function extractRuntime(files, stagingDir, extensionDir) {
+ await rm(stagingDir, { recursive: true, force: true });
+ await mkdir(stagingDir, { recursive: true });
+ const stagingRoot = `${resolve(stagingDir)}${sep}`;
+ for (const [name, blob] of files) {
+ const target = resolve(stagingDir, name);
+ if (!target.startsWith(stagingRoot)) throw new Error(`插件包包含不安全路径:${name}`);
+ await mkdir(dirname(target), { recursive: true });
+ await writeFile(target, new Uint8Array(await blob.arrayBuffer()));
+ }
+ await rm(extensionDir, { recursive: true, force: true });
+ await rename(stagingDir, extensionDir);
+}
+
+function cleanChannel(value) {
+ const channel = String(value ?? "").trim();
+ if (!channel) return "chromium";
+ if (channel !== "chromium") {
+ throw new Error(`本地插件自动化只支持 Chrome for Testing(channel=chromium),收到:${channel}`);
+ }
+ return channel;
+}
+
+async function runCaptured(command, args, env) {
+ return await new Promise((resolveResult, reject) => {
+ const child = spawn(command, args, { cwd: projectRoot, env, stdio: ["ignore", "pipe", "pipe"] });
+ let output = "";
+ child.stdout.on("data", (chunk) => {
+ const text = chunk.toString();
+ output += text;
+ process.stdout.write(text);
+ });
+ child.stderr.on("data", (chunk) => {
+ const text = chunk.toString();
+ output += text;
+ process.stderr.write(text);
+ });
+ child.once("error", reject);
+ child.once("close", (code) => resolveResult({ code: Number.isInteger(code) ? code : 1, output }));
+ });
+}
+
+async function readJsonIfExists(path) {
+ try {
+ return JSON.parse(await readFile(path, "utf8"));
+ } catch {
+ return null;
+ }
+}
+
+async function sourceFacts() {
+ const commit = await runQuiet("git", ["rev-parse", "HEAD"]);
+ const status = await runQuiet("git", ["status", "--short"]);
+ return { commit: commit.trim(), dirty: Boolean(status.trim()) };
+}
+
+async function runQuiet(command, args) {
+ return await new Promise((resolveResult) => {
+ const child = spawn(command, args, { cwd: projectRoot, stdio: ["ignore", "pipe", "ignore"] });
+ let output = "";
+ child.stdout.on("data", (chunk) => { output += chunk.toString(); });
+ child.once("error", () => resolveResult(""));
+ child.once("close", () => resolveResult(output));
+ });
+}
+
+if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+ await main();
+}
diff --git a/tools/zhipu-live-smoke.mjs b/tools/zhipu-live-smoke.mjs
new file mode 100644
index 0000000..dc0dfac
--- /dev/null
+++ b/tools/zhipu-live-smoke.mjs
@@ -0,0 +1,236 @@
+import { createHash } from "node:crypto";
+import { readFile } from "node:fs/promises";
+import { fileURLToPath } from "node:url";
+
+import { createAiProviderModule } from "../ai-provider-module.js";
+import { getAiProviderPreset } from "../ai-provider-presets.js";
+import { analyzeTextDetailedWithDeepSeek } from "../deepseek.js";
+import { createDefaultFacetCatalog } from "../facets.js";
+import {
+ aiConfigurationFromStorage,
+ projectAiRuntime,
+ resolveTextTaskSettings,
+ resolveVideoAnalysisTask,
+ resolveVisionTaskSettings
+} from "../ai-runtime.js";
+import { createComposerSession, normalizeComposerSettings } from "../composer.js";
+import { planComposerTurnWithService } from "../composer-service.js";
+import { extractCreativeSkillDraft } from "../creative-skill-service.js";
+import { analyzeVideoWithChatCompletions } from "../video-analysis.js";
+import { analyzeImageWithVision, createVisionRequestBudget } from "../vision.js";
+
+const MODEL = String(process.env.PROMPTDIRECTOR_ZHIPU_MODEL ?? "glm-5.3-flash").trim();
+const ALL_TASKS = ["textTags", "skillExtraction", "creativePlanning", "imageAnalysis", "videoAnalysis"];
+const requestedTasks = String(process.env.PROMPTDIRECTOR_ZHIPU_TASKS ?? "").split(",").map((item) => item.trim()).filter(Boolean);
+const TASKS = requestedTasks.length ? ALL_TASKS.filter((taskId) => requestedTasks.includes(taskId)) : ALL_TASKS;
+if (!TASKS.length || requestedTasks.some((taskId) => !ALL_TASKS.includes(taskId))) {
+ throw new Error(`PROMPTDIRECTOR_ZHIPU_TASKS 只接受:${ALL_TASKS.join(",")}`);
+}
+const IMAGE_URL = "https://cdn.bigmodel.cn/static/logo/register.png";
+const VIDEO_URL = "https://cdn.bigmodel.cn/agent-demos/lark/113123.mov";
+const USE_SYNTHETIC_LOCAL_VIDEO = process.env.PROMPTDIRECTOR_ZHIPU_SYNTHETIC_LOCAL_VIDEO === "1";
+const SYNTHETIC_LOCAL_VIDEO = fileURLToPath(new URL("../test/fixtures/zhipu-local-video-smoke.mp4", import.meta.url));
+const apiKey = String(process.env.PROMPTDIRECTOR_ZHIPU_API_KEY ?? "").trim();
+
+if (!apiKey) throw new Error("缺少 PROMPTDIRECTOR_ZHIPU_API_KEY;真实凭据不会从文件或默认值读取");
+
+const preset = getAiProviderPreset("zhipu");
+const provider = { ...preset, apiKey, consent: true };
+const module = createAiProviderModule({ timeoutMs: 120_000 });
+const receipt = {
+ schema: "promptdirector-zhipu-live-smoke",
+ version: 1,
+ startedAt: new Date().toISOString(),
+ provider: "zhipu",
+ requestedModel: MODEL,
+ requests: {},
+ tasks: {},
+ secretPersisted: false
+};
+
+try {
+ const discovery = await module.discoverModels(provider);
+ const model = discovery.models.find((item) => item.id === MODEL && item.status !== "unavailable");
+ const providerCatalogModels = discovery.models.filter((item) => item.source.includes("provider_models"));
+ receipt.discovery = {
+ passed: Boolean(model),
+ candidateModelCount: discovery.models.length,
+ providerCatalogModelCount: providerCatalogModels.length,
+ providerCatalogModelIds: providerCatalogModels.map((item) => item.id),
+ requestedModelStatus: model?.status ?? "missing",
+ requestedModelSource: model?.source ?? "",
+ requestedModelCatalogVisible: providerCatalogModels.some((item) => item.id === MODEL),
+ tasks: model?.tasks ?? [],
+ inputModalities: model?.inputModalities ?? [],
+ outputModalities: model?.outputModalities ?? []
+ };
+ if (!model && process.env.PROMPTDIRECTOR_ZHIPU_PROBE_UNLISTED === "1") {
+ const response = await fetch(preset.endpoint, {
+ method: "POST",
+ headers: { "content-type": "application/json", authorization: `Bearer ${apiKey}` },
+ body: JSON.stringify({
+ model: MODEL,
+ max_tokens: 8,
+ messages: [{ role: "user", content: "只回复 OK" }]
+ })
+ });
+ receipt.requests.accessProbe = 1;
+ const payload = await response.json().catch(() => ({}));
+ receipt.accessProbe = {
+ passed: response.ok && Boolean(payload?.choices?.[0]?.message?.content),
+ status: response.status,
+ model: String(payload?.model ?? MODEL),
+ usage: safeUsage(payload?.usage)
+ };
+ if (!receipt.accessProbe.passed) {
+ throw new Error(`未列出模型权限探针失败:HTTP ${response.status} ${String(payload?.error?.message ?? payload?.msg ?? "").slice(0, 300)}`);
+ }
+ throw new Error(`${MODEL} 虽未出现在 /models 目录,但真实调用成功;需要先修正模型发现契约再继续五项验收`);
+ }
+ if (!model) throw new Error(`${MODEL} 不在当前账号模型目录中,尚未执行付费请求`);
+
+ const configuration = aiConfigurationFromStorage({
+ aiProviderRegistry: {
+ version: 5,
+ providers: {
+ zhipu: {
+ apiKey,
+ consent: true,
+ models: Object.fromEntries(TASKS.map((taskId) => [taskId, MODEL])),
+ discoveredModels: discovery.models,
+ discovery: { discoveredAt: discovery.discoveredAt, source: discovery.source }
+ }
+ }
+ },
+ aiTaskAssignments: Object.fromEntries(TASKS.map((taskId) => [taskId, {
+ providerId: "zhipu",
+ model: MODEL,
+ evidence: "declared"
+ }]))
+ });
+ const runtime = projectAiRuntime(configuration);
+ const catalog = createDefaultFacetCatalog();
+
+ if (TASKS.includes("textTags")) await runTask("textTags", async (fetchImpl) => {
+ const result = await analyzeTextDetailedWithDeepSeek({
+ id: "live-text",
+ title: "三秒广告钩子",
+ text: "黑场中角色突然亮相,镜头快速推进,随后展示产品核心卖点。"
+ }, catalog, resolveTextTaskSettings("textTags", configuration), fetchImpl, { timeoutMs: 120_000 });
+ return resultFact(result, { tagCount: result.tags.length });
+ });
+
+ if (TASKS.includes("skillExtraction")) await runTask("skillExtraction", async (fetchImpl) => {
+ const result = await extractCreativeSkillDraft({
+ goal: "提炼前三秒广告钩子的可复用方法",
+ locale: "zh-CN",
+ targetType: "video",
+ aiProfile: { serviceId: "zhipu", model: MODEL, thinking: false },
+ sources: [{
+ prompt: "黑场后主体突然出现,镜头快速推进并在三秒内呈现核心卖点。",
+ analysis: "冲突建立快,主体和利益点出现顺序明确。"
+ }]
+ }, {
+ ai: resolveTextTaskSettings("skillExtraction", configuration),
+ vision: runtime.visionSettings,
+ composer: normalizeComposerSettings()
+ }, { fetchImpl, stream: false });
+ return resultFact(result, { markdownCharacters: result.markdown.length });
+ });
+
+ if (TASKS.includes("creativePlanning")) await runTask("creativePlanning", async (fetchImpl) => {
+ const session = createComposerSession({
+ targetType: "video",
+ routeMode: "compose",
+ outputLanguage: "zh-CN",
+ aiProfile: { serviceId: "zhipu", model: MODEL, thinking: false },
+ messages: [{ role: "user", type: "request", content: "为新品设计一个前三秒视频广告钩子" }]
+ });
+ const result = await planComposerTurnWithService({
+ session,
+ userMessage: "",
+ composerSettings: normalizeComposerSettings()
+ }, { ai: runtime.aiSettings, vision: runtime.visionSettings }, { fetchImpl });
+ return resultFact(result, { route: result.route, hasInstruction: Boolean(result.instruction) });
+ });
+
+ if (TASKS.includes("imageAnalysis")) await runTask("imageAnalysis", async (fetchImpl) => {
+ const imageResponse = await fetch(IMAGE_URL, { redirect: "error" });
+ if (!imageResponse.ok) throw new Error(`官方测试图片读取失败:HTTP ${imageResponse.status}`);
+ const mimeType = String(imageResponse.headers.get("content-type") || "image/png").split(";")[0];
+ const base64 = Buffer.from(await imageResponse.arrayBuffer()).toString("base64");
+ const result = await analyzeImageWithVision({
+ imageDataUrl: `data:${mimeType};base64,${base64}`,
+ catalog,
+ locale: "zh-CN",
+ settings: resolveVisionTaskSettings("imageAnalysis", configuration),
+ requestBudget: createVisionRequestBudget(1)
+ }, fetchImpl);
+ return resultFact(result, {
+ reconstructionCharacters: result.reconstructionPrompt.length,
+ tagCount: result.tags.length
+ });
+ });
+
+ if (TASKS.includes("videoAnalysis")) await runTask("videoAnalysis", async (fetchImpl) => {
+ const settings = resolveVideoAnalysisTask(configuration);
+ const localVideoBytes = USE_SYNTHETIC_LOCAL_VIDEO ? await readFile(SYNTHETIC_LOCAL_VIDEO) : null;
+ const result = await analyzeVideoWithChatCompletions({
+ ...settings,
+ ...(localVideoBytes
+ ? { videoBlob: new Blob([localVideoBytes], { type: "video/mp4" }) }
+ : { videoUrl: VIDEO_URL }),
+ preferPublicVideoUrl: true,
+ mode: "custom",
+ customQuestion: "请只用一句中文概括视频内容。"
+ }, { fetchImpl });
+ return resultFact(result, { sourceKind: result.sourceKind, textCharacters: result.text.length });
+ });
+} catch (error) {
+ receipt.fatalError = safeError(error);
+}
+
+receipt.finishedAt = new Date().toISOString();
+receipt.passed = receipt.discovery?.passed === true
+ && TASKS.every((taskId) => receipt.tasks[taskId]?.passed === true)
+ && TASKS.every((taskId) => receipt.requests[taskId] === 1);
+process.stdout.write(`${JSON.stringify(receipt, null, 2)}\n`);
+if (!receipt.passed) process.exitCode = 1;
+
+async function runTask(taskId, operation) {
+ let calls = 0;
+ const fetchImpl = async (...args) => {
+ calls += 1;
+ receipt.requests[taskId] = calls;
+ if (calls > 1) throw new Error(`${taskId} 已阻止第二次真实模型请求`);
+ return fetch(...args);
+ };
+ try {
+ receipt.tasks[taskId] = { passed: true, ...(await operation(fetchImpl)) };
+ } catch (error) {
+ receipt.tasks[taskId] = { passed: false, error: safeError(error) };
+ } finally {
+ receipt.requests[taskId] = calls;
+ }
+}
+
+function resultFact(result, extra = {}) {
+ const output = String(result?.markdown ?? result?.instruction ?? result?.reconstructionPrompt ?? result?.text
+ ?? (Array.isArray(result?.tags) ? JSON.stringify(result.tags) : ""));
+ return {
+ model: String(result?.model ?? MODEL),
+ outputSha256: createHash("sha256").update(output).digest("hex"),
+ usage: safeUsage(result?.usage),
+ ...extra
+ };
+}
+
+function safeUsage(value = {}) {
+ return Object.fromEntries(Object.entries(value && typeof value === "object" ? value : {})
+ .filter(([, item]) => Number.isFinite(Number(item)))
+ .map(([key, item]) => [key, Number(item)]));
+}
+
+function safeError(error) {
+ return String(error?.message ?? error ?? "未知错误").replaceAll(apiKey, "[REDACTED]").slice(0, 800);
+}
diff --git a/video-analysis.js b/video-analysis.js
index ec63960..58c5a12 100644
--- a/video-analysis.js
+++ b/video-analysis.js
@@ -10,7 +10,8 @@ export const GEMINI_VIDEO_API_ROOT = "https://generativelanguage.googleapis.com/
export const GEMINI_FILE_POLL_INTERVAL_MS = 2_000;
export const GEMINI_FILE_POLL_LIMIT = 150;
const GEMINI_VIDEO_MIME_TYPES = new Set(["video/mp4", "video/mpeg", "video/quicktime", "video/avi", "video/x-flv", "video/mpg", "video/webm", "video/wmv", "video/3gpp"]);
-const CHAT_COMPLETIONS_VIDEO_MIME_TYPES = new Set(["video/mp4", "video/mpeg", "video/quicktime", "video/webm"]);
+const CHAT_COMPLETIONS_VIDEO_MIME_TYPES = new Set(["video/mp4", "video/mpeg", "video/quicktime", "video/webm", "video/x-m4v"]);
+const EMBEDDED_VIDEO_PROVIDERS = new Set(["youtube", "vimeo", "bilibili", "douyin", "x"]);
export function requireVideoAnalysisConfirmation(value) {
if (value !== true) throw new Error("请从视频分析确认框开始本次付费媒体分析");
@@ -88,19 +89,17 @@ export async function analyzeVideoWithChatCompletions(input = {}, dependencies =
if (!apiKey || !model) throw new Error(`${providerLabel} 视频分析尚未完成配置`);
const prompt = videoAnalysisPrompt(input.mode, input.customQuestion);
const onStage = typeof input.onStage === "function" ? input.onStage : () => {};
- let videoUrl;
- let sourceKind;
- if (input.videoBlob instanceof Blob) {
+ const sourcePlan = chatCompletionsVideoSourcePlan(input);
+ let videoUrl = sourcePlan.videoUrl;
+ const sourceKind = sourcePlan.sourceKind;
+ if (sourceKind === "local-video") {
if (!CHAT_COMPLETIONS_VIDEO_MIME_TYPES.has(input.videoBlob.type)) {
throw new Error(`${providerLabel} 当前不支持 ${input.videoBlob.type || "未知格式"} 视频;请先转换为 MP4、WebM、MOV 或 MPEG`);
}
- sourceKind = "local-video";
onStage("encoding");
- videoUrl = await blobDataUrl(input.videoBlob);
- } else {
- videoUrl = safeHttpsUrl(input.youtubeUrl);
- if (!videoUrl) throw new Error(`该视频链接不能安全发送给 ${providerLabel},请改用 HTTPS 地址或附加本地视频文件`);
- sourceKind = "public-video-url";
+ videoUrl = input.localVideo === "base64"
+ ? await blobBase64(input.videoBlob)
+ : await blobDataUrl(input.videoBlob);
}
onStage("analyzing");
const endpoint = chatCompletionsEndpoint(input.endpoint, providerLabel);
@@ -142,6 +141,39 @@ export async function analyzeVideoWithChatCompletions(input = {}, dependencies =
};
}
+export function chatCompletionsVideoSourcePlan(input = {}) {
+ const providerLabel = clean(input.providerLabel) || "兼容视频服务";
+ const publicVideoUrl = safeHttpsUrl(input.videoUrl || input.youtubeUrl);
+ const hasLocalVideo = input.videoBlob instanceof Blob || input.hasLocalVideo === true;
+ const embeddedPage = input.referencePlaybackMode === "embed"
+ || EMBEDDED_VIDEO_PROVIDERS.has(clean(input.referenceProvider).toLocaleLowerCase("en-US"));
+ const directPublicUrlRequired = input.publicVideoUrl === "direct";
+ const assertUsablePublicUrl = () => {
+ if (directPublicUrlRequired && embeddedPage) {
+ throw new Error(`${providerLabel} 当前模型只确认了公网视频文件直链,不能分析 YouTube、Bilibili、抖音、X 或 Vimeo 播放页`);
+ }
+ };
+ if (input.preferPublicVideoUrl === true && publicVideoUrl) {
+ assertUsablePublicUrl();
+ return { videoUrl: publicVideoUrl, sourceKind: "public-video-url" };
+ }
+ if (hasLocalVideo) {
+ if (input.localVideo === "unsupported") {
+ throw new Error(`${providerLabel} 当前模型只确认了公网 HTTPS 视频文件直链,不能直接发送本地视频`);
+ }
+ const videoMimeType = clean(input.videoMimeType || input.videoBlob?.type).toLocaleLowerCase("en-US");
+ if (videoMimeType && !CHAT_COMPLETIONS_VIDEO_MIME_TYPES.has(videoMimeType)) {
+ throw new Error(`${providerLabel} 当前不能编码发送 ${videoMimeType} 视频;请先转换为 MP4、WebM、MOV、M4V 或 MPEG`);
+ }
+ return { videoUrl: "", sourceKind: "local-video" };
+ }
+ if (!publicVideoUrl) {
+ throw new Error(`该视频链接不能安全发送给 ${providerLabel},请改用 HTTPS 地址或附加本地视频文件`);
+ }
+ assertUsablePublicUrl();
+ return { videoUrl: publicVideoUrl, sourceKind: "public-video-url" };
+}
+
async function uploadGeminiVideo(blob, apiKey, fetchImpl) {
const start = await fetchImpl(GEMINI_VIDEO_UPLOAD_ENDPOINT, {
method: "POST",
@@ -232,12 +264,16 @@ function safeHttpsUrl(value) {
}
async function blobDataUrl(blob) {
+ return `data:${blob.type};base64,${await blobBase64(blob)}`;
+}
+
+async function blobBase64(blob) {
const bytes = new Uint8Array(await blob.arrayBuffer());
let binary = "";
for (let offset = 0; offset < bytes.length; offset += 0x8000) {
binary += String.fromCharCode(...bytes.subarray(offset, offset + 0x8000));
}
- return `data:${blob.type};base64,${btoa(binary)}`;
+ return btoa(binary);
}
function chatCompletionsMessageText(value) {
diff --git a/vision.js b/vision.js
index 1727042..b3738eb 100644
--- a/vision.js
+++ b/vision.js
@@ -146,7 +146,10 @@ function normalizeCompatibleSettings(value = {}) {
}
return {
protocol: value?.protocol === "responses" ? "responses" : "chat_completions",
- structuredOutput: value?.structuredOutput === "json_object" ? "json_object" : "json_schema",
+ structuredOutput: value?.structuredOutput === "prompt_only"
+ ? "prompt_only"
+ : value?.structuredOutput === "json_object" ? "json_object" : "json_schema",
+ imageBase64: value?.imageBase64 === "raw" ? "raw" : "data_url",
endpoint,
model: String(value?.model ?? "").trim(),
apiKey,
@@ -224,6 +227,7 @@ export function mergeVisionSettings(currentValue = {}, incomingValue = {}) {
compatible: {
protocol: incoming.compatible?.protocol ?? current.compatible.protocol,
structuredOutput: incoming.compatible?.structuredOutput ?? current.compatible.structuredOutput,
+ imageBase64: incoming.compatible?.imageBase64 ?? current.compatible.imageBase64,
endpoint: nextEndpoint,
model: incoming.compatible?.model ?? current.compatible.model,
apiKey: compatibleApiKey,
@@ -259,6 +263,7 @@ export function publicVisionSettings(value = {}) {
compatible: {
protocol: settings.compatible.protocol,
structuredOutput: settings.compatible.structuredOutput,
+ imageBase64: settings.compatible.imageBase64,
model: settings.compatible.model,
configured: Boolean(settings.compatible.endpoint && settings.compatible.model &&
(settings.compatible.apiKey || isLoopbackEndpoint(settings.compatible.endpoint))),
@@ -527,20 +532,22 @@ async function requestCompatible(settings, imageDataUrl, instruction, fetchImpl,
}
const schema = options.schema ?? VISUAL_MODEL_RESPONSE_SCHEMA;
const schemaName = options.schemaName ?? "vision_reconstruction_tags_v1";
- const structuredInstruction = settings.compatible.structuredOutput === "json_object"
+ const structuredInstruction = ["json_object", "prompt_only"].includes(settings.compatible.structuredOutput)
? `${instruction}\nJSON property names are case-sensitive. Return exactly one object matching this JSON Schema: ${JSON.stringify(schema)}`
: instruction;
const body = {
model: settings.compatible.model,
max_tokens: positiveTokenBudget(options.maxOutputTokens) || settings.maxOutputTokens,
- response_format: settings.compatible.structuredOutput === "json_object"
- ? { type: "json_object" }
- : { type: "json_schema", json_schema: { name: schemaName, strict: true, schema } },
+ ...(settings.compatible.structuredOutput === "prompt_only" ? {} : {
+ response_format: settings.compatible.structuredOutput === "json_object"
+ ? { type: "json_object" }
+ : { type: "json_schema", json_schema: { name: schemaName, strict: true, schema } }
+ }),
messages: [{
role: "user",
content: [
{ type: "text", text: structuredInstruction },
- { type: "image_url", image_url: { url: imageDataUrl, detail: "high" } }
+ compatibleImagePart(imageDataUrl, settings.compatible.imageBase64)
]
}]
};
@@ -554,6 +561,14 @@ async function requestCompatible(settings, imageDataUrl, instruction, fetchImpl,
return { content, model: String(payload.model ?? settings.compatible.model), usage: normalizeCompatibleUsage(payload.usage) };
}
+function compatibleImagePart(imageDataUrl, encoding) {
+ if (encoding === "raw") {
+ const raw = String(imageDataUrl).replace(/^data:image\/[^;]+;base64,/i, "");
+ return { type: "image_url", image_url: { url: raw } };
+ }
+ return { type: "image_url", image_url: { url: imageDataUrl, detail: "high" } };
+}
+
function compatibleEmptyResultMessage(finishReason, fallback = "") {
const reason = String(finishReason ?? "").trim();
const prefix = String(fallback ?? "").trim();