A local plugin for image-heavy chats in Codex, Claude Code and other agents: earlier images stop going out with every message, and the model fetches the one it needs by its ID, also after compaction.
TypeScript ai-agentsclaude-codeclaude-code-plugincodexcodex-plugincontext-managementdeveloper-toolsimage-generationmcpmultimodal
feat(spike): P3 面板试验——对话内界面(MCP App)试验服务、窗口截屏脚本、插件服务的配置段落
chipfighter5916e4b58edc
diff --git a/AGENTS.md b/AGENTS.mdindex f1e3ce2..3c6bdca 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -16,6 +16,7 @@ Codex 上下文素材管理器(暂名):让用户逐轮决定,Codex 任- 仓库里的文档和代码不写用户本机的具体路径;通用位置(如 `~/.codex`、`%LOCALAPPDATA%`)可以写。- 先在 Windows 上开发和测试。技术栈用 TypeScript/Node。- 界面开发:Claude 自己截图检查效果;做到界面的关键节点时,截图给用户看,停下来等反馈再继续(用户 2026-09-24 确认)。+- 测试时调用模型一律用 `gpt-6-sol`,推理强度 `low`,包括脚本自测和桌面版测试,为了省 token(用户 2026-09-24 要求)。- 分工:Claude 维护 `docs/` 下的规格和方案,并负责开发和测试(阶段 0 的 T0–T2 由 Codex 完成;测试需要在 Codex 之外启动 app-server,所以从 T3 起改由 Claude 执行)。其他协作者不修改 spec.md 和 plan.md。## Git 规范diff --git a/docs/plan.md b/docs/plan.mdindex 69bc290..6d86073 100644--- a/docs/plan.md+++ b/docs/plan.md@@ -1,6 +1,6 @@# 技术方案-> 目标、约束和验收关卡以 [spec.md](spec.md) 为准。本文件说明怎么做、为什么这样做。第 1–8 节是阶段 0 的设计(已完成,结果见 [phase0-report.md](phase0-report.md));第 9、10 节是正在验证的代理路线:第 9 节是接入代理(P1),第 10 节是按勾选改写(P2)。+> 目标、约束和验收关卡以 [spec.md](spec.md) 为准。本文件说明怎么做、为什么这样做。第 1–8 节是阶段 0 的设计(已完成,结果见 [phase0-report.md](phase0-report.md));第 9、10 节是代理路线的接入(P1)和按勾选改写(P2),都已在桌面版验证通过;第 11 节是正在做的面板(P3)。## 1. 已知事实(2026-09-24)@@ -227,3 +227,37 @@ P1 通过后,再按用户的节奏一步步探索改写。- 取消勾选时,如果这个任务正在一轮对话中间,而且还在用 WebSocket,这一轮剩下的请求可能仍带着原图。从下一轮起一定生效。- 生图和改图:当前版本要靠 `.env` 里的 `NO_PROXY` 才能经过代理,见第 9 节“不等更新的办法”。- 桌面版会在用户消息里附一段 “Files mentioned by the user”,写着上传文件的完整路径。代理不改这段用户消息的文字,靠占位符里“不要自己用工具去读取”的要求约束模型;P2-6 里模型遵守了。++## 11. 面板(P3)++### 已查证的事实++依据桌面版自带的 `0.155.0-alpha.16.4` 的源码、上游 issue 和本机检查:++- **对话里的界面(MCP Apps)**:+ - 工具在自己的说明里用 `_meta.ui.resourceUri` 指向一个 `ui://` 页面;模型调用这个工具时,桌面版在对话里显示这个页面。+ - `_meta["openai/ui"].preferredModelDisplayMode` 可以选 `inline`(嵌在对话里)或 `fullscreen`(全屏)。+ - 页面的类型是 `text/html;profile=mcp-app`,在沙箱里运行,通过 postMessage 和桌面版握手,再经桌面版回调同一个服务的工具。+ - 界面信息会存进聊天记录,重启后不用重新查询插件也能显示出来。+- **已知问题(上游未修)**:+ - [#47512](https://github.com/openai/codex/issues/47512):Windows 上页面偶尔整块空白,都出现在页面地址变化之后。所以页面地址要保持固定。+ - [#45913](https://github.com/openai/codex/issues/45913):页面访问不了 `http://localhost`,即使列进了 CSP。所以面板只经桌面版回调插件服务拿数据,缩略图用 data URL。+- **插件格式**:`.codex-plugin/plugin.json` 可以带技能(skills)、MCP 服务(`mcpServers`)和钩子(hooks)。钩子事件有 `SessionStart`、`UserPromptSubmit` 等,可以执行命令。+- **接入配置**:`config.toml` 的 `[mcp_servers.<名字>]` 写启动命令;`default_tools_approval_mode = "approve"` 让它的工具不用逐次确认。+- **桌面版的程序**:OpenAI.Codex 安装目录下的 `ChatGPT.exe`,窗口标题是“ChatGPT”。截屏脚本 `spike/scripts/capture-codex.ps1` 按安装目录找这个窗口,只截它。++### 设计(待 P3-1 实测确认)++- **面板**:由插件的 MCP 服务提供的一个 MCP App,页面地址固定。打开方式是用户让模型调用“打开面板”工具,这会花一轮很小的请求;能不能不经过模型、从侧栏“插件”直接打开,P3-1 里一并查看。+- **数据**:面板经桌面版回调服务的工具,拿到当前任务的图片列表:编号、名称、来源、轮次、尺寸、大小、内容相同、勾选状态和缩略图。勾选和取消写进勾选状态文件,代理照旧读取。+- **缩略图**:服务端从本地记录解码 PNG、缩成小图,以 data URL 放进只给面板看的 `_meta`,不进入模型的上下文。要确认 Codex 不把 `_meta` 发给模型。+- **代理随 Codex 启停**:插件的 `SessionStart` 和 `UserPromptSubmit` 钩子检查代理,没在运行就在后台拉起;代理发现 Codex 进程都退出后,自己退出。+- **交付**:本地插件,包含说明用法的技能、MCP 服务和钩子;需要的配置由工具写成带标记的段落。++### 验证顺序++1. P3-1 试验面板:在 `config.toml` 里临时接入试验服务;用户重启桌面版,在测试任务里让模型打开面板。截图并查看日志,确认页面能否显示、能否握手、能否回调服务、data URL 图片能否显示、`localhost` 是否被拦截,以及服务是每个任务一个还是共用一个。+2. P3-2 面板数据接口:服务端工具和单元测试。+3. P3-3 面板界面:做到关键节点,截图给用户看,等反馈。+4. P3-4 代理随 Codex 启停。+5. P3-5 打包成本地插件,在桌面版上端到端验证。diff --git a/docs/spec.md b/docs/spec.mdindex 632ff3e..af63456 100644--- a/docs/spec.md+++ b/docs/spec.md@@ -68,7 +68,7 @@- 原始对话记录和素材文件始终只读,不修改、不移动、不删除。不直接写入 `~/.codex` 下的任何文件,只有三个例外:- 官方 app-server 在创建和运行任务时自己产生的正常写入;- - 为接入本地代理,工具可以修改 `config.toml` 里由它自己管理、带标记的设置(用户 2026-09-24 授权)。改之前先备份,能一键恢复,其他设置一律不动;+ - 为接入本地代理和面板服务,工具可以修改 `config.toml` 里由它自己管理、带标记的设置(用户 2026-09-24 授权)。改之前先备份,能一键恢复,其他设置一律不动;- 为让生图和改图也能连到本地代理,工具可以在 `~/.codex/.env` 里写一段带标记的 `NO_PROXY`,只含本机地址,并保留用户自己的条目(用户 2026-09-24 同意)。`.env` 可能含密钥,工具不复制、不输出它,只增删自己的那一段;文件里只剩这一段时,恢复时连文件一起删掉。- 真实数据不进仓库:从真实会话和用户素材里提取的图片、记录片段、日志和备份,只能放在本机的 `local/` 目录(已被 git 忽略),任何时候都不提交。仓库里的样例数据只能是合成的或脱敏的。- 仓库里的文档和代码不写用户本机的具体路径(例如个人项目文件夹)。`~/.codex`、`%LOCALAPPDATA%` 这类通用位置可以写。本项目以后会开源。@@ -76,7 +76,7 @@- 代理只替换用户取消的图片,连同 Codex 附在它前后的路径标签;请求里的其他内容原样转发。- 选择权永远在用户:哪些素材进入上下文,只由用户在面板里勾选决定。工具和模型都不能自动勾选或取消;模型只能按约定索要(例如回复“需要 IMG-004”),给不给由用户决定。- 先在 Windows 上开发和测试。技术栈:TypeScript/Node。-- 交付形态:Codex 插件(skill 加本地 MCP 服务);面板位置见第 1 节“界面”。+- 交付形态:Codex 插件(技能、本地 MCP 服务和钩子);面板位置见第 1 节“界面”。## 4. 验收标准@@ -105,16 +105,16 @@### 首版-> 以下按路线 A 写成,代理路线确定后重写。+> 按代理路线写成(2026-09-24 改写,原来按路线 A 写的版本已作废)。-1. 选定一个任务后,按时间顺序列出全部图片出现记录,无法识别的项单独列出。-2. 可以勾选、取消、重新勾选,并预览本轮将发送什么(实际素材、重复引用、占位符、预计字节数)。-3. 一键在新任务中继续:未勾选的图片不进入请求,选中的图可以继续修改。-4. 取消过的素材在之后任何一轮都能重新勾选并被发送。-5. 操作前后,原始记录和素材文件的哈希不变。-6. 显示操作前后的请求大小对比。+1. 装好后不用手动配置:工具自动接入 Codex,代理跟着 Codex 一起启动、一起退出;关闭工具时一键恢复,只删掉自己写入的内容。+2. 在 Codex 里打开面板,按时间顺序列出当前任务的全部图片出现记录,包括缩略图、编号、名称、来源、轮次、尺寸、大小、内容相同的关系和勾选状态。无法识别或不能取消的项要单独标出。+3. 勾选、取消、重新勾选都从下一次请求起生效;对话始终在同一个任务里,连续可见。+4. 面板显示本次会发送什么:照常发送的图、重复占位符和普通占位符,以及取消前后的请求大小对比。+5. 模型回复“需要 IMG-xxx”时,用户能在面板里找到那一条并勾选。+6. 没有取消任何图的任务,请求原样转发;改写不了时原样发送,并在面板上提示。+7. 原始记录和素材文件只读,操作前后哈希不变。## 5. 待定问题- 路线:路线 A(用 `thread/inject_items` 在新任务里重建)技术上可行,但用户不接受“重建后在界面上看不到原对话”。代理路线(任务不换,每次请求发出前由本地代理按勾选改写)已在桌面版验证通过:P1 只转发、P2 按勾选改写,生图和改图也包括在内,见 plan.md 第 9、10 节和 p2-report.md。用户 2026-09-24 决定接下来做面板。-- 第 4 节的首版验收标准是按路线 A 写的,开始做面板之前要按代理路线重写。\ No newline at end of filediff --git a/docs/tasks.md b/docs/tasks.mdindex f4542a6..344d218 100644--- a/docs/tasks.md+++ b/docs/tasks.md@@ -2,7 +2,22 @@> 方法见 [plan.md](plan.md)。按顺序做,完成一项勾一项,并写明证据放在哪里。-## 当前:代理验证 P2(按勾选改写)+## 当前:面板 P3++设计见 plan.md 第 11 节,验收标准见 spec.md 第 4 节“首版”。测试时调用模型一律用 `gpt-6-sol`,推理强度 `low`。++- [ ] **P3-1 试验面板**【需用户】(`spike/src/panel-probe.ts`、`panel-probe.html`;截屏 `spike/scripts/capture-codex.ps1`;配置 `codexconfig.ts mcp-enable/mcp-disable`)+ - 在 `config.toml` 末尾临时接入试验服务(带标记,能单独恢复)。用户重启桌面版,新建测试任务,让模型打开面板。+ - 截图并查看 `local/p3/panel-probe.jsonl`,确认:页面能否显示、能否握手、能否回调服务、data URL 图片能否显示、`localhost` 是否被拦截、服务是每个任务一个还是共用一个、能否不经模型从侧栏“插件”打开。+ - 结束后恢复配置。+- [ ] **P3-2 面板数据接口**:列出当前任务的图片、缩略图、勾选和取消;配单元测试。+- [ ] **P3-3 面板界面**【需用户】:做到关键节点,截图给用户看,等反馈。+- [ ] **P3-4 代理随 Codex 启停**。+- [ ] **P3-5 打包成本地插件,桌面版端到端验证**【需用户】。++---++## 已完成:代理验证 P2(按勾选改写)设计见 plan.md 第 10 节,验收标准见 spec.md 第 4 节“代理路线 P2”。diff --git a/spike/scripts/capture-codex.ps1 b/spike/scripts/capture-codex.ps1new file mode 100644index 0000000..3959e34--- /dev/null+++ b/spike/scripts/capture-codex.ps1@@ -0,0 +1,40 @@+# Purpose: P3 — capture only the Codex Desktop window to a PNG for UI checks, even when other windows cover it+# (PrintWindow with PW_RENDERFULLCONTENT). The desktop app's process is ChatGPT.exe inside the OpenAI.Codex package.+# Input: -Out <path.png>. Output: the PNG file and one line with its size.+param([Parameter(Mandatory = $true)] [string] $Out)++Add-Type -AssemblyName System.Drawing+Add-Type @"+using System;+using System.Runtime.InteropServices;+public static class CamWindow {+ [StructLayout(LayoutKind.Sequential)] public struct RECT { public int Left, Top, Right, Bottom; }+ [DllImport("user32.dll")] public static extern bool SetProcessDPIAware();+ [DllImport("user32.dll")] public static extern bool GetWindowRect(IntPtr hWnd, out RECT rect);+ [DllImport("user32.dll")] public static extern bool PrintWindow(IntPtr hWnd, IntPtr hdc, uint flags);+ [DllImport("user32.dll")] public static extern bool IsIconic(IntPtr hWnd);+}+"@+# Physical pixels, so the window is not cropped on scaled displays.+[void][CamWindow]::SetProcessDPIAware()++$process = Get-Process | Where-Object { $_.MainWindowHandle -ne 0 -and $_.Path -like '*\OpenAI.Codex_*' } | Select-Object -First 1+if (-not $process) { throw 'Codex Desktop window not found' }+$handle = $process.MainWindowHandle+if ([CamWindow]::IsIconic($handle)) { throw 'the Codex window is minimized' }++$rect = New-Object CamWindow+RECT+[void][CamWindow]::GetWindowRect($handle, [ref]$rect)+$width = $rect.Right - $rect.Left+$height = $rect.Bottom - $rect.Top+$bitmap = New-Object System.Drawing.Bitmap $width, $height+$graphics = [System.Drawing.Graphics]::FromImage($bitmap)+$hdc = $graphics.GetHdc()+# 2 = PW_RENDERFULLCONTENT: needed for Chromium/Electron content.+$rendered = [CamWindow]::PrintWindow($handle, $hdc, 2)+$graphics.ReleaseHdc($hdc)+$graphics.Dispose()+New-Item -ItemType Directory -Force -Path (Split-Path -Parent $Out) | Out-Null+$bitmap.Save($Out, [System.Drawing.Imaging.ImageFormat]::Png)+$bitmap.Dispose()+"{0}x{1} rendered={2}" -f $width, $height, $rendereddiff --git a/spike/src/codexconfig.test.ts b/spike/src/codexconfig.test.tsindex a7f6257..e8c5531 100644--- a/spike/src/codexconfig.test.ts+++ b/spike/src/codexconfig.test.ts@@ -1,10 +1,10 @@-// Purpose: P1-1/P2 — the managed config.toml settings and the .env NO_PROXY block are inserted and removed-// without touching anything else.+// Purpose: P1-1/P2/P3 — the managed config.toml settings (proxy blocks and the panel's MCP server table) and the+// .env NO_PROXY block are inserted and removed without touching anything else.// Input: synthetic config texts; output: Node test assertions only.import assert from "node:assert/strict";import test from "node:test";-import { coversLoopback, disableProxy, enableNoProxy, enableProxy, noProxyStatus, noProxyValue, proxyStatus } from "./codexconfig.ts";+import { coversLoopback, disableMcpServer, disableProxy, enableMcpServer, enableNoProxy, enableProxy, noProxyStatus, noProxyValue, proxyStatus } from "./codexconfig.ts";const URL = "http://localhost:17891/backend-api/codex";const withFeatures = 'model = "gpt-6-sol"\n\n[features]\nview_image = true\n\n[mcp_servers.demo]\ncommand = "node"\n';@@ -129,3 +129,30 @@ test("NO_PROXY keeps the user's entries and adds only the missing loopback hostsassert.equal(coversLoopback("localhost,127.0.0.1"), false);assert.equal(coversLoopback(null), false);});++const NODE = "C:\\Program Files\\nodejs\\node.exe";+const SCRIPT = "D:\\tools\\cam\\panel-probe.ts";++test("MCP server: the table goes at the end with Windows paths escaped, and disable restores the file byte for byte", () => {+ const { text, changed } = enableMcpServer(withFeatures, "cam_probe", NODE, [SCRIPT]);+ assert.equal(changed, true);+ assert.ok(text.startsWith(withFeatures), "everything before the block is untouched");+ assert.match(text, /\[mcp_servers\.cam_probe\]\ncommand = "C:\\\\Program Files\\\\nodejs\\\\node\.exe"\nargs = \["D:\\\\tools\\\\cam\\\\panel-probe\.ts"\]\ndefault_tools_approval_mode = "approve"\n# <<< codex-attachment-manager <<<\n$/);+ for (const sample of [withFeatures, withoutFeatures, "", "\uFEFF" + withFeatures.replaceAll("\n", "\r\n")]) {+ assert.equal(disableMcpServer(enableMcpServer(sample, "cam_probe", NODE, [SCRIPT]).text).text, sample);+ }+});++test("MCP server block and proxy blocks are switched independently", () => {+ const both = enableMcpServer(enableProxy(withFeatures, URL).text, "cam_probe", NODE, [SCRIPT]).text;+ assert.equal(enableProxy(both, URL).changed, false, "re-enabling the proxy keeps the MCP block");+ assert.equal(disableProxy(both).text, enableMcpServer(withFeatures, "cam_probe", NODE, [SCRIPT]).text);+ assert.equal(disableMcpServer(both).text, enableProxy(withFeatures, URL).text);+ assert.equal(disableMcpServer(disableProxy(both).text).text, withFeatures);+});++test("MCP server: a user's own table with the same name is never overwritten", () => {+ const mine = withFeatures + "\n[mcp_servers.cam_probe]\ncommand = \"python\"\n";+ assert.throws(() => enableMcpServer(mine, "cam_probe", NODE, [SCRIPT]), /already defines mcp_servers\.cam_probe/);+ assert.equal(enableMcpServer(mine, "other", NODE, [SCRIPT]).changed, true);+});diff --git a/spike/src/codexconfig.ts b/spike/src/codexconfig.tsindex 9ef9448..65ca2da 100644--- a/spike/src/codexconfig.ts+++ b/spike/src/codexconfig.ts@@ -52,14 +52,35 @@ function userRespectSystemProxy(lines: string[]): string | null {return null;}+// P3: a local MCP server for the panel, as a table appended at the end of config.toml. Its own marker keeps it+// independent of the proxy blocks: each can be switched on and off alone.+export const MCP_BEGIN = "# >>> codex-attachment-manager: managed MCP server, remove it with the tool >>>";+// Removes every managed block; the blank line enable adds after the top block goes with it.export function disableProxy(text: string): { text: string; changed: boolean } {+ return removeBlocks(text, BEGIN);+}++export function disableMcpServer(text: string): { text: string; changed: boolean } {+ return removeBlocks(text, MCP_BEGIN);+}++export function enableMcpServer(text: string, name: string, command: string, args: string[]): { text: string; changed: boolean } {+ const { bom, body, eol } = parts(disableMcpServer(text).text);+ if (body.split(eol).some((line) => line.trim() === `[mcp_servers.${name}]`)) throw new Error(`config.toml already defines mcp_servers.${name}; not overwriting it`);+ // JSON string syntax is valid TOML basic-string syntax, including backslashes in Windows paths.+ const block = [MCP_BEGIN, `[mcp_servers.${name}]`, `command = ${JSON.stringify(command)}`, `args = ${JSON.stringify(args)}`, 'default_tools_approval_mode = "approve"', END];+ const next = bom + body + (body === "" || body.endsWith(eol) ? "" : eol) + block.join(eol) + eol;+ return { text: next, changed: next !== text };+}++function removeBlocks(text: string, begin: string): { text: string; changed: boolean } {const { bom, body, eol } = parts(text);const lines = body.split(eol);const kept: string[] = [];let changed = false;for (let i = 0; i < lines.length; i++) {- if (lines[i].trim() !== BEGIN) { kept.push(lines[i]); continue; }+ if (lines[i].trim() !== begin) { kept.push(lines[i]); continue; }const end = lines.findIndex((line, j) => j > i && line.trim() === END);if (end < 0) throw new Error("a managed block in config.toml has no end marker");const atTop = kept.length === 0;@@ -191,10 +212,21 @@ function main(): void {if (changed) write(text);if (env.changed) writeEnv(env.text!);console.log(JSON.stringify({ command, changed, envChanged: env.changed, ...proxyStatus(text), ...envStatus(existsSync(dotenv) ? env.text : null) }));+ } else if (command === "mcp-enable") {+ const name = process.argv[process.argv.indexOf("--name") + 1];+ const script = resolve(process.argv[process.argv.indexOf("--script") + 1]);+ if (!process.argv.includes("--name") || !process.argv.includes("--script") || !existsSync(script)) throw new Error("usage: codexconfig.ts mcp-enable --name <server> --script <path to .ts>");+ const { text, changed } = enableMcpServer(current, name, process.execPath, [script]);+ if (changed) write(text);+ console.log(JSON.stringify({ command, changed, mcpServer: name }));+ } else if (command === "mcp-disable") {+ const { text, changed } = disableMcpServer(current);+ if (changed) write(text);+ console.log(JSON.stringify({ command, changed }));} else if (command === "status") {- console.log(JSON.stringify({ command, configExists: existsSync(config), ...proxyStatus(current), ...envStatus(currentEnv) }));+ console.log(JSON.stringify({ command, configExists: existsSync(config), ...proxyStatus(current), mcpServerManaged: current.split(/\r?\n/).some((line) => line.trim() === MCP_BEGIN), ...envStatus(currentEnv) }));} else {- throw new Error("usage: codexconfig.ts enable [--port N] | disable | status");+ throw new Error("usage: codexconfig.ts enable [--port N] | disable | mcp-enable --name <server> --script <path> | mcp-disable | status");}}diff --git a/spike/src/p1-selftest.ts b/spike/src/p1-selftest.tsindex e75b0a8..78e9540 100644--- a/spike/src/p1-selftest.ts+++ b/spike/src/p1-selftest.ts@@ -10,6 +10,7 @@ import { dirname, join, resolve } from "node:path";import { fileURLToPath } from "node:url";import zlib from "node:zlib";import { AppServer, findBundledCodex } from "./appserver.ts";+import { TEST_MODEL } from "./testkit.ts";type Json = Record<string, any>;const localRoot = join(resolve(dirname(fileURLToPath(import.meta.url)), "../.."), "local");@@ -81,7 +82,7 @@ async function main(): Promise<void> {let threadId = "";try {await server.initialize();- const started = await server.request<{ thread: Json }>("thread/start", { cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: true });+ const started = await server.request<{ thread: Json }>("thread/start", { model: TEST_MODEL, cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: true });threadId = started.thread.id;const text = (value: string) => ({ type: "text", text: value, text_elements: [] });results.text = await turn(server, threadId, [text("请只回复“收到”两个字。")]);diff --git a/spike/src/p2-capture.ts b/spike/src/p2-capture.tsindex 0e8a6e0..0d16157 100644--- a/spike/src/p2-capture.ts+++ b/spike/src/p2-capture.ts@@ -7,7 +7,7 @@ import { copyFileSync, mkdirSync, writeFileSync } from "node:fs";import { dirname, join, resolve } from "node:path";import { fileURLToPath } from "node:url";import { AppServer, findBundledCodex } from "./appserver.ts";-import { runTurn, testImages, text } from "./testkit.ts";+import { runTurn, TEST_MODEL, testImages, text } from "./testkit.ts";type Json = Record<string, any>;const localRoot = join(resolve(dirname(fileURLToPath(import.meta.url)), "../.."), "local");@@ -30,7 +30,7 @@ async function main(): Promise<void> {const result: Json = { createdAt: startedAt, label, turns: {} };try {await server.initialize();- const { thread } = await server.request<{ thread: Json }>("thread/start", { cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });+ const { thread } = await server.request<{ thread: Json }>("thread/start", { model: TEST_MODEL, cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });result.threadId = thread.id;await server.request("thread/name/set", { threadId: thread.id, name: `[CAM测试] P2-1 请求结构 ${label}` });result.turns.upload = await runTurn(server, thread.id, [text("这是三张测试图,先不用描述,只回复“收到”。"), ...["a.png", "b.png", "b-copy.png"].map((name) => ({ type: "localImage", path: file(name) }))]);diff --git a/spike/src/p2-selftest.ts b/spike/src/p2-selftest.tsindex 22cfa3d..eb5a741 100644--- a/spike/src/p2-selftest.ts+++ b/spike/src/p2-selftest.ts@@ -12,7 +12,7 @@ import { fileURLToPath } from "node:url";import { AppServer, findBundledCodex } from "./appserver.ts";import { codexHome } from "./codexconfig.ts";import { readSelection, writeSelection } from "./selection.ts";-import { runTurn, testImages, text } from "./testkit.ts";+import { runTurn, TEST_MODEL, testImages, text } from "./testkit.ts";import { loadThreadIndex } from "./thread-index.ts";type Json = Record<string, any>;@@ -40,7 +40,7 @@ async function main(): Promise<void> {let threadId = "";try {await server.initialize();- const { thread } = await server.request<{ thread: Json }>("thread/start", { cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });+ const { thread } = await server.request<{ thread: Json }>("thread/start", { model: TEST_MODEL, cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });threadId = thread.id;result.threadId = threadId;await server.request("thread/name/set", { threadId, name: `[CAM测试] P2 改写自测 ${label}` });diff --git a/spike/src/panel-probe.html b/spike/src/panel-probe.htmlnew file mode 100644index 0000000..00f13e2--- /dev/null+++ b/spike/src/panel-probe.html@@ -0,0 +1,113 @@+<!doctype html>+<html lang="zh-CN">+<head>+<meta charset="utf-8">+<meta name="viewport" content="width=device-width, initial-scale=1">+<title>素材面板试验</title>+<style>+ :root { color-scheme: light dark; font: 13px/1.5 system-ui, "Microsoft YaHei", sans-serif; }+ body { margin: 0; padding: 12px 14px; }+ h1 { font-size: 15px; margin: 0 0 8px; }+ .row { display: flex; gap: 8px; align-items: center; margin: 3px 0; }+ .label { min-width: 9em; opacity: .75; }+ .ok { color: #1a7f37; } .bad { color: #cf222e; }+ pre { font-size: 11px; white-space: pre-wrap; word-break: break-all; max-height: 220px; overflow: auto; background: rgba(127,127,127,.12); padding: 8px; border-radius: 6px; }+ button { font: inherit; padding: 3px 10px; }+ img { width: 40px; height: 40px; border-radius: 6px; border: 1px solid rgba(127,127,127,.4); }+</style>+</head>+<body>+<h1>素材管理器 · 界面试验</h1>+<div class="row"><span class="label">静态页面</span><b class="ok">已加载</b></div>+<div class="row"><span class="label">缩略图(data URL)</span><img id="thumb" alt=""><b id="thumbState">加载中…</b></div>+<div class="row"><span class="label">与 Codex 握手</span><b id="handshake">进行中…</b></div>+<div class="row"><span class="label">回调插件服务</span><b id="ping">未开始</b></div>+<div class="row"><span class="label">访问本机地址</span><b id="fetch">未开始</b></div>+<div class="row"><button id="again">再回调一次</button><button id="fullscreen">请求全屏</button></div>+<pre id="log"></pre>+<script>+(() => {+ const $ = (id) => document.getElementById(id);+ const lines = [];+ const log = (line) => { lines.push(line); $("log").textContent = lines.join("\n"); resize(); };+ const set = (id, text, ok) => { $(id).textContent = text; $(id).className = ok ? "ok" : "bad"; resize(); };+ const report = { windowOpenai: typeof window.openai === "object" && window.openai ? Object.keys(window.openai) : null, received: [], csp: [] };+ let nextId = 1;+ const pending = new Map();+ const post = (message) => window.parent.postMessage({ jsonrpc: "2.0", ...message }, "*");+ const request = (method, params, timeoutMs = 10000) => new Promise((resolve, reject) => {+ const id = nextId++;+ const timer = setTimeout(() => { pending.delete(id); reject(new Error(method + " 超时")); }, timeoutMs);+ pending.set(id, { resolve: (value) => { clearTimeout(timer); resolve(value); }, reject: (error) => { clearTimeout(timer); reject(error); } });+ post({ id, method, params });+ });+ function resize() {+ post({ method: "ui/notifications/size-changed", params: { width: document.documentElement.scrollWidth, height: document.documentElement.scrollHeight } });+ }+ document.addEventListener("securitypolicyviolation", (event) => {+ report.csp.push(event.violatedDirective + " " + event.blockedURI);+ log("CSP 拦截:" + event.violatedDirective + " " + event.blockedURI);+ });+ window.addEventListener("message", (event) => {+ const message = event.data;+ if (!message || typeof message !== "object") return;+ if (message.id !== undefined && pending.has(message.id) && ("result" in message || "error" in message)) {+ const waiter = pending.get(message.id);+ pending.delete(message.id);+ if (message.error) waiter.reject(new Error(JSON.stringify(message.error))); else waiter.resolve(message.result);+ return;+ }+ report.received.push(message.method || "reply");+ log("收到:" + (message.method || "reply") + " " + JSON.stringify(message.params ?? message.result ?? {}).slice(0, 160));+ });+ log("window.openai:" + JSON.stringify(report.windowOpenai));++ // A data-URL image drawn on a canvas stands in for a thumbnail.+ const canvas = document.createElement("canvas");+ canvas.width = canvas.height = 40;+ const ctx = canvas.getContext("2d");+ ctx.fillStyle = "#fad23c"; ctx.fillRect(0, 0, 40, 40);+ ctx.fillStyle = "#2846d2"; ctx.beginPath(); ctx.arc(20, 20, 11, 0, Math.PI * 2); ctx.fill();+ $("thumb").onload = () => { set("thumbState", "显示正常", true); report.thumb = "ok"; };+ $("thumb").onerror = () => { set("thumbState", "显示失败", false); report.thumb = "error"; };+ $("thumb").src = canvas.toDataURL("image/png");++ const ping = async (reason) => {+ try {+ const result = await request("tools/call", { name: "cam_probe_ping", arguments: { reason, report } });+ set("ping", "成功:" + JSON.stringify(result.structuredContent ?? result.content).slice(0, 100), true);+ } catch (error) { set("ping", "失败:" + error.message, false); }+ };+ $("again").onclick = () => ping("button");+ $("fullscreen").onclick = async () => {+ try { log("全屏:" + JSON.stringify(await request("ui/request-display-mode", { mode: "fullscreen" }))); }+ catch (error) { log("全屏失败:" + error.message); }+ };++ (async () => {+ try {+ const init = await request("ui/initialize", { protocolVersion: "2026-01-26", appInfo: { name: "codex-attachment-manager-probe", version: "0.0.1" }, appCapabilities: {} });+ const context = init.hostContext || {};+ report.host = { protocolVersion: init.protocolVersion, hostInfo: init.hostInfo, hostCapabilities: init.hostCapabilities, contextKeys: Object.keys(context), displayMode: context.displayMode, availableDisplayModes: context.availableDisplayModes, theme: context.theme, locale: context.locale };+ post({ method: "ui/notifications/initialized", params: {} });+ set("handshake", "成功 " + JSON.stringify(report.host.hostInfo ?? {}).slice(0, 80), true);+ log("握手结果:" + JSON.stringify(report.host).slice(0, 600));+ } catch (error) {+ report.handshakeError = error.message;+ set("handshake", "失败:" + error.message, false);+ }+ try {+ const response = await fetch("http://localhost:17891/__cam/health");+ report.fetch = "status " + response.status;+ set("fetch", "成功(" + response.status + ")", true);+ } catch (error) {+ report.fetch = String(error);+ set("fetch", "失败:" + error.message, false);+ }+ await ping("load");+ })();+ resize();+})();+</script>+</body>+</html>diff --git a/spike/src/panel-probe.ts b/spike/src/panel-probe.tsnew file mode 100644index 0000000..4000669--- /dev/null+++ b/spike/src/panel-probe.ts@@ -0,0 +1,84 @@+// Purpose: P3-1 — probe what Codex Desktop offers an MCP App on Windows: whether the widget renders, finishes the+// MCP Apps handshake, calls back into this server, shows data-URL images and reaches localhost; and when and how often+// Codex starts this server (per thread or shared).+// Input: MCP JSON-RPC over stdio (newline-delimited); the widget page is panel-probe.html next to this file.+// Output: one JSON line per event in local/p3/panel-probe.jsonl (local only; no conversation content is logged).++import { appendFileSync, mkdirSync, readFileSync } from "node:fs";+import { dirname, join, resolve } from "node:path";+import { createInterface } from "node:readline";+import { fileURLToPath } from "node:url";++type Json = Record<string, any>;+const here = dirname(fileURLToPath(import.meta.url));+const logDir = join(resolve(here, "../.."), "local", "p3");+const RESOURCE = "ui://codex-attachment-manager/probe.html";+const MIME = "text/html;profile=mcp-app";+let calls = 0;++function log(entry: Json): void {+ mkdirSync(logDir, { recursive: true });+ appendFileSync(join(logDir, "panel-probe.jsonl"), `${JSON.stringify({ at: new Date().toISOString(), pid: process.pid, ...entry })}\n`);+}++const send = (message: Json) => process.stdout.write(`${JSON.stringify({ jsonrpc: "2.0", ...message })}\n`);++const TOOLS = [+ {+ name: "cam_panel_probe",+ title: "素材面板(试验)",+ description: "打开 Codex 素材管理器的界面试验面板。用户要求打开素材面板时调用,不需要参数。",+ inputSchema: { type: "object", properties: {}, additionalProperties: false },+ _meta: { ui: { resourceUri: RESOURCE }, "openai/ui": { preferredModelDisplayMode: "inline" } },+ },+ {+ name: "cam_probe_ping",+ title: "素材面板回调(试验)",+ description: "素材面板内部使用:面板页面回调插件服务。模型不需要调用。",+ inputSchema: { type: "object", properties: { reason: { type: "string" }, report: { type: "object" } } },+ _meta: { ui: { resourceUri: RESOURCE, visibility: ["app"] } },+ },+];++function handle(message: Json): void {+ const { id, method, params = {} } = message;+ if (method === "initialize") {+ log({ event: "initialize", client: params.clientInfo ?? null, protocolVersion: params.protocolVersion ?? null, capabilities: params.capabilities ?? null });+ send({ id, result: { protocolVersion: params.protocolVersion ?? "2025-06-18", capabilities: { tools: {}, resources: {} }, serverInfo: { name: "codex-attachment-manager-probe", version: "0.0.1" } } });+ } else if (method === "tools/list") {+ send({ id, result: { tools: TOOLS } });+ } else if (method === "tools/call") {+ calls++;+ log({ event: "tools/call", name: params.name, arguments: params.arguments ?? null, meta: params._meta ?? null });+ const meta = params._meta ?? {};+ const threadId = meta.threadId ?? meta.thread_id ?? null;+ if (params.name === "cam_panel_probe") {+ send({ id, result: { content: [{ type: "text", text: "已打开素材面板(试验版)。" }], structuredContent: { pid: process.pid, threadId, calls } } });+ } else if (params.name === "cam_probe_ping") {+ send({ id, result: { content: [{ type: "text", text: "pong" }], structuredContent: { pid: process.pid, calls, at: new Date().toISOString() } } });+ } else {+ send({ id, error: { code: -32602, message: `unknown tool ${params.name}` } });+ }+ } else if (method === "resources/list") {+ send({ id, result: { resources: [{ uri: RESOURCE, name: "素材面板(试验)", mimeType: MIME }] } });+ } else if (method === "resources/templates/list") {+ send({ id, result: { resourceTemplates: [] } });+ } else if (method === "resources/read") {+ log({ event: "resources/read", uri: params.uri ?? null });+ const text = readFileSync(join(here, "panel-probe.html"), "utf8");+ send({ id, result: { contents: [{ uri: RESOURCE, mimeType: MIME, text, _meta: { ui: { csp: { connectDomains: ["http://localhost:17891"], resourceDomains: [] }, prefersBorder: true } } }] } });+ } else if (method === "ping") {+ send({ id, result: {} });+ } else if (id !== undefined) {+ log({ event: "unsupported", method });+ send({ id, error: { code: -32601, message: `unsupported: ${method}` } });+ }+}++// Which process started us, and whether Codex passes thread identity in the environment (names and ids only).+log({ event: "start", ppid: process.ppid, argv: process.argv.slice(1), env: Object.fromEntries(Object.entries(process.env).filter(([name]) => /THREAD|SESSION/i.test(name) && !/TOKEN|KEY|SECRET/i.test(name))) });+process.stdin.on("close", () => { log({ event: "stdin-closed", calls }); process.exit(0); });+createInterface({ input: process.stdin, crlfDelay: Infinity }).on("line", (line) => {+ if (!line.trim()) return;+ try { handle(JSON.parse(line)); } catch (error) { log({ event: "error", error: String(error) }); }+});diff --git a/spike/src/testkit.ts b/spike/src/testkit.tsindex c51584c..3abd80f 100644--- a/spike/src/testkit.ts+++ b/spike/src/testkit.ts@@ -49,6 +49,9 @@ export const testImages = {},};+// Tests use gpt-6-sol at low effort to save the user's tokens (AGENTS.md).+export const TEST_MODEL = "gpt-6-sol";+export const text = (value: string) => ({ type: "text", text: value, text_elements: [] });export async function runTurn(server: AppServer, threadId: string, input: Json[], timeoutMs = 20 * 60_000): Promise<Json> {