Skip to content

Stop paying the multimodal tax. Your AI agent keeps every image it has seen but only sends the ones it needs: lighter chats, no endless reconnecting, no starting over. Works with Codex, Claude Code, Cline, Gemini CLI, OpenCode, WorkBuddy and more. Install once, forget it.

feat(spike): P3 面板试验——对话内界面(MCP App)试验服务、窗口截屏脚本、插件服务的配置段落

chipfighter5916e4b58edc
diff --git a/AGENTS.md b/AGENTS.md
index f1e3ce2..3c6bdca 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -16,6 +16,7 @@ Codex 上下文素材管理器(暂名):让用户逐轮决定,Codex 任
- 仓库里的文档和代码不写用户本机的具体路径;通用位置(如 `~/.codex`、`%LOCALAPPDATA%`)可以写。
- 先在 Windows 上开发和测试。技术栈用 TypeScript/Node。
- 界面开发:Claude 自己截图检查效果;做到界面的关键节点时,截图给用户看,停下来等反馈再继续(用户 2026-09-24 确认)。
+- 测试时调用模型一律用 `gpt-6-sol`,推理强度 `low`,包括脚本自测和桌面版测试,为了省 token(用户 2026-09-24 要求)。
- 分工:Claude 维护 `docs/` 下的规格和方案,并负责开发和测试(阶段 0 的 T0–T2 由 Codex 完成;测试需要在 Codex 之外启动 app-server,所以从 T3 起改由 Claude 执行)。其他协作者不修改 spec.md 和 plan.md。
## Git 规范
diff --git a/docs/plan.md b/docs/plan.md
index 69bc290..6d86073 100644
--- a/docs/plan.md
+++ b/docs/plan.md
@@ -1,6 +1,6 @@
# 技术方案
-> 目标、约束和验收关卡以 [spec.md](spec.md) 为准。本文件说明怎么做、为什么这样做。第 1–8 节是阶段 0 的设计(已完成,结果见 [phase0-report.md](phase0-report.md));第 9、10 节是正在验证的代理路线:第 9 节是接入代理(P1),第 10 节是按勾选改写(P2)。
+> 目标、约束和验收关卡以 [spec.md](spec.md) 为准。本文件说明怎么做、为什么这样做。第 1–8 节是阶段 0 的设计(已完成,结果见 [phase0-report.md](phase0-report.md));第 9、10 节是代理路线的接入(P1)和按勾选改写(P2),都已在桌面版验证通过;第 11 节是正在做的面板(P3)。
## 1. 已知事实(2026-09-24)
@@ -227,3 +227,37 @@ P1 通过后,再按用户的节奏一步步探索改写。
- 取消勾选时,如果这个任务正在一轮对话中间,而且还在用 WebSocket,这一轮剩下的请求可能仍带着原图。从下一轮起一定生效。
- 生图和改图:当前版本要靠 `.env` 里的 `NO_PROXY` 才能经过代理,见第 9 节“不等更新的办法”。
- 桌面版会在用户消息里附一段 “Files mentioned by the user”,写着上传文件的完整路径。代理不改这段用户消息的文字,靠占位符里“不要自己用工具去读取”的要求约束模型;P2-6 里模型遵守了。
+
+## 11. 面板(P3)
+
+### 已查证的事实
+
+依据桌面版自带的 `0.155.0-alpha.16.4` 的源码、上游 issue 和本机检查:
+
+- **对话里的界面(MCP Apps)**:
+ - 工具在自己的说明里用 `_meta.ui.resourceUri` 指向一个 `ui://` 页面;模型调用这个工具时,桌面版在对话里显示这个页面。
+ - `_meta["openai/ui"].preferredModelDisplayMode` 可以选 `inline`(嵌在对话里)或 `fullscreen`(全屏)。
+ - 页面的类型是 `text/html;profile=mcp-app`,在沙箱里运行,通过 postMessage 和桌面版握手,再经桌面版回调同一个服务的工具。
+ - 界面信息会存进聊天记录,重启后不用重新查询插件也能显示出来。
+- **已知问题(上游未修)**:
+ - [#47512](https://github.com/openai/codex/issues/47512):Windows 上页面偶尔整块空白,都出现在页面地址变化之后。所以页面地址要保持固定。
+ - [#45913](https://github.com/openai/codex/issues/45913):页面访问不了 `http://localhost`,即使列进了 CSP。所以面板只经桌面版回调插件服务拿数据,缩略图用 data URL。
+- **插件格式**:`.codex-plugin/plugin.json` 可以带技能(skills)、MCP 服务(`mcpServers`)和钩子(hooks)。钩子事件有 `SessionStart`、`UserPromptSubmit` 等,可以执行命令。
+- **接入配置**:`config.toml` 的 `[mcp_servers.<名字>]` 写启动命令;`default_tools_approval_mode = "approve"` 让它的工具不用逐次确认。
+- **桌面版的程序**:OpenAI.Codex 安装目录下的 `ChatGPT.exe`,窗口标题是“ChatGPT”。截屏脚本 `spike/scripts/capture-codex.ps1` 按安装目录找这个窗口,只截它。
+
+### 设计(待 P3-1 实测确认)
+
+- **面板**:由插件的 MCP 服务提供的一个 MCP App,页面地址固定。打开方式是用户让模型调用“打开面板”工具,这会花一轮很小的请求;能不能不经过模型、从侧栏“插件”直接打开,P3-1 里一并查看。
+- **数据**:面板经桌面版回调服务的工具,拿到当前任务的图片列表:编号、名称、来源、轮次、尺寸、大小、内容相同、勾选状态和缩略图。勾选和取消写进勾选状态文件,代理照旧读取。
+- **缩略图**:服务端从本地记录解码 PNG、缩成小图,以 data URL 放进只给面板看的 `_meta`,不进入模型的上下文。要确认 Codex 不把 `_meta` 发给模型。
+- **代理随 Codex 启停**:插件的 `SessionStart` 和 `UserPromptSubmit` 钩子检查代理,没在运行就在后台拉起;代理发现 Codex 进程都退出后,自己退出。
+- **交付**:本地插件,包含说明用法的技能、MCP 服务和钩子;需要的配置由工具写成带标记的段落。
+
+### 验证顺序
+
+1. P3-1 试验面板:在 `config.toml` 里临时接入试验服务;用户重启桌面版,在测试任务里让模型打开面板。截图并查看日志,确认页面能否显示、能否握手、能否回调服务、data URL 图片能否显示、`localhost` 是否被拦截,以及服务是每个任务一个还是共用一个。
+2. P3-2 面板数据接口:服务端工具和单元测试。
+3. P3-3 面板界面:做到关键节点,截图给用户看,等反馈。
+4. P3-4 代理随 Codex 启停。
+5. P3-5 打包成本地插件,在桌面版上端到端验证。
diff --git a/docs/spec.md b/docs/spec.md
index 632ff3e..af63456 100644
--- a/docs/spec.md
+++ b/docs/spec.md
@@ -68,7 +68,7 @@
- 原始对话记录和素材文件始终只读,不修改、不移动、不删除。不直接写入 `~/.codex` 下的任何文件,只有三个例外:
- 官方 app-server 在创建和运行任务时自己产生的正常写入;
- - 为接入本地代理,工具可以修改 `config.toml` 里由它自己管理、带标记的设置(用户 2026-09-24 授权)。改之前先备份,能一键恢复,其他设置一律不动;
+ - 为接入本地代理和面板服务,工具可以修改 `config.toml` 里由它自己管理、带标记的设置(用户 2026-09-24 授权)。改之前先备份,能一键恢复,其他设置一律不动;
- 为让生图和改图也能连到本地代理,工具可以在 `~/.codex/.env` 里写一段带标记的 `NO_PROXY`,只含本机地址,并保留用户自己的条目(用户 2026-09-24 同意)。`.env` 可能含密钥,工具不复制、不输出它,只增删自己的那一段;文件里只剩这一段时,恢复时连文件一起删掉。
- 真实数据不进仓库:从真实会话和用户素材里提取的图片、记录片段、日志和备份,只能放在本机的 `local/` 目录(已被 git 忽略),任何时候都不提交。仓库里的样例数据只能是合成的或脱敏的。
- 仓库里的文档和代码不写用户本机的具体路径(例如个人项目文件夹)。`~/.codex`、`%LOCALAPPDATA%` 这类通用位置可以写。本项目以后会开源。
@@ -76,7 +76,7 @@
- 代理只替换用户取消的图片,连同 Codex 附在它前后的路径标签;请求里的其他内容原样转发。
- 选择权永远在用户:哪些素材进入上下文,只由用户在面板里勾选决定。工具和模型都不能自动勾选或取消;模型只能按约定索要(例如回复“需要 IMG-004”),给不给由用户决定。
- 先在 Windows 上开发和测试。技术栈:TypeScript/Node。
-- 交付形态:Codex 插件(skill 加本地 MCP 服务);面板位置见第 1 节“界面”。
+- 交付形态:Codex 插件(技能、本地 MCP 服务和钩子);面板位置见第 1 节“界面”。
## 4. 验收标准
@@ -105,16 +105,16 @@
### 首版
-> 以下按路线 A 写成,代理路线确定后重写。
+> 按代理路线写成(2026-09-24 改写,原来按路线 A 写的版本已作废)。
-1. 选定一个任务后,按时间顺序列出全部图片出现记录,无法识别的项单独列出。
-2. 可以勾选、取消、重新勾选,并预览本轮将发送什么(实际素材、重复引用、占位符、预计字节数)。
-3. 一键在新任务中继续:未勾选的图片不进入请求,选中的图可以继续修改。
-4. 取消过的素材在之后任何一轮都能重新勾选并被发送。
-5. 操作前后,原始记录和素材文件的哈希不变。
-6. 显示操作前后的请求大小对比。
+1. 装好后不用手动配置:工具自动接入 Codex,代理跟着 Codex 一起启动、一起退出;关闭工具时一键恢复,只删掉自己写入的内容。
+2. 在 Codex 里打开面板,按时间顺序列出当前任务的全部图片出现记录,包括缩略图、编号、名称、来源、轮次、尺寸、大小、内容相同的关系和勾选状态。无法识别或不能取消的项要单独标出。
+3. 勾选、取消、重新勾选都从下一次请求起生效;对话始终在同一个任务里,连续可见。
+4. 面板显示本次会发送什么:照常发送的图、重复占位符和普通占位符,以及取消前后的请求大小对比。
+5. 模型回复“需要 IMG-xxx”时,用户能在面板里找到那一条并勾选。
+6. 没有取消任何图的任务,请求原样转发;改写不了时原样发送,并在面板上提示。
+7. 原始记录和素材文件只读,操作前后哈希不变。
## 5. 待定问题
- 路线:路线 A(用 `thread/inject_items` 在新任务里重建)技术上可行,但用户不接受“重建后在界面上看不到原对话”。代理路线(任务不换,每次请求发出前由本地代理按勾选改写)已在桌面版验证通过:P1 只转发、P2 按勾选改写,生图和改图也包括在内,见 plan.md 第 9、10 节和 p2-report.md。用户 2026-09-24 决定接下来做面板。
-- 第 4 节的首版验收标准是按路线 A 写的,开始做面板之前要按代理路线重写。
\ No newline at end of file
diff --git a/docs/tasks.md b/docs/tasks.md
index f4542a6..344d218 100644
--- a/docs/tasks.md
+++ b/docs/tasks.md
@@ -2,7 +2,22 @@
> 方法见 [plan.md](plan.md)。按顺序做,完成一项勾一项,并写明证据放在哪里。
-## 当前:代理验证 P2(按勾选改写)
+## 当前:面板 P3
+
+设计见 plan.md 第 11 节,验收标准见 spec.md 第 4 节“首版”。测试时调用模型一律用 `gpt-6-sol`,推理强度 `low`。
+
+- [ ] **P3-1 试验面板**【需用户】(`spike/src/panel-probe.ts`、`panel-probe.html`;截屏 `spike/scripts/capture-codex.ps1`;配置 `codexconfig.ts mcp-enable/mcp-disable`)
+ - 在 `config.toml` 末尾临时接入试验服务(带标记,能单独恢复)。用户重启桌面版,新建测试任务,让模型打开面板。
+ - 截图并查看 `local/p3/panel-probe.jsonl`,确认:页面能否显示、能否握手、能否回调服务、data URL 图片能否显示、`localhost` 是否被拦截、服务是每个任务一个还是共用一个、能否不经模型从侧栏“插件”打开。
+ - 结束后恢复配置。
+- [ ] **P3-2 面板数据接口**:列出当前任务的图片、缩略图、勾选和取消;配单元测试。
+- [ ] **P3-3 面板界面**【需用户】:做到关键节点,截图给用户看,等反馈。
+- [ ] **P3-4 代理随 Codex 启停**。
+- [ ] **P3-5 打包成本地插件,桌面版端到端验证**【需用户】。
+
+---
+
+## 已完成:代理验证 P2(按勾选改写)
设计见 plan.md 第 10 节,验收标准见 spec.md 第 4 节“代理路线 P2”。
diff --git a/spike/scripts/capture-codex.ps1 b/spike/scripts/capture-codex.ps1
new file mode 100644
index 0000000..3959e34
--- /dev/null
+++ b/spike/scripts/capture-codex.ps1
@@ -0,0 +1,40 @@
+# Purpose: P3 — capture only the Codex Desktop window to a PNG for UI checks, even when other windows cover it
+# (PrintWindow with PW_RENDERFULLCONTENT). The desktop app's process is ChatGPT.exe inside the OpenAI.Codex package.
+# Input: -Out <path.png>. Output: the PNG file and one line with its size.
+param([Parameter(Mandatory = $true)] [string] $Out)
+
+Add-Type -AssemblyName System.Drawing
+Add-Type @"
+using System;
+using System.Runtime.InteropServices;
+public static class CamWindow {
+ [StructLayout(LayoutKind.Sequential)] public struct RECT { public int Left, Top, Right, Bottom; }
+ [DllImport("user32.dll")] public static extern bool SetProcessDPIAware();
+ [DllImport("user32.dll")] public static extern bool GetWindowRect(IntPtr hWnd, out RECT rect);
+ [DllImport("user32.dll")] public static extern bool PrintWindow(IntPtr hWnd, IntPtr hdc, uint flags);
+ [DllImport("user32.dll")] public static extern bool IsIconic(IntPtr hWnd);
+}
+"@
+# Physical pixels, so the window is not cropped on scaled displays.
+[void][CamWindow]::SetProcessDPIAware()
+
+$process = Get-Process | Where-Object { $_.MainWindowHandle -ne 0 -and $_.Path -like '*\OpenAI.Codex_*' } | Select-Object -First 1
+if (-not $process) { throw 'Codex Desktop window not found' }
+$handle = $process.MainWindowHandle
+if ([CamWindow]::IsIconic($handle)) { throw 'the Codex window is minimized' }
+
+$rect = New-Object CamWindow+RECT
+[void][CamWindow]::GetWindowRect($handle, [ref]$rect)
+$width = $rect.Right - $rect.Left
+$height = $rect.Bottom - $rect.Top
+$bitmap = New-Object System.Drawing.Bitmap $width, $height
+$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
+$hdc = $graphics.GetHdc()
+# 2 = PW_RENDERFULLCONTENT: needed for Chromium/Electron content.
+$rendered = [CamWindow]::PrintWindow($handle, $hdc, 2)
+$graphics.ReleaseHdc($hdc)
+$graphics.Dispose()
+New-Item -ItemType Directory -Force -Path (Split-Path -Parent $Out) | Out-Null
+$bitmap.Save($Out, [System.Drawing.Imaging.ImageFormat]::Png)
+$bitmap.Dispose()
+"{0}x{1} rendered={2}" -f $width, $height, $rendered
diff --git a/spike/src/codexconfig.test.ts b/spike/src/codexconfig.test.ts
index a7f6257..e8c5531 100644
--- a/spike/src/codexconfig.test.ts
+++ b/spike/src/codexconfig.test.ts
@@ -1,10 +1,10 @@
-// Purpose: P1-1/P2 — the managed config.toml settings and the .env NO_PROXY block are inserted and removed
-// without touching anything else.
+// Purpose: P1-1/P2/P3 — the managed config.toml settings (proxy blocks and the panel's MCP server table) and the
+// .env NO_PROXY block are inserted and removed without touching anything else.
// Input: synthetic config texts; output: Node test assertions only.
import assert from "node:assert/strict";
import test from "node:test";
-import { coversLoopback, disableProxy, enableNoProxy, enableProxy, noProxyStatus, noProxyValue, proxyStatus } from "./codexconfig.ts";
+import { coversLoopback, disableMcpServer, disableProxy, enableMcpServer, enableNoProxy, enableProxy, noProxyStatus, noProxyValue, proxyStatus } from "./codexconfig.ts";
const URL = "http://localhost:17891/backend-api/codex";
const withFeatures = 'model = "gpt-6-sol"\n\n[features]\nview_image = true\n\n[mcp_servers.demo]\ncommand = "node"\n';
@@ -129,3 +129,30 @@ test("NO_PROXY keeps the user's entries and adds only the missing loopback hosts
assert.equal(coversLoopback("localhost,127.0.0.1"), false);
assert.equal(coversLoopback(null), false);
});
+
+const NODE = "C:\\Program Files\\nodejs\\node.exe";
+const SCRIPT = "D:\\tools\\cam\\panel-probe.ts";
+
+test("MCP server: the table goes at the end with Windows paths escaped, and disable restores the file byte for byte", () => {
+ const { text, changed } = enableMcpServer(withFeatures, "cam_probe", NODE, [SCRIPT]);
+ assert.equal(changed, true);
+ assert.ok(text.startsWith(withFeatures), "everything before the block is untouched");
+ assert.match(text, /\[mcp_servers\.cam_probe\]\ncommand = "C:\\\\Program Files\\\\nodejs\\\\node\.exe"\nargs = \["D:\\\\tools\\\\cam\\\\panel-probe\.ts"\]\ndefault_tools_approval_mode = "approve"\n# <<< codex-attachment-manager <<<\n$/);
+ for (const sample of [withFeatures, withoutFeatures, "", "\uFEFF" + withFeatures.replaceAll("\n", "\r\n")]) {
+ assert.equal(disableMcpServer(enableMcpServer(sample, "cam_probe", NODE, [SCRIPT]).text).text, sample);
+ }
+});
+
+test("MCP server block and proxy blocks are switched independently", () => {
+ const both = enableMcpServer(enableProxy(withFeatures, URL).text, "cam_probe", NODE, [SCRIPT]).text;
+ assert.equal(enableProxy(both, URL).changed, false, "re-enabling the proxy keeps the MCP block");
+ assert.equal(disableProxy(both).text, enableMcpServer(withFeatures, "cam_probe", NODE, [SCRIPT]).text);
+ assert.equal(disableMcpServer(both).text, enableProxy(withFeatures, URL).text);
+ assert.equal(disableMcpServer(disableProxy(both).text).text, withFeatures);
+});
+
+test("MCP server: a user's own table with the same name is never overwritten", () => {
+ const mine = withFeatures + "\n[mcp_servers.cam_probe]\ncommand = \"python\"\n";
+ assert.throws(() => enableMcpServer(mine, "cam_probe", NODE, [SCRIPT]), /already defines mcp_servers\.cam_probe/);
+ assert.equal(enableMcpServer(mine, "other", NODE, [SCRIPT]).changed, true);
+});
diff --git a/spike/src/codexconfig.ts b/spike/src/codexconfig.ts
index 9ef9448..65ca2da 100644
--- a/spike/src/codexconfig.ts
+++ b/spike/src/codexconfig.ts
@@ -52,14 +52,35 @@ function userRespectSystemProxy(lines: string[]): string | null {
return null;
}
+// P3: a local MCP server for the panel, as a table appended at the end of config.toml. Its own marker keeps it
+// independent of the proxy blocks: each can be switched on and off alone.
+export const MCP_BEGIN = "# >>> codex-attachment-manager: managed MCP server, remove it with the tool >>>";
+
// Removes every managed block; the blank line enable adds after the top block goes with it.
export function disableProxy(text: string): { text: string; changed: boolean } {
+ return removeBlocks(text, BEGIN);
+}
+
+export function disableMcpServer(text: string): { text: string; changed: boolean } {
+ return removeBlocks(text, MCP_BEGIN);
+}
+
+export function enableMcpServer(text: string, name: string, command: string, args: string[]): { text: string; changed: boolean } {
+ const { bom, body, eol } = parts(disableMcpServer(text).text);
+ if (body.split(eol).some((line) => line.trim() === `[mcp_servers.${name}]`)) throw new Error(`config.toml already defines mcp_servers.${name}; not overwriting it`);
+ // JSON string syntax is valid TOML basic-string syntax, including backslashes in Windows paths.
+ const block = [MCP_BEGIN, `[mcp_servers.${name}]`, `command = ${JSON.stringify(command)}`, `args = ${JSON.stringify(args)}`, 'default_tools_approval_mode = "approve"', END];
+ const next = bom + body + (body === "" || body.endsWith(eol) ? "" : eol) + block.join(eol) + eol;
+ return { text: next, changed: next !== text };
+}
+
+function removeBlocks(text: string, begin: string): { text: string; changed: boolean } {
const { bom, body, eol } = parts(text);
const lines = body.split(eol);
const kept: string[] = [];
let changed = false;
for (let i = 0; i < lines.length; i++) {
- if (lines[i].trim() !== BEGIN) { kept.push(lines[i]); continue; }
+ if (lines[i].trim() !== begin) { kept.push(lines[i]); continue; }
const end = lines.findIndex((line, j) => j > i && line.trim() === END);
if (end < 0) throw new Error("a managed block in config.toml has no end marker");
const atTop = kept.length === 0;
@@ -191,10 +212,21 @@ function main(): void {
if (changed) write(text);
if (env.changed) writeEnv(env.text!);
console.log(JSON.stringify({ command, changed, envChanged: env.changed, ...proxyStatus(text), ...envStatus(existsSync(dotenv) ? env.text : null) }));
+ } else if (command === "mcp-enable") {
+ const name = process.argv[process.argv.indexOf("--name") + 1];
+ const script = resolve(process.argv[process.argv.indexOf("--script") + 1]);
+ if (!process.argv.includes("--name") || !process.argv.includes("--script") || !existsSync(script)) throw new Error("usage: codexconfig.ts mcp-enable --name <server> --script <path to .ts>");
+ const { text, changed } = enableMcpServer(current, name, process.execPath, [script]);
+ if (changed) write(text);
+ console.log(JSON.stringify({ command, changed, mcpServer: name }));
+ } else if (command === "mcp-disable") {
+ const { text, changed } = disableMcpServer(current);
+ if (changed) write(text);
+ console.log(JSON.stringify({ command, changed }));
} else if (command === "status") {
- console.log(JSON.stringify({ command, configExists: existsSync(config), ...proxyStatus(current), ...envStatus(currentEnv) }));
+ console.log(JSON.stringify({ command, configExists: existsSync(config), ...proxyStatus(current), mcpServerManaged: current.split(/\r?\n/).some((line) => line.trim() === MCP_BEGIN), ...envStatus(currentEnv) }));
} else {
- throw new Error("usage: codexconfig.ts enable [--port N] | disable | status");
+ throw new Error("usage: codexconfig.ts enable [--port N] | disable | mcp-enable --name <server> --script <path> | mcp-disable | status");
}
}
diff --git a/spike/src/p1-selftest.ts b/spike/src/p1-selftest.ts
index e75b0a8..78e9540 100644
--- a/spike/src/p1-selftest.ts
+++ b/spike/src/p1-selftest.ts
@@ -10,6 +10,7 @@ import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import zlib from "node:zlib";
import { AppServer, findBundledCodex } from "./appserver.ts";
+import { TEST_MODEL } from "./testkit.ts";
type Json = Record<string, any>;
const localRoot = join(resolve(dirname(fileURLToPath(import.meta.url)), "../.."), "local");
@@ -81,7 +82,7 @@ async function main(): Promise<void> {
let threadId = "";
try {
await server.initialize();
- const started = await server.request<{ thread: Json }>("thread/start", { cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: true });
+ const started = await server.request<{ thread: Json }>("thread/start", { model: TEST_MODEL, cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: true });
threadId = started.thread.id;
const text = (value: string) => ({ type: "text", text: value, text_elements: [] });
results.text = await turn(server, threadId, [text("请只回复“收到”两个字。")]);
diff --git a/spike/src/p2-capture.ts b/spike/src/p2-capture.ts
index 0e8a6e0..0d16157 100644
--- a/spike/src/p2-capture.ts
+++ b/spike/src/p2-capture.ts
@@ -7,7 +7,7 @@ import { copyFileSync, mkdirSync, writeFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { AppServer, findBundledCodex } from "./appserver.ts";
-import { runTurn, testImages, text } from "./testkit.ts";
+import { runTurn, TEST_MODEL, testImages, text } from "./testkit.ts";
type Json = Record<string, any>;
const localRoot = join(resolve(dirname(fileURLToPath(import.meta.url)), "../.."), "local");
@@ -30,7 +30,7 @@ async function main(): Promise<void> {
const result: Json = { createdAt: startedAt, label, turns: {} };
try {
await server.initialize();
- const { thread } = await server.request<{ thread: Json }>("thread/start", { cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });
+ const { thread } = await server.request<{ thread: Json }>("thread/start", { model: TEST_MODEL, cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });
result.threadId = thread.id;
await server.request("thread/name/set", { threadId: thread.id, name: `[CAM测试] P2-1 请求结构 ${label}` });
result.turns.upload = await runTurn(server, thread.id, [text("这是三张测试图,先不用描述,只回复“收到”。"), ...["a.png", "b.png", "b-copy.png"].map((name) => ({ type: "localImage", path: file(name) }))]);
diff --git a/spike/src/p2-selftest.ts b/spike/src/p2-selftest.ts
index 22cfa3d..eb5a741 100644
--- a/spike/src/p2-selftest.ts
+++ b/spike/src/p2-selftest.ts
@@ -12,7 +12,7 @@ import { fileURLToPath } from "node:url";
import { AppServer, findBundledCodex } from "./appserver.ts";
import { codexHome } from "./codexconfig.ts";
import { readSelection, writeSelection } from "./selection.ts";
-import { runTurn, testImages, text } from "./testkit.ts";
+import { runTurn, TEST_MODEL, testImages, text } from "./testkit.ts";
import { loadThreadIndex } from "./thread-index.ts";
type Json = Record<string, any>;
@@ -40,7 +40,7 @@ async function main(): Promise<void> {
let threadId = "";
try {
await server.initialize();
- const { thread } = await server.request<{ thread: Json }>("thread/start", { cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });
+ const { thread } = await server.request<{ thread: Json }>("thread/start", { model: TEST_MODEL, cwd: workDir, approvalPolicy: "never", sandbox: "read-only", ephemeral: false });
threadId = thread.id;
result.threadId = threadId;
await server.request("thread/name/set", { threadId, name: `[CAM测试] P2 改写自测 ${label}` });
diff --git a/spike/src/panel-probe.html b/spike/src/panel-probe.html
new file mode 100644
index 0000000..00f13e2
--- /dev/null
+++ b/spike/src/panel-probe.html
@@ -0,0 +1,113 @@
+<!doctype html>
+<html lang="zh-CN">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>素材面板试验</title>
+<style>
+ :root { color-scheme: light dark; font: 13px/1.5 system-ui, "Microsoft YaHei", sans-serif; }
+ body { margin: 0; padding: 12px 14px; }
+ h1 { font-size: 15px; margin: 0 0 8px; }
+ .row { display: flex; gap: 8px; align-items: center; margin: 3px 0; }
+ .label { min-width: 9em; opacity: .75; }
+ .ok { color: #1a7f37; } .bad { color: #cf222e; }
+ pre { font-size: 11px; white-space: pre-wrap; word-break: break-all; max-height: 220px; overflow: auto; background: rgba(127,127,127,.12); padding: 8px; border-radius: 6px; }
+ button { font: inherit; padding: 3px 10px; }
+ img { width: 40px; height: 40px; border-radius: 6px; border: 1px solid rgba(127,127,127,.4); }
+</style>
+</head>
+<body>
+<h1>素材管理器 · 界面试验</h1>
+<div class="row"><span class="label">静态页面</span><b class="ok">已加载</b></div>
+<div class="row"><span class="label">缩略图(data URL)</span><img id="thumb" alt=""><b id="thumbState">加载中…</b></div>
+<div class="row"><span class="label">与 Codex 握手</span><b id="handshake">进行中…</b></div>
+<div class="row"><span class="label">回调插件服务</span><b id="ping">未开始</b></div>
+<div class="row"><span class="label">访问本机地址</span><b id="fetch">未开始</b></div>
+<div class="row"><button id="again">再回调一次</button><button id="fullscreen">请求全屏</button></div>
+<pre id="log"></pre>
+<script>
+(() => {
+ const $ = (id) => document.getElementById(id);
+ const lines = [];
+ const log = (line) => { lines.push(line); $("log").textContent = lines.join("\n"); resize(); };
+ const set = (id, text, ok) => { $(id).textContent = text; $(id).className = ok ? "ok" : "bad"; resize(); };
+ const report = { windowOpenai: typeof window.openai === "object" && window.openai ? Object.keys(window.openai) : null, received: [], csp: [] };
+ let nextId = 1;
+ const pending = new Map();
+ const post = (message) => window.parent.postMessage({ jsonrpc: "2.0", ...message }, "*");
+ const request = (method, params, timeoutMs = 10000) => new Promise((resolve, reject) => {
+ const id = nextId++;
+ const timer = setTimeout(() => { pending.delete(id); reject(new Error(method + " 超时")); }, timeoutMs);
+ pending.set(id, { resolve: (value) => { clearTimeout(timer); resolve(value); }, reject: (error) => { clearTimeout(timer); reject(error); } });
+ post({ id, method, params });
+ });
+ function resize() {
+ post({ method: "ui/notifications/size-changed", params: { width: document.documentElement.scrollWidth, height: document.documentElement.scrollHeight } });
+ }
+ document.addEventListener("securitypolicyviolation", (event) => {
+ report.csp.push(event.violatedDirective + " " + event.blockedURI);
+ log("CSP 拦截:" + event.violatedDirective + " " + event.blockedURI);
+ });
+ window.addEventListener("message", (event) => {
+ const message = event.data;
+ if (!message || typeof message !== "object") return;
+ if (message.id !== undefined && pending.has(message.id) && ("result" in message || "error" in message)) {
+ const waiter = pending.get(message.id);
+ pending.delete(message.id);
+ if (message.error) waiter.reject(new Error(JSON.stringify(message.error))); else waiter.resolve(message.result);
+ return;
+ }
+ report.received.push(message.method || "reply");
+ log("收到:" + (message.method || "reply") + " " + JSON.stringify(message.params ?? message.result ?? {}).slice(0, 160));
+ });
+ log("window.openai:" + JSON.stringify(report.windowOpenai));
+
+ // A data-URL image drawn on a canvas stands in for a thumbnail.
+ const canvas = document.createElement("canvas");
+ canvas.width = canvas.height = 40;
+ const ctx = canvas.getContext("2d");
+ ctx.fillStyle = "#fad23c"; ctx.fillRect(0, 0, 40, 40);
+ ctx.fillStyle = "#2846d2"; ctx.beginPath(); ctx.arc(20, 20, 11, 0, Math.PI * 2); ctx.fill();
+ $("thumb").onload = () => { set("thumbState", "显示正常", true); report.thumb = "ok"; };
+ $("thumb").onerror = () => { set("thumbState", "显示失败", false); report.thumb = "error"; };
+ $("thumb").src = canvas.toDataURL("image/png");
+
+ const ping = async (reason) => {
+ try {
+ const result = await request("tools/call", { name: "cam_probe_ping", arguments: { reason, report } });
+ set("ping", "成功:" + JSON.stringify(result.structuredContent ?? result.content).slice(0, 100), true);
+ } catch (error) { set("ping", "失败:" + error.message, false); }
+ };
+ $("again").onclick = () => ping("button");
+ $("fullscreen").onclick = async () => {
+ try { log("全屏:" + JSON.stringify(await request("ui/request-display-mode", { mode: "fullscreen" }))); }
+ catch (error) { log("全屏失败:" + error.message); }
+ };
+
+ (async () => {
+ try {
+ const init = await request("ui/initialize", { protocolVersion: "2026-01-26", appInfo: { name: "codex-attachment-manager-probe", version: "0.0.1" }, appCapabilities: {} });
+ const context = init.hostContext || {};
+ report.host = { protocolVersion: init.protocolVersion, hostInfo: init.hostInfo, hostCapabilities: init.hostCapabilities, contextKeys: Object.keys(context), displayMode: context.displayMode, availableDisplayModes: context.availableDisplayModes, theme: context.theme, locale: context.locale };
+ post({ method: "ui/notifications/initialized", params: {} });
+ set("handshake", "成功 " + JSON.stringify(report.host.hostInfo ?? {}).slice(0, 80), true);
+ log("握手结果:" + JSON.stringify(report.host).slice(0, 600));
+ } catch (error) {
+ report.handshakeError = error.message;
+ set("handshake", "失败:" + error.message, false);
+ }
+ try {
+ const response = await fetch("http://localhost:17891/__cam/health");
+ report.fetch = "status " + response.status;
+ set("fetch", "成功(" + response.status + ")", true);
+ } catch (error) {
+ report.fetch = String(error);
+ set("fetch", "失败:" + error.message, false);
+ }
+ await ping("load");
+ })();
+ resize();
+})();
+</script>
+</body>
+</html>
diff --git a/spike/src/panel-probe.ts b/spike/src/panel-probe.ts
new file mode 100644
index 0000000..4000669
--- /dev/null
+++ b/spike/src/panel-probe.ts
@@ -0,0 +1,84 @@
+// Purpose: P3-1 — probe what Codex Desktop offers an MCP App on Windows: whether the widget renders, finishes the
+// MCP Apps handshake, calls back into this server, shows data-URL images and reaches localhost; and when and how often
+// Codex starts this server (per thread or shared).
+// Input: MCP JSON-RPC over stdio (newline-delimited); the widget page is panel-probe.html next to this file.
+// Output: one JSON line per event in local/p3/panel-probe.jsonl (local only; no conversation content is logged).
+
+import { appendFileSync, mkdirSync, readFileSync } from "node:fs";
+import { dirname, join, resolve } from "node:path";
+import { createInterface } from "node:readline";
+import { fileURLToPath } from "node:url";
+
+type Json = Record<string, any>;
+const here = dirname(fileURLToPath(import.meta.url));
+const logDir = join(resolve(here, "../.."), "local", "p3");
+const RESOURCE = "ui://codex-attachment-manager/probe.html";
+const MIME = "text/html;profile=mcp-app";
+let calls = 0;
+
+function log(entry: Json): void {
+ mkdirSync(logDir, { recursive: true });
+ appendFileSync(join(logDir, "panel-probe.jsonl"), `${JSON.stringify({ at: new Date().toISOString(), pid: process.pid, ...entry })}\n`);
+}
+
+const send = (message: Json) => process.stdout.write(`${JSON.stringify({ jsonrpc: "2.0", ...message })}\n`);
+
+const TOOLS = [
+ {
+ name: "cam_panel_probe",
+ title: "素材面板(试验)",
+ description: "打开 Codex 素材管理器的界面试验面板。用户要求打开素材面板时调用,不需要参数。",
+ inputSchema: { type: "object", properties: {}, additionalProperties: false },
+ _meta: { ui: { resourceUri: RESOURCE }, "openai/ui": { preferredModelDisplayMode: "inline" } },
+ },
+ {
+ name: "cam_probe_ping",
+ title: "素材面板回调(试验)",
+ description: "素材面板内部使用:面板页面回调插件服务。模型不需要调用。",
+ inputSchema: { type: "object", properties: { reason: { type: "string" }, report: { type: "object" } } },
+ _meta: { ui: { resourceUri: RESOURCE, visibility: ["app"] } },
+ },
+];
+
+function handle(message: Json): void {
+ const { id, method, params = {} } = message;
+ if (method === "initialize") {
+ log({ event: "initialize", client: params.clientInfo ?? null, protocolVersion: params.protocolVersion ?? null, capabilities: params.capabilities ?? null });
+ send({ id, result: { protocolVersion: params.protocolVersion ?? "2025-06-18", capabilities: { tools: {}, resources: {} }, serverInfo: { name: "codex-attachment-manager-probe", version: "0.0.1" } } });
+ } else if (method === "tools/list") {
+ send({ id, result: { tools: TOOLS } });
+ } else if (method === "tools/call") {
+ calls++;
+ log({ event: "tools/call", name: params.name, arguments: params.arguments ?? null, meta: params._meta ?? null });
+ const meta = params._meta ?? {};
+ const threadId = meta.threadId ?? meta.thread_id ?? null;
+ if (params.name === "cam_panel_probe") {
+ send({ id, result: { content: [{ type: "text", text: "已打开素材面板(试验版)。" }], structuredContent: { pid: process.pid, threadId, calls } } });
+ } else if (params.name === "cam_probe_ping") {
+ send({ id, result: { content: [{ type: "text", text: "pong" }], structuredContent: { pid: process.pid, calls, at: new Date().toISOString() } } });
+ } else {
+ send({ id, error: { code: -32602, message: `unknown tool ${params.name}` } });
+ }
+ } else if (method === "resources/list") {
+ send({ id, result: { resources: [{ uri: RESOURCE, name: "素材面板(试验)", mimeType: MIME }] } });
+ } else if (method === "resources/templates/list") {
+ send({ id, result: { resourceTemplates: [] } });
+ } else if (method === "resources/read") {
+ log({ event: "resources/read", uri: params.uri ?? null });
+ const text = readFileSync(join(here, "panel-probe.html"), "utf8");
+ send({ id, result: { contents: [{ uri: RESOURCE, mimeType: MIME, text, _meta: { ui: { csp: { connectDomains: ["http://localhost:17891"], resourceDomains: [] }, prefersBorder: true } } }] } });
+ } else if (method === "ping") {
+ send({ id, result: {} });
+ } else if (id !== undefined) {
+ log({ event: "unsupported", method });
+ send({ id, error: { code: -32601, message: `unsupported: ${method}` } });
+ }
+}
+
+// Which process started us, and whether Codex passes thread identity in the environment (names and ids only).
+log({ event: "start", ppid: process.ppid, argv: process.argv.slice(1), env: Object.fromEntries(Object.entries(process.env).filter(([name]) => /THREAD|SESSION/i.test(name) && !/TOKEN|KEY|SECRET/i.test(name))) });
+process.stdin.on("close", () => { log({ event: "stdin-closed", calls }); process.exit(0); });
+createInterface({ input: process.stdin, crlfDelay: Infinity }).on("line", (line) => {
+ if (!line.trim()) return;
+ try { handle(JSON.parse(line)); } catch (error) { log({ event: "error", error: String(error) }); }
+});
diff --git a/spike/src/testkit.ts b/spike/src/testkit.ts
index c51584c..3abd80f 100644
--- a/spike/src/testkit.ts
+++ b/spike/src/testkit.ts
@@ -49,6 +49,9 @@ export const testImages = {
},
};
+// Tests use gpt-6-sol at low effort to save the user's tokens (AGENTS.md).
+export const TEST_MODEL = "gpt-6-sol";
+
export const text = (value: string) => ({ type: "text", text: value, text_elements: [] });
export async function runTurn(server: AppServer, threadId: string, input: Json[], timeoutMs = 20 * 60_000): Promise<Json> {