Stop paying the multimodal tax. Your AI agent keeps every image it has seen but only sends the ones it needs: lighter chats, no endless reconnecting, no starting over. Works with Codex, Claude Code, Cline, Gemini CLI, OpenCode, WorkBuddy and more. Install once, forget it.
TypeScript ai-agentsai-imageclaude-codeclaude-code-pluginclinecodexcodex-plugincomputer-usecontext-engineeringcontext-managementdeveloper-toolsgemini-cliimage-generationmcpmultimodalmultimodal-aiopencodeqwen-codetoken-optimizationvision-language-model
fix(rewrite): 仍发送的图标上编号;说明写清楚勾回的原图回到原来的位置
chipfighter41d8f49abc99
diff --git a/plugin/src/rewrite.ts b/plugin/src/rewrite.tsindex 975f59d..863275a 100644--- a/plugin/src/rewrite.ts+++ b/plugin/src/rewrite.ts@@ -5,6 +5,8 @@// v0.1-14 — in the user's language (Codex's interface language, language.ts): the Chinese wording is the one tested on// 2026-09-25; the English one says the same and is checked by the same self-test (v01-placeholder-selftest.ts --lang en).// v0.1-24 — how to ask for an image back is spelled out, for every GPT-6 model (self-tests with CAM_TEST_MODEL).+// v0.1-25 — once anything is left out, each image still sent carries its id, and the note says a checked image is back+// in its old place: GPT-6 Luna looked for it in the newest message and said it could not see it (user 2026-09-28).// Input: request items, a lookup into the thread index, the unchecked keys. Output: new items plus a metadata report.import { findImages, type ImageKind, type ImageRef } from "./images.ts";@@ -18,7 +20,7 @@ export type RewriteReport = { images: number; replaced: Replacement[]; locked: stype Words = {kind: Record<ImageKind, string>; unnamed: string; turn: (n: number) => string; thisTurn: string; newImage: string;- omitted: string; notRepeated: string; separator: string;+ omitted: string; notRepeated: string; included: string; separator: string;heading: (id: string, label: string | null, state: string, fields: string[]) => string;plain: (id: string) => string; duplicate: (id: string, where: string) => string; note: string;};@@ -36,14 +38,16 @@ const WORDS: Record<Lang, Words> = {newImage: "(新图片)",omitted: "已省略",notRepeated: "未重复提供",+ included: "已提供",separator: ",",heading: (id, label, state, fields) => `[图片 ${id}${label ? `(${label})` : ""} ${state}|${fields.join("|")}]`,plain: (id) => `原图被用户省略以节省上下文,情况见前面的“上下文管理说明”;需要重新看这张图时,在回复里写出“需要 ${id}”。`,duplicate: (id, where) => `这张图和 ${id}(${where})内容完全相同,那一条在本次请求里照常提供,请直接参考那一条。`,note: [- "上下文管理说明(来自用户安装的上下文素材管理工具):为了节省上下文,用户取消勾选了这个任务里的部分图片。后面的对话里,这些图片原来的位置被换成了“[图片 IMG-xxx 已省略…]”这样的占位文字。",+ "上下文管理说明(来自用户安装的上下文素材管理工具):为了节省上下文,用户取消勾选了这个任务里的部分图片。后面的对话里,这些图片原来的位置被换成了“[图片 IMG-xxx 已省略…]”这样的占位文字;仍在提供的图片前面标着“[图片 IMG-xxx 已提供…]”,紧接着就是原图。","这些图片在它们出现的那一轮真实存在,你当时收到并看过;你之前对它们的描述,以及据此做出的判断,都是看着原图做出的,仍然有效。不要因为现在看不到,就认为之前的回答是猜测或错误,也不要收回或道歉。",- "被问到现在能不能看到时,如实说明现在看不到(被用户省略了),并说明之前的描述仍然有效。现在看不到画面,就不要补充之前没说过的细节。需要重新看某张图时,在回复里原样写出“需要 IMG-xxx”(几张就写“需要 IMG-002、IMG-003”):工具靠这几个字认出你要的图,用户在工具里勾选后,下一条消息就会带上原图,所以不要让用户重新上传或附上图片。不要自己用工具去读取这些图片。",+ "被问到现在能不能看到某张被省略的图片时,如实说明现在看不到(被用户省略了),并说明之前的描述仍然有效。现在看不到画面,就不要补充之前没说过的细节。需要重新看某张图时,在回复里原样写出“需要 IMG-xxx”(几张就写“需要 IMG-002、IMG-003”):工具靠这几个字认出你要的图。用户重新勾选后,原图会回到它在对话里原来的位置:那里的占位文字换成“[图片 IMG-xxx 已提供…]”和原图;它不会出现在用户的新消息里,所以不要让用户重新上传或附上图片。不要自己用工具去读取这些图片。",+ "标着“[图片 IMG-xxx 已提供…]”的图片现在就在你面前,即使它之前被省略过、或者正是你要过的图:被问到它时,直接看图回答,不要说看不到。",].join("\n"),},en: {@@ -54,20 +58,23 @@ const WORDS: Record<Lang, Words> = {newImage: "(new image)",omitted: "omitted",notRepeated: "not repeated",+ included: "included",separator: ", ",heading: (id, label, state, fields) => `[Image ${id}${label ? ` (${label})` : ""} ${state} | ${fields.join(" | ")}]`,plain: (id) => `The user left out the original image to save context; see the "Context management note" above. If you need to see this image again, write "need ${id}" in your reply.`,duplicate: (id, where) => `This image is identical to ${id} (${where}), which is included in this request as usual; refer to that one.`,note: [- "Context management note (from the context asset manager the user installed): to save context, the user unchecked some images in this task. Further on in the conversation, where those images were, there is now placeholder text such as \"[Image IMG-xxx omitted…]\".",+ "Context management note (from the context asset manager the user installed): to save context, the user unchecked some images in this task. Further on in the conversation, where those images were, there is now placeholder text such as \"[Image IMG-xxx omitted…]\"; each image still included is marked \"[Image IMG-xxx included…]\", with the original image right after it.","These images really existed in the turn where they appeared, and you received and looked at them then; what you said about them before, and the judgments you based on it, were made looking at the original images and remain valid. Do not treat your earlier answers as guesses or mistakes because you cannot see the images now, and do not take them back or apologize.",- "If asked whether you can see them now, say truthfully that you cannot see them now (the user left them out), and that your earlier descriptions remain valid. Since you cannot see them now, do not add details you did not mention before. If you need to look at an image again, write exactly \"need IMG-xxx\" in your reply (for several: \"need IMG-002, IMG-003\"): the tool finds the images you want by these words, and once the user checks them, the next message brings back the originals, so do not ask the user to upload or attach them again. Do not read these images yourself with tools.",+ "If asked whether you can see an omitted image now, say truthfully that you cannot see it now (the user left it out), and that your earlier descriptions remain valid. Since you cannot see it now, do not add details you did not mention before. If you need to look at an image again, write exactly \"need IMG-xxx\" in your reply (for several: \"need IMG-002, IMG-003\"): the tool finds the images you want by these words. Once the user checks an image again, its original is back in its old place in the conversation: the placeholder there becomes \"[Image IMG-xxx included…]\" followed by the original image. It does not come in the user's new message, so do not ask the user to upload or attach it again. Do not read these images yourself with tools.",+ "An image marked \"[Image IMG-xxx included…]\" is in front of you now, even if it was omitted before or is one you asked for: when asked about it, look at it and answer from it, and do not say you cannot see it.",].join("\n"),},};// The Chinese note, tested on 2026-09-25; v0.1-24 (2026-09-28) spells out how to ask for an image back ("需要 IMG-xxx",-// several at once, no re-uploading), after GPT-6 Luna asked the user to "reattach" images instead.+// several at once, no re-uploading), after GPT-6 Luna asked the user to "reattach" images instead; v0.1-25 says where a+// checked image comes back (its old place, marked "已提供"), after Luna looked for it in the newest message instead.export const OMISSION_NOTE = WORDS.zh.note;export const omissionNote = (lang: Lang) => WORDS[lang].note;@@ -89,6 +96,12 @@ export function duplicatePlaceholder(image: Described, same: Described, lang: Lareturn `${heading(words, image, words.notRepeated)}\n${words.duplicate(same.id, where)}`;}+// Put right before an image that is still sent, so the model can tell which id it is: Codex's own labels ("[Image #2]")+// start again in every message.+export function includedLabel(image: Described, lang: Lang = "zh"): string {+ return heading(WORDS[lang], image, WORDS[lang].included);+}+export function rewriteItems(items: Json[],describe: (ref: ImageRef) => Described | undefined,@@ -110,7 +123,9 @@ export function rewriteItems(const pixels = pixelHash(a);return pixels !== null && pixels === pixelHash(b);};- const edits = new Map<number, Array<{ from: number; to: number; text: string }>>();+ // Each edit puts one text part at `at`, in place of `remove` parts (none for a label).+ const edits = new Map<number, Array<{ at: number; remove: number; text: string }>>();+ const edit = (item: number, at: number, remove: number, text: string) => edits.set(item, [...(edits.get(item) ?? []), { at, remove, text }]);let firstPlain: number | null = null;for (const ref of drop) {const image = describe(ref)!;@@ -119,16 +134,20 @@ export function rewriteItems(const text = copyImage ? duplicatePlaceholder(image, copyImage, lang) : plainPlaceholder(image, lang);if (!copyImage && firstPlain === null) firstPlain = ref.item;report.replaced.push({ id: image.id, key: ref.key, kind: ref.kind, mode: copyImage ? "duplicate" : "plain", sameAs: copyImage?.id ?? null, base64Chars: ref.base64Chars });- const list = edits.get(ref.item) ?? [];- list.push({ from: ref.openTag ?? ref.part!, to: ref.closeTag ?? ref.part!, text });- edits.set(ref.item, list);+ const at = ref.openTag ?? ref.part!;+ edit(ref.item, at, (ref.closeTag ?? ref.part!) - at + 1, text);+ }+ // v0.1-25: a checked image sits in its old place among placeholders, so every image still sent is marked with its id.+ for (const ref of sent) {+ const image = ref.replaceable && ref.part !== null ? describe(ref) : undefined;+ if (image) edit(ref.item, ref.openTag ?? ref.part!, 0, includedLabel(image, lang));}const next = items.map((item, index) => {const list = edits.get(index);if (!list) return item;const field = item.type === "message" ? "content" : "output";const parts: Json[] = [...item[field]];- for (const edit of list.sort((a, b) => b.from - a.from)) parts.splice(edit.from, edit.to - edit.from + 1, { type: "input_text", text: edit.text });+ for (const { at, remove, text } of list.sort((a, b) => b.at - a.at)) parts.splice(at, remove, { type: "input_text", text });return { ...item, [field]: parts };});// Only plain placeholders need the note (a duplicate's content is still in view). It goes before the user messagediff --git a/spike/src/rewrite.test.ts b/spike/src/rewrite.test.tsindex 432e547..d5779b8 100644--- a/spike/src/rewrite.test.ts+++ b/spike/src/rewrite.test.ts@@ -1,11 +1,12 @@// Purpose: P2 — the rewrite rules: only unchecked images change; duplicate vs plain placeholders; locked results.// v0.1 — one developer message explains omitted images, so the model does not take back earlier answers.+// v0.1-25 — once anything is left out, each image still sent is marked with its id.// Input: synthetic request items; output: Node test assertions only.import assert from "node:assert/strict";import test from "node:test";import { findImages, type ImageRef } from "../../plugin/src/images.ts";-import { OMISSION_NOTE, rewriteItems, type Described } from "../../plugin/src/rewrite.ts";+import { OMISSION_NOTE, omissionNote, rewriteItems, type Described } from "../../plugin/src/rewrite.ts";import { png } from "./testkit.ts";const url = (bytes: Buffer) => `data:image/png;base64,${bytes.toString("base64")}`;@@ -52,7 +53,8 @@ test("an unchecked upload becomes one plain placeholder in place of Codex's tag,const { items: out, report } = rewriteItems(input, describe, keys("IMG-001", "IMG-004"), noPixels);// out[1] is the inserted explanation (next test); the user message follows it.const parts = texts(out[2]);- assert.equal(parts.length, 1 + 1 + 3 + 3);+ // The text, IMG-001's placeholder, then IMG-002 and IMG-003, each with its label (a later test) and Codex's 3 parts.+ assert.equal(parts.length, 1 + 1 + 4 + 4);assert.match(parts[1], /^\[图片 IMG-001(\[Image #1\]) 已省略|a\.png|用户上传|第 1 轮|4×4\]\n原图被用户省略以节省上下文,情况见前面的“上下文管理说明”/);assert.match(parts[1], /需要 IMG-001/);assert.doesNotMatch(parts[1], /C:\\work/, "no local path in the placeholder");@@ -74,6 +76,35 @@ test("one developer message explains omitted images: seen when they appeared, eaassert.match(OMISSION_NOTE, /需要 IMG-xxx/);});+test("once anything is left out, each image still sent carries its id right before it", () => {+ const input = items();+ const { describe, keys } = index(input);+ const { items: out } = rewriteItems(input, describe, keys("IMG-001", "IMG-004"), noPixels);+ const parts = texts(out[2]);+ // Before Codex's own tag, whose label ("[Image #2]") starts again in every message.+ assert.equal(parts[2], "[图片 IMG-002([Image #2]) 已提供|b.png|用户上传|第 1 轮|4×4]");+ assert.match(parts[3], /^<image name=\[Image #2\]/);+ assert.equal(parts[4], "input_image");+ assert.equal(parts[6], "[图片 IMG-003([Image #3]) 已提供|b copy.png|用户上传|第 1 轮|4×4]");+ const english = rewriteItems(input, describe, keys("IMG-001", "IMG-004"), noPixels, "en").items;+ assert.equal(texts(english[2])[2], "[Image IMG-002 ([Image #2]) included | b.png | uploaded by the user | turn 1 | 4×4]");+ // A tool's image is marked the same way; a hosted generation result cannot carry text and stays as it is.+ const tool = rewriteItems(input, describe, keys("IMG-002", "IMG-003"), noPixels).items;+ assert.deepEqual(texts(tool[5]), ["Script completed", "[图片 IMG-004 已提供|a.png|工具查看的图片|第 1 轮|4×4]", "input_image"]);+ assert.equal(tool[6], input[5]);+});++test("the explanation says a checked image comes back in its old place, marked, not in the user's new message", () => {+ assert.match(OMISSION_NOTE, /仍在提供的图片前面标着“\[图片 IMG-xxx 已提供…\]”/);+ assert.match(OMISSION_NOTE, /原图会回到它在对话里原来的位置/);+ assert.match(OMISSION_NOTE, /不会出现在用户的新消息里/);+ assert.match(OMISSION_NOTE, /标着“\[图片 IMG-xxx 已提供…\]”的图片现在就在你面前.*不要说看不到/);+ assert.match(omissionNote("en"), /each image still included is marked "\[Image IMG-xxx included…\]"/);+ assert.match(omissionNote("en"), /back in its old place in the conversation/);+ assert.match(omissionNote("en"), /does not come in the user's new message/);+ assert.match(omissionNote("en"), /marked "\[Image IMG-xxx included…\]" is in front of you now.*do not say you cannot see it/);+});+test("the explanation never splits a tool call from its output, and is left out when only copies are omitted", () => {const input = items();const { describe, keys } = index(input);@@ -91,7 +122,7 @@ test("an unchecked copy points at the identical image that is still sent, in eitconst input = items();const { describe, keys } = index(input);const later = rewriteItems(input, describe, keys("IMG-003"), noPixels);- assert.match(texts(later.items[1])[7], /^\[图片 IMG-003(\[Image #3\]) 未重复提供|b copy\.png|用户上传|第 1 轮|4×4\]\n这张图和 IMG-002(b\.png,第 1 轮)内容完全相同/);+ assert.match(texts(later.items[1])[9], /^\[图片 IMG-003(\[Image #3\]) 未重复提供|b copy\.png|用户上传|第 1 轮|4×4\]\n这张图和 IMG-002(b\.png,第 1 轮)内容完全相同/);assert.deepEqual(modes(later.report), [["IMG-003", "duplicate", "IMG-002"]]);assert.deepEqual(modes(rewriteItems(input, describe, keys("IMG-001"), noPixels).report), [["IMG-001", "duplicate", "IMG-004"]]);});