Skip to content

A local plugin for image-heavy chats in Codex, Claude Code and other agents: earlier images stop going out with every message, and the model fetches the one it needs by its ID, also after compaction.

fix(rewrite): Claude 的特征行说明不再紧跟工具结果;说明讲清用途,评价时不表态

chipfighter2d79fbdacf72
diff --git a/plugin/src/claude-rewrite.ts b/plugin/src/claude-rewrite.ts
index a1a7e4b..13e00da 100644
--- a/plugin/src/claude-rewrite.ts
+++ b/plugin/src/claude-rewrite.ts
@@ -9,8 +9,9 @@
// — thinking blocks included — is passed on unchanged.
// Matching to the index (claude-index.ts): an image inside a tool result by its tool_use_id; pasted images by
// content, a request message to the transcript record(s) holding the same images in the same order.
-// v0.8: the feature line note follows the message holding the first image (a system message, as the other notes), and
-// every image sent gets its id, also the ones new in this turn; placeholders carry what the model noted.
+// v0.8: the feature line note follows the user's own message of the turn holding the first image (a system message, as
+// the other notes, but never right after a tool result), and every image sent gets its id, also the ones new in this
+// turn; placeholders carry what the model noted.
// Input: the request's messages, the session's index, the keys to leave out. Output: new messages plus a report.
import type { ClaudeIndex, ClaudeImage } from "./claude-index.ts";
@@ -190,15 +191,31 @@ export function rewriteMessages(
noteAfter(next, msg, `${msg === holding[0] && !noted ? `${claudeNote(lang, auto !== undefined)}\n` : ""}${hereNote(plainBy.get(msg)!, lang, auto !== undefined)}`);
}
if (holding.length && !noted) report.noteAt = holding[0] + 1;
- // v0.8: after the message holding the first image, at or before the others; joined with a note already there.
+ // v0.8: after the user's own message of the turn holding the first image, at or before the others; joined with a
+ // note already there.
if (first) {
- const added = noteAfter(next, first.msg, featureNote(lang));
- report.featureAt = first.msg + 1;
- if (added && report.noteAt !== undefined && report.noteAt > first.msg) report.noteAt += 1;
+ const at = promptOf(messages, first.msg);
+ const added = noteAfter(next, at, featureNote(lang));
+ report.featureAt = at + 1;
+ if (added && report.noteAt !== undefined && report.noteAt > at) report.noteAt += 1;
}
return { messages: next, report };
}
+// The user's own message of the turn holding message `msg` (its prompt, or the summary after a compaction). Sonnet 5.5
+// and Haiku 5.5 may take a system message right after a tool result for part of it and ignore it (Anthropic's page on
+// mid-conversation system messages; desktop test 2026-10-19: Haiku called the feature line note something the Read
+// tool returned, did not follow it and told the user). Without such a message right before an assistant's, `msg`.
+const holdsToolResult = (message: Json) => Array.isArray(message?.content) && message.content.some((block: Json) => block?.type === "tool_result");
+function promptOf(messages: Json[], msg: number): number {
+ for (let at = msg; at >= 0; at--) {
+ const message = messages[at];
+ if (message?.role !== "user" || holdsToolResult(message)) continue;
+ return messages[at + 1]?.role === "user" ? msg : at;
+ }
+ return msg;
+}
+
// A note goes right after a user message, the only place Anthropic allows a system message; when Claude Code put a
// system message there already, the note joins it, since two may not follow each other. True when a message was added.
function noteAfter(messages: Json[], msg: number, text: string): boolean {
diff --git a/plugin/src/rewrite.ts b/plugin/src/rewrite.ts
index 337e2eb..8d8f2f7 100644
--- a/plugin/src/rewrite.ts
+++ b/plugin/src/rewrite.ts
@@ -116,15 +116,17 @@ const WORDS: Record<Lang, Words> = {
unavailable: (id) => `${id} 的原图读不出来。`,
off: "这个任务没有开自动选图:被省略的图片要由用户勾选。需要时在回复里写“需要 IMG-xxx”。",
},
- // v0.8, the wording the user approved on 2026-10-19 (docs/v0.8/plan.md).
+ // v0.8, the wording the user approved on 2026-10-19 (docs/v0.8/plan.md). Revised the same day: it says why the user
+ // installed the tool and no longer that the user "does not know" (Haiku 5.5 read that as an injection), and that a
+ // rating needs no "记下了" (sol said so).
featureNote: [
- "特征行说明(来自用户安装的上下文素材管理工具):这个任务里的每张图都有编号,写在图前面的“[图片 IMG-xxx …]”里。为了以后(包括对话压缩之后)还能认出每张图,请在每一轮最后那条回复的末尾(不要写在中途调用工具时说的话里),给这一轮新出现的每张图写一行特征,格式是:",
+ "特征行说明(来自用户安装的上下文素材管理工具):这个任务里的每张图都有编号,写在图前面的“[图片 IMG-xxx …]”里。用户装这个工具,是为了让你以后(包括对话压缩之后)还能认出每张图、记得用户对它们的评价。所以请在每一轮最后那条回复的末尾(不要写在中途调用工具时说的话里),给这一轮新出现的每张图写一行特征,格式是:",
"[//]: # (cam IMG-015 场景|看得见的特征)",
"- 先写场景,再写看得见的特征(景别、主体、角度、光线、颜色、构图),一共 30 字以内,例如“[//]: # (cam IMG-015 深蓝雨夜|近景,正脸,霓虹反光)”。",
"- 同一个场景沿用你之前用过的叫法。",
"- 只写看得见的东西,不写“好看”“像谁”这类判断。每张图只写一次,以后不改。用户上传的、你生成的、你查看的、工具返回的图都要写;用 cam_view_image 取回的图和没有编号的图不用写。",
"- 用户评价某张图时(包括隔了几轮才说),先确定是哪一张,再照用户的原话记一行,15 字以内,例如“[//]: # (cam IMG-003 评价:太暗)”。拿不准是哪一张就不记。",
- "- 这些行和正文之间空一行,一张图写一行;括号里不要用半角括号。用户看不到这些行,所以正文里不要提到它们,也不要说你在记录。",
+ "- 这些行和正文之间空一行,一张图写一行;括号里不要用半角括号。这些行只给这个工具读,用户在界面上看不到。正文照常回答,不用提到这些行;用户评价图片时,也不用回应记录这件事,比如“记下了”“我会记住”“之后会参考”。",
"- 以后被省略的图,占位里会带着你写的特征和记下的评价,对话压缩之后也还在。要选图时先看这些;拿不准时,先把原图要回来看(怎么要,见“上下文管理说明”),或者问用户是哪一张。",
].join("\n"),
notes: (line, ratings) => [
@@ -182,13 +184,13 @@ const WORDS: Record<Lang, Words> = {
off: "Automatic image selection is off in this task: omitted images come back only when the user checks them. If you need one, write \"need IMG-xxx\" in your reply.",
},
featureNote: [
- "Feature line note (from the context asset manager the user installed): every image in this task has an ID, written in the \"[Image IMG-xxx …]\" line before it. So that each image can still be told apart later, also after the conversation is compacted, end the last reply of every turn (not what you say between tool calls) with one feature line for each image that is new in this turn, in this form:",
+ "Feature line note (from the context asset manager the user installed): every image in this task has an ID, written in the \"[Image IMG-xxx …]\" line before it. The user installed this tool so that you can still tell the images apart later, also after the conversation is compacted, and remember what the user said about each. So end the last reply of every turn (not what you say between tool calls) with one feature line for each image that is new in this turn, in this form:",
"[//]: # (cam IMG-015 scene | visible features)",
"- First the scene, then what can be seen (shot size, subject, angle, light, colors, composition), about 15 words at most, for example \"[//]: # (cam IMG-015 rainy blue night | close-up, facing the camera, neon reflections)\".",
"- For a scene you named before, reuse that name.",
"- Only what can be seen; no judgments such as \"beautiful\" or \"looks like someone\". One line per image, written once and never changed. Write one for the images the user uploaded, the ones you generated or viewed, and those in tool results; not for images you fetched with cam_view_image or images without an ID.",
"- When the user rates an image, even turns later, first make sure which one it is, then note the user's own words in a line of about 8 words at most, for example \"[//]: # (cam IMG-003 rated: too dark)\". If you are not sure which image it is, do not note it.",
- "- Leave a blank line between these lines and your reply, one image per line, and no parentheses inside them. The user cannot see these lines, so do not mention them in your reply or say that you are noting anything.",
+ "- Leave a blank line between these lines and your reply, one image per line, and no parentheses inside them. These lines are for this tool only; the user does not see them on screen. Answer as usual and do not mention these lines; when the user rates an image, do not acknowledge the noting either, such as \"noted\", \"I'll keep that in mind\" or \"I'll refer to this later\".",
"- Images left out later carry your feature line and the ratings you noted in their placeholders, also after a compaction. Check them when you pick an image; when unsure, get the original back first (the \"Context management note\" says how) or ask the user which one.",
].join("\n"),
notes: (line, ratings) => [
diff --git a/spike/src/notes.test.ts b/spike/src/notes.test.ts
index afd8119..f151206 100644
--- a/spike/src/notes.test.ts
+++ b/spike/src/notes.test.ts
@@ -154,6 +154,29 @@ test("Claude: the feature line note follows the message holding the first image,
assert.deepEqual([some.report.featureAt, some.report.noteAt], [1, 4]);
});
+// 2026-10-19: right after a tool result, Haiku 5.5 took the note for part of what the tool returned and refused it.
+test("Claude: when the first image came from a tool, the feature line note follows the user's message of that turn", () => {
+ const t = transcript();
+ t.prompt("Read a.png");
+ t.assistant({ type: "tool_use", id: "toolu_1", name: "Read", input: { file_path: "C:\\work\\a.png" } });
+ t.toolResult("toolu_1", [image(pngA)]);
+ const index = buildClaudeIndex("s", t.records);
+ const messages = [
+ { role: "user", content: [text("Read a.png")] },
+ { role: "system", content: "SessionStart hook additional context" },
+ { role: "assistant", content: [{ type: "tool_use", id: "toolu_1", name: "Read", input: { file_path: "C:\\work\\a.png" } }] },
+ { role: "user", content: [{ type: "tool_result", tool_use_id: "toolu_1", content: [image(pngA)] }] },
+ ];
+ const out = rewriteMessages(messages, index, new Set(), () => null, { features: true });
+ assert.deepEqual(out.messages.map((message) => message.role), ["user", "system", "assistant", "user"]);
+ assert.deepEqual(out.messages[1].content.map((block: Record<string, any>) => block.text), [featureNote("zh"), "SessionStart hook additional context"]);
+ assert.match(out.messages[3].content[0].content[0].text, /^\[图片 IMG-001 已提供|a\.png|工具查看的图片|第 1 轮|8×8\]$/);
+ assert.deepEqual([out.report.featureAt, out.report.labeled], [1, 1]);
+ // No message of the user's before it (a request cut short): after the image's own message, as before.
+ const bare = rewriteMessages(messages.slice(2), index, new Set(), () => null, { features: true });
+ assert.deepEqual([bare.messages.map((message) => message.role), bare.report.featureAt], [["assistant", "user", "system"], 2]);
+});
+
test("a subagent's model writes no feature lines; the user's own task does, also before its rollout's first line", () => {
const sessions = mkdtempSync(join(tmpdir(), "cam-notes-sessions-"));
const day = join(sessions, "2026", "10", "19");