A local-first multi-agent app development platform, built on PenguinHarness
fix(core): 这一轮输入自己就超过四张图时也裁,重放的卡死会话才真能继续
lmliheng20dde0d6edc9
diff --git a/changelog/unreleased/2026-10-23-image-budget.md b/changelog/unreleased/2026-10-23-image-budget.mdindex fdadae08..502ccb86 100644--- a/changelog/unreleased/2026-10-23-image-budget.md+++ b/changelog/unreleased/2026-10-23-image-budget.md@@ -29,12 +29,19 @@ conversation or its Trace.`getHistory` returns are never mutated in place). A tool result's images are trimmed inside itsown `images` array with the placeholder appended to its text. When the new input alone fillsthe budget, every replayed image becomes a placeholder.+- **The input is trimmed too when it overflows by itself.** Five pictures read in one turn are+ merged into a *single* input message, and an interrupted turn is resent as new input when the+ Session continues — trimming only the history would leave that request carrying five and+ refused again. The input is now capped first (the newest four stay, older ones become the+ placeholder), the history takes what is left of the budget, and `fitReplayedImages` hands back+ the message to send instead of mutating the caller's.- **The Session model carries it** (`core/src/agent.ts`): `buildLLM` looks the limit up in thecatalog and passes it as `GenerativeModelConfig.maxImages`. A context rebuilt after compaction,or for a switched model, goes through the same constructor.- Tests: `core/test/image-budget.test.ts` (counting, per-message trimming, tool-result trimming,- no mutation of the originals, a zero budget, the untouched path with no limit, and the wiring- that counts the new input) and `core/test/model-catalog.test.ts` (the four rows carry 4; no+ no mutation of the originals, a zero budget, the untouched path with no limit, the wiring that+ counts the new input, an input that overflows by itself keeping only the newest four, and the+ placeholder naming the seller's cap rather than the leftover budget) and `core/test/model-catalog.test.ts` (the four rows carry 4; noother model and no id outside the catalog does).- Docs: the Abliteration section of `models.zh.md` / `models.en.md` states the limit and whatthe trimming does.diff --git a/changelog/unreleased/2026-10-23-image-budget.zh.md b/changelog/unreleased/2026-10-23-image-budget.zh.mdindex 9c985051..292ef8c8 100644--- a/changelog/unreleased/2026-10-23-image-budget.zh.md+++ b/changelog/unreleased/2026-10-23-image-budget.zh.md@@ -21,11 +21,14 @@ Abliteration 一次请求最多收四张图,超过就把整条请求拒掉(``fitReplayedImages`:先把本次新输入的图算进预算,剩下的额度从最新往前给历史;放不下时用 `capHistoryImages`重建一份历史交给 AgentHub 的 `setHistory`(`getHistory` 交出来的对象不原地改动)。工具结果里的图按它自己的`images` 数组裁,占位那行追加到同一段文字后面。新输入自己就占满额度时,历史里的图全部换成占位。+- **输入自己就超上限时也裁**:一轮里读五张图,这一轮的五张会被合并进**同一条**输入消息,而打断后重放的一轮又是+ 作为新输入重新送出的 —— 只裁历史的话,那条请求仍然带着五张、照样 400。现在输入先按上限裁(同样留最新四张、+ 旧的换成占位),历史再从剩下的额度里分;`fitReplayedImages` 因此把要送出的那条消息交回去,不改调用方的对象。- **会话模型带上它**(`core/src/agent.ts`):`buildLLM` 从目录查出上限,作为 `GenerativeModelConfig.maxImages`传下去;压缩后重建、或切换模型后的新上下文都走同一个构造点。- 测试:`core/test/image-budget.test.ts`(计数、按消息裁剪、工具结果裁剪、不改原对象、零预算、没有上限时原样- 送出,以及把新输入的图算进预算的接线)与 `core/test/model-catalog.test.ts`(四条条目带 4,其余模型与目录外- 的 id 都没有)。+ 送出、把新输入的图算进预算的接线、输入自己超上限时只送最新的四张,以及占位写的是卖家的上限而不是剩余额度)+ 与 `core/test/model-catalog.test.ts`(四条条目带 4,其余模型与目录外的 id 都没有)。- 文档:`models.zh.md` / `models.en.md` 的 Abliteration 一节写清这条上限与裁剪行为。## 兼容性diff --git a/packages/core/src/llm/generative-model.ts b/packages/core/src/llm/generative-model.tsindex b29ebfd0..fb80fbd1 100644--- a/packages/core/src/llm/generative-model.ts+++ b/packages/core/src/llm/generative-model.ts@@ -310,14 +310,21 @@ function capMessageImages(msg: UniMessage, budget: number, limit: number): UniMe}/**- * Trims a replayed history to at most `limit` images in aggregate, newest kept (see- * {@link capMessageImages}). Returns a new array of the same length whose untouched messages- * are the very objects it was handed.+ * Trims a replayed history so it carries at most `budget` images in aggregate, newest kept (see+ * {@link capMessageImages}). `limit` is the seller's per-request cap and only names itself in+ * the placeholder — a request whose own input already used part of the budget keeps fewer+ * replayed images than the cap without misreporting the cap as smaller.+ * Returns a new array of the same length whose untouched messages are the very objects it was+ * handed.*/-export function capHistoryImages(history: UniMessage[], limit: number): UniMessage[] {+export function capHistoryImages(+ history: UniMessage[],+ budget: number,+ limit: number = budget,+): UniMessage[] {const counts = history.map(countMessageImages);const capped = new Array<UniMessage>(history.length);- let left = limit;+ let left = budget;for (let i = history.length - 1; i >= 0; i -= 1) {const msg = history[i]!;const count = counts[i]!;@@ -1311,10 +1318,10 @@ export class GenerativeModel implements LLMInterface {await this.refreshApiKey();// The seller counts images per request, not per message: a conversation that read five// pictures would fail every later turn with 400 media_count_exceeded, so the oldest- // replayed ones are dropped from THIS request (the history and Trace keep them).- this.fitReplayedImages(uniMessage);+ // replayed ones — and, when the input alone overflows, the input's own oldest ones — are+ // dropped from THIS request (the history and Trace keep them).const it = this.openStream(- uniMessage,+ this.fitReplayedImages(uniMessage),ac.signal,this.requestConfig(params.thinkingLevel, params.newMessages),)[Symbol.asyncIterator]();@@ -1490,24 +1497,32 @@ export class GenerativeModel implements LLMInterface {}/**- * Trims the replayed history so this request carries at most `maxImages` images in all: the- * new input's images are counted first (they are what the user is asking about now), the- * history keeps the rest, newest first — see capHistoryImages for the per-message rule.+ * Trims a request so it carries at most `maxImages` images in all, newest kept.+ *+ * The **input** is trimmed first and counts first: an interrupted turn replayed from Trace is+ * resent as carry-over input, and that input may itself hold more images than the seller+ * accepts (the exact case that used to fail every turn of a Session forever). The rest of the+ * budget goes to the replayed history, newest first — see capHistoryImages for the+ * per-message rule.*- * Nothing happens without a configured cap, or when the history already fits. When it does- * not, the client's history is replaced through setHistory rather than mutated: the objects- * getHistory hands out are the client's own. A conversation's Trace and stored history are- * untouched — this decides what one request carries, not what the conversation is.+ * Nothing happens without a configured cap, or when everything already fits. The input comes+ * back trimmed (the very object when it fits) and the caller sends that; the client's history+ * is replaced through setHistory rather than mutated, since the objects getHistory hands out+ * are the client's own. A conversation's Trace and stored history are untouched — this+ * decides what one request carries, not what the conversation is.*/- private fitReplayedImages(input: UniMessage): void {+ private fitReplayedImages(input: UniMessage): UniMessage {const limit = this.maxImages;- if (limit === undefined) return;- const budget = Math.max(0, limit - countMessageImages(input));+ if (limit === undefined) return input;+ const carried =+ countMessageImages(input) > limit ? capMessageImages(input, limit, limit) : input;+ const budget = Math.max(0, limit - countMessageImages(carried));const history = this.client.getHistory();- if (history.length === 0) return;+ if (history.length === 0) return carried;const total = history.reduce((sum, msg) => sum + countMessageImages(msg), 0);- if (total <= budget) return;- this.client.setHistory(capHistoryImages(history, budget));+ if (total <= budget) return carried;+ this.client.setHistory(capHistoryImages(history, budget, limit));+ return carried;}/**diff --git a/packages/core/test/image-budget.test.ts b/packages/core/test/image-budget.test.tsindex 40af3597..8d28c414 100644--- a/packages/core/test/image-budget.test.ts+++ b/packages/core/test/image-budget.test.ts@@ -8,18 +8,27 @@* What is checked here: the pure trimming (newest images kept, older ones replaced by a text* note, tool results trimmed inside their own image list) and the wiring — a configured* `maxImages` reaches the client's history before the request opens, and the new input's own- * images are counted against the same budget.+ * images are counted against the same budget. One turn can overflow on its own, too: the input+ * is a merge of everything the turn carries (a resumed turn replays its tool results), so the+ * input itself is trimmed to the cap before the history gets what is left.*/import { describe, expect, it, vi } from "vitest";import type { UniEvent, UniMessage } from "@prismshadow/agenthub";import type { OmniMessage } from "../src/omnimessage/index.js";+// The class itself is imported below, after `vi.mock` has had its say; this is the type-only+// half so annotations do not have to spell `InstanceType<typeof …>`.+import type { GenerativeModel as GenerativeModelClass } from "../src/llm/generative-model.js";-const state = vi.hoisted(() => ({ clients: [] as { history: UniMessage[] }[] }));+const state = vi.hoisted(() => ({+ clients: [] as { history: UniMessage[]; requests: UniMessage[] }[],+}));vi.mock("@prismshadow/agenthub", async (importOriginal) => {const mod = await importOriginal<typeof import("@prismshadow/agenthub")>();class FakeAutoLLMClient {history: UniMessage[] = [];+ /** Every message this client was asked to send, in order — the request side of the budget. */+ requests: UniMessage[] = [];constructor(_options: Record<string, unknown>) {state.clients.push(this);}@@ -29,8 +38,9 @@ vi.mock("@prismshadow/agenthub", async (importOriginal) => {setHistory(history: UniMessage[]): void {this.history = history;}- async *streamingResponseStateful(): AsyncIterable<UniEvent> {+ async *streamingResponseStateful(options: { message: UniMessage }): AsyncIterable<UniEvent> {// Nothing upstream: the tests only care about what the request would carry.+ this.requests.push(options.message);}appendHistory(message: UniMessage): void {this.history = [...this.history, message];@@ -134,10 +144,20 @@ describe("capHistoryImages", () => {expect(tool.images).toBeUndefined();expect(tool.text).toContain(imageOmittedNote(0));});++ it("names the seller's cap, not the leftover budget, when the input already used some", () => {+ // One image left for the history of a model that accepts four: the placeholder must not+ // tell the reader the model takes one.+ const capped = capHistoryImages([imageMessage(2)], 1, 4);+ expect(capped[0]!.content_items).toEqual([+ { type: "text", text: imageOmittedNote(4) },+ { type: "image_url", image_url: IMAGE },+ ]);+ });});describe("GenerativeModel image budget", () => {- const build = (over: { maxImages?: number } = {}): GenerativeModel =>+ const build = (over: { maxImages?: number } = {}): GenerativeModelClass =>new GenerativeModel({modelId: "abliterated-model-large-flash",apiKey: "ak-test",@@ -147,9 +167,10 @@ describe("GenerativeModel image budget", () => {...over,});- const lastClient = (): { history: UniMessage[] } => state.clients.at(-1)!;+ const lastClient = (): { history: UniMessage[]; requests: UniMessage[] } => state.clients.at(-1)!;+ const lastRequest = (): UniMessage => lastClient().requests.at(-1)!;- const run = async (model: GenerativeModel, newMessages: OmniMessage[]): Promise<void> => {+ const run = async (model: GenerativeModelClass, newMessages: OmniMessage[]): Promise<void> => {for await (const _ of model.streamGenerate({ newMessages })) {// Nothing is produced — the fake stream is empty.}@@ -182,6 +203,24 @@ describe("GenerativeModel image budget", () => {expect(imageCount(lastClient().history)).toBe(2);});+ it("trims the input itself when one turn carries more images than the cap", async () => {+ const model = build({ maxImages: 4 });+ model.setHistory([+ toolCallOutput({ toolCallId: "call_1", output: "image/png, 1 kB", images: [IMAGE] }),+ ]);+ // Five pictures in one turn: a resumed turn replays its five read_file results as one+ // merged input message, which is the shape that failed every request before.+ await run(+ model,+ Array.from({ length: 5 }, () => imageUrlMessage(IMAGE)),+ );+ const sent = lastRequest();+ expect(imageCount([sent])).toBe(4);+ expect(sent.content_items[0]).toEqual({ type: "text", text: imageOmittedNote(4) });+ // The input used the whole cap: not one replayed image goes out with it.+ expect(imageCount(lastClient().history)).toBe(0);+ });+it("sends the history untouched when no cap is configured", async () => {const model = build();model.setHistory([diff --git a/packages/docs/content/models.en.md b/packages/docs/content/models.en.mdindex f4af40a3..5b86b20d 100644--- a/packages/docs/content/models.en.md+++ b/packages/docs/content/models.en.md@@ -499,7 +499,7 @@ The preset rows carry no price. ModelScope bills for api-inference and its model- **Presets.** `abliterated-model-large-v2` (the vendor's recommended default), `abliterated-model-large`, `abliterated-model-large-flash` and `abliterated-model`. The first two take text only at a 1,000,000-token window; `flash` takes text and images at 1,048,576; `abliterated-model` takes both at 262,144.- **Protocol.** None of these ids matches a rule AgentHub routes by, so every row pins the Chat Completions client (`client_type = "openai-chat"`) and carries the endpoint on itself. The same endpoint also serves `/responses`; change the row's protocol if you prefer it. As in every gateway, a keyless row is refused rather than sent your OpenAI key.- **Prices.** Read from the service's own model list, in USD per million Tokens: input 1 or 3, output 2, 3 or 5, and cache hits read at 0.1 or 0.3. It charges no separate cache-write fee, so `cache_write` records the input rate.-- **At most four images per request.** That is the service's own limit, and a request past it is refused outright (`400 … media_count_exceeded`); because a conversation replays its images on every request, such a conversation fails for good. Adelie therefore replaces the **oldest** images with a one-line `[image omitted: …]` placeholder before each request, keeping the newest four (images in the new input count against them first). The conversation and its Trace keep every picture — only what one request carries is reduced.+- **At most four images per request.** That is the service's own limit, and a request past it is refused outright (`400 … media_count_exceeded`); because a conversation replays its images on every request, such a conversation fails for good. Adelie therefore replaces the **oldest** images with a one-line `[image omitted: …]` placeholder before each request, keeping the newest four (images in the new input count against them first; when a single turn carries five on its own, its oldest ones become placeholders the same way). The conversation and its Trace keep every picture — only what one request carries is reduced.### Preset modelsdiff --git a/packages/docs/content/models.zh.md b/packages/docs/content/models.zh.mdindex 38f586c3..2526239c 100644--- a/packages/docs/content/models.zh.md+++ b/packages/docs/content/models.zh.md@@ -498,7 +498,7 @@ OpenAI 兼容网关分组(openrouter / fireworks / siliconflow / tokendance /- **预置模型。** `abliterated-model-large-v2`(官方推荐的默认选择)、`abliterated-model-large`、`abliterated-model-large-flash` 与 `abliterated-model`。前两条是 1,000,000 Token 上下文、只收文本;`flash` 是 1,048,576 并接受图像;`abliterated-model` 是 262,144,也接受图像。- **协议。** 这些 id 没有一条能被 AgentHub 按名字认出来,所以每个条目都固定 Chat Completions 客户端(`client_type = "openai-chat"`)并把端点写在条目上;同一条端点上也有 `/responses`,要换协议在条目上改即可。和所有网关一样,没有 key 的条目会被拒绝,而不是把你的 OpenAI key 发过去。- **价格。** 取自服务自己的模型列表,单位是每百万 Token 多少美元:输入 1 或 3,输出 2、3 或 5,缓存命中的读价 0.1 或 0.3。它不额外收缓存写入费,所以 `cache_write` 记的就是输入价。-- **一次请求最多 4 张图。** 这是服务自己的限制,超过它会直接回 `400 … media_count_exceeded`,而对话里的图会被每一次请求重新带上——所以会话会一直失败。Adelie 因此在每次请求前把**最旧的**那几张换成一行 `[image omitted: …]` 文字占位,只保留最新的四张(新发出的图优先算进这四张)。对话本身和 Trace 里的图一张不少,改的只是这一次请求带过去的量。+- **一次请求最多 4 张图。** 这是服务自己的限制,超过它会直接回 `400 … media_count_exceeded`,而对话里的图会被每一次请求重新带上——所以会话会一直失败。Adelie 因此在每次请求前把**最旧的**那几张换成一行 `[image omitted: …]` 文字占位,只保留最新的四张(新发出的图优先算进这四张;一轮自己就带了五张时,这一次输入里的旧图也一样换成占位)。对话本身和 Trace 里的图一张不少,改的只是这一次请求带过去的量。### 预置模型