Skip to content

A local-first multi-agent app development platform, built on PenguinHarness

fix(core): 这一轮输入自己就超过四张图时也裁,重放的卡死会话才真能继续

lmliheng20dde0d6edc9
diff --git a/changelog/unreleased/2026-10-23-image-budget.md b/changelog/unreleased/2026-10-23-image-budget.md
index fdadae08..502ccb86 100644
--- a/changelog/unreleased/2026-10-23-image-budget.md
+++ b/changelog/unreleased/2026-10-23-image-budget.md
@@ -29,12 +29,19 @@ conversation or its Trace.
`getHistory` returns are never mutated in place). A tool result's images are trimmed inside its
own `images` array with the placeholder appended to its text. When the new input alone fills
the budget, every replayed image becomes a placeholder.
+- **The input is trimmed too when it overflows by itself.** Five pictures read in one turn are
+ merged into a *single* input message, and an interrupted turn is resent as new input when the
+ Session continues — trimming only the history would leave that request carrying five and
+ refused again. The input is now capped first (the newest four stay, older ones become the
+ placeholder), the history takes what is left of the budget, and `fitReplayedImages` hands back
+ the message to send instead of mutating the caller's.
- **The Session model carries it** (`core/src/agent.ts`): `buildLLM` looks the limit up in the
catalog and passes it as `GenerativeModelConfig.maxImages`. A context rebuilt after compaction,
or for a switched model, goes through the same constructor.
- Tests: `core/test/image-budget.test.ts` (counting, per-message trimming, tool-result trimming,
- no mutation of the originals, a zero budget, the untouched path with no limit, and the wiring
- that counts the new input) and `core/test/model-catalog.test.ts` (the four rows carry 4; no
+ no mutation of the originals, a zero budget, the untouched path with no limit, the wiring that
+ counts the new input, an input that overflows by itself keeping only the newest four, and the
+ placeholder naming the seller's cap rather than the leftover budget) and `core/test/model-catalog.test.ts` (the four rows carry 4; no
other model and no id outside the catalog does).
- Docs: the Abliteration section of `models.zh.md` / `models.en.md` states the limit and what
the trimming does.
diff --git a/changelog/unreleased/2026-10-23-image-budget.zh.md b/changelog/unreleased/2026-10-23-image-budget.zh.md
index 9c985051..292ef8c8 100644
--- a/changelog/unreleased/2026-10-23-image-budget.zh.md
+++ b/changelog/unreleased/2026-10-23-image-budget.zh.md
@@ -21,11 +21,14 @@ Abliteration 一次请求最多收四张图,超过就把整条请求拒掉(`
`fitReplayedImages`:先把本次新输入的图算进预算,剩下的额度从最新往前给历史;放不下时用 `capHistoryImages`
重建一份历史交给 AgentHub 的 `setHistory`(`getHistory` 交出来的对象不原地改动)。工具结果里的图按它自己的
`images` 数组裁,占位那行追加到同一段文字后面。新输入自己就占满额度时,历史里的图全部换成占位。
+- **输入自己就超上限时也裁**:一轮里读五张图,这一轮的五张会被合并进**同一条**输入消息,而打断后重放的一轮又是
+ 作为新输入重新送出的 —— 只裁历史的话,那条请求仍然带着五张、照样 400。现在输入先按上限裁(同样留最新四张、
+ 旧的换成占位),历史再从剩下的额度里分;`fitReplayedImages` 因此把要送出的那条消息交回去,不改调用方的对象。
- **会话模型带上它**(`core/src/agent.ts`):`buildLLM` 从目录查出上限,作为 `GenerativeModelConfig.maxImages`
传下去;压缩后重建、或切换模型后的新上下文都走同一个构造点。
- 测试:`core/test/image-budget.test.ts`(计数、按消息裁剪、工具结果裁剪、不改原对象、零预算、没有上限时原样
- 送出,以及把新输入的图算进预算的接线)与 `core/test/model-catalog.test.ts`(四条条目带 4,其余模型与目录外
- 的 id 都没有)。
+ 送出、把新输入的图算进预算的接线、输入自己超上限时只送最新的四张,以及占位写的是卖家的上限而不是剩余额度)
+ 与 `core/test/model-catalog.test.ts`(四条条目带 4,其余模型与目录外的 id 都没有)。
- 文档:`models.zh.md` / `models.en.md` 的 Abliteration 一节写清这条上限与裁剪行为。
## 兼容性
diff --git a/packages/core/src/llm/generative-model.ts b/packages/core/src/llm/generative-model.ts
index b29ebfd0..fb80fbd1 100644
--- a/packages/core/src/llm/generative-model.ts
+++ b/packages/core/src/llm/generative-model.ts
@@ -310,14 +310,21 @@ function capMessageImages(msg: UniMessage, budget: number, limit: number): UniMe
}
/**
- * Trims a replayed history to at most `limit` images in aggregate, newest kept (see
- * {@link capMessageImages}). Returns a new array of the same length whose untouched messages
- * are the very objects it was handed.
+ * Trims a replayed history so it carries at most `budget` images in aggregate, newest kept (see
+ * {@link capMessageImages}). `limit` is the seller's per-request cap and only names itself in
+ * the placeholder — a request whose own input already used part of the budget keeps fewer
+ * replayed images than the cap without misreporting the cap as smaller.
+ * Returns a new array of the same length whose untouched messages are the very objects it was
+ * handed.
*/
-export function capHistoryImages(history: UniMessage[], limit: number): UniMessage[] {
+export function capHistoryImages(
+ history: UniMessage[],
+ budget: number,
+ limit: number = budget,
+): UniMessage[] {
const counts = history.map(countMessageImages);
const capped = new Array<UniMessage>(history.length);
- let left = limit;
+ let left = budget;
for (let i = history.length - 1; i >= 0; i -= 1) {
const msg = history[i]!;
const count = counts[i]!;
@@ -1311,10 +1318,10 @@ export class GenerativeModel implements LLMInterface {
await this.refreshApiKey();
// The seller counts images per request, not per message: a conversation that read five
// pictures would fail every later turn with 400 media_count_exceeded, so the oldest
- // replayed ones are dropped from THIS request (the history and Trace keep them).
- this.fitReplayedImages(uniMessage);
+ // replayed ones — and, when the input alone overflows, the input's own oldest ones — are
+ // dropped from THIS request (the history and Trace keep them).
const it = this.openStream(
- uniMessage,
+ this.fitReplayedImages(uniMessage),
ac.signal,
this.requestConfig(params.thinkingLevel, params.newMessages),
)[Symbol.asyncIterator]();
@@ -1490,24 +1497,32 @@ export class GenerativeModel implements LLMInterface {
}
/**
- * Trims the replayed history so this request carries at most `maxImages` images in all: the
- * new input's images are counted first (they are what the user is asking about now), the
- * history keeps the rest, newest first — see capHistoryImages for the per-message rule.
+ * Trims a request so it carries at most `maxImages` images in all, newest kept.
+ *
+ * The **input** is trimmed first and counts first: an interrupted turn replayed from Trace is
+ * resent as carry-over input, and that input may itself hold more images than the seller
+ * accepts (the exact case that used to fail every turn of a Session forever). The rest of the
+ * budget goes to the replayed history, newest first — see capHistoryImages for the
+ * per-message rule.
*
- * Nothing happens without a configured cap, or when the history already fits. When it does
- * not, the client's history is replaced through setHistory rather than mutated: the objects
- * getHistory hands out are the client's own. A conversation's Trace and stored history are
- * untouched — this decides what one request carries, not what the conversation is.
+ * Nothing happens without a configured cap, or when everything already fits. The input comes
+ * back trimmed (the very object when it fits) and the caller sends that; the client's history
+ * is replaced through setHistory rather than mutated, since the objects getHistory hands out
+ * are the client's own. A conversation's Trace and stored history are untouched — this
+ * decides what one request carries, not what the conversation is.
*/
- private fitReplayedImages(input: UniMessage): void {
+ private fitReplayedImages(input: UniMessage): UniMessage {
const limit = this.maxImages;
- if (limit === undefined) return;
- const budget = Math.max(0, limit - countMessageImages(input));
+ if (limit === undefined) return input;
+ const carried =
+ countMessageImages(input) > limit ? capMessageImages(input, limit, limit) : input;
+ const budget = Math.max(0, limit - countMessageImages(carried));
const history = this.client.getHistory();
- if (history.length === 0) return;
+ if (history.length === 0) return carried;
const total = history.reduce((sum, msg) => sum + countMessageImages(msg), 0);
- if (total <= budget) return;
- this.client.setHistory(capHistoryImages(history, budget));
+ if (total <= budget) return carried;
+ this.client.setHistory(capHistoryImages(history, budget, limit));
+ return carried;
}
/**
diff --git a/packages/core/test/image-budget.test.ts b/packages/core/test/image-budget.test.ts
index 40af3597..8d28c414 100644
--- a/packages/core/test/image-budget.test.ts
+++ b/packages/core/test/image-budget.test.ts
@@ -8,18 +8,27 @@
* What is checked here: the pure trimming (newest images kept, older ones replaced by a text
* note, tool results trimmed inside their own image list) and the wiring — a configured
* `maxImages` reaches the client's history before the request opens, and the new input's own
- * images are counted against the same budget.
+ * images are counted against the same budget. One turn can overflow on its own, too: the input
+ * is a merge of everything the turn carries (a resumed turn replays its tool results), so the
+ * input itself is trimmed to the cap before the history gets what is left.
*/
import { describe, expect, it, vi } from "vitest";
import type { UniEvent, UniMessage } from "@prismshadow/agenthub";
import type { OmniMessage } from "../src/omnimessage/index.js";
+// The class itself is imported below, after `vi.mock` has had its say; this is the type-only
+// half so annotations do not have to spell `InstanceType<typeof …>`.
+import type { GenerativeModel as GenerativeModelClass } from "../src/llm/generative-model.js";
-const state = vi.hoisted(() => ({ clients: [] as { history: UniMessage[] }[] }));
+const state = vi.hoisted(() => ({
+ clients: [] as { history: UniMessage[]; requests: UniMessage[] }[],
+}));
vi.mock("@prismshadow/agenthub", async (importOriginal) => {
const mod = await importOriginal<typeof import("@prismshadow/agenthub")>();
class FakeAutoLLMClient {
history: UniMessage[] = [];
+ /** Every message this client was asked to send, in order — the request side of the budget. */
+ requests: UniMessage[] = [];
constructor(_options: Record<string, unknown>) {
state.clients.push(this);
}
@@ -29,8 +38,9 @@ vi.mock("@prismshadow/agenthub", async (importOriginal) => {
setHistory(history: UniMessage[]): void {
this.history = history;
}
- async *streamingResponseStateful(): AsyncIterable<UniEvent> {
+ async *streamingResponseStateful(options: { message: UniMessage }): AsyncIterable<UniEvent> {
// Nothing upstream: the tests only care about what the request would carry.
+ this.requests.push(options.message);
}
appendHistory(message: UniMessage): void {
this.history = [...this.history, message];
@@ -134,10 +144,20 @@ describe("capHistoryImages", () => {
expect(tool.images).toBeUndefined();
expect(tool.text).toContain(imageOmittedNote(0));
});
+
+ it("names the seller's cap, not the leftover budget, when the input already used some", () => {
+ // One image left for the history of a model that accepts four: the placeholder must not
+ // tell the reader the model takes one.
+ const capped = capHistoryImages([imageMessage(2)], 1, 4);
+ expect(capped[0]!.content_items).toEqual([
+ { type: "text", text: imageOmittedNote(4) },
+ { type: "image_url", image_url: IMAGE },
+ ]);
+ });
});
describe("GenerativeModel image budget", () => {
- const build = (over: { maxImages?: number } = {}): GenerativeModel =>
+ const build = (over: { maxImages?: number } = {}): GenerativeModelClass =>
new GenerativeModel({
modelId: "abliterated-model-large-flash",
apiKey: "ak-test",
@@ -147,9 +167,10 @@ describe("GenerativeModel image budget", () => {
...over,
});
- const lastClient = (): { history: UniMessage[] } => state.clients.at(-1)!;
+ const lastClient = (): { history: UniMessage[]; requests: UniMessage[] } => state.clients.at(-1)!;
+ const lastRequest = (): UniMessage => lastClient().requests.at(-1)!;
- const run = async (model: GenerativeModel, newMessages: OmniMessage[]): Promise<void> => {
+ const run = async (model: GenerativeModelClass, newMessages: OmniMessage[]): Promise<void> => {
for await (const _ of model.streamGenerate({ newMessages })) {
// Nothing is produced — the fake stream is empty.
}
@@ -182,6 +203,24 @@ describe("GenerativeModel image budget", () => {
expect(imageCount(lastClient().history)).toBe(2);
});
+ it("trims the input itself when one turn carries more images than the cap", async () => {
+ const model = build({ maxImages: 4 });
+ model.setHistory([
+ toolCallOutput({ toolCallId: "call_1", output: "image/png, 1 kB", images: [IMAGE] }),
+ ]);
+ // Five pictures in one turn: a resumed turn replays its five read_file results as one
+ // merged input message, which is the shape that failed every request before.
+ await run(
+ model,
+ Array.from({ length: 5 }, () => imageUrlMessage(IMAGE)),
+ );
+ const sent = lastRequest();
+ expect(imageCount([sent])).toBe(4);
+ expect(sent.content_items[0]).toEqual({ type: "text", text: imageOmittedNote(4) });
+ // The input used the whole cap: not one replayed image goes out with it.
+ expect(imageCount(lastClient().history)).toBe(0);
+ });
+
it("sends the history untouched when no cap is configured", async () => {
const model = build();
model.setHistory([
diff --git a/packages/docs/content/models.en.md b/packages/docs/content/models.en.md
index f4af40a3..5b86b20d 100644
--- a/packages/docs/content/models.en.md
+++ b/packages/docs/content/models.en.md
@@ -499,7 +499,7 @@ The preset rows carry no price. ModelScope bills for api-inference and its model
- **Presets.** `abliterated-model-large-v2` (the vendor's recommended default), `abliterated-model-large`, `abliterated-model-large-flash` and `abliterated-model`. The first two take text only at a 1,000,000-token window; `flash` takes text and images at 1,048,576; `abliterated-model` takes both at 262,144.
- **Protocol.** None of these ids matches a rule AgentHub routes by, so every row pins the Chat Completions client (`client_type = "openai-chat"`) and carries the endpoint on itself. The same endpoint also serves `/responses`; change the row's protocol if you prefer it. As in every gateway, a keyless row is refused rather than sent your OpenAI key.
- **Prices.** Read from the service's own model list, in USD per million Tokens: input 1 or 3, output 2, 3 or 5, and cache hits read at 0.1 or 0.3. It charges no separate cache-write fee, so `cache_write` records the input rate.
-- **At most four images per request.** That is the service's own limit, and a request past it is refused outright (`400 … media_count_exceeded`); because a conversation replays its images on every request, such a conversation fails for good. Adelie therefore replaces the **oldest** images with a one-line `[image omitted: …]` placeholder before each request, keeping the newest four (images in the new input count against them first). The conversation and its Trace keep every picture — only what one request carries is reduced.
+- **At most four images per request.** That is the service's own limit, and a request past it is refused outright (`400 … media_count_exceeded`); because a conversation replays its images on every request, such a conversation fails for good. Adelie therefore replaces the **oldest** images with a one-line `[image omitted: …]` placeholder before each request, keeping the newest four (images in the new input count against them first; when a single turn carries five on its own, its oldest ones become placeholders the same way). The conversation and its Trace keep every picture — only what one request carries is reduced.
### Preset models
diff --git a/packages/docs/content/models.zh.md b/packages/docs/content/models.zh.md
index 38f586c3..2526239c 100644
--- a/packages/docs/content/models.zh.md
+++ b/packages/docs/content/models.zh.md
@@ -498,7 +498,7 @@ OpenAI 兼容网关分组(openrouter / fireworks / siliconflow / tokendance /
- **预置模型。** `abliterated-model-large-v2`(官方推荐的默认选择)、`abliterated-model-large`、`abliterated-model-large-flash` 与 `abliterated-model`。前两条是 1,000,000 Token 上下文、只收文本;`flash` 是 1,048,576 并接受图像;`abliterated-model` 是 262,144,也接受图像。
- **协议。** 这些 id 没有一条能被 AgentHub 按名字认出来,所以每个条目都固定 Chat Completions 客户端(`client_type = "openai-chat"`)并把端点写在条目上;同一条端点上也有 `/responses`,要换协议在条目上改即可。和所有网关一样,没有 key 的条目会被拒绝,而不是把你的 OpenAI key 发过去。
- **价格。** 取自服务自己的模型列表,单位是每百万 Token 多少美元:输入 1 或 3,输出 2、3 或 5,缓存命中的读价 0.1 或 0.3。它不额外收缓存写入费,所以 `cache_write` 记的就是输入价。
-- **一次请求最多 4 张图。** 这是服务自己的限制,超过它会直接回 `400 … media_count_exceeded`,而对话里的图会被每一次请求重新带上——所以会话会一直失败。Adelie 因此在每次请求前把**最旧的**那几张换成一行 `[image omitted: …]` 文字占位,只保留最新的四张(新发出的图优先算进这四张)。对话本身和 Trace 里的图一张不少,改的只是这一次请求带过去的量。
+- **一次请求最多 4 张图。** 这是服务自己的限制,超过它会直接回 `400 … media_count_exceeded`,而对话里的图会被每一次请求重新带上——所以会话会一直失败。Adelie 因此在每次请求前把**最旧的**那几张换成一行 `[image omitted: …]` 文字占位,只保留最新的四张(新发出的图优先算进这四张;一轮自己就带了五张时,这一次输入里的旧图也一样换成占位)。对话本身和 Trace 里的图一张不少,改的只是这一次请求带过去的量。
### 预置模型