Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(sdk): send Anthropic messages to /v1/compress natively so thinking blocks survive (#4489)

Shubham Srivastavacd5e2b9e73eb
diff --git a/docs/content/docs/anthropic-sdk.mdx b/docs/content/docs/anthropic-sdk.mdx
index 8dc6fd65e..d06112721 100644
--- a/docs/content/docs/anthropic-sdk.mdx
+++ b/docs/content/docs/anthropic-sdk.mdx
@@ -41,26 +41,19 @@ Every call to `client.messages.create()` compresses messages first. The response
`withHeadroom()` returns a proxy around your Anthropic client that intercepts `messages.create()`:
-1. Converts Anthropic-format messages to OpenAI format
-2. Sends them to the Headroom proxy's `/v1/compress` endpoint
-3. Converts the compressed messages back to Anthropic format
-4. Forwards the request to Anthropic as normal
+1. Sends your `messages` to the Headroom proxy's `/v1/compress` endpoint as they are, in Anthropic format
+2. Forwards the request to Anthropic with the compressed messages in place
-### Message format conversion
+### Message format
-The adapter handles the full Anthropic message format including content blocks:
+Nothing is converted. `POST /v1/compress` compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format) — and returns the same shape it received, so every block reaches Anthropic exactly as you built it:
-| Anthropic format | OpenAI format |
-|-----------------|---------------|
-| `{ type: "text", text: "..." }` | `{ role: "user", content: "..." }` |
-| `{ type: "tool_use", id, name, input }` | `{ tool_calls: [{ id, function: { name, arguments } }] }` |
-| `{ type: "tool_result", tool_use_id, content }` | `{ role: "tool", tool_call_id, content }` |
+- `thinking` and `redacted_thinking` blocks keep their `signature` and `data`, which Anthropic requires back unmodified within a tool-use turn
+- `cache_control` markers stay on the blocks that carry them
+- `tool_use` and `tool_result` blocks keep their ids; only the content of large `tool_result` blocks changes
+- images, documents, citations and server-tool blocks pass through untouched
-This conversion is lossless. Your request and response behave identically to an unwrapped client.
-
-<Callout type="info" title="The conversion is an SDK choice, not an API requirement">
-`POST /v1/compress` does no format conversion and compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format). If you are calling the endpoint directly (from a gateway, or LiteLLM's `headroom` guardrail), send Anthropic-shaped messages as-is; you get the same shape back. Only this TypeScript adapter converts, because it normalises on OpenAI types internally.
-</Callout>
+If the proxy is unreachable and `fallback` is enabled, the original messages are forwarded unchanged.
## Options
diff --git a/sdk/typescript/src/adapters/anthropic.ts b/sdk/typescript/src/adapters/anthropic.ts
index 11693c7c5..3c1fc3a81 100644
--- a/sdk/typescript/src/adapters/anthropic.ts
+++ b/sdk/typescript/src/adapters/anthropic.ts
@@ -1,6 +1,5 @@
import { compress } from "../compress.js";
import type { CompressOptions } from "../types.js";
-import { anthropicToOpenAI, openAIToAnthropic } from "../utils/format.js";
/* eslint-disable @typescript-eslint/no-explicit-any */
@@ -15,6 +14,10 @@ interface AnthropicLike {
* Wrap an Anthropic client to auto-compress messages before each request.
*
* Intercepts `client.messages.create()` only. All other methods pass through.
+ * Messages reach `/v1/compress` in Anthropic shape and come back in Anthropic
+ * shape: the proxy compresses Anthropic content blocks natively, so thinking
+ * blocks, cache_control markers and tool_result blocks are forwarded exactly
+ * as the caller built them.
*
* @example
* ```typescript
@@ -40,20 +43,15 @@ export function withHeadroom<T extends AnthropicLike>(
const model =
options.model ?? params.model ?? "claude-sonnet-4-5-20250929";
- const openaiMessages = anthropicToOpenAI(messages);
- const result = await compress(openaiMessages, {
+ const result = await compress(messages, {
stack: "adapter_ts_anthropic",
...options,
model,
});
- const anthropicMessages = result.compressed
- ? openAIToAnthropic(result.messages)
- : messages;
-
return originalCreate({
...params,
- messages: anthropicMessages,
+ messages: result.compressed ? result.messages : messages,
});
};
diff --git a/sdk/typescript/src/compress.ts b/sdk/typescript/src/compress.ts
index 23bf50e05..c995449d1 100644
--- a/sdk/typescript/src/compress.ts
+++ b/sdk/typescript/src/compress.ts
@@ -14,7 +14,8 @@ import { extractUserQuery, countTurns, extractToolCalls } from "./hooks.js";
*
* Accepts messages in any format: OpenAI, Anthropic, Vercel AI SDK, or Google Gemini.
* Detects the format automatically, compresses via the proxy, and returns
- * compressed messages in the same format as the input.
+ * compressed messages in the same format as the input. Anthropic messages are
+ * sent to the proxy unconverted (it compresses Anthropic blocks natively).
*/
export async function compress(
messages: any[],
@@ -47,21 +48,26 @@ export async function compress(
// 2. Detect input format
const inputFormat = detectFormat(processedMessages);
- // 3. Convert to OpenAI format (the proxy's lingua franca)
- const openaiMessages = toOpenAI(processedMessages);
+ // 3. Shape the messages for the proxy. Anthropic messages go as-is: /v1/compress
+ // compresses Anthropic content blocks natively, and converting them would drop
+ // the thinking blocks Anthropic requires back unchanged, along with cache_control
+ // markers and server-tool blocks. Every other format is converted to OpenAI
+ // shape, the proxy's lingua franca.
+ const native = inputFormat === "anthropic";
+ const proxyMessages = native ? processedMessages : toOpenAI(processedMessages);
// 4. Compute biases
let biases: Record<number, number> = {};
if (hooks) {
- biases = await hooks.computeBiases(openaiMessages, ctx);
+ biases = await hooks.computeBiases(proxyMessages, ctx);
}
// 5. Compress via proxy
const client = providedClient ?? new HeadroomClient(clientOptions);
- const result = await client.compress(openaiMessages, { model, tokenBudget, config });
+ const result = await client.compress(proxyMessages, { model, tokenBudget, config });
// 6. Convert compressed messages back to original format
- const outputMessages = fromOpenAI(result.messages, inputFormat);
+ const outputMessages = native ? result.messages : fromOpenAI(result.messages, inputFormat);
const finalResult: CompressResult = {
...result,
diff --git a/sdk/typescript/src/utils/format.ts b/sdk/typescript/src/utils/format.ts
index ec744ad71..0c40851fb 100644
--- a/sdk/typescript/src/utils/format.ts
+++ b/sdk/typescript/src/utils/format.ts
@@ -54,6 +54,8 @@ export function detectFormat(messages: any[]): MessageFormat {
if (part.type === "tool_use" || part.type === "tool_result") return "anthropic";
// Anthropic: image / document with source.type
if ((part.type === "image" || part.type === "document") && part.source?.type) return "anthropic";
+ // Anthropic: thinking blocks (the Vercel AI SDK spells its equivalent part `reasoning`)
+ if (part.type === "thinking" || part.type === "redacted_thinking") return "anthropic";
// Vercel: `file` parts carry data/mediaType (OpenAI's `file` part carries a `file` object);
// `image` parts carry `image` (Anthropic's carry `source`)
if (part && typeof part === "object") {
diff --git a/sdk/typescript/test/adapters/anthropic.test.ts b/sdk/typescript/test/adapters/anthropic.test.ts
index b78d530af..6664e7a57 100644
--- a/sdk/typescript/test/adapters/anthropic.test.ts
+++ b/sdk/typescript/test/adapters/anthropic.test.ts
@@ -21,6 +21,39 @@ function mockCompressSuccess(
);
}
+// A thinking-enabled tool-use turn. Anthropic requires the thinking blocks
+// back "complete and unmodified" with the tool result, or the request is a 400.
+const thinkingToolTurn = [
+ {
+ role: "user",
+ content: [
+ { type: "text", text: "Read the gateway log.", cache_control: { type: "ephemeral" } },
+ ],
+ },
+ {
+ role: "assistant",
+ content: [
+ { type: "thinking", thinking: "I should read the log first.", signature: "SIG-abc123" },
+ { type: "redacted_thinking", data: "REDACTED-xyz" },
+ { type: "text", text: "Reading the log now." },
+ { type: "tool_use", id: "toolu_01", name: "fetch_gateway_metrics", input: { path: "/var/log/gateway.json" } },
+ ],
+ },
+ {
+ role: "user",
+ content: [
+ {
+ type: "tool_result",
+ tool_use_id: "toolu_01",
+ content: [
+ { type: "text", text: "row 1\nrow 2\nrow 3" },
+ { type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },
+ ],
+ },
+ ],
+ },
+];
+
describe("withHeadroom (Anthropic)", () => {
beforeEach(() => {
mockFetch.mockReset();
@@ -102,7 +135,7 @@ describe("withHeadroom (Anthropic)", () => {
expect(result).toEqual(expectedResponse);
});
- it("converts Anthropic content blocks to OpenAI format for compression", async () => {
+ it("posts Anthropic content blocks to /v1/compress unchanged", async () => {
mockFetch.mockResolvedValueOnce(mockCompressSuccess());
const mockCreate = vi.fn().mockResolvedValue({});
@@ -112,46 +145,47 @@ describe("withHeadroom (Anthropic)", () => {
baseUrl: "http://localhost:8787",
});
+ const messages = [
+ { role: "user", content: [{ type: "text", text: "analyze this data" }] },
+ ];
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
- messages: [
- {
- role: "user",
- content: [{ type: "text", text: "analyze this data" }],
- },
- ],
+ messages,
max_tokens: 1024,
});
- // Verify the compress call used OpenAI format
+ // The proxy compresses Anthropic blocks natively; nothing is converted.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
- expect(body.messages[0].role).toBe("user");
- expect(body.messages[0].content).toBe("analyze this data");
+ expect(body.messages).toEqual(messages);
+ expect(body.model).toBe("claude-sonnet-4-5-20250929");
});
- it("handles Anthropic tool_use and tool_result blocks", async () => {
- // Proxy returns compressed messages
- mockFetch.mockResolvedValueOnce(
- mockCompressSuccess([
- { role: "user", content: "search for X" },
- {
- role: "assistant",
- content: null,
- tool_calls: [
- {
- id: "tu_1",
- type: "function",
- function: { name: "search", arguments: '{"q":"X"}' },
- },
- ],
- },
- {
- role: "tool",
- content: '{"results":["a","b"]}',
- tool_call_id: "tu_1",
- },
- ]),
- );
+ it("posts tool_use and array-content tool_result blocks as-is and forwards the proxy's result", async () => {
+ const turn = [
+ { role: "user", content: "run the query" },
+ {
+ role: "assistant",
+ content: [
+ { type: "tool_use", id: "tu_1", name: "query", input: { sql: "select 1" } },
+ ],
+ },
+ {
+ role: "user",
+ content: [
+ {
+ type: "tool_result",
+ tool_use_id: "tu_1",
+ content: [
+ { type: "text", text: "row 1" },
+ { type: "text", text: "row 2" },
+ ],
+ },
+ ],
+ },
+ ];
+ const compressed = structuredClone(turn);
+ (compressed[2].content as any[])[0].content = [{ type: "text", text: "[2 rows]" }];
+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(compressed));
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
@@ -162,36 +196,33 @@ describe("withHeadroom (Anthropic)", () => {
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
- messages: [
- { role: "user", content: "search for X" },
- {
- role: "assistant",
- content: [
- { type: "tool_use", id: "tu_1", name: "search", input: { q: "X" } },
- ],
- },
- {
- role: "user",
- content: [
- {
- type: "tool_result",
- tool_use_id: "tu_1",
- content: '{"results":["a","b"]}',
- },
- ],
- },
- ],
+ messages: turn,
max_tokens: 1024,
});
+ // The proxy reads Anthropic tool_result blocks natively, list content included.
+ const body = JSON.parse(mockFetch.mock.calls[0][1].body);
+ expect(body.messages).toEqual(turn);
+
expect(mockCreate).toHaveBeenCalledOnce();
- // Messages should be converted back to Anthropic format
const createArgs = mockCreate.mock.calls[0][0];
- expect(createArgs.messages.length).toBeGreaterThan(0);
+ expect(createArgs.messages).toEqual(compressed);
});
- it("flattens array-content tool_result blocks to text for the compression request", async () => {
- mockFetch.mockResolvedValueOnce(mockCompressSuccess());
+ it("keeps image blocks native in the compression request and in the forwarded messages", async () => {
+ const turn = [
+ {
+ role: "user",
+ content: [
+ { type: "text", text: "what is this?" },
+ {
+ type: "image",
+ source: { type: "base64", media_type: "image/png", data: "AAAA" },
+ },
+ ],
+ },
+ ];
+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(turn));
const mockCreate = vi.fn().mockResolvedValue({});
const fakeClient = { messages: { create: mockCreate } };
@@ -202,172 +233,55 @@ describe("withHeadroom (Anthropic)", () => {
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
- messages: [
- { role: "user", content: "run the query" },
- {
- role: "assistant",
- content: [
- { type: "tool_use", id: "tu_1", name: "query", input: { sql: "select 1" } },
- ],
- },
- {
- role: "user",
- content: [
- {
- type: "tool_result",
- tool_use_id: "tu_1",
- content: [
- { type: "text", text: "row 1" },
- { type: "text", text: "row 2" },
- ],
- },
- ],
- },
- ],
+ messages: turn,
max_tokens: 1024,
});
- // The proxy must receive the tool output as text, not a JSON-encoded block array.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
- expect(body.messages[2]).toEqual({
- role: "tool",
- content: "row 1\nrow 2",
- tool_call_id: "tu_1",
- });
- });
-
- it("preserves image blocks instead of dropping them during compression", async () => {
- mockFetch.mockResolvedValueOnce(mockCompressSuccess());
-
- const mockCreate = vi.fn().mockResolvedValue({});
- const fakeClient = { messages: { create: mockCreate } };
-
- const wrapped = withHeadroom(fakeClient as any, {
- baseUrl: "http://localhost:8787",
- });
+ expect(body.messages).toEqual(turn);
- await wrapped.messages.create({
- model: "claude-sonnet-4-5-20250929",
- messages: [
- {
- role: "user",
- content: [
- { type: "text", text: "what is this?" },
- {
- type: "image",
- source: { type: "base64", media_type: "image/png", data: "AAAA" },
- },
- ],
- },
- ],
- max_tokens: 1024,
- });
-
- // The image must not be silently dropped from the compression request.
- const body = JSON.parse(mockFetch.mock.calls[0][1].body);
- expect(body.messages).toHaveLength(1);
- expect(body.messages[0].content).toEqual([
- { type: "text", text: "what is this?" },
- { type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },
- ]);
+ expect(mockCreate).toHaveBeenCalledOnce();
+ const createArgs = mockCreate.mock.calls[0][0];
+ expect(createArgs.messages).toEqual(turn);
});
- it("converts a retained image_url back to an Anthropic image block for the final messages.create call (image-only turn)", async () => {
- // Proxy retains the image through compression (successful-compression path).
- mockFetch.mockResolvedValueOnce(
- mockCompressSuccess([
- {
- role: "user",
- content: [
- { type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },
- ],
- },
- ]),
- );
-
+ it("posts a thinking-enabled tool-use turn unchanged, thinking blocks and tool_result blocks included", async () => {
+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(thinkingToolTurn));
const mockCreate = vi.fn().mockResolvedValue({});
- const fakeClient = { messages: { create: mockCreate } };
-
- const wrapped = withHeadroom(fakeClient as any, {
+ const wrapped = withHeadroom({ messages: { create: mockCreate } } as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
- messages: [
- {
- role: "user",
- content: [
- {
- type: "image",
- source: { type: "base64", media_type: "image/png", data: "AAAA" },
- },
- ],
- },
- ],
+ messages: thinkingToolTurn,
max_tokens: 1024,
});
- expect(mockCreate).toHaveBeenCalledOnce();
- const createArgs = mockCreate.mock.calls[0][0];
- expect(createArgs.messages).toEqual([
- {
- role: "user",
- content: [
- { type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },
- ],
- },
- ]);
+ const body = JSON.parse(mockFetch.mock.calls[0][1].body);
+ expect(body.messages).toEqual(thinkingToolTurn);
+ expect(body.model).toBe("claude-sonnet-4-5-20250929");
});
- it("converts a retained mixed text/image turn back to Anthropic blocks, order preserved, for the final messages.create call", async () => {
- mockFetch.mockResolvedValueOnce(
- mockCompressSuccess([
- {
- role: "user",
- content: [
- { type: "text", text: "what is this?" },
- { type: "image_url", image_url: { url: "data:image/jpeg;base64,BBBB" } },
- ],
- },
- ]),
- );
-
+ it("forwards the proxy's returned Anthropic messages to messages.create as-is", async () => {
+ const compressed = structuredClone(thinkingToolTurn);
+ (compressed[2].content[0] as any).content[0].text = "[3 rows]";
+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(compressed));
const mockCreate = vi.fn().mockResolvedValue({});
- const fakeClient = { messages: { create: mockCreate } };
-
- const wrapped = withHeadroom(fakeClient as any, {
+ const wrapped = withHeadroom({ messages: { create: mockCreate } } as any, {
baseUrl: "http://localhost:8787",
});
await wrapped.messages.create({
model: "claude-sonnet-4-5-20250929",
- messages: [
- {
- role: "user",
- content: [
- { type: "text", text: "what is this?" },
- {
- type: "image",
- source: { type: "base64", media_type: "image/jpeg", data: "BBBB" },
- },
- ],
- },
- ],
+ messages: thinkingToolTurn,
max_tokens: 1024,
});
- expect(mockCreate).toHaveBeenCalledOnce();
const createArgs = mockCreate.mock.calls[0][0];
- expect(createArgs.messages).toEqual([
- {
- role: "user",
- content: [
- { type: "text", text: "what is this?" },
- { type: "image", source: { type: "base64", media_type: "image/jpeg", data: "BBBB" } },
- ],
- },
- ]);
+ expect(createArgs.messages).toEqual(compressed);
+ // The thinking blocks reach the provider exactly as the model produced them.
+ expect(createArgs.messages[1].content.slice(0, 2)).toEqual(thinkingToolTurn[1].content.slice(0, 2));
});
it("returns original messages on compression fallback", async () => {
diff --git a/sdk/typescript/test/compress.test.ts b/sdk/typescript/test/compress.test.ts
index 49ec2d934..78a4be809 100644
--- a/sdk/typescript/test/compress.test.ts
+++ b/sdk/typescript/test/compress.test.ts
@@ -184,21 +184,54 @@ describe("compress() keeps Anthropic text documents through the public path", ()
mockFetch.mockReset();
});
- it("sends a document-only turn as a media part and restores the document block", async () => {
+ it("posts a document-only turn as the native document block and returns it unchanged", async () => {
const doc = {
type: "document",
source: { type: "text", media_type: "text/plain", data: "Critical contract: do not delete account 123" },
title: "Contract",
citations: { enabled: true },
};
- // The proxy passes non-text parts through unchanged; echo the request's messages back.
+ // The proxy passes non-text blocks through unchanged; echo the request's messages back.
mockFetch.mockImplementationOnce(async (_url: string, init: any) => okResponse({ messages: JSON.parse(init.body).messages }));
const result = await compress([{ role: "user", content: [doc] }], { baseUrl: "http://localhost:8787" });
+ // Anthropic input is not converted: the document block itself reaches the proxy.
const body = JSON.parse(mockFetch.mock.calls[0][1].body);
- expect(body.messages).toHaveLength(1);
- expect(body.messages[0].content[0].type).toBe("image_url");
+ expect(body.messages).toEqual([{ role: "user", content: [doc] }]);
expect(result.messages).toEqual([{ role: "user", content: [doc] }]);
});
});
+
+describe("compress() sends Anthropic-format input to the proxy natively", () => {
+ beforeEach(() => {
+ mockFetch.mockReset();
+ });
+
+ const anthropicTurn = [
+ { role: "user", content: "search for X" },
+ {
+ role: "assistant",
+ content: [
+ { type: "thinking", thinking: "Search first.", signature: "SIG-1" },
+ { type: "tool_use", id: "toolu_1", name: "search", input: { q: "X" } },
+ ],
+ },
+ {
+ role: "user",
+ content: [{ type: "tool_result", tool_use_id: "toolu_1", content: "a\nb\nc" }],
+ },
+ ];
+
+ it("posts the Anthropic blocks as-is and returns the proxy's messages unconverted", async () => {
+ const returned = structuredClone(anthropicTurn);
+ (returned[2].content as any[])[0].content = "[3 lines]";
+ mockFetch.mockResolvedValueOnce(okResponse({ messages: returned }));
+
+ const result = await compress(anthropicTurn, { baseUrl: "http://localhost:8787" });
+
+ const body = JSON.parse(mockFetch.mock.calls[0][1].body);
+ expect(body.messages).toEqual(anthropicTurn);
+ expect(result.messages).toEqual(returned);
+ });
+});
diff --git a/sdk/typescript/test/utils/detect-format.test.ts b/sdk/typescript/test/utils/detect-format.test.ts
index d74bb8216..4b7a267b8 100644
--- a/sdk/typescript/test/utils/detect-format.test.ts
+++ b/sdk/typescript/test/utils/detect-format.test.ts
@@ -185,3 +185,26 @@ describe("detectFormat: Anthropic document blocks", () => {
).toBe("anthropic");
});
});
+
+describe("detectFormat: Anthropic thinking blocks", () => {
+ it("detects a turn whose only Anthropic marker is a thinking block", () => {
+ const messages = [
+ { role: "user", content: "hi" },
+ {
+ role: "assistant",
+ content: [
+ { type: "thinking", thinking: "...", signature: "SIG" },
+ { type: "text", text: "hello" },
+ ],
+ },
+ ];
+ expect(detectFormat(messages)).toBe("anthropic");
+ });
+
+ it("detects a turn whose only Anthropic marker is a redacted_thinking block", () => {
+ const messages = [
+ { role: "assistant", content: [{ type: "redacted_thinking", data: "REDACTED" }, { type: "text", text: "hello" }] },
+ ];
+ expect(detectFormat(messages)).toBe("anthropic");
+ });
+});