Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(sdk): send Anthropic messages to /v1/compress natively so thinking blocks survive (#4489)
Shubham Srivastavacd5e2b9e73eb
diff --git a/docs/content/docs/anthropic-sdk.mdx b/docs/content/docs/anthropic-sdk.mdxindex 8dc6fd65e..d06112721 100644--- a/docs/content/docs/anthropic-sdk.mdx+++ b/docs/content/docs/anthropic-sdk.mdx@@ -41,26 +41,19 @@ Every call to `client.messages.create()` compresses messages first. The response`withHeadroom()` returns a proxy around your Anthropic client that intercepts `messages.create()`:-1. Converts Anthropic-format messages to OpenAI format-2. Sends them to the Headroom proxy's `/v1/compress` endpoint-3. Converts the compressed messages back to Anthropic format-4. Forwards the request to Anthropic as normal+1. Sends your `messages` to the Headroom proxy's `/v1/compress` endpoint as they are, in Anthropic format+2. Forwards the request to Anthropic with the compressed messages in place-### Message format conversion+### Message format-The adapter handles the full Anthropic message format including content blocks:+Nothing is converted. `POST /v1/compress` compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format) — and returns the same shape it received, so every block reaches Anthropic exactly as you built it:-| Anthropic format | OpenAI format |-|-----------------|---------------|-| `{ type: "text", text: "..." }` | `{ role: "user", content: "..." }` |-| `{ type: "tool_use", id, name, input }` | `{ tool_calls: [{ id, function: { name, arguments } }] }` |-| `{ type: "tool_result", tool_use_id, content }` | `{ role: "tool", tool_call_id, content }` |+- `thinking` and `redacted_thinking` blocks keep their `signature` and `data`, which Anthropic requires back unmodified within a tool-use turn+- `cache_control` markers stay on the blocks that carry them+- `tool_use` and `tool_result` blocks keep their ids; only the content of large `tool_result` blocks changes+- images, documents, citations and server-tool blocks pass through untouched-This conversion is lossless. Your request and response behave identically to an unwrapped client.--<Callout type="info" title="The conversion is an SDK choice, not an API requirement">-`POST /v1/compress` does no format conversion and compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format). If you are calling the endpoint directly (from a gateway, or LiteLLM's `headroom` guardrail), send Anthropic-shaped messages as-is; you get the same shape back. Only this TypeScript adapter converts, because it normalises on OpenAI types internally.-</Callout>+If the proxy is unreachable and `fallback` is enabled, the original messages are forwarded unchanged.## Optionsdiff --git a/sdk/typescript/src/adapters/anthropic.ts b/sdk/typescript/src/adapters/anthropic.tsindex 11693c7c5..3c1fc3a81 100644--- a/sdk/typescript/src/adapters/anthropic.ts+++ b/sdk/typescript/src/adapters/anthropic.ts@@ -1,6 +1,5 @@import { compress } from "../compress.js";import type { CompressOptions } from "../types.js";-import { anthropicToOpenAI, openAIToAnthropic } from "../utils/format.js";/* eslint-disable @typescript-eslint/no-explicit-any */@@ -15,6 +14,10 @@ interface AnthropicLike {* Wrap an Anthropic client to auto-compress messages before each request.** Intercepts `client.messages.create()` only. All other methods pass through.+ * Messages reach `/v1/compress` in Anthropic shape and come back in Anthropic+ * shape: the proxy compresses Anthropic content blocks natively, so thinking+ * blocks, cache_control markers and tool_result blocks are forwarded exactly+ * as the caller built them.** @example* ```typescript@@ -40,20 +43,15 @@ export function withHeadroom<T extends AnthropicLike>(const model =options.model ?? params.model ?? "claude-sonnet-4-5-20250929";- const openaiMessages = anthropicToOpenAI(messages);- const result = await compress(openaiMessages, {+ const result = await compress(messages, {stack: "adapter_ts_anthropic",...options,model,});- const anthropicMessages = result.compressed- ? openAIToAnthropic(result.messages)- : messages;-return originalCreate({...params,- messages: anthropicMessages,+ messages: result.compressed ? result.messages : messages,});};diff --git a/sdk/typescript/src/compress.ts b/sdk/typescript/src/compress.tsindex 23bf50e05..c995449d1 100644--- a/sdk/typescript/src/compress.ts+++ b/sdk/typescript/src/compress.ts@@ -14,7 +14,8 @@ import { extractUserQuery, countTurns, extractToolCalls } from "./hooks.js";** Accepts messages in any format: OpenAI, Anthropic, Vercel AI SDK, or Google Gemini.* Detects the format automatically, compresses via the proxy, and returns- * compressed messages in the same format as the input.+ * compressed messages in the same format as the input. Anthropic messages are+ * sent to the proxy unconverted (it compresses Anthropic blocks natively).*/export async function compress(messages: any[],@@ -47,21 +48,26 @@ export async function compress(// 2. Detect input formatconst inputFormat = detectFormat(processedMessages);- // 3. Convert to OpenAI format (the proxy's lingua franca)- const openaiMessages = toOpenAI(processedMessages);+ // 3. Shape the messages for the proxy. Anthropic messages go as-is: /v1/compress+ // compresses Anthropic content blocks natively, and converting them would drop+ // the thinking blocks Anthropic requires back unchanged, along with cache_control+ // markers and server-tool blocks. Every other format is converted to OpenAI+ // shape, the proxy's lingua franca.+ const native = inputFormat === "anthropic";+ const proxyMessages = native ? processedMessages : toOpenAI(processedMessages);// 4. Compute biaseslet biases: Record<number, number> = {};if (hooks) {- biases = await hooks.computeBiases(openaiMessages, ctx);+ biases = await hooks.computeBiases(proxyMessages, ctx);}// 5. Compress via proxyconst client = providedClient ?? new HeadroomClient(clientOptions);- const result = await client.compress(openaiMessages, { model, tokenBudget, config });+ const result = await client.compress(proxyMessages, { model, tokenBudget, config });// 6. Convert compressed messages back to original format- const outputMessages = fromOpenAI(result.messages, inputFormat);+ const outputMessages = native ? result.messages : fromOpenAI(result.messages, inputFormat);const finalResult: CompressResult = {...result,diff --git a/sdk/typescript/src/utils/format.ts b/sdk/typescript/src/utils/format.tsindex ec744ad71..0c40851fb 100644--- a/sdk/typescript/src/utils/format.ts+++ b/sdk/typescript/src/utils/format.ts@@ -54,6 +54,8 @@ export function detectFormat(messages: any[]): MessageFormat {if (part.type === "tool_use" || part.type === "tool_result") return "anthropic";// Anthropic: image / document with source.typeif ((part.type === "image" || part.type === "document") && part.source?.type) return "anthropic";+ // Anthropic: thinking blocks (the Vercel AI SDK spells its equivalent part `reasoning`)+ if (part.type === "thinking" || part.type === "redacted_thinking") return "anthropic";// Vercel: `file` parts carry data/mediaType (OpenAI's `file` part carries a `file` object);// `image` parts carry `image` (Anthropic's carry `source`)if (part && typeof part === "object") {diff --git a/sdk/typescript/test/adapters/anthropic.test.ts b/sdk/typescript/test/adapters/anthropic.test.tsindex b78d530af..6664e7a57 100644--- a/sdk/typescript/test/adapters/anthropic.test.ts+++ b/sdk/typescript/test/adapters/anthropic.test.ts@@ -21,6 +21,39 @@ function mockCompressSuccess();}+// A thinking-enabled tool-use turn. Anthropic requires the thinking blocks+// back "complete and unmodified" with the tool result, or the request is a 400.+const thinkingToolTurn = [+ {+ role: "user",+ content: [+ { type: "text", text: "Read the gateway log.", cache_control: { type: "ephemeral" } },+ ],+ },+ {+ role: "assistant",+ content: [+ { type: "thinking", thinking: "I should read the log first.", signature: "SIG-abc123" },+ { type: "redacted_thinking", data: "REDACTED-xyz" },+ { type: "text", text: "Reading the log now." },+ { type: "tool_use", id: "toolu_01", name: "fetch_gateway_metrics", input: { path: "/var/log/gateway.json" } },+ ],+ },+ {+ role: "user",+ content: [+ {+ type: "tool_result",+ tool_use_id: "toolu_01",+ content: [+ { type: "text", text: "row 1\nrow 2\nrow 3" },+ { type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },+ ],+ },+ ],+ },+];+describe("withHeadroom (Anthropic)", () => {beforeEach(() => {mockFetch.mockReset();@@ -102,7 +135,7 @@ describe("withHeadroom (Anthropic)", () => {expect(result).toEqual(expectedResponse);});- it("converts Anthropic content blocks to OpenAI format for compression", async () => {+ it("posts Anthropic content blocks to /v1/compress unchanged", async () => {mockFetch.mockResolvedValueOnce(mockCompressSuccess());const mockCreate = vi.fn().mockResolvedValue({});@@ -112,46 +145,47 @@ describe("withHeadroom (Anthropic)", () => {baseUrl: "http://localhost:8787",});+ const messages = [+ { role: "user", content: [{ type: "text", text: "analyze this data" }] },+ ];await wrapped.messages.create({model: "claude-sonnet-4-5-20250929",- messages: [- {- role: "user",- content: [{ type: "text", text: "analyze this data" }],- },- ],+ messages,max_tokens: 1024,});- // Verify the compress call used OpenAI format+ // The proxy compresses Anthropic blocks natively; nothing is converted.const body = JSON.parse(mockFetch.mock.calls[0][1].body);- expect(body.messages[0].role).toBe("user");- expect(body.messages[0].content).toBe("analyze this data");+ expect(body.messages).toEqual(messages);+ expect(body.model).toBe("claude-sonnet-4-5-20250929");});- it("handles Anthropic tool_use and tool_result blocks", async () => {- // Proxy returns compressed messages- mockFetch.mockResolvedValueOnce(- mockCompressSuccess([- { role: "user", content: "search for X" },- {- role: "assistant",- content: null,- tool_calls: [- {- id: "tu_1",- type: "function",- function: { name: "search", arguments: '{"q":"X"}' },- },- ],- },- {- role: "tool",- content: '{"results":["a","b"]}',- tool_call_id: "tu_1",- },- ]),- );+ it("posts tool_use and array-content tool_result blocks as-is and forwards the proxy's result", async () => {+ const turn = [+ { role: "user", content: "run the query" },+ {+ role: "assistant",+ content: [+ { type: "tool_use", id: "tu_1", name: "query", input: { sql: "select 1" } },+ ],+ },+ {+ role: "user",+ content: [+ {+ type: "tool_result",+ tool_use_id: "tu_1",+ content: [+ { type: "text", text: "row 1" },+ { type: "text", text: "row 2" },+ ],+ },+ ],+ },+ ];+ const compressed = structuredClone(turn);+ (compressed[2].content as any[])[0].content = [{ type: "text", text: "[2 rows]" }];+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(compressed));const mockCreate = vi.fn().mockResolvedValue({});const fakeClient = { messages: { create: mockCreate } };@@ -162,36 +196,33 @@ describe("withHeadroom (Anthropic)", () => {await wrapped.messages.create({model: "claude-sonnet-4-5-20250929",- messages: [- { role: "user", content: "search for X" },- {- role: "assistant",- content: [- { type: "tool_use", id: "tu_1", name: "search", input: { q: "X" } },- ],- },- {- role: "user",- content: [- {- type: "tool_result",- tool_use_id: "tu_1",- content: '{"results":["a","b"]}',- },- ],- },- ],+ messages: turn,max_tokens: 1024,});+ // The proxy reads Anthropic tool_result blocks natively, list content included.+ const body = JSON.parse(mockFetch.mock.calls[0][1].body);+ expect(body.messages).toEqual(turn);+expect(mockCreate).toHaveBeenCalledOnce();- // Messages should be converted back to Anthropic formatconst createArgs = mockCreate.mock.calls[0][0];- expect(createArgs.messages.length).toBeGreaterThan(0);+ expect(createArgs.messages).toEqual(compressed);});- it("flattens array-content tool_result blocks to text for the compression request", async () => {- mockFetch.mockResolvedValueOnce(mockCompressSuccess());+ it("keeps image blocks native in the compression request and in the forwarded messages", async () => {+ const turn = [+ {+ role: "user",+ content: [+ { type: "text", text: "what is this?" },+ {+ type: "image",+ source: { type: "base64", media_type: "image/png", data: "AAAA" },+ },+ ],+ },+ ];+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(turn));const mockCreate = vi.fn().mockResolvedValue({});const fakeClient = { messages: { create: mockCreate } };@@ -202,172 +233,55 @@ describe("withHeadroom (Anthropic)", () => {await wrapped.messages.create({model: "claude-sonnet-4-5-20250929",- messages: [- { role: "user", content: "run the query" },- {- role: "assistant",- content: [- { type: "tool_use", id: "tu_1", name: "query", input: { sql: "select 1" } },- ],- },- {- role: "user",- content: [- {- type: "tool_result",- tool_use_id: "tu_1",- content: [- { type: "text", text: "row 1" },- { type: "text", text: "row 2" },- ],- },- ],- },- ],+ messages: turn,max_tokens: 1024,});- // The proxy must receive the tool output as text, not a JSON-encoded block array.const body = JSON.parse(mockFetch.mock.calls[0][1].body);- expect(body.messages[2]).toEqual({- role: "tool",- content: "row 1\nrow 2",- tool_call_id: "tu_1",- });- });-- it("preserves image blocks instead of dropping them during compression", async () => {- mockFetch.mockResolvedValueOnce(mockCompressSuccess());-- const mockCreate = vi.fn().mockResolvedValue({});- const fakeClient = { messages: { create: mockCreate } };-- const wrapped = withHeadroom(fakeClient as any, {- baseUrl: "http://localhost:8787",- });+ expect(body.messages).toEqual(turn);- await wrapped.messages.create({- model: "claude-sonnet-4-5-20250929",- messages: [- {- role: "user",- content: [- { type: "text", text: "what is this?" },- {- type: "image",- source: { type: "base64", media_type: "image/png", data: "AAAA" },- },- ],- },- ],- max_tokens: 1024,- });-- // The image must not be silently dropped from the compression request.- const body = JSON.parse(mockFetch.mock.calls[0][1].body);- expect(body.messages).toHaveLength(1);- expect(body.messages[0].content).toEqual([- { type: "text", text: "what is this?" },- { type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },- ]);+ expect(mockCreate).toHaveBeenCalledOnce();+ const createArgs = mockCreate.mock.calls[0][0];+ expect(createArgs.messages).toEqual(turn);});- it("converts a retained image_url back to an Anthropic image block for the final messages.create call (image-only turn)", async () => {- // Proxy retains the image through compression (successful-compression path).- mockFetch.mockResolvedValueOnce(- mockCompressSuccess([- {- role: "user",- content: [- { type: "image_url", image_url: { url: "data:image/png;base64,AAAA" } },- ],- },- ]),- );-+ it("posts a thinking-enabled tool-use turn unchanged, thinking blocks and tool_result blocks included", async () => {+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(thinkingToolTurn));const mockCreate = vi.fn().mockResolvedValue({});- const fakeClient = { messages: { create: mockCreate } };-- const wrapped = withHeadroom(fakeClient as any, {+ const wrapped = withHeadroom({ messages: { create: mockCreate } } as any, {baseUrl: "http://localhost:8787",});await wrapped.messages.create({model: "claude-sonnet-4-5-20250929",- messages: [- {- role: "user",- content: [- {- type: "image",- source: { type: "base64", media_type: "image/png", data: "AAAA" },- },- ],- },- ],+ messages: thinkingToolTurn,max_tokens: 1024,});- expect(mockCreate).toHaveBeenCalledOnce();- const createArgs = mockCreate.mock.calls[0][0];- expect(createArgs.messages).toEqual([- {- role: "user",- content: [- { type: "image", source: { type: "base64", media_type: "image/png", data: "AAAA" } },- ],- },- ]);+ const body = JSON.parse(mockFetch.mock.calls[0][1].body);+ expect(body.messages).toEqual(thinkingToolTurn);+ expect(body.model).toBe("claude-sonnet-4-5-20250929");});- it("converts a retained mixed text/image turn back to Anthropic blocks, order preserved, for the final messages.create call", async () => {- mockFetch.mockResolvedValueOnce(- mockCompressSuccess([- {- role: "user",- content: [- { type: "text", text: "what is this?" },- { type: "image_url", image_url: { url: "data:image/jpeg;base64,BBBB" } },- ],- },- ]),- );-+ it("forwards the proxy's returned Anthropic messages to messages.create as-is", async () => {+ const compressed = structuredClone(thinkingToolTurn);+ (compressed[2].content[0] as any).content[0].text = "[3 rows]";+ mockFetch.mockResolvedValueOnce(mockCompressSuccess(compressed));const mockCreate = vi.fn().mockResolvedValue({});- const fakeClient = { messages: { create: mockCreate } };-- const wrapped = withHeadroom(fakeClient as any, {+ const wrapped = withHeadroom({ messages: { create: mockCreate } } as any, {baseUrl: "http://localhost:8787",});await wrapped.messages.create({model: "claude-sonnet-4-5-20250929",- messages: [- {- role: "user",- content: [- { type: "text", text: "what is this?" },- {- type: "image",- source: { type: "base64", media_type: "image/jpeg", data: "BBBB" },- },- ],- },- ],+ messages: thinkingToolTurn,max_tokens: 1024,});- expect(mockCreate).toHaveBeenCalledOnce();const createArgs = mockCreate.mock.calls[0][0];- expect(createArgs.messages).toEqual([- {- role: "user",- content: [- { type: "text", text: "what is this?" },- { type: "image", source: { type: "base64", media_type: "image/jpeg", data: "BBBB" } },- ],- },- ]);+ expect(createArgs.messages).toEqual(compressed);+ // The thinking blocks reach the provider exactly as the model produced them.+ expect(createArgs.messages[1].content.slice(0, 2)).toEqual(thinkingToolTurn[1].content.slice(0, 2));});it("returns original messages on compression fallback", async () => {diff --git a/sdk/typescript/test/compress.test.ts b/sdk/typescript/test/compress.test.tsindex 49ec2d934..78a4be809 100644--- a/sdk/typescript/test/compress.test.ts+++ b/sdk/typescript/test/compress.test.ts@@ -184,21 +184,54 @@ describe("compress() keeps Anthropic text documents through the public path", ()mockFetch.mockReset();});- it("sends a document-only turn as a media part and restores the document block", async () => {+ it("posts a document-only turn as the native document block and returns it unchanged", async () => {const doc = {type: "document",source: { type: "text", media_type: "text/plain", data: "Critical contract: do not delete account 123" },title: "Contract",citations: { enabled: true },};- // The proxy passes non-text parts through unchanged; echo the request's messages back.+ // The proxy passes non-text blocks through unchanged; echo the request's messages back.mockFetch.mockImplementationOnce(async (_url: string, init: any) => okResponse({ messages: JSON.parse(init.body).messages }));const result = await compress([{ role: "user", content: [doc] }], { baseUrl: "http://localhost:8787" });+ // Anthropic input is not converted: the document block itself reaches the proxy.const body = JSON.parse(mockFetch.mock.calls[0][1].body);- expect(body.messages).toHaveLength(1);- expect(body.messages[0].content[0].type).toBe("image_url");+ expect(body.messages).toEqual([{ role: "user", content: [doc] }]);expect(result.messages).toEqual([{ role: "user", content: [doc] }]);});});++describe("compress() sends Anthropic-format input to the proxy natively", () => {+ beforeEach(() => {+ mockFetch.mockReset();+ });++ const anthropicTurn = [+ { role: "user", content: "search for X" },+ {+ role: "assistant",+ content: [+ { type: "thinking", thinking: "Search first.", signature: "SIG-1" },+ { type: "tool_use", id: "toolu_1", name: "search", input: { q: "X" } },+ ],+ },+ {+ role: "user",+ content: [{ type: "tool_result", tool_use_id: "toolu_1", content: "a\nb\nc" }],+ },+ ];++ it("posts the Anthropic blocks as-is and returns the proxy's messages unconverted", async () => {+ const returned = structuredClone(anthropicTurn);+ (returned[2].content as any[])[0].content = "[3 lines]";+ mockFetch.mockResolvedValueOnce(okResponse({ messages: returned }));++ const result = await compress(anthropicTurn, { baseUrl: "http://localhost:8787" });++ const body = JSON.parse(mockFetch.mock.calls[0][1].body);+ expect(body.messages).toEqual(anthropicTurn);+ expect(result.messages).toEqual(returned);+ });+});diff --git a/sdk/typescript/test/utils/detect-format.test.ts b/sdk/typescript/test/utils/detect-format.test.tsindex d74bb8216..4b7a267b8 100644--- a/sdk/typescript/test/utils/detect-format.test.ts+++ b/sdk/typescript/test/utils/detect-format.test.ts@@ -185,3 +185,26 @@ describe("detectFormat: Anthropic document blocks", () => {).toBe("anthropic");});});++describe("detectFormat: Anthropic thinking blocks", () => {+ it("detects a turn whose only Anthropic marker is a thinking block", () => {+ const messages = [+ { role: "user", content: "hi" },+ {+ role: "assistant",+ content: [+ { type: "thinking", thinking: "...", signature: "SIG" },+ { type: "text", text: "hello" },+ ],+ },+ ];+ expect(detectFormat(messages)).toBe("anthropic");+ });++ it("detects a turn whose only Anthropic marker is a redacted_thinking block", () => {+ const messages = [+ { role: "assistant", content: [{ type: "redacted_thinking", data: "REDACTED" }, { type: "text", text: "hello" }] },+ ];+ expect(detectFormat(messages)).toBe("anthropic");+ });+});