Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
gateway: Codex gives Claude its signed thinking back through encrypted_content
yetone79c8667c9215
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.mdindex 62c0c83a2..adde7c18d 100644--- a/docs/subsystems/gateway-routing.md+++ b/docs/subsystems/gateway-routing.md@@ -219,11 +219,12 @@ recorded as a prediction failure (`noPrediction`).- A translated Chat stream's tool calls are read by `index`, not as one after another (`chatDecoder.toolCall` in [`chat.go`](../../internal/gateway/chat.go)). An upstream may send several calls' fragments by turns (index 0, 1, 0, 1…), the first of each with its id and name; read as a new call at each change of index, two calls reached Codex as six `function_call` items, the first two with `{}` and four with no name (mysterious_dove on Discord). The open call streams as it comes; a call that starts while the open one's arguments aren't yet whole JSON is held by its index and given whole, in index order, at `finish_reason` or `[DONE]`. Every translation from Chat (Responses, Messages, Gemini, a whole reply) reads the stream through it. `TestInterleavedToolCallsToResponses`, `TestInterleavedToolCallsToAnthropic`, `TestInterleavedToolCallsWhole`, `TestToolCallsNamedFirstThenArgued` in [`translate_stream_test.go`](../../internal/gateway/translate_stream_test.go).- A relayed Chat reply whose choice made calls and ended `stop` ends `tool_calls`, streamed (`chatTidy`) and whole (`chatWhole`), as the translations' collector already ends it (`collector.finish`): told it stopped, a client that reads `finish_reason` ends its turn with the calls not run (喜欢一个人 on Discord, a relay in front of deepseek-v4.1-flash). Any other reason stays as it came. `TestChatCallsEndToolCalls` in [`chattidy_finish_test.go`](../../internal/gateway/chattidy_finish_test.go).- Codex's model list from the backend path (`codexModels`) is the ChatGPT backend's for its sign-in and then magpie's (`provider.CodexListed`), its ETag and every reply's `X-Models-Etag` tagged with `provider.CodexListTag` so a change has Codex ask again. A Codex there only for account failover, not connected (`CodexOwnOnly`, `agent.CodexOwnOnly`; #1385), gets the backend's alone, tagged with `provider.CodexOwnListTag`; neither tag contains the other, so connecting or disconnecting has Codex fetch the list again. The user's picks on the codex provider (`provider.CodexNativePicked`) narrow the backend's listed models. They keep its hidden ones (`visibility: "hide"`), which no picker shows. One of those is `codex-auto-review`: when the list lacks it and no reviewer is set, Codex's auto-review runs on the conversation's model at low effort (#1460, `TestCodexModelListPicksKeepHidden`). The account's own models kept out of agents' lists for routing groups (`provider.CodexNativeUnlisted`) are dropped from it too, hidden ones kept: every one when the codex provider is "Only through routing groups" (`Provider.Unlisted`), and those a group that unlists its members names (`Group.Unlists`). A Codex there only for failover keeps them, as it is handed no groups; `spawnChoices` drops the same ones (#1602, `TestCodexModelListUnlistedForGroups`).-- Reasoning state goes through magpie's translation as the vendor wrote it when the agent and the upstream speak the same API (#1445, XYenon): a non-streaming Messages request to a stream-only account, web search on the agent's own API, a plugin, failover. The IR carries it sealed (`Part.Sealed`, `SealedBy`; events `KThinkStart`, `KSealed`, `KTextSig` in [`ir.go`](../../internal/gateway/ir.go)), so each thinking block or reasoning item stays apart, whole or streamed, and a seal is written only for its own API (`sealedBy`). Anthropic's `redacted_thinking` goes back raw (to DeepSeek as a thinking block, below), and a thinking block signed with no text keeps its signature (`aBlock` still writes `"thinking": ""`, #1447), on Claude's subscription too. A Responses reasoning item keeps its id, summary and `encrypted_content`, and goes back to a Responses upstream as it came when it has no reasoning text (`sealedReasoning`; text, another vendor's, is still left out for OpenAI, #1411). Gemini's `thoughtSignature` stays on the call it signed and on text; a later call of a signed step goes without one, and only an unsigned step or Claude on Antigravity gets `skip_thought_signature_validator`. Another API's agent gets the call's signature in its id, as on chat (#687), with its length in the mark (`__ts<n>__`), so an id a client shortened carries no signature and the call goes as unsigned (#1536); it gets no one else's seal. The signature itself is kept in magpie's folder, `gemini-signatures/<key>`, named by its hash (`keepSignature` in [`gemini_sigstore.go`](../../internal/gateway/gemini_sigstore.go)), and the id carries only the key (`__tsk16__<key>`), so an id stays under 64 characters however long the step's signature (#2411: LobeHub's PostgreSQL index turned away a 3KB id). A key magpie no longer has, or whose file doesn't hash to it, reads as no signature; one not read for 90 days is removed. Where the folder can't be written, the id carries the signature as before. Carrying reasoning across APIs (an envelope) isn't done.+- Reasoning state goes through magpie's translation as the vendor wrote it when the agent and the upstream speak the same API (#1445, XYenon): a non-streaming Messages request to a stream-only account, web search on the agent's own API, a plugin, failover. The IR carries it sealed (`Part.Sealed`, `SealedBy`; events `KThinkStart`, `KSealed`, `KTextSig` in [`ir.go`](../../internal/gateway/ir.go)), so each thinking block or reasoning item stays apart, whole or streamed, and a seal is written only for its own API (`sealedBy`). Anthropic's `redacted_thinking` goes back raw (to DeepSeek as a thinking block, below), and a thinking block signed with no text keeps its signature (`aBlock` still writes `"thinking": ""`, #1447), on Claude's subscription too. A Responses reasoning item keeps its id, summary and `encrypted_content`, and goes back to a Responses upstream as it came when it has no reasoning text (`sealedReasoning`; text, another vendor's, is still left out for OpenAI, #1411). Gemini's `thoughtSignature` stays on the call it signed and on text; a later call of a signed step goes without one, and only an unsigned step or Claude on Antigravity gets `skip_thought_signature_validator`. Another API's agent gets the call's signature in its id, as on chat (#687), with its length in the mark (`__ts<n>__`), so an id a client shortened carries no signature and the call goes as unsigned (#1536); it gets no one else's seal. The signature itself is kept in magpie's folder, `gemini-signatures/<key>`, named by its hash (`keepSignature` in [`gemini_sigstore.go`](../../internal/gateway/gemini_sigstore.go)), and the id carries only the key (`__tsk16__<key>`), so an id stays under 64 characters however long the step's signature (#2411: LobeHub's PostgreSQL index turned away a 3KB id). A key magpie no longer has, or whose file doesn't hash to it, reads as no signature; one not read for 90 days is removed. Where the folder can't be written, the id carries the signature as before. Across APIs, only Claude's reasoning is carried to a Responses client, below.+- Claude's reasoning goes to a Responses client (Codex) in an envelope and comes back as Claude wrote it (mysterious_dove_96933 on Discord). Anthropic wants a tool loop's next request to have the turn's thinking blocks, signed, and its `redacted_thinking` whole, or answers 400; Codex with `store: false` keeps only a reply's items, and of a reasoning item only `encrypted_content` is opaque. So the Anthropic decoders name a signature as Claude's (`KSig` with `sealAnthropic`, `Part.SignedBy`), and the Responses encoder and `renderResponses` give each thinking block a reasoning item whose `encrypted_content` is `magpie-claude1:` and the block's JSON in base64 (`claudeThinking`, [`internal/claudeseal`](../../internal/claudeseal/claudeseal.go)): its summary with it, an empty summary for a block signed with no text, and a `redacted_thinking` block as an item of its own. `parseResponses` reads such an item back into the block (`claudePart`), in order, so `anthropicBody` sends it as it was; an envelope that can't be read is its summary, as before. A Gemini thought's signature isn't named, and isn't sealed. The envelope goes to no Responses API: `passthrough` and `forwardOnce` take it out (`withoutClaudeSeals`: the item keeps its summary, one with none goes), `codexInputAs` leaves such an item out for Codex's own models as it does one with nothing sealed, and `codexUpstreamOn` takes it out before the ChatGPT backend; OpenAI's own `encrypted_content` is untouched. `redact.MaskJSON` keeps it whole (a reasoning item's seal), so the placeholders Claude was sent stay in what its signature covers. Claude's subscription names its signatures too, so Codex gets the same items, though its tool loop goes on inside Claude Code's own run, which keeps the thinking itself. `TestCodexToolLoopGivesClaudeItsThinkingBack`, `TestClaudeBlocksNeverGoToAResponsesAPI`, `TestGeminiSignatureIsNotSealedAsClaudes`, `TestMaskJSONKeepsClaudeBlocks`.- A translated Responses reply gives Codex the reasoning items a Responses upstream sent, as it sent them; any other upstream's reasoning becomes items magpie made: an `rs_` id and a summary, nothing sealed. Codex hands them back on every turn. A request relayed as is to OpenAI's API or Azure OpenAI goes without them (`withoutBareReasoning`) unless `store` is true, because both look the id up among stored items and answer 400 "Item with id 'rs_…' not found" (#1008). Any other Responses upstream gets them, because a DeepSeek model wants its reasoning back (#388). When one answers 400 naming the input (`refusesInput`: `param` is `input`, or the error names an item by id), as a relay in front of OpenAI does with a message of its own, `passthrough` asks once more without them and, once that works, leaves them out for that provider and model from then on (`bareReasoningRefused`, #1044). Codex's compaction offers no tools, so it is relayed even when its turns were translated. The summary request magpie makes offers none either: `codexInputAs` sends `tools: []` and leaves out the `additional_tools` input item, where Codex's Responses Lite (the entry of a ChatGPT account's gpt-6 model, copied from Codex's own) puts every tool; left in, the model may call one in place of the summary (the likely cause of WuSix's codex/gpt-6-astra "the model wrote no summary" on Discord, not confirmed on a real reply), and a reply with no summary says what it held instead (`answeredWith`). When a provider answers its summary request 404, 405 or 501 (`lacksEndpoint`), or 400 for an item it didn't find (`itemNotFound`, a relay in front of either), `codexCompact` asks once more with the conversation as text (`plainCompact`), then compacts locally (#866, #1008). Any other failure goes back with its status, said as the compaction's with who answered it (`compactFailed`: "compaction: <provider> (<model>, <base URL>) answered the summary request 502 Bad Gateway: …"), the vendor's code, param and Retry-After kept. A compaction relayed to OpenAI (`/responses/compact` on Codex's own model, as a `/responses` turn) that OpenAI answers with an error status and an empty body, which Codex shows as "unexpected status 502 Bad Gateway: Unknown error", or with a web page, is said in words naming the model, the status, and the account it went to (`codexSaidNothing`, outrageous_piglet on Discord); an error OpenAI says in JSON goes as it came. `TestCodexOwnCompactFailureIsSaid`, `TestCodexCompactFailureSaysCompaction`, `TestCodexCompactNoEndpointCompactsLocally`. So does the ChatGPT backend's 502 "response protection is unavailable" (`protectionRefused`): a long history refused so was answered as text. Codex also compacts remotely, a `compaction_trigger` on `/responses` that wants exactly one compaction item back, on any provider table named "OpenAI", as CC Switch names its relay tables. magpie points those at `/v1` while a magpie model is on, so `handle` makes the summary there too (`codexInput`, `codexCompact`), and magpie's summaries go back to the model as text (#1301). A summary is wanted only whole (`askWhole`, `holdWriter.wantWhole`): its stream is held to its end, not let through at its first words, so a reply cut short (`errEndedShort`), failed halfway (a 429 "Individual quota reached" after some of it) or ended `incomplete` (`endedIncomplete`, `incompleteReply`) fails over to the next candidate like a failure before any of it, and rests the account as that failure would; one that reached its output limit rests nobody. With nobody left, `codexCompact` answers 502 and never hands Codex the words before the cut as the summary (#2274, huangzuomin: Antigravity cutting each summary short). Codex with context_management's experimental mode compacts in the background at every step while the conversation is over its threshold and drops a failed compaction without a word, so a conversation whose summaries keep failing is asked again every few seconds, each paying for the whole history (1169 compactions in 6 hours there). After `compactFailsMost` (3) failed summaries in a row of one conversation (`prompt_cache_key`) on one model, each within `compactHeldFor` (5 minutes) of the last, the next is answered at once with a 400 naming the last failure, which Codex doesn't ask again, and no model is asked until 5 minutes have passed (`compactFails` in [`codex_compact_guard.go`](../../internal/gateway/codex_compact_guard.go)); a summary that comes clears the count, and a sign-in's 401 or 403 isn't counted. `TestCodexCompactCutSummaryFailsOver`, `TestCodexCompactCutSummaryIsNoSummary`, `TestCodexCompactKeepsFailingIsHeld`.- A reasoning item with its text in it (`content` parts: DeepSeek's Responses API writes `reasoning_text`, relayed to Codex as it came) is another vendor's. OpenAI's API and the ChatGPT backend take a reasoning item's `content` only empty, and refuse the whole request over one with 400 "Invalid 'input[n].content': array too long. Expected an array with maximum length 0" (`array_above_max_length`). Codex hands the item back on every later turn, so it never goes to them: for one of Codex's own models, `codexInput(body, false)` leaves it out (and a reasoning item with nothing sealed in it) on `/backend-api/codex`'s `/responses` and on `/responses/compact`, before Codex's own sign-in, its API key or a magpie Codex account (`compactOn`) is asked. Codex compacts a thread before its first turn back on its own model, and a refused compaction shows as that turn's error (jue on Discord). `passthrough` leaves it out for OpenAI, Azure OpenAI and a Codex account (`withoutReasoningText`), and for any other upstream once it has refused one (`refusesReasoningText`, `reasoningTextRefused`, #1411). OpenAI's own sealed reasoning (`encrypted_content`, no text) stays, and a DeepSeek model (`replaysReasoning`) still gets the text. `TestForeignReasoningNeverReachesOpenAI`, `TestReasoningTextLeftOutWhereRefused`, `TestReasoningTextNotSentToCodexAccount`.- Native compaction (#1099, off by default): a provider with `nativeCompaction` on (`provider.Provider.CompactsOn`; the editor's Compaction row, `magpie provider set <id> compaction=native`) is asked Codex's `compaction_trigger` as Codex sent it, and the compaction item it seals goes back to Codex unchanged, not as a `magpie1:` summary. Only a wire that carries it can be set so (`CarriesCompaction`): a Responses API, a ChatGPT account, GitHub Copilot (its models asked on Responses; its Claude models keep magpie's summary). The decision is made where `handle` and `codexBackend` read the input (`nativeCompactionFor`, `codexInputAs`): a group compacts natively only when every member does. Such a request, and a later one carrying a sealed compaction to a provider set so or to the provider recorded as its writer, is relayed, never translated, which would drop the item (`keepsCompaction` in `attempt`); a provider that can't be asked it on Responses answers a 400 saying so. The key or account that answered is recorded, whoever it is: a fallback or another member that answered the trigger sealed it whatever its own setting, and only it can read the seal (#2271, JasonBuildAI: a fallback's seal went unrecorded, and the next turn was answered 200 without the compacted history; `TestNativeCompactionFallbackSealStaysWithItsWriter`). It is recorded by the hash of what it sealed (`wroteCheckpoint`, `compactions.json` beside `affinity.json`, 0600, at most 1024 for 30 days). A later request carrying it goes only to that writer (`checkpointReaders`); a writer whose own list has since dropped the model is tried still, as the last resort an account whose list lacks the model is — it sealed the model's compaction, and no other can read it (`planned.unlisted`, `TestNativeCompactionWriterWhoseListLacksTheModel`, #2642: it was refused as "switched off, removed, or no longer serves" though it was on, and switching it on changed nothing). With the writer gone from the candidates, magpie answers 400 `checkpointError` with the reason its settings give (`checkpointGone`: the provider removed, the account signed out, its sign-in lapsed, paused while others are on, switched off, kept from the model in Names & levels, the key off or gone, the model disabled, or the model or group asked for not going to it), what lets the conversation go on, and whether native compaction is on for the writer's provider now, and never sends it to another key, drops it, or summarises it in its place (`TestNativeCompactionWriterGoneSaysWhy`). One of Codex's own models asked on the backend path is decided as a magpie model is when magpie's Codex accounts serve it (`codexAccounts`): the trigger is kept only with the codex provider's native compaction on, else `codexCompact` gives magpie's summary, which any account reads, and no writer is recorded (#2582, ump45nose; before, every such compaction was sealed natively whatever the setting, and the conversation was held to its account; `TestCodexOwnModelCompactionHonoursTheToggle`). Codex's own sign-in alone, relayed as it came, still compacts natively, unrecorded, as does `/responses/compact`; but a request carrying a compaction magpie recorded the writer of goes through routing even then (`codexSealedFor`), so with the writer switched off the one account left isn't sent it. A legacy `/responses/compact` carrying one is judged at its passthrough (`sealedCompact`): the writer on in magpie answers it signed as the writer — its own seal while the sign-in can be asked, a key's seal, a request signing with its own `chatgpt-account-id`, or an unreadable sign-in state relays as it came — and a writer gone is refused with `checkpointError`'s reason, the same words as on `/responses` (`TestNativeCompactionCompactJudgesItsWriter`, #2727). The writer matched is the account user the record holds, so two Team seats of one email read as one and such a compact relays as it came. A gateway key held to some accounts (#905) is asked of the account the compact goes to (`compactSigninHeld`): the writer when it answers it, so a writer the key may not use is refused 403 (`sealedCompactHeldError`) with nothing asked, and one it may use answers though the sign-in isn't the key's (`TestNativeCompactionCompactKeepsTheKeysAccounts`). With the writer held at its usage cap or for credits it may not spend, the answer is 429 `checkpointCappedError` (record `checkpointCappedRecord`, "native compaction's account at its usage cap"), naming that account, that the others serving the model weren't tried for the compaction, and that neither the routing mode nor how conversations stay changes that: wait for the window, raise or lift its cap, or start a new conversation (#2517, lm0711; before, it read "every account at its usage cap" though another had room; `TestNativeCompactionWriterAtItsCap`). `withoutCompaction` (the drop after a refused reasoning item) isn't used for such a request. A route carrying a compaction names its sealer (`Route.Compaction`), and Routing says that is why no other account was asked, not that it is "the only account on" for the model (#2639; `TestNativeCompactionRouteNamesItsWriter`, `routing-compaction.test.cjs`). A refusal of the trigger reaches Codex as the upstream sent it. Not tried against a real upstream; Copilot may refuse a history that holds Codex's `web_search` items. [`native_compaction.go`](../../internal/gateway/native_compaction.go), `TestNativeCompaction*`.-- A compaction magpie answers itself (`codexCompact`: a magpie model, Codex's own models through magpie's Codex accounts, a CC Switch table) goes to Codex as a `compaction` item whose `encrypted_content` is `magpie1:` and the summary in base64 (`codexsummary.Encrypted`), and `codexInputAs` sends Codex's own summary message in its place (`codexsummary.Message`: Codex's summary prefix, a newline and the summary, as a user message). Codex keeps the item in its rollout and replays it, and OpenAI refuses it once Codex talks to OpenAI itself (`invalid_encrypted_content`, LukeMa on X). So `sessions.PortCodexRollouts` writes that same message into the rollouts in its place ([`codex_portable.go`](../../internal/sessions/codex_portable.go)). It changes only magpie's items, keeps every other line's bytes and the file's line count, mode and time, and drops an unreadable item together with its `replacement_history_metadata` entry. It publishes as Codex publishes a compressed rollout: under `thread-writer-locks/.coordination.lock`, only with the thread's lock free, through a temp file and a rename. A thread a Codex has open is left for a later pass, and a Codex home without writer locks is left alone. It runs 2 minutes after the GUI or `magpie serve` starts and every 10 minutes after, on Codex Disconnect (3s), and as the GUI quits (3s). Each pass reads what changed since the last whole pass, plus what was held (`codex-portable.json`). A ported thread was resumed by Codex 0.162.1 against an upstream that refuses `magpie1:` as OpenAI does. Not tried: a Codex in a WSL distro, whose locks this machine can't take, so it is skipped. `TestPortCodexRollouts*`, `TestPortedCompactionReadsTheSame`.+- A compaction magpie answers itself (`codexCompact`: a magpie model, Codex's own models through magpie's Codex accounts, a CC Switch table) goes to Codex as a `compaction` item whose `encrypted_content` is `magpie1:` and the summary in base64 (`codexsummary.Encrypted`), and `codexInputAs` sends Codex's own summary message in its place (`codexsummary.Message`: Codex's summary prefix, a newline and the summary, as a user message). Codex keeps the item in its rollout and replays it, and OpenAI refuses it once Codex talks to OpenAI itself (`invalid_encrypted_content`, LukeMa on X). So `sessions.PortCodexRollouts` writes that same message into the rollouts in its place ([`codex_portable.go`](../../internal/sessions/codex_portable.go)). A reasoning item carrying Claude's block (`magpie-claude1:`, above) is written as its summary, as the gateway sends it to OpenAI (`TestPortLineDropsClaudeBlocks`). It changes only magpie's items, keeps every other line's bytes and the file's line count, mode and time, and drops an unreadable item together with its `replacement_history_metadata` entry. It publishes as Codex publishes a compressed rollout: under `thread-writer-locks/.coordination.lock`, only with the thread's lock free, through a temp file and a rename. A thread a Codex has open is left for a later pass, and a Codex home without writer locks is left alone. It runs 2 minutes after the GUI or `magpie serve` starts and every 10 minutes after, on Codex Disconnect (3s), and as the GUI quits (3s). Each pass reads what changed since the last whole pass, plus what was held (`codex-portable.json`). A ported thread was resumed by Codex 0.162.1 against an upstream that refuses `magpie1:` as OpenAI does. Not tried: a Codex in a WSL distro, whose locks this machine can't take, so it is skipped. `TestPortCodexRollouts*`, `TestPortedCompactionReadsTheSame`.- A Responses request Codex sends a Responses provider is fitted for a backend that isn't OpenAI's before it goes (`passthrough`): Responses Lite's shape taken off (`fromLite`): its `additional_tools` item lifted into `tools` (`liftAdditionalTools`, #350), its `reasoning.context` left out, and its `X-OpenAI-Internal-Codex-Responses-Lite` header not sent on (`liteKept`). A subagent a Lite lead (gpt-6.1-sol) spawns on another model comes in that shape, its model name alone changed, and OpenCode Go's DeepSeek answered it 503 "Endpoint is unavailable" (TangForX on X, `TestLiteSubagentLeavesLite`). Also the tool search as a function, OpenAI-only fields left out. A relay that checks a request is Codex's own turns that shape away (#1758: 400 "Codex 客户端请求特征无效", 10014). A provider set `codexAsSent` (`magpie provider set <id> codex=as-sent`) gets a request from Codex (`fromCodex`) as Codex sent it, its model name alone swapped, with Codex's own headers (`codexClientHeader`). That holds when Codex offers `web_search` too: the request is relayed with it, not translated for magpie's search (`searchedForIt`), which merged the developer messages into `instructions`, sent `apply_patch` as a function, left `web_search` out and added `max_output_tokens` (`codexAsSent` in `route`; maralica on Discord). `spawn_agent`'s description is still widened with the catalog's models (`spawnAgentModels`). Off is the default, and a subscription account is never sent so. `TestCodexAsSent`, `TestCodexAsSentWebSearch`.- A tool a Chat request's message carries joins the request's `tools` (`liftMessageTools` in [`message_tools.go`](../../internal/gateway/message_tools.go), #2322). Pi with `supportsMidConvoToolAdditions` gives a tool it adds mid-conversation as `{"role":"system","tools":[…]}` with no content, and keeps top-level `tools` as they were at the start; no Chat API reads tools there, and `parseChat` read only the top-level ones, so the tool could be called on no route. `handle` lifts them for every Chat body before `serveAgent`, so the Chat relay, the schema fixers, redaction (the tool is a tool definition, not message text), the prompt shown and routing all see them; `parseChat` lifts too, for a body that never came through `handle` (the Claude subscription bridge's direct callers). Each tool joins once by name and the first definition wins: a top-level tool keeps its own, then the earliest message's, as `liftAdditionalTools` and a `tool_addition` (`withoutDeferredTools`) do. A system or developer message that carried only tools is left out, so it is neither relayed as an empty system message nor joined into the system text as an empty part; one with content keeps its place without `tools`. Messages stay where they are: the in-place system message the issue also asks for is not this change. `TestPiAddedToolReachesEveryRoute` sends deephbz's Pi 1.2.0 bytes through the Chat relay, Messages, Responses, Gemini and Codex, and `TestPiAddedToolOfferedToClaudeCode` through the Claude subscription bridge.- A Chat request's tool in another API's shape is put in the Chat shape, `{"type":"function","function":{name, description, parameters, strict}}` (`shapeChatTools` in [`chat_tool_shape.go`](../../internal/gateway/chat_tool_shape.go), #2622, jinyue806): MCP's `{"name","description","inputSchema"}` (ZCode), Anthropic's `input_schema`, a bare `{"name","parameters"}` and Responses' flat `{"type":"function","name",…}`. No Chat API takes those (SenseNova: "field Tools[0].Type invalid, should be set" and "function cannot be empty"), and `parseChat` read no name from them, so a translated request offered a tool named "". It runs right after `liftMessageTools`, in `handle` for every Chat body and in `parseChat`, so it holds for every Chat upstream, not SenseNova's alone. A tool with its `function`, a hosted tool (any other `type`) and one with no name are left as they came, and a request with none to shape is relayed byte for byte. `TestChatToolInAnotherShapeIsShaped` sends the report's request with each of its four tools through the Chat relay, Messages and Responses; `TestShapeChatToolsLeavesChatShape`.diff --git a/internal/claudeseal/claudeseal.go b/internal/claudeseal/claudeseal.gonew file mode 100644index 000000000..2093fd767--- /dev/null+++ b/internal/claudeseal/claudeseal.go@@ -0,0 +1,47 @@+// Package claudeseal carries Claude's signed reasoning through a Responses+// client (mysterious_dove_96933 on Discord).+//+// Claude signs each thinking block and seals a redacted_thinking block+// whole, and Anthropic wants both back as they came on the next request+// of a tool loop. A Responses client such as Codex, with store off, keeps+// only what a reply's reasoning items carry, and of those only+// encrypted_content is opaque. So magpie writes the block there: "magpie-+// claude1:" and the block's JSON in base64, which only magpie reads. Sent+// to OpenAI it would be refused as invalid_encrypted_content, so the+// gateway drops it before any Responses upstream, and a rollout is made+// portable by dropping it there (sessions.PortCodexRollouts).+package claudeseal++import (+ "encoding/base64"+ "encoding/json"+ "strings"+)++// Mark begins the encrypted_content of a reasoning item carrying a+// Claude block.+const Mark = "magpie-claude1:"++// Seal is the encrypted_content carrying block, a Messages content block+// (thinking with its signature, or redacted_thinking).+func Seal(block []byte) string {+ return Mark + base64.StdEncoding.EncodeToString(block)+}++// Open is the block an encrypted_content carries: ours is false for one+// magpie didn't seal, and block nil for one of magpie's that can't be+// read.+func Open(encrypted string) (block json.RawMessage, ours bool) {+ b64, ok := strings.CutPrefix(encrypted, Mark)+ if !ok {+ return nil, false+ }+ b, err := base64.StdEncoding.DecodeString(b64)+ if err != nil || !json.Valid(b) {+ return nil, true+ }+ return b, true+}++// Is reports whether encrypted is one magpie sealed.+func Is(encrypted string) bool { return strings.HasPrefix(encrypted, Mark) }diff --git a/internal/gateway/anthropic.go b/internal/gateway/anthropic.goindex db6459990..249a417f7 100644--- a/internal/gateway/anthropic.go+++ b/internal/gateway/anthropic.go@@ -1295,7 +1295,7 @@ func decodeAnthropic(data string, emit func(Event)) error {emit(Event{Kind: KThink, Text: ev.ContentBlock.Thinking})}if ev.ContentBlock.Signature != "" {- emit(Event{Kind: KSig, Text: ev.ContentBlock.Signature})+ emit(Event{Kind: KSig, Name: sealAnthropic, Text: ev.ContentBlock.Signature})}case "redacted_thinking":// sealed whole: it goes on as it came, to Messages only@@ -1315,7 +1315,7 @@ func decodeAnthropic(data string, emit func(Event)) error {case "thinking_delta":emit(Event{Kind: KThink, Text: ev.Delta.Thinking})case "signature_delta":- emit(Event{Kind: KSig, Text: ev.Delta.Signature})+ emit(Event{Kind: KSig, Name: sealAnthropic, Text: ev.Delta.Signature})}case "message_delta":if ev.Delta.StopReason != "" {diff --git a/internal/gateway/claude_seal.go b/internal/gateway/claude_seal.gonew file mode 100644index 000000000..55de6e4d2--- /dev/null+++ b/internal/gateway/claude_seal.go@@ -0,0 +1,95 @@+package gateway++import (+ "bytes"+ "encoding/json"++ "github.com/tidwall/gjson"+ "github.com/yetone/magpie/internal/claudeseal"+)++// Claude's signed reasoning through a Responses client (mysterious_dove_96933+// on Discord): a reply's reasoning item carries Claude's thinking block,+// signature and all, or its redacted_thinking block, in encrypted_content+// (claudeseal), and the next request reads it back into the block as+// Claude wrote it, which Anthropic wants back on a tool loop's next turn.++// claudeThinking is the encrypted_content of a reasoning item for Claude's+// thinking block with its signature.+func claudeThinking(text, signature string) string {+ b, _ := json.Marshal(struct {+ Type string `json:"type"`+ Thinking string `json:"thinking"`+ Signature string `json:"signature"`+ }{"thinking", text, signature})+ return claudeseal.Seal(b)+}++// claudePart is the reasoning a block magpie sealed stands for: Claude's+// thinking, signed, or its redacted_thinking whole; summary (the item's+// own reasoning, as it is without one) when the block can't be read.+func claudePart(block json.RawMessage, summary Part) Part {+ var b struct {+ Type string `json:"type"`+ Thinking string `json:"thinking"`+ Signature string `json:"signature"`+ Data string `json:"data"`+ }+ if block == nil || decodeJSON(block, &b) != nil {+ return summary+ }+ switch {+ case b.Type == "thinking" && b.Signature != "":+ return Part{Kind: Thinking, Text: b.Thinking, Signature: b.Signature, SignedBy: sealAnthropic}+ case b.Type == "redacted_thinking" && b.Data != "":+ return Part{Kind: Thinking, Sealed: block, SealedBy: sealAnthropic}+ }+ return summary+}++// withoutClaudeSeals is a Responses request body with no Claude block in+// its reasoning items, for a Responses API: OpenAI would refuse the+// envelope as invalid_encrypted_content, and Claude's thinking is no other+// vendor's to read. An item keeps its summary, as it was before magpie+// carried the block; one with none (a block signed with no text, or+// redacted) goes. Every other item, and a body with none, goes byte for+// byte.+func withoutClaudeSeals(body []byte) []byte {+ if !bytes.Contains(body, []byte(claudeseal.Mark)) {+ return body+ }+ var q map[string]json.RawMessage+ var input []json.RawMessage+ if decodeJSON(body, &q) != nil || decodeJSON(q["input"], &input) != nil {+ return body+ }+ out := make([]json.RawMessage, 0, len(input))+ changed := false+ for _, it := range input {+ r := gjson.ParseBytes(it)+ if r.Get("type").String() != "reasoning" || !claudeseal.Is(r.Get("encrypted_content").String()) {+ out = append(out, it)+ continue+ }+ changed = true+ if len(r.Get("summary").Array()) == 0 && len(r.Get("content").Array()) == 0 {+ continue+ }+ var m map[string]json.RawMessage+ if decodeJSON(it, &m) != nil {+ continue+ }+ delete(m, "encrypted_content")+ b, _ := json.Marshal(m)+ out = append(out, b)+ }+ if !changed {+ return body+ }+ q["input"], _ = json.Marshal(out)+ b, err := json.Marshal(q)+ if err != nil {+ return body+ }+ return b+}diff --git a/internal/gateway/claude_subscription.go b/internal/gateway/claude_subscription.goindex 56cbb5150..8304cf700 100644--- a/internal/gateway/claude_subscription.go+++ b/internal/gateway/claude_subscription.go@@ -2506,7 +2506,7 @@ func (r *subscriptionRun) readOutput(rd io.Reader) {r.emit(Event{Kind: KThink, Text: c.Thinking})}if c.Signature != "" {- r.emit(Event{Kind: KSig, Text: c.Signature})+ r.emit(Event{Kind: KSig, Name: sealAnthropic, Text: c.Signature})}case "text":said(c.Text)@@ -2585,7 +2585,7 @@ func (r *subscriptionRun) readOutput(rd io.Reader) {case "thinking_delta":r.emit(Event{Kind: KThink, Text: e.Delta.Thinking})case "signature_delta":- r.emit(Event{Kind: KSig, Text: e.Delta.Signature})+ r.emit(Event{Kind: KSig, Name: sealAnthropic, Text: e.Delta.Signature})case "input_json_delta":if !own[e.Index] {r.emit(Event{Kind: KToolArgs, Text: e.Delta.PartialJSON})diff --git a/internal/gateway/codex_backend.go b/internal/gateway/codex_backend.goindex e7d7e8e63..c58d15ee6 100644--- a/internal/gateway/codex_backend.go+++ b/internal/gateway/codex_backend.go@@ -20,6 +20,7 @@ import ("github.com/yetone/magpie/internal/access""github.com/yetone/magpie/internal/catalog"+ "github.com/yetone/magpie/internal/claudeseal""github.com/yetone/magpie/internal/codexcat""github.com/yetone/magpie/internal/codexsummary""github.com/yetone/magpie/internal/provider"@@ -897,6 +898,9 @@ func (s *Server) codexUpstreamOn(w http.ResponseWriter, r *http.Request, rest ststart := time.Now()usage.Saw(agentOf(r))w0, body0 := w, body // as they came, for the fallbacks (codexFallBack)+ // Claude's blocks magpie gave Codex (claudeseal) are Claude's alone:+ // to OpenAI each item is its summary, as before magpie carried them+ body = withoutClaudeSeals(body)var maskedHits maskHitsif r.Method == http.MethodPost {var unmask func()@@ -2102,7 +2106,10 @@ func codexInputAs(body []byte, magpieModel, native bool) (_ []byte, compact bool// Nor reasoning with nothing sealed in it — another vendor's,// only an id (rs_…): OpenAI, which keeps nothing (store is// false), looks the id up and answers 404 "Item … not found".- if enc, _ := it["encrypted_content"].(string); enc == "" {+ // Nor Claude's block magpie carried in one (claudeseal),+ // which OpenAI would refuse as invalid_encrypted_content:+ // before magpie carried it, the item had nothing sealed+ if enc, _ := it["encrypted_content"].(string); enc == "" || claudeseal.Is(enc) {changed = truecontinue}@@ -2179,7 +2186,7 @@ func codexInputItemChanges(it []byte, magpieModel, native bool) bool {}}enc, _ := stringBytes(memberOrNil(it, "encrypted_content"))- return len(enc) == 0+ return len(enc) == 0 || hasPrefix(enc, claudeseal.Mark)}return false}diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.goindex 71ea0e615..d17c50d1e 100644--- a/internal/gateway/gateway.go+++ b/internal/gateway/gateway.go@@ -4126,6 +4126,10 @@ func (s *Server) forwardOnce(ctx context.Context, p provider.Provider, to providbody = kimiLockedSampling(p, to, body)body = clinePin(p, to, body)body = codexImagesFit(p, to, body)+ if to == provider.Responses {+ // whatever the route, Claude's blocks never reach a Responses API+ body = withoutClaudeSeals(body)+ }if to == provider.Responses && !keepsAccessPrograms(p, in) {// Codex's Cyber program, relayed or built, goes to OpenAI alone// (#3172); before the user's own fields, which may name one@@ -4405,6 +4409,11 @@ func codexClientHeader(k string) bool {func (s *Server) passthrough(w http.ResponseWriter, r *http.Request, p provider.Provider, proto provider.Protocol, model string, body []byte, u *Usage) (status int, msg string, done bool) {upstream := provider.UpstreamNameIn(wiresOf(r.Context()), p.ID, model)body = rewriteModel(body, upstream)+ if proto == provider.Responses {+ // Claude's blocks magpie gave a Responses client go back to Claude+ // alone; to this API the item is its summary, as before+ body = withoutClaudeSeals(body)+ }// a client that asked for one body, of an account that only streams:// asked streamed, the stream read here (gathered)var gather *gathereddiff --git a/internal/gateway/ir.go b/internal/gateway/ir.goindex 080a37e7d..2909c4fce 100644--- a/internal/gateway/ir.go+++ b/internal/gateway/ir.go@@ -64,6 +64,11 @@ type Part struct {// comes only from Google, as gemini_signature.go carries it, and// text's only from a Gemini API (#1445)Signature string+ // SignedBy is the API that signed a thinking Part (sealAnthropic for+ // Claude's), empty when it isn't known: only Claude's signature goes+ // to a Responses client, in its reasoning item's encrypted_content+ // (claudeseal)+ SignedBy string// thinking an upstream sealed, as its API wrote it: Anthropic's// redacted_thinking block, a Responses reasoning item with its id,// summary and encrypted_content. Only that API's builder, encoder and@@ -302,7 +307,7 @@ const (KStart EventKind = iota // MsgID, Model, Usage (input side)KText // TextKThink // Text (reasoning)- KSig // Text (thinking signature)+ KSig // Text (thinking signature), Name the API that signed it if knownKToolStart // ID, NameKToolArgs // Text (partial JSON of the arguments)KStop // Stop@@ -601,9 +606,12 @@ func (c *collector) add(ev Event) {// display is omitted) is a block all the same (#1445)if p := c.open(); p != nil {p.Signature += ev.Text+ if ev.Name != "" {+ p.SignedBy = ev.Name+ }} else if c.fresh {c.closeTool()- c.res.Parts = append(c.res.Parts, Part{Kind: Thinking, Signature: ev.Text})+ c.res.Parts = append(c.res.Parts, Part{Kind: Thinking, Signature: ev.Text, SignedBy: ev.Name})}case KSealed:// a Responses item's summary came before it as reasoning of itsdiff --git a/internal/gateway/reasoning_state_claude_codex_test.go b/internal/gateway/reasoning_state_claude_codex_test.gonew file mode 100644index 000000000..57ed2e1f8--- /dev/null+++ b/internal/gateway/reasoning_state_claude_codex_test.go@@ -0,0 +1,275 @@+package gateway++import (+ "encoding/json"+ "io"+ "net/http"+ "net/http/httptest"+ "strings"+ "sync"+ "testing"++ "github.com/yetone/magpie/internal/claudeseal"+ "github.com/yetone/magpie/internal/provider"+)++// Codex calling a Claude model through magpie (mysterious_dove_96933 on+// Discord): Claude's thinking is signed, and a redacted_thinking block is+// sealed whole, and Anthropic wants both back intact on the next request+// of a tool loop. Codex keeps only what a reply's reasoning items carry,+// so they carry Claude's blocks in encrypted_content, and the next+// request has them back, in order, as Claude wrote them.++// claudeToolTurn is a Messages stream as Anthropic's extended thinking+// docs give one ending in a tool call: a signed thinking block, a+// redacted_thinking block, a block signed with no text (display omitted),+// then the call.+var claudeToolTurn = []string{+ `{"type":"message_start","message":{"id":"msg_01","type":"message","role":"assistant","model":"claude-sonnet-4-5","content":[],"usage":{"input_tokens":10,"output_tokens":1}}}`,+ `{"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":"","signature":""}}`,+ `{"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"I should list "}}`,+ `{"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"the files."}}`,+ `{"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"ErUBCkYIBxgCIkB2signed"}}`,+ `{"type":"content_block_stop","index":0}`,+ `{"type":"content_block_start","index":1,"content_block":` + redactedBlock + `}`,+ `{"type":"content_block_stop","index":1}`,+ `{"type":"content_block_start","index":2,"content_block":{"type":"thinking","thinking":"","signature":""}}`,+ `{"type":"content_block_delta","index":2,"delta":{"type":"signature_delta","signature":"EqQBCkYIBxgCKkDomitted"}}`,+ `{"type":"content_block_stop","index":2}`,+ `{"type":"content_block_start","index":3,"content_block":{"type":"tool_use","id":"toolu_01","name":"shell","input":{}}}`,+ `{"type":"content_block_delta","index":3,"delta":{"type":"input_json_delta","partial_json":"{\"command\":[\"ls\"]}"}}`,+ `{"type":"content_block_stop","index":3}`,+ `{"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":30}}`,+ `{"type":"message_stop"}`,+}++// claudeTurnBlocks are the blocks claudeToolTurn's message is made of.+var claudeTurnBlocks = []string{+ `{"type":"thinking","thinking":"I should list the files.","signature":"ErUBCkYIBxgCIkB2signed"}`,+ redactedBlock,+ `{"type":"thinking","thinking":"","signature":"EqQBCkYIBxgCKkDomitted"}`,+ `{"type":"tool_use","id":"toolu_01","name":"shell","input":{"command":["ls"]}}`,+}++// claudeUp answers a request without a tool result with claudeToolTurn,+// and one with a result with a line of text, keeping what it was sent.+type claudeUp struct {+ mu sync.Mutex+ sent []string+}++func (u *claudeUp) ServeHTTP(w http.ResponseWriter, r *http.Request) {+ b, _ := io.ReadAll(r.Body)+ u.mu.Lock()+ u.sent = append(u.sent, string(b))+ u.mu.Unlock()+ w.Header().Set("Content-Type", "text/event-stream")+ lines := claudeToolTurn+ if strings.Contains(string(b), `"tool_result"`) {+ lines = []string{+ `{"type":"message_start","message":{"id":"msg_02","type":"message","role":"assistant","model":"claude-sonnet-4-5","content":[],"usage":{"input_tokens":10,"output_tokens":1}}}`,+ `{"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}`,+ `{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Two files."}}`,+ `{"type":"content_block_stop","index":0}`,+ `{"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":3}}`,+ `{"type":"message_stop"}`,+ }+ }+ for _, l := range lines {+ var ev struct {+ Type string `json:"type"`+ }+ json.Unmarshal([]byte(l), &ev)+ io.WriteString(w, "event: "+ev.Type+"\ndata: "+l+"\n\n")+ }+}++// codexLoopAsk is a request as Codex sends one with store off: the reasoning+// items of a reply come back in input with their encrypted_content.+func codexLoopAsk(stream bool, input string) string {+ s := "false"+ if stream {+ s = "true"+ }+ return `{"model":"a/claude-sonnet-4-5","instructions":"You are a coding agent.","input":` + input + `,` ++ `"tools":[{"type":"function","name":"shell","description":"Runs a command","strict":false,"parameters":{"type":"object","properties":{"command":{"type":"array","items":{"type":"string"}}},"required":["command"]}}],` ++ `"tool_choice":"auto","parallel_tool_calls":false,"reasoning":{"effort":"medium","summary":"auto"},"store":false,"stream":` + s ++ `,"include":["reasoning.encrypted_content"],"prompt_cache_key":"t1"}`+}++// replyItems are a Responses reply's output items, from its+// response.output_item.done events when it streamed.+func replyItems(t *testing.T, body []byte, stream bool) []map[string]any {+ t.Helper()+ var items []map[string]any+ if !stream {+ var out struct {+ Output []map[string]any `json:"output"`+ }+ if err := json.Unmarshal(body, &out); err != nil {+ t.Fatalf("%v: %s", err, body)+ }+ return out.Output+ }+ for _, ev := range sseData(t, body) {+ if ev["type"] == "response.output_item.done" {+ items = append(items, ev["item"].(map[string]any))+ }+ }+ return items+}++// The tool loop as Codex runs it against a Claude model on an Anthropic+// API key: the second request reaches Anthropic with the first reply's+// signed thinking, redacted_thinking and text-less signed block, in+// order and as Claude wrote them, before the tool_use, streamed or not.+func TestCodexToolLoopGivesClaudeItsThinkingBack(t *testing.T) {+ for _, stream := range []bool{true, false} {+ fresh(t)+ up := &claudeUp{}+ srv := httptest.NewServer(up)+ t.Cleanup(srv.Close)+ if err := provider.Save(provider.Provider{ID: "a", Name: "A", Key: "k", Anthropic: srv.URL, Models: []string{"claude-sonnet-4-5"}}); err != nil {+ t.Fatal(err)+ }+ s := New()+ user := `{"type":"message","role":"user","content":[{"type":"input_text","text":"What is here?"}]}`+ rec := httptest.NewRecorder()+ s.Handler().ServeHTTP(rec, httptest.NewRequest("POST", "/v1/responses", strings.NewReader(codexLoopAsk(stream, "["+user+"]"))))+ if rec.Code != 200 {+ t.Fatalf("stream=%t turn 1: %d %s", stream, rec.Code, rec.Body)+ }+ // Codex keeps each item as it came, less a reasoning item's id+ // (it doesn't send one back with store off), then the call's output+ input := []string{user}+ var call string+ for _, it := range replyItems(t, rec.Body.Bytes(), stream) {+ delete(it, "status")+ if it["type"] == "reasoning" {+ delete(it, "id")+ }+ if it["type"] == "function_call" {+ call, _ = it["call_id"].(string)+ }+ b, _ := json.Marshal(it)+ input = append(input, string(b))+ }+ if call == "" {+ t.Fatalf("stream=%t: no call in %s", stream, rec.Body)+ }+ input = append(input, `{"type":"function_call_output","call_id":"`+call+`","output":"a.go\nb.go"}`)+ rec = httptest.NewRecorder()+ s.Handler().ServeHTTP(rec, httptest.NewRequest("POST", "/v1/responses", strings.NewReader(codexLoopAsk(stream, "["+strings.Join(input, ",")+"]"))))+ if rec.Code != 200 || !strings.Contains(rec.Body.String(), "Two files.") {+ t.Fatalf("stream=%t turn 2: %d %s", stream, rec.Code, rec.Body)+ }+ up.mu.Lock()+ last := up.sent[len(up.sent)-1]+ up.mu.Unlock()+ var q struct {+ Messages []struct {+ Role string `json:"role"`+ Content []json.RawMessage `json:"content"`+ } `json:"messages"`+ }+ if err := json.Unmarshal([]byte(last), &q); err != nil {+ t.Fatal(err)+ }+ var got []json.RawMessage+ for _, m := range q.Messages {+ if m.Role == "assistant" {+ got = append(got, m.Content...)+ }+ }+ if len(got) != len(claudeTurnBlocks) {+ t.Fatalf("stream=%t: Anthropic was sent %d assistant blocks, want %d:\n%s", stream, len(got), len(claudeTurnBlocks), last)+ }+ for i, w := range claudeTurnBlocks {+ var g map[string]any+ json.Unmarshal(got[i], &g)+ delete(g, "cache_control")+ if !jsonIs(t, g, w) {+ t.Fatalf("stream=%t: block %d:\n got %s\nwant %s\nsent %s", stream, i, got[i], w, last)+ }+ }+ }+}++// claudeConversation is a Codex conversation that went through Claude and+// now goes on with another model: a reasoning item with a summary and+// Claude's block, one with Claude's block alone, and one OpenAI sealed.+func claudeConversation() string {+ think := claudeThinking("I should list the files.", "ErUBCkYIBxgCIkB2signed")+ redacted := claudeseal.Seal([]byte(redactedBlock))+ return `{"type":"message","role":"user","content":[{"type":"input_text","text":"What is here?"}]},` ++ `{"type":"reasoning","summary":[{"type":"summary_text","text":"I should list the files."}],"encrypted_content":"` + think + `"},` ++ `{"type":"reasoning","summary":[],"encrypted_content":"` + redacted + `"},` ++ `{"type":"function_call","call_id":"call_1","name":"shell","arguments":"{\"command\":[\"ls\"]}"},` ++ `{"type":"function_call_output","call_id":"call_1","output":"a.go"},` ++ `{"type":"reasoning","id":"rs_openai","summary":[],"encrypted_content":"gAAAA-openai-sealed"},` ++ `{"type":"message","role":"user","content":[{"type":"input_text","text":"and now?"}]}`+}++// Claude's blocks are Claude's: a Responses API (a relay here, the+// ChatGPT backend for Codex's own models) is sent each item as it was+// before magpie carried them, its summary alone or nothing, and never the+// envelope, which OpenAI would refuse as invalid_encrypted_content. What+// OpenAI sealed goes on untouched.+func TestClaudeBlocksNeverGoToAResponsesAPI(t *testing.T) {+ fresh(t)+ u := &relayUpstream{}+ saveRelay(t, u)+ body := `{"model":"relay/gpt-5.4","stream":true,"store":false,"include":["reasoning.encrypted_content"],"input":[` + claudeConversation() + `]}`+ if rec := codexPostTo(New(), body); rec.Code != 200 {+ t.Fatalf("%d %s", rec.Code, rec.Body)+ }+ asked := u.asked()+ if len(asked) != 1 {+ t.Fatalf("relay asked %d times", len(asked))+ }+ sent, _ := json.Marshal(asked[0])+ if strings.Contains(string(sent), claudeseal.Mark) || !strings.Contains(string(sent), "gAAAA-openai-sealed") {+ t.Fatalf("relay was sent %s", sent)+ }+ if !strings.Contains(string(sent), "I should list the files.") {+ t.Fatalf("the summary went missing: %s", sent)+ }++ // Codex's own model on the ChatGPT backend+ out, _ := codexInputAs([]byte(`{"model":"gpt-5.4","input":[`+claudeConversation()+`]}`), false, false)+ if strings.Contains(string(out), claudeseal.Mark) || !strings.Contains(string(out), "gAAAA-openai-sealed") {+ t.Fatalf("ChatGPT backend would be sent %s", out)+ }+ if strings.Count(string(out), `"reasoning"`) != 1 {+ t.Fatalf("want only OpenAI's reasoning item: %s", out)+ }++ // and nothing to take out, the body goes byte for byte+ plain := []byte(`{"input":[{"type":"reasoning","summary":[],"encrypted_content":"gAAAA"}],"x":1}`)+ if got := withoutClaudeSeals(plain); string(got) != string(plain) {+ t.Fatalf("%s became %s", plain, got)+ }+}++// Only Claude's signature goes in encrypted_content: a Gemini thought's+// (an unnamed KSig, or a Part with no SignedBy) isn't sealed as Claude's,+// streamed or not.+func TestGeminiSignatureIsNotSealedAsClaudes(t *testing.T) {+ rec := httptest.NewRecorder()+ enc := &responsesEncoder{w: newSSEWriter(rec), model: "m"}+ enc.event(Event{Kind: KThinkStart})+ enc.event(Event{Kind: KThink, Text: "thought"})+ enc.event(Event{Kind: KSig, Text: "gemini-sig"})+ enc.event(Event{Kind: KThinkStart})+ enc.event(Event{Kind: KSig, Text: "gemini-bare"})+ enc.event(Event{Kind: KText, Text: "hi"})+ enc.event(Event{Kind: KStop, Stop: "stop"})+ enc.finish()+ if strings.Contains(rec.Body.String(), "encrypted_content") || strings.Contains(rec.Body.String(), "gemini-") {+ t.Fatalf("stream: %s", rec.Body)+ }+ b := renderResponses(Result{Parts: []Part{{Kind: Thinking, Text: "thought", Signature: "gemini-sig"}, {Kind: Thinking, Signature: "gemini-bare"}, {Kind: Text, Text: "hi"}}}, "m", nil)+ if strings.Contains(string(b), "encrypted_content") || strings.Contains(string(b), "gemini-") {+ t.Fatalf("render: %s", b)+ }+}diff --git a/internal/gateway/responses.go b/internal/gateway/responses.goindex 2e219558d..ccc7880f1 100644--- a/internal/gateway/responses.go+++ b/internal/gateway/responses.go@@ -11,6 +11,7 @@ import ("time""github.com/tidwall/gjson"+ "github.com/yetone/magpie/internal/claudeseal""github.com/yetone/magpie/internal/provider")@@ -439,7 +440,13 @@ func readResponses(body []byte, facts bool) (*Request, error) {}}part := Part{Kind: Thinking, Text: b.String()}- if it.EncryptedContent != "" {+ if block, ours := claudeseal.Open(it.EncryptedContent); ours {+ // Claude's block, which magpie gave the client to carry+ // back: as Claude wrote it, and never to a Responses API+ // (withoutClaudeSeals). One that can't be read is its+ // summary, as before.+ part = claudePart(block, part)+ } else if it.EncryptedContent != "" {// sealed: the item goes back to a Responses API as it// came, id, summary and all (#1445)if rawItems == nil {@@ -449,7 +456,7 @@ func readResponses(body []byte, facts bool) (*Request, error) {part.Sealed, part.SealedBy = rawItems[i], sealResponses}}- if b.Len() > 0 || part.Sealed != nil {+ if part.Text != "" || part.Sealed != nil || part.Signature != "" {r.Messages = append(r.Messages, Message{Role: "assistant", Parts: []Part{part}})}}@@ -1689,9 +1696,14 @@ type responsesEncoder struct {// opens an item of its own, by thinkID when the upstream named itthinkNew boolthinkID string- output []any- col collector- named map[string]nsTool // the request's namespaced tools+ // sig is Claude's signature of the open reasoning item's block, which+ // its encrypted_content carries back (claudeseal); bare is an item of+ // a block signed with no text, opened with no summary part+ sig strings.Builder+ bare bool+ output []any+ col collector+ named map[string]nsTool // the request's namespaced tools// call is the open function_call item's id and name, kept here: the// collector's last part isn't it once a hidden thinking block (signed// with no text, or redacted) has come after the call (#1786)@@ -1845,10 +1857,19 @@ func (e *responsesEncoder) closeAs(sealed json.RawMessage) {item, e.heldMsg = m, mcase Thinking:t := e.text.String()- e.send("response.reasoning_summary_text.done", map[string]any{"item_id": e.itemID, "output_index": e.item, "summary_index": 0, "text": t})- part := map[string]any{"type": "summary_text", "text": t}- e.send("response.reasoning_summary_part.done", map[string]any{"item_id": e.itemID, "output_index": e.item, "summary_index": 0, "part": part})- item = map[string]any{"id": e.itemID, "type": "reasoning", "status": "completed", "summary": []map[string]any{part}}+ summary := []map[string]any{}+ if !e.bare {+ e.send("response.reasoning_summary_text.done", map[string]any{"item_id": e.itemID, "output_index": e.item, "summary_index": 0, "text": t})+ part := map[string]any{"type": "summary_text", "text": t}+ e.send("response.reasoning_summary_part.done", map[string]any{"item_id": e.itemID, "output_index": e.item, "summary_index": 0, "part": part})+ summary = append(summary, part)+ }+ m := map[string]any{"id": e.itemID, "type": "reasoning", "status": "completed", "summary": summary}+ if e.sig.Len() > 0 {+ // Claude's block, signed, for the next request to give back+ m["encrypted_content"] = claudeThinking(t, e.sig.String())+ }+ item = mif sealed != nil {item = sealed}@@ -1871,6 +1892,7 @@ func (e *responsesEncoder) closeAs(sealed json.RawMessage) {e.send("response.output_item.done", map[string]any{"output_index": e.item, "item": item})e.output = append(e.output, item)e.open, e.text = "", strings.Builder{}+ e.sig, e.bare = strings.Builder{}, false}func (e *responsesEncoder) openItem(k Kind, prefix string, item map[string]any) {@@ -1911,8 +1933,11 @@ func (e *responsesEncoder) event(ev Event) {if ev.Text == "" {return}- if e.open != Thinking || e.thinkNew {- e.openItem(Thinking, "rs_", map[string]any{"type": "reasoning", "summary": []any{}})+ if e.open != Thinking || e.thinkNew || e.bare {+ if e.open != Thinking || e.thinkNew {+ e.openItem(Thinking, "rs_", map[string]any{"type": "reasoning", "summary": []any{}})+ }+ e.bare = falsee.send("response.reasoning_summary_part.added", map[string]any{"item_id": e.itemID, "output_index": e.item, "summary_index": 0,"part": map[string]any{"type": "summary_text", "text": ""}})}@@ -1923,7 +1948,32 @@ func (e *responsesEncoder) event(ev Event) {if strings.HasPrefix(ev.ID, "rs_") {e.thinkID = ev.ID}+ case KSig:+ if ev.Name != sealAnthropic || ev.Text == "" {+ break // Gemini's, which goes back on its own path+ }+ if e.open != Thinking || e.thinkNew {+ // a block signed with no text in it: an item all the same,+ // with no summary, for its signature to come back+ e.openItem(Thinking, "rs_", map[string]any{"type": "reasoning", "summary": []any{}})+ e.bare = true+ }+ e.sig.WriteString(ev.Text)case KSealed:+ if ev.Name == sealAnthropic {+ // Claude's redacted_thinking: an item with no summary, the+ // block whole in its encrypted_content+ e.closeItem()+ e.item+++ e.thinkNew, e.thinkID = false, ""+ id := "rs_" + newID()+ e.send("response.output_item.added", map[string]any{"output_index": e.item,+ "item": map[string]any{"id": id, "type": "reasoning", "status": "in_progress", "summary": []any{}}})+ item := map[string]any{"id": id, "type": "reasoning", "status": "completed", "summary": []any{}, "encrypted_content": claudeseal.Seal([]byte(ev.Text))}+ e.send("response.output_item.done", map[string]any{"output_index": e.item, "item": item})+ e.output = append(e.output, item)+ break+ }if ev.Name != sealResponses {break // another API's, which a Responses client can't read}@@ -2059,11 +2109,25 @@ func renderResponses(res Result, model string, named map[string]nsTool) []byte {output = append(output, p.Sealed)continue}- if p.Text == "" {+ if p.sealedBy(sealAnthropic) {+ // Claude's redacted_thinking, whole, to come back (claudeseal)+ output = append(output, map[string]any{"id": "rs_" + newID(), "type": "reasoning", "status": "completed",+ "summary": []any{}, "encrypted_content": claudeseal.Seal(p.Sealed)})+ continue+ }+ claude := p.SignedBy == sealAnthropic && p.Signature != ""+ if p.Text == "" && !claude {continue // another API's sealed thinking, or only its signature}- output = append(output, map[string]any{"id": "rs_" + newID(), "type": "reasoning", "status": "completed",- "summary": []map[string]any{{"type": "summary_text", "text": p.Text}}})+ summary := []map[string]any{}+ if p.Text != "" {+ summary = append(summary, map[string]any{"type": "summary_text", "text": p.Text})+ }+ item := map[string]any{"id": "rs_" + newID(), "type": "reasoning", "status": "completed", "summary": summary}+ if claude {+ item["encrypted_content"] = claudeThinking(p.Text, p.Signature)+ }+ output = append(output, item)case ToolCall:id := p.IDif id == "" {diff --git a/internal/gateway/responses_hidden_thinking_test.go b/internal/gateway/responses_hidden_thinking_test.goindex 37d06d61c..6606fb9b6 100644--- a/internal/gateway/responses_hidden_thinking_test.go+++ b/internal/gateway/responses_hidden_thinking_test.go@@ -8,6 +8,7 @@ import ("strings""testing"+ "github.com/yetone/magpie/internal/claudeseal""github.com/yetone/magpie/internal/provider")@@ -66,7 +67,7 @@ func TestResponsesStreamToolThenHiddenThinking(t *testing.T) {}var completed map[string]anyvar call map[string]any- var reasoning int+ var reasoning []map[string]anyfor _, ev := range events(string(body)) {switch ev["type"] {case "response.completed":@@ -77,7 +78,7 @@ func TestResponsesStreamToolThenHiddenThinking(t *testing.T) {case "function_call":call = itemcase "reasoning":- reasoning+++ reasoning = append(reasoning, item)}}}@@ -88,8 +89,19 @@ func TestResponsesStreamToolThenHiddenThinking(t *testing.T) {b, _ := json.Marshal(call)t.Fatalf("function_call = %s, want toolu_01 shell {\"command\":[\"ls\"]}\n%s", b, body)}- if want := map[bool]int{true: 1, false: 0}[c.visible]; reasoning != want {- t.Errorf("%d reasoning items, want %d", reasoning, want)+ // Thinking Claude didn't show is still given back to it on the+ // next request (mysterious_dove_96933 on Discord): one reasoning+ // item carrying the block, its summary empty unless shown.+ if len(reasoning) != 1 {+ t.Fatalf("%d reasoning items, want 1\n%s", len(reasoning), body)+ }+ enc, _ := reasoning[0]["encrypted_content"].(string)+ if !claudeseal.Is(enc) {+ t.Errorf("reasoning item's encrypted_content = %q, want Claude's block sealed", enc)+ }+ summary, _ := reasoning[0]["summary"].([]any)+ if shown := len(summary) > 0; shown != c.visible {+ t.Errorf("reasoning summary = %v, want shown %v", summary, c.visible)}})}diff --git a/internal/redact/redact_test.go b/internal/redact/redact_test.goindex 1379f36ac..ca5220061 100644--- a/internal/redact/redact_test.go+++ b/internal/redact/redact_test.go@@ -6,6 +6,8 @@ import ("net/http/httptest""strings""testing"++ "github.com/yetone/magpie/internal/claudeseal")const (@@ -207,6 +209,21 @@ func TestMaskJSONSignedGoesBackAsWritten(t *testing.T) {}}+// Claude's block magpie gave a Responses client to carry back+// (claudeseal) goes back byte for byte: Claude's own text in it, with the+// placeholders Claude was sent, is what its signature is checked against.+func TestMaskJSONKeepsClaudeBlocks(t *testing.T) {+ o := Options{Secrets: true, Personal: true}+ const email = "[email protected]"+ pe, _ := Mask(email, o)+ sealed := claudeseal.Seal([]byte(`{"type":"thinking","thinking":"Write to ` + pe + `, or call 13800138000.","signature":"Eqk1"}`))+ body := `{"input":[{"role":"user","content":"I am ` + email + `"},{"type":"reasoning","summary":[{"type":"summary_text","text":"Write to ` + email + `"}],"encrypted_content":"` + sealed + `"}]}`+ out, _ := MaskJSON([]byte(body), o)+ if !strings.Contains(string(out), `"`+sealed+`"`) || strings.Contains(string(out), email) {+ t.Errorf("Claude's block not as it was sent, or a value went out:\n%s", out)+ }+}+// An id goes as the vendor gave it, on both sides of what it matches.func TestMaskJSONKeepsIDs(t *testing.T) {o := Options{Secrets: true, Personal: true}diff --git a/internal/sessions/codex_portable.go b/internal/sessions/codex_portable.goindex 2bf00fbd6..a543ded2d 100644--- a/internal/sessions/codex_portable.go+++ b/internal/sessions/codex_portable.go@@ -17,6 +17,7 @@ import ("github.com/klauspost/compress/zstd"+ "github.com/yetone/magpie/internal/claudeseal""github.com/yetone/magpie/internal/codexsummary""github.com/yetone/magpie/internal/filelock""github.com/yetone/magpie/internal/settings"@@ -32,10 +33,15 @@ import (// local compaction keeps, and what the gateway sends for it anyway, so a// rollout so written reads the same through magpie and goes on without it.//-// Only magpie's compaction items change, in whichever line holds them (a-// response_item, a compacted line's replacement_history); every other-// line keeps its bytes, and the file its line count, mode and time. A-// rollout is changed as Codex publishes a compressed one+// A reasoning item magpie gave Codex with Claude's block in its+// encrypted_content (claudeseal) is refused the same way; it is written+// as its summary, as the gateway sends it to OpenAI.+//+// Only magpie's compaction items and those reasoning items change, in+// whichever line holds them (a response_item, a compacted line's+// replacement_history); every other line keeps its bytes, and the file+// its line count, mode and time. A rollout is changed as Codex publishes+// a compressed one// (codex-rs/rollout/src/writer_lock.rs, try_acquire_for_publication):// under its home's coordination lock, and only when no Codex holds the// thread's writer lock, so never under a Codex that has the thread open.@@ -204,6 +210,10 @@ func codexThreadOf(name string) string {var magpieMark = []byte(codexsummary.Mark)+// claudeMark begins a reasoning item's encrypted_content carrying Claude's+// block (claudeseal), which only magpie reads.+var claudeMark = []byte(claudeseal.Mark)+// portRollout writes the rollout at path portable, if it has magpie's// compaction item, and says whether it did.func portRollout(ctx context.Context, home, path string) (bool, error) {@@ -321,7 +331,7 @@ func portRollout(ctx context.Context, home, path string) (bool, error) {// gateway leaves it out; a line that is one alone stays as it is, as// leaving a line out would move every line after it.func portLine(line []byte) ([]byte, bool) {- if !bytes.Contains(line, magpieMark) {+ if !bytes.Contains(line, magpieMark) && !bytes.Contains(line, claudeMark) {return line, false}body := bytes.TrimRight(line, "\r\n")@@ -360,6 +370,16 @@ func portValue(v any) (_ any, changed, drop bool) {}return x, false, false}+ if t, _ := x["type"].(string); t == "reasoning" {+ // Claude's block magpie gave Codex to carry back: sent to+ // OpenAI it would be refused as invalid_encrypted_content, so+ // the item goes as its summary, as magpie sends it to OpenAI+ if enc, _ := x["encrypted_content"].(string); claudeseal.Is(enc) {+ delete(x, "encrypted_content")+ return x, true, false+ }+ return x, false, false+ }for k, c := range x {list, isList := c.([]any)if !isList {diff --git a/internal/sessions/codex_portable_test.go b/internal/sessions/codex_portable_test.goindex 7084966c1..5f1c8ec0a 100644--- a/internal/sessions/codex_portable_test.go+++ b/internal/sessions/codex_portable_test.go@@ -13,6 +13,7 @@ import ("github.com/klauspost/compress/zstd"+ "github.com/yetone/magpie/internal/claudeseal""github.com/yetone/magpie/internal/codexsummary""github.com/yetone/magpie/internal/filelock")@@ -237,3 +238,28 @@ func TestPortLineLeavesOthers(t *testing.T) {t.Errorf("compaction_summary: %q", got)}}++// A reasoning item carrying Claude's block (claudeseal) is written as its+// summary, as the gateway sends it to OpenAI, in a response_item and in a+// compacted line's replacement_history; OpenAI's own encrypted_content and+// a mention of the mark in text keep their bytes.+func TestPortLineDropsClaudeBlocks(t *testing.T) {+ sealed := claudeseal.Seal([]byte(`{"type":"thinking","thinking":"hm","signature":"sig"}`))+ for _, l := range []string{+ `{"type":"response_item","payload":{"type":"reasoning","summary":[],"encrypted_content":"gAAAAopenai"}}` + "\n",+ `{"type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"` + sealed + `"}]}}` + "\n",+ } {+ if got, ok := portLine([]byte(l)); ok || string(got) != l {+ t.Errorf("%q became %q", l, got)+ }+ }+ for _, l := range []string{+ `{"type":"response_item","payload":{"type":"reasoning","summary":[{"type":"summary_text","text":"hm"}],"encrypted_content":"` + sealed + `"}}` + "\n",+ `{"type":"compacted","payload":{"message":"","replacement_history":[{"type":"reasoning","summary":[],"encrypted_content":"` + sealed + `"}]}}` + "\n",+ } {+ got, ok := portLine([]byte(l))+ if !ok || bytes.Contains(got, []byte(claudeseal.Mark)) || bytes.Contains(got, []byte("encrypted_content")) || !bytes.Contains(got, []byte(`"reasoning"`)) {+ t.Errorf("%q became %q", l, got)+ }+ }+}