The open-source app everyone uses to manage agents at work
TypeScript acpxagent-orchestrationagent-skillsagentic-aiai-agentsai-engineeringai-governanceai-toolsautonomous-agentsclaudeclaude-codeclaude-skillscodexhermesllmmcpmulti-agentopenclawproject-managementworkflow-automation
feat(adapter-utils): carry binary bodies and attachment routes over the HTTP/2 sandbox bridge (#12923)
Nicky Leach6019e2bd6e61
diff --git a/doc/observability.md b/doc/observability.mdindex 3d45b420fd..9d7939a9c6 100644--- a/doc/observability.md+++ b/doc/observability.md@@ -754,18 +754,49 @@ To add a name or an enum value, extend the literal constant in### Known behavior: aggregate retained body bytes-The HTTP/2 bridge bounds retained body bytes for one route only. Each route-holds up to 8,388,608 bytes (8 MiB) at its own peak (see-`HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS` in `http2-bridge-server.ts`). The host-process admits up to 128 concurrent routes (see-`DEFAULT_MAX_CONCURRENT_DUPLEX_ROUTES` in `plugin-worker-manager.ts`). The-process can therefore retain up to 1,073,741,824 bytes (1 GiB) of body data-across every route at the same time.--This is accepted, known behavior. The process tracks no aggregate byte-ledger across routes: a per-route bound stops one busy route from starving-another route's own budget, but the host enforces no smaller ceiling on the-sum across every route.+Each HTTP/2 bridge route holds up to 168,820,736 bytes (161 MiB) at its own+peak (see `HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS` in `http2-bridge-server.ts`).+The host process admits up to 128 concurrent routes (see+`DEFAULT_MAX_CONCURRENT_DUPLEX_ROUTES` in `plugin-worker-manager.ts`). Those+two figures alone would let the process retain up to 21,609,054,208 bytes+(about 20.1 GiB) of body data across every route at the same time.++The process does not reach that figure, on two levels.+`HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES` (`http2-bridge-server.ts`) enforces a+real, live ledger: 1,073,741,824 bytes (1 GiB) across every route, not merely+an accepted paper ceiling. Every HTTP/2 stream creates one `BridgeBodyReservation` owner over+its lifetime, and every source-level full-body buffer that stream retains —+its request-body chunk array, the concatenated request body, the+response-body chunk array, and the concatenated response body — reserves+against that one owner before it allocates. A reservation that would pass the+process total is denied before it copies anything, and the host answers 503+instead of accepting the body. The reservation stays live for the response+body until the HTTP/2 write actually finishes flowing to the peer or the+stream closes, not merely until the write call returns, so a slow or+backpressured peer cannot hold response bytes in memory the ledger no longer+counts.++`HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES` adds a second, per-route ledger on top of+that process-wide one: each route's own reservations also check a ceiling+scoped to that one route (its own 168,820,736-byte peak from above), so one+busy or malicious route can pass its own ceiling and get denied with a 503,+but it can never spend the whole process-wide total and deny every sibling+route admission. This accounting covers source-level full-body buffers only:+internal Node.js and Undici copies (socket buffers, HTTP/2 frame buffers,+decompression buffers) stay outside it.++The generated gateway process inside the sandbox (`getSandboxCallbackBridgeServerSource`+in `sandbox-callback-bridge.ts`) enforces its own separate ledger, independent+of the two host-side ledgers above: each side bounds only the memory in its+own process. `readBodyBytes` reserves a request body's chunk bytes as they+arrive, then reserves the concatenated buffer's own byte count before+`Buffer.concat` allocates it, against a ceiling of `maxBodyBytes * 8` (4+concurrent bodies, each counted twice for its two live copies). A denied+reservation answers 503 with no forward call. Each request handler releases+its own reservation once the whole request settles: a completed response, a+thrown error, a client abort, or a deadline timeout all reach the same+release call.+Keep every dimension low-cardinality and free of user content.### Shared skill preparationdiff --git a/packages/adapter-utils/src/execution-target-sandbox.test.ts b/packages/adapter-utils/src/execution-target-sandbox.test.tsindex 0b3c5134af..ce1d9f6faf 100644--- a/packages/adapter-utils/src/execution-target-sandbox.test.ts+++ b/packages/adapter-utils/src/execution-target-sandbox.test.ts@@ -10,6 +10,13 @@ import { promisify } from "node:util";import { afterEach, describe, expect, it, vi } from "vitest";import { getSandboxDuplexGatewayCodecSource } from "./sandbox-callback-bridge.js";+import {+ createBridgeBodyReservation,+ getBridgeBodyReservedBytesForTest,+ resetBridgeBodyReservationsForTest,+ HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS,+ HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES,+} from "./http2-bridge-server.js";import {__duplexReadinessTesting,@@ -3177,10 +3184,10 @@ describe("sandbox adapter execution targets", () => {*/function http2TestRequest(session: http2.ClientHttp2Session,- request: { method: string; path: string; headers?: Record<string, string>; body?: string },- ): Promise<{ status: number; headers: Record<string, string>; body: string }> {+ request: { method: string; path: string; headers?: Record<string, string>; body?: Buffer },+ ): Promise<{ status: number; headers: Record<string, string>; body: Buffer }> {return new Promise((resolve, reject) => {- const body = Buffer.from(request.body ?? "", "utf8");+ const body = request.body ?? Buffer.alloc(0);const stream = session.request({ ":method": request.method, ":path": request.path, ...request.headers },{ endStream: body.length === 0 },@@ -3197,13 +3204,35 @@ describe("sandbox adapter execution targets", () => {}});stream.on("data", (chunk: Buffer) => chunks.push(chunk));- stream.once("end", () => resolve({ status, headers, body: Buffer.concat(chunks).toString("utf8") }));+ stream.once("end", () => resolve({ status, headers, body: Buffer.concat(chunks) }));stream.once("error", (error) => reject(error));if (body.length > 0) stream.end(body);else if (!stream.writableEnded) stream.end();});}+ // Fixed binary payload for an attachment-content download. Byte 0x89 opens+ // the PNG signature and is not valid UTF-8 on its own, so a round trip+ // through a text decode step would corrupt it.+ const ATTACHMENT_DOWNLOAD_BYTES = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0xff, 0x00, 0x7f]);++ // Build a minimal multipart/form-data request body with one binary file+ // part, plus the matching `content-type` header value.+ function buildMultipartAttachmentUpload(fileBytes: Buffer): { body: Buffer; contentType: string } {+ const boundary = "paperclip-test-boundary";+ const head = Buffer.from(+ `--${boundary}\r\n` ++ `Content-Disposition: form-data; name="file"; filename="upload.bin"\r\n` ++ `Content-Type: application/octet-stream\r\n\r\n`,+ "utf8",+ );+ const tail = Buffer.from(`\r\n--${boundary}--\r\n`, "utf8");+ return {+ body: Buffer.concat([head, fileBytes, tail]),+ contentType: `multipart/form-data; boundary=${boundary}`,+ };+ }+// Start a host API server that records each forwarded request, so a test can// assert the real token and the run id reach the host, or that a rejected// request never forwards.@@ -3215,6 +3244,7 @@ describe("sandbox adapter execution targets", () => {auth: string | null;runId: string | null;headers: Record<string, string>;+ body: Buffer;}>;close: () => Promise<void>;}> {@@ -3224,21 +3254,35 @@ describe("sandbox adapter execution targets", () => {auth: string | null;runId: string | null;headers: Record<string, string>;+ body: Buffer;}> = [];const server = createServer((req, res) => {const headers: Record<string, string> = {};for (const [key, value] of Object.entries(req.headers)) {if (typeof value === "string") headers[key] = value;}- requests.push({- method: req.method ?? "GET",- url: req.url ?? "/",- auth: req.headers.authorization ?? null,- runId: typeof req.headers["x-paperclip-run-id"] === "string" ? req.headers["x-paperclip-run-id"] : null,- headers,+ const chunks: Buffer[] = [];+ req.on("data", (chunk: Buffer) => chunks.push(chunk));+ req.on("end", () => {+ requests.push({+ method: req.method ?? "GET",+ url: req.url ?? "/",+ auth: req.headers.authorization ?? null,+ runId: typeof req.headers["x-paperclip-run-id"] === "string" ? req.headers["x-paperclip-run-id"] : null,+ headers,+ body: Buffer.concat(chunks),+ });+ // An attachment-content download answers with a binary body, so a+ // test can assert the bytes reach the caller unchanged. Every other+ // route keeps the fixed JSON acknowledgement.+ if (req.method === "GET" && /^\/api\/attachments\/[^/]+\/content$/.test(req.url ?? "")) {+ res.writeHead(200, { "content-type": "application/octet-stream" });+ res.end(ATTACHMENT_DOWNLOAD_BYTES);+ return;+ }+ res.writeHead(200, { "content-type": "application/json" });+ res.end(JSON.stringify({ ok: true }));});- res.writeHead(200, { "content-type": "application/json" });- res.end(JSON.stringify({ ok: true }));});await new Promise<void>((resolve, reject) => {server.once("error", reject);@@ -3298,6 +3342,23 @@ describe("sandbox adapter execution targets", () => {expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("queue_v1");const fallback = counters.find((c) => c.metric === DUPLEX_COUNTER_FALLBACK_TOTAL);expect(fallback?.dimensions.fallback_reason).toBe("channel_open_failed");++ // The queue fallback keeps the 415 gate for the attachment upload path:+ // it never admits a binary body, and it never forwards the request to+ // the host.+ const uploadResponse = await fetch(+ `${bridge!.env.PAPERCLIP_API_URL}/api/companies/co-1/issues/issue-1/attachments`,+ {+ method: "POST",+ headers: {+ authorization: `Bearer ${bridge!.env.PAPERCLIP_API_KEY}`,+ "content-type": "application/octet-stream",+ },+ body: Buffer.from([0x50, 0x4b, 0x03, 0x04]),+ },+ );+ expect(uploadResponse.status).toBe(415);+ expect(api.requests).toHaveLength(0);} finally {await bridge?.stop();await api.close();@@ -3762,7 +3823,7 @@ describe("sandbox adapter execution targets", () => {method: "POST",path: "/api/secret-admin-route",headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/json" },- body: JSON.stringify({ escalate: true }),+ body: Buffer.from(JSON.stringify({ escalate: true }), "utf8"),});expect(forbidden.status).toBe(403);expect(api.requests).toHaveLength(0);@@ -3784,6 +3845,39 @@ describe("sandbox adapter execution targets", () => {runId: "run-http2-403",});expect(api.requests[0].headers["x-not-allowed"]).toBeUndefined();++ // The two attachment routes are admitted on the http2 path, and a+ // binary body reaches the host and returns unchanged in both+ // directions: no re-encoding step touches the multipart upload or the+ // binary download.+ const uploadBytes = Buffer.from([0x00, 0x01, 0xff, 0x7f, 0x80, 0x0d, 0x0a]);+ const upload = buildMultipartAttachmentUpload(uploadBytes);+ const uploadResponse = await http2TestRequest(sessionRef.current!, {+ method: "POST",+ path: "/api/companies/co-1/issues/issue-1/attachments",+ headers: {+ authorization: `Bearer ${bridgeToken}`,+ "content-type": upload.contentType,+ },+ body: upload.body,+ });+ expect(uploadResponse.status).toBe(200);+ expect(api.requests).toHaveLength(2);+ expect(api.requests[1]).toMatchObject({+ method: "POST",+ url: "/api/companies/co-1/issues/issue-1/attachments",+ });+ expect(api.requests[1].headers["content-type"]).toBe(upload.contentType);+ expect(api.requests[1].body.equals(upload.body)).toBe(true);++ const downloadResponse = await http2TestRequest(sessionRef.current!, {+ method: "GET",+ path: "/api/attachments/att-1/content",+ headers: { authorization: `Bearer ${bridgeToken}` },+ });+ expect(downloadResponse.status).toBe(200);+ expect(api.requests).toHaveLength(3);+ expect(downloadResponse.body.equals(ATTACHMENT_DOWNLOAD_BYTES)).toBe(true);} finally {sessionRef.current?.close();await bridge?.stop();@@ -3791,6 +3885,83 @@ describe("sandbox adapter execution targets", () => {}}, 20000);+ it("test_http2_forward_preserves_request_and_response_bytes", async () => {+ // Byte 0xC3 opens a two-byte UTF-8 sequence; 0x28 is not a valid+ // continuation byte, so this body is not valid UTF-8. The forward path+ // must carry these exact bytes on the way in, and the host's own+ // response bytes on the way out, with no re-encoding step on either leg.+ const malformedBytes = Buffer.from([0x7b, 0x22, 0x61, 0x22, 0x3a, 0xc3, 0x28, 0x7d]);+ const receivedRequestBodies: Buffer[] = [];+ const echoServer = createServer((req, res) => {+ const chunks: Buffer[] = [];+ req.on("data", (chunk: Buffer) => chunks.push(chunk));+ req.on("end", () => {+ receivedRequestBodies.push(Buffer.concat(chunks));+ res.writeHead(200, { "content-type": "application/octet-stream" });+ res.end(malformedBytes);+ });+ });+ await new Promise<void>((resolve, reject) => {+ echoServer.once("error", reject);+ echoServer.listen(0, "127.0.0.1", () => resolve());+ });+ const echoAddress = echoServer.address();+ if (!echoAddress || typeof echoAddress === "string") {+ throw new Error("Expected the echo server to listen on a TCP port.");+ }++ const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-http2-raw-bytes-"));+ cleanupDirs.push(rootDir);+ const remoteCwd = path.join(rootDir, "workspace");+ await mkdir(remoteCwd, { recursive: true });+ const sessionRef: { current: http2.ClientHttp2Session | null } = { current: null };+ let bridgeToken = "";+ const { runner } = makeHttp2SelectionRunner((ctx) => {+ bridgeToken = ctx.bridgeToken;+ ctx.emitReady();+ sessionRef.current = ctx.connectHttp2();+ });+ const target: AdapterSandboxExecutionTarget = {+ kind: "remote",+ transport: "sandbox",+ providerKey: "daytona",+ remoteCwd,+ timeoutMs: 30_000,+ runner,+ effectiveCapabilities: duplexCapabilities(true),+ };++ const bridge = await startAdapterExecutionTargetPaperclipBridge({+ runId: "run-raw-bytes",+ target,+ runtimeRootDir: path.join(remoteCwd, ".paperclip-runtime", "codex"),+ adapterKey: "codex",+ hostApiToken: "real-run-jwt",+ hostApiUrl: `http://127.0.0.1:${echoAddress.port}`,+ enableSandboxDuplexBridge: true,+ });+ try {+ expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("http2_v1");+ await waitForCondition(() => sessionRef.current !== null, "the http2 client session to open", 4000);++ const response = await http2TestRequest(sessionRef.current!, {+ method: "POST",+ path: "/api/issues/issue-1/comments",+ headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/octet-stream" },+ body: malformedBytes,+ });++ expect(response.status).toBe(200);+ expect(receivedRequestBodies).toHaveLength(1);+ expect(receivedRequestBodies[0]?.equals(malformedBytes)).toBe(true);+ expect(response.body.equals(malformedBytes)).toBe(true);+ } finally {+ sessionRef.current?.close();+ await bridge?.stop();+ await new Promise<void>((resolve) => echoServer.close(() => resolve()));+ }+ }, 20000);+// One recording telemetry recorder. It captures every span, counter, and event// the fixed duplex surface produces, so a test asserts the exact names,// dimensions, and values. An optional `failEvery` flag makes every method throw,@@ -4234,7 +4405,7 @@ describe("sandbox adapter execution targets", () => {method: "POST",path: `/api/issues/${ROUTE_SENTINEL}/comments?secret=${QUERY_SENTINEL}`,headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/json" },- body: JSON.stringify({ body: BODY_SENTINEL }),+ body: Buffer.from(JSON.stringify({ body: BODY_SENTINEL }), "utf8"),});expect(response.status).toBe(200);await waitForCondition(@@ -5350,6 +5521,11 @@ describe("sandbox adapter execution targets", () => {// failed, so a retry could double-apply it. The host answers a// non-retryable 504 with the indeterminate marker instead — the same// rule `forwardBridgeRequest` already applies on every transport.+ //+ // `maxBodyBytes: 1` below now bounds the request body too, since the+ // host and the gateway share one resolved ceiling: this request carries+ // no body, so only the mock host's own response — comfortably over one+ // byte — trips the size check this test exists to force.const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-http2-unsafe-indeterminate-"));cleanupDirs.push(rootDir);const remoteCwd = path.join(rootDir, "workspace");@@ -5389,7 +5565,6 @@ describe("sandbox adapter execution targets", () => {method: "POST",path: "/api/issues/issue-1/comments",headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/json" },- body: JSON.stringify({ body: "hello" }),});expect(response.status).toBe(504);expect(response.headers["x-paperclip-bridge-outcome"]).toBe("indeterminate");@@ -5481,6 +5656,509 @@ describe("sandbox adapter execution targets", () => {}}, 20000);+ it("test_a_denied_response_chunk_cancels_the_reader_before_it_copies_the_chunk", async () => {+ // Fill the process ledger so only a sliver of headroom remains, then let+ // the mock host answer with a response chunk far bigger than that+ // sliver. `readBridgeForwardResponseBody` (`execution-target.ts`) must+ // deny that chunk's reservation, cancel its reader (closing the+ // outbound connection to the mock host), and retain no copy of it.+ resetBridgeBodyReservationsForTest();+ const filler = createBridgeBodyReservation();+ expect(filler.reserve(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES - 100)).toBe(true);++ let resolveHostConnectionClosed: (() => void) | undefined;+ const hostConnectionClosed = new Promise<void>((resolve) => {+ resolveHostConnectionClosed = resolve;+ });+ const api = createServer((_req, res) => {+ res.writeHead(200, { "content-type": "application/octet-stream" });+ // Far bigger than the 100 bytes of headroom the filler above left:+ // this one chunk alone must pass the process ceiling.+ res.write(Buffer.alloc(1_000, "a"));+ res.on("close", () => resolveHostConnectionClosed!());+ });+ await new Promise<void>((resolve, reject) => {+ api.once("error", reject);+ api.listen(0, "127.0.0.1", () => resolve());+ });+ const apiAddress = api.address();+ if (!apiAddress || typeof apiAddress === "string") {+ throw new Error("Expected the mock host server to listen on a TCP port.");+ }+ const apiOrigin = `http://127.0.0.1:${apiAddress.port}`;++ const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-http2-denied-response-chunk-"));+ cleanupDirs.push(rootDir);+ const remoteCwd = path.join(rootDir, "workspace");+ await mkdir(remoteCwd, { recursive: true });+ const sessionRef: { current: http2.ClientHttp2Session | null } = { current: null };+ let bridgeToken = "";+ const { runner } = makeHttp2SelectionRunner((ctx) => {+ bridgeToken = ctx.bridgeToken;+ ctx.emitReady();+ sessionRef.current = ctx.connectHttp2();+ });+ const target: AdapterSandboxExecutionTarget = {+ kind: "remote",+ transport: "sandbox",+ providerKey: "daytona",+ remoteCwd,+ timeoutMs: 30_000,+ runner,+ effectiveCapabilities: duplexCapabilities(true),+ };++ const bridge = await startAdapterExecutionTargetPaperclipBridge({+ runId: "run-denied-response-chunk",+ target,+ runtimeRootDir: path.join(remoteCwd, ".paperclip-runtime", "codex"),+ adapterKey: "codex",+ hostApiToken: "real-run-jwt",+ hostApiUrl: apiOrigin,+ enableSandboxDuplexBridge: true,+ });+ try {+ expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("http2_v1");+ await waitForCondition(() => sessionRef.current !== null, "the http2 client session to open", 4000);+ // A capacity denial reaches the client as the retryable 503 the+ // HTTP/2 bridge server's own capacity-denial path answers+ // (`forwardBridgeRequest` rethrows `BridgeProcessCapacityError` before+ // the method-safety classification runs), not the generic 502 that+ // classification would give any other response-body read fault.+ const response = await http2TestRequest(sessionRef.current!, {+ method: "GET",+ path: "/api/agents/me",+ headers: { authorization: `Bearer ${bridgeToken}` },+ });+ expect(response.status).toBe(503);+ // The reader actually cancelled: the mock host observes its+ // connection close, instead of staying open with the chunk+ // unacknowledged.+ await hostConnectionClosed;+ } finally {+ sessionRef.current?.close();+ await bridge?.stop();+ await new Promise<void>((resolve) => api.close(() => resolve()));+ // The denied chunk was never retained: releasing the filler is the+ // only release this test needs to reach zero. If the denied response+ // copy had reserved anything despite being denied, this would be+ // nonzero.+ filler.release();+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ }+ }, 20000);++ it("test_a_denied_concatenated_response_body_never_allocates_the_copy", async () => {+ // Leave room for the one response chunk but not for the second,+ // concatenated copy `readBridgeForwardResponseBody` (`execution-target.ts`)+ // builds from it: a correct reader checks the reservation before+ // `Buffer.concat` allocates the copy, so the denied concatenated copy+ // must never call `Buffer.concat` at all. The same denial must also+ // cancel the upstream response reader, instead of leaving it open after+ // the throw.+ resetBridgeBodyReservationsForTest();+ const chunkBytes = 200_000;+ const filler = createBridgeBodyReservation();+ expect(filler.reserve(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES - Math.floor(chunkBytes * 1.5))).toBe(true);++ const api = createServer((_req, res) => {+ res.writeHead(200, { "content-type": "application/octet-stream" });+ res.end(Buffer.alloc(chunkBytes, "a"));+ });+ await new Promise<void>((resolve, reject) => {+ api.once("error", reject);+ api.listen(0, "127.0.0.1", () => resolve());+ });+ const apiAddress = api.address();+ if (!apiAddress || typeof apiAddress === "string") {+ throw new Error("Expected the mock host server to listen on a TCP port.");+ }+ const apiOrigin = `http://127.0.0.1:${apiAddress.port}`;++ const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-http2-denied-response-concat-"));+ cleanupDirs.push(rootDir);+ const remoteCwd = path.join(rootDir, "workspace");+ await mkdir(remoteCwd, { recursive: true });+ const sessionRef: { current: http2.ClientHttp2Session | null } = { current: null };+ let bridgeToken = "";+ const { runner } = makeHttp2SelectionRunner((ctx) => {+ bridgeToken = ctx.bridgeToken;+ ctx.emitReady();+ sessionRef.current = ctx.connectHttp2();+ });+ const target: AdapterSandboxExecutionTarget = {+ kind: "remote",+ transport: "sandbox",+ providerKey: "daytona",+ remoteCwd,+ timeoutMs: 30_000,+ runner,+ effectiveCapabilities: duplexCapabilities(true),+ };++ const bridge = await startAdapterExecutionTargetPaperclipBridge({+ runId: "run-denied-response-concat",+ target,+ runtimeRootDir: path.join(remoteCwd, ".paperclip-runtime", "codex"),+ adapterKey: "codex",+ hostApiToken: "real-run-jwt",+ hostApiUrl: apiOrigin,+ enableSandboxDuplexBridge: true,+ });+ const concatSpy = vi.spyOn(Buffer, "concat");+ const readerCancelSpy = vi.spyOn(ReadableStreamDefaultReader.prototype, "cancel");+ try {+ expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("http2_v1");+ await waitForCondition(() => sessionRef.current !== null, "the http2 client session to open", 4000);+ // A capacity denial must reach the client as the retryable 503 the+ // HTTP/2 bridge server's own capacity-denial path answers, not the+ // generic 502 the method-safety classification would otherwise apply+ // to any other response-body read fault (`forwardBridgeRequest`+ // rethrows `BridgeProcessCapacityError` before that classification+ // runs). This reads the response with a plain string accumulator, not+ // `Buffer.concat`, so the spy below counts only the calls the bridge+ // code under test makes.+ const response = await new Promise<{ status: number; body: string }>((resolve, reject) => {+ const stream = sessionRef.current!.request({+ ":method": "GET",+ ":path": "/api/agents/me",+ authorization: `Bearer ${bridgeToken}`,+ });+ let status = 0;+ let body = "";+ stream.setEncoding("utf8");+ stream.on("response", (headers) => {+ status = Number(headers[":status"]) || 0;+ });+ stream.on("data", (chunk) => (body += chunk));+ stream.once("end", () => resolve({ status, body }));+ stream.once("error", reject);+ stream.end();+ });+ expect(response.status).toBe(503);+ expect(JSON.parse(response.body)).toEqual({+ error: "The bridge host reached its reserved process body byte ceiling. Retry later.",+ });+ // The GET request itself carries no body, so the host's own read of+ // that empty request body still calls `Buffer.concat` on an empty+ // array — that call is unrelated to this test. No call may carry any+ // response byte, since the denied concatenated response copy must+ // never allocate.+ for (const [chunks] of concatSpy.mock.calls) {+ expect((chunks as Buffer[]).reduce((sum, chunk) => sum + chunk.length, 0)).toBe(0);+ }+ // The denied concatenated-body reservation must cancel the upstream+ // response reader before it throws, instead of leaving it open.+ expect(readerCancelSpy).toHaveBeenCalled();+ } finally {+ readerCancelSpy.mockRestore();+ concatSpy.mockRestore();+ sessionRef.current?.close();+ await bridge?.stop();+ await new Promise<void>((resolve) => api.close(() => resolve()));+ // The denied concatenated copy reserved nothing: releasing the filler+ // is the only release this test needs to reach zero.+ filler.release();+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ }+ }, 20000);++ it("test_a_denied_response_chunk_on_a_mutating_method_is_indeterminate_not_retryable", async () => {+ // A POST may already have committed on the host by the time the+ // response-body read hits the process capacity ceiling: the host+ // delivered response headers before the read even starts. Unlike the+ // safe-method case above, this denial must not reach the client as a+ // retryable 503 — a caller that retries would apply the mutation twice.+ // It must fall through to the same non-retryable indeterminate 504 any+ // other response-body read fault on a mutating method gets.+ resetBridgeBodyReservationsForTest();+ const filler = createBridgeBodyReservation();+ expect(filler.reserve(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES - 100)).toBe(true);++ const api = createServer((_req, res) => {+ res.writeHead(200, { "content-type": "application/octet-stream" });+ // Far bigger than the 100 bytes of headroom the filler above left.+ res.end(Buffer.alloc(1_000, "a"));+ });+ await new Promise<void>((resolve, reject) => {+ api.once("error", reject);+ api.listen(0, "127.0.0.1", () => resolve());+ });+ const apiAddress = api.address();+ if (!apiAddress || typeof apiAddress === "string") {+ throw new Error("Expected the mock host server to listen on a TCP port.");+ }+ const apiOrigin = `http://127.0.0.1:${apiAddress.port}`;++ const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-http2-denied-response-mutation-"));+ cleanupDirs.push(rootDir);+ const remoteCwd = path.join(rootDir, "workspace");+ await mkdir(remoteCwd, { recursive: true });+ const sessionRef: { current: http2.ClientHttp2Session | null } = { current: null };+ let bridgeToken = "";+ const { runner } = makeHttp2SelectionRunner((ctx) => {+ bridgeToken = ctx.bridgeToken;+ ctx.emitReady();+ sessionRef.current = ctx.connectHttp2();+ });+ const target: AdapterSandboxExecutionTarget = {+ kind: "remote",+ transport: "sandbox",+ providerKey: "daytona",+ remoteCwd,+ timeoutMs: 30_000,+ runner,+ effectiveCapabilities: duplexCapabilities(true),+ };++ const bridge = await startAdapterExecutionTargetPaperclipBridge({+ runId: "run-denied-response-mutation",+ target,+ runtimeRootDir: path.join(remoteCwd, ".paperclip-runtime", "codex"),+ adapterKey: "codex",+ hostApiToken: "real-run-jwt",+ hostApiUrl: apiOrigin,+ enableSandboxDuplexBridge: true,+ });+ try {+ expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("http2_v1");+ await waitForCondition(() => sessionRef.current !== null, "the http2 client session to open", 4000);+ const response = await http2TestRequest(sessionRef.current!, {+ method: "POST",+ path: "/api/issues/issue-1/comments",+ headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/json" },+ });+ expect(response.status).toBe(504);+ expect(response.headers["x-paperclip-bridge-outcome"]).toBe("indeterminate");+ expect(JSON.parse(response.body.toString("utf8"))).toMatchObject({+ outcome: "indeterminate",+ retryable: false,+ });+ } finally {+ sessionRef.current?.close();+ await bridge?.stop();+ await new Promise<void>((resolve) => api.close(() => resolve()));+ filler.release();+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ }+ }, 20000);++ it("test_concurrent_request_and_response_bodies_never_pass_the_process_ceiling", async () => {+ resetBridgeBodyReservationsForTest();+ const bodyBytes = 2 * 1024 * 1024;+ const requestBody = Buffer.alloc(bodyBytes, "a");+ const samples: number[] = [];+ const api = createServer((req, res) => {+ const chunks: Buffer[] = [];+ req.on("data", (chunk: Buffer) => chunks.push(chunk));+ req.on("end", () => {+ // Sampled while this stream's own request-body copies are still+ // live and at least one sibling stream may also be mid-flight: the+ // real, concurrent, multi-stream shape this test exists to prove.+ samples.push(getBridgeBodyReservedBytesForTest());+ res.writeHead(200, { "content-type": "application/octet-stream" });+ res.end(Buffer.alloc(bodyBytes, "b"));+ });+ });+ await new Promise<void>((resolve, reject) => {+ api.once("error", reject);+ api.listen(0, "127.0.0.1", () => resolve());+ });+ const apiAddress = api.address();+ if (!apiAddress || typeof apiAddress === "string") {+ throw new Error("Expected the mock host server to listen on a TCP port.");+ }+ const apiOrigin = `http://127.0.0.1:${apiAddress.port}`;++ const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-http2-ceiling-aggregate-"));+ cleanupDirs.push(rootDir);+ const remoteCwd = path.join(rootDir, "workspace");+ await mkdir(remoteCwd, { recursive: true });+ const sessionRef: { current: http2.ClientHttp2Session | null } = { current: null };+ let bridgeToken = "";+ const { runner } = makeHttp2SelectionRunner((ctx) => {+ bridgeToken = ctx.bridgeToken;+ ctx.emitReady();+ sessionRef.current = ctx.connectHttp2();+ });+ const target: AdapterSandboxExecutionTarget = {+ kind: "remote",+ transport: "sandbox",+ providerKey: "daytona",+ remoteCwd,+ timeoutMs: 30_000,+ runner,+ effectiveCapabilities: duplexCapabilities(true),+ };+ const bridge = await startAdapterExecutionTargetPaperclipBridge({+ runId: "run-ceiling-aggregate",+ target,+ runtimeRootDir: path.join(remoteCwd, ".paperclip-runtime", "codex"),+ adapterKey: "codex",+ hostApiToken: "real-run-jwt",+ hostApiUrl: apiOrigin,+ enableSandboxDuplexBridge: true,+ });+ try {+ expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("http2_v1");+ await waitForCondition(() => sessionRef.current !== null, "the http2 client session to open", 4000);++ const responses = await Promise.all(+ Array.from({ length: HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS }, () =>+ http2TestRequest(sessionRef.current!, {+ method: "POST",+ path: "/api/issues/abc/comments",+ headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/octet-stream" },+ body: requestBody,+ }),+ ),+ );+ for (const response of responses) {+ expect(response.status).toBe(200);+ expect(response.body.byteLength).toBe(bodyBytes);+ }+ expect(samples).toHaveLength(HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS);+ for (const sample of samples) {+ expect(sample).toBeGreaterThan(0);+ expect(sample).toBeLessThanOrEqual(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES);+ }+ // Every stream's owner released once its forward settled.+ await waitForCondition(+ () => getBridgeBodyReservedBytesForTest() === 0,+ "the process reservation total to return to zero",+ 2_000,+ );+ } finally {+ sessionRef.current?.close();+ await bridge?.stop();+ await new Promise<void>((resolve) => api.close(() => resolve()));+ }+ }, 20000);++ it("test_the_host_denies_a_body_over_the_resolved_limit_not_only_the_gateway", async () => {+ const api = await startRecordingApiServer();+ const sessionRef: { current: http2.ClientHttp2Session | null } = { current: null };+ let bridgeToken = "";+ const { runner } = makeHttp2SelectionRunner((ctx) => {+ bridgeToken = ctx.bridgeToken;+ ctx.emitReady();+ sessionRef.current = ctx.connectHttp2();+ });+ const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-http2-host-body-limit-"));+ cleanupDirs.push(rootDir);+ const remoteCwd = path.join(rootDir, "workspace");+ await mkdir(remoteCwd, { recursive: true });+ const target: AdapterSandboxExecutionTarget = {+ kind: "remote",+ transport: "sandbox",+ providerKey: "daytona",+ remoteCwd,+ timeoutMs: 30_000,+ runner,+ effectiveCapabilities: duplexCapabilities(true),+ };++ const bridge = await startAdapterExecutionTargetPaperclipBridge({+ runId: "run-host-body-limit",+ target,+ runtimeRootDir: path.join(remoteCwd, ".paperclip-runtime", "codex"),+ adapterKey: "codex",+ hostApiToken: "real-run-jwt",+ hostApiUrl: api.origin,+ enableSandboxDuplexBridge: true,+ // A ceiling far under the body this test sends, resolved for this+ // run. `createHttp2BridgeServer()` must receive this same resolved+ // value, so the host itself enforces it on the raw wire — this test+ // talks to the host directly, with no sandbox-side gateway script in+ // between to enforce anything on its own.+ maxBodyBytes: 100,+ });+ try {+ expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("http2_v1");+ await waitForCondition(() => sessionRef.current !== null, "the http2 client session to open", 4000);+ const response = await http2TestRequest(sessionRef.current!, {+ method: "POST",+ path: "/api/issues/abc/comments",+ headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/json" },+ body: Buffer.alloc(1_000, "a"),+ });+ // The oversized body never reaches the host API: the resolved ceiling+ // rejects it before any forward call runs, so the mock host records+ // no request. (The host's own size-violation path resets the stream+ // before it can write a status line, so the client sees no ordinary+ // response — this test asserts the one thing that channel does prove:+ // the forward call itself never ran.)+ expect(api.requests).toHaveLength(0);+ expect(response.status).not.toBe(200);+ } finally {+ sessionRef.current?.close();+ await bridge?.stop();+ await api.close();+ }+ }, 20000);++ it("test_the_queue_transport_still_forwards_with_no_reservation_owner", async () => {+ // The queue transport's `handleRequest` callback+ // (`execution-target.ts`'s queue callback) calls `forwardBridgeRequest`+ // with no `reservation` option at all. `readBridgeForwardResponseBody`+ // must behave exactly as it did before that option existed: it still+ // forwards the request and still returns the host's body.+ const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-queue-no-reservation-"));+ cleanupDirs.push(rootDir);+ const remoteCwd = path.join(rootDir, "workspace");+ const runtimeRootDir = path.join(remoteCwd, ".paperclip-runtime", "codex");+ await mkdir(runtimeRootDir, { recursive: true });++ const apiServer = createServer((req, res) => {+ res.writeHead(201, { "content-type": "application/json" });+ res.end(JSON.stringify({ ok: true, echoedMethod: req.method }));+ });+ await new Promise<void>((resolve, reject) => {+ apiServer.once("error", reject);+ apiServer.listen(0, "127.0.0.1", () => resolve());+ });+ const address = apiServer.address();+ if (!address || typeof address === "string") {+ throw new Error("Expected the queue-transport test API server to listen on a TCP port.");+ }++ const target: AdapterSandboxExecutionTarget = {+ kind: "remote",+ transport: "sandbox",+ providerKey: "e2b",+ environmentId: "env-1",+ leaseId: "lease-1",+ remoteCwd,+ runner: createLocalSandboxRunner(),+ timeoutMs: 30_000,+ };++ const bridge = await startAdapterExecutionTargetPaperclipBridge({+ runId: "run-queue-no-reservation",+ target,+ runtimeRootDir,+ adapterKey: "codex",+ hostApiToken: "real-run-jwt",+ hostApiUrl: `http://127.0.0.1:${address.port}`,+ });+ try {+ expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe("queue_v1");+ const response = await fetch(`${bridge!.env.PAPERCLIP_API_URL}/api/issues/abc/comments`, {+ method: "POST",+ headers: {+ authorization: `Bearer ${bridge!.env.PAPERCLIP_API_KEY}`,+ "content-type": "application/json",+ },+ body: JSON.stringify({ body: "hello" }),+ });+ expect(response.status).toBe(201);+ expect(await response.json()).toEqual({ ok: true, echoedMethod: "POST" });+ } finally {+ await bridge?.stop();+ await new Promise<void>((resolve) => apiServer.close(() => resolve()));+ }+ });+// ---------------------------------------------------------------------------// Real-PTY replay.//diff --git a/packages/adapter-utils/src/execution-target.ts b/packages/adapter-utils/src/execution-target.tsindex ccc4a9e2ab..dcede3ea14 100644--- a/packages/adapter-utils/src/execution-target.ts+++ b/packages/adapter-utils/src/execution-target.ts@@ -35,6 +35,7 @@ import {createSandboxCallbackBridgeAsset,createSandboxCallbackBridgeToken,DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES,+ HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST,SANDBOX_CALLBACK_BRIDGE_ENTRYPOINT,SANDBOX_CALLBACK_BRIDGE_HTTP2_MODE,sandboxCallbackBridgeDirectories,@@ -45,6 +46,8 @@ import {} from "./sandbox-callback-bridge.js";import {createHttp2BridgeServer,+ BridgeProcessCapacityError,+ type BridgeBodyReservation,type Http2BridgeForwardHandler,} from "./http2-bridge-server.js";import {@@ -87,12 +90,18 @@ import type { RunnerIngressEndpoint } from "./runner-connectivity.js";export type { RuntimeProgressSink } from "./runtime-progress.js";-export function postedIssueCommentLogMarker(method: string, requestPath: string, status: number, body: string) {+export function postedIssueCommentLogMarker(+ method: string,+ requestPath: string,+ status: number,+ body: Buffer | string,+) {if (method !== "POST" || !/^\/api\/issues\/[^/]+\/comments$/.test(requestPath) || status < 200 || status >= 300) {return null;}+ const bodyText = typeof body === "string" ? body : body.toString("utf8");try {- const parsed = JSON.parse(body) as { id?: unknown };+ const parsed = JSON.parse(bodyText) as { id?: unknown };return typeof parsed.id === "string" && parsed.id.length > 0 ? `comment id: ${parsed.id}\n` : null;} catch {return null;@@ -646,12 +655,19 @@ function preferredSandboxShell(target: AdapterSandboxExecutionTarget): "bash" |type AdapterCommandCapableExecutionTarget = AdapterSshExecutionTarget | AdapterSandboxExecutionTarget;+// The Secure Shell command runner's own output buffer. This value used to+// derive from the bridge body limit (`DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES+// * 4`), so a bridge limit rise silently grew it too. It now stands on its+// own local constant, independent of the bridge body limit, so a later+// bridge limit change never resizes this buffer as a side effect.+const SSH_COMMAND_MAX_BUFFER_BYTES = 1024 * 1024;+function adapterExecutionTargetCommandRunner(target: AdapterCommandCapableExecutionTarget): CommandManagedRuntimeRunner {if (target.transport === "ssh") {return createSshCommandManagedRuntimeRunner({spec: target.spec,defaultCwd: target.remoteCwd,- maxBufferBytes: DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES * 4,+ maxBufferBytes: SSH_COMMAND_MAX_BUFFER_BYTES,});}return requireSandboxRunner(target);@@ -1617,15 +1633,30 @@ function bridgeResponseBodyLimitError(maxBodyBytes: number): Error {}/**- * Read the forward response body into a string. The per-request `maxBodyBytes`- * limit rejects a body larger than the configured per-request ceiling.+ * Read the forward response body into a `Buffer`, with no text decoding. The+ * per-request `maxBodyBytes` limit rejects a body larger than the configured+ * per-request ceiling.+ *+ * When the caller passes a `reservation`, this reserves each chunk's bytes+ * against it immediately after `reader.read()` yields the chunk, and before+ * `Buffer.from(value)` copies it — the allocation happens inside that+ * expression, so reserving only before the later `chunks.push` would let the+ * copy happen first. It also reserves the concatenated buffer's own byte+ * count before `Buffer.concat` allocates it: the chunk array and the+ * concatenated buffer are two separate live copies. A denied reservation+ * cancels the reader and throws {@link BridgeProcessCapacityError}, copying+ * no further chunk. This function never releases the reservation; the+ * stream owner that created it does, once the whole forward call settles.*- * This function reserves no process-wide byte budget: it enforces only the- * one request's own ceiling. See the "Known behavior: aggregate retained- * body bytes" section in `doc/observability.md` for the accepted aggregate- * ceiling this leaves across every concurrent route.+ * With no `reservation`, this function enforces only the one request's own+ * ceiling, exactly as it did before this parameter existed — the queue+ * transport calls it with no reservation, and its behavior must not change.*/-async function readBridgeForwardResponseBody(response: Response, maxBodyBytes: number): Promise<string> {+async function readBridgeForwardResponseBody(+ response: Response,+ maxBodyBytes: number,+ reservation?: BridgeBodyReservation,+): Promise<Buffer> {const rawContentLength = response.headers.get("content-length");if (rawContentLength) {const contentLength = Number.parseInt(rawContentLength, 10);@@ -1635,7 +1666,7 @@ async function readBridgeForwardResponseBody(response: Response, maxBodyBytes: n}if (!response.body) {- return "";+ return Buffer.alloc(0);}const reader = response.body.getReader();@@ -1651,9 +1682,17 @@ async function readBridgeForwardResponseBody(response: Response, maxBodyBytes: nawait reader.cancel().catch(() => undefined);throw bridgeResponseBodyLimitError(maxBodyBytes);}+ if (reservation && !reservation.reserve(chunkBytes)) {+ await reader.cancel().catch(() => undefined);+ throw new BridgeProcessCapacityError();+ }chunks.push(Buffer.from(value));}- return Buffer.concat(chunks, totalBytes).toString("utf8");+ if (reservation && !reservation.reserve(totalBytes)) {+ await reader.cancel().catch(() => undefined);+ throw new BridgeProcessCapacityError();+ }+ return Buffer.concat(chunks, totalBytes);}const PROCESS_SESSION_PROXY_SCRIPT = "paperclip-process-session-proxy.mjs";@@ -4099,14 +4138,24 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {path: string;query: string;headers: Record<string, string>;- /** The file bridge passes the whole request body here as one string. */- body?: string;+ /** The file bridge passes the whole request body here as one string.+ * The HTTP/2 bridge passes it as the raw `Buffer` it read off the wire. */+ body?: string | Buffer;},signal?: AbortSignal,options?: {suppressDebugLog?: boolean;+ /**+ * The caller's stream reservation owner, if it has one. The HTTP/2+ * bridge passes the stream's own owner here, so the response body copy+ * reserves against the same ceiling the request body copy already+ * reserved against. The queue transport passes no owner, so its+ * response-body read enforces only the per-request size ceiling, exactly+ * as it did before this option existed.+ */+ reservation?: BridgeBodyReservation;},- ): Promise<{ status: number; headers: Record<string, string>; body: string }> => {+ ): Promise<{ status: number; headers: Record<string, string>; body: Buffer }> => {const method = request.method.trim().toUpperCase() || "GET";// The per-request debug log prints the method, the path, and the query. The// duplex path suppresses it, so no route or query rides a log line there. The@@ -4131,14 +4180,19 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {const timeoutSignal = AbortSignal.timeout(forwardTimeoutMs);const forwardSignal = signal ? AbortSignal.any([signal, timeoutSignal]) : timeoutSignal;// Build the request-body init. A GET or a HEAD carries no body. The file- // bridge passes the whole body as one string.+ // bridge passes the whole body as one string; the HTTP/2 bridge passes it+ // as a raw `Buffer`. Undici accepts a `Buffer` request body directly (a+ // `Buffer` is an `ArrayBufferView`), so neither shape needs a conversion.+ // The cast below only bridges a `BodyInit` typing gap: the DOM library+ // type this project's ambient `RequestInit` resolves to excludes a+ // `Buffer`, though Undici accepts one at runtime.const forwardInit: RequestInit = {method,headers,signal: forwardSignal,};- if (method !== "GET" && method !== "HEAD" && typeof request.body === "string") {- forwardInit.body = request.body;+ if (method !== "GET" && method !== "HEAD" && request.body !== undefined) {+ forwardInit.body = request.body as BodyInit;}const response = await fetch(buildBridgeForwardUrl(hostApiUrl, request), forwardInit);if (emitDebugLog) {@@ -4159,10 +4213,23 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {// non-retryable 504 and marks the outcome indeterminate, exactly like an// aborted in-flight forward. The in-sandbox server maps the indeterminate 504// to a non-retryable 409 for both the file bridge and the duplex broker.- let responseBody: string;+ let responseBody: Buffer;try {- responseBody = await readBridgeForwardResponseBody(response, maxBodyBytes);+ responseBody = await readBridgeForwardResponseBody(response, maxBodyBytes, options?.reservation);} catch (error) {+ // A denied reservation is retryable capacity pressure for a safe+ // method, not a body-read fault: rethrow it before the method-safety+ // classification below runs, so it reaches the HTTP/2 bridge's own+ // capacity-denial catch (which answers the retryable 503 and settles+ // the stream) instead of this function turning it into a 502. For an+ // unsafe (mutating) method, the host has already delivered response+ // headers by this point, so it may already have committed the+ // mutation. Rethrowing there too would let the retryable 503 reach a+ // caller that repeats the request, applying the mutation twice. An+ // unsafe method's capacity denial falls through to the same+ // non-retryable indeterminate 504 any other response-body read fault+ // gets below.+ if (error instanceof BridgeProcessCapacityError && isSafeBridgeMethod(method)) throw error;if (isSafeBridgeMethod(method)) {// The method is safe, so a retry cannot double-apply a mutation. Return a// retryable 502 with no indeterminate marker, so the gateway passes it@@ -4170,9 +4237,12 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {return {status: 502,headers: { "content-type": "application/json" },- body: JSON.stringify({- error: error instanceof Error ? error.message : String(error),- }),+ body: Buffer.from(+ JSON.stringify({+ error: error instanceof Error ? error.message : String(error),+ }),+ "utf8",+ ),};}return {@@ -4181,11 +4251,14 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {"content-type": "application/json","x-paperclip-bridge-outcome": "indeterminate",},- body: JSON.stringify({- error: error instanceof Error ? error.message : String(error),- outcome: "indeterminate",- retryable: false,- }),+ body: Buffer.from(+ JSON.stringify({+ error: error instanceof Error ? error.message : String(error),+ outcome: "indeterminate",+ retryable: false,+ }),+ "utf8",+ ),};}const commentMarker = postedIssueCommentLogMarker(method, request.path, response.status, responseBody);@@ -4387,10 +4460,10 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {path: request.pathname,query: request.query,headers: request.headers,- body: request.body.toString("utf8"),+ body: request.body,},request.signal,- { suppressDebugLog: true },+ { suppressDebugLog: true, reservation: request.reservation },);duplexObservability.recordRequest({ latencyMs: Date.now() - dispatchStartMs, outcome: "ok" });return { status: result.status, headers: result.headers, body: result.body };@@ -4403,6 +4476,13 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {const http2Server = createHttp2BridgeServer({bridgeToken,forwardRequest: http2ForwardRequest,+ routes: HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST,+ // The same resolved limit the launch environment hands the+ // sandbox-side gateway (`PAPERCLIP_BRIDGE_MAX_BODY_BYTES`,+ // below), so the host check and the gateway check enforce one+ // value instead of the host silently falling back to the+ // package default.+ maxBodyBytes,onGoaway: () => recordHttp2Loss("session_goaway"),onSessionError: () => recordHttp2Loss("session_error"),});@@ -4510,7 +4590,13 @@ export async function startAdapterExecutionTargetPaperclipBridge(input: {maxBodyBytes,getRuntimeParentContext: input.getRuntimeParentContext,runtimeSpan: input.runtimeSpan,- handleRequest: async (request, options) => forwardBridgeRequest(request, options?.signal),+ // The queue transport writes the response body to a text file, so this+ // is the one place the forward path decodes the response `Buffer` to a+ // UTF-8 string. The queue's own on-wire behavior does not change.+ handleRequest: async (request, options) => {+ const result = await forwardBridgeRequest(request, options?.signal);+ return { status: result.status, headers: result.headers, body: result.body.toString("utf8") };+ },});server = await startSandboxCallbackBridgeServer({runner,diff --git a/packages/adapter-utils/src/http2-bridge-server.test.ts b/packages/adapter-utils/src/http2-bridge-server.test.tsindex 5613620a38..c3acb87aa6 100644--- a/packages/adapter-utils/src/http2-bridge-server.test.ts+++ b/packages/adapter-utils/src/http2-bridge-server.test.ts@@ -2,14 +2,19 @@ import { duplexPair } from "node:stream";import type { Duplex } from "node:stream";import http2 from "node:http2";-import { describe, expect, it } from "vitest";+import { beforeEach, describe, expect, it, vi } from "vitest";import {buildHttp2BridgeForwardUrl,classifyStreamAgainstGoaway,+ createBridgeBodyReservation,+ createBridgeRouteBodyLedger,createHttp2BridgeServer,+ getBridgeBodyReservedBytesForTest,parseCanonicalBridgeRequestPath,+ resetBridgeBodyReservationsForTest,wrapDuplexChannelAsNodeDuplex,+ BridgeProcessCapacityError,DEFAULT_HTTP2_BRIDGE_MAX_BUFFERED_READ_BYTES,DEFAULT_HTTP2_BRIDGE_PING_INTERVAL_MS,DEFAULT_HTTP2_BRIDGE_PING_STALL_MS,@@ -19,6 +24,8 @@ import {HTTP2_BRIDGE_MAX_DEFLATE_DYNAMIC_TABLE_SIZE,HTTP2_BRIDGE_MAX_HEADER_LIST_PAIRS,HTTP2_BRIDGE_MAX_HEADER_LIST_SIZE,+ HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES,+ HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES,HTTP2_BRIDGE_MAX_SESSION_INVALID_FRAMES,HTTP2_BRIDGE_MAX_SESSION_MEMORY,HTTP2_BRIDGE_MAX_SESSION_REJECTED_STREAMS,@@ -32,6 +39,8 @@ import {import {createSandboxHttp2BridgeGateway,DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES,+ HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST,+ type SandboxCallbackBridgeRouteRule,} from "./sandbox-callback-bridge.js";import type { CommandManagedDuplexChannel } from "./command-managed-runtime.js";@@ -82,6 +91,10 @@ interface TestPairOptions {requestBodyTimeoutMs?: number;requestBodyLifetimeCeilingMs?: number;closeGraceMs?: number;+ capacityDenialSettleDeadlineMs?: number;+ responseWriteSettleDeadlineMs?: number;+ maxBodyBytes?: number;+ routes?: readonly SandboxCallbackBridgeRouteRule[];onGoaway?: (record: Http2BridgeGoawayRecord) => void;onSessionError?: (error: Error) => void;onSession?: (session: http2.ServerHttp2Session) => void;@@ -97,7 +110,7 @@ function bindTestServer(options: TestPairOptions = {}) {(async (request: Http2BridgeForwardRequest) => ({status: 200,headers: { "content-type": "application/json" },- body: JSON.stringify({ echoedMethod: request.method, echoedPath: request.pathname }),+ body: Buffer.from(JSON.stringify({ echoedMethod: request.method, echoedPath: request.pathname }), "utf8"),}));const handle = createHttp2BridgeServer({bridgeToken,@@ -107,6 +120,10 @@ function bindTestServer(options: TestPairOptions = {}) {requestBodyTimeoutMs: options.requestBodyTimeoutMs,requestBodyLifetimeCeilingMs: options.requestBodyLifetimeCeilingMs,closeGraceMs: options.closeGraceMs,+ capacityDenialSettleDeadlineMs: options.capacityDenialSettleDeadlineMs,+ responseWriteSettleDeadlineMs: options.responseWriteSettleDeadlineMs,+ maxBodyBytes: options.maxBodyBytes,+ routes: options.routes,onGoaway: options.onGoaway,onSessionError: options.onSessionError,onSession: options.onSession,@@ -130,9 +147,15 @@ function createTestPair(options: TestPairOptions = {}) {/** Open a raw HTTP/2 client session directly against one side of the pair,* bypassing the sandbox gateway. Some tests need direct stream control (an* explicit RST_STREAM, an explicit GOAWAY) the gateway's `forwardRequest`- * abstraction does not expose. */-function connectRawClient(clientSide: Duplex): http2.ClientHttp2Session {- return http2.connect("http://bridge.internal", { createConnection: () => clientSide });+ * abstraction does not expose. `settings`, when given, rides the client's+ * own initial SETTINGS frame — for example `{ initialWindowSize: 0 }` to+ * deny the server any flow-control credit to send response bytes with, a+ * deterministic stall independent of the fake transport's own buffering. */+function connectRawClient(+ clientSide: Duplex,+ settings?: http2.Settings,+): http2.ClientHttp2Session {+ return http2.connect("http://bridge.internal", { createConnection: () => clientSide, settings });}/** Track whether `forwardRequest` ran, so a test can prove a denied or@@ -175,6 +198,13 @@ async function expectSessionStillServesARequest(}describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {+ // Every test in this file shares the module-scope process reservation+ // total. A reservation one test leaves unreleased would otherwise lower+ // the ceiling every later test sees, so each test starts from zero.+ beforeEach(() => {+ resetBridgeBodyReservationsForTest();+ });+it("test_one_session_over_a_fake_channel_forwards_a_request", async () => {const { handle, gateway } = createTestPair();try {@@ -208,7 +238,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {return {status: 200,headers: {},- body: JSON.stringify({ echoedPath: request.pathname }),+ body: Buffer.from(JSON.stringify({ echoedPath: request.pathname }), "utf8"),};},});@@ -316,7 +346,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {// Hold this one open long enough for the test to RST it.await new Promise((resolve) => setTimeout(resolve, 200));}- return { status: 200, headers: {}, body: JSON.stringify({ path: request.pathname }) };+ return { status: 200, headers: {}, body: Buffer.from(JSON.stringify({ path: request.pathname }), "utf8") };},});const rawClient = connectRawClient(clientSide);@@ -362,7 +392,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {// Hold the handler open, so the test controls exactly when the// client-side stream close happens relative to the forward.await forwardHeld;- return { status: 200, body: "{}" };+ return { status: 200, body: Buffer.from("{}", "utf8") };},});const rawClient = connectRawClient(clientSide);@@ -414,7 +444,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {} else if (request.pathname === "/api/agents/survivor") {await survivorHeld;}- return { status: 200, body: JSON.stringify({ path: request.pathname }) };+ return { status: 200, body: Buffer.from(JSON.stringify({ path: request.pathname }), "utf8") };},});const rawClient = connectRawClient(clientSide);@@ -491,7 +521,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {}request.signal.addEventListener("abort", () => reject(new Error("aborted")), { once: true });});- return { status: 200, body: JSON.stringify({ path: request.pathname }) };+ return { status: 200, body: Buffer.from(JSON.stringify({ path: request.pathname }), "utf8") };} finally {liveForwards -= 1;}@@ -579,7 +609,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {requestBodyTimeoutMs: 80,forwardRequest: async (request) => ({status: 200,- body: JSON.stringify({ bodyLength: request.body.byteLength }),+ body: Buffer.from(JSON.stringify({ bodyLength: request.body.byteLength }), "utf8"),}),});const rawClient = connectRawClient(clientSide);@@ -687,7 +717,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {requestBodyLifetimeCeilingMs: 5_000,forwardRequest: async (request) => ({status: 200,- body: JSON.stringify({ bodyLength: request.body.byteLength }),+ body: Buffer.from(JSON.stringify({ bodyLength: request.body.byteLength }), "utf8"),}),});const rawClient = connectRawClient(clientSide);@@ -728,7 +758,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {requestBodyLifetimeCeilingMs: 5_000,forwardRequest: async (request) => {forwarderTracker.markCalled();- return { status: 200, body: JSON.stringify({ bodyLength: request.body.byteLength }) };+ return { status: 200, body: Buffer.from(JSON.stringify({ bodyLength: request.body.byteLength }), "utf8") };},});const rawClient = connectRawClient(clientSide);@@ -845,17 +875,877 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {});it("test_the_host_body_budget_matches_the_stream_limit", () => {- // The multiplier counts every retained `Buffer` and string copy of one- // live forward's request and response body: four exact `Buffer` rows,- // plus two string rows. Each string row applies two bytes to each UTF-16- // code unit of the body limit (`sandbox-callback-bridge.ts`), for an- // accounting peak of eight times the body limit for one live forward.+ // The multiplier counts every retained `Buffer` copy of one live+ // forward's request and response body: four exact `Buffer` rows. The+ // forward path decodes no body to a string, so no row applies the+ // two-bytes-per-UTF-16-code-unit string overhead any more.// `test_live_forward_work_never_passes_the_stream_limit` proves the// count of live forwards never passes `HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS`,// so this multiplier bounds live forwards, not merely open streams.- expect(- HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS * 8 * DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES,- ).toBe(8_388_608);+ // `HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES` uses this exact same formula to+ // enforce it as a real per-route cap, not merely a derived figure.+ const expectedRouteBudget =+ HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS * 4 * DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES;+ expect(expectedRouteBudget).toBe(168_820_736);+ expect(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES).toBe(expectedRouteBudget);+ });++ describe("createBridgeBodyReservation", () => {+ it("test_a_reservation_denies_a_chunk_that_passes_the_process_ceiling", () => {+ const owner = createBridgeBodyReservation();+ try {+ expect(owner.reserve(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES)).toBe(true);+ expect(owner.heldBytes).toBe(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES);+ expect(getBridgeBodyReservedBytesForTest()).toBe(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES);++ // One more byte passes the ceiling: denied, and it holds no bytes.+ expect(owner.reserve(1)).toBe(false);+ expect(owner.heldBytes).toBe(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES);+ expect(getBridgeBodyReservedBytesForTest()).toBe(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES);+ } finally {+ owner.release();+ }+ });++ it("test_a_reservation_denies_the_concatenated_copy_that_passes_the_process_ceiling", () => {+ const owner = createBridgeBodyReservation();+ try {+ // The chunk-array reservation alone passes comfortably.+ const chunkArrayBytes = Math.floor(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES * 0.6);+ expect(owner.reserve(chunkArrayBytes)).toBe(true);++ // The concatenated `Buffer.concat` copy is a second, separate live+ // copy of the same bytes: reserving it on top of the chunk array+ // this owner already holds passes the ceiling, even though the+ // chunk-array reservation alone did not.+ expect(owner.reserve(chunkArrayBytes)).toBe(false);+ expect(owner.heldBytes).toBe(chunkArrayBytes);+ expect(getBridgeBodyReservedBytesForTest()).toBe(chunkArrayBytes);+ } finally {+ owner.release();+ }+ });++ it("test_a_reservation_releases_every_held_byte_one_time_only", () => {+ const owner = createBridgeBodyReservation();+ expect(owner.reserve(1_000)).toBe(true);+ expect(getBridgeBodyReservedBytesForTest()).toBe(1_000);++ owner.release();+ expect(owner.heldBytes).toBe(0);+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);++ // A second release must not double-subtract: the total must not go+ // below zero.+ owner.release();+ expect(owner.heldBytes).toBe(0);+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ });+ });++ describe("route isolation", () => {+ it("test_a_route_ledger_denies_a_reservation_that_passes_its_own_ceiling_with_the_process_ceiling_still_open", () => {+ // The process-wide ceiling has ample room (1 GiB); only this one+ // route's own share is tight. A route-scoped owner must still deny+ // the second reservation, proving the route ceiling is a real,+ // independent check, not merely a reflection of the process total.+ const routeLedger = createBridgeRouteBodyLedger();+ const owner = createBridgeBodyReservation(routeLedger);+ try {+ expect(owner.reserve(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES)).toBe(true);+ expect(routeLedger.reservedBytes).toBe(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES);++ expect(owner.reserve(1)).toBe(false);+ expect(owner.heldBytes).toBe(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES);+ expect(routeLedger.reservedBytes).toBe(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES);+ // The process-wide total only ever grew by what this owner actually+ // holds: the denied byte reserved against neither total.+ expect(getBridgeBodyReservedBytesForTest()).toBe(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES);+ } finally {+ owner.release();+ }+ });++ it("test_one_route_at_its_own_ceiling_never_blocks_a_sibling_routes_reservation", () => {+ // Two routes, two ledgers. Route A spends its own entire ceiling.+ // Route B's reservation, against its own separate ledger, must still+ // succeed: the process-wide total (1 GiB) has room for both routes'+ // ceilings many times over, so only route isolation — not the shared+ // total — could explain a denial here.+ const routeLedgerA = createBridgeRouteBodyLedger();+ const routeLedgerB = createBridgeRouteBodyLedger();+ const ownerA = createBridgeBodyReservation(routeLedgerA);+ const ownerB = createBridgeBodyReservation(routeLedgerB);+ try {+ expect(ownerA.reserve(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES)).toBe(true);+ expect(ownerA.reserve(1)).toBe(false);++ expect(ownerB.reserve(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES)).toBe(true);+ } finally {+ ownerA.release();+ ownerB.release();+ }+ });++ it("test_two_bridge_server_routes_isolate_their_own_reservations_end_to_end", async () => {+ // The end-to-end proof: while route A's own forward holds its own+ // entire per-route ceiling in flight, a second, independent route's+ // request must still succeed. Each `bindTestServer` call is its own+ // route (its own `createHttp2BridgeServer` call, so its own route+ // ledger). Route A's forward parks on `releaseRouteAHold` after it+ // reserves, so its reservation stays live — not merely reserved and+ // immediately released — for the whole time route B's request runs.+ let markRouteAReserved: (() => void) | undefined;+ const routeAReserved = new Promise<void>((resolve) => {+ markRouteAReserved = resolve;+ });+ let releaseRouteAHold: (() => void) | undefined;+ const routeAHold = new Promise<void>((resolve) => {+ releaseRouteAHold = resolve;+ });+ const routeA = bindTestServer({+ forwardRequest: async (request) => {+ // Reserve this route's own entire ceiling against its own ledger,+ // simulating route A at its own documented peak.+ if (!request.reservation.reserve(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES)) {+ throw new BridgeProcessCapacityError();+ }+ markRouteAReserved!();+ await routeAHold;+ return { status: 200 };+ },+ });+ const routeB = bindTestServer({+ forwardRequest: async () => ({ status: 200 }),+ });+ const rawClientA = connectRawClient(routeA.clientSide);+ const rawClientB = connectRawClient(routeB.clientSide);+ try {+ const pendingResponseA = expectSessionStillServesARequest(rawClientA, {+ method: "GET",+ path: "/api/agents/me",+ token: routeA.bridgeToken,+ });++ // Wait until route A's forward actually holds its own ceiling —+ // not merely until the request was sent — before checking route B.+ await routeAReserved;+ expect(getBridgeBodyReservedBytesForTest()).toBeGreaterThanOrEqual(HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES);++ // Route A now holds its own entire per-route ceiling, live. The+ // process-wide total still has headroom (1 GiB minus one 161 MiB+ // route), so route B's independent request succeeds only if its own+ // ledger is genuinely separate from route A's.+ const responseB = await expectSessionStillServesARequest(rawClientB, {+ method: "GET",+ path: "/api/agents/me",+ token: routeB.bridgeToken,+ });+ expect(responseB.status).toBe(200);++ releaseRouteAHold!();+ const responseA = await pendingResponseA;+ expect(responseA.status).toBe(200);+ } finally {+ rawClientA.close();+ rawClientB.close();+ await routeA.handle.close();+ await routeB.handle.close();+ }+ });+ });++ it("test_the_stream_handler_releases_its_reservation_on_completion_error_abort_timeout_and_close", async () => {+ // Every stream, no matter how it ends, must leave the process-wide+ // reservation total at zero: `handleStream`'s `finally` block releases+ // exactly one owner exactly one time on every exit path.++ async function runCompletionCase(): Promise<void> {+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async (request) => {+ request.reservation.reserve(1_000);+ return { status: 200, body: Buffer.from("{}", "utf8") };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const response = await expectSessionStillServesARequest(rawClient, {+ method: "GET",+ path: "/api/agents/me",+ token: bridgeToken,+ });+ expect(response.status).toBe(200);+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ }++ async function runErrorCase(): Promise<void> {+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async (request) => {+ request.reservation.reserve(1_000);+ throw new Error("forward handler fault");+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const response = await expectSessionStillServesARequest(rawClient, {+ method: "GET",+ path: "/api/agents/me",+ token: bridgeToken,+ });+ expect(response.status).toBe(502);+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ }++ async function runAbortCase(): Promise<void> {+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async (request) => {+ request.reservation.reserve(1_000);+ await new Promise<void>((resolve) => {+ if (request.signal.aborted) {+ resolve();+ return;+ }+ request.signal.addEventListener("abort", () => resolve(), { once: true });+ });+ return { status: 200 };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const stream = rawClient.request({+ ":method": "GET",+ ":path": "/api/agents/me",+ authorization: `Bearer ${bridgeToken}`,+ });+ const streamClosed = new Promise<void>((resolve) => {+ stream.on("error", () => resolve());+ stream.on("close", () => resolve());+ });+ stream.end();+ await new Promise((resolve) => setTimeout(resolve, 20));+ stream.close(http2.constants.NGHTTP2_CANCEL);+ await streamClosed;+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ }++ async function runTimeoutCase(): Promise<void> {+ const { handle, bridgeToken, clientSide } = bindTestServer({+ requestBodyTimeoutMs: 30,+ forwardRequest: async () => ({ status: 200 }),+ });+ const rawClient = connectRawClient(clientSide);+ try {+ await new Promise<void>((resolve) => {+ const req = rawClient.request({+ ":method": "POST",+ ":path": "/api/issues/abc/comments",+ authorization: `Bearer ${bridgeToken}`,+ });+ req.on("error", () => resolve());+ req.on("close", () => resolve());+ req.write("partial-body");+ });+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ }++ async function runCloseCase(): Promise<void> {+ const { handle, bridgeToken, clientSide } = bindTestServer({+ requestBodyTimeoutMs: 60_000,+ closeGraceMs: 30,+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const req = rawClient.request({+ ":method": "POST",+ ":path": "/api/issues/abc/comments",+ authorization: `Bearer ${bridgeToken}`,+ });+ req.write("partial-body");+ await new Promise((resolve) => setTimeout(resolve, 20));+ await handle.close();+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ }+ }++ await runCompletionCase();+ await runErrorCase();+ await runAbortCase();+ await runTimeoutCase();+ await runCloseCase();+ }, 10_000);++ it("test_a_backpressured_response_holds_its_reservation_until_the_write_settles", async () => {+ // `stream.end(body)` only queues `body` for asynchronous transmission.+ // A client that never reads its response leaves those bytes in process+ // memory well after `end()` returns, so the reservation covering them+ // must stay held until the write actually settles, not merely until the+ // call to `end()` returns.+ const responseBytes = 4 * 1024 * 1024;+ const responseBody = Buffer.alloc(responseBytes, 0x61);+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async (request) => {+ // Mirror the real forward path: the handler reserves the response+ // body it read from the sandbox target before returning it.+ if (!request.reservation.reserve(responseBytes)) {+ throw new BridgeProcessCapacityError();+ }+ return { status: 200, body: responseBody };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const req = rawClient.request({+ ":method": "GET",+ ":path": "/api/agents/me",+ authorization: `Bearer ${bridgeToken}`,+ });+ // Deliberately paused: no `resume()` and no `data` listener yet, so+ // the client never issues the flow-control credit the host needs to+ // finish writing a response this size.+ const headersReceived = new Promise<void>((resolve) => {+ req.once("response", () => resolve());+ });+ req.end();+ await headersReceived;+ // Give the host's `stream.end()` call, and its microtask queue, a+ // turn to run — the write is now queued but the client still is not+ // draining it.+ await new Promise((resolve) => setImmediate(resolve));+ expect(getBridgeBodyReservedBytesForTest()).toBeGreaterThan(0);++ const drained = new Promise<void>((resolve) => {+ req.on("data", () => {+ // Discard: this test only cares that the bytes left the host.+ });+ req.once("end", () => resolve());+ });+ req.resume();+ await drained;++ // Give the host's `finally` block a turn to run after the write+ // settles.+ await new Promise((resolve) => setImmediate(resolve));+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ });++ it("test_a_peer_reset_holds_the_reservation_until_the_active_forward_settles", async () => {+ let capturedRequest: Http2BridgeForwardRequest | undefined;+ let releaseForward: (() => void) | undefined;+ const forwardHeld = new Promise<void>((resolve) => {+ releaseForward = resolve;+ });+ let forwardSettled = false;+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async (request) => {+ capturedRequest = request;+ // Stand in for a live response-body copy `readBridgeForwardResponseBody`+ // (`execution-target.ts`) would reserve against this same owner.+ request.reservation.reserve(10_000);+ // The forward stays active past the peer's own reset: it settles+ // only when the test releases it below, not merely when the abort+ // signal fires — the same shape a real outbound `fetch` bound to+ // `request.signal` has, since an abort does not settle the fetch+ // promise synchronously.+ await new Promise<void>((resolve) => {+ if (request.signal.aborted) {+ resolve();+ return;+ }+ request.signal.addEventListener("abort", () => resolve(), { once: true });+ });+ await forwardHeld;+ forwardSettled = true;+ return { status: 200, body: Buffer.from("{}", "utf8") };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const stream = rawClient.request({+ ":method": "GET",+ ":path": "/api/agents/me",+ authorization: `Bearer ${bridgeToken}`,+ });+ const streamClosed = new Promise<void>((resolve) => {+ stream.on("error", () => resolve());+ stream.on("close", () => resolve());+ });+ stream.end();+ // Give the request time to reach the server, enter `forwardRequest`,+ // and land its reservation.+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(capturedRequest).toBeDefined();+ expect(getBridgeBodyReservedBytesForTest()).toBe(10_000);++ // The peer resets the stream. The forward call is still pending.+ stream.close(http2.constants.NGHTTP2_CANCEL);+ await streamClosed;+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(capturedRequest!.signal.aborted).toBe(true);+ // The owner still holds its bytes: the active forward has not+ // settled yet, so `handleStream`'s `finally` has not released it.+ expect(forwardSettled).toBe(false);+ expect(getBridgeBodyReservedBytesForTest()).toBe(10_000);++ // Let the forward settle. The `finally` block releases the owner+ // exactly one time, and the process total returns to zero.+ releaseForward!();+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(forwardSettled).toBe(true);+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ });++ it("test_a_stalled_body_releases_its_reservation_while_other_streams_hold_theirs", async () => {+ const releaseByPath = new Map<string, () => void>();+ const { handle, bridgeToken, clientSide } = bindTestServer({+ requestBodyTimeoutMs: 60,+ forwardRequest: async (request) => {+ // Stand in for a live response-body buffer: reserve, then hold the+ // forward open until the test releases this exact path.+ request.reservation.reserve(20_000);+ await new Promise<void>((resolve) => {+ releaseByPath.set(request.pathname, resolve);+ });+ return { status: 200, body: Buffer.from("{}", "utf8") };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const streamA = rawClient.request({+ ":method": "GET",+ ":path": "/api/agents/a",+ authorization: `Bearer ${bridgeToken}`,+ });+ const streamB = rawClient.request({+ ":method": "GET",+ ":path": "/api/agents/b",+ authorization: `Bearer ${bridgeToken}`,+ });+ // Drain each response as it arrives: `session.close()` later in this+ // test's teardown waits for every stream to fully end on both sides,+ // and an unread response can hold a stream open past that point.+ streamA.resume();+ streamB.resume();+ streamA.end();+ streamB.end();+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(40_000);++ // The third stream's request body stalls past the idle bound. It+ // never reaches `forwardRequest`, so it never reserves anything of+ // its own; its owner still holds zero bytes at the moment it releases.+ const streamC = rawClient.request({+ ":method": "POST",+ ":path": "/api/issues/abc/comments",+ authorization: `Bearer ${bridgeToken}`,+ });+ const streamCClosed = new Promise<void>((resolve) => {+ streamC.on("error", () => resolve());+ streamC.on("close", () => resolve());+ });+ streamC.write("partial-body");+ await streamCClosed;+ await new Promise((resolve) => setTimeout(resolve, 20));++ // The stalled stream's release changed nothing: the other two streams+ // keep their reservations.+ expect(getBridgeBodyReservedBytesForTest()).toBe(40_000);++ releaseByPath.get("/api/agents/a")?.();+ releaseByPath.get("/api/agents/b")?.();+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ });++ it("test_a_denied_stream_drains_its_body_and_holds_no_bytes", async () => {+ const { handle, clientSide } = bindTestServer({+ forwardRequest: async () => ({ status: 200 }),+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const response = await new Promise<{ status: number }>((resolve, reject) => {+ // No `authorization` header: the token check denies this stream+ // before its body is ever read.+ const req = rawClient.request({ ":method": "POST", ":path": "/api/issues/abc/comments" });+ let status = 0;+ req.on("response", (headers) => {+ status = Number(headers[":status"]) || 0;+ });+ req.on("data", () => undefined);+ req.on("end", () => resolve({ status }));+ req.on("error", reject);+ req.write(Buffer.alloc(50_000, "a"));+ req.end();+ });+ expect(response.status).toBe(401);+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ });++ it("test_a_request_body_reservation_denial_answers_503_on_the_wire", async () => {+ // A denied request-body chunk must leave the stream alive long enough+ // to answer 503 for real, on the wire — not merely destroy the stream+ // and leave the caller with a bare reset. `respondJson` only queues the+ // 503 write; the handler must wait for that write to settle before it+ // destroys the stream, or the client can see a reset instead of the+ // full response body asserted below.+ const filler = createBridgeBodyReservation();+ expect(filler.reserve(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES - 100)).toBe(true);+ const forwarderTracker = createForwarderCallTracker();+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async () => {+ forwarderTracker.markCalled();+ return { status: 200 };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const response = await new Promise<{ status: number; body: string }>((resolve, reject) => {+ const req = rawClient.request({+ ":method": "POST",+ ":path": "/api/issues/abc/comments",+ authorization: `Bearer ${bridgeToken}`,+ });+ let status = 0;+ let body = "";+ req.setEncoding("utf8");+ req.on("response", (headers) => {+ status = Number(headers[":status"]) || 0;+ });+ req.on("data", (chunk) => (body += chunk));+ req.on("end", () => resolve({ status, body }));+ req.on("error", reject);+ // Ten times the 100 bytes of headroom the filler above left.+ req.end(Buffer.alloc(1_000, "a"));+ });+ expect(response.status).toBe(503);+ expect(forwarderTracker.called).toBe(false);+ // A raw reset delivers no body at all: parsing it here proves the+ // full JSON response actually reached the client, not merely the+ // status line.+ expect(JSON.parse(response.body)).toEqual({+ error: "The bridge host reached its reserved process body byte ceiling. Retry later.",+ });++ // The stream ended with a real response, not a raw reset: the session+ // stays healthy, so a second, complete request still succeeds.+ const survivingResponse = await expectSessionStillServesARequest(rawClient, {+ method: "GET",+ path: "/api/agents/me",+ token: bridgeToken,+ });+ expect(survivingResponse.status).toBe(200);+ } finally {+ rawClient.close();+ await handle.close();+ filler.release();+ }+ });++ it("test_a_stalled_client_still_frees_the_capacity_denial_reservation_and_slot", async () => {+ // The capacity-denial (503) path must not wait forever for its queued+ // write to settle: a client that grants no flow-control credit for the+ // response would otherwise hold this stream's reservation and+ // concurrent-stream slot open indefinitely. A zero initial window+ // denies the server any credit to send the 503 body with, at the+ // protocol layer, regardless of the fake transport's own buffering —+ // the same deterministic stall a real stalled peer produces.+ const filler = createBridgeBodyReservation();+ expect(filler.reserve(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES - 100)).toBe(true);+ const forwarderTracker = createForwarderCallTracker();+ const { handle, bridgeToken, clientSide } = bindTestServer({+ capacityDenialSettleDeadlineMs: 30,+ forwardRequest: async () => {+ forwarderTracker.markCalled();+ return { status: 200 };+ },+ });+ const rawClient = connectRawClient(clientSide, { initialWindowSize: 0 });+ try {+ const startMs = Date.now();+ const closed = new Promise<void>((resolve) => {+ const req = rawClient.request({+ ":method": "POST",+ ":path": "/api/issues/abc/comments",+ authorization: `Bearer ${bridgeToken}`,+ });+ req.on("error", () => undefined);+ req.on("close", () => resolve());+ // Ten times the 100 bytes of headroom the filler above left: the+ // request body itself denies the reservation and enters the 503+ // path.+ req.end(Buffer.alloc(1_000, "a"));+ });+ await closed;+ // The deadline bounded the wait: the stream closed near the deadline+ // bound, not after some much longer natural settle that never comes.+ expect(Date.now() - startMs).toBeLessThan(5_000);+ expect(forwarderTracker.called).toBe(false);++ // The stream's own reservation released even though the client never+ // drained the 503 body — only the filler's own bytes remain held.+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(filler.heldBytes);++ // The stream's slot freed too. Restore a normal flow-control window+ // first: only the denied stream above needs to stall.+ await new Promise<void>((resolve) => rawClient.settings({ initialWindowSize: 65_535 }, () => resolve()));+ const survivingResponse = await expectSessionStillServesARequest(rawClient, {+ method: "GET",+ path: "/api/agents/me",+ token: bridgeToken,+ });+ expect(survivingResponse.status).toBe(200);+ } finally {+ rawClient.close();+ await handle.close();+ filler.release();+ }+ });++ it("test_a_stalled_normal_response_still_frees_its_reservation_and_slot", async () => {+ // The completed-response (200) write path must not wait forever for its+ // queued write to settle either: a client that grants no flow-control+ // credit for the response would otherwise hold this stream's+ // reservation and concurrent-stream slot open indefinitely, the same+ // failure mode the capacity-denial path already guards against. A zero+ // initial window denies the server any credit to send the response body+ // with, at the protocol layer, regardless of the fake transport's own+ // buffering.+ const forwarderTracker = createForwarderCallTracker();+ const { handle, bridgeToken, clientSide } = bindTestServer({+ responseWriteSettleDeadlineMs: 30,+ forwardRequest: async () => {+ forwarderTracker.markCalled();+ return { status: 200, headers: {}, body: Buffer.alloc(1_000, "a") };+ },+ });+ const rawClient = connectRawClient(clientSide, { initialWindowSize: 0 });+ try {+ const startMs = Date.now();+ const closed = new Promise<void>((resolve) => {+ const req = rawClient.request({+ ":method": "GET",+ ":path": "/api/agents/me",+ authorization: `Bearer ${bridgeToken}`,+ });+ req.on("error", () => undefined);+ req.on("close", () => resolve());+ req.end();+ });+ await closed;+ expect(forwarderTracker.called).toBe(true);+ // The deadline bounded the wait: the stream closed near the deadline+ // bound, not after some much longer natural settle that never comes.+ expect(Date.now() - startMs).toBeLessThan(5_000);++ // This stream's own reservation released even though the client never+ // drained the response body.+ await new Promise((resolve) => setTimeout(resolve, 20));+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);++ // The stream's slot freed too. Restore a normal flow-control window+ // first: only the stalled stream above needs to stall.+ await new Promise<void>((resolve) => rawClient.settings({ initialWindowSize: 65_535 }, () => resolve()));+ const survivingResponse = await expectSessionStillServesARequest(rawClient, {+ method: "GET",+ path: "/api/agents/me",+ token: bridgeToken,+ });+ expect(survivingResponse.status).toBe(200);+ } finally {+ rawClient.close();+ await handle.close();+ }+ });++ it("test_a_normal_response_that_settles_in_time_is_left_alone", async () => {+ // A response that genuinely settles inside the deadline must not be+ // force-destroyed: the bounded wait exists only for a stream that never+ // settles on its own.+ const { handle, bridgeToken, clientSide } = bindTestServer({+ responseWriteSettleDeadlineMs: 5_000,+ forwardRequest: async () => ({+ status: 200,+ headers: { "content-type": "application/json" },+ body: Buffer.from(JSON.stringify({ ok: true }), "utf8"),+ }),+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const response = await expectSessionStillServesARequest(rawClient, {+ method: "GET",+ path: "/api/agents/me",+ token: bridgeToken,+ });+ expect(response.status).toBe(200);+ expect(JSON.parse(response.body)).toEqual({ ok: true });+ expect(getBridgeBodyReservedBytesForTest()).toBe(0);+ } finally {+ rawClient.close();+ await handle.close();+ }+ });++ it("test_a_response_reservation_denial_answers_503_on_the_wire", async () => {+ // The response reader on the sandbox target side+ // (`readBridgeForwardResponseBody`) denies its own reservation and+ // throws `BridgeProcessCapacityError` from inside `forwardRequest`,+ // after the forward handler already ran — a different call site than+ // the request-body denial above, but the same 503-before-destroy+ // contract must hold: the stream must stay alive long enough to answer+ // 503 for real, on the wire, not merely destroy and leave the caller+ // with a bare reset.+ const forwarderTracker = createForwarderCallTracker();+ // Only the first call denies: the surviving-request check below reuses+ // the same session for a second, independent request, which must+ // succeed once this first stream's reservation released.+ let forwardCalls = 0;+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async () => {+ forwarderTracker.markCalled();+ forwardCalls += 1;+ if (forwardCalls === 1) {+ throw new BridgeProcessCapacityError();+ }+ return { status: 200 };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const response = await new Promise<{ status: number; body: string }>((resolve, reject) => {+ const req = rawClient.request({+ ":method": "GET",+ ":path": "/api/agents/me",+ authorization: `Bearer ${bridgeToken}`,+ });+ let status = 0;+ let body = "";+ req.setEncoding("utf8");+ req.on("response", (headers) => {+ status = Number(headers[":status"]) || 0;+ });+ req.on("data", (chunk) => (body += chunk));+ req.on("end", () => resolve({ status, body }));+ req.on("error", reject);+ req.end();+ });+ expect(response.status).toBe(503);+ expect(forwarderTracker.called).toBe(true);+ // A raw reset delivers no body at all: parsing it here proves the+ // full JSON response actually reached the client, not merely the+ // status line.+ expect(JSON.parse(response.body)).toEqual({+ error: "The bridge host reached its reserved process body byte ceiling. Retry later.",+ });++ // The stream ended with a real response, not a raw reset: the session+ // stays healthy, so a second, complete request still succeeds.+ const survivingResponse = await expectSessionStillServesARequest(rawClient, {+ method: "GET",+ path: "/api/agents/me",+ token: bridgeToken,+ });+ expect(survivingResponse.status).toBe(200);+ } finally {+ rawClient.close();+ await handle.close();+ }+ });++ it("test_a_denied_concatenated_request_body_never_allocates_the_copy", async () => {+ // The chunk-array copy and the concatenated copy are two separate live+ // buffers, so each must reserve on its own. Leave room for the first but+ // not the second: a correct reader checks the reservation before+ // `Buffer.concat` allocates the copy, so the denied concatenated copy+ // must never call `Buffer.concat` at all.+ const chunkBytes = 200_000;+ const filler = createBridgeBodyReservation();+ expect(filler.reserve(HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES - Math.floor(chunkBytes * 1.5))).toBe(true);+ const concatSpy = vi.spyOn(Buffer, "concat");+ const forwarderTracker = createForwarderCallTracker();+ const { handle, bridgeToken, clientSide } = bindTestServer({+ forwardRequest: async () => {+ forwarderTracker.markCalled();+ return { status: 200 };+ },+ });+ const rawClient = connectRawClient(clientSide);+ try {+ const response = await new Promise<{ status: number; body: string }>((resolve, reject) => {+ const req = rawClient.request({+ ":method": "POST",+ ":path": "/api/issues/abc/comments",+ authorization: `Bearer ${bridgeToken}`,+ });+ let status = 0;+ let body = "";+ req.setEncoding("utf8");+ req.on("response", (headers) => {+ status = Number(headers[":status"]) || 0;+ });+ req.on("data", (chunk) => (body += chunk));+ req.on("end", () => resolve({ status, body }));+ req.on("error", reject);+ req.write(Buffer.alloc(chunkBytes, "a"), () => {+ // Give the host a turn to receive the one chunk and reserve its+ // bytes before this test ends the stream and triggers the+ // concatenated-copy reservation attempt.+ setTimeout(() => {+ // The chunk-array reservation alone must already hold, on top of+ // the filler above, exactly the one chunk's bytes: the+ // concatenated-copy reservation has not run yet, because the+ // stream has not ended.+ expect(getBridgeBodyReservedBytesForTest()).toBe(filler.heldBytes + chunkBytes);+ req.end();+ }, 50);+ });+ });+ expect(response.status).toBe(503);+ expect(forwarderTracker.called).toBe(false);+ expect(concatSpy).not.toHaveBeenCalled();+ } finally {+ concatSpy.mockRestore();+ rawClient.close();+ await handle.close();+ filler.release();+ }});describe("parseCanonicalBridgeRequestPath", () => {@@ -940,7 +1830,7 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {const { handle, clientSide } = bindTestServer({forwardRequest: async (request) => {forwarderTracker.markCalled();- return { status: 200, body: JSON.stringify({ path: request.pathname }) };+ return { status: 200, body: Buffer.from(JSON.stringify({ path: request.pathname }), "utf8") };},});const rawClient = connectRawClient(clientSide);@@ -1045,6 +1935,56 @@ describe("createHttp2BridgeServer + createSandboxHttp2BridgeGateway", () => {}});+ it("test_a_maximum_size_multipart_attachment_upload_fits_through_the_bridge", async () => {+ // A file at the exact attachment content ceiling (10 MiB — the same+ // figure `MAX_ATTACHMENT_BYTES` in `server/src/attachment-types.ts`+ // enforces) still crosses the bridge wrapped in a multipart body: the+ // boundary line and the part's own headers add bytes on top of that file+ // content. The bridge's own body limit needs headroom for that framing,+ // or a valid maximum-size attachment fails here before the server ever+ // sees it.+ const maxAttachmentBytes = 10 * 1024 * 1024;+ const boundary = "----PaperclipTestBoundary1234567890abcdef";+ const preamble = Buffer.from(+ `--${boundary}\r\n` ++ `Content-Disposition: form-data; name="file"; filename="attachment.bin"\r\n` ++ `Content-Type: application/octet-stream\r\n\r\n`,+ "utf8",+ );+ const fileContent = Buffer.alloc(maxAttachmentBytes, 0x61);+ const epilogue = Buffer.from(`\r\n--${boundary}--\r\n`, "utf8");+ const multipartBody = Buffer.concat([preamble, fileContent, epilogue]);++ // The wrapper really does add bytes on top of the file content alone —+ // otherwise this test would prove nothing about framing headroom.+ expect(multipartBody.byteLength).toBeGreaterThan(maxAttachmentBytes);+ expect(multipartBody.byteLength).toBeLessThanOrEqual(DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES);++ let receivedBodyBytes = 0;+ const { gateway, handle } = createTestPair({+ routes: HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST,+ forwardRequest: async (request) => {+ receivedBodyBytes = request.body.byteLength;+ return { status: 201, body: Buffer.from("{}", "utf8") };+ },+ });+ try {+ const response = await gateway.forwardRequest({+ method: "POST",+ path: "/api/companies/co-1/issues/issue-1/attachments",+ query: "",+ headers: { "content-type": `multipart/form-data; boundary=${boundary}` },+ body: multipartBody,+ receivedToken: BRIDGE_TOKEN,+ });+ expect(response.status).toBe(201);+ expect(receivedBodyBytes).toBe(multipartBody.byteLength);+ } finally {+ await gateway.close();+ await handle.close();+ }+ }, 20_000);+it("the sandbox gateway keeps its own token check before it opens a stream", async () => {const forwarderTracker = createForwarderCallTracker();const { gateway, handle } = createTestPair({diff --git a/packages/adapter-utils/src/http2-bridge-server.ts b/packages/adapter-utils/src/http2-bridge-server.tsindex 0d07fc5ae9..3d96787ff2 100644--- a/packages/adapter-utils/src/http2-bridge-server.ts+++ b/packages/adapter-utils/src/http2-bridge-server.ts@@ -52,23 +52,32 @@ export const HTTP2_BRIDGE_ENABLE_PUSH = false;* Open streams. The host keeps one forward, its request body, and its* response body alive for the life of a stream, and — before this file binds* each forward to its own stream's abort signal — a forward can outlive its- * stream's own HTTP/2 slot until the forward's own timeout runs out. Counting- * every retained `Buffer` and string copy of one stream's request and- * response body against the {@link DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES}- * body limit (`sandbox-callback-bridge.ts`) gives an accounting peak of eight- * times that limit for one live forward. This bound is the per-route- * in-flight-body budget: `HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS * 8 *- * DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES` bytes = 4 * 8 * 262,144- * bytes = 8,388,608 bytes for one route.+ * stream's own HTTP/2 slot until the forward's own timeout runs out. The+ * forward path carries a request body and a response body as raw `Buffer`+ * values with no string copy. Counting every retained `Buffer` copy of one+ * stream's request and response body against the+ * {@link DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES} body limit+ * (`sandbox-callback-bridge.ts`) gives an accounting peak of four times that+ * limit for one live forward. This bound is the per-route in-flight-body+ * budget: `HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS * 4 *+ * DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES` bytes = 4 * 4 * 10,551,296+ * bytes = 168,820,736 bytes (161 MiB) for one route. {@link+ * HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES} enforces this figure as a real, live+ * cap on every reservation, so one busy route cannot pass it, no matter how+ * much of the process-wide ceiling below still sits free.*- * Known aggregate behavior: this budget applies to one route only. The host- * process admits up to `DEFAULT_MAX_CONCURRENT_DUPLEX_ROUTES` (128, in- * `plugin-worker-manager.ts`) routes at the same time, and each route holds- * its own 8,388,608-byte peak. The process can therefore retain up to- * 1,073,741,824 bytes (1 GiB) of live body data across every route at once.- * This document accepts that ceiling: the host tracks no process-wide byte- * total, so no single route can starve another route's own budget, but the- * host also enforces no smaller sum across every route.+ * Aggregate behavior: the host process admits up to+ * `DEFAULT_MAX_CONCURRENT_DUPLEX_ROUTES` (128, in `plugin-worker-manager.ts`)+ * routes at the same time. The aggregate across every route is bounded too:+ * every stream's {@link BridgeBodyReservation} owner also reserves against+ * the shared {@link HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES} total+ * (1,073,741,824 bytes, 1 GiB), so the process retains no more than that+ * many live body bytes no matter how many routes or streams run at once. One+ * full-size stream's four retained copies cost `4 * 10,551,296` =+ * 42,205,184 bytes of that total, so the process admits at least 25+ * concurrent full-size streams, spread across at least six routes each at+ * their own per-route ceiling, before it starts denying the rest with a 503+ * response.*/export const HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS = 4;/** One decompressed header list. The Node default is 65535. */@@ -112,6 +121,176 @@ export const HTTP2_BRIDGE_SERVER_OPTIONS: http2.ServerOptions = {streamResetBurst: HTTP2_BRIDGE_STREAM_RESET_BURST,};+// ---------------------------------------------------------------------------+// Process-wide body byte reservation+// ---------------------------------------------------------------------------++/**+ * The most process memory, in bytes, this file lets every route hold in live+ * request and response body buffers at the same time. Every+ * {@link BridgeBodyReservation} owner reserves against this one shared+ * total, so no combination of concurrent streams, across every route, can+ * retain more than this many bytes at once. This value keeps the accepted+ * process ceiling at 1,073,741,824 bytes (1 GiB) — the same ceiling+ * `doc/observability.md` already accepted before the per-body limit rose to+ * 10 MiB — now enforced by this reservation instead of left as an unenforced+ * document note. See the per-route budget comment above for the full+ * accounting.+ */+export const HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES = 1024 * 1024 * 1024;++/**+ * The most memory, in bytes, one route (one {@link createHttp2BridgeServer}+ * call, one sandbox run's bridge session) may hold in live request and+ * response body buffers at the same time, on top of the shared process-wide+ * ceiling above. This is the same per-route figure the budget comment above+ * already derives from stream concurrency: naming it here and checking it on+ * every reservation stops one busy route from spending the whole+ * process-wide ceiling and denying every sibling route admission. See that+ * comment for the full accounting.+ */+export const HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES =+ HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS * 4 * DEFAULT_SANDBOX_CALLBACK_BRIDGE_MAX_BODY_BYTES;++// The process-wide running total, in bytes, every `BridgeBodyReservation`+// owner reserves against. Module-scope state is correct here: one host+// process runs one bridge, and every route and every stream in that process+// must share the same ceiling.+let reservedProcessBodyBytes = 0;++/**+ * One route's own running total, in bytes, against+ * {@link HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES}. `createHttp2BridgeServer`+ * creates exactly one ledger per route and every stream that route ever+ * handles reserves against it, so one route's own activity can never pass+ * its own ceiling, regardless of how much of the process-wide total remains+ * free for other routes.+ */+export interface BridgeRouteBodyLedger {+ /**+ * Reserve `byteCount` more bytes against this route's own ceiling. Returns+ * `false`, and reserves nothing, when the new route total would pass+ * {@link HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES}.+ */+ reserve(byteCount: number): boolean;+ /** Release `byteCount` bytes this route previously reserved. */+ release(byteCount: number): void;+ /** The bytes this route currently holds. */+ readonly reservedBytes: number;+}++/** Create one fresh {@link BridgeRouteBodyLedger}, holding zero bytes. One+ * `createHttp2BridgeServer` call creates exactly one, before its first+ * stream, and every stream that route ever handles shares it. */+export function createBridgeRouteBodyLedger(): BridgeRouteBodyLedger {+ let reservedBytes = 0;+ return {+ reserve(byteCount: number): boolean {+ if (reservedBytes + byteCount > HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES) {+ return false;+ }+ reservedBytes += byteCount;+ return true;+ },+ release(byteCount: number): void {+ reservedBytes -= byteCount;+ },+ get reservedBytes(): number {+ return reservedBytes;+ },+ };+}++/**+ * One HTTP/2 stream's reservation owner. `handleStream` creates exactly one+ * owner per stream and releases it in its existing `finally` block, so every+ * live request or response body buffer that stream produces reserves+ * against the same owner, and the process reclaims those bytes exactly one+ * time when the stream ends.+ */+export interface BridgeBodyReservation {+ /**+ * Reserve `byteCount` more bytes against the process-wide total, and+ * against this owner's route ledger when it has one. Returns `false` and+ * reserves nothing against either total when either check fails. A failed+ * reservation allocates nothing: the caller must not copy the bytes it+ * asked to reserve.+ */+ reserve(byteCount: number): boolean;+ /**+ * Release every byte this owner currently holds. Safe to call more than+ * one time: a second call releases nothing.+ */+ release(): void;+ /** The bytes this owner currently holds. */+ readonly heldBytes: number;+}++/**+ * Create one fresh {@link BridgeBodyReservation} owner, holding zero bytes.+ * A caller that passes `routeLedger` also checks and reserves against that+ * route's own ceiling on every call, isolating this owner's route from every+ * other route sharing the process-wide total. A caller with no route to+ * isolate (a test filling only the process-wide total, for example) omits+ * it, and this owner checks the process-wide ceiling alone.+ */+export function createBridgeBodyReservation(routeLedger?: BridgeRouteBodyLedger): BridgeBodyReservation {+ let heldBytes = 0;+ let released = false;+ return {+ reserve(byteCount: number): boolean {+ if (released) return false;+ if (reservedProcessBodyBytes + byteCount > HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES) {+ return false;+ }+ if (routeLedger && !routeLedger.reserve(byteCount)) {+ return false;+ }+ reservedProcessBodyBytes += byteCount;+ heldBytes += byteCount;+ return true;+ },+ release(): void {+ if (released) return;+ released = true;+ reservedProcessBodyBytes -= heldBytes;+ routeLedger?.release(heldBytes);+ heldBytes = 0;+ },+ get heldBytes(): number {+ return heldBytes;+ },+ };+}++/**+ * A reservation owner denied a request or response body copy because the+ * process-wide ceiling would otherwise be passed. The stream handler answers+ * 503 for this error, not 413: a 413 tells a caller its own body is too+ * large; a 503 tells a caller the host is busy and to retry later.+ */+export class BridgeProcessCapacityError extends Error {+ constructor() {+ super("The bridge host reached its reserved process body byte ceiling. Retry later.");+ this.name = "BridgeProcessCapacityError";+ }+}++/**+ * Test-only. Reset the process-wide reservation total to zero. A test file+ * that exercises {@link createBridgeBodyReservation} must call this between+ * tests, so a reservation one test left unreleased cannot lower the ceiling+ * for a later test.+ */+export function resetBridgeBodyReservationsForTest(): void {+ reservedProcessBodyBytes = 0;+}++/** Test-only. Read the current process-wide reservation total. */+export function getBridgeBodyReservedBytesForTest(): number {+ return reservedProcessBodyBytes;+}+// ---------------------------------------------------------------------------// Duplex channel adapter// ---------------------------------------------------------------------------@@ -463,6 +642,24 @@ export const DEFAULT_HTTP2_BRIDGE_REQUEST_BODY_LIFETIME_CEILING_MS = 480_000;* session that carries a stalled stream would otherwise hold `close()` open* forever, because `session.close()` waits for every open stream to end. */export const DEFAULT_HTTP2_BRIDGE_CLOSE_GRACE_MS = 5_000;+/** The default bound the capacity-denial (503) response path waits for its+ * queued write to settle before it force-destroys the stream. A normal,+ * draining peer settles well inside this bound, so it still receives the+ * full 503 body. A stalled peer that never grants the flow-control credit+ * the write needs would otherwise hold this stream's reservation and+ * {@link HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS} slot open forever. */+export const DEFAULT_HTTP2_BRIDGE_CAPACITY_DENIAL_SETTLE_DEADLINE_MS = 5_000;+/** The default bound the completed-response (normal, non-denial) write path+ * waits for its queued write to settle before it force-destroys the stream.+ * A normal, draining peer settles well inside this bound. A stalled peer+ * that grants no flow-control credit would otherwise hold this stream's+ * reservation and {@link HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS} slot open+ * forever — the same failure mode the capacity-denial path already guards+ * against. Set above {@link DEFAULT_HTTP2_BRIDGE_CAPACITY_DENIAL_SETTLE_DEADLINE_MS}+ * because a completed response can carry a full-size body (up to the+ * configured body-byte ceiling), not just a small JSON error payload, so a+ * slow-but-genuine peer needs more room to drain it. */+export const DEFAULT_HTTP2_BRIDGE_RESPONSE_WRITE_SETTLE_DEADLINE_MS = 30_000;function startHttp2BridgePingWatchdog(session: http2.ServerHttp2Session,@@ -517,7 +714,7 @@ function startHttp2BridgePingWatchdog(export interface Http2BridgeForwardResult {status: number;headers?: Record<string, string>;- body?: Buffer | string;+ body?: Buffer;}/**@@ -538,6 +735,16 @@ export interface Http2BridgeForwardRequest {* never fires for any other stream or for the session.*/signal: AbortSignal;+ /**+ * This stream's one {@link BridgeBodyReservation} owner. A forward handler+ * that itself retains a full response body buffer — `execution-target.ts`+ * does, through `forwardBridgeRequest`'s optional `reservation` option —+ * reserves against this same owner, so the request body and the response+ * body of one stream share one ceiling. `handleStream` releases this owner+ * exactly one time, after the forward call settles; the forward handler+ * must never release it.+ */+ reservation: BridgeBodyReservation;}export type Http2BridgeForwardHandler = (@@ -587,6 +794,14 @@ export interface CreateHttp2BridgeServerOptions {* session to close on its own before it force-destroys the session. The* default is {@link DEFAULT_HTTP2_BRIDGE_CLOSE_GRACE_MS}. */closeGraceMs?: number;+ /** The bound the capacity-denial (503) response path waits for its queued+ * write to settle before it force-destroys the stream. The default is+ * {@link DEFAULT_HTTP2_BRIDGE_CAPACITY_DENIAL_SETTLE_DEADLINE_MS}. */+ capacityDenialSettleDeadlineMs?: number;+ /** The bound the completed-response (normal) write path waits for its+ * queued write to settle before it force-destroys the stalled stream. The+ * default is {@link DEFAULT_HTTP2_BRIDGE_RESPONSE_WRITE_SETTLE_DEADLINE_MS}. */+ responseWriteSettleDeadlineMs?: number;/** The cap, in bytes, on data this server holds once a bound `Duplex`* reports its readable side is full (`push()` returns `false`). Past this* cap the server treats the channel as stuck, not merely slow: see@@ -667,13 +882,21 @@ export interface Http2BridgeBodyBounds {* stream ends for any reason at all — a normal end, an error, a timeout- or* shutdown-triggered `destroy()`, or a peer reset — so the promise always* settles and the caller never awaits a stream that already went away.+ *+ * `onChunk` and `onEnd` decide what the read retains, if anything:+ * {@link readHttp2StreamBody} accumulates chunks and reserves against a+ * {@link BridgeBodyReservation}; {@link drainHttp2StreamBody} discards every+ * chunk and reserves nothing. Either callback may throw to reject the read+ * (a denied reservation, for example) — the throw destroys the stream the+ * same way a size, idle, or lifetime fault does.*/-function readHttp2StreamBody(+function readOrDrainHttp2StreamBody<T>(stream: http2.ServerHttp2Stream,bounds: Http2BridgeBodyBounds,-): Promise<Buffer> {+ onChunk: (chunk: Buffer, totalBytes: number) => void,+ onEnd: () => T,+): Promise<T> {return new Promise((resolve, reject) => {- const chunks: Buffer[] = [];let totalBytes = 0;let settled = false;let idleTimer: ReturnType<typeof setTimeout>;@@ -709,18 +932,153 @@ function readHttp2StreamBody(stream.destroy();return;}- chunks.push(chunk);+ try {+ onChunk(chunk, totalBytes);+ } catch (error) {+ settle(() => reject(error instanceof Error ? error : new Error(String(error))));+ // A denied reservation answers 503 through the caller's own+ // `respondJson` call, after this promise rejects — destroying the+ // stream here, before that call runs, would make it a no-op (a+ // destroyed stream refuses `.respond()`). The caller destroys the+ // stream itself, once its response is actually on the wire. Every+ // other body-read fault has no such response to protect, so it+ // destroys the stream immediately, exactly as before.+ if (!(error instanceof BridgeProcessCapacityError)) {+ stream.destroy();+ }+ return;+ }// The chunk is real progress, so the peer is not stalled: reset the// idle bound. The lifetime ceiling timer above does not reset here.armIdleTimer();});- stream.once("end", () => settle(() => resolve(Buffer.concat(chunks))));+ stream.once("end", () => {+ let result: T;+ try {+ result = onEnd();+ } catch (error) {+ settle(() => reject(error instanceof Error ? error : new Error(String(error))));+ return;+ }+ settle(() => resolve(result));+ });stream.once("error", (error) => settle(() => reject(error instanceof Error ? error : new Error(String(error)))));stream.once("aborted", () => settle(() => reject(new Error("Bridge request stream aborted."))));stream.once("close", () => settle(() => reject(new Error("Bridge request stream closed before it completed."))));});}+/**+ * Read one request body into a `Buffer`. When the caller passes a+ * `reservation`, this reserves each chunk's bytes against it before the+ * chunk joins the retained array, and reserves the concatenated buffer's own+ * byte count before `Buffer.concat` allocates it — the chunk array and the+ * concatenated buffer are two separate live copies, so both reserve. A+ * denied reservation rejects with {@link BridgeProcessCapacityError} and+ * destroys the stream, retaining no further chunk.+ */+function readHttp2StreamBody(+ stream: http2.ServerHttp2Stream,+ bounds: Http2BridgeBodyBounds,+ reservation?: BridgeBodyReservation,+): Promise<Buffer> {+ const chunks: Buffer[] = [];+ let retainedBytes = 0;+ return readOrDrainHttp2StreamBody(+ stream,+ bounds,+ (chunk, totalBytes) => {+ if (reservation && !reservation.reserve(chunk.byteLength)) {+ throw new BridgeProcessCapacityError();+ }+ chunks.push(chunk);+ retainedBytes = totalBytes;+ },+ () => {+ if (reservation && !reservation.reserve(retainedBytes)) {+ throw new BridgeProcessCapacityError();+ }+ return Buffer.concat(chunks);+ },+ );+}++/**+ * Drain and discard one denied stream's request body, under the same size,+ * idle, and lifetime bounds an authenticated request gets, but retaining no+ * chunk and reserving no bytes. `denyRequest` calls this instead of+ * {@link readHttp2StreamBody}, so a stream that never carries a valid bridge+ * token cannot retain a full body buffer merely by sending one.+ */+function drainHttp2StreamBody(stream: http2.ServerHttp2Stream, bounds: Http2BridgeBodyBounds): Promise<void> {+ return readOrDrainHttp2StreamBody(+ stream,+ bounds,+ () => {+ // No `chunks.push`: a denied stream's body content never reaches the+ // forward handler, so this reader retains no chunk and reserves+ // nothing against the process ceiling.+ },+ () => undefined,+ );+}++/**+ * Wait until a stream's queued write actually leaves process memory, or+ * until the stream closes for any other reason. `stream.end(body)` only+ * queues `body` for asynchronous transmission: Node keeps the bytes in+ * memory until HTTP/2 flow control lets them flow, which a slow or+ * backpressured peer can delay well past the moment `end()` returns. A+ * caller that reserves the response body's bytes must hold that reservation+ * across this whole wait, not merely across the call to `end()`, or a+ * backpressured response keeps bytes in memory the ledger already believes+ * it reclaimed.+ *+ * `finish` is the normal settle: every queued byte reached the session. A+ * `close` or an `error` settle the wait the same way, so a peer reset or a+ * destroyed stream cannot leave a caller waiting forever for a `finish`+ * that will never come.+ *+ * `deadlineMs`, when given, bounds the wait itself: past that many+ * milliseconds with none of the three events above, this resolves anyway.+ * A peer that neither drains the write nor resets nor errors the stream —+ * one that simply stalls, granting no flow-control credit — would otherwise+ * hold the wait open forever with none of the three settle events ever+ * firing. The caller stays responsible for destroying the stream once this+ * resolves; a bounded resolve here does not by itself free the stream's+ * slot or its reservation.+ *+ * The returned `settled` flag tells the caller which way this resolved:+ * `true` for a genuine `finish`/`close`/`error` event, `false` for the+ * deadline. A caller that must free the stream's slot only when the peer+ * truly stalled reads this flag instead of destroying an already-finished+ * stream unconditionally.+ */+function waitForHttp2StreamWriteToSettle(+ stream: http2.ServerHttp2Stream,+ deadlineMs?: number,+): Promise<{ settled: boolean }> {+ if (stream.writableFinished || stream.destroyed || stream.closed) return Promise.resolve({ settled: true });+ return new Promise((resolve) => {+ let deadlineTimer: ReturnType<typeof setTimeout> | undefined;+ const onSettle = (settled: boolean): void => {+ stream.removeListener("finish", onFinish);+ stream.removeListener("close", onFinish);+ stream.removeListener("error", onFinish);+ if (deadlineTimer) clearTimeout(deadlineTimer);+ resolve({ settled });+ };+ const onFinish = (): void => onSettle(true);+ stream.once("finish", onFinish);+ stream.once("close", onFinish);+ stream.once("error", onFinish);+ if (deadlineMs !== undefined) {+ deadlineTimer = setTimeout(() => onSettle(false), deadlineMs);+ deadlineTimer.unref?.();+ }+ });+}+function respondJson(stream: http2.ServerHttp2Stream, status: number, body: unknown): void {if (stream.destroyed || stream.closed) return;try {@@ -741,7 +1099,10 @@ function respondJson(stream: http2.ServerHttp2Stream, status: number, body: unkn* peer that leaves the body unfinished keeps the stream open, holding one* of the {@link HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS} slots for as long as* it chooses. Nothing awaits the discard; the caller has already answered- * the request and moves on to the next stream.+ * the request and moves on to the next stream. The discard retains no+ * chunk and reserves no process byte budget: a stream that fails the bridge+ * token check gets no reservation, so it cannot retain a full body buffer+ * merely by sending one.*/function denyRequest(stream: http2.ServerHttp2Stream,@@ -751,7 +1112,7 @@ function denyRequest(): void {respondJson(stream, status, body);if (stream.destroyed || stream.closed) return;- readHttp2StreamBody(stream, bounds).catch(() => {+ drainHttp2StreamBody(stream, bounds).catch(() => {// The idle or lifetime bound above already destroyed the stream, or the// peer reset it first. Either way the slot is free; the discarded body// content is irrelevant to a denial.@@ -773,6 +1134,10 @@ export function createHttp2BridgeServer(options: CreateHttp2BridgeServerOptions)const requestBodyLifetimeCeilingMs =options.requestBodyLifetimeCeilingMs ?? DEFAULT_HTTP2_BRIDGE_REQUEST_BODY_LIFETIME_CEILING_MS;const closeGraceMs = options.closeGraceMs ?? DEFAULT_HTTP2_BRIDGE_CLOSE_GRACE_MS;+ const capacityDenialSettleDeadlineMs =+ options.capacityDenialSettleDeadlineMs ?? DEFAULT_HTTP2_BRIDGE_CAPACITY_DENIAL_SETTLE_DEADLINE_MS;+ const responseWriteSettleDeadlineMs =+ options.responseWriteSettleDeadlineMs ?? DEFAULT_HTTP2_BRIDGE_RESPONSE_WRITE_SETTLE_DEADLINE_MS;const maxBufferedReadBytes = options.maxBufferedReadBytes ?? DEFAULT_HTTP2_BRIDGE_MAX_BUFFERED_READ_BYTES;const readBackpressureStallMs =options.readBackpressureStallMs ?? DEFAULT_HTTP2_BRIDGE_READ_BACKPRESSURE_STALL_MS;@@ -787,6 +1152,11 @@ export function createHttp2BridgeServer(options: CreateHttp2BridgeServerOptions)const server = http2.createServer(HTTP2_BRIDGE_SERVER_OPTIONS);const activeSessions = new Set<http2.ServerHttp2Session>();+ // One route ledger for this one `createHttp2BridgeServer` call. Every+ // stream this route ever handles reserves against it, so this route's own+ // activity can never pass its own share of the process-wide ceiling and+ // deny a sibling route admission.+ const routeLedger = createBridgeRouteBodyLedger();async function handleStream(stream: http2.ServerHttp2Stream,@@ -804,6 +1174,17 @@ export function createHttp2BridgeServer(options: CreateHttp2BridgeServerOptions)stream.once("close", abortForThisStream);stream.once("aborted", abortForThisStream);stream.once("error", abortForThisStream);+ // One reservation owner for this one stream's entire life. Every live+ // request or response body buffer this stream produces reserves against+ // this same owner; the `finally` block below releases it exactly one+ // time, on every exit path (a deny, a body-read fault, a forward fault,+ // or a completed response), including a peer reset or a timeout, both of+ // which route through the abort listeners above into the forward call's+ // combined signal, so the owner keeps its bytes reserved until that+ // forward call actually settles. It reserves against this route's own+ // ledger too, so it can never spend more than this route's own share of+ // the process-wide ceiling.+ const reservation = createBridgeBodyReservation(routeLedger);try {// Accepted security fix 4: the constant-time bridge-token compare runs// before route processing and before header processing. This host check@@ -838,8 +1219,24 @@ export function createHttp2BridgeServer(options: CreateHttp2BridgeServerOptions)let body: Buffer;try {- body = await readHttp2StreamBody(stream, bodyBounds);+ body = await readHttp2StreamBody(stream, bodyBounds, reservation);} catch (error) {+ if (error instanceof BridgeProcessCapacityError) {+ // The reader above left this stream open (undestroyed) exactly so+ // this response could reach the wire. `respondJson` only queues+ // the 503 write; destroying the stream right after queues a+ // RST_STREAM before a backpressured peer drains it, so the client+ // can see a reset instead of the 503. Wait for the write to+ // settle first, then destroy to free this stream's+ // concurrent-stream slot. The wait carries its own deadline: a+ // stalled peer that grants no flow-control credit would otherwise+ // never settle the write, holding this stream's reservation and+ // slot open forever.+ respondJson(stream, 503, { error: error.message });+ await waitForHttp2StreamWriteToSettle(stream, capacityDenialSettleDeadlineMs);+ if (!stream.destroyed) stream.destroy();+ return;+ }respondJson(stream, 413, { error: error instanceof Error ? error.message : String(error) });return;}@@ -853,8 +1250,23 @@ export function createHttp2BridgeServer(options: CreateHttp2BridgeServerOptions)headers: sanitizedHeaders,body,signal: controller.signal,+ reservation,});} catch (error) {+ if (error instanceof BridgeProcessCapacityError) {+ // The forward handler's own response reader denies its reservation+ // and cancels there, but leaves this stream open exactly so this+ // 503 can reach the wire — the same contract the request-body+ // capacity denial above keeps. `respondJson` only queues the+ // write, so wait for it to settle first (bounded, so a stalled+ // peer that grants no flow-control credit cannot hold this+ // stream's reservation and slot open forever), then destroy to+ // free the slot.+ respondJson(stream, 503, { error: error.message });+ await waitForHttp2StreamWriteToSettle(stream, capacityDenialSettleDeadlineMs);+ if (!stream.destroyed) stream.destroy();+ return;+ }respondJson(stream, 502, { error: error instanceof Error ? error.message : String(error) });return;}@@ -868,6 +1280,21 @@ export function createHttp2BridgeServer(options: CreateHttp2BridgeServerOptions)try {stream.respond(responseHeaders);stream.end(result.body);+ // `end()` only queues `result.body`; a slow or backpressured peer+ // can hold those bytes in process memory well after this call+ // returns. Wait for the write to actually settle before the+ // `finally` block below releases the reservation those bytes hold.+ // The wait carries its own deadline: a peer that grants no+ // flow-control credit and never resets or errors the stream would+ // otherwise hold this stream's reservation and+ // HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS slot open forever, the same+ // failure mode the capacity-denial path above already guards+ // against. A genuinely stalled stream gets force-destroyed once the+ // deadline passes; a stream that settled on its own (`finish`,+ // `close`, or `error`) is left alone, since it is already ending or+ // ended.+ const { settled } = await waitForHttp2StreamWriteToSettle(stream, responseWriteSettleDeadlineMs);+ if (!settled && !stream.destroyed) stream.destroy();} catch {// The peer reset the stream (RST_STREAM) between dispatch and response.// One stream's write fault stays local to that stream.@@ -876,7 +1303,12 @@ export function createHttp2BridgeServer(options: CreateHttp2BridgeServerOptions)// Every path above reaches this exactly once: a deny, a body-read// fault, a forward fault, or a completed response. A completed stream// must leak no listener; the session, not this one stream, outlives- // the handler.+ // the handler. The reservation release is idempotent, but this is+ // still the one place this stream's owner ever releases: releasing+ // here, after the response write above has actually settled, keeps+ // the owner's bytes reserved for the whole time Node still holds a+ // live copy of them, not merely until the write call returns.+ reservation.release();stream.removeListener("close", abortForThisStream);stream.removeListener("aborted", abortForThisStream);stream.removeListener("error", abortForThisStream);diff --git a/packages/adapter-utils/src/sandbox-callback-bridge.test.ts b/packages/adapter-utils/src/sandbox-callback-bridge.test.tsindex a967201428..6cc661f5f2 100644--- a/packages/adapter-utils/src/sandbox-callback-bridge.test.ts+++ b/packages/adapter-utils/src/sandbox-callback-bridge.test.ts@@ -14,7 +14,9 @@ import {createFileSystemSandboxCallbackBridgeQueueClient,createSandboxCallbackBridgeAsset,createSandboxCallbackBridgeToken,+ getSandboxBridgeProcessBodyLedgerSource,getSandboxCallbackBridgeServerSource,+ HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST,sandboxCallbackBridgeDirectories,syncRemoteTextFileWithHashSkip,syncSandboxCallbackBridgeEntrypoint,@@ -966,6 +968,24 @@ describe("sandbox callback bridge", () => {await expect(nonJsonResponse.json()).resolves.toEqual({error: "Bridge only accepts JSON request bodies.",});++ // The queue transport keeps its 415 gate for the attachment upload path+ // too: it carries a string envelope only, so it never admits a binary body.+ const attachmentOctetStreamResponse = await fetch(+ `${bridge.baseUrl}/api/companies/co-1/issues/issue-1/attachments`,+ {+ method: "POST",+ headers: {+ authorization: `Bearer ${bridgeToken}`,+ "content-type": "application/octet-stream",+ },+ body: Buffer.from([0x50, 0x4b, 0x03, 0x04]),+ },+ );+ expect(attachmentOctetStreamResponse.status).toBe(415);+ await expect(attachmentOctetStreamResponse.json()).resolves.toEqual({+ error: "Bridge only accepts JSON request bodies.",+ });});it("returns a 502 when the host response times out", async () => {@@ -1425,6 +1445,44 @@ describe("sandbox callback bridge", () => {`Route not allowed: ${request.method} ${request.path}`,);}++ // The HTTP/2 route list adds the two binary attachment routes on top of+ // the documented heartbeat surface.+ const http2Allowed: Array<{ method: string; path: string }> = [+ { method: "POST", path: "/api/companies/co-1/issues/issue-1/attachments" },+ { method: "GET", path: "/api/attachments/att-1/content" },+ ];+ for (const request of http2Allowed) {+ expect(+ authorizeSandboxCallbackBridgeRequestWithRoutes(request, HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST),+ ).toBeNull();+ }++ const http2Denied: Array<{ method: string; path: string }> = [+ // Wrong method for each attachment rule.+ { method: "GET", path: "/api/companies/co-1/issues/issue-1/attachments" },+ { method: "POST", path: "/api/attachments/att-1/content" },+ // Extra path segment for each attachment rule.+ { method: "POST", path: "/api/companies/co-1/issues/issue-1/attachments/att-1" },+ { method: "GET", path: "/api/attachments/att-1/content/extra" },+ ];+ for (const request of http2Denied) {+ expect(+ authorizeSandboxCallbackBridgeRequestWithRoutes(request, HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST),+ ).toBe(`Route not allowed: ${request.method} ${request.path}`);+ }+ });++ it("denies both attachment routes on the default (queue) route list", () => {+ const attachmentRequests: Array<{ method: string; path: string }> = [+ { method: "POST", path: "/api/companies/co-1/issues/issue-1/attachments" },+ { method: "GET", path: "/api/attachments/att-1/content" },+ ];+ for (const request of attachmentRequests) {+ expect(authorizeSandboxCallbackBridgeRequestWithRoutes(request)).toBe(+ `Route not allowed: ${request.method} ${request.path}`,+ );+ }});it("marks command-managed bridge operations with the bridge execution channel", async () => {@@ -3252,7 +3310,11 @@ describe("sandbox callback bridge", () => {bridgeToken,forwardRequest: async (request) => {seenBodies.push(request.body);- return { status: 200, headers: { "content-type": "application/json" }, body: JSON.stringify({ ok: true }) };+ return {+ status: 200,+ headers: { "content-type": "application/json" },+ body: Buffer.from(JSON.stringify({ ok: true }), "utf8"),+ };},});@@ -3278,19 +3340,22 @@ describe("sandbox callback bridge", () => {it("forwards malformed UTF-8 bytes to the HTTP/2 host handler unchanged", async () => {const bridgeToken = createSandboxCallbackBridgeToken();const seenBodies: Buffer[] = [];+ // Byte 0xC3 opens a two-byte UTF-8 sequence; 0x28 is not a valid+ // continuation byte, so this body is not valid UTF-8. The gateway does+ // not decode or validate the body, so these exact bytes must still+ // arrive at the host handler unchanged, and the same bytes must return+ // to the caller unchanged. The host answers with a non-JSON content+ // type, so the round trip proves the gateway applies no format-specific+ // handling on the response leg either.+ const malformedBytes = Buffer.from([0x7b, 0x22, 0x61, 0x22, 0x3a, 0xc3, 0x28, 0x7d]);const gateway = await startHttp2GatewayForTest({bridgeToken,forwardRequest: async (request) => {seenBodies.push(request.body);- return { status: 200, headers: {}, body: "" };+ return { status: 200, headers: { "content-type": "application/octet-stream" }, body: malformedBytes };},});- // Byte 0xC3 opens a two-byte UTF-8 sequence; 0x28 is not a valid- // continuation byte, so this body is not valid UTF-8. The gateway does- // not decode or validate the body, so these exact bytes must still- // arrive at the host handler unchanged.- const malformedBytes = Buffer.from([0x7b, 0x22, 0x61, 0x22, 0x3a, 0xc3, 0x28, 0x7d]);const response = await fetch(`${gateway.baseUrl}/api/issues/issue-1/comments`, {method: "POST",headers: {@@ -3302,6 +3367,8 @@ describe("sandbox callback bridge", () => {expect(response.status).toBe(200);expect(seenBodies).toHaveLength(1);expect(seenBodies[0]?.equals(malformedBytes)).toBe(true);+ const responseBytes = Buffer.from(await response.arrayBuffer());+ expect(responseBytes.equals(malformedBytes)).toBe(true);}, 15_000);it("rejects a request body over maxBodyBytes on the HTTP/2 path before it forwards a byte", async () => {@@ -3313,7 +3380,7 @@ describe("sandbox callback bridge", () => {maxBodyBytes,forwardRequest: async () => {forwardCalls += 1;- return { status: 200, headers: {}, body: "" };+ return { status: 200, headers: {}, body: Buffer.alloc(0) };},});@@ -3333,6 +3400,133 @@ describe("sandbox callback bridge", () => {expect(forwardCalls).toBe(0);}, 15_000);+ /** A resolvable gate a test can await, then release on its own schedule. */+ function createGate(): { reached: Promise<void>; reach: () => void } {+ let reach!: () => void;+ const reached = new Promise<void>((resolve) => {+ reach = resolve;+ });+ return { reached, reach };+ }++ it("denies a body once concurrent reservations reach the gateway's own process ledger ceiling, then admits again once they release", async () => {+ // The per-body maxBodyBytes cap alone does not bound how many bodies+ // this gateway process holds in memory at once. The generated gateway's+ // own process ledger gives it a second, independent ceiling: 4+ // concurrent max-size bodies, each counted twice (the retained chunk+ // array and the concatenated copy), exactly fills maxBodyBytes * 8. This+ // drives 4 concurrent requests to that exact ceiling, held open by a+ // forward call that does not return, then proves a 5th is denied and a+ // later one succeeds once the 4 held requests release their bytes.+ const bridgeToken = createSandboxCallbackBridgeToken();+ const maxBodyBytes = 400;+ const holdBody = Buffer.alloc(maxBodyBytes, 0x42);+ const HOLD_COUNT = 4;+ const reachedGates = Array.from({ length: HOLD_COUNT }, () => createGate());+ const releaseGates = Array.from({ length: HOLD_COUNT }, () => createGate());+ let forwardCalls = 0;+ const gateway = await startHttp2GatewayForTest({+ bridgeToken,+ maxBodyBytes,+ forwardRequest: async () => {+ const callIndex = forwardCalls;+ forwardCalls += 1;+ if (callIndex < HOLD_COUNT) {+ reachedGates[callIndex]!.reach();+ await releaseGates[callIndex]!.reached;+ }+ return { status: 200, headers: {}, body: Buffer.alloc(0) };+ },+ });++ const postHoldBody = () =>+ fetch(`${gateway.baseUrl}/api/issues/issue-1/comments`, {+ method: "POST",+ headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/json" },+ body: holdBody,+ });++ const holdResponses = Promise.all(Array.from({ length: HOLD_COUNT }, () => postHoldBody()));+ // Each held request's own forward call started, which only happens+ // after readBodyBytes already reserved that request's bytes, so all 4+ // reservations are live once every gate below resolves.+ await Promise.all(reachedGates.map((gate) => gate.reached));++ const deniedResponse = await postHoldBody();+ expect(deniedResponse.status).toBe(503);+ await expect(deniedResponse.json()).resolves.toMatchObject({+ error: "The bridge gateway process reached its reserved body byte ceiling. Retry later.",+ });+ // The denial happened during the body read, before this stream's own+ // forward call ever ran.+ expect(forwardCalls).toBe(HOLD_COUNT);++ releaseGates.forEach((gate) => gate.reach());+ const responses = await holdResponses;+ for (const response of responses) {+ expect(response.status).toBe(200);+ }++ // Every held reservation released once its own forward call settled: a+ // fresh request at the same size now succeeds again.+ const recoveredResponse = await postHoldBody();+ expect(recoveredResponse.status).toBe(200);+ }, 15_000);++ it("maps an indeterminate host outcome to a non-retryable 409 on the HTTP/2 gateway for an unsafe method", async () => {+ // The host answers a mutating request's response-body capacity denial with a+ // 504 and the indeterminate outcome header, exactly like an aborted in-flight+ // forward: the host may have already committed the mutation, so the status+ // must not be retryable. The HTTP/2 gateway must map that 504 to a 409, the+ // same map the file gateway already applies, so a standard retry policy does+ // not repeat the mutation. The outcome header and body must survive the map.+ const bridgeToken = createSandboxCallbackBridgeToken();+ const indeterminateBody = JSON.stringify({ error: "response body over limit", outcome: "indeterminate" });+ const gateway = await startHttp2GatewayForTest({+ bridgeToken,+ forwardRequest: async () => ({+ status: 504,+ headers: { "content-type": "application/json", "x-paperclip-bridge-outcome": "indeterminate" },+ body: Buffer.from(indeterminateBody, "utf8"),+ }),+ });++ const response = await fetch(`${gateway.baseUrl}/api/issues/issue-1/comments`, {+ method: "POST",+ headers: { authorization: `Bearer ${bridgeToken}`, "content-type": "application/json" },+ body: JSON.stringify({ note: "test" }),+ });++ expect(response.status).toBe(409);+ expect(response.status).toBeLessThan(500);+ expect(response.headers.get("x-paperclip-bridge-outcome")).toBe("indeterminate");+ const responseBytes = Buffer.from(await response.arrayBuffer());+ expect(responseBytes.toString("utf8")).toBe(indeterminateBody);+ }, 15_000);++ it("passes through a safe method's retryable 503 on the HTTP/2 gateway unchanged", async () => {+ // A safe method's response-body capacity denial carries no indeterminate+ // outcome header, so the gateway must keep it retryable: the map above must+ // not fire on a plain 503.+ const bridgeToken = createSandboxCallbackBridgeToken();+ const gateway = await startHttp2GatewayForTest({+ bridgeToken,+ forwardRequest: async () => ({+ status: 503,+ headers: { "content-type": "application/json" },+ body: Buffer.from(JSON.stringify({ error: "response body over limit" }), "utf8"),+ }),+ });++ const response = await fetch(`${gateway.baseUrl}/api/issues/issue-1/comments`, {+ method: "GET",+ headers: { authorization: `Bearer ${bridgeToken}` },+ });++ expect(response.status).toBe(503);+ expect(response.headers.get("x-paperclip-bridge-outcome")).toBeNull();+ }, 15_000);+it("rejects a request body over maxBodyBytes on the queue path before it writes the queue file", async () => {const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-bridge-queue-maxbody-"));cleanupDirs.push(rootDir);@@ -3453,3 +3647,40 @@ describe("sandbox callback bridge", () => {expect(seenRequests[0]?.body).toBe(requestBodyText);});});++interface EmbeddedBridgeProcessBodyLedger {+ reserve(byteCount: number): boolean;+ release(byteCount: number): void;+ readonly reservedBytes: number;+}++// This describe block covers the zero-dependency process body-byte ledger+// every generated gateway embeds (`BRIDGE_PROCESS_BODY_LEDGER_SOURCE`),+// exercised directly with no spawned process involved, the same way the+// codec source in `execution-target-sandbox.test.ts` gets its own direct+// coverage.+describe("embedded sandbox gateway process body ledger", () => {+ it("reserves and releases bytes against its own ceiling, denying only once it is exceeded", () => {+ const ledgerFactory = new Function(+ `${getSandboxBridgeProcessBodyLedgerSource()}\nreturn createBridgeProcessBodyLedger;`,+ ) as unknown as () => (maxBytes: number) => EmbeddedBridgeProcessBodyLedger;+ const createBridgeProcessBodyLedger = ledgerFactory();+ const ledger = createBridgeProcessBodyLedger(100);++ expect(ledger.reserve(60)).toBe(true);+ expect(ledger.reservedBytes).toBe(60);+ // A denied reservation reserves nothing: the total stays exactly what+ // the first call reserved.+ expect(ledger.reserve(41)).toBe(false);+ expect(ledger.reservedBytes).toBe(60);+ expect(ledger.reserve(40)).toBe(true);+ expect(ledger.reservedBytes).toBe(100);++ ledger.release(60);+ expect(ledger.reservedBytes).toBe(40);+ // Room freed by the release admits a request the full ceiling would+ // have denied.+ expect(ledger.reserve(60)).toBe(true);+ expect(ledger.reservedBytes).toBe(100);+ });+});diff --git a/packages/adapter-utils/src/sandbox-callback-bridge.ts b/packages/adapter-utils/src/sandbox-callback-bridge.tsindex de4bdb1c50..8a9ac43b70 100644--- a/packages/adapter-utils/src/sandbox-callback-bridge.ts+++ b/packages/adapter-utils/src/sandbox-callback-bridge.ts@@ -20,7 +20,23 @@ const DEFAULT_BRIDGE_POLL_INTERVAL_MS = 100;const DEFAULT_BRIDGE_RESPONSE_TIMEOUT_MS = 30_000;const DEFAULT_BRIDGE_STOP_TIMEOUT_MS = 2_000;const DEFAULT_BRIDGE_MAX_QUEUE_DEPTH = 64;-const DEFAULT_BRIDGE_MAX_BODY_BYTES = 256 * 1024;+// A `BridgeBodyReservation` owner (`http2-bridge-server.ts`) now bounds the+// process-wide total of live request and response body bytes at+// `HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES` (1 GiB), so this per-body limit can+// rise to the same ceiling `MAX_ATTACHMENT_BYTES`+// (`server/src/attachment-types.ts`) already accepts, with no unbounded+// growth in process memory.+//+// The attachment upload route carries its file inside a multipart body, so a+// file at the exact `MAX_ATTACHMENT_BYTES` ceiling needs more than+// `MAX_ATTACHMENT_BYTES` raw bytes to cross the bridge: the multipart+// boundary line, each part's `Content-Disposition` and `Content-Type`+// headers, and the small JSON metadata field this route also accepts all add+// bytes on top of the file content the server measures. This headroom+// covers that framing with a wide margin, so a valid maximum-size attachment+// never fails at the bridge before the server ever sees it.+const BRIDGE_MULTIPART_FRAMING_HEADROOM_BYTES = 64 * 1024;+const DEFAULT_BRIDGE_MAX_BODY_BYTES = 10 * 1024 * 1024 + BRIDGE_MULTIPART_FRAMING_HEADROOM_BYTES;// Per-iteration timeout for one poll-loop client call. A healthy control-plane// round trip finishes in well under one second, so 10s is far above a normal// iteration and never false-fires on a slow-but-live call. It is also well@@ -183,6 +199,16 @@ export const DEFAULT_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST: readonly SandboxCa{ method: "DELETE", path: /^\/api\/routine-triggers\/[^/]+$/ },] as const;+// The HTTP/2 bridge carries the request body as raw bytes, so it can admit+// the two binary attachment routes. The queue transport keeps+// `DEFAULT_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST`, because its envelope+// carries a string body only.+export const HTTP2_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST: readonly SandboxCallbackBridgeRouteRule[] = [+ ...DEFAULT_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST,+ { method: "POST", path: /^\/api\/companies\/[^/]+\/issues\/[^/]+\/attachments$/ },+ { method: "GET", path: /^\/api\/attachments\/[^/]+\/content$/ },+] as const;+export const DEFAULT_SANDBOX_CALLBACK_BRIDGE_HEADER_ALLOWLIST = ["accept","content-type",@@ -2039,6 +2065,48 @@ export function getSandboxDuplexGatewayCodecSource(): string {return DUPLEX_GATEWAY_CODEC_SOURCE;}+/**+ * Zero-dependency source for the gateway's own aggregate body-byte ledger.+ * `readBodyBytes` in the generated gateway reserves against one instance of+ * this ledger before it retains a chunk and before it concatenates the final+ * buffer, so a burst of concurrent requests cannot grow the gateway+ * process's own memory without limit. This ledger is separate from, and+ * independent of, the ceiling `http2-bridge-server.ts` enforces on the host+ * side of the bridge connection: each side bounds only the memory in its own+ * process. It uses no global beyond plain JavaScript, so it embeds inside the+ * gateway template literal with no escape. A test wraps this source directly+ * to exercise `reserve`/`release`, the same way+ * {@link getSandboxDuplexGatewayCodecSource} lets a test exercise the codec.+ */+const BRIDGE_PROCESS_BODY_LEDGER_SOURCE = `function createBridgeProcessBodyLedger(maxBytes) {+ let reservedBytes = 0;+ return {+ reserve(byteCount) {+ if (reservedBytes + byteCount > maxBytes) {+ return false;+ }+ reservedBytes += byteCount;+ return true;+ },+ release(byteCount) {+ reservedBytes -= byteCount;+ },+ get reservedBytes() {+ return reservedBytes;+ },+ };+}`;++/**+ * Return the exact zero-dependency ledger source the generated gateway+ * embeds. A test wraps this source and calls `createBridgeProcessBodyLedger`+ * to prove the embedded copy reserves and releases bytes correctly, with no+ * spawned process involved.+ */+export function getSandboxBridgeProcessBodyLedgerSource(): string {+ return BRIDGE_PROCESS_BODY_LEDGER_SOURCE;+}+export function getSandboxCallbackBridgeServerSource(): string {return `import { randomUUID, timingSafeEqual } from "node:crypto";import { createServer } from "node:http";@@ -2119,6 +2187,32 @@ process.on("unhandledRejection", (reason) => {// gateway ignores it.${DUPLEX_GATEWAY_CODEC_SOURCE}+// The embedded zero-dependency process body-byte ledger. Both gateway modes+// use it: readBodyBytes reserves against it, and each mode's request+// handler releases what it reserved once the body is no longer needed.+${BRIDGE_PROCESS_BODY_LEDGER_SOURCE}++// The multiplier matches HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS (4) in+// http2-bridge-server.ts, doubled because readBodyBytes reserves a body's+// bytes twice: once for the retained chunk array, once for the concatenated+// copy, since both are live buffers at once. This gives the gateway process+// its own aggregate ceiling on live request-body bytes, so a burst of+// concurrent requests cannot grow this process's memory without limit, even+// though every individual body already passes the maxBodyBytes check below.+// This ceiling is independent of, and separate from, the ceiling the host+// enforces on its own side of the bridge connection.+const maxProcessBodyBytes = maxBodyBytes * 8;+const processBodyLedger = createBridgeProcessBodyLedger(maxProcessBodyBytes);++// A denied process-ledger reservation answers 503: the sandbox client should+// retry once other in-flight bodies finish and release their bytes, the same+// retry contract the host side gives for its own capacity denial.+class BridgeProcessCapacityError extends Error {+ constructor() {+ super("The bridge gateway process reached its reserved body byte ceiling. Retry later.");+ }+}+function sleep(ms) {return new Promise((resolve) => setTimeout(resolve, ms));}@@ -2136,22 +2230,49 @@ function normalizeHeaders(headers) {return out;}+// Reserves each chunk's bytes against the process ledger before the chunk+// joins the retained array, and reserves the concatenated buffer's own byte+// count before Buffer.concat allocates it, mirroring the order+// readHttp2StreamBody enforces on the host side. Returns the body buffer+// together with a release function: the caller must call release exactly+// once, after the body is no longer needed, so its reserved bytes return to+// the ledger on completion, on an error the caller raises later, on a client+// abort, and on a timeout — every path funnels through the caller's own+// finally block. A read that fails here (the size limit, or a denied+// process reservation) releases its own partial reservation immediately, so+// no caller-side release call is needed for that path.async function readBodyBytes(req) {const chunks = [];let totalBytes = 0;- for await (const chunk of req) {- const nextChunk = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);- chunks.push(nextChunk);- totalBytes += nextChunk.byteLength;- if (totalBytes > maxBodyBytes) {- throw new Error("Bridge request body exceeded the configured size limit.");+ let reservedBytes = 0;+ try {+ for await (const chunk of req) {+ const nextChunk = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);+ totalBytes += nextChunk.byteLength;+ if (totalBytes > maxBodyBytes) {+ throw new Error("Bridge request body exceeded the configured size limit.");+ }+ if (!processBodyLedger.reserve(nextChunk.byteLength)) {+ throw new BridgeProcessCapacityError();+ }+ reservedBytes += nextChunk.byteLength;+ chunks.push(nextChunk);+ }+ if (!processBodyLedger.reserve(totalBytes)) {+ throw new BridgeProcessCapacityError();}+ reservedBytes += totalBytes;+ const body = Buffer.concat(chunks);+ return { body, release: () => processBodyLedger.release(reservedBytes) };+ } catch (error) {+ processBodyLedger.release(reservedBytes);+ throw error;}- return Buffer.concat(chunks);}async function readBody(req) {- return (await readBodyBytes(req)).toString("utf8");+ const { body, release } = await readBodyBytes(req);+ return { body: body.toString("utf8"), release };}function tokensMatch(received) {@@ -2211,6 +2332,12 @@ async function runFileGateway() {}const server = createServer(async (req, res) => {+ // readBody reserves the body's bytes against the process ledger and+ // hands back a release function; this holds it so the finally below+ // releases those bytes exactly once no matter how this handler ends —+ // its normal completion, a thrown error, a client abort, or a deadline+ // timeout all reach the same finally.+ let releaseBodyReservation = null;try {const auth = req.headers.authorization || "";const receivedToken = auth.startsWith("Bearer ") ? auth.slice("Bearer ".length) : "";@@ -2236,7 +2363,8 @@ async function runFileGateway() {return;}const requestId = randomUUID();- const requestBody = await readBody(req);+ const { body: requestBody, release } = await readBody(req);+ releaseBodyReservation = release;const payload = {id: requestId,method: req.method || "GET",@@ -2284,7 +2412,13 @@ async function runFileGateway() {}res.end(typeof response.body === "string" ? response.body : "");} catch (error) {- writeJsonResponse(res, 502, { error: error instanceof Error ? error.message : String(error) });+ // A denied process-ledger reservation is retryable: the caller should+ // try again once other in-flight bodies release their bytes. Every+ // other body-read or handling fault stays a generic 502.+ const status = error instanceof BridgeProcessCapacityError ? 503 : 502;+ writeJsonResponse(res, status, { error: error instanceof Error ? error.message : String(error) });+ } finally {+ releaseBodyReservation?.();}});@@ -2465,6 +2599,12 @@ function runHttp2Gateway() {}const server = createServer(async (req, res) => {+ // readBodyBytes reserves the body's bytes against the process ledger+ // and hands back a release function; this holds it so the finally+ // below releases those bytes exactly once no matter how this handler+ // ends — its normal completion, a thrown error, a client abort, or a+ // deadline timeout all reach the same finally.+ let releaseBodyReservation = null;try {const auth = req.headers.authorization || "";const receivedToken = auth.startsWith("Bearer ") ? auth.slice("Bearer ".length) : "";@@ -2477,12 +2617,8 @@ function runHttp2Gateway() {return;}const url = new URL(req.url || "/", "http://127.0.0.1");- const contentType = typeof req.headers["content-type"] === "string" ? req.headers["content-type"] : "";- if (req.method && req.method !== "GET" && req.method !== "HEAD" && !/json/i.test(contentType)) {- writeJsonResponse(res, 415, { error: "Bridge only accepts JSON request bodies." });- return;- }- const requestBodyBuffer = await readBodyBytes(req);+ const { body: requestBodyBuffer, release } = await readBodyBytes(req);+ releaseBodyReservation = release;let response;try {response = await forwardOverHttp2({@@ -2496,14 +2632,32 @@ function runHttp2Gateway() {writeJsonResponse(res, 502, { error: error instanceof Error ? error.message : String(error) });return;}- res.statusCode = typeof response.status === "number" ? response.status : 200;+ // The host marks a possibly-committed mutation with an indeterminate outcome.+ // The host cannot cancel a host operation that is in flight, so the mutation+ // may have committed before the worker aborted the handler. A 5xx status is+ // retryable by convention, so a caller that retries 5xx would apply the+ // mutation twice. Map the indeterminate outcome to a non-retryable 409, so a+ // standard retry policy does not repeat the request. The outcome header and+ // body stay, so a caller that reads them still sees the indeterminate result.+ const bridgeOutcome = (response.headers || {})["x-paperclip-bridge-outcome"];+ if (bridgeOutcome === "indeterminate") {+ res.statusCode = 409;+ } else {+ res.statusCode = typeof response.status === "number" ? response.status : 200;+ }for (const [key, value] of Object.entries(response.headers || {})) {if (typeof value !== "string" || key.toLowerCase() === "content-length") continue;res.setHeader(key, value);}res.end(response.body);} catch (error) {- writeJsonResponse(res, 502, { error: error instanceof Error ? error.message : String(error) });+ // A denied process-ledger reservation is retryable: the caller should+ // try again once other in-flight bodies release their bytes. Every+ // other body-read or handling fault stays a generic 502.+ const status = error instanceof BridgeProcessCapacityError ? 503 : 502;+ writeJsonResponse(res, status, { error: error instanceof Error ? error.message : String(error) });+ } finally {+ releaseBodyReservation?.();}});diff --git a/server/src/services/plugin-worker-manager.ts b/server/src/services/plugin-worker-manager.tsindex bdcf10cb8b..af6721f3d4 100644--- a/server/src/services/plugin-worker-manager.ts+++ b/server/src/services/plugin-worker-manager.ts@@ -3558,11 +3558,18 @@ export interface PluginWorkerManagerOptions {** Known aggregate behavior: this ceiling bounds route count only, not* retained bytes. Each HTTP/2 bridge route bounds its own retained body- * bytes to 8,388,608 bytes (see `HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS` in- * `http2-bridge-server.ts`), so this route ceiling caps the process's- * aggregate retained body bytes at 128 * 8,388,608 = 1,073,741,824 bytes- * (1 GiB). This is accepted, known behavior, not a defect: the process- * tracks no aggregate byte ledger across routes.+ * bytes to 168,820,736 bytes (see `HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS` in+ * `http2-bridge-server.ts`), so this route ceiling alone would let the+ * process's aggregate retained body bytes reach 128 * 168,820,736 =+ * 21,609,054,208 bytes (about 20.1 GiB) if nothing else bounded it. It does+ * not reach that figure: every stream's `BridgeBodyReservation` owner+ * (`http2-bridge-server.ts`) reserves against the shared+ * `HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES` total (1,073,741,824 bytes, 1 GiB)+ * across every route, so that reservation — not this route ceiling — is the+ * process's real aggregate-byte bound. `HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES`+ * (`http2-bridge-server.ts`) adds a second bound scoped to one route at a+ * time, so one busy route cannot spend that whole process-wide total by+ * itself and deny every sibling route admission.*/export const DEFAULT_MAX_CONCURRENT_DUPLEX_ROUTES = 128;