Reverse engineer anything with agents, from app behavior down to native binaries.
TypeScript agent-skillsai-agentsbinary-analysisclaude-codeclicodexcordisdecompilerdeveloper-toolsdshdsh-pluginghidrahopperllmmcpmodel-context-protocolreareverse-engineeringstatic-analysistouhou
feat(captures): make the HAR/SAZ/qlog decoder heap configurable (#6299)
morluto306ed799320e
diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.mdindex adf475e0f..a75b279f0 100644--- a/.agents/skills/reverse-engineer-anything/SKILL.md+++ b/.agents/skills/reverse-engineer-anything/SKILL.md@@ -126,7 +126,10 @@ explicit path or endpoint and do not need it.connect Everywhere's separate MCP server for live traffic.- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Selectoriginal record ordinals when useful; inspect producer fields and byte/number- sidecars without fetching recorded URLs or claiming live attribution. See+ sidecars without fetching recorded URLs or claiming live attribution. HAR/SAZ/qlog+ old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,+ floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not+ raise it. See[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstreamprofile and credential exclusions. Qlog support starts with aioquic 1.5.0JSON 0.3; stream coverage and endpoint loss declarations do not recover bodiesdiff --git a/docs/web-network-captures.md b/docs/web-network-captures.mdindex a998083c8..e4afbde3f 100644--- a/docs/web-network-captures.md+++ b/docs/web-network-captures.md@@ -716,8 +716,17 @@ budget. Those budgets have no runtime override. `capture_timeout_ms` does notraise them. Exceeding a budget returns no partial success.Oversized selected captures return an input error identifying `capture_path`and the byte budget; owned reply/output failures remain output or provider errors.-HAR, SAZ and qlog run in an owned-192 MiB Node old-generation heap with one V8 worker. Mitmproxy decoding applies+HAR, SAZ and qlog run in an owned Node old-generation heap with one V8 worker.+`REA_CAPTURE_DECODER_OLD_GENERATION_MIB` is a decimal integer of mebibytes from+8 to 16384. Unset uses 192, the previous constant. Invalid values fail+configuration and name the setting; they do not fall back to 192. The setting+is read from the environment that starts REA, so a change needs a restart.+A memory failure names the effective mebibytes, whether they came from the+default or the setting, the default, the floor, and the maximum. The decoder+passes that value as `--max-old-space-size` and removes inherited+`NODE_OPTIONS`, so additional host memory and `NODE_OPTIONS` do not raise the+heap. The 8 MiB semi-space stays fixed; a smaller old generation aborts Node+during startup instead of enforcing a limit. Mitmproxy decoding appliesLinux limits of 768 MiB address space, 30 seconds CPU and 96 MiB file output;both commands use `capture_timeout_ms` (default 30000) and independently supervisedprocess cleanup. Owned decoder diagnostics retain at most 1 MiB across stdout@@ -725,15 +734,18 @@ and stderr; the collector drops overflow as it arrives, reports the limitfailure and verifies cleanup. Heap limits alone do not establish aggregate RSS limits.SAZ and qlog refuse decoding when neither a Node heap limit nor a Windows Job Object bound can be enforced. Existing HAR availability is preserved.When REA runs on Bun, the HAR/SAZ/qlog decoder runs on Bun, which ignores these V8 heap-flags; its limitations and resource failures then omit the heap sizes. On+flags; its limitations and resource failures then omit the V8 heap sizes. OnWindows, the decoder's owned Job Object instead limits its committed memory to-704 MiB, the 192 MiB budget plus a 512 MiB reserve for Bun itself. The-limitations state that limit, and reaching it returns `resource_constraint`-with `process_memory_limit_bytes`, whatever exit status Bun reports.+the effective `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` value plus a 512 MiB+reserve for Bun itself. The default 192 MiB setting therefore yields 704 MiB.+The limitations state that limit, and reaching it returns `resource_constraint`+with `process_memory_limit_bytes`, whatever exit status Bun reports. Other Bun+hosts enforce no decoder heap limit, so raising the setting does not bound them.Observed decoder memory exhaustion returns `resource_constraint`, with the-selected capture, applicable limits and retained process diagnostics. HAR/SAZ/qlog's-old-generation heap is fixed; inherited `NODE_OPTIONS` cannot raise it.+selected capture, applicable limits and retained process diagnostics. Raising+host memory or `NODE_OPTIONS` does not raise the old-generation heap. Set+`REA_CAPTURE_DECODER_OLD_GENERATION_MIB` and restart REA.Record selection happens after complete decoding and does not reduce thatworkload. Use a smaller capture exported by its producer or another decoderwith sufficient capacity, retaining the original for provenance. Subsets dodiff --git a/src/application/Doctor.test.ts b/src/application/Doctor.test.tsindex 03957e636..3187a4b88 100644--- a/src/application/Doctor.test.ts+++ b/src/application/Doctor.test.ts@@ -165,6 +165,7 @@ describe("doctor native analysis configuration", () => {it.each([["REA_ANALYSIS_PROVIDER", "Hopper"],["REA_REGEX_MATCHING_DEADLINE_MS", "0"],+ ["REA_CAPTURE_DECODER_OLD_GENERATION_MIB", "7"],["REA_REGEX_WORKER_HEAP_REQUEST_MIB", "6"],["REA_GHIDRA_STARTUP_TIMEOUT_MS", "abc"],["REA_SWIFTC_COMPILE_TIMEOUT_MS", "0"],diff --git a/src/application/SetupRegistrationEnvironment.ts b/src/application/SetupRegistrationEnvironment.tsindex 027acb77e..864c788d5 100644--- a/src/application/SetupRegistrationEnvironment.ts+++ b/src/application/SetupRegistrationEnvironment.ts@@ -3,6 +3,7 @@ import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";const SELECTED_REGISTRATION_SETTINGS = ["REA_ANALYSIS_PROVIDER",+ "REA_CAPTURE_DECODER_OLD_GENERATION_MIB","REA_GHIDRA_STARTUP_TIMEOUT_MS","REA_HOPPER_STARTUP_TIMEOUT_MS","REA_JEB_MCP_TIMEOUT_MS",diff --git a/src/browser/history/CaptureFailures.test.ts b/src/browser/history/CaptureFailures.test.tsindex d6268645a..7508da5b0 100644--- a/src/browser/history/CaptureFailures.test.ts+++ b/src/browser/history/CaptureFailures.test.ts@@ -36,10 +36,12 @@ it.each([expect(projected).toMatchObject({code: "resource_constraint",retryable: false,- message: expect.stringContaining("fixed V8 heap"),+ message: expect.stringContaining(+ "exhausted its 192 MiB V8 old-generation heap (source default)",+ ),remediation: {action: expect.stringContaining(- "smaller capture exported by its producer",+ "The effective REA_CAPTURE_DECODER_OLD_GENERATION_MIB value is 192 MiB (source default; default 192 MiB, floor 8 MiB, maximum 16384 MiB).",),},details: {@@ -52,8 +54,13 @@ it.each([},});expect(JSON.stringify(projected)).not.toContain("rea doctor");+ expect(JSON.stringify(projected)).not.toContain("no runtime override");// Bun ignores the decoder's V8 heap flags, so no fixed heap is reported.- const heapLimits = { old_generation_heap_mib: 192, semi_space_heap_mib: 8 };+ const heapLimits = {+ old_generation_heap_mib: 192,+ old_generation_heap_source: "default",+ semi_space_heap_mib: 8,+ };if (process.versions.bun === undefined)expect(projected).toMatchObject({details: { reported_limits: heapLimits },diff --git a/src/browser/history/CaptureFailures.ts b/src/browser/history/CaptureFailures.tsindex 4fc854501..34600a7a6 100644--- a/src/browser/history/CaptureFailures.ts+++ b/src/browser/history/CaptureFailures.ts@@ -21,11 +21,16 @@ import {webNetworkCaptureTimeoutRecovery,type InspectWebNetworkCaptureInput,} from "../../domain/webNetworkCapture.js";+import {+ captureDecoderHeapRecovery,+ DEFAULT_CAPTURE_DECODER_HEAP,+ type CaptureDecoderHeap,+} from "../../config/captureDecoderHeap.js";import { OwnedCommandFailure } from "../../process/OwnedCommand.js";import {NODE_CAPTURE_HEAP_ENFORCED,NODE_CAPTURE_HEAP_LIMITS,- NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES,+ captureDecoderProcessMemoryLimitBytes,} from "./CaptureRelease.js";const OPERATION = "inspect_web_network_capture";@@ -39,6 +44,7 @@ export const historicalCaptureFailure = (cause: unknown,phase: "capture-read" | "descriptor-read" | "decoder",options?: ExecutionOptions,+ heap: CaptureDecoderHeap = DEFAULT_CAPTURE_DECODER_HEAP,): AnalysisError => {if (cause instanceof AnalysisError) return cause;const descriptorRead = phase === "descriptor-read";@@ -100,6 +106,7 @@ export const historicalCaptureFailure = (input,CAPTURE_PROCESS_MEMORY_LIMIT_REASON,cause,+ heap,);if (phase === "decoder" &&@@ -111,8 +118,9 @@ export const historicalCaptureFailure = ()return captureMemoryFailure(input,- `The ${input.format} decoder exhausted its fixed V8 heap`,+ `The ${input.format} decoder exhausted its ${String(heap.oldGenerationMiB)} MiB V8 old-generation heap (source ${heap.source})`,cause,+ heap,);}if (@@ -224,8 +232,13 @@ export const captureMemoryFailure = (input: InspectWebNetworkCaptureInput,reason: string,cause?: unknown,-): AnalysisResourceConstraintError =>- new AnalysisResourceConstraintError(+ heap: CaptureDecoderHeap = DEFAULT_CAPTURE_DECODER_HEAP,+): AnalysisResourceConstraintError => {+ const processMemoryLimitBytes = captureDecoderProcessMemoryLimitBytes(+ heap.oldGenerationMiB,+ );+ const recovery = captureDecoderHeapRecovery(heap);+ return new AnalysisResourceConstraintError(OPERATION,"memory",`${reason}: ${input.capture_path}. No complete capture result is available.`,@@ -237,14 +250,14 @@ export const captureMemoryFailure = (maximum_reply_bytes: WEB_NETWORK_CAPTURE_LIMITS.outputBytes,...(input.format !== "mitmproxy" && NODE_CAPTURE_HEAP_ENFORCED? {- old_generation_heap_mib: NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB,+ old_generation_heap_mib: heap.oldGenerationMiB,+ old_generation_heap_source: heap.source,semi_space_heap_mib: NODE_CAPTURE_HEAP_LIMITS.semiSpaceMiB,}: {}),- ...(input.format !== "mitmproxy" &&- NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null+ ...(input.format !== "mitmproxy" && processMemoryLimitBytes !== null? {- process_memory_limit_bytes: NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES,+ process_memory_limit_bytes: processMemoryLimitBytes,}: {}),...(cause instanceof OwnedCommandFailure@@ -256,7 +269,7 @@ export const captureMemoryFailure = (},{cause,- remediationAction: `Use a smaller capture exported by its producer, or another decoder with sufficient capacity for the original capture. Preserve the original for provenance; a subset does not establish complete-capture coverage.${input.format !== "mitmproxy" && NODE_CAPTURE_HEAP_ENFORCED ? ` REA's Node capture old-generation heap is fixed at ${NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB} MiB; inherited NODE_OPTIONS cannot raise it.` : ""}${input.format !== "mitmproxy" && NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null ? ` REA's Node capture decoder process commit limit is fixed at ${String(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)} bytes.` : ""}`,+ remediationAction: `Use a smaller capture exported by its producer, or another decoder with sufficient capacity for the original capture. Preserve the original for provenance; a subset does not establish complete-capture coverage.${input.format !== "mitmproxy" ? ` ${recovery}` : ""}${input.format !== "mitmproxy" && processMemoryLimitBytes !== null ? ` The Windows Job Object commit limit for this decoder is ${String(processMemoryLimitBytes)} bytes, ${String(heap.oldGenerationMiB)} MiB plus a 512 MiB Bun reserve.` : ""}`,...(cause instanceof OwnedCommandFailure && cause.snapshot !== null? {capturedOutput: {@@ -268,6 +281,7 @@ export const captureMemoryFailure = (: {}),},);+};/** Preserve the exact file owner when capture or descriptor acquisition needs cleanup. */export const withCaptureReaderCleanup = (diff --git a/src/browser/history/CaptureRelease.ts b/src/browser/history/CaptureRelease.tsindex 77f29df51..62dfecee4 100644--- a/src/browser/history/CaptureRelease.ts+++ b/src/browser/history/CaptureRelease.ts@@ -1,3 +1,7 @@+import {+ CAPTURE_DECODER_SEMI_SPACE_MIB,+ DEFAULT_CAPTURE_DECODER_OLD_GENERATION_MIB,+} from "../../config/captureDecoderHeap.js";import { javaScriptRuntime } from "../../domain/runtimeVersion.js";import { javaScriptWorkerMemoryLimitBytes } from "../../process/JavaScriptWorkerMemoryLimit.js";@@ -8,19 +12,30 @@ export const HAR_CAPTURE_PROVIDER_IDENTITY = {} as const;-/** Fixed V8 heap settings used by the owned Node capture decoder process. */+/**+ * Default V8 heap settings for the owned Node capture decoder process.+ * The old generation is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB`; the+ * semi-space stays fixed.+ */export const NODE_CAPTURE_HEAP_LIMITS = {- oldGenerationMiB: 192,- semiSpaceMiB: 8,+ oldGenerationMiB: DEFAULT_CAPTURE_DECODER_OLD_GENERATION_MIB,+ semiSpaceMiB: CAPTURE_DECODER_SEMI_SPACE_MIB,} as const;+/** Windows Job Object commit limit for one resolved old-generation budget. */+export const captureDecoderProcessMemoryLimitBytes = (+ oldGenerationMiB: number,+): number | null => javaScriptWorkerMemoryLimitBytes(oldGenerationMiB);+/** The Node capture decoder runs on this executable; Bun accepts but ignores V8 heap flags. */export const NODE_CAPTURE_HEAP_ENFORCED =javaScriptRuntime(process.versions).id === "node";-/** Windows Job Object commit limit replacing the ignored heap flags under Bun; otherwise null. */+/** Commit limit for the default heap. A configured heap uses `captureDecoderProcessMemoryLimitBytes`. */export const NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES =- javaScriptWorkerMemoryLimitBytes(NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB);+ captureDecoderProcessMemoryLimitBytes(+ NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB,+ );/** Native executable profile verified with unchanged mitmproxy raw-state decoding. */export const MITMPROXY_CAPTURE_PROVIDER_IDENTITY = {diff --git a/src/browser/history/HistoricalCaptureDecoder.ts b/src/browser/history/HistoricalCaptureDecoder.tsindex 6ec81366d..bb0c61cb6 100644--- a/src/browser/history/HistoricalCaptureDecoder.ts+++ b/src/browser/history/HistoricalCaptureDecoder.ts@@ -9,10 +9,14 @@ import {captureMemoryFailure,withCaptureReaderCleanup,} from "./CaptureFailures.js";+import {+ captureDecoderHeapRecovery,+ resolveCaptureDecoderHeap,+ type CaptureDecoderHeap,+} from "../../config/captureDecoderHeap.js";import {NODE_CAPTURE_HEAP_ENFORCED,- NODE_CAPTURE_HEAP_LIMITS,- NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES,+ captureDecoderProcessMemoryLimitBytes,} from "./CaptureRelease.js";import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";import type { HistoricalCaptureFormatAdapter } from "./HistoricalCaptureFormatAdapter.js";@@ -76,6 +80,7 @@ const replySchema = z.discriminatedUnion("ok", [/** One owned snapshot/process/root lifecycle for replaceable historical format decoders. */export class HistoricalCaptureDecoder {readonly #resources: ArtifactResourceScope;+ readonly #heap: CaptureDecoderHeap;constructor(readonly adapters: readonly HistoricalCaptureFormatAdapter[],@@ -86,6 +91,18 @@ export class HistoricalCaptureDecoder {resources = new ArtifactResourceScope(),) {this.#resources = resources;+ // Adapters that launch the Node decoder already carry its parsed heap.+ // Reuse that fact. This environment is only the fallback when none do,+ // or when those facts disagree.+ const shared = sharedCaptureDecoderHeap(adapters);+ this.#heap = shared ?? resolveCaptureDecoderHeap(environment);+ }++ /** Heap the selected adapter launches, or the shared fallback when it has none. */+ #reportedHeap(+ adapter: HistoricalCaptureFormatAdapter | undefined,+ ): CaptureDecoderHeap {+ return adapter?.heap ?? this.#heap;}/** Retry retained readers and private roots at service shutdown. */@@ -108,6 +125,7 @@ export class HistoricalCaptureDecoder {cause,"capture-read",options,+ this.#heap,);return err(redactExplicitFailure(@@ -126,10 +144,12 @@ export class HistoricalCaptureDecoder {let result: Result<WebNetworkCapture, AnalysisError>;let phase: "capture-read" | "decoder" = "capture-read";let encodedReplyBytes = 0;+ let reportedHeap = this.#heap;try {const adapter = this.adapters.find((candidate) => candidate.format === input.format,);+ reportedHeap = this.#reportedHeap(adapter);if (adapter === undefined)throw new AnalysisCapabilityUnavailableError(input.format,@@ -186,7 +206,12 @@ export class HistoricalCaptureDecoder {// A decoder that survived a failed allocation may still reply; its// reply is not trusted as complete once the limit was reached.if (completed.memoryLimitReached === true)- throw captureMemoryFailure(input, CAPTURE_PROCESS_MEMORY_LIMIT_REASON);+ throw captureMemoryFailure(+ input,+ CAPTURE_PROCESS_MEMORY_LIMIT_REASON,+ undefined,+ reportedHeap,+ );let reply: z.output<typeof replySchema>;try {const replyFile = await readStableArtifact(@@ -248,7 +273,12 @@ export class HistoricalCaptureDecoder {},]);if (reply.reason === "resource-limit")- throw captureMemoryFailure(input, reply.message);+ throw captureMemoryFailure(+ input,+ reply.message,+ undefined,+ reportedHeap,+ );if (reply.reason === "decoder")throw new ProviderAdapterError(input.format, OPERATION, {diagnostics: {@@ -288,7 +318,7 @@ export class HistoricalCaptureDecoder {"This is retained producer evidence, without live browser transaction IDs, scenario provenance, execution attribution or deployment authenticity. Recorded URLs are never fetched.","Every record is decoded before selection; no partial success is returned on malformed input or a resource limit.","Known authentication headers/cookies and explicitly marked literal values are excluded. Unknown extension names do not establish sensitivity.",- `Complete evidence is bounded by a ${String(WEB_NETWORK_CAPTURE_LIMITS.inputBytes)}-byte input budget, a ${String(WEB_NETWORK_CAPTURE_LIMITS.outputBytes)}-byte reply budget, and a ${String(WEB_NETWORK_CAPTURE_LIMITS.depth)}-level nesting cap. Those have no runtime override. capture_timeout_ms is ${String(selectedWebNetworkCaptureTimeoutMs(input.capture_timeout_ms))} ms (omitted means ${String(WEB_NETWORK_CAPTURE_LIMITS.timeoutMs)}). ${nodeMemoryLimitation()}; mitmproxy decoding uses a 768 MiB Linux address-space limit.`,+ `Complete evidence is bounded by a ${String(WEB_NETWORK_CAPTURE_LIMITS.inputBytes)}-byte input budget, a ${String(WEB_NETWORK_CAPTURE_LIMITS.outputBytes)}-byte reply budget, and a ${String(WEB_NETWORK_CAPTURE_LIMITS.depth)}-level nesting cap. Those have no runtime override. capture_timeout_ms is ${String(selectedWebNetworkCaptureTimeoutMs(input.capture_timeout_ms))} ms (omitted means ${String(WEB_NETWORK_CAPTURE_LIMITS.timeoutMs)}). ${nodeMemoryLimitation(reportedHeap)} Mitmproxy decoding uses a 768 MiB Linux address-space limit.`,],};if (input.format === "qlog") {@@ -348,13 +378,25 @@ export class HistoricalCaptureDecoder {message: cause.message,},])- : historicalCaptureFailure(input, cause, phase, options);+ : historicalCaptureFailure(+ input,+ cause,+ phase,+ options,+ reportedHeap,+ );if (safePartialSource) error.retainPartialObservation(result.value);throw error;}}} catch (cause: unknown) {- const error = historicalCaptureFailure(input, cause, phase, options);+ const error = historicalCaptureFailure(+ input,+ cause,+ phase,+ options,+ reportedHeap,+ );result = err(redactExplicitFailure(withCaptureReaderCleanup(error, cause),@@ -418,13 +460,36 @@ export class HistoricalCaptureDecoder {}}+/** One shared parsed heap, when every Node adapter agrees. Disagreement stays unresolved here. */+const sharedCaptureDecoderHeap = (+ adapters: readonly HistoricalCaptureFormatAdapter[],+): CaptureDecoderHeap | undefined => {+ const heaps = adapters.flatMap((adapter) =>+ adapter.heap === undefined ? [] : [adapter.heap],+ );+ const first = heaps[0];+ if (first === undefined) return undefined;+ return heaps.every(+ (heap) =>+ heap === first ||+ (heap.oldGenerationMiB === first.oldGenerationMiB &&+ heap.source === first.source),+ )+ ? first+ : undefined;+};+/** The Node capture decoder runs on this executable; only Node.js enforces its V8 heap flags. */-const nodeMemoryLimitation = (): string => {+const nodeMemoryLimitation = (heap: CaptureDecoderHeap): string => {+ const recovery = captureDecoderHeapRecovery(heap);if (NODE_CAPTURE_HEAP_ENFORCED)- return `HAR/SAZ/qlog use a ${String(NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB)} MiB Node old-generation heap`;- return NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null- ? "HAR/SAZ/qlog run on Bun, which does not enforce REA's V8 heap flags"- : `HAR/SAZ/qlog run on Bun, which does not enforce REA's V8 heap flags, under a ${String(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)}-byte Windows Job Object commit limit`;+ return `HAR/SAZ/qlog use a ${String(heap.oldGenerationMiB)} MiB Node old-generation heap (source ${heap.source}). ${recovery}`;+ const commitLimit = captureDecoderProcessMemoryLimitBytes(+ heap.oldGenerationMiB,+ );+ return commitLimit === null+ ? `HAR/SAZ/qlog run on Bun, which does not enforce REA's V8 heap flags. ${recovery}`+ : `HAR/SAZ/qlog run on Bun, which does not enforce REA's V8 heap flags, under a ${String(commitLimit)}-byte Windows Job Object commit limit. ${recovery}`;};const replyFailureReason = (cause: unknown): string => {diff --git a/src/browser/history/HistoricalCaptureFormatAdapter.ts b/src/browser/history/HistoricalCaptureFormatAdapter.tsindex 9660fd579..88e811b73 100644--- a/src/browser/history/HistoricalCaptureFormatAdapter.ts+++ b/src/browser/history/HistoricalCaptureFormatAdapter.ts@@ -1,9 +1,15 @@+import type { CaptureDecoderHeap } from "../../config/captureDecoderHeap.js";import type { InspectWebNetworkCaptureInput } from "../../domain/webNetworkCapture.js";/** Format-specific process profile keeps mature upstream code behind a replaceable adapter seam. */export interface HistoricalCaptureFormatAdapter {readonly format: InspectWebNetworkCaptureInput["format"];readonly limitations?: readonly string[];+ /**+ * Parsed heap this adapter launches. Reports reuse this fact. Adapters that+ * do not launch the Node decoder omit it.+ */+ readonly heap?: CaptureDecoderHeap;readonly identity: {readonly id: string;readonly name: string;diff --git a/src/browser/history/NodeCaptureAdapter.test.ts b/src/browser/history/NodeCaptureAdapter.test.tsindex bf2528f5b..3e47e0973 100644--- a/src/browser/history/NodeCaptureAdapter.test.ts+++ b/src/browser/history/NodeCaptureAdapter.test.ts@@ -5,7 +5,7 @@ import { NodeCaptureAdapter } from "./NodeCaptureAdapter.js";vi.mock("./CaptureRelease.js", async (original) => ({...(await original<typeof import("./CaptureRelease.js")>()),NODE_CAPTURE_HEAP_ENFORCED: false,- NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES: null,+ captureDecoderProcessMemoryLimitBytes: () => null,}));it.each(["saz", "qlog"] as const)(diff --git a/src/browser/history/NodeCaptureAdapter.ts b/src/browser/history/NodeCaptureAdapter.tsindex a366459a4..249572c26 100644--- a/src/browser/history/NodeCaptureAdapter.ts+++ b/src/browser/history/NodeCaptureAdapter.ts@@ -1,12 +1,17 @@import { fileURLToPath } from "node:url";import type { HistoricalCaptureFormatAdapter } from "./HistoricalCaptureFormatAdapter.js";+import {+ CAPTURE_DECODER_SEMI_SPACE_MIB,+ DEFAULT_CAPTURE_DECODER_HEAP,+ resolveCaptureDecoderHeap,+ type CaptureDecoderHeap,+} from "../../config/captureDecoderHeap.js";import {NODE_CAPTURE_HEAP_ENFORCED,HAR_CAPTURE_PROVIDER_IDENTITY,SAZ_CAPTURE_PROVIDER_IDENTITY,QLOG_CAPTURE_PROVIDER_IDENTITY,- NODE_CAPTURE_HEAP_LIMITS,- NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES,+ captureDecoderProcessMemoryLimitBytes,} from "./CaptureRelease.js";import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";@@ -14,7 +19,22 @@ import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCoexport class NodeCaptureAdapter implements HistoricalCaptureFormatAdapter {readonly identity;readonly limitations;- constructor(readonly format: "har" | "saz" | "qlog") {+ readonly heap: CaptureDecoderHeap;+ /**+ * `environment` is the selected startup environment. Omitted means the+ * default heap, not ambient `process.env`. An invalid setting is refused.+ * `heap` is an already parsed fact and is not read from `environment` again.+ */+ constructor(+ readonly format: "har" | "saz" | "qlog",+ environment?: Readonly<Record<string, string | undefined>>,+ heap?: CaptureDecoderHeap,+ ) {+ this.heap =+ heap ??+ (environment === undefined+ ? DEFAULT_CAPTURE_DECODER_HEAP+ : resolveCaptureDecoderHeap(environment));this.identity =format === "har"? HAR_CAPTURE_PROVIDER_IDENTITY@@ -32,10 +52,13 @@ export class NodeCaptureAdapter implements HistoricalCaptureFormatAdapter {: [];}async command(requestPath: string) {+ const memoryLimitBytes = captureDecoderProcessMemoryLimitBytes(+ this.heap.oldGenerationMiB,+ );if (this.format !== "har" &&!NODE_CAPTURE_HEAP_ENFORCED &&- NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null+ memoryLimitBytes === null)throw new AnalysisCapabilityUnavailableError(this.identity.id,@@ -45,17 +68,15 @@ export class NodeCaptureAdapter implements HistoricalCaptureFormatAdapter {return {command: process.execPath,arguments: [- `--max-old-space-size=${NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB}`,- `--max-semi-space-size=${NODE_CAPTURE_HEAP_LIMITS.semiSpaceMiB}`,+ `--max-old-space-size=${String(this.heap.oldGenerationMiB)}`,+ `--max-semi-space-size=${String(CAPTURE_DECODER_SEMI_SPACE_MIB)}`,"--v8-pool-size=1","--single-threaded",fileURLToPath(new URL("./NodeCaptureProcess.js", import.meta.url)),requestPath,this.format,],- ...(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null- ? {}- : { memoryLimitBytes: NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES }),+ ...(memoryLimitBytes === null ? {} : { memoryLimitBytes }),};}}diff --git a/src/cli/webNetworkCaptureCommands.ts b/src/cli/webNetworkCaptureCommands.tsindex 5a3d82b96..659d9c023 100644--- a/src/cli/webNetworkCaptureCommands.ts+++ b/src/cli/webNetworkCaptureCommands.ts@@ -23,7 +23,6 @@ export const registerWebNetworkCaptureCommands = (logger: Logger,environment: Readonly<NodeJS.ProcessEnv>,): void => {- const service = createWebNetworkCaptureService(environment);cli.command(CLI_COMMANDS.inspectWebNetworkCapture, {description:"Inspect retained HAR, native mitmproxy, Fiddler SAZ or aioquic qlog records without fetching recorded URLs",@@ -65,6 +64,7 @@ export const registerWebNetworkCaptureCommands = (logger,CLI_COMMANDS.inspectWebNetworkCapture,async () => {+ const service = createWebNetworkCaptureService(environment);return deliverCliAnalysisEvidence({run: () =>service.inspect(diff --git a/src/composition/webNetworkCaptures.ts b/src/composition/webNetworkCaptures.tsindex 9d9e8c6cc..1653f644a 100644--- a/src/composition/webNetworkCaptures.ts+++ b/src/composition/webNetworkCaptures.ts@@ -2,19 +2,22 @@ import { WebNetworkCaptureService } from "../application/WebNetworkCaptureServicimport { HistoricalCaptureDecoder } from "../browser/history/HistoricalCaptureDecoder.js";import { NodeCaptureAdapter } from "../browser/history/NodeCaptureAdapter.js";import { MitmproxyCaptureAdapter } from "../browser/history/MitmproxyCaptureAdapter.js";+import { resolveCaptureDecoderHeap } from "../config/captureDecoderHeap.js";/** Compose offline format adapters without import-time reads, installation or process startup. */export const createWebNetworkCaptureService = (environment: Readonly<NodeJS.ProcessEnv> = process.env,-): WebNetworkCaptureService =>- new WebNetworkCaptureService(+): WebNetworkCaptureService => {+ const heap = resolveCaptureDecoderHeap(environment);+ return new WebNetworkCaptureService(new HistoricalCaptureDecoder([- new NodeCaptureAdapter("har"),- new NodeCaptureAdapter("saz"),- new NodeCaptureAdapter("qlog"),+ new NodeCaptureAdapter("har", environment, heap),+ new NodeCaptureAdapter("saz", environment, heap),+ new NodeCaptureAdapter("qlog", environment, heap),new MitmproxyCaptureAdapter(environment),],environment,),);+};diff --git a/src/config/captureDecoderHeap.test.ts b/src/config/captureDecoderHeap.test.tsnew file mode 100644index 000000000..e6bb48f5e--- /dev/null+++ b/src/config/captureDecoderHeap.test.ts@@ -0,0 +1,260 @@+import { mkdtemp, rm, writeFile } from "node:fs/promises";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { expect, it, vi } from "vitest";++import { NodeCaptureAdapter } from "../browser/history/NodeCaptureAdapter.js";+import { HistoricalCaptureDecoder } from "../browser/history/HistoricalCaptureDecoder.js";+import { decodeHarCapture } from "../browser/history/HarCapture.js";+import { historicalCaptureFailure } from "../browser/history/CaptureFailures.js";+import { ConfigurationError } from "../domain/configurationErrors.js";+import { projectAnalysisError } from "../domain/analysisErrorProjection.js";+import { inspectWebNetworkCaptureInputSchema } from "../domain/webNetworkCapture.js";+import { OwnedCommandFailure } from "../process/OwnedCommand.js";+import { createWebNetworkCaptureService } from "../composition/webNetworkCaptures.js";+import { createCli } from "../cli.js";+import { historicalHar } from "../../tests/fixtures/historicalHar.js";+import { parseEnvironment } from "./environment.js";+import { parseConfig } from "./parseConfig.js";+import {+ CAPTURE_DECODER_OLD_GENERATION_SETTING,+ captureDecoderHeapConstraint,+ captureDecoderHeapRecovery,+ DEFAULT_CAPTURE_DECODER_HEAP,+ MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB,+ parseCaptureDecoderHeap,+} from "./captureDecoderHeap.js";++const configured = (oldGenerationMiB: number) =>+ ({ oldGenerationMiB, source: "setting" }) as const;++const heapExhaustion = () => {+ const stderr =+ "FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory\n";+ return new OwnedCommandFailure("process", "Decoder failed", {+ stdout: { text: "", bytes: 0, observedBytes: 0 },+ stderr: {+ text: stderr,+ bytes: Buffer.byteLength(stderr),+ observedBytes: Buffer.byteLength(stderr),+ },+ exitCode: null,+ signal: "SIGABRT",+ });+};++it("keeps the 192 MiB decoder heap when the setting is unset", async () => {+ expect(parseCaptureDecoderHeap(undefined)).toEqual({+ ok: true,+ value: DEFAULT_CAPTURE_DECODER_HEAP,+ });+ const parsed = parseConfig({});+ if (!parsed.ok) throw new Error("unset heap was rejected");+ const recovery = captureDecoderHeapRecovery(DEFAULT_CAPTURE_DECODER_HEAP);+ expect(recovery).toContain(+ "The effective REA_CAPTURE_DECODER_OLD_GENERATION_MIB value is 192 MiB (source default; default 192 MiB, floor 8 MiB, maximum 16384 MiB).",+ );+ expect(recovery).toContain(+ "Set REA_CAPTURE_DECODER_OLD_GENERATION_MIB in the environment that starts REA to a decimal integer from 8 to 16384, then restart REA.",+ );+ expect(recovery).not.toContain("no runtime override");+ const command = await new NodeCaptureAdapter("har").command(+ "/owned/request.json",+ );+ expect(command.arguments).toContain("--max-old-space-size=192");+ expect(command.arguments).toContain("--max-semi-space-size=8");+});++it("uses a valid REA_CAPTURE_DECODER_OLD_GENERATION_MIB override", async () => {+ expect(parseCaptureDecoderHeap("256")).toEqual({+ ok: true,+ value: configured(256),+ });+ expect(+ parseCaptureDecoderHeap(String(MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)),+ ).toEqual({+ ok: true,+ value: configured(MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB),+ });+ const environment = { REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "256" };+ const parsed = parseConfig(environment);+ if (!parsed.ok) throw new Error("valid heap was rejected");+ const adapter = new NodeCaptureAdapter("har", environment);+ expect(adapter.heap).toEqual(configured(256));+ const command = await adapter.command("/owned/request.json");+ expect(command.arguments).toContain("--max-old-space-size=256");+ expect(command.arguments).toContain("--max-semi-space-size=8");+ const input = inspectWebNetworkCaptureInputSchema.parse({+ capture_path: "/selected/input.har",+ format: "har",+ });+ const projected = projectAnalysisError(+ historicalCaptureFailure(+ input,+ heapExhaustion(),+ "decoder",+ undefined,+ configured(256),+ ),+ );+ const recovery = captureDecoderHeapRecovery(configured(256));+ expect(projected.message).toContain(+ "exhausted its 256 MiB V8 old-generation heap (source setting)",+ );+ expect(projected.remediation.action).toContain(recovery);+ expect(projected.message).not.toContain("no runtime override");+ if (process.versions.bun === undefined)+ expect(projected.details).toMatchObject({+ reported_limits: {+ old_generation_heap_mib: 256,+ old_generation_heap_source: "setting",+ },+ });+});++it("keeps a parsed heap instead of reading the adapter environment again", async () => {+ const heap = configured(256);+ const adapter = new NodeCaptureAdapter(+ "har",+ { REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "512" },+ heap,+ );+ expect(adapter.heap).toBe(heap);+ const command = await adapter.command("/owned/request.json");+ expect(command.arguments).toContain("--max-old-space-size=256");+ expect(command.arguments).not.toContain("--max-old-space-size=512");+});++it("reports the heap the selected adapter launches when decoder environment differs", async () => {+ const launched = new NodeCaptureAdapter("har", {+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "256",+ });+ const other = new NodeCaptureAdapter("saz", {+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "512",+ });+ const root = await mkdtemp(join(tmpdir(), "rea-capture-heap-report-"));+ const capturePath = join(root, "input.har");+ const text = JSON.stringify(historicalHar());+ await writeFile(capturePath, text);+ const input = inspectWebNetworkCaptureInputSchema.parse({+ capture_path: capturePath,+ format: "har",+ });+ const decoder = new HistoricalCaptureDecoder([launched, other], {});+ try {+ vi.spyOn(launched, "command").mockImplementation(async (requestPath) => {+ const runtimePath = join(requestPath, "..");+ await writeFile(+ join(runtimePath, "reply.json"),+ JSON.stringify({ ok: true, value: decodeHarCapture(text, []) }),+ );+ return {+ command: process.execPath,+ arguments: ["-e", "process.exit(0)"],+ };+ });+ const reported = await decoder.inspect(input);+ if (!reported.ok)+ throw new Error(projectAnalysisError(reported.error).message);+ const limitations = reported.value.limitations.join("\n");+ expect(limitations).toContain(+ "The effective REA_CAPTURE_DECODER_OLD_GENERATION_MIB value is 256 MiB (source setting",+ );+ expect(limitations).not.toContain("192 MiB (source default)");+ expect(limitations).not.toContain("512 MiB (source setting)");++ vi.spyOn(launched, "command").mockRejectedValue(heapExhaustion());+ const exhausted = await decoder.inspect(input);+ if (exhausted.ok) throw new Error("Heap exhaustion must fail inspection");+ const projected = projectAnalysisError(exhausted.error);+ expect(projected.message).toContain(+ "exhausted its 256 MiB V8 old-generation heap (source setting)",+ );+ expect(projected.remediation.action).toContain(+ "The effective REA_CAPTURE_DECODER_OLD_GENERATION_MIB value is 256 MiB (source setting",+ );+ expect(projected.message).not.toContain("192 MiB");+ expect(projected.remediation.action).not.toContain(+ "512 MiB (source setting)",+ );+ } finally {+ await decoder.close();+ await rm(root, { recursive: true, force: true });+ }+});++it("registers CLI commands when the capture decoder heap setting is invalid", async () => {+ const cli = createCli({+ REA_LOG_LEVEL: "silent",+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "7",+ });+ let text = "";+ await cli.serve(["inspect-web-network-capture", "--help"], {+ env: {},+ exit: () => undefined,+ stdout: (chunk: string) => {+ text += chunk;+ },+ });+ expect(text).toContain("inspect-web-network-capture");+});++it.each([+ "",+ "0",+ "7",+ "-1",+ "0192",+ "192.5",+ " 192",+ "192 ",+ "1e3",+ String(MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB + 1),+ "9".repeat(40),+])("rejects invalid %j as a configuration error", (value) => {+ const parsed = parseCaptureDecoderHeap(value);+ if (parsed.ok) throw new Error(`Accepted ${JSON.stringify(value)}`);+ expect(parsed.error).toBeInstanceOf(ConfigurationError);+ expect(parsed.error.settings).toEqual([+ {+ setting: CAPTURE_DECODER_OLD_GENERATION_SETTING,+ constraint: captureDecoderHeapConstraint,+ },+ ]);+ expect(parsed.error.message).toBe(captureDecoderHeapConstraint);+ const startup = parseEnvironment({+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: value,+ });+ if (startup.ok) throw new Error(`Startup accepted ${JSON.stringify(value)}`);+ expect(startup.error.settings).toContainEqual({+ setting: "REA_CAPTURE_DECODER_OLD_GENERATION_MIB",+ constraint: expect.stringContaining("from 8 to 16384 MiB"),+ });+ const config = parseConfig({+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: value,+ });+ if (config.ok)+ throw new Error(`parseConfig accepted ${JSON.stringify(value)}`);+ expect(projectAnalysisError(config.error)).toMatchObject({+ code: "configuration_invalid",+ message: expect.stringContaining("REA_CAPTURE_DECODER_OLD_GENERATION_MIB"),+ });+ expect(+ () =>+ new NodeCaptureAdapter("har", {+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: value,+ }),+ ).toThrow(ConfigurationError);+ expect(+ () =>+ new HistoricalCaptureDecoder([], {+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: value,+ }),+ ).toThrow(ConfigurationError);+ expect(() =>+ createWebNetworkCaptureService({+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: value,+ }),+ ).toThrow(ConfigurationError);+});diff --git a/src/config/captureDecoderHeap.ts b/src/config/captureDecoderHeap.tsnew file mode 100644index 000000000..aed3b0619--- /dev/null+++ b/src/config/captureDecoderHeap.ts@@ -0,0 +1,105 @@+import { z } from "zod";++import { ConfigurationError } from "../domain/configurationErrors.js";+import { err, ok, type Result } from "../domain/result.js";++export const CAPTURE_DECODER_OLD_GENERATION_SETTING =+ "REA_CAPTURE_DECODER_OLD_GENERATION_MIB";++/** Old-generation mebibytes used when the setting is unset. Today's constant. */+export const DEFAULT_CAPTURE_DECODER_OLD_GENERATION_MIB = 192;++/**+ * Semi-space the owned decoder always passes. It is not a setting. An old+ * generation smaller than this semi-space aborts Node during isolate startup+ * instead of enforcing a heap limit, so the setting's floor is this value.+ */+export const CAPTURE_DECODER_SEMI_SPACE_MIB = 8;++/** Smallest old generation that still starts the decoder with that semi-space. */+export const MINIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB =+ CAPTURE_DECODER_SEMI_SPACE_MIB;++/**+ * Largest old generation one owned decoder may request. 16 GiB is the same+ * ceiling REA already uses for an isolated JavaScript analysis worker. Node+ * will start with a larger flag; this maximum is REA's refusal, so one+ * decoder cannot ask for an unbounded old generation.+ */+export const MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB = 16_384;++/** Effective HAR/SAZ/qlog old-generation heap and where its value came from. */+export interface CaptureDecoderHeap {+ readonly oldGenerationMiB: number;+ readonly source: "default" | "setting";+}++export const DEFAULT_CAPTURE_DECODER_HEAP: CaptureDecoderHeap = {+ oldGenerationMiB: DEFAULT_CAPTURE_DECODER_OLD_GENERATION_MIB,+ source: "default",+};++export const captureDecoderHeapConstraint = `${CAPTURE_DECODER_OLD_GENERATION_SETTING} must be a decimal integer from ${String(MINIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)} to ${String(MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)} MiB; the default is ${String(DEFAULT_CAPTURE_DECODER_OLD_GENERATION_MIB)} MiB. An old generation below the decoder's ${String(CAPTURE_DECODER_SEMI_SPACE_MIB)} MiB semi-space aborts Node during startup instead of enforcing a limit, and a larger value is refused.`;++const decimalInteger = /^[1-9]\d*$/u;++const withinHeapRange = (value: string): boolean => {+ // The pattern check reports a non-decimal value. Refine only bounds integers.+ if (!decimalInteger.test(value)) return true;+ const parsed = BigInt(value);+ return (+ parsed >= BigInt(MINIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB) &&+ parsed <= BigInt(MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)+ );+};++/** Optional decimal mebibyte count for the HAR/SAZ/qlog old-generation heap. */+export const captureDecoderOldGenerationMiBSchema = z+ .string()+ .regex(decimalInteger, { error: captureDecoderHeapConstraint })+ .refine(withinHeapRange, { error: captureDecoderHeapConstraint })+ .transform(Number);++/** Resolve the effective heap from one environment value. */+export const parseCaptureDecoderHeap = (+ value: string | undefined,+): Result<CaptureDecoderHeap, ConfigurationError> => {+ if (value === undefined) return ok(DEFAULT_CAPTURE_DECODER_HEAP);+ const parsed = captureDecoderOldGenerationMiBSchema.safeParse(value);+ return parsed.success+ ? ok({ oldGenerationMiB: parsed.data, source: "setting" })+ : err(+ new ConfigurationError(captureDecoderHeapConstraint, {+ settings: [+ {+ setting: CAPTURE_DECODER_OLD_GENERATION_SETTING,+ constraint: captureDecoderHeapConstraint,+ },+ ],+ }),+ );+};++/**+ * Resolve the heap from a composition root's selected environment. An invalid+ * setting is refused, not ignored. Callers that already resolved it at startup+ * pass that value through instead of reading the environment again.+ */+export const resolveCaptureDecoderHeap = (+ environment: Readonly<Record<string, string | undefined>>,+): CaptureDecoderHeap => {+ const parsed = parseCaptureDecoderHeap(+ environment[CAPTURE_DECODER_OLD_GENERATION_SETTING],+ );+ if (!parsed.ok) throw parsed.error;+ return parsed.value;+};++/** How to read and change the effective HAR/SAZ/qlog old-generation heap. */+export const captureDecoderHeapRecovery = (heap: CaptureDecoderHeap): string =>+ [+ `The effective ${CAPTURE_DECODER_OLD_GENERATION_SETTING} value is ${String(heap.oldGenerationMiB)} MiB (source ${heap.source}; default ${String(DEFAULT_CAPTURE_DECODER_OLD_GENERATION_MIB)} MiB, floor ${String(MINIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)} MiB, maximum ${String(MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)} MiB).`,+ `Set ${CAPTURE_DECODER_OLD_GENERATION_SETTING} in the environment that starts REA to a decimal integer from ${String(MINIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)} to ${String(MAXIMUM_CAPTURE_DECODER_OLD_GENERATION_MIB)}, then restart REA.`,+ `On Node.js the decoder passes --max-old-space-size=${String(heap.oldGenerationMiB)} and removes inherited NODE_OPTIONS, so additional host memory and NODE_OPTIONS do not raise this heap. The ${String(CAPTURE_DECODER_SEMI_SPACE_MIB)} MiB semi-space stays fixed.`,+ "On Bun the V8 flags are ignored. Windows limits the decoder's committed memory to this value plus a 512 MiB reserve. Other Bun hosts enforce no decoder heap limit.",+ ].join(" ");diff --git a/src/config/environment.ts b/src/config/environment.tsindex d90715c8d..09ca39fb5 100644--- a/src/config/environment.ts+++ b/src/config/environment.ts@@ -19,6 +19,7 @@ import { mcpInputSchemaProfileSchema } from "./mcpInputSchemaProfile.js";import { mcpOutputSchemaProfileSchema } from "./mcpOutputSchemaProfile.js";import { mcpResponseBudgetSchema } from "./mcpResponseBudget.js";import { asarMaxHeaderBytesSchema } from "./asarHeaderAdmission.js";+import { captureDecoderOldGenerationMiBSchema } from "./captureDecoderHeap.js";import { regexMatchingDeadlineMsSchema } from "./regexMatchingDeadline.js";import { regexWorkerHeapRequestMiBSchema } from "./regexWorkerHeapRequest.js";import { parseLoaderArgs, parseStringArray } from "./parsers.js";@@ -27,6 +28,8 @@ const environmentSchema = z.object({REA_ANALYSIS_PROVIDER: analysisProviderSelectorSchema.default("auto"),REA_MCP_MAX_RESPONSE_BYTES: mcpResponseBudgetSchema.optional(),REA_ASAR_MAX_HEADER_BYTES: asarMaxHeaderBytesSchema.optional(),+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB:+ captureDecoderOldGenerationMiBSchema.optional(),REA_REGEX_MATCHING_DEADLINE_MS: regexMatchingDeadlineMsSchema.optional(),REA_REGEX_WORKER_HEAP_REQUEST_MIB: regexWorkerHeapRequestMiBSchema.optional(),REA_MCP_INPUT_SCHEMA_PROFILE: mcpInputSchemaProfileSchema.optional(),diff --git a/src/setupEnvironment.test.ts b/src/setupEnvironment.test.tsindex d24b4e574..015e1badb 100644--- a/src/setupEnvironment.test.ts+++ b/src/setupEnvironment.test.ts@@ -185,6 +185,7 @@ it.each([REA_GHIDRA_STARTUP_TIMEOUT_MS: "2700000",REA_HOPPER_STARTUP_TIMEOUT_MS: "120000",REA_MCP_INPUT_SCHEMA_PROFILE: "full",+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "256",REA_REGEX_MATCHING_DEADLINE_MS: "10000",REA_REGEX_WORKER_HEAP_REQUEST_MIB: "256",},@@ -219,6 +220,7 @@ it.each([REA_GHIDRA_STARTUP_TIMEOUT_MS: "2700000",REA_HOPPER_STARTUP_TIMEOUT_MS: "120000",REA_MCP_INPUT_SCHEMA_PROFILE: "full",+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "256",REA_REGEX_MATCHING_DEADLINE_MS: "10000",REA_REGEX_WORKER_HEAP_REQUEST_MIB: "256",});diff --git a/tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts b/tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.tsindex cfe958909..142426834 100644--- a/tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts+++ b/tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts@@ -486,6 +486,7 @@ cliTest(REA_HOPPER_STARTUP_TIMEOUT_MS: "2800002",REA_JEB_MCP_TIMEOUT_MS: "2900003",REA_MCP_INPUT_SCHEMA_PROFILE: "full",+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "256",REA_REGEX_MATCHING_DEADLINE_MS: "10000",REA_REGEX_WORKER_HEAP_REQUEST_MIB: "256",REA_UNRELATED_MARKER: "do-not-persist",@@ -516,6 +517,9 @@ cliTest();expect(codexRegistration).toMatchObject({ tool_timeout_sec: 3021 });expect(codexConfig).toContain('REA_JEB_MCP_TIMEOUT_MS = "2900003"');+ expect(codexConfig).toContain(+ 'REA_CAPTURE_DECODER_OLD_GENERATION_MIB = "256"',+ );expect(codexConfig).toContain('REA_REGEX_MATCHING_DEADLINE_MS = "10000"');expect(codexConfig).toContain('REA_REGEX_WORKER_HEAP_REQUEST_MIB = "256"');expect(codexConfig).not.toContain("REA_UNRELATED_MARKER");@@ -533,6 +537,7 @@ cliTest(REA_HOPPER_STARTUP_TIMEOUT_MS: "2800002",REA_JEB_MCP_TIMEOUT_MS: "2900003",REA_MCP_INPUT_SCHEMA_PROFILE: "full",+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "256",REA_REGEX_MATCHING_DEADLINE_MS: "10000",REA_REGEX_WORKER_HEAP_REQUEST_MIB: "256",},diff --git a/tests/acceptance/tools/historicalCaptureResources.test.ts b/tests/acceptance/tools/historicalCaptureResources.test.tsindex 64a7b6638..0073fc606 100644--- a/tests/acceptance/tools/historicalCaptureResources.test.ts+++ b/tests/acceptance/tools/historicalCaptureResources.test.ts@@ -64,13 +64,14 @@ it.skipIf(process.versions.bun !== undefined && jobLimit === null)(sensitive_values: ["caller-marked"],});expect(response.isError).toBe(true);- expect(parseMcpToolError(response)).toMatchObject({+ const exhausted = parseMcpToolError(response);+ expect(exhausted).toMatchObject({error: {code: "resource_constraint",retryable: false,message: expect.stringContaining(jobLimit === null- ? "fixed V8 heap"+ ? "192 MiB V8 old-generation heap (source default)": "Windows Job Object memory limit",),remediation: {@@ -101,6 +102,14 @@ it.skipIf(process.versions.bun !== undefined && jobLimit === null)(});expect(JSON.stringify(response)).not.toContain("caller-marked");expect(JSON.stringify(response)).not.toContain("rea doctor");+ const remediation = String(exhausted.error.remediation?.action ?? "");+ expect(remediation).toContain(+ "The effective REA_CAPTURE_DECODER_OLD_GENERATION_MIB value is 192 MiB (source default; default 192 MiB, floor 8 MiB, maximum 16384 MiB).",+ );+ expect(remediation).toContain(+ "Set REA_CAPTURE_DECODER_OLD_GENERATION_MIB in the environment that starts REA",+ );+ expect(remediation).not.toContain("no runtime override");const failed: unknown = await execute(process.execPath, [cli,"inspect-web-network-capture",@@ -117,6 +126,9 @@ it.skipIf(process.versions.bun !== undefined && jobLimit === null)(? String(failed.stdout): "";expect(JSON.parse(stdout)).toMatchObject({ code: "resource_constraint" });+ expect(stdout).toContain("REA_CAPTURE_DECODER_OLD_GENERATION_MIB");+ expect(stdout).toContain("source default");+ expect(stdout).not.toContain("no runtime override");const small = join(root, "small.har");await writeFile(small, capture(2));await client.ping();@@ -128,6 +140,16 @@ it.skipIf(process.versions.bun !== undefined && jobLimit === null)(expect(control.structuredContent).toMatchObject({normalized_result: { total_records: 2 },});+ const controlText = JSON.stringify(control.structuredContent);+ expect(controlText).toContain(+ "The effective REA_CAPTURE_DECODER_OLD_GENERATION_MIB value is 192 MiB (source default; default 192 MiB, floor 8 MiB, maximum 16384 MiB).",+ );+ expect(controlText).toContain("source default");+ expect(controlText).toContain("floor 8 MiB");+ expect(controlText).toContain("maximum 16384 MiB");+ expect(controlText).toContain(+ "Set REA_CAPTURE_DECODER_OLD_GENERATION_MIB in the environment that starts REA",+ );await truncate(path, WEB_NETWORK_CAPTURE_LIMITS.inputBytes + 1);const oversized = await call("inspect_web_network_capture", {capture_path: path,diff --git a/tests/boundary/cli/captureDecoderHeapDoctor.test.ts b/tests/boundary/cli/captureDecoderHeapDoctor.test.tsnew file mode 100644index 000000000..94c4ebb42--- /dev/null+++ b/tests/boundary/cli/captureDecoderHeapDoctor.test.ts@@ -0,0 +1,28 @@+import { expect } from "vitest";++import { cliTest } from "../../support/cli/cliFixture.js";++cliTest(+ "reports an invalid capture decoder heap from doctor without failing registration",+ async ({ cli }) => {+ const doctor = await cli.run({+ arguments: ["doctor", "--json"],+ environment: {+ REA_LOG_LEVEL: "silent",+ REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "7",+ },+ });+ expect(doctor.exitCode, doctor.stderr).toBe(1);+ expect(doctor.stderr).toBe("");+ expect(doctor.json).toMatchObject({+ healthy: false,+ scope_checks: expect.arrayContaining([+ expect.objectContaining({+ name: "runtime-configuration:REA_CAPTURE_DECODER_OLD_GENERATION_MIB",+ ok: false,+ classification: "config_drift",+ }),+ ]),+ });+ },+);diff --git a/tests/boundary/filesystem/clientConfigurationEnvironment.test.ts b/tests/boundary/filesystem/clientConfigurationEnvironment.test.tsindex 53aca18e9..c13d2615f 100644--- a/tests/boundary/filesystem/clientConfigurationEnvironment.test.ts+++ b/tests/boundary/filesystem/clientConfigurationEnvironment.test.ts@@ -496,6 +496,11 @@ it.each([{ REA_REGEX_MATCHING_DEADLINE_MS: "0" },"REA_REGEX_MATCHING_DEADLINE_MS",],+ [+ "capture decoder heap below its floor",+ { REA_CAPTURE_DECODER_OLD_GENERATION_MIB: "7" },+ "REA_CAPTURE_DECODER_OLD_GENERATION_MIB",+ ],["regex worker heap request below its floor",{ REA_REGEX_WORKER_HEAP_REQUEST_MIB: "6" },