Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
fix(deploy): give grok-llm-proxy a 1Gi memory limit for its visual slot
Alfredo Lopeze5cbb043a0f4
diff --git a/deploy/base/control-plane/grok-llm-proxy.yaml b/deploy/base/control-plane/grok-llm-proxy.yamlindex 6894b84e1..557bf8425 100644--- a/deploy/base/control-plane/grok-llm-proxy.yaml+++ b/deploy/base/control-plane/grok-llm-proxy.yaml@@ -79,11 +79,12 @@ spec:secretKeyRef:name: grok-llm-proxy-secretskey: GROK_LLM_PROXY_CONTROL_API_TOKEN- # Caps V8's old space well under the 768Mi limit, so heap growth+ # Caps V8's old space well under the 1Gi limit, so heap growth# is bounded and collected before the kernel kills the pod. The# cap does not lower the peak: with eight 8 MiB streams and three# queued 8 MiB bodies (#739 D5), RSS measured 510 MiB with it and- # 480-511 MiB without it.+ # 480-511 MiB without it. One ~36 MB V2 stream in the visual slot+ # on top of that load (#784) raised the capped peak to 775 MiB.- name: NODE_OPTIONSvalue: "--max-old-space-size=384"ports:@@ -112,7 +113,7 @@ spec:memory: 768Milimits:cpu: 250m- memory: 768Mi+ memory: 1GisecurityContext:allowPrivilegeEscalation: falsereadOnlyRootFilesystem: truediff --git a/grok-llm-proxy/src/requestLimits.ts b/grok-llm-proxy/src/requestLimits.tsindex 344fbb18c..2412d4cda 100644--- a/grok-llm-proxy/src/requestLimits.ts+++ b/grok-llm-proxy/src/requestLimits.ts@@ -22,11 +22,12 @@ export const DEFAULT_MAX_BODY_BYTES = CONTRACT_LIMITS.maxRequestBodyBytes + ENVE* canonical serialization). Without this bound the stream gate would let 24* bodies in.* Bodies over the ordinary cap never take this budget: they are V2 visual- * envelopes, bounded by `visualStreamGate` instead. Measured with the full load the gates admit (eight 8 MiB streams- * and three queued 8 MiB bodies, #739 D5), the process peaked at 510 MiB of+ * envelopes, bounded by `visualStreamGate` instead. With eight 8 MiB streams+ * and three queued 8 MiB bodies (#739 D5) the process peaked at 510 MiB of* RSS with `--max-old-space-size=384` and at 480-511 MiB with an uncapped* heap. That is past the former 256Mi limit, which is why the deployment sets- * that cap and a 768Mi memory limit.+ * that cap. The full load the gates admit adds one visual stream; see+ * `VISUAL_STREAM_LIMITS` for that peak and the memory limit it sets.*/export const IN_FLIGHT_BODY_BUDGET_BODIES = 3@@ -68,8 +69,12 @@ export const STREAM_LIMITS = {* a 35 MiB envelope is more than four ordinary bodies, so those requests take* a 1-wide sibling of the 8-wide stream gate and keep the slot for the* stream. Small bodies, including every valid V1, must not enter this gate.- * The 768Mi limit is sized for this slot plus the ordinary body budget;- * widening either one needs a new memory measurement first.+ * Measured with the full load the gates admit (the D5 load above plus one+ * ~36 MB V2 stream: a 20 MiB PNG and 8 MiB of text; tsc build, one process,+ * `--max-old-space-size=384`, upstream request through undici), the process+ * peaked at 775 MiB of RSS, against 511 MiB for D5 alone. The deployment's+ * 1Gi memory limit is that peak plus 25 %, rounded up; widening this gate or+ * the ordinary body budget needs a new memory measurement first.*/export const VISUAL_STREAM_LIMITS = {maxConcurrentStreams: 1,diff --git a/grok-llm-proxy/test/bodyAdmission.test.ts b/grok-llm-proxy/test/bodyAdmission.test.tsindex fc05f2a9a..5673401cf 100644--- a/grok-llm-proxy/test/bodyAdmission.test.ts+++ b/grok-llm-proxy/test/bodyAdmission.test.ts@@ -4,8 +4,7 @@* At an 8 MiB request cap, the stream gate alone would let 24 bodies in (8* running plus 16 queued). Eleven bodies in flight (eight streams, three* queued; #739 D5) already peaked at 510 MiB of RSS with the heap capped, so- * 24 would not fit the proxy's 768Mi limit by that ratio (not- * measured at 24). These tests drive the real runtime app over HTTP and use+ * 24 would not fit the proxy's 1Gi limit by that ratio (not measured at 24). These tests drive the real runtime app over HTTP and use* the control-api `redeem` call as the witness: it runs only after the whole* body was read, JSON-parsed, contract-parsed and hash-checked, so the number* of attempts held there is the number of bodies in memory.diff --git a/grok-llm-proxy/test/requestLimits.test.ts b/grok-llm-proxy/test/requestLimits.test.tsindex 834365998..7ba6426d8 100644--- a/grok-llm-proxy/test/requestLimits.test.ts+++ b/grok-llm-proxy/test/requestLimits.test.ts@@ -388,7 +388,7 @@ describe('StreamGate', () => {// A V2 body above the ordinary cap keeps its image bytes resident for the// whole upstream stream, so it takes a 1-wide sibling of the 8-wide gate.- // Widening it needs a new memory measurement against the 768Mi limit.+ // Widening it needs a new memory measurement against the 1Gi limit.it('pins the visual 1/4 sibling against the ordinary 8/16 stream gate', () => {expect(VISUAL_STREAM_LIMITS).toEqual({ maxConcurrentStreams: 1, maxQueuedRequests: 4 })expect(STREAM_LIMITS.maxConcurrentStreams).toBe(8)