Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

fix(deploy): give grok-llm-proxy a 1Gi memory limit for its visual slot

Alfredo Lopeze5cbb043a0f4
diff --git a/deploy/base/control-plane/grok-llm-proxy.yaml b/deploy/base/control-plane/grok-llm-proxy.yaml
index 6894b84e1..557bf8425 100644
--- a/deploy/base/control-plane/grok-llm-proxy.yaml
+++ b/deploy/base/control-plane/grok-llm-proxy.yaml
@@ -79,11 +79,12 @@ spec:
secretKeyRef:
name: grok-llm-proxy-secrets
key: GROK_LLM_PROXY_CONTROL_API_TOKEN
- # Caps V8's old space well under the 768Mi limit, so heap growth
+ # Caps V8's old space well under the 1Gi limit, so heap growth
# is bounded and collected before the kernel kills the pod. The
# cap does not lower the peak: with eight 8 MiB streams and three
# queued 8 MiB bodies (#739 D5), RSS measured 510 MiB with it and
- # 480-511 MiB without it.
+ # 480-511 MiB without it. One ~36 MB V2 stream in the visual slot
+ # on top of that load (#784) raised the capped peak to 775 MiB.
- name: NODE_OPTIONS
value: "--max-old-space-size=384"
ports:
@@ -112,7 +113,7 @@ spec:
memory: 768Mi
limits:
cpu: 250m
- memory: 768Mi
+ memory: 1Gi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
diff --git a/grok-llm-proxy/src/requestLimits.ts b/grok-llm-proxy/src/requestLimits.ts
index 344fbb18c..2412d4cda 100644
--- a/grok-llm-proxy/src/requestLimits.ts
+++ b/grok-llm-proxy/src/requestLimits.ts
@@ -22,11 +22,12 @@ export const DEFAULT_MAX_BODY_BYTES = CONTRACT_LIMITS.maxRequestBodyBytes + ENVE
* canonical serialization). Without this bound the stream gate would let 24
* bodies in.
* Bodies over the ordinary cap never take this budget: they are V2 visual
- * envelopes, bounded by `visualStreamGate` instead. Measured with the full load the gates admit (eight 8 MiB streams
- * and three queued 8 MiB bodies, #739 D5), the process peaked at 510 MiB of
+ * envelopes, bounded by `visualStreamGate` instead. With eight 8 MiB streams
+ * and three queued 8 MiB bodies (#739 D5) the process peaked at 510 MiB of
* RSS with `--max-old-space-size=384` and at 480-511 MiB with an uncapped
* heap. That is past the former 256Mi limit, which is why the deployment sets
- * that cap and a 768Mi memory limit.
+ * that cap. The full load the gates admit adds one visual stream; see
+ * `VISUAL_STREAM_LIMITS` for that peak and the memory limit it sets.
*/
export const IN_FLIGHT_BODY_BUDGET_BODIES = 3
@@ -68,8 +69,12 @@ export const STREAM_LIMITS = {
* a 35 MiB envelope is more than four ordinary bodies, so those requests take
* a 1-wide sibling of the 8-wide stream gate and keep the slot for the
* stream. Small bodies, including every valid V1, must not enter this gate.
- * The 768Mi limit is sized for this slot plus the ordinary body budget;
- * widening either one needs a new memory measurement first.
+ * Measured with the full load the gates admit (the D5 load above plus one
+ * ~36 MB V2 stream: a 20 MiB PNG and 8 MiB of text; tsc build, one process,
+ * `--max-old-space-size=384`, upstream request through undici), the process
+ * peaked at 775 MiB of RSS, against 511 MiB for D5 alone. The deployment's
+ * 1Gi memory limit is that peak plus 25 %, rounded up; widening this gate or
+ * the ordinary body budget needs a new memory measurement first.
*/
export const VISUAL_STREAM_LIMITS = {
maxConcurrentStreams: 1,
diff --git a/grok-llm-proxy/test/bodyAdmission.test.ts b/grok-llm-proxy/test/bodyAdmission.test.ts
index fc05f2a9a..5673401cf 100644
--- a/grok-llm-proxy/test/bodyAdmission.test.ts
+++ b/grok-llm-proxy/test/bodyAdmission.test.ts
@@ -4,8 +4,7 @@
* At an 8 MiB request cap, the stream gate alone would let 24 bodies in (8
* running plus 16 queued). Eleven bodies in flight (eight streams, three
* queued; #739 D5) already peaked at 510 MiB of RSS with the heap capped, so
- * 24 would not fit the proxy's 768Mi limit by that ratio (not
- * measured at 24). These tests drive the real runtime app over HTTP and use
+ * 24 would not fit the proxy's 1Gi limit by that ratio (not measured at 24). These tests drive the real runtime app over HTTP and use
* the control-api `redeem` call as the witness: it runs only after the whole
* body was read, JSON-parsed, contract-parsed and hash-checked, so the number
* of attempts held there is the number of bodies in memory.
diff --git a/grok-llm-proxy/test/requestLimits.test.ts b/grok-llm-proxy/test/requestLimits.test.ts
index 834365998..7ba6426d8 100644
--- a/grok-llm-proxy/test/requestLimits.test.ts
+++ b/grok-llm-proxy/test/requestLimits.test.ts
@@ -388,7 +388,7 @@ describe('StreamGate', () => {
// A V2 body above the ordinary cap keeps its image bytes resident for the
// whole upstream stream, so it takes a 1-wide sibling of the 8-wide gate.
- // Widening it needs a new memory measurement against the 768Mi limit.
+ // Widening it needs a new memory measurement against the 1Gi limit.
it('pins the visual 1/4 sibling against the ordinary 8/16 stream gate', () => {
expect(VISUAL_STREAM_LIMITS).toEqual({ maxConcurrentStreams: 1, maxQueuedRequests: 4 })
expect(STREAM_LIMITS.maxConcurrentStreams).toBe(8)