Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

fix(codex-llm-proxy): raise the memory limit to 1Gi from the D5 measurement

Alfredo Lopez6cd1d03f43fe
diff --git a/codex-llm-proxy/src/requestLimits.ts b/codex-llm-proxy/src/requestLimits.ts
index 6c2950af8..f0607af43 100644
--- a/codex-llm-proxy/src/requestLimits.ts
+++ b/codex-llm-proxy/src/requestLimits.ts
@@ -23,10 +23,11 @@ export const DEFAULT_MAX_BODY_BYTES = CONTRACT_LIMITS.maxRequestBodyBytes + ENVE
* let 24 bodies in.
* Bodies over the ordinary cap never take this budget: they are V2 visual
* envelopes, bounded by `visualStreamGate` instead, so the declared bodies a pod
- * can hold at once are 3 x 8 MiB here plus 2 x 24 MiB there. Measured with all
- * five in flight, the process peaked at 714 MiB of RSS with an uncapped heap
- * and at 574 MiB with `--max-old-space-size=384`, which is why the deployment
- * sets that cap and a 768Mi memory limit.
+ * can hold at once are 3 x 8 MiB here plus 2 x 24 MiB there. Measured with the
+ * full load the gates admit (eight 8 MiB streams, two 24 MiB visual streams
+ * and three queued 8 MiB bodies, #739 D5), the process peaked at 790 MiB of
+ * RSS with `--max-old-space-size=384` and at 886-1009 MiB with an uncapped
+ * heap, which is why the deployment sets that cap and a 1Gi memory limit.
*/
export const IN_FLIGHT_BODY_BUDGET_BODIES = 3
@@ -66,7 +67,7 @@ export const STREAM_LIMITS = {
* The pod cannot hold the ordinary 8-stream gate across a 24 MiB image, so
* those requests are a tighter sibling and a V2 request keeps the slot until
* the stream ends. Small bodies, including every valid V1, must not enter this
- * gate. The 768Mi limit is sized for these two slots plus the ordinary body
+ * gate. The 1Gi limit is sized for these two slots plus the ordinary body
* budget; widening either one needs a new memory measurement first.
*/
export const VISUAL_STREAM_LIMITS = {
diff --git a/codex-llm-proxy/test/bodyAdmission.test.ts b/codex-llm-proxy/test/bodyAdmission.test.ts
index 027ab5db5..425ef4f8b 100644
--- a/codex-llm-proxy/test/bodyAdmission.test.ts
+++ b/codex-llm-proxy/test/bodyAdmission.test.ts
@@ -2,8 +2,9 @@
* #731 R3-2 - body admission before `express.json`.
*
* At an 8 MiB request cap, the stream gate alone would let 24 bodies in (8
- * running plus 16 queued). Five bodies in flight already peaked at 574-714 MiB
- * of RSS, so 24 would not fit the proxy's 768Mi limit by that ratio (not
+ * running plus 16 queued). Thirteen bodies in flight (eight streams, two
+ * visual, three queued; #739 D5) already peaked at 790 MiB of RSS with the
+ * heap capped, so 24 would not fit the proxy's 1Gi limit by that ratio (not
* measured at 24). These tests drive the real runtime app over HTTP and use
* the control-api `redeem` call as the witness: it runs only after the whole
* body was read, JSON-parsed, contract-parsed and hash-checked, so the number
diff --git a/deploy/base/control-plane/codex-llm-proxy.yaml b/deploy/base/control-plane/codex-llm-proxy.yaml
index 807ef1c4d..481afda23 100644
--- a/deploy/base/control-plane/codex-llm-proxy.yaml
+++ b/deploy/base/control-plane/codex-llm-proxy.yaml
@@ -81,10 +81,10 @@ spec:
secretKeyRef:
name: codex-llm-proxy-secrets
key: CODEX_LLM_PROXY_CONTROL_API_TOKEN
- # Caps V8's old space well under the 768Mi limit so the heap is
- # collected before the kernel kills the pod. Measured peak with
- # three 8 MiB and two 24 MiB bodies in flight: 574 MiB of RSS with
- # this cap, 714 MiB without it.
+ # Caps V8's old space under the 1Gi limit so the heap is collected
+ # before the kernel kills the pod. Measured peak with eight 8 MiB
+ # streams, two 24 MiB visual streams and three queued 8 MiB bodies
+ # (#739 D5): 790 MiB of RSS with this cap, 886-1009 MiB without it.
- name: NODE_OPTIONS
value: "--max-old-space-size=384"
ports:
@@ -113,7 +113,7 @@ spec:
memory: 256Mi
limits:
cpu: 250m
- memory: 768Mi
+ memory: 1Gi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
diff --git a/deploy/base/control-plane/grok-llm-proxy.yaml b/deploy/base/control-plane/grok-llm-proxy.yaml
index 79f994a80..f5d9b4547 100644
--- a/deploy/base/control-plane/grok-llm-proxy.yaml
+++ b/deploy/base/control-plane/grok-llm-proxy.yaml
@@ -78,9 +78,9 @@ spec:
key: GROK_LLM_PROXY_CONTROL_API_TOKEN
# Caps V8's old space well under the 768Mi limit, so heap growth
# is bounded and collected before the kernel kills the pod. The
- # cap does not lower the peak: with three 8 MiB bodies in flight,
- # RSS measured 249-292 MiB with it and 230-252 MiB without it
- # (one run per mode).
+ # cap does not lower the peak: with eight 8 MiB streams and three
+ # queued 8 MiB bodies (#739 D5), RSS measured 510 MiB with it and
+ # 480-511 MiB without it.
- name: NODE_OPTIONS
value: "--max-old-space-size=384"
ports:
diff --git a/grok-llm-proxy/src/requestLimits.ts b/grok-llm-proxy/src/requestLimits.ts
index 58181eb42..592d7b87d 100644
--- a/grok-llm-proxy/src/requestLimits.ts
+++ b/grok-llm-proxy/src/requestLimits.ts
@@ -20,11 +20,11 @@ export const DEFAULT_MAX_BODY_BYTES = CONTRACT_LIMITS.maxRequestBodyBytes + ENVE
* Several copies of a body are alive while it is parsed and hashed (the raw
* buffer, the decoded string, the parsed object, the contract copy and the
* canonical serialization). Without this bound the stream gate would let 24
- * bodies in. Measured with three 8 MiB bodies in flight, the process peaked at
- * 230-252 MiB of RSS with an uncapped heap and at 249-292 MiB with
- * `--max-old-space-size=384` (one run per mode). The capped run went past the
- * former 256Mi limit and the uncapped one came within 4 MiB of it, which is
- * why the deployment sets that cap and a 768Mi memory limit.
+ * bodies in. Measured with the full load the gates admit (eight 8 MiB streams
+ * and three queued 8 MiB bodies, #739 D5), the process peaked at 510 MiB of
+ * RSS with `--max-old-space-size=384` and at 480-511 MiB with an uncapped
+ * heap. That is past the former 256Mi limit, which is why the deployment sets
+ * that cap and a 768Mi memory limit.
*/
export const IN_FLIGHT_BODY_BUDGET_BODIES = 3
diff --git a/grok-llm-proxy/test/bodyAdmission.test.ts b/grok-llm-proxy/test/bodyAdmission.test.ts
index 3eb572d25..7b20d3aa5 100644
--- a/grok-llm-proxy/test/bodyAdmission.test.ts
+++ b/grok-llm-proxy/test/bodyAdmission.test.ts
@@ -2,8 +2,9 @@
* #731 R3-2 - body admission before `express.json`.
*
* At an 8 MiB request cap, the stream gate alone would let 24 bodies in (8
- * running plus 16 queued). Three bodies in flight already peaked at 230-292 MiB
- * of RSS, so 24 would not fit the proxy's 768Mi limit by that ratio (not
+ * running plus 16 queued). Eleven bodies in flight (eight streams, three
+ * queued; #739 D5) already peaked at 510 MiB of RSS with the heap capped, so
+ * 24 would not fit the proxy's 768Mi limit by that ratio (not
* measured at 24). These tests drive the real runtime app over HTTP and use
* the control-api `redeem` call as the witness: it runs only after the whole
* body was read, JSON-parsed, contract-parsed and hash-checked, so the number
diff --git a/scripts/tests/test-grok-llm-proxy-deploy-contract.sh b/scripts/tests/test-grok-llm-proxy-deploy-contract.sh
index 131e86df7..4640a82a0 100755
--- a/scripts/tests/test-grok-llm-proxy-deploy-contract.sh
+++ b/scripts/tests/test-grok-llm-proxy-deploy-contract.sh
@@ -85,11 +85,10 @@ if "DATABASE_URL" in text or "POSTGRES" in text:
if "CONTROL_API_INTERNAL_SERVICE_TOKENS" in text:
errors.append("proxy must not receive the full token map")
-# Three 8 MiB bodies in flight peaked at 230-252 MiB of RSS with an uncapped
-# heap and at 249-292 MiB with a 384 MiB old space (one run per mode). The
-# capped run went past the former 256Mi limit and the uncapped one came within
-# 4 MiB of it, so the pod gets the same 768Mi limit and heap cap as
-# codex-llm-proxy.
+# Eight 8 MiB streams and three queued 8 MiB bodies (#739 D5) peaked at 510 MiB
+# of RSS with a 384 MiB old space and at 480-511 MiB with an uncapped heap. That
+# is past the former 256Mi limit and under 768Mi, so the pod gets a 768Mi limit
+# and the same heap cap as codex-llm-proxy (whose visual slots need 1Gi).
memory_limit = re.search(r"limits:\n\s+cpu: \S+\n\s+memory: (\S+)", text)
memory_request = re.search(r"requests:\n\s+cpu: \S+\n\s+memory: (\S+)", text)
heap_cap = re.search(