Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
fix(deploy): give grok-llm-proxy 768Mi and a 384 MiB heap cap (#739)
Alfredo Lopezaa506853b882
diff --git a/deploy/base/control-plane/grok-llm-proxy.yaml b/deploy/base/control-plane/grok-llm-proxy.yamlindex 831653870..f22147197 100644--- a/deploy/base/control-plane/grok-llm-proxy.yaml+++ b/deploy/base/control-plane/grok-llm-proxy.yaml@@ -73,6 +73,12 @@ spec:secretKeyRef:name: grok-llm-proxy-secretskey: GROK_LLM_PROXY_CONTROL_API_TOKEN+ # Caps V8's old space well under the 768Mi limit so the heap is+ # collected before the kernel kills the pod. Measured peak with+ # three 8 MiB bodies in flight: 230-252 MiB of RSS without this+ # cap and 249-292 MiB with it (one run per mode).+ - name: NODE_OPTIONS+ value: "--max-old-space-size=384"ports:- name: runtimecontainerPort: 8080@@ -96,10 +102,10 @@ spec:resources:requests:cpu: 50m- memory: 64Mi+ memory: 256Milimits:cpu: 250m- memory: 256Mi+ memory: 768MisecurityContext:allowPrivilegeEscalation: falsereadOnlyRootFilesystem: truediff --git a/grok-llm-proxy/src/requestLimits.ts b/grok-llm-proxy/src/requestLimits.tsindex d76a0d975..fda0c39fa 100644--- a/grok-llm-proxy/src/requestLimits.ts+++ b/grok-llm-proxy/src/requestLimits.ts@@ -15,10 +15,13 @@ export const DEFAULT_MAX_BODY_BYTES = CONTRACT_LIMITS.maxRequestBodyBytes + ENVE/*** #731 R3-2 — how many maximum-size bodies may be read and parsed at once.- * About five copies of a body are alive while it is parsed and hashed (the raw+ * Several copies of a body are alive while it is parsed and hashed (the raw* buffer, the decoded string, the parsed object, the contract copy and the- * canonical serialization), so three 8 MiB bodies hold about 120 MiB of the- * pod's 256Mi. Without this bound the stream gate would let 24 bodies in.+ * canonical serialization). Without this bound the stream gate would let 24+ * bodies in. Measured with three 8 MiB bodies in flight, the process peaked at+ * 230-252 MiB of RSS with an uncapped heap and at 249-292 MiB with+ * `--max-old-space-size=384` (one run per mode), past the former 256Mi limit,+ * which is why the deployment sets that cap and a 768Mi memory limit.*/export const IN_FLIGHT_BODY_BUDGET_BODIES = 3diff --git a/grok-llm-proxy/test/bodyAdmission.test.ts b/grok-llm-proxy/test/bodyAdmission.test.tsindex d15bdec75..ec5fbb0a7 100644--- a/grok-llm-proxy/test/bodyAdmission.test.ts+++ b/grok-llm-proxy/test/bodyAdmission.test.ts@@ -1,9 +1,10 @@/*** #731 R3-2 - body admission before `express.json`.*- * At an 8 MiB request cap, parsing every body the stream gate lets in (8- * running plus 16 queued) holds about five copies of each in memory, far past- * the proxy's 256Mi. These tests drive the real runtime app over HTTP and use+ * At an 8 MiB request cap, the stream gate alone would let 24 bodies in (8+ * running plus 16 queued). Three bodies in flight already peaked at 230-292 MiB+ * of RSS, so 24 would not fit the proxy's 768Mi limit by that ratio (not+ * measured at 24). These tests drive the real runtime app over HTTP and use* the control-api `redeem` call as the witness: it runs only after the whole* body was read, JSON-parsed, contract-parsed and hash-checked, so the number* of attempts held there is the number of bodies in memory.