Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

docs(llm-proxy): cite where each idle timeout comes from and call the cap policy

Alfredo Lopez64f269850525
diff --git a/codex-llm-proxy/src/requestLimits.ts b/codex-llm-proxy/src/requestLimits.ts
index 60beb47ef..6c2950af8 100644
--- a/codex-llm-proxy/src/requestLimits.ts
+++ b/codex-llm-proxy/src/requestLimits.ts
@@ -54,7 +54,10 @@ export const STREAM_LIMITS = {
// timeout.
maxQueueWaitMs: 60_000,
// Longest silence tolerated while waiting on the upstream (response headers
- // or the next SSE chunk). Matches the Codex CLI stream idle timeout.
+ // or the next SSE chunk). Same value as the Codex CLI default,
+ // `DEFAULT_STREAM_IDLE_TIMEOUT_MS` (300 s) in openai/codex
+ // codex-rs/model-provider-info/src/lib.rs:63 (commit 6824dabe0, read
+ // 2026-09-23).
upstreamIdleTimeoutMs: 300_000,
} as const
diff --git a/docs/architecture/codex-subscription-transport-contract.md b/docs/architecture/codex-subscription-transport-contract.md
index 7ab3ef91a..606342e1d 100644
--- a/docs/architecture/codex-subscription-transport-contract.md
+++ b/docs/architecture/codex-subscription-transport-contract.md
@@ -468,8 +468,14 @@ code the proxy constructs, and every code it refuses a request with
upstream body and returns HTTP 504, or an SSE error frame when text had
already been streamed. The Host maps it to `LLM_STREAM_DURATION_EXCEEDED`.
It is not retryable and not failover-eligible: another attempt would spend
- the same budget on the same turn.
-- Idle timeout: when the upstream sends no byte for `upstreamIdleTimeoutMs`,
+ the same budget on the same turn. The 1800000 ms value is a policy choice,
+ not an upstream limit: the upstream publishes no maximum stream length.
+ Observed durations are recorded in
+ `codex_llm_proxy_stream_duration_seconds` (buckets up to 1800 s), which is
+ the data the cap should be revisited with.
+- Idle timeout: when the upstream sends no byte for `upstreamIdleTimeoutMs`
+ (the Codex CLI default `DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000`,
+ openai/codex `codex-rs/model-provider-info/src/lib.rs:63` at 6824dabe0),
the proxy cancels the upstream body and fails the attempt with
`provider_unavailable` (HTTP 503, reason `upstream stream idle timeout`).
That code stays retryable and failover-eligible, because a silent upstream
diff --git a/docs/architecture/grok-subscription-transport-contract.md b/docs/architecture/grok-subscription-transport-contract.md
index 36d2ffeac..39ed6af4d 100644
--- a/docs/architecture/grok-subscription-transport-contract.md
+++ b/docs/architecture/grok-subscription-transport-contract.md
@@ -328,9 +328,15 @@ code the proxy constructs, and every code it refuses a request with
upstream body and returns HTTP 504, or an SSE error frame when text had
already been streamed. The Host maps it to `LLM_STREAM_DURATION_EXCEEDED`.
It is not retryable and not failover-eligible: another attempt would spend
- the same budget on the same turn.
+ the same budget on the same turn. The 1800000 ms value is a policy choice,
+ not an upstream limit: the upstream publishes no maximum stream length.
+ Observed durations are recorded in
+ `grok_llm_proxy_stream_duration_seconds` (buckets up to 1800 s), which is
+ the data the cap should be revisited with.
- Idle timeout: when the upstream sends no byte for `upstreamIdleTimeoutMs`
- (the value read from the Grok Build client source), the proxy cancels the upstream
+ (the Grok Build CLI default `inference_idle_timeout_secs = 600`, read from
+ the config embedded in the closed grok-build 1.0.41 binary; no public
+ source exists to cite), the proxy cancels the upstream
body and fails the attempt with `provider_unavailable` (HTTP 503, reason
`upstream stream idle timeout`). That code stays retryable and
failover-eligible, because a silent upstream is an outage of that provider,
diff --git a/grok-llm-proxy/src/requestLimits.ts b/grok-llm-proxy/src/requestLimits.ts
index 6c79b1391..58181eb42 100644
--- a/grok-llm-proxy/src/requestLimits.ts
+++ b/grok-llm-proxy/src/requestLimits.ts
@@ -51,7 +51,10 @@ export const STREAM_LIMITS = {
// keepalive stays below the Host HTTP client's 300 s header timeout.
maxQueueWaitMs: 60_000,
// Longest silence tolerated while waiting on the upstream (response headers
- // or the next SSE chunk). Matches the Grok Build inference idle timeout.
+ // or the next SSE chunk). Same value as the Grok Build CLI default,
+ // `inference_idle_timeout_secs = 600` in the default config embedded in the
+ // grok-build 1.0.41 macOS binary (read with `strings`, 2026-09-23). The CLI
+ // source is not public, so no file and line can be cited.
upstreamIdleTimeoutMs: 600_000,
} as const