Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

docs(contracts): describe the repeated-text shadow and the rollout band (R4-L2/L3/L5/L7)

Alfredo Lopeze9028d81c98b
diff --git a/docs/architecture/codex-subscription-transport-contract.md b/docs/architecture/codex-subscription-transport-contract.md
index ac5660040..3f5e6f8b0 100644
--- a/docs/architecture/codex-subscription-transport-contract.md
+++ b/docs/architecture/codex-subscription-transport-contract.md
@@ -56,9 +56,9 @@ HTTP envelope, including base64, history and the signed execution ticket, with
no allowance on top: the authorize route's JSON parser, the gateway's
`client_max_body_size` and `buildCodexProxyEnvelope` all hold the whole V2
envelope to `maxVisualRequestBodyBytes`. The ticket is about 3.5 KB of it. V2 text, tools and other
-non-image fields remain bounded to `maxRequestBodyBytes`, measured with only
-image data blanked in a temporary size projection; the actual request and its
-hash are not modified. The two caps are not additive: a V2 request carrying a
+non-image fields remain bounded to `maxRequestBodyBytes`, measured with image
+data and text parts that repeat `content` blanked in a temporary size
+projection; the actual request and its hash are not modified. The two caps are not additive: a V2 request carrying a
hard-ceiling image (16 MiB decoded, about 21.3 MiB encoded) has about 2.7 MiB
left for non-image data before the 24 MiB envelope refuses it.
The authorizer builds the exact V2 proxy envelope inside its transaction after
@@ -100,8 +100,13 @@ not keep a second model-vision allowlist. To stop all Codex traffic, use the
existing kill switches (`MCP_HOST_CODEX_SUBSCRIPTION_ENABLED` and
`CODEX_LLM_PROXY_EXECUTION_ENABLED`). Missing image provenance returns
`image_source_invalid`. Limit errors, including HTTP 413 without a JSON body,
-remain non-retryable. Images are never silently stripped and do not trigger
-automatic fallback.
+remain non-retryable and do not fail over. Images are never silently stripped.
+Two 503s the proxy gives an image request before redeem are different:
+`visual_gate` (the visual gate is full) and `visual_host_share` (the
+principal's share of it is full) answer `provider_unavailable`, which the Host
+classifies as `LLM_MODEL_OVERLOADED`, retryable, and a Host with a fallback
+provider configured fails over on it. Whether that should stay retryable is
+tracked in #868.
Residual risk. Tool screenshots reach the upstream in a user-role message,
after the tool messages, behind the fixed text "These images are output of the
@@ -540,9 +545,16 @@ behavior changes:
It rejects explicit disagreeing pairs with 422
`subscriptionAnnotationsDisagree`. Broker changes that omit the annotations
return 422 `providerChangeRequiresGrant`.
-- **Rollout.** Roll out control-api first, and wait until every pod runs the
- new image. An older control-api rejects the entire workflow-control token
- issue when HCC or WRC request the unknown `llm:grok:execute` scope.
+- **Rollout.** Roll out `codex-llm-proxy` before control-api and the Host
+ images. #784 stops counting V2 text parts that repeat `content` toward the
+ non-image budget; a proxy from before #784 still counts that text twice. A
+ V2 request with about 4 to 8 MiB of text passes the new Host and control-api
+ and is then refused by the old proxy with 413 `payload_too_large`, after the
+ attempt was authorized (measured: 5 MiB of text and one image measure
+ 5.0 MiB on the new side and 10.0 MiB on the old proxy); the user sees
+ "Conversation Too Long". Then roll out control-api, and wait until every pod
+ runs the new image. An older control-api rejects the entire workflow-control
+ token issue when HCC or WRC request the unknown `llm:grok:execute` scope.
- **Rollback.** Turn the Grok flags off and wait for HCC and WRC to drop
`llm:grok:execute` before rolling back control-api. While an older
control-api serves, a reassigned Codex recipe grant updates only
diff --git a/docs/architecture/grok-subscription-transport-contract.md b/docs/architecture/grok-subscription-transport-contract.md
index ceb8079c5..d4d0176f8 100644
--- a/docs/architecture/grok-subscription-transport-contract.md
+++ b/docs/architecture/grok-subscription-transport-contract.md
@@ -52,8 +52,8 @@ encoded image budget (`4 × ceil(20971520 / 3)` = 27962028 bytes) plus the
serialized V2 request and its HTTP envelope, signed ticket included, with no
envelope allowance on top, as for Codex. V1 keeps `maxRequestBodyBytes` plus
the 16 KiB `ENVELOPE_ALLOWANCE_BYTES`. Text, tools and other non-image fields
-of a V2 request stay bounded to `maxRequestBodyBytes`, measured with the image
-data blanked in a temporary projection.
+of a V2 request stay bounded to `maxRequestBodyBytes`, measured with image
+data and text parts that repeat `content` blanked in a temporary projection.
The contract checks a V2 request in this order:
@@ -70,11 +70,17 @@ The contract checks a V2 request in this order:
`request exceeds maxVisualRequestBodyBytes`;
4. each image and the request's image totals, while the parts are parsed.
-A request over 35 MiB because of its text is therefore reported as text. With
-the non-image share within 8 MiB, a request over 35 MiB carries more than
-28311552 encoded image bytes, which is already over the 20 MiB image budget, so
-the Host reports the whole-body refusal as an image refusal
-(`attachment_too_large`, below).
+The non-image share does not count text parts that repeat `content`, so a V2
+body can carry up to about 16 MiB of text while 8 MiB is counted. Such a body
+can exceed 35 MiB with its images inside the 20 MiB budget: one 16 MiB image
+plus 7 MiB of text, or two 10 MiB images plus 4.5 MiB of text. It gets the
+whole-body refusal, and the Host reports every whole-body refusal as an image
+refusal (`attachment_too_large`, below), so the user is told to send fewer or
+smaller images although shorter text would also fit. The Host refuses it
+before authorize, so no attempt is spent. Codex behaves the same way. For the
+same reason a V2 request accepted close to 35 MiB can leave less room than the
+authorize wrapper and the signed ticket need; the Host's authorize-body check
+refuses it, also before any attempt.
The value is a runtime limit, not a published fixture limit: the
`grok-llm-proxy/test/contractFreeze.test.ts` fixture describes the measured
@@ -965,8 +971,10 @@ Grok annotations. A new control-api also republishes on boot.
follows automatically through HCC and WRC.
Image input (#784) on a deployment where the Grok flags are already on: roll
-out `grok-llm-proxy` first, then control-api together with the control-plane
-ConfigMap, then the four Host images, and wait until every pod of each runs
+out `grok-llm-proxy` first, together with `codex-llm-proxy` (#784 also changes
+the Codex non-image measurement; see the Codex transport contract, Rollout),
+then control-api together with the control-plane ConfigMap, then the four Host
+images, and wait until every pod of each runs
the new image before starting the next. With the ConfigMap, restart
`nginx-workflow-approval-gateway`: it mounts `nginx.conf` through `subPath`,
so it keeps the old authorize `client_max_body_size` (25165824) until its pods
diff --git a/packages/grok-provider-attempt-contract/index.cjs b/packages/grok-provider-attempt-contract/index.cjs
index 9348dcf03..641d5b919 100644
--- a/packages/grok-provider-attempt-contract/index.cjs
+++ b/packages/grok-provider-attempt-contract/index.cjs
@@ -36,9 +36,12 @@ const LIMITS = Object.freeze({
maxRequestBodyBytes: MAX_REQUEST_BODY_BYTES,
// V2 request/envelope ceiling: the whole image budget encoded as base64
// (27962028 bytes) plus the non-image share, rounded up to a whole MiB,
- // which gives 36700160 (35 MiB). The rounding leaves more than
- // ENVELOPE_ALLOWANCE_BYTES of headroom, so a V2 envelope is bounded by this
- // value as a whole and gets no allowance on top, as in the Codex contract.
+ // which gives 36700160 (35 MiB). A V2 envelope is bounded by this value as
+ // a whole and gets no allowance on top, as in the Codex contract. The
+ // non-image share does not count text parts that repeat `content`, so a V2
+ // body carries up to twice that share in text and can reach this ceiling
+ // with its images inside their budget; the whole-body refusal then reports
+ // it, and no envelope headroom is guaranteed.
maxVisualRequestBodyBytes:
Math.ceil((MAX_ENCODED_TOTAL_IMAGE_BYTES + MAX_REQUEST_BODY_BYTES) / MIB) * MIB,
maxMessages: 1024,
@@ -675,9 +678,11 @@ function parseGrokCompletionRequestRoot(input, schemaVersion, messageKeys) {
} catch {
return fail('invalid', 'request is not JSON-serializable')
}
- // The non-image check runs first, so a V2 request over the visual ceiling
- // because of its text is reported as text; only image data can reach the
- // whole-body message below.
+ // The non-image check runs first. It does not count text parts that repeat
+ // `content`, so a V2 request can pass it and still exceed the visual
+ // ceiling below because its text travels twice (one 16 MiB image plus
+ // 7 MiB of text); that request gets the whole-body message, which the Host
+ // reports as an attachment refusal.
if (visualSchema && measureNonImageRequestBytes(input) > LIMITS.maxRequestBodyBytes) {
return fail('limit', 'request exceeds maxRequestBodyBytes outside image data', 'size')
}
@@ -884,9 +889,11 @@ function hashGrokCompletionRequest(request) {
* A deployment that lowers that proxy limit below the contract limit is not
* covered by this measurement.
*
- * The request inside the envelope already passed the V2 non-image budget, so
- * the V2 headroom here can only be spent by the ticket and the digest the
- * authorizer produced; a caller cannot reach this ceiling with text.
+ * The request inside the envelope already passed the V2 non-image budget,
+ * which does not count text parts that repeat `content`. A request accepted
+ * close to the ceiling can leave less room than the authorize wrapper, the
+ * ticket and the digest need; the Host's authorize-body check or this check
+ * then refuses it.
*/
function buildGrokProxyEnvelope(input) {
if (!isPlainObject(input)) return fail('invalid', 'proxy envelope must be an object')
diff --git a/packages/grok-provider-attempt-contract/index.d.ts b/packages/grok-provider-attempt-contract/index.d.ts
index ddfc73043..4e33cdef1 100644
--- a/packages/grok-provider-attempt-contract/index.d.ts
+++ b/packages/grok-provider-attempt-contract/index.d.ts
@@ -292,19 +292,20 @@ export declare function stableStringify(value: unknown): string
* BEFORE parsing: 35 MiB for a document that declares
* `grok-completion-request.v2`, `maxRequestBodyBytes` (8 MiB) for anything
* else. Declaring V2 does not raise the budget for text or tool definitions —
- * the parser measures the body with every image payload blanked and keeps that
- * share on the `maxRequestBodyBytes` ceiling.
+ * the parser measures the body with image data and text parts that repeat
+ * `content` blanked, and keeps that share on the `maxRequestBodyBytes` ceiling.
*/
export declare function requestBodyLimitBytes(request: unknown): number
/**
- * Byte length of an authorize document after every image payload inside
- * `body.request` is blanked. Wrapper fields stay on the non-image budget
- * even when the nested request declares V2.
+ * Byte length of an authorize document with image data and text parts that
+ * repeat `content` blanked inside `body.request`. Wrapper fields stay on the
+ * non-image budget even when the nested request declares V2.
*/
export declare function measureNonImageAuthorizeBytes(body: unknown): number
/**
- * Byte length of a proxy completion document after image payloads inside
- * `body.request` are blanked and `executionTicket` is omitted. The ticket is
+ * Byte length of a proxy completion document with image data and text parts
+ * that repeat `content` blanked inside `body.request`, and `executionTicket`
+ * omitted. The ticket is
* issued after authorize, so it must not consume the non-image budget
* that authorize already applied to the pre-ticket wrapper.
*/
diff --git a/packages/grok-provider-attempt-contract/index.test.cjs b/packages/grok-provider-attempt-contract/index.test.cjs
index 96b88935c..11e70940a 100644
--- a/packages/grok-provider-attempt-contract/index.test.cjs
+++ b/packages/grok-provider-attempt-contract/index.test.cjs
@@ -1059,6 +1059,31 @@ test('non-image share over maxRequestBodyBytes is refused outside image data', (
)
})
+// Review R4-L2, case A: text parts that repeat `content` are not counted in
+// the non-image share, so text travelling twice can take a V2 body over the
+// visual ceiling with its image inside the budget. The whole-body message
+// reports it (the Host shows it as an attachment refusal); this pins that
+// labeling, documented in the transport contract.
+test('a V2 body over the visual ceiling because of repeated text gets the whole-body refusal', () => {
+ const image = imagePart(declaredHeaderPngOfSize(16 * MIB).toString('base64'))
+ const request = text => v2WithParts([{ type: 'text', text }, image], text)
+ const over = request('x'.repeat(7 * MIB))
+ // The non-image check runs first, so this message also shows the 7 MiB of
+ // text passed it.
+ assert.ok(
+ Buffer.byteLength(JSON.stringify(over), 'utf8') > contract.LIMITS.maxVisualRequestBodyBytes
+ )
+ assert.deepEqual(contract.parseGrokCompletionRequest(over), {
+ ok: false,
+ code: 'limit',
+ kind: 'size',
+ message: 'request exceeds maxVisualRequestBodyBytes',
+ })
+ // Witness: the same image with less text is accepted.
+ const within = contract.parseGrokCompletionRequest(request('x'.repeat(3 * MIB)))
+ assert.equal(within.ok, true, within.message)
+})
+
// The non-image check runs before the whole-body check, so text past the
// visual ceiling is reported as text, not as an attachment over budget. The
// whole-body message stays reachable only through image data.
diff --git a/packages/llm-provider-attempt-contract/index.cjs b/packages/llm-provider-attempt-contract/index.cjs
index ea465c122..8741d051e 100644
--- a/packages/llm-provider-attempt-contract/index.cjs
+++ b/packages/llm-provider-attempt-contract/index.cjs
@@ -646,8 +646,9 @@ function parseTransportHints(raw) {
* Two size gates run first, before any key, id or payload work:
* 1. the whole body against this version's ceiling (24 MiB for V2, 8 MiB
* otherwise), then
- * 2. for V2 only, the body with every image payload blanked against the V1
- * ceiling — the images, not the text, are what the larger budget buys.
+ * 2. for V2 only, the body with image data and text parts that repeat
+ * `content` blanked against the V1 ceiling — the images, not the text,
+ * are what the larger budget buys.
* Both are pure measurements, so the projected value and its hash are
* unaffected by them.
*/
@@ -880,9 +881,11 @@ function hashCodexCompletionRequest(request) {
* A deployment that lowers that proxy limit below the contract limit is not
* covered by this measurement.
*
- * The request inside the envelope already passed the V2 non-image budget, so
- * the V2 headroom here can only be spent by the ticket and the digest the
- * authorizer produced; a caller cannot reach this ceiling with text.
+ * The request inside the envelope already passed the V2 non-image budget,
+ * which does not count text parts that repeat `content`. A request accepted
+ * close to the ceiling can leave less room than the authorize wrapper, the
+ * ticket and the digest need; the Host's authorize-body check or this check
+ * then refuses it.
*/
function buildCodexProxyEnvelope(input) {
if (!isPlainObject(input)) return fail('invalid', 'proxy envelope must be an object')
diff --git a/packages/llm-provider-attempt-contract/index.d.ts b/packages/llm-provider-attempt-contract/index.d.ts
index 1feaecbac..c184aae43 100644
--- a/packages/llm-provider-attempt-contract/index.d.ts
+++ b/packages/llm-provider-attempt-contract/index.d.ts
@@ -305,20 +305,21 @@ export declare function stableStringify(value: unknown): string
* BEFORE parsing: 24 MiB for a document that declares
* `codex-completion-request.v2`, `maxRequestBodyBytes` (8 MiB) for anything
* else. Declaring V2 does not raise the budget for text or tool definitions —
- * the parser measures the body with every image payload blanked and keeps that
- * share on the `maxRequestBodyBytes` ceiling.
+ * the parser measures the body with image data and text parts that repeat
+ * `content` blanked, and keeps that share on the `maxRequestBodyBytes` ceiling.
*/
export declare function requestBodyLimitBytes(request: unknown): number
/**
- * Byte length of an authorize document after every image payload inside
- * `body.request` is blanked. Wrapper fields stay on the non-image budget
- * (`maxRequestBodyBytes` plus the caller's envelope allowance) even when the
- * nested request declares V2.
+ * Byte length of an authorize document with image data and text parts that
+ * repeat `content` blanked inside `body.request`. Wrapper fields stay on the
+ * non-image budget (`maxRequestBodyBytes` plus the caller's envelope
+ * allowance) even when the nested request declares V2.
*/
export declare function measureNonImageAuthorizeBytes(body: unknown): number
/**
- * Byte length of a proxy completion document after image payloads inside
- * `body.request` are blanked and `executionTicket` is omitted. The ticket is
+ * Byte length of a proxy completion document with image data and text parts
+ * that repeat `content` blanked inside `body.request`, and `executionTicket`
+ * omitted. The ticket is
* issued after authorize, so it must not consume the non-image budget
* that authorize already applied to the pre-ticket wrapper.
*/