Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
Grok subscription validation
Evidence lanes for grok-subscription image input (issue #784). One lane does
not stand in for another.
Purpose
cli-chat-proxy.grok.com/v1/responses, the endpoint grok-llm-proxy calls,
is unmeasured for images. The limits and the wire shape in
the transport contract
come from the xAI API documentation for api.x.ai/v1 (docs.x.ai, read
2026-09-23): 20 MiB per image, JPEG and PNG, any text/image order, and
{ "type": "input_image", "image_url": "data:<mime>;base64,<data>", "detail": "high" }
next to input_text parts. The contract, proxy, control-api, Host and Desktop
suites prove what Evenfire sends and refuses. None of them proves that the
subscription endpoint accepts that body or that the model sees the pixels.
The live probe below is the only check that does. Run it when a Grok subscription is available. Until it has run, report the endpoint as unmeasured for images.
Preconditions
- Separate authorization for the run: it calls the real upstream and spends subscription quota.
- A branch-owned local profile, bootstrapped by the canonical Minikube entry points, with the images built from the commit under test. Do not touch another branch's profile or port-forwards.
GROK_LLM_PROXY_EXECUTION_ENABLEDon, a connected Grok grant, and a Host whose Grok projection mintsllm:grok:execute(HCC then injectsMCP_HOST_GROK_SUBSCRIPTION_ENABLED). There is no image-specific flag.- A catalog model that the xAI documentation lists with image input (grok-4.5, grok-4.6 or grok-4.7). Record the canonical model id the runtime returns, not an alias.
- Never read
~/.grok/auth.json, and never read theapi_keyfields of~/.grok/models_cache.json. The probe runs through the real Host → control-api →grok-llm-proxypath and needs neither.
Steps
Send every request through the Host, from the Desktop composer or as a tool
screenshot, never by calling the upstream directly. Use a fresh 64-bit
hexadecimal challenge per image, rendered only into the pixels. The generators
in desktop-app/test/e2e-playwright/codexImageChallenge.ts
(challengeImage, paddedChallengeImage, challengeImageAt) produce such
images; the filename and the prompt must not carry the answer.
- PNG challenge. One small PNG. Ask the model for the hexadecimal code in the image.
- JPEG challenge. The same with a JPEG.
- Order. One user message of text, image, text, where the answer depends on the order (for example: "the code in the image, then the word after it").
- Limits. The image budget (20 images, 20 MiB decoded) is checked per
request, and a request carries the images of the current user message plus
every tool screenshot taken earlier in the same turn. Earlier turns are
rebuilt as text (
buildMessageHistoryinmcp-host/src/core/conversation/conversation.ts), so a case never carries a previous case's images. Keep each case to one turn that adds only its own images:- one image padded close to the ingress cap of 16 MiB decoded from the composer, and one tool screenshot close to 20 MiB decoded, the contract's per-image limit, which only tool screenshots can reach;
- 20 small images in one request, each with its own challenge;
- one PNG whose long side is 9000 px. The contract sets no pixel limit, so this case measures whether the upstream does.
Expected
- Steps 1 to 3: the answer contains the challenge (hexadecimal case ignored), the turn ends without an error, no tool step substitutes OCR or file loading for direct image input, and no fallback badge appears.
- Step 4: every case is accepted by Evenfire, since each is inside the contract. Whatever the upstream answers is the measurement. An upstream refusal is a limit mismatch between the xAI documentation and the subscription endpoint. Report it, and change the contract only in a new pull request that cites the recorded status.
- A local refusal in any step (
attachment_too_large,invalid_request,image_source_invalid, HTTP 413) is a defect in Evenfire, not an upstream measurement.
Recording
For each step record:
- the commit, the profile, the model id and the date;
- the image format, the decoded size in bytes, the dimensions and the image count;
- the upstream HTTP status and, on failure, the bounded
upstreamHintfrom the proxy'sgrok_upstream_httplog line and the error code from itsgrok_proxy_attempt_finishedline; - whether the answer contained the challenge.
Never record tickets, tokens, grant credentials or image bytes. Put the results in the pull request or issue that closes the measurement, and update the "unmeasured" statement in the transport contract only with that evidence.