Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript
5.6 KB
Raw Blame

Desktop image-capability fixture

This optional, unpublished image derives from the reviewed MCP Host image. It simulates only the external ZAI chat-completions peer. Electron, sign-in, IPC, RPC, the catalog, model selection, persistence, the Host and its SDK serializer remain real. No paid inference is made by this lane.

The fixture authenticates its own generated key by a SHA-256 digest. The key is held in a run-owned Kubernetes Secret; only the digest enters the fixture configuration. Responses to image requests are computed from the received PNG pixels. Evidence records model, image digest and counters, never bodies or authorization data. Other external origins are refused.

The pinned OpenAI SDK uses node-fetch by default. The fixture entrypoint selects the SDK's web shim before starting the real CommonJS Host, and refuses startup unless the SDK binds to the isolated fetch boundary. sdk.test.mjs checks the compiled ZAI provider with the PNG generated by the browser journey. The preload runs with Node's --import; the production main.js remains the main program so its normal startup guard and crash handlers execute.

Documents (issue #678)

The same fixture also answers the two provider turns of a document attachment. The Host never puts a file's text in the prompt: it announces the file in the turn context and the model reads it through the clerum__attachment_read tool. When that tool is offered and the last user message announces an attached_file, the fixture first replies with a tool call for it. After the tool result arrives, it replies DOCUMENT_FIXTURE_SHA256: plus the first 16 hex characters of the SHA-256 of the text in that result. The prompt alone can never produce that answer.

The ledger records document-read-requested (no digest yet, with documentByteLength taken from the bytes= field the Host wrote on the attached_file line) and document-answer (with documentSha256, never the text). An attached_file line without a positive bytes=, malformed, non-text or repeated tool results and streamed document turns are refused and counted in documentFailures. Image rows keep their previous shape.

sdk.test.mjs drives both turns through the compiled Host: the turn-context block, the safety wrapper and the ZAI provider are the production ones.

The browser journey has its own lane, selected by the runner:

make minikube-build-image-capabilities-fixture
make minikube-t2
make minikube-run-document-upload

make minikube-run-document-upload is minikube-run-image-capabilities with IMAGE_CAPABILITIES_LANE=document-upload; the config is desktop-app/test/e2e-playwright/playwright.document-upload.config.ts. It attaches a text file through Add context -> Upload Files, waits for a ready file chip, sends, and requires the digest of the attached bytes in the answer, one document-read-requested row, one document-answer row carrying that digest, no image row and no documentFailures.

Local execution

Use Node 24 and Bash 5, a clean development branch based on current origin/dev, and the profile/context resolved by the primary checkout's branch.mk. Both MINIKUBE_PROFILE and CONTROL_API_REAL_PG_CONTEXT must name that owned profile. Run runtime commands outside the native sandbox. Keep its registered UI forwards alive in a host session. Do not use a shared profile or a remote recorder config.

make minikube-build-image-capabilities-fixture
make minikube-t2
make minikube-run-image-capabilities

Acquisition is separate from the journey and records the exact source revision, image IDs and derived-base binding under the normal mutation lease. The runner requires those identities, the owned local ports and one isolated chatllm Host. It uses the existing admin-login resolver and the real admin API to establish catalog preconditions. Login and all chat interactions under test happen through the visible Desktop UI. Read-only IPC observations after the response correlate both rendered messages with the durable Host task; they never drive the flow.

The runner journals restoration before mutation. It restores the original model, allowlist, configuration and production image, checks resource ownership, and removes only its own key and catalog rows. On SIGINT or SIGTERM it restores before exiting (after the current kubectl/npm call returns). If the process was killed or restoration failed, pass the printed canonical run directory to:

make minikube-restore-image-capabilities IMAGE_CAPABILITIES_RUN_DIR=<run directory>

After pod replacements, revalidate the registered forwards through the branch helper. Close runtime certification with make minikube-t2-runtime only when T0/T1 are already attested on the same HEAD. T2 refuses a remaining fixture image, configuration or marker. The browser result and the production-runtime verdict are separate evidence lanes. A paid-provider smoke remains separately opt-in.

Verification

node --test scripts/e2e/image-capabilities-fixture.test.mjs
node --test tests/e2e/fixtures/image-capabilities/provider.test.mjs
node --test tests/e2e/fixtures/image-capabilities/sdk.test.mjs
bash scripts/tests/test-minikube-image-capability-fixture.sh

Build mcp-host before the SDK integration test. The browser config is desktop-app/test/e2e-playwright/playwright.image-capabilities.config.ts; it does not load global env files. The journey verifies unknown capability, supported image delivery, retention while switching to text-only, blocked Enter/send, and text delivery after explicit image removal. Missing pixels, a wrong model, duplicate visual delivery, stale evidence, or missing durable-task correlation cannot satisfy its positive oracle.