Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript
9.3 KB
Raw Blame

Desktop App — headful journey tests

Status snapshot of the optional Playwright QA recorder journeys for the Evenfire Desktop App. These drive the real Electron window headfully (visible UI, recorded WebM + PNG) against a live example-dev port-forward. They are separate from the CI Playwright suite and never run in CI by default.

Recorder runs are local and opt-in. Configure the exact QA identity in .env.qa-recorder, use only approved development endpoints, and enable a QA_RECORDER_CONFIRM_* flag only for the side effect you intend to exercise. Recordings stay under the git-ignored .local-notes/qa-recorder/ directory and must be reviewed before sharing.

Coverage

Every Desktop App user journey has at least one headful spec. Each uses the exact QA identity from .env.qa-recorder (no first-available fallback), shares launch / login / proof plumbing in qa-recorder-helpers.ts, and records a WebM + PNG per test under .local-notes/qa-recorder/runs/desktop-app/ (git-ignored).

Journey Spec Tests Confirm flag Status
Auth & session qa-recorder-auth.spec.ts 3 — sign-in screen, authenticated shell, logout — ✅
Navigation qa-recorder-navigation.spec.ts 3 — primary sidebar, footer settings menu, resources pages — ✅
Agents qa-recorder-agents.spec.ts 3 — fleet, chat workspace, workspace routes — ✅
Connectors qa-recorder-connectors.spec.ts 1 — MCP/connector health inventory — ✅
Shared files qa-recorder-shared-files.spec.ts 1 — filesystem + directory browser — ✅
Plugins & workflows qa-recorder-plugins.spec.ts 1 — inventory, detail, runs/artifacts (+optional trigger) QA_RECORDER_CONFIRM_MUTATIONS (trigger only) ✅
Apps qa-recorder-apps.spec.ts 1 — catalog + embedded session — ✅
Settings qa-recorder-settings.spec.ts 2 — appearance/notifications, information/configuration (+optional endpoint switch) QA_RECORDER_CONFIRM_MUTATIONS (switch only) ✅
Inbox & search qa-recorder-inbox-search.spec.ts 2 — inbox, global search — ✅
Chat qa-recorder-chat.spec.ts 1 — composer, thread, task progress QA_RECORDER_CONFIRM_CHAT (required) ✅
Settings + chat smoke qa-recorder-settings-chat.spec.ts 1 — Settings tabs + one chat message QA_RECORDER_CONFIRM_CHAT (required) ✅
Image capabilities qa-recorder-image-capabilities.spec.ts 2 — capability gate on a text-only model, fixture-backed image answer QA_RECORDER_CONFIRM_CHAT (required) ✅
Model selector qa-recorder-model-selector.spec.ts 1 — a multi-model catalog lists rows without capability tags — ✅
Document upload qa-recorder-document-upload.spec.ts 1 — attached text file read through the read tool (fixture lane only); a small file is answered with its whole-file digest, and a 6 MiB file with the digest of the first page the Host delivered (byteRange and truncated: true from the fixture ledger) IMAGE_CAPABILITIES_RUN_ID (runner-set) ✅

Totals: 14 specs, 23 tests. Read-only journeys need no confirmation flag; every journey still calls the loopback health guard on both API URLs — except against a non-loopback target, where QA_RECORDER_ALLOW_REMOTE=1 skips the precheck, so a wrong URL surfaces as an Electron timeout instead of a fast refusal. Mutating / paid steps (chat message, workflow trigger, endpoint switch) run only under their QA_RECORDER_CONFIRM_* flag and are otherwise skipped — the read-only part of the journey always runs.

Run

cd desktop-app
# dev Evenfire must be closed (process-wide single-instance lock)
# keep control-api / external-rest-api / rpc-proxy port-forwarded to loopback
# (standard single minikube profile: make -C .. minikube-pf-desktop,
#  fixed ports 8090 / 8091 / 8094)

npm run qa:recorder:navigation   # one journey
npm run qa:recorder:all          # all 14 specs / 23 tests

On a branch-owned profile the forwards use profile-owned ports instead of 8091 / 8094. Hold them on the host and point the recorder at them:

MINIKUBE_PROFILE=<owned-profile> \
  make -f scripts/minikube-profiles/branch.mk branch-profile-pf    # from the repo root
MINIKUBE_PROFILE=<owned-profile> \
  make -f scripts/minikube-profiles/branch.mk branch-profile-info  # prints the ports

cd desktop-app
EXTERNAL_REST_API_BASE_URL=http://127.0.0.1:<external-rest-api-port> \
RPC_PROXY_BASE_URL=http://127.0.0.1:<rpc-proxy-port> \
  npm run qa:recorder:navigation

qa:recorder:all includes qa-recorder-model-selector.spec.ts, which fails loudly against a catalog of fewer than two models. Run it against an environment whose catalog has several, or run the other journeys individually.

Proof artifacts (local only, git-ignored)

find .local-notes/qa-recorder/runs/desktop-app -name '*.webm'   # WebM recordings
find .local-notes/qa-recorder/runs/desktop-app -name '*.png'    # full-page screenshots

Videos may contain emails, agent names, endpoint URLs, prompts, or model responses and must stay local unless reviewed and shared intentionally.

Last verified

2026-07-17 against example-dev (GKE) port-forwards — the then-current suite passed with WebM + PNG proof for every test. This is a point-in-time snapshot; re-run npm run qa:recorder:all to refresh.