Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

feat(qa-recorder): add webm-to-mp4 evidence conversion script

Miguelon Peluchonb72e4ca2dd5c
diff --git a/control-ui/package-lock.json b/control-ui/package-lock.json
index 459efda58..e391146bb 100644
--- a/control-ui/package-lock.json
+++ b/control-ui/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "clerum-control-ui",
- "version": "0.1.466",
+ "version": "0.1.467",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "clerum-control-ui",
- "version": "0.1.466",
+ "version": "0.1.467",
"dependencies": {
"@clerum/display-field": "file:../packages/display-field",
"@clerum/llm-providers": "file:../packages/llm-providers",
diff --git a/control-ui/package.json b/control-ui/package.json
index e64bc61ba..ffaff7c7d 100644
--- a/control-ui/package.json
+++ b/control-ui/package.json
@@ -1,6 +1,6 @@
{
"name": "clerum-control-ui",
- "version": "0.1.466",
+ "version": "0.1.467",
"private": true,
"scripts": {
"dev": "NEXT_IGNORE_INCORRECT_LOCKFILE=1 next dev",
@@ -55,6 +55,7 @@
"qa:recorder:member-removed-access": "PLAYWRIGHT_BROWSERS_PATH=../.local-notes/qa-recorder/browsers playwright test --config=playwright.qa-recorder.config.ts qa-recorder-member-removed-access.spec.ts",
"qa:recorder:connectors-search-agent": "PLAYWRIGHT_BROWSERS_PATH=../.local-notes/qa-recorder/browsers playwright test --config=playwright.qa-recorder.config.ts qa-recorder-connectors-search-agent.spec.ts",
"qa:recorder:all": "PLAYWRIGHT_BROWSERS_PATH=../.local-notes/qa-recorder/browsers playwright test --config=playwright.qa-recorder.config.ts",
+ "qa:recorder:convert-mp4": "bash ../scripts/qa-recorder/convert-videos-mp4.sh",
"test": "vitest run",
"test:watch": "vitest"
},
diff --git a/docs/features/control-ui-context-removal.md b/docs/features/control-ui-context-removal.md
index 50a5197a9..b8746aa87 100644
--- a/docs/features/control-ui-context-removal.md
+++ b/docs/features/control-ui-context-removal.md
@@ -236,7 +236,7 @@ Tests updated in the same commit: `HostTable.test.tsx` (hover card now asserts a
| `qa:recorder:member-removed-access` | "Removed access" section for a mapping whose scope was deleted | yes |
| `qa:recorder:connectors-search-agent` | Connectors search filters by agent display name | yes |
- Prerequisites for video evidence: `npm run qa:recorder:install` (once), `make minikube-pf-all`, and a repo-root `.env.qa-recorder` copied from the committed `.env.qa-recorder.example` containing the admin identity (`E2E_ADMIN_USER`/`E2E_ADMIN_PASSWORD`) and `QA_RECORDER_CONFIRM_MUTATIONS=1` for the mutating journeys. Journeys follow the recorder contract (`docs/testing/optional-playwright-qa-recorder.md`): loopback-only target guard, explicit opt-in before any write, and full API cleanup in `finally`.
+ Prerequisites for video evidence: `npm run qa:recorder:install` (once), `make minikube-pf-all`, and a repo-root `.env.qa-recorder` copied from the committed `.env.qa-recorder.example` containing the admin identity (`E2E_ADMIN_USER`/`E2E_ADMIN_PASSWORD`) and `QA_RECORDER_CONFIRM_MUTATIONS=1` for the mutating journeys. Journeys follow the recorder contract (`docs/testing/optional-playwright-qa-recorder.md`): loopback-only target guard, explicit opt-in before any write, and full API cleanup in `finally`. Output per journey: a WebM video (1280×720, Playwright native format) + full-page PNGs under `.local-notes/qa-recorder/runs/control-ui/`. For MP4 evidence run `npm run qa:recorder:convert-mp4` (`scripts/qa-recorder/convert-videos-mp4.sh [--replace]` — H.264 + faststart via ffmpeg; verified working with ffmpeg 7.x).
- Verification per phase: `cd control-ui && npx tsc --noEmit && npm test` (no lint script exists in control-ui; typecheck via `tsc --noEmit`).
## 10. Progress log (append-only)
@@ -255,6 +255,7 @@ Tests updated in the same commit: `HostTable.test.tsx` (hover card now asserts a
| 2026-08-27 | e2e | Built the full e2e regression catalog for the §12 flows: 15 new specs + `contexts.spec.ts` extended with the real-slug resolver case (112 tests / 24 files listed for the control-ui project). Extended `helpers/api-client.ts` with context/mcp-server/user-team-contexts/team/budget seeding + teardown helpers (all against the unchanged wire API). Added `tests/e2e/playwright/tsconfig.typecheck.json` so the package typechecks standalone (`npx tsc -p tsconfig.typecheck.json` → clean; the shipped tsconfig lacks node types + a covering rootDir). No specs executed here (no cluster) — they run under the normal `make test-playwright-control-ui` / T2 lane. control-ui unit suite re-verified 1700/1700. No backend changes. |
| 2026-08-27 | e2e | Built the QA-recorder video-evidence journeys: 11 new `qa-recorder-*` specs + `qa-recorder-connector-edit` extended with the legacy-tab redirect, each with a dedicated `qa:recorder:<name>` npm script (62 tests / 45 files listed under the recorder config). Journeys follow the recorder contract (loopback guard, `QA_RECORDER_CONFIRM_MUTATIONS` opt-in for mutating journeys, `screenshotAndLog` at every step, full API cleanup in `finally`). Committed the previously-missing `.env.qa-recorder.example` (referenced by the recorder doc but never tracked) with the exact keys the journeys need. Verification: control-ui `tsc --noEmit` clean, recorder `--list` clean, unit suite 1700/1700. No backend changes. |
| 2026-08-27 | e2e | Added 3 gap journeys for error/degraded paths: `agent-connectors-conflict` (out-of-band context PUT stales the page → 409 banner "This agent's connectors changed since they were loaded." → reload + retry succeeds), `member-removed-access` (mapping whose scope was deleted renders under "Removed access" with the agent label + Deleted marker), `connectors-search-agent` (search-by-agent-name filtering + empty state). Recorder catalog now 65 tests / 48 files listed; `tsc` clean. **Execution status: no Playwright spec (main or recorder) has been executed in this lane — no runnable cluster exists in the dev session** (both minikube profiles Stopped, no branch.mk ownership record → fail-closed per repo rules; the live `:3000` server belongs to the unrelated `feat/desktop-home` worktree and `:8090` is a dead forward). Verified-only: unit suite 1700/1700 (three clean full runs; one intermediate run under heavy parallel load showed 4 flakes that did not reproduce), `tsc --noEmit` everywhere, both Playwright configs `--list` clean, `next build` green. First cluster run should budget for small selector polish. |
+| 2026-08-27 | e2e | Video-format answer + MP4 pipeline: Playwright/Chromium records **WebM** natively (not MP4) — `qa:recorder:convert-mp4` added (`scripts/qa-recorder/convert-videos-mp4.sh`, ffmpeg H.264 + faststart, `--replace` to drop originals; round-trip verified against a synthetic clip). Recorder doc + §9 updated. **No video evidence exists yet anywhere in `.local-notes/` — zero journeys have been executed in this lane; videos are produced only when a journey runs against a live cluster.** |
## 11. Post-implementation audit (2026-08-27)
diff --git a/docs/testing/optional-playwright-qa-recorder.md b/docs/testing/optional-playwright-qa-recorder.md
index 6cbfbe647..c32cc4ab2 100644
--- a/docs/testing/optional-playwright-qa-recorder.md
+++ b/docs/testing/optional-playwright-qa-recorder.md
@@ -43,7 +43,10 @@ the exact admin identity from `.env.qa-recorder`, and records a WebM + PNG under
needed); only the `agent-create` journey above mutates. See
[control-ui-headful-journeys.md](./control-ui-headful-journeys.md) for the full
status table. Run one with `npm run qa:recorder:<journey>` or all with
-`npm run qa:recorder:all`.
+`npm run qa:recorder:all`. Videos are WebM (Playwright/Chromium native); convert
+the evidence bundle to H.264 MP4 with `npm run qa:recorder:convert-mp4`
+(`scripts/qa-recorder/convert-videos-mp4.sh`, `--replace` drops the WebM
+originals).
### Desktop App: Settings and chat
diff --git a/scripts/qa-recorder/convert-videos-mp4.sh b/scripts/qa-recorder/convert-videos-mp4.sh
new file mode 100755
index 000000000..dc32c2c4a
--- /dev/null
+++ b/scripts/qa-recorder/convert-videos-mp4.sh
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+# Convert QA-recorder journey videos from Playwright's native WebM to MP4
+# (H.264 + faststart) so evidence can be attached anywhere without codec
+# questions. Screenshots are already PNG and need no conversion.
+#
+# Usage:
+# scripts/qa-recorder/convert-videos-mp4.sh [--replace] [root]
+#
+# --replace delete the source .webm after a successful conversion
+# root recorder root (default: <repo>/.local-notes/qa-recorder;
+# honored from $QA_RECORDER_ROOT like the playwright config)
+set -euo pipefail
+
+replace=0
+root="${QA_RECORDER_ROOT:-}"
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --replace) replace=1 ;;
+ -h|--help)
+ sed -n '2,9p' "$0" | sed 's/^# \{0,1\}//'
+ exit 0
+ ;;
+ *)
+ if [ -n "$root" ]; then
+ echo "unexpected argument: $1" >&2
+ exit 2
+ fi
+ root="$1"
+ ;;
+ esac
+ shift
+done
+
+repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
+root="${root:-$repo_root/.local-notes/qa-recorder}"
+runs_dir="$root/runs"
+
+if ! command -v ffmpeg >/dev/null 2>&1; then
+ echo "ffmpeg is required for MP4 conversion but was not found on PATH." >&2
+ echo "Install ffmpeg (e.g. apt install ffmpeg) or keep the WebM originals." >&2
+ exit 1
+fi
+
+if [ ! -d "$runs_dir" ]; then
+ echo "No recorder runs found under $runs_dir — run a journey first:"
+ echo " cd control-ui && npm run qa:recorder:<journey>"
+ exit 0
+fi
+
+count=0
+failed=0
+while IFS= read -r -d '' webm; do
+ mp4="${webm%.webm}.mp4"
+ if ffmpeg -y -hide_banner -loglevel error -i "$webm" \
+ -c:v libx264 -pix_fmt yuv420p -movflags +faststart "$mp4"; then
+ echo "[mp4] $mp4"
+ count=$((count + 1))
+ if [ "$replace" -eq 1 ]; then
+ rm -- "$webm"
+ fi
+ else
+ echo "[fail] $webm" >&2
+ failed=$((failed + 1))
+ fi
+done < <(find "$runs_dir" -type f -name '*.webm' -print0)
+
+echo "converted $count video(s)$([ "$replace" -eq 1 ] && echo ', sources removed' || echo '')"
+[ "$failed" -eq 0 ] || exit 1