Skip to content

Speakr is a personal, self-hosted web application designed for transcribing audio recordings

Python

Browser smoke test: every page loads in headless Chromium with a working header (no 4xx, no JS errors, no template text, user menu opens); runs from source in test.yml and against the lite image in docker-publish.yml before anything is pushed

Murtaza Nasir02b540f12319
diff --git a/.github/workflows/docker-publish.yml b/.github/workflows/docker-publish.yml
index f367d4fe..ba23d2aa 100644
--- a/.github/workflows/docker-publish.yml
+++ b/.github/workflows/docker-publish.yml
@@ -60,9 +60,71 @@ jobs:
TEST_DATABASE_URI: postgresql://speakr:speakr@localhost:5432/speakr_test
run: pytest tests/ -q --cov=src --cov-report=term-missing --cov-fail-under=60
- build:
+ smoke-lite:
+ # Build the lite image for amd64 into the local daemon and open every page
+ # of it in a browser (tests/e2e/test_browser_smoke.py) before anything is
+ # pushed. The build jobs below wait for this job, so a broken image is
+ # never published. It shares the GitHub Actions build cache with them.
needs: test
runs-on: ubuntu-latest
+
+ steps:
+ - name: Free disk space
+ run: |
+ sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache
+ sudo docker image prune -af
+ df -h /
+
+ - name: Checkout repository
+ uses: actions/checkout@v5
+
+ - name: Set up Docker Buildx
+ uses: docker/setup-buildx-action@v3
+
+ - name: Build lite image (amd64, local)
+ uses: docker/build-push-action@v6
+ with:
+ context: .
+ platforms: linux/amd64
+ load: true
+ push: false
+ tags: speakr:smoke
+ cache-from: type=gha
+ cache-to: type=gha,mode=max
+ build-args: |
+ PRODUCTION=1
+ LIGHTWEIGHT=1
+
+ - name: Start the image with the example settings
+ run: |
+ python3 tests/e2e/smoke_env.py > smoke.env
+ docker run -d --name speakr-smoke -p 8899:8899 --env-file smoke.env speakr:smoke
+ for i in $(seq 1 90); do curl -fs -o /dev/null http://127.0.0.1:8899/login && break; sleep 2; done
+ curl -fs -o /dev/null http://127.0.0.1:8899/login
+
+ - name: Set up Python
+ uses: actions/setup-python@v6
+ with:
+ python-version: '3.11'
+
+ - name: Install Playwright
+ run: |
+ pip install pytest playwright
+ python -m playwright install --with-deps chromium
+
+ - name: Browser smoke test against the image
+ env:
+ SPEAKR_BROWSER_SMOKE: '1'
+ SPEAKR_BASE_URL: http://127.0.0.1:8899
+ run: pytest tests/e2e/test_browser_smoke.py -q -p no:cacheprovider --noconftest -rs
+
+ - name: Container log
+ if: always()
+ run: docker logs speakr-smoke 2>&1 | tail -n 200 || true
+
+ build:
+ needs: [test, smoke-lite]
+ runs-on: ubuntu-latest
permissions:
contents: read
packages: write
@@ -125,7 +187,7 @@ jobs:
PRODUCTION=1
build-lite:
- needs: test
+ needs: [test, smoke-lite]
runs-on: ubuntu-latest
permissions:
contents: read
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 6bf31046..f975d745 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -118,6 +118,49 @@ jobs:
if: always()
run: tail -n 200 /tmp/whisperx.log || true
+ browser-smoke:
+ # Opens every page in headless Chromium and fails on failed requests,
+ # JavaScript errors, leftover template text, or a header whose user menu
+ # does not open (tests/e2e/test_browser_smoke.py). Vendor files are
+ # downloaded exactly as the Dockerfile does it, in production mode, so a
+ # reference to a file the image lacks fails here.
+ runs-on: ubuntu-latest
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v5
+
+ - name: Set up Python
+ uses: actions/setup-python@v6
+ with:
+ python-version: '3.11'
+ cache: pip
+
+ - name: Install dependencies
+ run: pip install -r requirements.txt pytest playwright
+
+ - name: Download vendor files as the image build does
+ run: PRODUCTION=1 python scripts/download_offline_deps.py
+
+ - name: Install Chromium
+ run: python -m playwright install --with-deps chromium
+
+ - name: Start Speakr (config/env.whisperx.example)
+ run: |
+ python tests/e2e/run_speakr.py --port 8899 --workdir /tmp/speakr-smoke > /tmp/speakr.log 2>&1 &
+ for i in $(seq 1 90); do curl -fs -o /dev/null http://127.0.0.1:8899/login && break; sleep 2; done
+ curl -fs -o /dev/null http://127.0.0.1:8899/login
+
+ - name: Browser smoke test
+ env:
+ SPEAKR_BROWSER_SMOKE: '1'
+ SPEAKR_BASE_URL: http://127.0.0.1:8899
+ run: pytest tests/e2e/test_browser_smoke.py -q -p no:cacheprovider --noconftest -rs
+
+ - name: Speakr log
+ if: always()
+ run: tail -n 200 /tmp/speakr.log || true
+
js-tests:
runs-on: ubuntu-latest
diff --git a/tests/e2e/README.md b/tests/e2e/README.md
index d67283ec..d2bdfe6f 100644
--- a/tests/e2e/README.md
+++ b/tests/e2e/README.md
@@ -27,3 +27,24 @@ To test the service setup behind #409 (no `PRELOAD_MODEL` in the service's
`.env`), start the stub server with `--set PRELOAD_MODEL=`.
`KEEP_E2E_LOGS=1` keeps Speakr's log and database after the run.
+
+## Browser smoke test
+
+`test_browser_smoke.py` logs in through the login form and opens `/`, `/account`, `/admin`, `/inquire` and a recording (when one exists) in headless Chromium. It fails on any same-origin response of 400 or higher, any JavaScript error, template text left in the page (`${` or `{{`), or a header whose user menu does not open with its entries. The API-level tests never render a page; this is the check for regressions such as a missing vendor file or a template that Vue cannot compile.
+
+It runs in two places:
+
+- `test.yml`, job `browser-smoke`: Speakr from source via `run_speakr.py`, after `PRODUCTION=1 python scripts/download_offline_deps.py`, exactly as the image build downloads the vendor files.
+- `docker-publish.yml`, job `smoke-lite`: the lite image, built for amd64 and started with `smoke_env.py`'s settings, before either image is pushed.
+
+By hand (Speakr running at the URL, with Playwright installed):
+
+```bash
+python tests/e2e/smoke_env.py > smoke.env # for docker run --env-file
+python tests/e2e/run_speakr.py --port 8899 & # or: from source
+SPEAKR_BROWSER_SMOKE=1 SPEAKR_BASE_URL=http://127.0.0.1:8899 \
+ pytest tests/e2e/test_browser_smoke.py -q --noconftest -rs
+```
+
+`--noconftest` keeps the repository's `conftest.py`, which imports the app, out of a run that only needs Playwright.
+
diff --git a/tests/e2e/run_speakr.py b/tests/e2e/run_speakr.py
new file mode 100644
index 00000000..a40ad1be
--- /dev/null
+++ b/tests/e2e/run_speakr.py
@@ -0,0 +1,54 @@
+"""Start Speakr from source the way the image runs it, for the browser smoke test.
+
+ python tests/e2e/run_speakr.py --port 8899 --workdir /tmp/speakr-smoke
+
+Settings come from smoke_env.py (config/env.whisperx.example plus a few
+test changes). Like docker-entrypoint.sh, the database and the example's
+admin account are created first; gunicorn then runs in the foreground until
+it is stopped. Run it in the background and wait for /login to answer.
+"""
+
+import argparse
+import os
+import subprocess
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+ROOT = os.path.dirname(os.path.dirname(HERE))
+sys.path.insert(0, HERE)
+
+from smoke_env import smoke_settings # noqa: E402
+
+
+def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--port", type=int, default=8899)
+ parser.add_argument("--host", default="127.0.0.1")
+ parser.add_argument("--workdir", default="/tmp/speakr-smoke")
+ args = parser.parse_args()
+
+ work = args.workdir
+ for d in ("uploads", "exports", "auto-process", "instance"):
+ os.makedirs(os.path.join(work, d), exist_ok=True)
+ env = {k: os.environ[k] for k in ("PATH", "HOME", "LANG", "LC_ALL", "TMPDIR") if k in os.environ}
+ env.update(smoke_settings())
+ env.update({
+ "SQLALCHEMY_DATABASE_URI": f"sqlite:///{work}/instance/transcriptions.db",
+ "UPLOAD_FOLDER": f"{work}/uploads",
+ "AUTO_EXPORT_DIR": f"{work}/exports",
+ "AUTO_PROCESS_WATCH_DIR": f"{work}/auto-process",
+ "PYTHONUNBUFFERED": "1",
+ })
+
+ def run(cmd):
+ subprocess.run(cmd, cwd=ROOT, env=env, check=True)
+
+ run([sys.executable, "-c", "from src.app import app, db; app.app_context().push(); db.create_all()"])
+ run([sys.executable, "scripts/docker_create_admin.py"])
+ os.execvpe(sys.executable, [sys.executable, "-m", "gunicorn", "--workers", "2", "--worker-class", "gthread",
+ "--threads", "4", "--bind", f"{args.host}:{args.port}", "--timeout", "600",
+ "--chdir", ROOT, "src.app:app"], env)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tests/e2e/smoke_env.py b/tests/e2e/smoke_env.py
new file mode 100644
index 00000000..317dd100
--- /dev/null
+++ b/tests/e2e/smoke_env.py
@@ -0,0 +1,50 @@
+"""Settings for the browser smoke test: config/env.whisperx.example plus the
+few changes a page-load test needs.
+
+Used two ways:
+- ``python tests/e2e/smoke_env.py > smoke.env`` writes a file for
+ ``docker run --env-file`` (the image's entrypoint then creates the database
+ and the example's admin account);
+- ``run_speakr.py`` imports ``smoke_settings()`` to start Speakr from source.
+
+Changes to the example, and why:
+- DISABLE_VOICE_EMBEDDING_CHECK=true: no ASR service runs in this test, and
+ the check would only log a warning about it.
+- ENABLE_INQUIRE_MODE=true: so the Inquire page is loaded as well.
+- SECRET_KEY: a fixed test value.
+"""
+
+import os
+import sys
+
+ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+EXAMPLE = os.path.join(ROOT, "config", "env.whisperx.example")
+
+OVERRIDES = {
+ "DISABLE_VOICE_EMBEDDING_CHECK": "true",
+ "ENABLE_INQUIRE_MODE": "true",
+ "SECRET_KEY": "browser-smoke-test",
+}
+
+
+def read_example(path=EXAMPLE):
+ values = {}
+ with open(path, encoding="utf-8") as f:
+ for line in f:
+ line = line.strip()
+ if not line or line.startswith("#") or "=" not in line:
+ continue
+ key, value = line.split("=", 1)
+ values[key.strip()] = value.split(" #")[0].strip().strip('"').strip("'")
+ return values
+
+
+def smoke_settings():
+ values = read_example()
+ values.update(OVERRIDES)
+ return values
+
+
+if __name__ == "__main__":
+ for key, value in smoke_settings().items():
+ sys.stdout.write(f"{key}={value}\n")
diff --git a/tests/e2e/test_browser_smoke.py b/tests/e2e/test_browser_smoke.py
new file mode 100644
index 00000000..358571c6
--- /dev/null
+++ b/tests/e2e/test_browser_smoke.py
@@ -0,0 +1,179 @@
+"""Browser smoke test: every page loads and its header works, in a real browser.
+
+The API-level tests never render a page, so two header regressions reached a
+release: a Vue template error (an element with both v-text and fallback text)
+that left the header blank, and a request for vendor/js/vue.global.prod.js, a
+file the image does not contain, which left the settings-page headers empty.
+This test logs in through the login form, opens each page in headless
+Chromium, and fails on:
+
+- any same-origin response with status 400 or higher;
+- any JavaScript error (uncaught exception or console error);
+- template text left in the page (``${`` or ``{{``);
+- a header that is not mounted, or a user menu that does not open with its
+ entries.
+
+It runs only with SPEAKR_BROWSER_SMOKE=1 against a running Speakr at
+SPEAKR_BASE_URL (the test.yml browser-smoke job starts one from source with
+tests/e2e/run_speakr.py; docker-publish.yml runs the built image before it is
+pushed). SPEAKR_SMOKE_USER / SPEAKR_SMOKE_PASSWORD default to the admin in
+config/env.whisperx.example.
+
+Console messages that are not failures (documented allowlist):
+- messages from browser extensions (chrome-extension://);
+- "Failed to load resource": the browser's echo of a failed response, which
+ is already reported with its URL by the response check.
+"""
+
+import os
+import re
+import sys
+
+import pytest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+
+pytestmark = pytest.mark.skipif(os.environ.get("SPEAKR_BROWSER_SMOKE") != "1",
+ reason="browser smoke test; set SPEAKR_BROWSER_SMOKE=1 with Speakr running")
+
+playwright_api = pytest.importorskip("playwright.sync_api") if os.environ.get("SPEAKR_BROWSER_SMOKE") == "1" else None
+
+from smoke_env import read_example # noqa: E402
+
+BASE = os.environ.get("SPEAKR_BASE_URL", "http://127.0.0.1:8899").rstrip("/")
+_example = read_example()
+USER = os.environ.get("SPEAKR_SMOKE_USER", _example.get("ADMIN_EMAIL", "[email protected]"))
+PASSWORD = os.environ.get("SPEAKR_SMOKE_PASSWORD", _example.get("ADMIN_PASSWORD", "changeme"))
+
+MENU_ENTRIES = ("Notifications", "Settings", "Shared Transcripts", "Sign Out")
+IGNORED_CONSOLE = (re.compile(r"chrome-extension://"), re.compile(r"Failed to load resource"))
+
+
+class PageProblems:
+ """Collects failed responses and JavaScript errors for the current page."""
+
+ def __init__(self, page):
+ self.failed, self.errors = [], []
+ page.on("response", self._response)
+ page.on("pageerror", lambda exc: self.errors.append(f"uncaught: {exc}"))
+ page.on("console", self._console)
+
+ def _response(self, response):
+ if response.url.startswith(BASE) and response.status >= 400:
+ self.failed.append(f"{response.status} {response.url}")
+
+ def _console(self, msg):
+ if msg.type != "error":
+ return
+ text = msg.text
+ location = (msg.location or {}).get("url", "")
+ if any(p.search(text) or p.search(location) for p in IGNORED_CONSOLE):
+ return
+ self.errors.append(f"console: {text}")
+
+ def reset(self):
+ self.failed.clear()
+ self.errors.clear()
+
+
[email protected](scope="module")
+def browser_page():
+ with playwright_api.sync_playwright() as p:
+ browser = p.chromium.launch()
+ context = browser.new_context(viewport={"width": 1400, "height": 900}, locale="en-US")
+ page = context.new_page()
+ problems = PageProblems(page)
+ page.goto(f"{BASE}/login")
+ page.fill("input[name=email]", USER)
+ page.fill("input[name=password]", PASSWORD)
+ page.locator("form:has(input[name=password]) [type=submit]:visible").first.click()
+ page.wait_for_load_state("load")
+ assert "/login" not in page.url, f"could not log in as {USER}"
+ problems.reset()
+ yield page, problems
+ context.close()
+ browser.close()
+
+
+def _open(page, problems, path):
+ # Leave the previous page first: its requests still in flight are
+ # aborted on unload and logged as "Failed to fetch", which belongs to
+ # that page, not the one being checked.
+ page.goto("about:blank")
+ page.wait_for_timeout(300)
+ problems.reset()
+ response = page.goto(f"{BASE}{path}")
+ page.wait_for_load_state("load")
+ # Headers mount after the page's scripts run; give polling apps a moment.
+ page.wait_for_timeout(2500)
+ return response
+
+
+def _check_common(page, problems, path):
+ body = page.inner_text("body")
+ leftovers = [m for m in ("${", "{{") if m in body]
+ assert not leftovers, f"{path}: template text left in the page: {leftovers}"
+ assert not problems.failed, f"{path}: failed requests: {problems.failed}"
+ assert not problems.errors, f"{path}: JavaScript errors: {problems.errors}"
+
+
+def _check_user_menu(page, path):
+ toggle = page.locator("[data-user-menu-toggle]:visible").first
+ assert toggle.count() == 1, f"{path}: no user menu button in the header"
+ toggle.click()
+ menu = page.locator("[data-user-menu-dropdown]:visible").first
+ menu.wait_for(state="visible", timeout=5000)
+ text = menu.inner_text()
+ missing = [entry for entry in MENU_ENTRIES if entry not in text]
+ assert not missing, f"{path}: user menu is missing {missing}"
+ page.keyboard.press("Escape")
+ page.mouse.click(5, 400)
+
+
+def test_main_view(browser_page):
+ page, problems = browser_page
+ _open(page, problems, "/")
+ assert page.locator("[data-v-app]").count() >= 1, "/: the app did not mount"
+ _check_user_menu(page, "/")
+ _check_common(page, problems, "/")
+
+
[email protected]("path", ["/account", "/admin"])
+def test_settings_pages(browser_page, path):
+ page, problems = browser_page
+ response = _open(page, problems, path)
+ assert response is not None and response.status == 200, f"{path}: HTTP {response and response.status}"
+ header = page.locator("#global-header")
+ assert header.count() == 1, f"{path}: no shared page header"
+ assert header.get_attribute("data-v-app") is not None, f"{path}: the page header did not mount"
+ label = header.locator(".t-meta").first.inner_text().strip()
+ assert label, f"{path}: the page label in the header is empty"
+ _check_user_menu(page, path)
+ if path == "/account":
+ tabs = page.locator(".tabs button:visible, .tabs a:visible")
+ assert tabs.count() >= 3, "/account: the tab strip did not render"
+ _check_common(page, problems, path)
+
+
+def test_inquire_page(browser_page):
+ page, problems = browser_page
+ response = _open(page, problems, "/inquire")
+ if response is None or response.status != 200 or "/inquire" not in page.url:
+ pytest.skip("Inquire mode is not enabled")
+ problems.failed[:] = [f for f in problems.failed if "/inquire " not in f]
+ assert page.locator("#inquire-app[data-v-app]").count() == 1, "/inquire: the app did not mount"
+ _check_user_menu(page, "/inquire")
+ _check_common(page, problems, "/inquire")
+
+
+def test_a_recording_detail_view(browser_page):
+ page, problems = browser_page
+ listing = page.request.get(f"{BASE}/api/recordings?per_page=1")
+ items = listing.json().get("recordings", []) if listing.ok else []
+ if not items:
+ pytest.skip("no recordings on this instance")
+ path = f"/recordings/{items[0]['id']}"
+ _open(page, problems, path)
+ _check_user_menu(page, path)
+ _check_common(page, problems, path)