Speakr is a personal, self-hosted web application designed for transcribing audio recordings
Python
Browser smoke test: every page loads in headless Chromium with a working header (no 4xx, no JS errors, no template text, user menu opens); runs from source in test.yml and against the lite image in docker-publish.yml before anything is pushed
Murtaza Nasir02b540f12319
diff --git a/.github/workflows/docker-publish.yml b/.github/workflows/docker-publish.ymlindex f367d4fe..ba23d2aa 100644--- a/.github/workflows/docker-publish.yml+++ b/.github/workflows/docker-publish.yml@@ -60,9 +60,71 @@ jobs:TEST_DATABASE_URI: postgresql://speakr:speakr@localhost:5432/speakr_testrun: pytest tests/ -q --cov=src --cov-report=term-missing --cov-fail-under=60- build:+ smoke-lite:+ # Build the lite image for amd64 into the local daemon and open every page+ # of it in a browser (tests/e2e/test_browser_smoke.py) before anything is+ # pushed. The build jobs below wait for this job, so a broken image is+ # never published. It shares the GitHub Actions build cache with them.needs: testruns-on: ubuntu-latest++ steps:+ - name: Free disk space+ run: |+ sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache+ sudo docker image prune -af+ df -h /++ - name: Checkout repository+ uses: actions/checkout@v5++ - name: Set up Docker Buildx+ uses: docker/setup-buildx-action@v3++ - name: Build lite image (amd64, local)+ uses: docker/build-push-action@v6+ with:+ context: .+ platforms: linux/amd64+ load: true+ push: false+ tags: speakr:smoke+ cache-from: type=gha+ cache-to: type=gha,mode=max+ build-args: |+ PRODUCTION=1+ LIGHTWEIGHT=1++ - name: Start the image with the example settings+ run: |+ python3 tests/e2e/smoke_env.py > smoke.env+ docker run -d --name speakr-smoke -p 8899:8899 --env-file smoke.env speakr:smoke+ for i in $(seq 1 90); do curl -fs -o /dev/null http://127.0.0.1:8899/login && break; sleep 2; done+ curl -fs -o /dev/null http://127.0.0.1:8899/login++ - name: Set up Python+ uses: actions/setup-python@v6+ with:+ python-version: '3.11'++ - name: Install Playwright+ run: |+ pip install pytest playwright+ python -m playwright install --with-deps chromium++ - name: Browser smoke test against the image+ env:+ SPEAKR_BROWSER_SMOKE: '1'+ SPEAKR_BASE_URL: http://127.0.0.1:8899+ run: pytest tests/e2e/test_browser_smoke.py -q -p no:cacheprovider --noconftest -rs++ - name: Container log+ if: always()+ run: docker logs speakr-smoke 2>&1 | tail -n 200 || true++ build:+ needs: [test, smoke-lite]+ runs-on: ubuntu-latestpermissions:contents: readpackages: write@@ -125,7 +187,7 @@ jobs:PRODUCTION=1build-lite:- needs: test+ needs: [test, smoke-lite]runs-on: ubuntu-latestpermissions:contents: readdiff --git a/.github/workflows/test.yml b/.github/workflows/test.ymlindex 6bf31046..f975d745 100644--- a/.github/workflows/test.yml+++ b/.github/workflows/test.yml@@ -118,6 +118,49 @@ jobs:if: always()run: tail -n 200 /tmp/whisperx.log || true+ browser-smoke:+ # Opens every page in headless Chromium and fails on failed requests,+ # JavaScript errors, leftover template text, or a header whose user menu+ # does not open (tests/e2e/test_browser_smoke.py). Vendor files are+ # downloaded exactly as the Dockerfile does it, in production mode, so a+ # reference to a file the image lacks fails here.+ runs-on: ubuntu-latest++ steps:+ - name: Checkout repository+ uses: actions/checkout@v5++ - name: Set up Python+ uses: actions/setup-python@v6+ with:+ python-version: '3.11'+ cache: pip++ - name: Install dependencies+ run: pip install -r requirements.txt pytest playwright++ - name: Download vendor files as the image build does+ run: PRODUCTION=1 python scripts/download_offline_deps.py++ - name: Install Chromium+ run: python -m playwright install --with-deps chromium++ - name: Start Speakr (config/env.whisperx.example)+ run: |+ python tests/e2e/run_speakr.py --port 8899 --workdir /tmp/speakr-smoke > /tmp/speakr.log 2>&1 &+ for i in $(seq 1 90); do curl -fs -o /dev/null http://127.0.0.1:8899/login && break; sleep 2; done+ curl -fs -o /dev/null http://127.0.0.1:8899/login++ - name: Browser smoke test+ env:+ SPEAKR_BROWSER_SMOKE: '1'+ SPEAKR_BASE_URL: http://127.0.0.1:8899+ run: pytest tests/e2e/test_browser_smoke.py -q -p no:cacheprovider --noconftest -rs++ - name: Speakr log+ if: always()+ run: tail -n 200 /tmp/speakr.log || true+js-tests:runs-on: ubuntu-latestdiff --git a/tests/e2e/README.md b/tests/e2e/README.mdindex d67283ec..d2bdfe6f 100644--- a/tests/e2e/README.md+++ b/tests/e2e/README.md@@ -27,3 +27,24 @@ To test the service setup behind #409 (no `PRELOAD_MODEL` in the service's`.env`), start the stub server with `--set PRELOAD_MODEL=`.`KEEP_E2E_LOGS=1` keeps Speakr's log and database after the run.++## Browser smoke test++`test_browser_smoke.py` logs in through the login form and opens `/`, `/account`, `/admin`, `/inquire` and a recording (when one exists) in headless Chromium. It fails on any same-origin response of 400 or higher, any JavaScript error, template text left in the page (`${` or `{{`), or a header whose user menu does not open with its entries. The API-level tests never render a page; this is the check for regressions such as a missing vendor file or a template that Vue cannot compile.++It runs in two places:++- `test.yml`, job `browser-smoke`: Speakr from source via `run_speakr.py`, after `PRODUCTION=1 python scripts/download_offline_deps.py`, exactly as the image build downloads the vendor files.+- `docker-publish.yml`, job `smoke-lite`: the lite image, built for amd64 and started with `smoke_env.py`'s settings, before either image is pushed.++By hand (Speakr running at the URL, with Playwright installed):++```bash+python tests/e2e/smoke_env.py > smoke.env # for docker run --env-file+python tests/e2e/run_speakr.py --port 8899 & # or: from source+SPEAKR_BROWSER_SMOKE=1 SPEAKR_BASE_URL=http://127.0.0.1:8899 \+ pytest tests/e2e/test_browser_smoke.py -q --noconftest -rs+```++`--noconftest` keeps the repository's `conftest.py`, which imports the app, out of a run that only needs Playwright.+diff --git a/tests/e2e/run_speakr.py b/tests/e2e/run_speakr.pynew file mode 100644index 00000000..a40ad1be--- /dev/null+++ b/tests/e2e/run_speakr.py@@ -0,0 +1,54 @@+"""Start Speakr from source the way the image runs it, for the browser smoke test.++ python tests/e2e/run_speakr.py --port 8899 --workdir /tmp/speakr-smoke++Settings come from smoke_env.py (config/env.whisperx.example plus a few+test changes). Like docker-entrypoint.sh, the database and the example's+admin account are created first; gunicorn then runs in the foreground until+it is stopped. Run it in the background and wait for /login to answer.+"""++import argparse+import os+import subprocess+import sys++HERE = os.path.dirname(os.path.abspath(__file__))+ROOT = os.path.dirname(os.path.dirname(HERE))+sys.path.insert(0, HERE)++from smoke_env import smoke_settings # noqa: E402+++def main():+ parser = argparse.ArgumentParser()+ parser.add_argument("--port", type=int, default=8899)+ parser.add_argument("--host", default="127.0.0.1")+ parser.add_argument("--workdir", default="/tmp/speakr-smoke")+ args = parser.parse_args()++ work = args.workdir+ for d in ("uploads", "exports", "auto-process", "instance"):+ os.makedirs(os.path.join(work, d), exist_ok=True)+ env = {k: os.environ[k] for k in ("PATH", "HOME", "LANG", "LC_ALL", "TMPDIR") if k in os.environ}+ env.update(smoke_settings())+ env.update({+ "SQLALCHEMY_DATABASE_URI": f"sqlite:///{work}/instance/transcriptions.db",+ "UPLOAD_FOLDER": f"{work}/uploads",+ "AUTO_EXPORT_DIR": f"{work}/exports",+ "AUTO_PROCESS_WATCH_DIR": f"{work}/auto-process",+ "PYTHONUNBUFFERED": "1",+ })++ def run(cmd):+ subprocess.run(cmd, cwd=ROOT, env=env, check=True)++ run([sys.executable, "-c", "from src.app import app, db; app.app_context().push(); db.create_all()"])+ run([sys.executable, "scripts/docker_create_admin.py"])+ os.execvpe(sys.executable, [sys.executable, "-m", "gunicorn", "--workers", "2", "--worker-class", "gthread",+ "--threads", "4", "--bind", f"{args.host}:{args.port}", "--timeout", "600",+ "--chdir", ROOT, "src.app:app"], env)+++if __name__ == "__main__":+ main()diff --git a/tests/e2e/smoke_env.py b/tests/e2e/smoke_env.pynew file mode 100644index 00000000..317dd100--- /dev/null+++ b/tests/e2e/smoke_env.py@@ -0,0 +1,50 @@+"""Settings for the browser smoke test: config/env.whisperx.example plus the+few changes a page-load test needs.++Used two ways:+- ``python tests/e2e/smoke_env.py > smoke.env`` writes a file for+ ``docker run --env-file`` (the image's entrypoint then creates the database+ and the example's admin account);+- ``run_speakr.py`` imports ``smoke_settings()`` to start Speakr from source.++Changes to the example, and why:+- DISABLE_VOICE_EMBEDDING_CHECK=true: no ASR service runs in this test, and+ the check would only log a warning about it.+- ENABLE_INQUIRE_MODE=true: so the Inquire page is loaded as well.+- SECRET_KEY: a fixed test value.+"""++import os+import sys++ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))+EXAMPLE = os.path.join(ROOT, "config", "env.whisperx.example")++OVERRIDES = {+ "DISABLE_VOICE_EMBEDDING_CHECK": "true",+ "ENABLE_INQUIRE_MODE": "true",+ "SECRET_KEY": "browser-smoke-test",+}+++def read_example(path=EXAMPLE):+ values = {}+ with open(path, encoding="utf-8") as f:+ for line in f:+ line = line.strip()+ if not line or line.startswith("#") or "=" not in line:+ continue+ key, value = line.split("=", 1)+ values[key.strip()] = value.split(" #")[0].strip().strip('"').strip("'")+ return values+++def smoke_settings():+ values = read_example()+ values.update(OVERRIDES)+ return values+++if __name__ == "__main__":+ for key, value in smoke_settings().items():+ sys.stdout.write(f"{key}={value}\n")diff --git a/tests/e2e/test_browser_smoke.py b/tests/e2e/test_browser_smoke.pynew file mode 100644index 00000000..358571c6--- /dev/null+++ b/tests/e2e/test_browser_smoke.py@@ -0,0 +1,179 @@+"""Browser smoke test: every page loads and its header works, in a real browser.++The API-level tests never render a page, so two header regressions reached a+release: a Vue template error (an element with both v-text and fallback text)+that left the header blank, and a request for vendor/js/vue.global.prod.js, a+file the image does not contain, which left the settings-page headers empty.+This test logs in through the login form, opens each page in headless+Chromium, and fails on:++- any same-origin response with status 400 or higher;+- any JavaScript error (uncaught exception or console error);+- template text left in the page (``${`` or ``{{``);+- a header that is not mounted, or a user menu that does not open with its+ entries.++It runs only with SPEAKR_BROWSER_SMOKE=1 against a running Speakr at+SPEAKR_BASE_URL (the test.yml browser-smoke job starts one from source with+tests/e2e/run_speakr.py; docker-publish.yml runs the built image before it is+pushed). SPEAKR_SMOKE_USER / SPEAKR_SMOKE_PASSWORD default to the admin in+config/env.whisperx.example.++Console messages that are not failures (documented allowlist):+- messages from browser extensions (chrome-extension://);+- "Failed to load resource": the browser's echo of a failed response, which+ is already reported with its URL by the response check.+"""++import os+import re+import sys++import pytest++HERE = os.path.dirname(os.path.abspath(__file__))+sys.path.insert(0, HERE)++pytestmark = pytest.mark.skipif(os.environ.get("SPEAKR_BROWSER_SMOKE") != "1",+ reason="browser smoke test; set SPEAKR_BROWSER_SMOKE=1 with Speakr running")++playwright_api = pytest.importorskip("playwright.sync_api") if os.environ.get("SPEAKR_BROWSER_SMOKE") == "1" else None++from smoke_env import read_example # noqa: E402++BASE = os.environ.get("SPEAKR_BASE_URL", "http://127.0.0.1:8899").rstrip("/")+_example = read_example()+USER = os.environ.get("SPEAKR_SMOKE_USER", _example.get("ADMIN_EMAIL", "[email protected]"))+PASSWORD = os.environ.get("SPEAKR_SMOKE_PASSWORD", _example.get("ADMIN_PASSWORD", "changeme"))++MENU_ENTRIES = ("Notifications", "Settings", "Shared Transcripts", "Sign Out")+IGNORED_CONSOLE = (re.compile(r"chrome-extension://"), re.compile(r"Failed to load resource"))+++class PageProblems:+ """Collects failed responses and JavaScript errors for the current page."""++ def __init__(self, page):+ self.failed, self.errors = [], []+ page.on("response", self._response)+ page.on("pageerror", lambda exc: self.errors.append(f"uncaught: {exc}"))+ page.on("console", self._console)++ def _response(self, response):+ if response.url.startswith(BASE) and response.status >= 400:+ self.failed.append(f"{response.status} {response.url}")++ def _console(self, msg):+ if msg.type != "error":+ return+ text = msg.text+ location = (msg.location or {}).get("url", "")+ if any(p.search(text) or p.search(location) for p in IGNORED_CONSOLE):+ return+ self.errors.append(f"console: {text}")++ def reset(self):+ self.failed.clear()+ self.errors.clear()++[email protected](scope="module")+def browser_page():+ with playwright_api.sync_playwright() as p:+ browser = p.chromium.launch()+ context = browser.new_context(viewport={"width": 1400, "height": 900}, locale="en-US")+ page = context.new_page()+ problems = PageProblems(page)+ page.goto(f"{BASE}/login")+ page.fill("input[name=email]", USER)+ page.fill("input[name=password]", PASSWORD)+ page.locator("form:has(input[name=password]) [type=submit]:visible").first.click()+ page.wait_for_load_state("load")+ assert "/login" not in page.url, f"could not log in as {USER}"+ problems.reset()+ yield page, problems+ context.close()+ browser.close()+++def _open(page, problems, path):+ # Leave the previous page first: its requests still in flight are+ # aborted on unload and logged as "Failed to fetch", which belongs to+ # that page, not the one being checked.+ page.goto("about:blank")+ page.wait_for_timeout(300)+ problems.reset()+ response = page.goto(f"{BASE}{path}")+ page.wait_for_load_state("load")+ # Headers mount after the page's scripts run; give polling apps a moment.+ page.wait_for_timeout(2500)+ return response+++def _check_common(page, problems, path):+ body = page.inner_text("body")+ leftovers = [m for m in ("${", "{{") if m in body]+ assert not leftovers, f"{path}: template text left in the page: {leftovers}"+ assert not problems.failed, f"{path}: failed requests: {problems.failed}"+ assert not problems.errors, f"{path}: JavaScript errors: {problems.errors}"+++def _check_user_menu(page, path):+ toggle = page.locator("[data-user-menu-toggle]:visible").first+ assert toggle.count() == 1, f"{path}: no user menu button in the header"+ toggle.click()+ menu = page.locator("[data-user-menu-dropdown]:visible").first+ menu.wait_for(state="visible", timeout=5000)+ text = menu.inner_text()+ missing = [entry for entry in MENU_ENTRIES if entry not in text]+ assert not missing, f"{path}: user menu is missing {missing}"+ page.keyboard.press("Escape")+ page.mouse.click(5, 400)+++def test_main_view(browser_page):+ page, problems = browser_page+ _open(page, problems, "/")+ assert page.locator("[data-v-app]").count() >= 1, "/: the app did not mount"+ _check_user_menu(page, "/")+ _check_common(page, problems, "/")++[email protected]("path", ["/account", "/admin"])+def test_settings_pages(browser_page, path):+ page, problems = browser_page+ response = _open(page, problems, path)+ assert response is not None and response.status == 200, f"{path}: HTTP {response and response.status}"+ header = page.locator("#global-header")+ assert header.count() == 1, f"{path}: no shared page header"+ assert header.get_attribute("data-v-app") is not None, f"{path}: the page header did not mount"+ label = header.locator(".t-meta").first.inner_text().strip()+ assert label, f"{path}: the page label in the header is empty"+ _check_user_menu(page, path)+ if path == "/account":+ tabs = page.locator(".tabs button:visible, .tabs a:visible")+ assert tabs.count() >= 3, "/account: the tab strip did not render"+ _check_common(page, problems, path)+++def test_inquire_page(browser_page):+ page, problems = browser_page+ response = _open(page, problems, "/inquire")+ if response is None or response.status != 200 or "/inquire" not in page.url:+ pytest.skip("Inquire mode is not enabled")+ problems.failed[:] = [f for f in problems.failed if "/inquire " not in f]+ assert page.locator("#inquire-app[data-v-app]").count() == 1, "/inquire: the app did not mount"+ _check_user_menu(page, "/inquire")+ _check_common(page, problems, "/inquire")+++def test_a_recording_detail_view(browser_page):+ page, problems = browser_page+ listing = page.request.get(f"{BASE}/api/recordings?per_page=1")+ items = listing.json().get("recordings", []) if listing.ok else []+ if not items:+ pytest.skip("no recordings on this instance")+ path = f"/recordings/{items[0]['id']}"+ _open(page, problems, path)+ _check_user_menu(page, path)+ _check_common(page, problems, path)