Skip to content

SigNoz is an open-source, OpenTelemetry-native observability platform for your team and their AI agents. Get logs, metrics, and traces in one tool with features like APM, distributed tracing, log management, infra monitoring, etc. Combined with SigNoz MCP and a native AI teammate (in SigNoz Cloud) it helps you build more resilient apps.

20.4 KB
Raw Blame

E2E Tests

SigNoz uses end-to-end tests to verify the frontend works correctly against a real backend. These tests use Playwright to drive a real browser against a containerized SigNoz stack that pytest brings up — the same fixture graph integration tests use, with an extra HTTP seeder container for per-spec telemetry seeding.

How to set up the E2E test environment?

Prerequisites

Before running E2E tests, ensure you have the following installed:

  • Python 3.13+
  • uv
  • Docker (for containerized services)
  • Node 18+ and pnpm

Initial Setup

  1. Install Python deps for the shared tests project:
cd tests
uv sync
  1. Install Node deps and Playwright browsers:
cd e2e
pnpm install
pnpm install:browsers   # one-time Playwright browser install

Starting the Test Environment

To spin up the backend stack (SigNoz, ClickHouse, Postgres, ClickHouse Keeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:

cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --with-web \
  e2e/bootstrap/setup.py::test_setup

This command will:

  • Bring up all containers via pytest fixtures
  • Register the admin user ([email protected] / password123Z$)
  • Apply the enterprise license (via a WireMock stub of Zeus) and dismiss the org-onboarding prompt so specs can navigate directly to feature pages
  • Start the HTTP seeder container (tests/seeder/ — exposing /telemetry/{traces,logs,metrics} POST + DELETE)
  • Write backend coordinates to tests/e2e/.env.local (loaded by playwright.config.ts via dotenv)
  • Keep containers running via the --reuse flag
  • Rebuild the SigNoz container from the current sources via the --rebuild flag

The --with-web flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start; later builds are incremental.

Rebuilding After Source Changes

The --with-web image bakes the built frontend in, so neither backend nor frontend changes are picked up while --reuse keeps the container running. --rebuild fixes that for both: it kills the SigNoz container, rebuilds the image incrementally (go build cache + pnpm store — a frontend-only change rebuilds in about a minute), and starts a fresh one while databases, mocks, migrations, and the seeder stay reused. The setup command above passes it, so the iteration loop is: change code → re-run the setup command → re-run your specs. --rebuild requires --reuse and cannot be combined with --teardown or --clean.

Stopping the Test Environment

When you're done writing E2E tests, clean up the environment:

cd tests
uv run pytest --basetemp=./tmp/ -vv --teardown \
  e2e/bootstrap/setup.py::test_teardown

Understanding the E2E Test Framework

Playwright drives a real browser (Chromium / Firefox / WebKit) against the running SigNoz frontend. The backend is brought up by the same pytest fixture graph integration tests use, so both suites share one source of truth for container lifecycle, license seeding, and test-user accounts.

  • Why Playwright? First-class TypeScript support, network interception, automatic wait-for-visibility, built-in trace viewer that captures every request/response the UI triggers — so specs rarely need separate API probes alongside UI clicks.
  • Why pytest for lifecycle? The integration suite already owns container bring-up. Reusing it keeps the E2E stack exactly in sync with the integration stack and avoids a parallel lifecycle framework.
  • Why a separate seeder container? Per-spec telemetry seeding (traces / logs / metrics) needs a thin HTTP wrapper around the ClickHouse insert helpers so a browser spec can POST from inside the test. The seeder lives at tests/seeder/, is built from tests/Dockerfile.seeder, and reuses the same fixtures/{traces,logs,metrics}.py as integration tests.
tests/
├── fixtures/                  # shared with integration (see integration.md)
├── integration/               # pytest integration suite
├── seeder/                    # standalone HTTP seeder container
│   ├── __init__.py
│   ├── Dockerfile
│   └── server.py              # FastAPI app wrapping fixtures.{traces,logs,metrics}
└── e2e/
    ├── package.json
    ├── playwright.config.ts   # loads .env + .env.local via dotenv
    ├── .env.example           # staging-mode template
    ├── .env.local             # generated by bootstrap/setup.py (gitignored)
    ├── AGENTS.md              # read this before writing a spec
    ├── bootstrap/
    │   └── setup.py           # test_setup / test_teardown — pytest lifecycle
    ├── drivers/               # the driver kit: one module per feature
    │   ├── kit.ts             # layer(), seeded(), the root `test`
    │   ├── auth.ts            # login + per-worker storageState
    │   ├── api.ts             # authed HTTP client + seeder URL
    │   ├── tanstack.ts        # the shared table driver
    │   ├── alerts/            # feature driver, seeds, payloads
    │   └── infra/             # feature driver, seeds, __generated__/datasets.ts
    ├── docs/flake-notes.md    # measurements and timeout budgets
    ├── testdata/              # static data files (marked generated)
    ├── tests/                 # Playwright .spec.ts files, one dir per feature
    │   └── alerts/
    │       └── list.spec.ts   # (example)
    └── artifacts/             # per-run output (gitignored)
        ├── html/              # HTML reporter output
        ├── json/              # JSON reporter output
        └── results/           # per-test traces / screenshots / videos on failure

The driver kit

A spec imports one module: its feature's test constant. That constant is built by composing layers, and each layer adds one named driver to the test arguments — so a spec body never handles a Page:

import { expect, test } from '../../drivers/alerts';

test('TC-01 seeded rules render', async ({ alerts, alertList }) => {
  await alerts.list.goto({ search: alertList.namePrefix });
  await expect(alerts.list.rows()).toHaveCount(10);
});
  • drivers/kit.ts holds the composition primitives. driver(name, build, dispose?) binds a factory to the authenticated page, and dispose runs after the test on the same page so a driver can delete what it seeded; seeded(name, open, close) seeds data once per worker on a throwaway admin page and tears it down after; allowFor(floor) raises the test timeout to a floor and never lowers it. test.slow() is banned in drivers, because it compounds when called in a loop.
  • drivers/<feature>.ts (or drivers/<feature>/) exports the feature's driver, its layers, and the composed test. A driver is a plain function returning an object of closures — no classes, and no page parameter on anything a spec calls.
  • Shared drivers — tanstack.ts (the table), querysearch.ts (the expression editor), antd.ts (select primitives), console.ts, api.ts, storage.ts — are composed into feature drivers by direct call, not by reaching for another fixture.
  • drivers/ is AI-owned: coding agents generate and maintain it with the specs, and .gitattributes collapses it in diffs.
  • testdata/ and any __generated__/ directory hold transcribed data. Both are marked linguist-generated; something in the suite re-derives them at runtime, so a stale value fails loudly rather than silently.

Fixture scopes:

  • test scope — fresh data per test. Use for mutations (edit, delete, rename).
  • worker scope — shared across a worker. Use for read-only data; it pays the setup cost once per worker instead of once per test. Playwright builds them lazily, so a spec that never destructures a seed never pays for it.
  • suite scope: a dataset only one file uses. Seed it in test.beforeAll through onAdminPage(browser, …), since authedPage is test-scoped.

Which layer does this test belong to?

Four layers run against this codebase, in cost order: jest → Storybook + msw → pytest integration → Playwright. Before writing an E2E, check the cheaper three. tests/e2e/AGENTS.md has the full rule; the short version:

  • A pure function, a reducer, a config literal or a component's own behaviour is a jest test. Most containers under frontend/src/container/ already have one.
  • A rendered state that needs no backend (empty, error, loading) is a Storybook story or a jest test with a fixed mock.
  • Anything the API does — ordering, pagination, filter semantics, validation, roll-ups — is a pytest integration test, and for infra monitoring and alert evaluation it very likely already exists. Grep tests/integration/ first.
  • An E2E is for a real response reaching a real DOM: URL and history, write paths that must persist, cross-page navigation, and one seeded proof per surface.

Each spec follows these principles:

  1. Directory per feature: tests/e2e/tests/<feature>/*.spec.ts. Cross-resource junction concerns (e.g. cascade-delete) go in their own file, not packed into one giant spec.
  2. Test titles use TC-NN: test('TC-01 alerts page — tabs render', ...). Preserves ordering at a glance and maps to external coverage tracking.
  3. UI-first: drive flows through the UI. Playwright traces capture every BE request/response the UI triggers, so asserting on UI outcomes implicitly validates BE contracts. To assert a write, attach page.waitForResponse before the click, then reload and re-read from the server.
  4. Self-contained state: each spec seeds its own data, scopes its assertions to the names it seeded, and cleans up. The pytest harness creates a fresh stack with zero dashboards / alerts / etc. Names carry the worker index, so parallel workers never collide. Pick the scope from the list above.
  5. Seed through the kit when the UI flow is not under test. apiDriver(page) (drivers/api.ts) creates control-plane objects with the session's token, and the seeder takes telemetry. The JWT lives in localStorage, so a spec never improvises with page.request.*. Feature drivers wrap both (dashboards.seed, alerts.seed.threshold).
  6. Reusable static data lives in tests/e2e/testdata/. For example, traces/large-trace.json is POSTed to the seeder by drivers/traces.ts. Infra monitoring reads the integration suite's fixtures through seedDataset (drivers/infra/seed.ts) instead of keeping a copy, and asserts on what it returned.
  7. Locate by testid, in the driver. A missing testid is added to the product in the same change. Role, label and CSS lookups are a stopgap, converted as that code is touched; antd internals (.ant-select-dropdown) are the exception, since the markup is not ours.

How to write an E2E test?

Create a new file tests/e2e/tests/alerts/smoke.spec.ts:

import { ALERT_LIST_PAGE_SIZE, expect, test } from '../../drivers/alerts';

test('TC-01 seeded rules render and paginate', async ({ alerts, alertList }) => {
  await alerts.list.goto({ search: alertList.namePrefix });
  await expect(alerts.list.table.totalCount()).toContainText(`of ${alertList.count}`);
  await expect(alerts.list.rows()).toHaveCount(ALERT_LIST_PAGE_SIZE);
});

alertList is a worker-scoped seed: the rules exist before the test starts, are shared by every test in the worker that asks for them, and are deleted when the worker ends.

The authedPage fixture (from tests/e2e/drivers/kit.ts, re-exported by every feature's test) gives you a Page whose browser context is already authenticated as the admin user. First use per worker triggers one login; the resulting storageState is held in memory and reused for later requests. Take it only when the driver genuinely does not cover what you need — navigation, goBack, waitForResponse, a second tab.

To run just this test (assuming the stack is up via test_setup):

cd tests/e2e
npx playwright test tests/alerts/smoke.spec.ts --project=chromium

Here's a more comprehensive example, driven entirely through the feature's driver:

import { expect, test } from '../../drivers/alerts';

test('TC-02 row actions persist: disable, then enable', async ({
  authedPage: page,
  alerts,
  ownedRules,
}) => {
  const name = `smoke-rule-${Date.now()}`;
  const ruleId = await ownedRules.threshold(name);

  await alerts.list.goto({ search: name });
  await expect(alerts.list.stateCell(ruleId)).toHaveText('OK');

  await alerts.list.openFirstRowActions();
  await Promise.all([
    // waitForResponse captures the call the UI triggers — no parallel fetch.
    page.waitForResponse(
      (res) =>
        res.url().includes(`/api/v2/rules/${ruleId}`) &&
        res.request().method() === 'PATCH',
    ),
    alerts.list.menuItem('Disable').click(),
  ]);

  // Re-read from the server rather than trusting the optimistic update.
  await page.reload();
  await expect(alerts.list.stateCell(ruleId)).toHaveText('Disabled');
});

ownedRules is a test-scoped seed: every rule it creates is deleted when the test finishes, so the spec owns its data and nothing leaks into the next one.

Commits

Commit driver changes apart from specs, product testids and tests moved to another layer, so each commit can be reviewed on its own. tests/e2e/AGENTS.md has the order. pnpm typecheck, pnpm lint and pnpm fmt:check gate the suite.

Agents

Three Claude agents in .claude/agents/ accelerate writing and maintaining E2E specs:

  • playwright-test-planner — explores a feature in a real browser plus the local frontend source and writes a test plan as a scratch markdown file (under tests/e2e/specs/, which is gitignored — plans are working artifacts for the generator, not committed docs).
  • playwright-test-generator — converts a test plan into Playwright spec files under tests/e2e/tests/<feature>/. Drives each scenario through MCP browser tools and emits TC-NN-titled tests on the feature's driver, adding the driver methods and seeds the plan needs.
  • playwright-test-healer — runs failing specs, debugs them with snapshots / console / network introspection, and edits the spec to fix selector drift, timing, or state-leak issues.

The agents rely on the Playwright-test MCP server (mcp__playwright-test__* tools). Configure it in your Claude MCP settings; the permission allowlist lives in .claude/settings.local.json.

How to run E2E tests?

Running All Tests

With the stack already up, from tests/e2e/:

pnpm test                 # headless, all projects

Running Specific Projects

pnpm test:chromium        # chromium only
pnpm test:firefox
pnpm test:webkit

Running Specific Tests

cd tests/e2e

# Single feature dir
npx playwright test tests/alerts/ --project=chromium

# Single sub-area
npx playwright test tests/alerts/history/ --project=chromium

# Single file
npx playwright test tests/alerts/page.spec.ts --project=chromium

# Single test by title grep
npx playwright test --project=chromium -g "TC-01"

Iterative modes

pnpm test:ui              # Playwright UI mode — watch + step through
pnpm test:headed          # headed browser
pnpm test:debug           # Playwright inspector, pause-on-breakpoint
pnpm codegen              # record-and-replay locator generation
pnpm report               # open the last HTML report (artifacts/html)

Staging fallback

Point SIGNOZ_E2E_BASE_URL at a remote env via .env — no local backend bring-up, no .env.local generated, Playwright hits the URL directly:

cd tests/e2e
cp .env.example .env      # fill SIGNOZ_E2E_USERNAME / PASSWORD
pnpm test:staging

How to configure different options for E2E tests?

Environment variables

Variable Description
SIGNOZ_E2E_BASE_URL Base URL the browser targets. Written by bootstrap/setup.py for local mode; set manually for staging.
SIGNOZ_E2E_USERNAME Admin email. Bootstrap writes [email protected].
SIGNOZ_E2E_PASSWORD Admin password. Bootstrap writes the integration-test default.
SIGNOZ_E2E_SEEDER_URL Seeder HTTP base URL — hit by specs that need per-test telemetry.

Precedence in playwright.config.ts, lowest to highest: .env (user-provided, staging) → .env.local (bootstrap-generated, local mode) → whatever is already in process.env. The config parses both files itself and only fills in keys the environment does not already define, so exporting a variable always wins:

# runs against a locally served frontend, not whatever .env.local points at
SIGNOZ_E2E_BASE_URL=http://127.0.0.1:3301 pnpm test tests/alerts

This is deliberately not dotenv.config({ override: true }). That flag makes the file beat process.env, which silently discarded exported values — including the SIGNOZ_E2E_BASE_URL in pnpm test:staging, whenever a .env.local happened to exist.

Playwright options

The full playwright.config.ts is the source of truth. Common things to tweak:

  • projects — Chromium / Firefox / WebKit are enabled by default. Disable to speed up iteration.
  • retries — 2 on CI (process.env.CI), 0 locally.
  • fullyParallel: true — files run in parallel by worker; within a file, use test.describe.configure({ mode: 'serial' }) if tests share list pages / mutate shared state.
  • trace: 'on-first-retry', screenshot: 'only-on-failure', video: 'retain-on-failure' — default diagnostic artifacts land in artifacts/results/<test>/.

Pytest options (bootstrap side)

The same pytest flags integration tests expose work here, since E2E reuses the shared fixture graph:

  • --reuse — keep containers warm between runs (required for all iteration).
  • --rebuild — recreate the SigNoz container from the current sources (backend and, with --with-web, frontend) while the rest of the stack stays up. Requires --reuse.
  • --teardown — tear everything down.
  • --clean — prune the docker build caches, forcing the next image build to start cold.
  • --with-web — build the frontend into the SigNoz container. Required for E2E; integration tests don't need it.
  • --sqlstore-provider, --postgres-version, --clickhouse-version, etc. — see docs/contributing/tests/integration.md.

What should I remember?

  • Always use the --reuse flag when setting up the E2E stack. --with-web adds a ~4 min frontend build on a cold start; later builds are incremental.
  • Changed backend or frontend code? Re-run the setup command — it passes --rebuild, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
  • Don't teardown before setup. --reuse correctly handles partially-set-up state, so chaining teardown → setup wastes time.
  • Prefer UI-driven flows. Playwright captures BE requests in the trace; a parallel fetch probe is almost always redundant. Seed through the kit, never with page.request.*.
  • Use page.waitForResponse on UI clicks to assert BE contracts — it still exercises the UI trigger path.
  • Title every test TC-NN <short description> — keeps the suite navigable and reportable.
  • Split by resource, not by regression suite. One spec per feature resource; cross-resource junction concerns (cascade-delete, linked-edit) get their own file.
  • Name resources by what they are, plus the worker index (e2e-alert-list-w${workerIndex}-${Date.now()}). Six workers share one stack, so assert only on names you seeded.
  • Never commit test.only — a pre-commit check or CI runs with forbidOnly: true.
  • Prefer explicit waits over page.waitForTimeout(ms). await expect(locator).toBeVisible() is always better than waitForTimeout(5000).
  • Unique test names won't save you from shared-tenant state. When two tests hit the same list page, either serialize (describe.configure({ mode: 'serial' })) or isolate cleanup religiously.
  • Green locally, red on CI? Run STRESS=1 CI=1 pnpm test <path> (4x CPU throttle, 2 workers, 2 retries) before filing a product bug.
  • Commit drivers apart from specs. One concern per commit; see Commits.
  • Artifacts go to tests/e2e/artifacts/ — HTML report at artifacts/html, traces at artifacts/results/<test>/. All gitignored; archive the dir in CI.