Skip to content

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

8.1 KB
Raw Blame

Frontend Testing

Testing Strategy

Type Tool Speed When to use
Integration (primary) Vitest + React Testing Library + MSW Fast (~100ms) ~90% of tests — page-level rendering with mocked API
E2E Playwright Slow (~5s) Critical flows: auth, payments, cross-page navigation
Visual Storybook + Chromatic N/A Design system components

Integration tests are the default. Since most of our code is client-only, we test at the page level: render the page with React Testing Library, mock API requests with MSW (handlers auto-generated by Orval), and assert with testing-library queries.

Integration Tests (Vitest + RTL + MSW)

Running

pnpm test:unit              # run all integration/unit tests with coverage
pnpm test:unit:watch        # watch mode for development

File location

Tests live in a __tests__/ folder next to the page or component they test:

app/(platform)/library/
  __tests__/
    main.test.tsx           # tests the main page rendering & interactions
    search.test.tsx         # tests search-specific behavior
  components/
    AgentCard/
      AgentCard.tsx
      __tests__/
        AgentCard.test.tsx  # only when testing the component in isolation
  page.tsx
  useLibraryPage.ts

Naming: use descriptive names like main.test.tsx, search.test.tsx, filters.test.tsx — not page.test.tsx or index.test.tsx.

Writing an integration test

  1. Render the page using the custom render() from @/tests/integrations/test-utils (wraps providers)
  2. Mock API responses using Orval-generated MSW handlers from @/app/api/__generated__/endpoints/{tag}/{tag}.msw.ts
  3. Assert with React Testing Library queries (screen.findByText, screen.getByRole, etc.)
import { render, screen } from "@/tests/integrations/test-utils";
import { server } from "@/mocks/mock-server";
import {
  getGetV2ListLibraryAgentsMockHandler200,
  getGetV2ListLibraryAgentsMockHandler422,
} from "@/app/api/__generated__/endpoints/library/library.msw";
import LibraryPage from "../page";

describe("LibraryPage", () => {
  test("renders agent list from API", async () => {
    server.use(getGetV2ListLibraryAgentsMockHandler200());

    render(<LibraryPage />);

    expect(await screen.findByText("All Workflows")).toBeDefined();
  });

  test("shows error state on API failure", async () => {
    server.use(getGetV2ListLibraryAgentsMockHandler422());

    render(<LibraryPage />);

    expect(await screen.findByText(/error/i)).toBeDefined();
  });
});

MSW handlers

Orval generates typed MSW handlers for every endpoint and HTTP status code:

  • getGetV2ListLibraryAgentsMockHandler200() — success response with faker data
  • getGetV2ListLibraryAgentsMockHandler422() — validation error response
  • getGetV2ListLibraryAgentsMockHandler401() — unauthorized response

To override with custom data, pass a resolver:

import { http, HttpResponse } from "msw";

server.use(
  http.get("http://localhost:3000/api/proxy/api/library/agents", () => {
    return HttpResponse.json({
      agents: [{ id: "1", name: "My Agent" }],
      pagination: { total: 1 },
    });
  }),
);

All handlers are aggregated in src/mocks/mock-handlers.ts and the MSW server is set up in src/mocks/mock-server.ts.

Test utilities

  • @/tests/integrations/test-utils — custom render() that wraps components with QueryClientProvider, BackendAPIProvider, OnboardingProvider, NuqsTestingAdapter, and TooltipProvider, so query-state hooks and tooltips work out of the box in page-level tests
  • @/tests/integrations/setup-nextjs-mocks — mocks for next/navigation, next/image, next/headers, next/link
  • @/tests/integrations/mock-auth-request — mocks Better Auth (returns null session by default)

What to test at page level

  • Page renders with API data (happy path)
  • Loading and error states
  • User interactions that trigger mutations (clicks, form submissions)
  • Conditional rendering based on API responses
  • Search, filtering, pagination behavior

When to test a component in isolation

Only when the component has complex internal logic that is hard to exercise through the page test. Prefer page-level tests as the default.

E2E Tests (Playwright)

Running

pnpm test                   # build + run the Playwright E2E suite used in CI
pnpm test-ui                # run the same E2E suite with Playwright UI
pnpm test:e2e:no-build      # run the same E2E suite against a running dev server
pnpm exec playwright test   # run the same eight-spec Playwright suite directly

Setup

  1. Start the backend stack (Postgres, Redis, RabbitMQ, API):
    • From autogpt_platform: docker compose --profile local up deps_backend -d
  2. Seed rich E2E data (creates [email protected] with library agents):
    • From autogpt_platform/backend: poetry run python test/e2e_test_data.py

How Playwright setup works

  • Playwright runs from frontend/playwright.config.ts and keeps browser-only code in frontend/src/playwright/
  • Global setup creates reusable auth states for deterministic seeded accounts in frontend/.auth/states/
  • getTestUser() (from src/playwright/utils/auth.ts) picks one seeded account for general auth coverage
  • getTestUserWithLibraryAgents() uses the rich user created by the data script

Test users

  • Seeded E2E accounts — created by backend fixtures and logged in during Playwright global setup. Used by getTestUser() and E2E_AUTH_STATES
  • Rich user with library agents — created by backend/test/e2e_test_data.py. Used by getTestUserWithLibraryAgents()

Current Playwright E2E suite

The CI suite is intentionally limited to the cross-page journeys we still require a real browser for. Playwright discovers the PR-gating specs by the *-happy-path.spec.ts naming pattern inside src/playwright/:

  • src/playwright/auth-happy-path.spec.ts
  • src/playwright/settings-happy-path.spec.ts
  • src/playwright/api-keys-happy-path.spec.ts
  • src/playwright/builder-happy-path.spec.ts
  • src/playwright/library-happy-path.spec.ts
  • src/playwright/marketplace-happy-path.spec.ts
  • src/playwright/publish-happy-path.spec.ts
  • src/playwright/copilot-happy-path.spec.ts

Resetting the DB

If you reset the Docker DB and logins start failing:

  1. Delete frontend/.auth/states/* and frontend/.auth/user-pool.json if it exists
  2. Re-run poetry run python test/e2e_test_data.py

Storybook

  • pnpm storybook — run locally
  • pnpm build-storybook — build static
  • pnpm test-storybook — CI runner
  • When changing components in src/components, update or add stories and verify in Storybook/Chromatic

Performance measurement

pnpm perf measures re-renders, main-thread time and JS heap for a fixed scenario (copilot, builder, library) against NEXT_PUBLIC_RENDER_COUNTER=1 pnpm dev, and pnpm perf:compare prints two labelled sets side by side. It is a local tool, not part of CI. See frontend-perf-measurement.md for setup, the scenarios and how to read the output.

Performance PRs

A PR that claims to make the frontend faster includes the pnpm perf:compare --a before --b after table for the scenario it targets, measured on the same machine. A PR that adds a scenario adds its baseline to frontend-perf-baseline.md.

TDD Workflow

When fixing a bug or adding a feature:

  1. Write a failing test first — for integration tests, write the test and confirm it fails. For Playwright, use .fixme annotation
  2. Implement the fix/feature — write the minimal code to make the test pass
  3. Remove annotations — once passing, remove .fixme and run the full suite