OpenPanel is an open-source web and product analytics platform, an open-source alternative to Mixpanel with optional self-hosting.
ralph(M12-010): Polish: fika swept from jobs/naming.ts, root package.json scripts point only at workspaces and scripts that exist, CLAUDE.md / .claude/CLAUDE.md / AGENTS.md rewritten for the current tree and paths
ralphbcf07cbc7c45
diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.mdindex 6ada80df8..a9cf654c1 100644--- a/.claude/CLAUDE.md+++ b/.claude/CLAUDE.md@@ -12,22 +12,22 @@ Openpanel is an open-source web/product analytics platform (Mixpanel alternative```bash# Development-pnpm dev # Run all services (api, worker, dashboard) in parallel+pnpm dev # Run apps/api (ROLE=all, port 3333) + apps/start dashboard (port 3000) in parallelpnpm dev:public # Run public/docs site onlypnpm dock:up / dock:down # Start/stop Docker (PostgreSQL, Redis, ClickHouse)# Code quality-pnpm check # Lint check (Biome via Ultracite)+pnpm check # Lint (Biome via Ultracite) + dependency-cruiser (check:deps)pnpm fix # Auto-fix lint/format issuespnpm typecheck # Typecheck all packages# Testing-pnpm test # Run all tests (vitest)-pnpm vitest run <path> # Run a single test file-# Workspace: packages/* and apps/* (excluding apps/start)+pnpm test # vitest run — packages/* and apps/* (excluding apps/start, packages/core)+cd packages/core && bun test # core's own suite (ADR-010; bun:test cannot run under vitest)+pnpm vitest run <path> # Run a single vitest test file# Database-pnpm codegen # Generate Prisma types + geo data+pnpm codegen # Generate the Prisma client + core's geo/bot/ASN datapnpm migrate # Run Prisma migrations (dev)pnpm migrate:deploy # Deploy migrations (production - never run this)@@ -42,37 +42,35 @@ pnpm dock:redis # Redis CLI| App | Stack | Port | Purpose ||-----|-------|------|---------|-| `apps/api` | Fastify + tRPC | 3333 | REST/RPC API server |-| `apps/start` | TanStack Start (Vite + React 19) | 3000 | Dashboard SPA |+| `apps/api` | Elysia on Bun, `ROLE=api\|worker\|all` | 3000 (`API_PORT`) | REST/tRPC API, BullMQ workers, Kafka ingest consumer — one process, role-gated |+| `apps/start` | TanStack Start (Vite + React 19) | 3000 (dev default; run alongside `apps/api` needs a distinct `API_PORT`) | Dashboard SPA || `apps/public` | Next.js 16 + Fumadocs | - | Marketing/docs site |-| `apps/worker` | Express + BullMQ | 9999 | Background job processor |++The standalone worker app is gone — M9 merged it into `apps/api`, selected at boot by `ROLE`.### Key Packages| Package | Purpose ||---------|---------|-| `packages/db` | Prisma ORM (PostgreSQL) + ClickHouse client |-| `packages/trpc` | tRPC router definitions, context, middleware |-| `packages/auth` | Authentication (Arctic OAuth, Oslo sessions, argon2) |-| `packages/queue` | BullMQ + GroupMQ job queue definitions |-| `packages/redis` | Redis client + LRU caching |-| `packages/validation` | Zod schemas shared across apps |-| `packages/common` | Shared utilities (date-fns, ua-parser, nanoid) |+| `packages/core` | The backend: every module (`<name>.rpc.ts` / `.routes.ts` / `.jobs.ts` / `.service.ts`), the tRPC router, Elysia route surfaces, the BullMQ job registry, auth macros, the ClickHouse `sql` tag consumer code, shared utilities. `apps/api` is a thin shell that mounts it |+| `packages/db` | Prisma ORM (PostgreSQL) + ClickHouse client, the `sql` tagged-template query builder, migrations |+| `packages/redis` | Redis client (Bun `RedisClient`) + caching, locks, pub/sub || `packages/email` | React Email templates via Resend |+| `packages/payments` | Billing/subscription integration || `packages/sdks/*` | Client SDKs (web, react, next, express, react-native, etc.) |### Data Flow-1. **Event ingestion**: Client SDKs → `apps/api` (track routes) → Redis queue-2. **Processing**: `apps/worker` picks up jobs from BullMQ, batches events into ClickHouse-3. **Dashboard queries**: `apps/start` → tRPC → `apps/api` → ClickHouse (analytics) / PostgreSQL (config)-4. **Real-time**: WebSocket via Fastify, pub/sub via Redis+1. **Event ingestion**: Client SDKs → `apps/api` `/track` → Kafka/Redpanda topic `events`+2. **Processing**: `apps/api`'s Kafka consumer (any non-`api` role) batches events into ClickHouse+3. **Dashboard queries**: `apps/start` → tRPC (fetch adapter) → `apps/api` → ClickHouse (analytics) / PostgreSQL (config)+4. **Real-time**: WebSocket via Elysia, pub/sub via Redis### Three-Database Strategy- **PostgreSQL**: Relational data (users, orgs, projects, dashboards). Managed by Prisma.- **ClickHouse**: Analytics event storage (OLAP). High-volume reads/writes.-- **Redis**: Caching, job queues (BullMQ), rate limiting, pub/sub.+- **Redis**: Caching, BullMQ job queues, rate limiting, pub/sub.### Dashboard (apps/start)@@ -80,7 +78,7 @@ Uses TanStack Router with file-based routing (`src/routes/`). State management v### API (apps/api)-Fastify server with tRPC integration. Route files in `src/routes/`. Hooks for IP extraction, request logging, timestamps. Built with `tsdown`.+Elysia on Bun, mounting `@openpanel/core`'s routers. No bundler — `bun run src/main.ts` runs the TypeScript directly (ADR-010). `main.ts` boots deps, `app.ts` wires the four Elysia scopes (root, dashboard, public API, ops); `ROLE` selects whether workers, schedulers and the Kafka consumer also start in this process.---## Core Principlesdiff --git a/.claude/commands/pr-review.md b/.claude/commands/pr-review.mdindex 5bcf18242..80b8a8348 100644--- a/.claude/commands/pr-review.md+++ b/.claude/commands/pr-review.md@@ -12,21 +12,21 @@ Review the current PR by running `git diff main...HEAD` and examining all change## Authorization & Data Access-- Every tRPC procedure that accesses project/org data uses the appropriate access check (`getProjectAccess`, `getOrganizationAccess`, `getClientAccess` from `@openpanel/db`)+- Every `protectedProcedure` gets the `enforceAccess` middleware (`rpc/base.ts`) for free — it reads the raw, top-level `projectId`/`organizationId` and calls `ctx.services.auth`, which is `shared/access.ts`'s single `createAccessChecks` ladder, bound once centrally in `auth.service.ts` (M10-002)- All ClickHouse queries filter by `project_id` — no cross-project data leaks- All Prisma queries scope to the authenticated user's org/project — no missing `organizationId`/`projectId` where clauses- No client-provided IDs trusted for authorization without server-side validation-- Note: all `protectedProcedure` which either have `organizationId` or `projectId` will be ensured correct access in the middleware+- Note: `enforceAccess` only checks `projectId`/`organizationId` in *input* — secondary object ids (reportId, dashboardId, …) need an in-handler check (ADR-011)## Architecture: Service Layer-- Data fetching and mutation logic lives in `packages/db/src/services/` — not inline in tRPC routers or API route handlers+- Data fetching and mutation logic lives in `packages/core/src/modules/<name>/<name>.service.ts` — not inline in tRPC routers or route handlers- tRPC routers should call service functions, not query ClickHouse/Prisma directly-- New queue job types defined in `packages/queue/src/queues.ts`, not inline in app code+- New job types are declared in the owning module's `<name>.jobs.ts` and spread into `packages/core/src/jobs.registry.ts` — not inline in app code. A cron schedule is the job's own `cron` field (ADR-021), not a second registry## Architecture: Validation-- Zod schemas used in more than one package/app belong in `packages/validation/src/` — not defined locally in a router or component+- Zod schemas used by more than one module, or by `apps/start`/an SDK, belong in the owning module's `<name>.constants.ts` (imports only zod, other constants, or types — ADR-008) — not defined locally in a router or component- All tRPC procedures use `.input(zodSchema)` for input validation- Schemas used only within a single file can stay local@@ -48,8 +48,8 @@ Review the current PR by running `git diff main...HEAD` and examining all change## Clickhouse- Queries should be optimized for very large datasets-- `chQuery` can be used for simple queries-- `clix` should be used when we have a lot of dynamic sql queries+- Static queries are raw SQL with ClickHouse-native `{name:Type}` bound params+- Dynamic composition (chart engine, filters, breakdowns) uses `@openpanel/db`'s `sql` tagged template — never `${}` of a raw value, never hand-rolled string concatenation (ADR-013)---diff --git a/CLAUDE.md b/CLAUDE.mdindex 5797fb5f0..d3fb55bdc 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -6,7 +6,7 @@ anything.## SafetyYour world is exactly two directories: this repository and-`/data/apps/rewrite-openpanel` (the controller). Do not read, list, or modify+`/home/deploy/rewrite-openpanel` (the controller). Do not read, list, or modifyanything else on this machine.Never use production credentials — production does not exist on this machine,@@ -18,7 +18,7 @@ and nothing here should reach it. Local Postgres / ClickHouse / Redis only.ClickHouse (8123) are already running on localhost, started by another user.Do not try to start, stop or install them.- **No sudo.** Anything needing root is a human task. Say so; do not attempt it.-- Writes are confined to this repo and `/data/apps/rewrite-openpanel`. Every+- Writes are confined to this repo and `/home/deploy/rewrite-openpanel`. Everyother path on the box is read-only to you.## Commands@@ -27,7 +27,7 @@ and nothing here should reach it. Local Postgres / ClickHouse / Redis only.pnpm installpnpm codegen # prisma client; needs .envpnpm run typecheck-pnpm run lint+pnpm check # ultracite + check:depspnpm test```@@ -43,18 +43,18 @@ count. When replacing a query, diff the old and new result sets on the same dataLocal is a single node; production is 2 shards x 2 replicas. Before editing anyquery against a `Distributed` table read-`/data/apps/rewrite-openpanel/docs/ENVIRONMENT.md` - a plain `IN (subquery)` on a+`/home/deploy/rewrite-openpanel/docs/ENVIRONMENT.md` - a plain `IN (subquery)` on adistributed table silently returns per-shard results.## Architecture-Architecture decisions live in `/data/apps/rewrite-openpanel/decisions`.+Architecture decisions live in `/home/deploy/rewrite-openpanel/decisions`.**Accepted ADRs are binding.** If you need a decision no accepted ADR covers,stop and say `BLOCKED: needs ADR — <question>`. Do not decide it yourself.-The target architecture is `/data/apps/rewrite-openpanel/docs/TARGET_ARCHITECTURE.md`.+The target architecture is `/home/deploy/rewrite-openpanel/docs/TARGET_ARCHITECTURE.md`.The documented behaviour of the system you are replacing is in-`/data/apps/rewrite-openpanel/docs/current/`. Read the relevant document before+`/home/deploy/rewrite-openpanel/docs/current/`. Read the relevant document beforechanging that area — it exists so you do not have to rediscover it.## Ralph@@ -62,9 +62,9 @@ changing that area — it exists so you do not have to rediscover it.Do not modify:```-/data/apps/rewrite-openpanel/plan/tasks.json-/data/apps/rewrite-openpanel/state-/data/apps/rewrite-openpanel/verification+/home/deploy/rewrite-openpanel/plan/tasks.json+/home/deploy/rewrite-openpanel/state+/home/deploy/rewrite-openpanel/verification```The orchestrator owns task state and owns the definition of "passing".diff --git a/docs/TECH_DEBT.md b/docs/TECH_DEBT.mdindex b7ef60b5b..65ea89aaa 100644--- a/docs/TECH_DEBT.md+++ b/docs/TECH_DEBT.md@@ -1339,3 +1339,32 @@ once-qualified, no dashes, no backticks) cannot spell either name. **If afuture task widens `sql.id` to accept a backtick-quoted, multi-part name,these two rows and `compiled.ts` with them can go.** That is the standing exitcondition; recorded here, not done in M12-009 (out of its scope).++## Root scripts (M12-010)++Every `--filter <name>` in the root `package.json` verified against the+workspace it selects and the script that workspace actually exports.+`gen:bots` pointed at `api`, which has no such script — the generator has+always lived in `core`; fixed to match `gen:referrers`, which was already+correct. `dev` assumed a `testing` script existed on every dev-facing+workspace (a V1-era convention); only `apps/api` still has one, so+`pnpm -r --parallel testing` silently ran nothing but the API. Fixed to start+`apps/api` and `apps/start`'s dashboard explicitly, on non-colliding ports+(both default to 3000) — verified with a real 10s boot (Kafka consumer joined,+7 BullMQ workers started, cron upserted, Vite ready) and a clean `SIGTERM`+shutdown, 2026-09-07.++| Script | Target workspace(s) | Target script | Exists |+|---|---|---|---|+| `gen:bots` | `core` | `gen:bots` | yes |+| `gen:referrers` | `core` | `gen:referrers` | yes |+| `dev` | `api` (testing), `start` (dev) | `testing`, `dev` | yes |+| `dev:public` | `public` | `dev` | yes |+| `db:codegen` | `db` | `codegen` | yes |+| `codegen` | `db`, `core` | `codegen` (both) | yes |+| `migrate` | `db` | `migrate` | yes |+| `migrate:deploy` | `db` | `migrate:deploy` | yes |++`typecheck`, `check`, `check:workspace`, `check:deps` and `fix` run across the+whole workspace (`pnpm -r` / no filter) and name no specific package, so they+are not in this table.diff --git a/package.json b/package.jsonindex c7352f4c3..c2271226c 100644--- a/package.json+++ b/package.json@@ -8,7 +8,7 @@"packageManager": "[email protected]","scripts": {"test": "vitest run",- "gen:bots": "pnpm -r --filter api gen:bots",+ "gen:bots": "pnpm -r --filter core gen:bots","gen:referrers": "pnpm -r --filter core gen:referrers","dock:up": "docker compose up -d","dock:down": "docker compose down",@@ -18,7 +18,7 @@"codegen": "pnpm -r --filter db --filter core run codegen","migrate": "pnpm -r --filter db run migrate","migrate:deploy": "pnpm -r --filter db run migrate:deploy",- "dev": "pnpm -r --parallel testing",+ "dev": "pnpm --filter api run testing & pnpm --filter start run dev & wait","dev:public": "pnpm -r --filter public dev","typecheck": "pnpm -r --no-bail typecheck","update-simple-git-hooks": "npx simple-git-hooks",diff --git a/packages/core/src/jobs/naming.test.ts b/packages/core/src/jobs/naming.test.tsindex 2d71efc38..59eae683b 100644--- a/packages/core/src/jobs/naming.test.ts+++ b/packages/core/src/jobs/naming.test.ts@@ -48,7 +48,7 @@ test('the golden table covers exactly the queues the registry declares', () => {).toEqual(golden);});-test('braces appear only under QUEUE_CLUSTER — the divergence from fika', () => {+test('braces appear only under QUEUE_CLUSTER', () => {expect(queueKey('cron')).toBe('cron');expect(queueKey('cron', {})).toBe('cron');expect(queueKey('cron', { cluster: false })).toBe('cron');diff --git a/packages/core/src/jobs/naming.ts b/packages/core/src/jobs/naming.tsindex 87443e3e9..9896f2e45 100644--- a/packages/core/src/jobs/naming.ts+++ b/packages/core/src/jobs/naming.ts@@ -13,11 +13,9 @@ export interface QueueKeyOptions {/*** The Redis key a queue lives under.*- * Deliberate divergence from the fika template this is ported from: fika- * braces unconditionally, we brace only under `QUEUE_CLUSTER`, because V1's- * `getQueueName` did and every existing queue in every deployment is named- * that way (ADR-005). `-` joins the namespace because BullMQ rejects a `:`- * in a queue name.+ * Braces only under `QUEUE_CLUSTER`, because V1's `getQueueName` did and every+ * existing queue in every deployment is named that way (ADR-005). `-` joins+ * the namespace because BullMQ rejects a `:` in a queue name.*/export function queueKey(name: string, options?: QueueKeyOptions): string {const scoped = options?.namespace ? `${name}-${options.namespace}` : name;