Skip to content

OpenPanel is an open-source web and product analytics platform, an open-source alternative to Mixpanel with optional self-hosting.

ralph(M14-001): packages/core/AGENTS.md gains the Services / graph / module / layers / procedures / clients / queues / config / frontend sections from ADR-022, and its Layout line stops describing the wrong factory signature

ralph2967de0dd4cc
diff --git a/packages/core/AGENTS.md b/packages/core/AGENTS.md
index 538bdac8b..27cc6764d 100644
--- a/packages/core/AGENTS.md
+++ b/packages/core/AGENTS.md
@@ -15,6 +15,7 @@ src/
index.ts curated barrel — NOT export *
context.ts AppDeps | Ctx | HttpCtx | JobCtx + createCtx
services.ts Services INTERFACE + createServices(deps)
+ every factory is (deps, services) — see "Services"
logger.ts structural Logger interface
metrics.ts the one prom-client registry
rpc.router.ts appRouter + export type AppRouter
@@ -114,6 +115,188 @@ import into a browser bundle silently, and the bundle still builds;
`intervals` and friends are rewritten to plain `Date` math when they move here,
for exactly this reason.
+## Services
+
+`services.ts` is the composition root. Nothing else builds a service graph.
+
+A service is a factory. No classes, no DI framework, no per-service interface:
+
+```ts
+export function createTenantService(deps: ServiceDeps, services: () => Services) {
+ const { db, logger, queues, clients } = deps;
+
+ // Private helpers first. Not in the returned object, so not on the type.
+ const view = (row: TenantRow): TenantView => ({ ... });
+
+ // Every public method has an EXPLICIT return type.
+ async function get(tenantId: string): Promise<TenantView | null> { ... }
+
+ async function storeMark(tenantId: string, bytes: Uint8Array): Promise<TenantMarkView> {
+ const upload = await services().files.store({ ... }); // a sibling, lazily
+ await clients.storage.put(upload.key, bytes, 'image/png');
+ await queues.tenant.deriveMarkVariants.add({ tenantId });
+ logger.info({ tenantId }, 'tenant mark stored');
+ return view(...);
+ }
+
+ // The return line IS the module's index.
+ return { get, storeMark };
+}
+```
+
+Rules:
+
+- **R3** — the signature is `(deps: ServiceDeps, services: () => Services)`, always. A factory
+ that needs no sibling still takes the thunk, as `_services`, so the composition root stays a
+ flat list. Reach any sibling through `services()`. Two services may call each other.
+- **R4** — declare functions, then `return { one, two, three }`. That line is the index.
+- **R5** — `Services` stays an `interface` whose members are `ReturnType<typeof createXService>`.
+ Never hand-write `export interface XService`.
+
+Both typing rules exist for one reason, and skipping either breaks the build in a confusing way:
+
+1. `Services` must be an **interface**. A type alias over `ReturnType<typeof createServices>` is
+ circular — resolving it needs every factory's signature, and every signature names `Services`.
+ Interface members resolve lazily, which breaks the loop.
+2. Every public method needs an **explicit return type**. TS derives a factory's type from
+ signatures alone; an inferred return type that depends on `services()` forces it to check the
+ body, which references `Services`, which references this factory → `ts7022`/`ts7023`. The error
+ names the offending method, so it is quick to fix — but do not "fix" it by widening a type.
+
+The cost this pays for: a frontend's `tsc` walks the server type graph to resolve `AppRouter`.
+Explicit return types are what keep that tractable.
+
+## Where the graph is built
+
+Rebuilt per unit of work, never once at boot. It is object literals, no I/O — and it is what
+makes `deps.logger` and `deps.queues` already bound to this request or job.
+
+| Unit of work | Who builds it | Logger | Queues |
+|---|---|---|---|
+| procedure | the rpc mount | the request logger | scoped to the request id |
+| plain route | lazily, on first `request.services` read | the request logger | scoped to the request id |
+| job | the worker runtime, per job | child bound to queue/job/attempt/requestId | carries the job's meta onward |
+| queue boot hook | the worker runtime | child bound to the queue | unscoped |
+| the app's boot | `apps/api/src/main.ts` | the boot logger | unscoped |
+
+**R6** — nothing outside those places calls `createServices`. A procedure and a job handler read
+`ctx.services`; a plain route reads `request.services`. **There is no "no context" path.**
+
+MCP and the assistant are not exceptions. They are mounted as ordinary routes
+(`.use(mcpRoutes(deps))`, `.use(assistantRoutes(deps))`) and already hold `deps`. Their tool
+handlers take a fixed SDK signature with no context parameter — that is a *closure* problem, not
+a context problem. Build the server where `deps` is in hand (`createMcpServer(deps)`) and let the
+handlers close over it. Never reach for a module-scope singleton.
+
+**Lazy-load assets, never dependencies.** Reading a MaxMind `.mmdb` from disk on first use is
+fine. Lazily importing a sibling service or a db handle you were already handed is the pattern
+this package spent 2,104 lines of `v1-compat.ts` learning not to do.
+
+## A module
+
+```
+src/modules/<name>/
+ <name>.rpc.ts procedures: schemas + one service call each
+ <name>.service.ts business logic; knows nothing about HTTP or rpc
+ <name>.routes.ts plain routes: webhooks, redirects, probes, file serving
+ <name>.jobs.ts queue: payload schemas + handlers
+ <name>.constants.ts isomorphic vocabulary (see the isomorphism rule above)
+ src/ everything else, plain names — no <name>. prefix inside src/
+```
+
+- **R1** — the `<name>.` prefix stays on root files and stops at `src/`.
+- **R2** — take only the transports that have callers. A health module is routes-only. An
+ integration has `rpc` (our dashboard), `routes` (the provider's webhooks), one `service` under
+ both. Do not add an rpc file with no frontend caller.
+- **R15** — a module joins the app by adding **one line** to each registry it needs
+ (`rpc.router.ts`, `rest.routes.ts`, `jobs.registry.ts`, `services.ts`). Those registries plus
+ the app's boot are the only construction sites in the codebase. Nothing in this package opens a
+ socket, connection or client at import time.
+
+## Layers
+
+**R22** — imports only point downward, by *layer*, not by how many `../` a specifier has:
+
+```
+shared < clients < rpc/base, http/define, jobs/define < modules < services.ts < registries < index.ts
+```
+
+A module importing `defineJob` at `../../jobs/define` is going **down**. That is correct and
+expected.
+
+Two upward edges are legal, both type-only:
+
+1. `ServiceDeps` / `Services` from the composition root — R3 requires them.
+2. Another module's `<name>.constants.ts` — the isomorphism rule blesses that file.
+
+Anything else pointing up is a bug: `shared/` importing `modules/`, or infrastructure
+deep-importing `modules/<name>/src/*` instead of reaching behaviour through `deps`/`ctx`.
+
+`shared/` is for helpers that need nothing above them. If a file in `shared/` needs
+`ServiceDeps`, it is service infrastructure and is in the wrong directory. Truly dumb,
+dependency-free helpers belong in `@openpanel/shared` (**R21**), not here.
+
+## Procedures
+
+A procedure is schemas plus one service call.
+
+- **R10** — the builders carry auth and the common error catalogue. Do not write `requireLogin`
+ or `requireReadAccess` in an rpc file; access checks live in the auth service and the builders.
+- Declare the errors a procedure raises **on that procedure** — that is what types the error
+ payload on the client.
+- Business rules live in the service. The exception is a rule about the caller themself (for
+ example, refusing to let an admin remove their own access), which reads better at the boundary.
+
+## Clients
+
+`src/clients/<name>.ts` is one transport each: config in, typed result or `ProviderError` out.
+
+- **R9** — a client holds **no logger**. It is built once at boot, so it would capture the boot
+ logger and every line would lose its request. Services log; theirs is bound.
+- A missing thing is `null`, not an error.
+- An optional API key means the client still builds and the first call throws. Construct the
+ vendor SDK lazily.
+- **R19** — `ProviderError.retryable` is load-bearing. The client classifies once (429 and 5xx
+ retryable, 4xx not); a job handler rethrows a retryable failure and **logs** a permanent
+ refusal, because a retry cannot change it.
+- A client only one module will ever call may live in that module's `src/` instead.
+
+## Queues
+
+A `.jobs.ts` **declares**; it constructs nothing. No connection exists until the registry reaches
+the producer or worker runtime.
+
+- **R11** — payload schemas and handlers together, `cron`/`schedule` on the job (ADR-021). One
+ way to enqueue, off the context. Enqueue **after** the transaction commits, never inside it.
+- **R18** — a payload must survive a JSON round-trip (no `z.date()`). The worker re-parses it,
+ because a job may have been enqueued by the previous deploy. The wire format is an envelope
+ `{ payload, meta }` so the originating requestId follows a chain of jobs. Schedulers are
+ registered by the **consumer** at worker boot, and retired ones removed. Deploy-owed work runs
+ in a boot hook, not as a job — a job enqueued at boot can be taken by the old instance still
+ draining.
+- Every job runs at least once and may run twice. **The row is the state machine; the job only
+ pokes it.**
+
+## Config and lifecycle
+
+- **R7** — nothing in this package reads `process.env`. New env goes in
+ `apps/api/src/config/env.ts` and `.env.example`, and arrives here as a value.
+- **R17** — that loader is one zod object: cross-field invariants as `.refine` *before* the
+ `.transform`; every derived value (URLs, namespaces, `IS_PRODUCTION`) computed in the transform
+ so nothing downstream re-derives it; a blank `KEY=` treated as absent; and the loader takes the
+ environment **as a parameter** so a test can pass a minimal object.
+- **R16** — whoever opens a connection closes it. This package never closes a connection it was
+ handed. `apps/api/src/main.ts` shuts down in the exact reverse of the order it opened things.
+
+## How a frontend consumes this
+
+- **R12** — a web app imports exactly one thing from this package as a value: a module's
+ `<name>.constants.ts`. Everything else arrives as `import type`, which is erased, so no server
+ code reaches a browser bundle.
+- Enforced by the `frontend-values-only-constants` cruiser rule and the SDK dist gate. If you
+ find yourself needing a runtime helper in the dashboard, it belongs in `@openpanel/shared`
+ (**R21**), not in an exception to this rule.
+
## Tests
`bun:test`, from birth. vitest stays for `apps/start` only.