Comp AI CRM is an open source, CRM designed for AI agents. Agentic-first CRM.
TypeScript
Replace the ported environment doc with the one for this repo
Lewis Carhartca8eae0a6a58
diff --git a/docs/environment.md b/docs/environment.mdindex 04ecfca7..4877d893 100644--- a/docs/environment.md+++ b/docs/environment.md@@ -1,213 +1,82 @@# Environment-One root `.env` is the only env file. Copy `.env.example` to `.env` and fill it-in. There are no per-app or per-package `.env` files.--## How it's loaded--`packages/db/src/env.ts` loads the root `.env` with `dotenv` (never overriding-vars already set in the real environment). Anything that imports `@trycompai/db`-(the API, the better-auth CLI, Prisma via `prisma.config.ts`) gets the root-`.env` from any cwd. `apps/app/next.config.ts` loads it too, so `NEXT_PUBLIC_*`-are inlined at build time.--## Typed, validated env (no scattered `process.env`)--Application config is read through a zod-validated `env` object per app/package,-not via ad-hoc `process.env.X` reads. Each validated module fails fast at first-use with a message naming the bad var:--- `packages/auth/src/env.ts` — Stripe secret/webhook, `REDIS_URL`, better-auth- URLs/origins, `CONTEXT_DEV_API_KEY`-- `packages/email/src/env.ts` — `RESEND_API_KEY`, `EMAIL_FROM`, `EMAIL_APP_URL`-- `apps/api/src/env.ts` — all API secrets/deployment values-- `apps/app/lib/env.ts` — `API_URL`, `NEXT_PUBLIC_POSTHOG_*`--Deliberate exceptions (do NOT add new ad-hoc reads outside these):--- `packages/db/src/env.ts` is the dotenv loader only (side-effect import); it must- not validate/throw, so `import "@trycompai/db/env"` stays safe for test files- that compute a skip condition before touching the DB. `DATABASE_URL` is- validated in `client.ts` where the Prisma client is built.-- `NODE_ENV` is read live via `process.env.NODE_ENV` (not a frozen `env` object)- in `apps/api/src/logging/context-logger.ts` and `packages/db/src/client.ts` — it- is a platform var and must be runtime-togglable (e.g. the logger test sets it).-- `packages/tracking/src/config.ts` reads `process.env` lazily at call time: the- package is imported into Next client bundles (can't pull in `@trycompai/db/env`- → `node:fs`) and is imported before dotenv loads, so a parse-at-import `env`- object would capture empty values.--turbo runs in strict env mode; each cached task declares its `env` keys and `.env`-is a task input so caching invalidates correctly.--## What is NOT an env var (by design)--- **Stripe catalog** (meter event names, per-unit prices, free-seat counts, seat- amounts, price `lookup_key`s) lives in `packages/auth/src/billing.config.ts`.- `unitPriceCents` MUST equal the Stripe metered price's `unit_amount_decimal`.- `stripe:setup` creates and verifies against the catalog — it throws on a metered- unit-price mismatch (including a `null` unit amount) and on a reused seat price- whose graduated tiers (free-seat boundary or per-seat amount) drift from config.-- **Stripe price IDs** are resolved at boot from Stripe by `lookup_key` and cached- in memory (`packages/auth/src/stripe-prices.ts`) — never stored in env.-- **Log format/level** default off `NODE_ENV` (json + log-level in prod, pretty +- all-levels in dev).-- **AI model/gateway URL**: the gateway base URL and default model- (`DEFAULT_CODE_SECURITY_MODEL`) are code constants- (`code-security/shared/contracts.ts`);- the Vercel AI Gateway (`AI_GATEWAY_API_KEY`) is the single AI path. The per-org- model override lives in the DB (`OrganizationModelConfig`, set in **general- workspace settings**) and applies to **every AI call in the app** — code- security scans/reviews, cluster/promote/triage, cloud remediation, and pentest —- via `resolveModel()` in `apps/api/src/ai/model.ts` (default `DEFAULT_AI_MODEL`).- The **selectable model list + per-model billing rates are fetched- live from the gateway `/v1/models` endpoint** (`billing/model-pricing.ts`,- cached in memory) — not env, not a hardcoded list.--## Stripe setup--Create/verify meters, metered prices, and graduated seat prices (with stable-lookup keys) in the target Stripe account:+## Per-app env files, not one root file-```-bun run --filter @trycompai/stripe-dev stripe:setup-```+There is no root `.env`. Every surface that runs as its own process owns its own file, copied from the `.env.example` sitting next to it:++| File | Loaded by | Used by |+| --- | --- | --- |+| `apps/api/.env` | Bun's automatic `.env` loading, plus `ConfigModule.forRoot()` | the NestJS API |+| `apps/app/.env` | Next.js | the Next.js app; `NEXT_PUBLIC_*` are inlined at build time |+| `packages/db/.env` | `import "dotenv/config"` in `packages/db/prisma.config.ts` | the Prisma CLI — `db:migrate`, `db:push`, `db:studio`, `db:seed` |++`packages/auth/.env.example` documents what `@crm/auth` reads, but there is deliberately no `packages/auth/.env`. The package is a library: it never loads a file, it just reads whatever `process.env` its host process already has. If a var is missing there, fix the API's or the app's `.env` — adding one next to the package would do nothing.++Because each loader reads from its own process's working directory, the API only sees `apps/api/.env` when started from `apps/api` — which is what `bun dev` / `turbo run dev` does. Running `bun src/main.ts` from the repo root silently gets you no env at all.++## Values that must match across files++Three variables are duplicated, and nothing checks that the copies agree:++- **`DATABASE_URL`** — `apps/api/.env`, `apps/app/.env`, `packages/db/.env`. The app reads sessions straight from the database, and Prisma's CLI needs its own copy because it runs outside both apps.+- **`BETTER_AUTH_SECRET`** — `apps/api/.env` and `apps/app/.env`.+- **`BETTER_AUTH_URL`** — `apps/api/.env` and `apps/app/.env`.++A mismatched `BETTER_AUTH_SECRET` is the one to watch: the API mints a session cookie the app cannot verify, so `requireSession()` rejects every request and you get an endless bounce back to `/sign-in` with no error anywhere. If sign-in "succeeds" but you land on the sign-in page again, compare the secrets before debugging anything else.++## Typed, validated env++Only the API validates. Everything else fails later and less clearly, so know which layer you are in:++- **`apps/api/src/config/env.validation.ts`** — the real one. A class-validator `EnvironmentVariables` class run through `ConfigModule.forRoot({ validate: validateEnv })`; it fails fast at boot with a message naming each bad variable. New API config belongs here.+- **`packages/db/src/client.ts`** — throws if `DATABASE_URL` is unset, pointing at `packages/db/.env.example`.+- **`packages/auth/src/env.ts`** — no schema. Plain `process.env` reads through an `optional()` helper, with one rule enforced: `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` must be set together or it throws. Anything else missing just becomes `undefined`.+- **`apps/app/lib/api.ts`** — `process.env.NEXT_PUBLIC_API_URL ?? "http://localhost:3001"`. A silent fallback, so a missing var in a deployed app looks like "the API is down" rather than a config error.-Nothing to paste into `.env` — price IDs resolve via lookup key at boot. Run this-against prod Stripe before deploying, or the People plan won't be offered. (The-`mdm_device_v1` IT Security price is no longer created — that module was retired-into People on 2026-07-27; archive the existing Stripe price rather than deleting-it.)+### Two sharp edges-## Database migrations+**`@crm/auth` reads env at import time.** `packages/auth/src/env.ts` builds its `env` object, and `auth.ts` builds `socialProviders`, while the module is being imported — not on first use. `app.module.ts` imports `@crm/auth` before `ConfigModule.forRoot()` runs, so the vars have to already be in `process.env` by then. Under Bun they are, because Bun loads `.env` before executing anything. Under plain Node or `nest start` they would not be: `@nestjs/config` loads the file *after* that import, so Google sign-in would vanish silently (`socialProviders: {}`) while `validateEnv` still passed, because it reads the same variables a moment later. If the API ever moves off Bun, load the env file explicitly before importing `AppModule`.-**Migrations apply themselves on deploy.** The `buildCommand` in-`apps/app/vercel.json` runs `bun run vercel-build`, which is:+**`main.ts` bypasses the validated `PORT`.** It reads `process.env.PORT ?? 3001` directly, so the `@IsInt() @Min(1) @Max(65535)` constraint on `EnvironmentVariables.PORT` never applies to the value actually used. Read it off `ConfigService` instead.++## Turbo and env++Turborepo 2 runs in strict env mode: a task sees only the variables it declares, so **adding a variable to a `.env` file is not enough — the task that needs it must list it too.**++- Root `turbo.json` sets `globalEnv: ["NODE_ENV"]` and makes `.env*` a `build` input, so editing any env file correctly invalidates the build cache.+- `apps/api/turbo.json` declares `passThroughEnv` for `dev` and `test` (auth, database, Google, cache, port).+- `apps/app/turbo.json` declares `env` for the two `NEXT_PUBLIC_*` values — they are build inputs because Next inlines them — and `passThroughEnv` for the server-side ones.+- `packages/db/turbo.json` and `packages/auth/turbo.json` pass `DATABASE_URL` through to every `db:*` / `auth:generate` task.++The split matters: `env` participates in the cache key, `passThroughEnv` does not. Secrets belong in `passThroughEnv` so changing one doesn't churn the cache; anything baked into build output (the `NEXT_PUBLIC_*` pair) belongs in `env`, or Turbo will serve a stale build that inlined the old value.++## Google OAuth++Google is the only sign-in method, wired unconditionally in `packages/auth/src/auth.ts`. Credentials live in `apps/api/.env` only — the app never mounts the auth handler.++Create a web OAuth client in Google Cloud → Credentials, and register the redirect URI:```-prisma generate → db:deploy → trpc codegen → next build+http://localhost:3001/api/auth/callback/google```-The history is a single squashed `20260726140000_init` migration. Every-environment has been baselined onto it, so `migrate deploy` is a no-op until the-next migration is added. Add new migrations with `db:migrate` as normal.+Add the deployed equivalent (`https://<api-host>/api/auth/callback/google`) for each environment. `AUTH_TRUSTED_ORIGINS` is the comma-separated allow-list of origins permitted to call the API with credentials, and doubles as the allow-list Better Auth validates post-sign-in `callbackURL`s against — the Next.js origin belongs there.-**Migrations must not run through the connection pooler.** `DATABASE_URL` points-at Neon's `-pooler` host (PgBouncer, transaction pooling). That is correct for-the app, but DDL and anything session-scoped must use the direct host — drop-`-pooler` from the hostname. Session-level advisory locks in particular are-silently broken through the pooler: the lock is stranded on an arbitrary backend-and the matching unlock lands on a different one, so it is never released and-every later connection waiting on it blocks forever.+## Database-## Admin access (bootstrap)+Prisma is driven through turbo from the repo root: `db:generate`, `db:migrate`, `db:push`, `db:reset`, `db:seed`, `db:studio`, `db:deploy`. Config lives in `packages/db/prisma.config.ts`, which loads `packages/db/.env` itself, so the CLI works without any app running.-The database `admin` role is the only admin path (there is no `ADMIN_USER_IDS`-env). To grant the first admin, set the role directly:+## What is not an env var-```sql-UPDATE "user" SET role = 'admin' WHERE email = '[email protected]';-```+- **Cache TTL default** — `DEFAULT_TTL_MS` (60s) is a constant in `apps/api/src/cache/cache.module.ts`. `CACHE_TTL_MS` only overrides it.+- **Redis** — optional. Without `REDIS_URL` the cache falls back to a per-instance in-memory store, which is fine for local work and wrong for any multi-instance deploy (see `docs/api.md`).+- **Sign-in method** — Google-only, in code, not configurable.++## Secrets hygiene++`.env` files are gitignored, but the patterns are name-exact and inconsistent between packages. Two things to know:++- **Root and `apps/api/.gitignore` list specific names** (`.env`, `.env.local`, `.env.*.local`). A file like `.env.bak` or `.env.old` is **not** ignored and will be committed if you `git add -A`. Never leave a timestamped backup of a real `.env` in the tree.+- **`apps/app/.gitignore` uses `.env*`**, which is broader — and catches the template too. `apps/app/.env.example` is currently untracked and ignored because of it, so a fresh clone gets no template for the web app while the other three packages ship theirs. Fix with a negation (`!.env.example`) in that file.-After that, promote others from the in-app admin panel.--## Advanced / rarely set (not in `.env.example`)--- `PENTEST_AGENT_DIST` — override path to the built pentest agent- (`packages/pentest/dist`); auto-discovered otherwise.-- `BUN_BIN` — bun binary path for `apps/api/scripts/build-func.mjs` (default- `bun`); a build-tool override, not app config.-- `VERCEL`, `NODE_ENV`, `CI` — set by the platform; turbo passes them through.--## Pentest traffic enforcement--Production pentest runs require `REDIS_URL` and `PENTEST_PROXY_BASE_URL` in-addition to the sandbox and Blob variables in `.env.example`.-`PENTEST_PROXY_BASE_URL` is the public HTTPS origin of the API deployment. The-Vercel Sandbox firewall forwards every in-scope HTTPS request to the run-scoped-proxy endpoint at that origin. Redis provides the shared organization, run,-host, and route counters plus the temporary redacted traffic ledger. Runs fail-closed when either dependency is absent.--Vercel Sandbox request forwarding must be enabled for the project plan. HTTP-targets are rejected because the managed firewall forwarding path is HTTPS-only.--`PENTEST_ARTIFACT_RETENTION_DAYS` optionally enforces a 1–3650 day private-artifact retention window through the signed artifact manifest. When unset,-retention remains `not-configured`.--## Cloud Security connectors (keyless customer onboarding)--Both Google connectors are keyless for the customer — nobody creates or downloads-a service-account key on the customer side, so they work under Google's-secure-by-default `constraints/iam.disableServiceAccountKeyCreation` org policy.-The trust model mirrors AWS: a single Comp AI-owned identity, granted access from-the customer side, mints short-lived credentials per scan.--- **AWS** — `AWS_CONNECTOR_ACCESS_KEY_ID` / `AWS_CONNECTOR_SECRET_ACCESS_KEY`- (optional `AWS_CONNECTOR_SESSION_TOKEN`, `AWS_CONNECTOR_REGION`). The customer- creates an IAM role trusting this account with `sts:ExternalId` = their org id;- the API `AssumeRole`s it per scan.-- **GCP + Google Workspace** — `GCP_CONNECTOR_SA_KEY` (the JSON key of Comp AI's- own connector service account). For **GCP**, the customer creates a read-only- auditor SA (no key) and grants the connector SA- `roles/iam.serviceAccountTokenCreator` on it; the API impersonates it to mint- short-lived tokens per scan. For **Workspace**, the customer authorizes the- connector SA's OAuth client ID for domain-wide delegation and the connector- impersonates a delegated admin. The connector SA's email and OAuth client ID- are derived from the key and surfaced to the onboarding UI via- `GET /api/cloud-security/gcp-connector` (and, for People, the- `peopleConnections.googleConnector` tRPC query). Set up the connector SA once- in Comp AI's own GCP project (where key creation is allowed); enabling- `iamcredentials.googleapis.com` on the customer project is part of the- onboarding script.--## One Google Workspace delegation, shared by People and Cloud Security--Both modules connect to the same Workspace tenant through **one** domain-wide-delegation grant. Whichever module's setup wizard the customer completes first-activates both: `WorkspaceConnectionService.activate`-(`apps/api/src/integrations/google-workspace/workspace-connection.service.ts`)-writes the shared `google_workspace_connection` row plus the-`people_provider_connection` row, and the caller links its own-`cloud_connection`. Because the delegation is shared, neither module's layout-gate sends the customer back to a setup wizard once either has been run.--- **Scopes** live in `apps/api/src/integrations/google-workspace/scopes.ts` and- nowhere else. The customer authorizes `WORKSPACE_ALL_SCOPES` (read **and**- write). Both Cloud Security and People request `WORKSPACE_READ_SCOPES` — the- same six read-only scopes Cloud Security always requested, unchanged, so- existing grants keep working. `WORKSPACE_DIRECTORY_SCOPES` (write) is still- probed at activation to record `writeAccessGranted`, but nothing requests a- write token today; People reads the directory and never writes to it.- Both the `.readonly` and the write variant of a scope must be in the grant:- domain-wide delegation matches scope strings exactly, so a grant holding only- `admin.directory.user` rejects a request for `admin.directory.user.readonly`.-- **Cloud Security must never request write scopes.** `mintDelegatedAccessToken`- asserts it for `purpose: "read"`, and- `apps/api/test/google-workspace-delegation.spec.ts` fails if any file under- `apps/api/src/cloud-security/` so much as mentions a write scope or- `purpose: "write"`.-- **Known residual risk:** the scan pipeline writes the connector SA's raw- private key into the Vercel Sandbox for Prowler- (`scan-pipeline.service.ts`, `google_workspace` case). Because the grant now- includes write scopes, that sandbox credential *could* mint a write-scoped- token — the read-only guarantee is enforced in our code, not by the grant. The- fix, if this becomes unacceptable, is a second read-only connector SA whose- client ID is authorized for the read scopes only; the scope-selection seam- (`scopesForPurpose`) is where that would plug in.-- **Domain-wide delegation is the only credential path.** There is no stored- secret: every call mints a short-lived token from the shared delegation. The- old "sign in with a Google super admin" OAuth mode (refresh token in- `people_provider_connection.encryptedPayload`) and its backfill script were- removed along with the rest of the People module; `credentialMode`,- `encryptedPayload`, `scheme`, and `tokenExpiresAt` are gone from the table.-- **Re-authorization.** A delegation refused with `unauthorized_client` /- `access_denied` flips the shared row to `status = "scopes_outdated"` and- `writeAccessGranted = false`, and degrades the People connection; the People- setup wizard is where the customer re-authorizes.--Note: customer org policies that restrict cross-org IAM members-(`constraints/iam.allowedPolicyMemberDomains`, domain-restricted sharing) can-block granting the connector SA on the customer project — this is inherent to any-cross-org GCP access model and is surfaced as a connect error.+`packages/auth/.env.example` ships a filled-in `BETTER_AUTH_SECRET` as an illustration. Treat it as a placeholder, not a value: generate your own with `openssl rand -base64 32`.