Skip to content

armgit / crm

Public Imported from GitHub
0

Comp AI CRM is an open source, CRM designed for AI agents. Agentic-first CRM.

TypeScript

Replace the ported environment doc with the one for this repo

Lewis Carhartca8eae0a6a58
diff --git a/docs/environment.md b/docs/environment.md
index 04ecfca7..4877d893 100644
--- a/docs/environment.md
+++ b/docs/environment.md
@@ -1,213 +1,82 @@
# Environment
-One root `.env` is the only env file. Copy `.env.example` to `.env` and fill it
-in. There are no per-app or per-package `.env` files.
-
-## How it's loaded
-
-`packages/db/src/env.ts` loads the root `.env` with `dotenv` (never overriding
-vars already set in the real environment). Anything that imports `@trycompai/db`
-(the API, the better-auth CLI, Prisma via `prisma.config.ts`) gets the root
-`.env` from any cwd. `apps/app/next.config.ts` loads it too, so `NEXT_PUBLIC_*`
-are inlined at build time.
-
-## Typed, validated env (no scattered `process.env`)
-
-Application config is read through a zod-validated `env` object per app/package,
-not via ad-hoc `process.env.X` reads. Each validated module fails fast at first
-use with a message naming the bad var:
-
-- `packages/auth/src/env.ts` — Stripe secret/webhook, `REDIS_URL`, better-auth
- URLs/origins, `CONTEXT_DEV_API_KEY`
-- `packages/email/src/env.ts` — `RESEND_API_KEY`, `EMAIL_FROM`, `EMAIL_APP_URL`
-- `apps/api/src/env.ts` — all API secrets/deployment values
-- `apps/app/lib/env.ts` — `API_URL`, `NEXT_PUBLIC_POSTHOG_*`
-
-Deliberate exceptions (do NOT add new ad-hoc reads outside these):
-
-- `packages/db/src/env.ts` is the dotenv loader only (side-effect import); it must
- not validate/throw, so `import "@trycompai/db/env"` stays safe for test files
- that compute a skip condition before touching the DB. `DATABASE_URL` is
- validated in `client.ts` where the Prisma client is built.
-- `NODE_ENV` is read live via `process.env.NODE_ENV` (not a frozen `env` object)
- in `apps/api/src/logging/context-logger.ts` and `packages/db/src/client.ts` — it
- is a platform var and must be runtime-togglable (e.g. the logger test sets it).
-- `packages/tracking/src/config.ts` reads `process.env` lazily at call time: the
- package is imported into Next client bundles (can't pull in `@trycompai/db/env`
- → `node:fs`) and is imported before dotenv loads, so a parse-at-import `env`
- object would capture empty values.
-
-turbo runs in strict env mode; each cached task declares its `env` keys and `.env`
-is a task input so caching invalidates correctly.
-
-## What is NOT an env var (by design)
-
-- **Stripe catalog** (meter event names, per-unit prices, free-seat counts, seat
- amounts, price `lookup_key`s) lives in `packages/auth/src/billing.config.ts`.
- `unitPriceCents` MUST equal the Stripe metered price's `unit_amount_decimal`.
- `stripe:setup` creates and verifies against the catalog — it throws on a metered
- unit-price mismatch (including a `null` unit amount) and on a reused seat price
- whose graduated tiers (free-seat boundary or per-seat amount) drift from config.
-- **Stripe price IDs** are resolved at boot from Stripe by `lookup_key` and cached
- in memory (`packages/auth/src/stripe-prices.ts`) — never stored in env.
-- **Log format/level** default off `NODE_ENV` (json + log-level in prod, pretty +
- all-levels in dev).
-- **AI model/gateway URL**: the gateway base URL and default model
- (`DEFAULT_CODE_SECURITY_MODEL`) are code constants
- (`code-security/shared/contracts.ts`);
- the Vercel AI Gateway (`AI_GATEWAY_API_KEY`) is the single AI path. The per-org
- model override lives in the DB (`OrganizationModelConfig`, set in **general
- workspace settings**) and applies to **every AI call in the app** — code
- security scans/reviews, cluster/promote/triage, cloud remediation, and pentest —
- via `resolveModel()` in `apps/api/src/ai/model.ts` (default `DEFAULT_AI_MODEL`).
- The **selectable model list + per-model billing rates are fetched
- live from the gateway `/v1/models` endpoint** (`billing/model-pricing.ts`,
- cached in memory) — not env, not a hardcoded list.
-
-## Stripe setup
-
-Create/verify meters, metered prices, and graduated seat prices (with stable
-lookup keys) in the target Stripe account:
+## Per-app env files, not one root file
-```
-bun run --filter @trycompai/stripe-dev stripe:setup
-```
+There is no root `.env`. Every surface that runs as its own process owns its own file, copied from the `.env.example` sitting next to it:
+
+| File | Loaded by | Used by |
+| --- | --- | --- |
+| `apps/api/.env` | Bun's automatic `.env` loading, plus `ConfigModule.forRoot()` | the NestJS API |
+| `apps/app/.env` | Next.js | the Next.js app; `NEXT_PUBLIC_*` are inlined at build time |
+| `packages/db/.env` | `import "dotenv/config"` in `packages/db/prisma.config.ts` | the Prisma CLI — `db:migrate`, `db:push`, `db:studio`, `db:seed` |
+
+`packages/auth/.env.example` documents what `@crm/auth` reads, but there is deliberately no `packages/auth/.env`. The package is a library: it never loads a file, it just reads whatever `process.env` its host process already has. If a var is missing there, fix the API's or the app's `.env` — adding one next to the package would do nothing.
+
+Because each loader reads from its own process's working directory, the API only sees `apps/api/.env` when started from `apps/api` — which is what `bun dev` / `turbo run dev` does. Running `bun src/main.ts` from the repo root silently gets you no env at all.
+
+## Values that must match across files
+
+Three variables are duplicated, and nothing checks that the copies agree:
+
+- **`DATABASE_URL`** — `apps/api/.env`, `apps/app/.env`, `packages/db/.env`. The app reads sessions straight from the database, and Prisma's CLI needs its own copy because it runs outside both apps.
+- **`BETTER_AUTH_SECRET`** — `apps/api/.env` and `apps/app/.env`.
+- **`BETTER_AUTH_URL`** — `apps/api/.env` and `apps/app/.env`.
+
+A mismatched `BETTER_AUTH_SECRET` is the one to watch: the API mints a session cookie the app cannot verify, so `requireSession()` rejects every request and you get an endless bounce back to `/sign-in` with no error anywhere. If sign-in "succeeds" but you land on the sign-in page again, compare the secrets before debugging anything else.
+
+## Typed, validated env
+
+Only the API validates. Everything else fails later and less clearly, so know which layer you are in:
+
+- **`apps/api/src/config/env.validation.ts`** — the real one. A class-validator `EnvironmentVariables` class run through `ConfigModule.forRoot({ validate: validateEnv })`; it fails fast at boot with a message naming each bad variable. New API config belongs here.
+- **`packages/db/src/client.ts`** — throws if `DATABASE_URL` is unset, pointing at `packages/db/.env.example`.
+- **`packages/auth/src/env.ts`** — no schema. Plain `process.env` reads through an `optional()` helper, with one rule enforced: `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` must be set together or it throws. Anything else missing just becomes `undefined`.
+- **`apps/app/lib/api.ts`** — `process.env.NEXT_PUBLIC_API_URL ?? "http://localhost:3001"`. A silent fallback, so a missing var in a deployed app looks like "the API is down" rather than a config error.
-Nothing to paste into `.env` — price IDs resolve via lookup key at boot. Run this
-against prod Stripe before deploying, or the People plan won't be offered. (The
-`mdm_device_v1` IT Security price is no longer created — that module was retired
-into People on 2026-07-27; archive the existing Stripe price rather than deleting
-it.)
+### Two sharp edges
-## Database migrations
+**`@crm/auth` reads env at import time.** `packages/auth/src/env.ts` builds its `env` object, and `auth.ts` builds `socialProviders`, while the module is being imported — not on first use. `app.module.ts` imports `@crm/auth` before `ConfigModule.forRoot()` runs, so the vars have to already be in `process.env` by then. Under Bun they are, because Bun loads `.env` before executing anything. Under plain Node or `nest start` they would not be: `@nestjs/config` loads the file *after* that import, so Google sign-in would vanish silently (`socialProviders: {}`) while `validateEnv` still passed, because it reads the same variables a moment later. If the API ever moves off Bun, load the env file explicitly before importing `AppModule`.
-**Migrations apply themselves on deploy.** The `buildCommand` in
-`apps/app/vercel.json` runs `bun run vercel-build`, which is:
+**`main.ts` bypasses the validated `PORT`.** It reads `process.env.PORT ?? 3001` directly, so the `@IsInt() @Min(1) @Max(65535)` constraint on `EnvironmentVariables.PORT` never applies to the value actually used. Read it off `ConfigService` instead.
+
+## Turbo and env
+
+Turborepo 2 runs in strict env mode: a task sees only the variables it declares, so **adding a variable to a `.env` file is not enough — the task that needs it must list it too.**
+
+- Root `turbo.json` sets `globalEnv: ["NODE_ENV"]` and makes `.env*` a `build` input, so editing any env file correctly invalidates the build cache.
+- `apps/api/turbo.json` declares `passThroughEnv` for `dev` and `test` (auth, database, Google, cache, port).
+- `apps/app/turbo.json` declares `env` for the two `NEXT_PUBLIC_*` values — they are build inputs because Next inlines them — and `passThroughEnv` for the server-side ones.
+- `packages/db/turbo.json` and `packages/auth/turbo.json` pass `DATABASE_URL` through to every `db:*` / `auth:generate` task.
+
+The split matters: `env` participates in the cache key, `passThroughEnv` does not. Secrets belong in `passThroughEnv` so changing one doesn't churn the cache; anything baked into build output (the `NEXT_PUBLIC_*` pair) belongs in `env`, or Turbo will serve a stale build that inlined the old value.
+
+## Google OAuth
+
+Google is the only sign-in method, wired unconditionally in `packages/auth/src/auth.ts`. Credentials live in `apps/api/.env` only — the app never mounts the auth handler.
+
+Create a web OAuth client in Google Cloud → Credentials, and register the redirect URI:
```
-prisma generate → db:deploy → trpc codegen → next build
+http://localhost:3001/api/auth/callback/google
```
-The history is a single squashed `20260726140000_init` migration. Every
-environment has been baselined onto it, so `migrate deploy` is a no-op until the
-next migration is added. Add new migrations with `db:migrate` as normal.
+Add the deployed equivalent (`https://<api-host>/api/auth/callback/google`) for each environment. `AUTH_TRUSTED_ORIGINS` is the comma-separated allow-list of origins permitted to call the API with credentials, and doubles as the allow-list Better Auth validates post-sign-in `callbackURL`s against — the Next.js origin belongs there.
-**Migrations must not run through the connection pooler.** `DATABASE_URL` points
-at Neon's `-pooler` host (PgBouncer, transaction pooling). That is correct for
-the app, but DDL and anything session-scoped must use the direct host — drop
-`-pooler` from the hostname. Session-level advisory locks in particular are
-silently broken through the pooler: the lock is stranded on an arbitrary backend
-and the matching unlock lands on a different one, so it is never released and
-every later connection waiting on it blocks forever.
+## Database
-## Admin access (bootstrap)
+Prisma is driven through turbo from the repo root: `db:generate`, `db:migrate`, `db:push`, `db:reset`, `db:seed`, `db:studio`, `db:deploy`. Config lives in `packages/db/prisma.config.ts`, which loads `packages/db/.env` itself, so the CLI works without any app running.
-The database `admin` role is the only admin path (there is no `ADMIN_USER_IDS`
-env). To grant the first admin, set the role directly:
+## What is not an env var
-```sql
-UPDATE "user" SET role = 'admin' WHERE email = '[email protected]';
-```
+- **Cache TTL default** — `DEFAULT_TTL_MS` (60s) is a constant in `apps/api/src/cache/cache.module.ts`. `CACHE_TTL_MS` only overrides it.
+- **Redis** — optional. Without `REDIS_URL` the cache falls back to a per-instance in-memory store, which is fine for local work and wrong for any multi-instance deploy (see `docs/api.md`).
+- **Sign-in method** — Google-only, in code, not configurable.
+
+## Secrets hygiene
+
+`.env` files are gitignored, but the patterns are name-exact and inconsistent between packages. Two things to know:
+
+- **Root and `apps/api/.gitignore` list specific names** (`.env`, `.env.local`, `.env.*.local`). A file like `.env.bak` or `.env.old` is **not** ignored and will be committed if you `git add -A`. Never leave a timestamped backup of a real `.env` in the tree.
+- **`apps/app/.gitignore` uses `.env*`**, which is broader — and catches the template too. `apps/app/.env.example` is currently untracked and ignored because of it, so a fresh clone gets no template for the web app while the other three packages ship theirs. Fix with a negation (`!.env.example`) in that file.
-After that, promote others from the in-app admin panel.
-
-## Advanced / rarely set (not in `.env.example`)
-
-- `PENTEST_AGENT_DIST` — override path to the built pentest agent
- (`packages/pentest/dist`); auto-discovered otherwise.
-- `BUN_BIN` — bun binary path for `apps/api/scripts/build-func.mjs` (default
- `bun`); a build-tool override, not app config.
-- `VERCEL`, `NODE_ENV`, `CI` — set by the platform; turbo passes them through.
-
-## Pentest traffic enforcement
-
-Production pentest runs require `REDIS_URL` and `PENTEST_PROXY_BASE_URL` in
-addition to the sandbox and Blob variables in `.env.example`.
-`PENTEST_PROXY_BASE_URL` is the public HTTPS origin of the API deployment. The
-Vercel Sandbox firewall forwards every in-scope HTTPS request to the run-scoped
-proxy endpoint at that origin. Redis provides the shared organization, run,
-host, and route counters plus the temporary redacted traffic ledger. Runs fail
-closed when either dependency is absent.
-
-Vercel Sandbox request forwarding must be enabled for the project plan. HTTP
-targets are rejected because the managed firewall forwarding path is HTTPS-only.
-
-`PENTEST_ARTIFACT_RETENTION_DAYS` optionally enforces a 1–3650 day private
-artifact retention window through the signed artifact manifest. When unset,
-retention remains `not-configured`.
-
-## Cloud Security connectors (keyless customer onboarding)
-
-Both Google connectors are keyless for the customer — nobody creates or downloads
-a service-account key on the customer side, so they work under Google's
-secure-by-default `constraints/iam.disableServiceAccountKeyCreation` org policy.
-The trust model mirrors AWS: a single Comp AI-owned identity, granted access from
-the customer side, mints short-lived credentials per scan.
-
-- **AWS** — `AWS_CONNECTOR_ACCESS_KEY_ID` / `AWS_CONNECTOR_SECRET_ACCESS_KEY`
- (optional `AWS_CONNECTOR_SESSION_TOKEN`, `AWS_CONNECTOR_REGION`). The customer
- creates an IAM role trusting this account with `sts:ExternalId` = their org id;
- the API `AssumeRole`s it per scan.
-- **GCP + Google Workspace** — `GCP_CONNECTOR_SA_KEY` (the JSON key of Comp AI's
- own connector service account). For **GCP**, the customer creates a read-only
- auditor SA (no key) and grants the connector SA
- `roles/iam.serviceAccountTokenCreator` on it; the API impersonates it to mint
- short-lived tokens per scan. For **Workspace**, the customer authorizes the
- connector SA's OAuth client ID for domain-wide delegation and the connector
- impersonates a delegated admin. The connector SA's email and OAuth client ID
- are derived from the key and surfaced to the onboarding UI via
- `GET /api/cloud-security/gcp-connector` (and, for People, the
- `peopleConnections.googleConnector` tRPC query). Set up the connector SA once
- in Comp AI's own GCP project (where key creation is allowed); enabling
- `iamcredentials.googleapis.com` on the customer project is part of the
- onboarding script.
-
-## One Google Workspace delegation, shared by People and Cloud Security
-
-Both modules connect to the same Workspace tenant through **one** domain-wide
-delegation grant. Whichever module's setup wizard the customer completes first
-activates both: `WorkspaceConnectionService.activate`
-(`apps/api/src/integrations/google-workspace/workspace-connection.service.ts`)
-writes the shared `google_workspace_connection` row plus the
-`people_provider_connection` row, and the caller links its own
-`cloud_connection`. Because the delegation is shared, neither module's layout
-gate sends the customer back to a setup wizard once either has been run.
-
-- **Scopes** live in `apps/api/src/integrations/google-workspace/scopes.ts` and
- nowhere else. The customer authorizes `WORKSPACE_ALL_SCOPES` (read **and**
- write). Both Cloud Security and People request `WORKSPACE_READ_SCOPES` — the
- same six read-only scopes Cloud Security always requested, unchanged, so
- existing grants keep working. `WORKSPACE_DIRECTORY_SCOPES` (write) is still
- probed at activation to record `writeAccessGranted`, but nothing requests a
- write token today; People reads the directory and never writes to it.
- Both the `.readonly` and the write variant of a scope must be in the grant:
- domain-wide delegation matches scope strings exactly, so a grant holding only
- `admin.directory.user` rejects a request for `admin.directory.user.readonly`.
-- **Cloud Security must never request write scopes.** `mintDelegatedAccessToken`
- asserts it for `purpose: "read"`, and
- `apps/api/test/google-workspace-delegation.spec.ts` fails if any file under
- `apps/api/src/cloud-security/` so much as mentions a write scope or
- `purpose: "write"`.
-- **Known residual risk:** the scan pipeline writes the connector SA's raw
- private key into the Vercel Sandbox for Prowler
- (`scan-pipeline.service.ts`, `google_workspace` case). Because the grant now
- includes write scopes, that sandbox credential *could* mint a write-scoped
- token — the read-only guarantee is enforced in our code, not by the grant. The
- fix, if this becomes unacceptable, is a second read-only connector SA whose
- client ID is authorized for the read scopes only; the scope-selection seam
- (`scopesForPurpose`) is where that would plug in.
-- **Domain-wide delegation is the only credential path.** There is no stored
- secret: every call mints a short-lived token from the shared delegation. The
- old "sign in with a Google super admin" OAuth mode (refresh token in
- `people_provider_connection.encryptedPayload`) and its backfill script were
- removed along with the rest of the People module; `credentialMode`,
- `encryptedPayload`, `scheme`, and `tokenExpiresAt` are gone from the table.
-- **Re-authorization.** A delegation refused with `unauthorized_client` /
- `access_denied` flips the shared row to `status = "scopes_outdated"` and
- `writeAccessGranted = false`, and degrades the People connection; the People
- setup wizard is where the customer re-authorizes.
-
-Note: customer org policies that restrict cross-org IAM members
-(`constraints/iam.allowedPolicyMemberDomains`, domain-restricted sharing) can
-block granting the connector SA on the customer project — this is inherent to any
-cross-org GCP access model and is surfaced as a connect error.
+`packages/auth/.env.example` ships a filled-in `BETTER_AUTH_SECRET` as an illustration. Treat it as a placeholder, not a value: generate your own with `openssl rand -base64 32`.