OpenPanel is an open-source web and product analytics platform, an open-source alternative to Mixpanel with optional self-hosting.
ralph(M13-003): Both Dockerfiles on bun install: no corepack, no pnpm; the dashboard image's stale packages/auth + @openpanel/tsconfig breakage fixed; both images build and boot; sizes vs the public images recorded
ralph0407d33e1ab1
diff --git a/.dockerignore b/.dockerignoreindex 2ff6521c4..7301ad57d 100644--- a/.dockerignore+++ b/.dockerignore@@ -9,4 +9,19 @@ README.md.next.gitdocker-converage\ No newline at end of file+converage++# P13 (M13-003): both images now COPY the whole workspace, because a bun+# workspace install has to see every member's manifest and the old+# hand-maintained per-package COPY list drifted silently. That makes the build+# context worth trimming — everything below is either regenerated inside the+# image or never read by it.+**/.next+**/.output+**/.turbo+**/*.tsbuildinfo+# Regenerated by `bun run codegen` in the api image's build stage; shipping the+# host's copies risks baking a stale Prisma client or an old geo database.+packages/db/src/generated+**/GeoLite2-*.mmdb+tmpdiff --git a/apps/api/Dockerfile b/apps/api/Dockerfileindex 3c526b209..fca89b5c4 100644--- a/apps/api/Dockerfile+++ b/apps/api/Dockerfile@@ -2,16 +2,23 @@# `.bun-version` + this ARG; the file does not exist in the tree yet, so this# ARG is the single home until it does. Keep the two in sync when it lands.ARG BUN_VERSION=1.4.0-# ADR-014: pnpm stays through P12 — this only pins the pnpm *version* used to-# install the workspace, it does not touch which tool runs installs. P9.5-# stops that installer reaching the runtime stage at all (ADR-014 Problems-# §4, option 2): pnpm and the build toolchain live only in `build`/`prod-deps`-# below, neither of which ships.-ARG PNPM_VERSION=11.23.0++# P13 (ADR-014) made `bun install` the installer for the whole repo, so this+# image installs with bun and nothing else: no second package manager, no+# Node-side installer shim, no lockfile or workspace file bun does not read.+# ADR-014 Problems §4 already ruled that a bun-base image cannot bootstrap+# another installer, and its answer is the shape below — ONE full-workspace+# install per image, from the root manifest bun.lock was resolved against.+#+# `bunfig.toml` is copied with the tree on purpose: it carries+# `[install] ignoreScripts = true` (ADR-014 amendment 2 — no dependency build+# script has ever run in an image this repo ships) and `linker = "isolated"`.+# The two install lines below name `--linker=isolated` explicitly anyway, so+# the layout is a property of the command and not only of a file.# BASE — OS packages every stage needs. curl and netcat-openbsd are here for# the docker-compose HEALTHCHECKs, which exec *inside* the running container,-# so they must reach the runtime stage too, not just the pnpm stages.+# so they must reach the runtime stage too, not just the install stages.FROM oven/bun:${BUN_VERSION}-slim AS baseRUN apt-get update && \@@ -24,100 +31,65 @@ RUN apt-get update && \&& apt-get clean && \rm -rf /var/lib/apt/lists/*-# TOOLCHAIN — pnpm + the native-addon build toolchain, shared by the two-# stages below. Neither of them ships, so neither does this.+# TOOLCHAIN — the full workspace, installed once with devDependencies, because+# codegen needs them (core's `tar` for scripts/download-geo.ts, db's `prisma`+# CLI for schema generation). Neither this stage nor `build` ships.+#+# The whole tree is copied rather than a hand-maintained list of manifests:+# a workspace install has to see every member's `package.json` anyway, Docker's+# COPY cannot glob a directory level, and the old per-package list failed+# SILENTLY when it drifted (a missing manifest omitted a package from+# node_modules and surfaced as a runtime "Cannot find module"). The cost is+# that a source edit invalidates the install layer; the gate builds from+# scratch regardless.+#+# No python3/make/g++: with `ignoreScripts` nothing compiles at install time,+# and the two native addons this image needs (@node-rs/argon2, sharp) resolve+# to prebuilt linux-x64 packages.FROM base AS toolchain-RUN apt-get update && apt-get install -y --no-install-recommends \- python3 \- make \- g++ \- && apt-get clean && \- rm -rf /var/lib/apt/lists/*--ARG PNPM_VERSION-RUN bun install -g pnpm@${PNPM_VERSION}--ENV PNPM_HOME="/pnpm"-ENV PATH="$PNPM_HOME:$PATH"-ENV npm_config_build_from_source=true-WORKDIR /app-# Workspace manifest-COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./-# Apps-COPY apps/api/package.json ./apps/api/-# Packages — @openpanel/api's workspace closure, exactly as pnpm computes it-# (verified against `pnpm --filter "@openpanel/api..." list --depth -1`: these-# 9, plus the private tooling/typescript devDependency, which --prod below-# never installs, so its manifest is deliberately not copied here). This list-# is still hand-maintained: Docker's COPY only accepts a glob for the *last*-# path segment, so `packages/*/package.json` flattens every match into one-# `packages/package.json`, overwriting instead of preserving each-# subdirectory — there is no wildcard that copies these into their own-# directories. `--filter "@openpanel/api..."` below is the actual-# closure boundary; if a manifest is missing here, the install below does-# NOT fail loud — it silently omits that package from `node_modules`-# (surfacing later as a runtime "Cannot find module", not a build failure) —-# so keep this list in sync with the filter output when api's dependency-# graph changes.-COPY packages/core/package.json packages/core/-COPY packages/db/package.json packages/db/-COPY packages/email/package.json packages/email/-COPY packages/redis/package.json packages/redis/-COPY packages/payments/package.json packages/payments/-COPY patches ./patches--# BUILD — full install (codegen needs devDependencies: core's `tar` for-# scripts/download-geo.ts, db's `prisma` CLI for schema generation) then the-# Prisma client + geo/ASN codegen. Never ships: only its generated *source*-# output (baked into packages/db and packages/core below) reaches runtime.-FROM toolchain AS build+COPY . .-RUN pnpm install --frozen-lockfile && \- pnpm store prune+RUN bun install --frozen-lockfile --linker=isolated-COPY apps/api ./apps/api-COPY packages ./packages-COPY tooling ./tooling+# BUILD — the Prisma client and core's geo/ASN data. No JS build step: ADR-010+# deleted the backend bundler with the V1 boot, so the only thing this stage+# produces is generated *source* under packages/, which `prod-deps` inherits.+FROM toolchain AS buildARG DATABASE_URLENV DATABASE_URL=$DATABASE_URL-# No JS build step: ADR-010 deleted the backend bundler with the V1 boot, so-# the only thing this stage produces is the generated Prisma client and-# core's downloaded geo/ASN data, both written to disk under packages/.-RUN pnpm codegen+RUN bun run codegen-# PROD-DEPS — a from-scratch production-only install. Deliberately NOT-# derived from `build`'s node_modules: pnpm's --prod install does not reliably-# purge an existing devDependency-laden modules dir (verified empirically —-# it left every devDependency in place), so shipping node_modules must come-# from an install that was production-only from the start.+# PROD-DEPS — the node_modules that ships. Derived from `build` so the+# generated output above survives, but with every node_modules directory+# deleted first: a `--production` install must not be a filtered view of a+# devDependency-laden tree, it must be an install that was production-only+# from the start.#-# `--filter "@openpanel/api..."` (the three-dot suffix: api plus everything it-# depends on, direct or transitive) is the actual closure boundary, chosen-# over `pnpm deploy` — deploy needs either `inject-workspace-packages=true`-# (a repo-wide install-behavior change, out of scope here) or `--legacy`-# (which re-resolves the full 26-project lockfile graph every build, ~8x-# slower than a scoped install, for the same output). Verified empirically-# against this workspace: with only the manifests above present, plain-# `pnpm install --frozen-lockfile --prod` and this filtered form produce-# byte-identical node_modules (same 1039 packages) — the manifest list-# already happened to equal the closure. The filter is kept anyway as the-# explicit, fails-closed boundary: it is what makes "only api's closure"-# a property of the command instead of an accident of which package.json-# files got copied above.-FROM toolchain AS prod-deps--RUN pnpm --filter "@openpanel/api..." install --frozen-lockfile --prod && \- pnpm rebuild && \- pnpm store prune--# RUNTIME — oven/bun:${BUN_VERSION}-slim, nothing but the app: node_modules-# from `prod-deps`, source (with `build`'s generated output baked in) from-# `build`. No pnpm, no python3/make/g++, no per-package COPY manifest.+# `--filter '@openpanel/api'` is the closure boundary, and it is the+# "equivalent" a bare `bun install --production --frozen-lockfile` is traded+# for. The bare form installs every workspace member's production closure —+# apps/start's React/Nitro tree included, ~1.1GB the API never loads+# (docs/TECH_DEBT.md §5d). Verified on Bun 1.4.0: `--filter` follows+# `workspace:` edges TRANSITIVELY (api -> core -> db -> redis/email/payments)+# and installs nothing outside them, which is exactly the boundary the+# previous image expressed with its own installer's `...` filter suffix. It+# fails visibly rather than silently: a filter that matches nothing leaves the+# app unable to resolve its imports, and tooling/gates/p13-images.sh boots the+# image and asserts /healthz/ready before it passes.+FROM build AS prod-deps++RUN find . -maxdepth 4 -type d -name node_modules -prune -exec rm -rf {} + && \+ bun install --production --frozen-lockfile --linker=isolated --filter '@openpanel/api'++# RUNTIME — oven/bun:${BUN_VERSION}-slim, nothing but the app: the isolated+# node_modules and the source (with `build`'s generated output baked in), both+# from `prod-deps` so the store and the per-package symlinks into it come from+# one install and stay consistent.FROM base AS runtimeENV NODE_ENV=production@@ -128,11 +100,10 @@ COPY --from=prod-deps /app/package.json ./COPY --from=prod-deps /app/node_modules ./node_modules# Apps — src/ is the artifact now; there is no dist/.-COPY --from=build /app/apps/api ./apps/api+COPY --from=prod-deps /app/apps/api ./apps/api-# Packages — one COPY, not the per-package manifest the pnpm stages needed.-COPY --from=build /app/packages ./packages-COPY --from=build /app/tooling/typescript ./tooling/typescript+COPY --from=prod-deps /app/packages ./packages+COPY --from=prod-deps /app/tooling/typescript ./tooling/typescriptWORKDIR /app/apps/apidiff --git a/apps/start/Dockerfile b/apps/start/Dockerfileindex 4265bad36..23c07f837 100644--- a/apps/start/Dockerfile+++ b/apps/start/Dockerfile@@ -1,11 +1,42 @@+# Dashboard image.+#+# THE TWO ROOT CAUSES THIS FILE FIXES. Measured at 4925d2f7 (2026-09-07) this+# image did not build, and neither break came from P13:+#+# 1. STALE COPIES OF THE DELETED `@openpanel/auth` PACKAGE. Two lines copied+# its manifest and its whole directory out of the build stage. ADR-007+# dissolved that package into `@openpanel/core` and P11 deleted its+# directory, so both COPYs referenced a path that no longer exists and+# the build would have died on them. The grep gate for this file asserts+# that neither the old installer nor that path is named anywhere in it.+#+# 2. THE WORKSPACE-SUBSET INSTALL. The old base stage copied ~10 of the+# workspace's ~20 member manifests and none of `packages/core`, which was+# fine while the dashboard imported nothing from it. P11 made+# `apps/start` import `@openpanel/core/modules/*.constants`, and+# `vite-tsconfig-paths` then reads `packages/core/tsconfig.json`, whose+# `"extends": "@openpanel/tsconfig/base.json"` cannot resolve when+# `tooling/typescript` was never installed as a workspace member:+# `failed to resolve "extends":"@openpanel/tsconfig/base.json" in+# /app/packages/core/tsconfig.json`. The fix is not another manifest in+# the list — it is ONE FULL WORKSPACE INSTALL, which is the shape ADR-014+# Problems §4 rules for a bun-base image anyway. A subset install is what+# the deleted `verifyDepsBeforeRun: false` setting existed to paper over;+# that setting has no successor, so the subset it protected has to go.+#+# ADR-014 scopes P13 to the INSTALLER only. The build runs on bun; the runtime+# deliberately stays Node — nothing here decides otherwise.+ARG BUN_VERSION=1.4.0ARG NODE_VERSION=24.19.0-FROM node:${NODE_VERSION}-slim AS base+# INSTALL — one full workspace install, exactly what bun.lock was resolved+# against. `bunfig.toml` rides along with the tree and carries+# `[install] ignoreScripts = true` (ADR-014 amendment 2) and+# `linker = "isolated"`; `--linker=isolated` is named here too so the layout is+# a property of the command, not only of a file.+FROM oven/bun:${BUN_VERSION}-slim AS install-# FIX: Bad workaround (https://github.com/nodejs/corepack/issues/612)-ENV COREPACK_INTEGRITY_KEYS=0--RUN corepack enable && apt-get update && \+RUN apt-get update && \apt-get install -y --no-install-recommends \ca-certificates \openssl \@@ -14,99 +45,56 @@ RUN corepack enable && apt-get update && \&& apt-get clean && \rm -rf /var/lib/apt/lists/*-ENV NITRO=1-ENV SELF_HOSTED=1-ENV PNPM_HOME="/pnpm"-ENV PATH="$PNPM_HOME:$PATH"-WORKDIR /app-# Workspace - Copy package.json files for caching-COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./-# Apps-COPY apps/start/package.json ./apps/start/-# Packages - Only copy what start app needs-COPY packages/payments/package.json packages/payments/-COPY packages/sdks/_info/package.json packages/sdks/_info/-COPY patches ./patches-# Copy tracking script to self-hosting dashboard-COPY apps/public/public/op1.js ./apps/start/public/op1.js--# BUILD-FROM base AS build--RUN apt-get update && apt-get install -y --no-install-recommends \- python3 \- make \- g++ && \- apt-get clean && \- rm -rf /var/lib/apt/lists/*+COPY . .-WORKDIR /app--# Install all dependencies (including dev dependencies for build)-RUN pnpm install --frozen-lockfile && \- pnpm store prune+# The tracking script the self-hosted dashboard serves.+COPY apps/public/public/op1.js ./apps/start/public/op1.js-# Copy source code-COPY apps/start ./apps/start-COPY packages ./packages-COPY tooling ./tooling+RUN bun install --frozen-lockfile --linker=isolated-# Generate Prisma client and build the app-RUN pnpm --filter start run build+# BUILD — Nitro's standalone server output. NITRO=1 selects the node-server+# preset over the Cloudflare one; SELF_HOSTED=1 is read at build time.+#+# No codegen step: verified on this tree (2026-09-07) that the build succeeds+# with `packages/db/src/generated` absent — every `@openpanel/db` import in+# apps/start is `import type`, and the `@openpanel/core` imports are all+# `*.constants` subpaths.+FROM install AS build-# PROD - Install only production dependencies-FROM base AS prod+ENV NITRO=1+ENV SELF_HOSTED=1-ENV npm_config_build_from_source=true+RUN cd apps/start && NITRO=1 SELF_HOSTED=1 bun run build-RUN apt-get update && apt-get install -y --no-install-recommends \- python3 \- make \- g++ && \- apt-get clean && \- rm -rf /var/lib/apt/lists/*--WORKDIR /app-COPY --from=build /app/package.json ./-COPY --from=build /app/pnpm-lock.yaml ./-COPY --from=build /app/pnpm-workspace.yaml ./--# Copy package.json files for production install-COPY --from=build /app/apps/start/package.json ./apps/start/-COPY --from=build /app/packages/db/package.json ./packages/db/-COPY --from=build /app/packages/auth/package.json ./packages/auth/-COPY --from=build /app/packages/payments/package.json ./packages/payments/-COPY --from=build /app/packages/sdks/_info/package.json ./packages/sdks/_info/-COPY --from=build /app/patches ./patches--# Install production dependencies only-RUN pnpm install --frozen-lockfile --prod && \- pnpm rebuild && \- pnpm store prune--# FINAL - Minimal runtime image-FROM base AS runner+# RUNNER — node:${NODE_VERSION}-slim, and only `.output`.+#+# The Nitro output is standalone: it ships its own `.output/server/node_modules`+# (222 packages) and `.output/public` holds every asset `/login` references.+# Verified by running `.output/server/index.mjs` from an otherwise empty+# directory on 2026-09-07: `/api/healthcheck` 200, `/login` 200 / 136710 bytes,+# no resolution errors. So the workspace `node_modules` and the per-package+# source copies the previous image carried are dead weight, and dropping them+# is what removes the last reference to the deleted `@openpanel/auth` tree.+FROM node:${NODE_VERSION}-slim AS runnerENV NODE_ENV=production-ENV npm_config_build_from_source=true--WORKDIR /app-# Copy workspace files-COPY --from=build /app/package.json ./-COPY --from=build /app/pnpm-workspace.yaml ./+RUN apt-get update && \+ apt-get install -y --no-install-recommends \+ ca-certificates \+ openssl \+ libssl3 \+ curl \+ && apt-get clean && \+ rm -rf /var/lib/apt/lists/*-# Copy production node_modules-COPY --from=prod /app/node_modules ./node_modules+WORKDIR /app/apps/start-# Copy built app with .output directory-COPY --from=build /app/apps/start/.output ./apps/start/.output-COPY --from=build /app/apps/start/dist ./apps/start/dist-COPY --from=build /app/apps/start/package.json ./apps/start/+COPY --from=build /app/apps/start/.output ./.output-# The standalone Nitro output preserves the target package name for the pnpm+# The standalone Nitro output preserves the target package name for the# esm-env -> esm-env-runtime override. @number-flow/react imports "esm-env"# at runtime, so keep that package name resolvable inside the SSR bundle.RUN set -eux; \@@ -127,15 +115,6 @@ RUN set -eux; \cd /app/apps/start/.output/server; \node --input-type=module -e 'const assert = await import("node:assert/strict"); const env = await import("esm-env"); assert.equal(env.BROWSER, false, "esm-env BROWSER should be false in the Node SSR runtime"); assert.equal(env.NODE, true, "esm-env NODE should be true in the Node SSR runtime"); assert.equal(env.DEV, false, "esm-env DEV should be false in the production runtime");'-# Copy necessary packages-COPY --from=build /app/packages/db ./packages/db-COPY --from=build /app/packages/auth ./packages/auth-COPY --from=build /app/packages/payments ./packages/payments-COPY --from=build /app/packages/sdks/_info ./packages/sdks/_info-COPY --from=build /app/tooling/typescript ./tooling/typescript--WORKDIR /app/apps/start-EXPOSE 3000# Start the Tanstack Start serverdiff --git a/docs/OPS_DEPLOY_MIGRATION.md b/docs/OPS_DEPLOY_MIGRATION.mdindex b8b48ce39..2d9964f11 100644--- a/docs/OPS_DEPLOY_MIGRATION.md+++ b/docs/OPS_DEPLOY_MIGRATION.md@@ -94,6 +94,10 @@ delta to report — only this absolute number.## P9.5 slim-down (M9-006): pnpm and the build toolchain leave the runtime image+> **Superseded by *P13 (M13-003)* at the end of this file.** Both Dockerfiles+> now install with bun and this section's stage list no longer matches the tree.+> Kept as the record of what M9-006 did.+`apps/api/Dockerfile` is now five stages: `base` (shared OS packages) →`toolchain` (pnpm + python3/make/g++, never ships) → `build` (full install,runs `pnpm codegen` — Prisma client + core's geo/ASN data — never ships) and@@ -354,3 +358,53 @@ was created. Pre-existing tags `openpanel-v2:slim` and `openpanel-v2:test`each is within that budget, but if disk pressure appears, `openpanel-v2:slim`and `openpanel-v2:test` are the pre-existing tags to reclaim first (this taskdid not create them and does not need them kept).++---++## P13 (M13-003, 2026-09-07): both Dockerfiles move onto `bun install`++Supersedes the *P9.5 slim-down (M9-006)* section above wherever the two+disagree. That section is left in place as the record of what M9-006 did; the+image shape it describes no longer exists.++**What changed.**++| | before (M9-006) | now |+|---|---|---|+| `apps/api/Dockerfile` stages | `base` → `toolchain` (a second package manager + python3/make/g++) → `build` / `prod-deps` in parallel → `runtime` | `base` → `toolchain` → `build` → `prod-deps` → `runtime`, all on `oven/bun:${BUN_VERSION}-slim` except nothing |+| api install | root install, then a from-scratch `--prod` install filtered to api's closure | `bun install --frozen-lockfile --linker=isolated`, then a from-scratch `bun install --production --frozen-lockfile --linker=isolated --filter '@openpanel/api'` |+| api codegen | `pnpm codegen` | `bun run codegen` |+| manifest COPYs | a hand-maintained list of `packages/*/package.json` | `COPY . .` — one full workspace install per image (ADR-014 Problems §4) |+| native build toolchain | `python3 make g++` in two stages | gone: `[install] ignoreScripts = true` means nothing compiles at install time, and `@node-rs/argon2` / `sharp` resolve to prebuilt linux-x64 packages |+| `apps/start/Dockerfile` | did not build at all — see below | install + build on `oven/bun:1.4.0-slim`, runtime unchanged at `node:24-slim` running `.output/server/index.mjs` |++**The dashboard image was already broken before P13**, for two reasons neither+of which is the installer, both now fixed and named in that file's header:+stale COPYs of the `@openpanel/auth` package P11 deleted, and a workspace-subset+install that left `@openpanel/tsconfig` unresolvable for+`packages/core/tsconfig.json` once P11 made `apps/start` import+`@openpanel/core`. The fix for the second is the full workspace install, which+is the shape a bun-base image has to take anyway.++The dashboard runner stage now copies **only** `.output`. The Nitro output is+standalone — 222 bundled packages of its own, and `.output/public` holds every+asset `/login` references — verified on 2026-09-07 by running+`.output/server/index.mjs` from an otherwise empty directory (`/api/healthcheck`+200, `/login` 200 / 136710 bytes, no resolution errors). The workspace+`node_modules` and per-package source copies the previous runner carried were+dead weight.++**Gate.** `bash tooling/gates/p13-images.sh` builds both images, boots each+against this box's local services and asserts the api's `/healthz/ready` = 200+and the dashboard's `/login` server-renders (HTTP 200, `<html`, ≥1000 bytes,+clean container log) — the `.github/smoke/smoke.sh` assertions, because+"the image built" is exactly what main-8e60 passed. It prunes the build cache+between the two builds and again on exit; one from-scratch build leaves ~6GB of+cache on a box with ~15GB free.++Full run, 2026-09-07: **PASSED in 3m16s**, `/login` 152348 bytes, both container+logs clean, 7.8G free after the final prune.++**Sizes and the one known break** (the shipped compose templates' `prisma+migrate deploy` line, pre-existing): `docs/TECH_DEBT.md` → *M13-003 image+sizes*.diff --git a/docs/OPS_P11_DASHBOARD_CHECK.md b/docs/OPS_P11_DASHBOARD_CHECK.mdindex 365fe697e..368ac4d3a 100644--- a/docs/OPS_P11_DASHBOARD_CHECK.md+++ b/docs/OPS_P11_DASHBOARD_CHECK.md@@ -3,7 +3,7 @@> This box has no browser. `curl http://127.0.0.1:4173/login` proves the SPA> shell boots and serves HTML (the automated boot gate); it cannot prove a> chart renders or a form validates. That half is this checklist — run it by-> hand against `pnpm run build && pnpm run preview` (or `pnpm dev`) in a real+> hand against `bun run build && bun run preview` (or `bun run dev`) in a real> browser after any change that touches `@openpanel/core`'s> `*.constants.ts` files or `apps/start/src/utils/{math,slug,super-json,> union-omit}.ts` — the P11 dissolution's actual runtime surface.diff --git a/docs/SQL_MIGRATION_RECIPE.md b/docs/SQL_MIGRATION_RECIPE.mdindex 6ca7a2a5e..3dcfb40a6 100644--- a/docs/SQL_MIGRATION_RECIPE.md+++ b/docs/SQL_MIGRATION_RECIPE.md@@ -623,7 +623,7 @@ tasks should expect:or a stated reason.4. `bash tooling/gates/p12-grep-gates.sh --report | tail -3` — the numbers wentdown, and nothing else moved.-5. `pnpm run typecheck` and `cd packages/core && bun test`.+5. `bun run typecheck` and `cd packages/core && bun test`.6. When all three totals reach 0, delete `query-builder.ts`,`query-builder.test.ts`, `sql-builder.ts` and their `packages/db/index.ts`re-exports, drop `sqlstring` and `@types/sqlstring` from the three manifests,diff --git a/docs/TECH_DEBT.md b/docs/TECH_DEBT.mdindex a65c3e897..e280c687b 100644--- a/docs/TECH_DEBT.md+++ b/docs/TECH_DEBT.md@@ -1635,3 +1635,110 @@ space cannot scope a version to one importer — see the recipe.enforcement point. `[install] ignoreScripts = true` in `bunfig.toml` is, andthe property the item was really after — no dependency build script runs, asunder pnpm's twelve `allowBuilds: false` — does hold.++## M13-003 image sizes++Carl asked for the two P13 images to be compared against the published ones.+All four numbers below are `docker image inspect --format '{{.Size}}'`, run on+this box on **2026-09-07**, Docker 29.7.2 with the **containerd snapshotter**.++**Read the caveat before the numbers.** Under the containerd snapshotter+`.Size` is the sum of the image's **compressed** layer blobs — what a registry+stores and what a `docker pull` transfers. `docker images`' `DISK USAGE` column+is the **uncompressed**, unpacked size on disk. They differ by ~5x here and+mixing them produces a meaningless comparison, so both are given, each labelled,+and the comparison column uses only the compressed pair.++| Image | Source | COMPRESSED (`inspect .Size`) | uncompressed (`docker images` DISK USAGE) |+|---|---|---:|---:|+| `openpanel-api:p13-gate` | built here from `apps/api/Dockerfile` | **412,899,646 B** (412.9 MB / 393.8 MiB) | 2.11 GB |+| `lindesvard/openpanel-api:latest` | pulled from Docker Hub | **476,399,142 B** (476.4 MB / 454.3 MiB) | 2.39 GB |+| `openpanel-dashboard:p13-gate` | built here from `apps/start/Dockerfile` | **95,217,524 B** (95.2 MB / 90.8 MiB) | 419 MB |+| `lindesvard/openpanel-dashboard:latest` | pulled from Docker Hub | **524,333,374 B** (524.3 MB / 500.0 MiB) | 2.77 GB |++Compressed deltas: api **−63.5 MB (−13.3%)**, dashboard **−429.1 MB (−81.8%)**.++### Provenance++The two `p13-gate` tags are exactly what `bash tooling/gates/p13-images.sh`+builds; the sizes above are the two lines that run prints — from the last of+four full builds on 2026-09-07. Rebuilding the same tree moves these numbers by+a few hundred to a few thousand bytes, because build timestamps sit inside the+layer metadata: the four runs gave the api 412,899,207 / 412,902,099 /+412,906,098 / 412,899,646 (spread 6.9 KB) and the dashboard 95,217,685 /+95,217,881 / 95,217,702 / 95,217,524 (spread 357 B). Anything larger than+~10 KB of drift is a real content change, not this noise. The published images+were pulled at their `latest` tags on 2026-09-07 and measured with the same+command:++| Image | digest | image `Created` |+|---|---|---|+| `lindesvard/openpanel-api:latest` | `sha256:cee4855cd715a7248c5c49090fe93a28ffc505541de20cb238d8884a6bcb9223` | 2026-08-18T21:22:28Z |+| `lindesvard/openpanel-dashboard:latest` | `sha256:b773cf864454a0ec89b2a133b48bf06202880ed650a75cc2f1f5c87aaed1cffa` | 2026-08-18T20:50:28Z |++Both were `docker rmi`'d after measuring — this box had ~4.5 GB free at that+point and two more from-scratch builds had to fit. Re-pull by digest to+reproduce.++**What the two deltas are not.** They are not a like-for-like measurement of+"the installer changed". The published images are ~3 weeks older than this+branch's HEAD and were built from a tree with a different dependency set, so the+api's −63.5 MB mixes the installer swap with three weeks of unrelated content.+The dashboard's −429.1 MB is dominated by one deliberate change in this task+and not by the installer at all: the previous runner stage copied a workspace+`node_modules`, `packages/db`, `packages/payments`, `packages/sdks/_info` and+the deleted `@openpanel/auth` tree alongside `.output`, and this one copies only+`.output`, which is verified standalone (222 bundled packages of its own; it+serves `/login` from an otherwise empty directory).++### Known break in the shipped compose templates — pre-existing, not P13's++`self-hosting/docker-compose.template.yml`, `self-hosting/coolify.yml` and+`.github/smoke/docker-compose.yml` all start the api container with++```+cd /app/packages/db && ./node_modules/.bin/prisma migrate deploy+```++**That binary is not in the image**, and was not before this task either.+`prisma` is a `devDependency` of `packages/db`, and both the old and the new+image install production dependencies only, so it is correctly omitted. Measured+2026-09-07 against a freshly built `openpanel-api:p13-gate`:++```+$ docker run --rm --entrypoint sh openpanel-api:p13-gate \+ -c 'cd /app/packages/db && ./node_modules/.bin/prisma migrate deploy'+sh: 1: ./node_modules/.bin/prisma: not found+```++(`./node_modules/.bin/jiti`, the second migration command, **is** present —+`jiti` is a real dependency of `packages/core`.)++`docs/OPS_DEPLOY_MIGRATION.md`'s M9-006 section records these commands as+"verified", but the verification it describes was run against this box's local+Postgres/ClickHouse — on the **host**, not inside the container — so the+image's contents were never the thing under test.++Left unfixed on purpose: the two candidate fixes are "ship the Prisma CLI in the+production image" and "run migrations from somewhere that has it", and choosing+between them is a decision about what the image contains, not an installer+change. M13-003's scope is the swap. The note is repeated in both compose+templates so a self-hoster reading the file sees it.++### Also recorded++- **`.bun-version` still does not exist.** ADR-016 names it as Bun's declared+ home alongside `ARG BUN_VERSION`; both Dockerfiles carry the ARG and nothing+ carries the file. Unchanged from M13-002's note.+- **Both images now `COPY . .` and install the whole workspace**, which is the+ shape ADR-014 Problems §4 rules for a bun-base image. The cost is that a+ source edit invalidates the install layer, so neither image has a+ manifests-only cache tier any more. The benefit is that the hand-maintained+ per-package `COPY packages/x/package.json` list — whose own comment recorded+ that drifting silently omitted a package from `node_modules` — is gone from+ both files.+- **The api image's production install is `--filter '@openpanel/api'`**, not a+ bare `--production`. A bare install at the workspace root pulls every member's+ production closure, including `apps/start`'s (§5d). Verified on Bun 1.4.0 that+ `--filter` follows `workspace:` edges transitively and installs nothing+ outside the filtered closure.diff --git a/self-hosting/coolify.yml b/self-hosting/coolify.ymlindex b0f63b0ea..a5cb809a8 100644--- a/self-hosting/coolify.yml+++ b/self-hosting/coolify.yml@@ -110,6 +110,10 @@ services:opapi:image: lindesvard/openpanel-api:latestrestart: always+ # Same contract as docker-compose.template.yml's op-api: no second package+ # manager exists in this image (P13 / ADR-014), so nothing below may invoke+ # one. The same KNOWN, pre-existing `prisma` break applies — see the note+ # there and docs/TECH_DEBT.md.command: >sh -c "echo 'Waiting for PostgreSQL to be ready...'diff --git a/self-hosting/docker-compose.template.yml b/self-hosting/docker-compose.template.ymlindex 7e01eba3d..a36aaae55 100644--- a/self-hosting/docker-compose.template.yml+++ b/self-hosting/docker-compose.template.yml@@ -129,11 +129,20 @@ services:op-api:image: lindesvard/openpanel-api:2restart: always- # P9.5 collapsed the two-Dockerfile setup onto oven/bun: pnpm no longer- # exists in this image (ADR-014 Problems §4), so migrations and boot run- # as their bun equivalents — the prisma/jiti binaries directly (their- # `#!/bin/sh` shims resolve `node` to Bun's node-fallback shim) and- # `bun run start` for apps/api's own script.+ # P9.5 collapsed the two-Dockerfile setup onto oven/bun, and P13+ # (ADR-014, M13-003) made bun the installer as well, so no second package+ # manager exists in this image or in the repo it is built from. Nothing+ # here may invoke one: migrations and boot run the installed binaries+ # directly (their `#!/bin/sh` shims resolve `node` to Bun's node-fallback+ # shim) and `bun run start` for apps/api's own script.+ #+ # KNOWN BREAK, pre-existing and NOT introduced by P13: `prisma` is a+ # devDependency of packages/db, so the image's production-only install+ # correctly omits it and the first line below cannot run. Measured against+ # a freshly built image on 2026-09-07:+ # `sh: ./node_modules/.bin/prisma: not found`. Recorded in+ # docs/TECH_DEBT.md; fixing it is a decision about what the image ships,+ # not an installer change.command: >sh -c "echo 'Running migrations...'diff --git a/tooling/gates/p13-images.sh b/tooling/gates/p13-images.shnew file mode 100755index 000000000..9274af4c5--- /dev/null+++ b/tooling/gates/p13-images.sh@@ -0,0 +1,274 @@+#!/usr/bin/env bash+#+# P13 image gate (ADR-014, M13-003) — do BOTH images build on `bun install`,+# and does what they build actually BOOT?+#+# ADR-014's own gate line is "all Dockerfiles build", and `.github/smoke/smoke.sh:6-11`+# is the repo's record of why that is not enough: main-8e60 built green, pushed+# green, and every dashboard route returned 500. So this gate does the same two+# things that script does — start the image and assert a real server-rendered+# page — rather than stopping at a successful build.+#+# What it does:+# 1. builds apps/api/Dockerfile (with a dummy DATABASE_URL build arg — the+# build stage's `bun run codegen` only needs the variable to exist, it+# never connects) and apps/start/Dockerfile;+# 2. runs the api image with `--network host` against this box's already+# running Postgres/ClickHouse/Redis/Redpanda and asserts /healthz/ready = 200;+# 3. runs the dashboard image the same way and asserts smoke.sh's+# assert_ssr_route conditions for /login — HTTP 200, an `<html`, at least+# 1000 bytes — plus assert_no_server_errors on the container log;+# 4. stops and removes both containers, always;+# 5. prunes the build cache.+#+# The prune also runs BETWEEN the two builds. Measured on this box on+# 2026-09-07: one from-scratch build leaves ~6GB of cache and the box had 12GB+# free, so building both without an intermediate prune fills the disk. The two+# builds share no cache anyway — they start from different base layers.+#+# DATABASES: the isolated `openpanel_test` Postgres and ClickHouse, never the+# prod-copy `openpanel` ones — both containers are read-only against them in+# practice (a session lookup and a readiness probe), but the rule is the rule.+#+# ENVIRONMENT: the repo `.env` is sourced into this shell for the two values a+# gate must not invent — ENCRYPTION_KEY and COOKIE_SECRET — and every other+# variable the containers get is written out explicitly below, so the api's+# database URLs and ports cannot be inherited from a developer's `.env`+# (which pins API_PORT=3333 and points CLICKHOUSE_URL at the prod copy).+# `--env-file` is deliberately not used: `.env` contains+# `DATABASE_URL_DIRECT="$DATABASE_URL"`, a shell expansion docker does not+# perform.+#+# DISK is the operational risk here: two from-scratch workspace installs+# produce several GB of build cache on a box that has ~15G free, so the prune+# runs on success, on failure and on an interrupt. It is a+# `docker builder prune -f --all` — build cache only; it does not touch images+# or the containers already running on this box.+#+# `--all` is not decoration. Measured on this box on 2026-09-07, immediately+# after a full gate run: a plain `docker builder prune -f` reclaimed **0B** and+# left **22.91GB** of cache (buildkit's default policy keeps recent records);+# `docker builder prune -af` on the same state reclaimed **20.89GB** and took+# the disk from 3.4G to 23G free. Without `--all` this gate fills the box in+# two or three runs, which is the exact failure it is supposed to prevent.+# Nothing else on this box builds images, so discarding all cache costs only+# rebuild time — one from-scratch build of either image is ~1.5-3 min.+#+# bash tooling/gates/p13-images.sh++set -uo pipefail++REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"+cd "$REPO_ROOT" || exit 1++readonly RUN_DIR="${P13_IMAGES_RUN_DIR:-/tmp/openpanel-p13-images}"+readonly API_BUILD_LOG="$RUN_DIR/api-build.log"+readonly DASHBOARD_BUILD_LOG="$RUN_DIR/dashboard-build.log"+readonly API_LOG="$RUN_DIR/api.log"+readonly DASHBOARD_LOG="$RUN_DIR/dashboard.log"+readonly BODY_FILE="$RUN_DIR/ssr-body.html"++readonly API_IMAGE='openpanel-api:p13-gate'+readonly DASHBOARD_IMAGE='openpanel-dashboard:p13-gate'+readonly API_CONTAINER='op-p13-gate-api'+readonly DASHBOARD_CONTAINER='op-p13-gate-dashboard'++# The build stage exports DATABASE_URL so `prisma generate` has one; it is+# never dialled. Deliberately not a reachable host.+readonly DUMMY_DATABASE_URL='postgresql://p13-gate:[email protected]:1/p13-gate'++# smoke.sh's own thresholds, reproduced rather than referenced.+readonly MIN_SSR_BODY_BYTES=1000+readonly SSR_REQUEST_TIMEOUT_SECONDS=30+readonly RESOLUTION_ERROR_PATTERN='is not a function|Cannot find (module|package)|ERR_MODULE_NOT_FOUND|ERR_PACKAGE_PATH_NOT_EXPORTED'++readonly API_READY_TIMEOUT_SECONDS=180+readonly DASHBOARD_READY_TIMEOUT_SECONDS=120+readonly READY_POLL_INTERVAL_SECONDS=2++FAILED=0++# --- lifecycle ---------------------------------------------------------------++remove_container() {+ local name="$1"+ docker rm -f "$name" >/dev/null 2>&1 || true+}++cleanup() {+ docker logs "$DASHBOARD_CONTAINER" >"$DASHBOARD_LOG" 2>&1 || true+ docker logs "$API_CONTAINER" >"$API_LOG" 2>&1 || true+ remove_container "$DASHBOARD_CONTAINER"+ remove_container "$API_CONTAINER"+ prune_build_cache+}+trap cleanup EXIT INT TERM++prune_build_cache() {+ echo '==> pruning the build cache'+ docker builder prune -f --all 2>&1 | tail -1+ df -h / | awk 'NR==2{print " disk: "$4" free"}'+}++fail() {+ echo "FAIL: $*" >&2+ FAILED=1+}++die() {+ fail "$@"+ exit 1+}++free_port() {+ bun -e 'const s = Bun.listen({ hostname: "127.0.0.1", port: 0, socket: { data() {} } }); console.log(s.port); s.stop(true);'+}++wait_ready() {+ local name="$1" url="$2" timeout="$3" container="$4" waited=0+ until curl -fsS -o /dev/null --max-time 5 "$url" 2>/dev/null; do+ if [[ "$(docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null)" != 'true' ]]; then+ docker logs "$container" 2>&1 | tail -40 >&2+ die "$name exited before it became ready"+ fi+ sleep "$READY_POLL_INTERVAL_SECONDS"+ waited=$((waited + READY_POLL_INTERVAL_SECONDS))+ if ((waited >= timeout)); then+ docker logs "$container" 2>&1 | tail -40 >&2+ die "$name never became ready at $url within ${timeout}s"+ fi+ done+ echo " $name ready at $url (${waited}s)"+}++# --- assertions --------------------------------------------------------------++assert_ssr_route() {+ local path="$1" status bytes++ status=$(curl -sS -o "$BODY_FILE" -w '%{http_code}' \+ --max-time "$SSR_REQUEST_TIMEOUT_SECONDS" "http://localhost:$DASHBOARD_PORT$path" || echo '000')+ bytes=$(wc -c <"$BODY_FILE" 2>/dev/null | tr -d ' ')+ bytes=${bytes:-0}++ if [[ "$status" != '200' ]]; then+ head -60 "$BODY_FILE" >&2 2>/dev/null+ fail "$path returned HTTP $status (expected 200)"+ return 1+ fi+ if ! grep -q '<html' "$BODY_FILE"; then+ head -40 "$BODY_FILE" >&2+ fail "$path returned 200 but no <html> — SSR did not render"+ return 1+ fi+ if ((bytes < MIN_SSR_BODY_BYTES)); then+ cat "$BODY_FILE" >&2+ fail "$path rendered only ${bytes} bytes — suspiciously empty for an SSR page"+ return 1+ fi++ echo " $path OK (HTTP 200, ${bytes} bytes)"+}++assert_no_server_errors() {+ local name="$1"+ local container="$2"+ local log="$RUN_DIR/$name-check.log"+ docker logs "$container" >"$log" 2>&1 || true+ if grep -qE "$RESOLUTION_ERROR_PATTERN" "$log"; then+ grep -nE "$RESOLUTION_ERROR_PATTERN" "$log" | head -20 >&2+ fail "$name logged a module/runtime resolution error"+ return 1+ fi+ echo " $name log clean"+}++# --- run ---------------------------------------------------------------------++mkdir -p "$RUN_DIR"+command -v docker >/dev/null || die 'docker is not available'++remove_container "$API_CONTAINER"+remove_container "$DASHBOARD_CONTAINER"++echo '==> building the api image'+if ! docker build -f apps/api/Dockerfile -t "$API_IMAGE" \+ --build-arg "DATABASE_URL=$DUMMY_DATABASE_URL" . >"$API_BUILD_LOG" 2>&1; then+ tail -40 "$API_BUILD_LOG" >&2+ die "the api image did not build — see $API_BUILD_LOG"+fi+echo " built $API_IMAGE"+prune_build_cache++echo '==> building the dashboard image'+if ! docker build -f apps/start/Dockerfile -t "$DASHBOARD_IMAGE" . >"$DASHBOARD_BUILD_LOG" 2>&1; then+ tail -40 "$DASHBOARD_BUILD_LOG" >&2+ die "the dashboard image did not build — see $DASHBOARD_BUILD_LOG"+fi+echo " built $DASHBOARD_IMAGE"++# .env supplies what the block below does not name. Sourced FIRST so the+# isolated test databases and the freshly-picked ports override it.+set -a+# shellcheck disable=SC1091+[[ -f .env ]] && source .env+set +a++API_PORT="$(free_port)"+DASHBOARD_PORT="$(free_port)"+[[ -n "$API_PORT" && -n "$DASHBOARD_PORT" ]] || die 'could not pick free ports'++echo "==> starting the api image (ROLE=api) on :$API_PORT"+docker run -d --name "$API_CONTAINER" --network host \+ -e NODE_ENV=production \+ -e ROLE=api \+ -e API_PORT="$API_PORT" \+ -e SELF_HOSTED=true \+ -e ALLOW_REGISTRATION=true \+ -e ALLOW_INVITATION=true \+ -e BATCH_SIZE=5000 \+ -e BATCH_INTERVAL=10000 \+ -e REDIS_URL='redis://localhost:6379' \+ -e CLICKHOUSE_URL='http://localhost:8123/openpanel_test' \+ -e DATABASE_URL='postgresql://postgres:postgres@localhost:5432/openpanel_test?schema=public' \+ -e DATABASE_URL_DIRECT='postgresql://postgres:postgres@localhost:5432/openpanel_test?schema=public' \+ -e DASHBOARD_URL="http://localhost:$DASHBOARD_PORT" \+ -e API_URL="http://localhost:$API_PORT" \+ -e ENCRYPTION_KEY="${ENCRYPTION_KEY:-}" \+ -e COOKIE_SECRET="${COOKIE_SECRET:-p13-images-gate-cookie-secret-not-a-real-secret}" \+ -e EMAIL_SENDER='[email protected]' \+ -e RESEND_API_KEY='re_p13_images_gate_key' \+ -e KAFKA_BROKERS='127.0.0.1:19092' \+ -e KAFKA_EVENTS_TOPIC_PARTITIONS=24 \+ "$API_IMAGE" >/dev/null || die 'could not start the api container'+wait_ready api "http://localhost:$API_PORT/healthz/ready" \+ "$API_READY_TIMEOUT_SECONDS" "$API_CONTAINER"++echo "==> starting the dashboard image on :$DASHBOARD_PORT"+docker run -d --name "$DASHBOARD_CONTAINER" --network host \+ -e NODE_ENV=production \+ -e SELF_HOSTED=true \+ -e PORT="$DASHBOARD_PORT" \+ -e DASHBOARD_URL="http://localhost:$DASHBOARD_PORT" \+ -e API_URL="http://localhost:$API_PORT" \+ "$DASHBOARD_IMAGE" >/dev/null || die 'could not start the dashboard container'+wait_ready dashboard "http://localhost:$DASHBOARD_PORT/api/healthcheck" \+ "$DASHBOARD_READY_TIMEOUT_SECONDS" "$DASHBOARD_CONTAINER"++echo '==> SSR rendering'+assert_ssr_route '/login'++echo '==> container logs'+assert_no_server_errors dashboard "$DASHBOARD_CONTAINER"+assert_no_server_errors api "$API_CONTAINER"++echo '==> image sizes (docker image inspect .Size — COMPRESSED on this box'\''s containerd snapshotter)'+printf ' %-34s %s bytes\n' "$API_IMAGE" "$(docker image inspect --format '{{.Size}}' "$API_IMAGE")"+printf ' %-34s %s bytes\n' "$DASHBOARD_IMAGE" "$(docker image inspect --format '{{.Size}}' "$DASHBOARD_IMAGE")"++if ((FAILED != 0)); then+ echo "p13-images: FAILED (logs in $RUN_DIR)" >&2+ exit 1+fi++echo 'p13-images: PASSED (both images built; api /healthz/ready 200; dashboard /login server-rendered; logs clean)'