Skip to content

Open-source manufacturing ERP, MES and QMS. Quoting, MRP, inventory, shop floor, quality and lot/serial traceability on one Postgres schema, with a REST API and MCP server. Self-host or use Carbon Cloud.

fix(datasets): re-applying demo data no longer trips the contract invoice line CHECK, and the dataset check now re-applies each dataset

aashu21cdeb3ed5dc
diff --git a/.ai/lessons.md b/.ai/lessons.md
index ff06a33468..6400e4437e 100644
--- a/.ai/lessons.md
+++ b/.ai/lessons.md
@@ -3393,3 +3393,13 @@ of `salesInvoice`; backfilled by `20261006220901_sales-invoice-opportunity-backf
**Rule:** Do not patch drei. Anything that owns `camera.up` must re-assert it in its own frame loop whenever it drifts outside its own animation.
**Applies to:** `packages/viewer/src/useBuildFrame.ts` (its `useFrame`), any code that sets `camera.up` under `GizmoHelper` or `OrbitControls`.
+
+## Re-applying demo data runs code a first apply never reaches
+
+**Context:** A new table (`customerContractInvoiceLine`) got a CHECK that needs `customerContractInvoiceId` OR `memoId`. The demo datasets seed it with only the invoice.
+
+**Problem:** Settings → Demo Data on a company that already had data failed with "new row … violates check constraint". The wipe that runs first nulls every nullable FK pointing at a table it clears, which emptied both columns. `pnpm db:check:datasets` stayed green: it applied each dataset once, to an empty scratch company, so the wipe never met a seeded row.
+
+**Rule:** A seeded table whose CHECK needs one of several nullable FKs goes in `PARENT_CHECK_TABLES` (`packages/database/src/datasets/wipe.ts`). The dataset check now re-applies each dataset over itself, so this fails before commit. When a test only exercises the first run of an idempotent operation, also run it a second time.
+
+**Applies to:** `packages/database/src/datasets/` (wipe, verify), migrations that add CHECKs over nullable FK columns.
diff --git a/.claude/rules/conventions-database.md b/.claude/rules/conventions-database.md
index 672bb538d4..df596871c5 100644
--- a/.claude/rules/conventions-database.md
+++ b/.claude/rules/conventions-database.md
@@ -169,3 +169,7 @@ Validate in route actions with `validator(schema).validate(formData)` from
and shipped with `pnpm --filter @carbon/database authz migration <name>`
- [ ] Zod validators in `{module}.models.ts`
- [ ] Applied locally with `pnpm db:migrate` (regenerates types) — never `db:build`
+- [ ] A CHECK that needs one of several nullable FK columns set (`a IS NOT NULL OR b IS NOT NULL`,
+ a discriminator CHECK): if the demo datasets seed the table, add it to `PARENT_CHECK_TABLES`
+ in `packages/database/src/datasets/wipe.ts`. Re-applying demo data nulls every nullable FK
+ into a table it clears, which trips that CHECK (`pnpm db:check:datasets` catches it)
diff --git a/.claude/rules/onboarding-company-templates.md b/.claude/rules/onboarding-company-templates.md
index 8ac1dcbaa5..54a5fd1206 100644
--- a/.claude/rules/onboarding-company-templates.md
+++ b/.claude/rules/onboarding-company-templates.md
@@ -101,7 +101,11 @@ around its generic FK-null + topological delete, all inside the apply transactio
customer-XOR-supplier CHECK the FK-nulling pass would violate.
3. The existing header-first deletes (invoices, orders, quotes, RFQs), MRP output, and
the four model-only step-slide tables (`MODEL_SLIDE_TABLES`) — their
- `imagePath OR modelUploadId` CHECK cannot survive the FK-nulling pass.
+ `imagePath OR modelUploadId` CHECK cannot survive the FK-nulling pass. Any other
+ seeded table with a CHECK that needs one of several nullable FKs set goes in
+ `PARENT_CHECK_TABLES` (today `customerContractInvoiceLine`: an invoice OR a credit
+ memo). Missing one fails every re-apply with "new row … violates check constraint",
+ while a first apply onto an empty company still works.
4. After the delete pass, **orphaned customer/supplier org groups and customer/supplier
type groups are removed** — the partner interceptors mint one `group` per partner and
type, and `group` is preserved, so without this every re-apply left the previous
@@ -541,6 +545,14 @@ which is exactly what lets one caller commit and the other roll back.
The scratch company is attributed to the `system` user, never a real developer's.
+After the coverage check it applies the dataset AGAIN over the seeded scratch company with
+`wipeFirst: true` — the Settings → Demo Data path on a company that already has data. That
+is the only way to exercise the wipe against real rows: a new CHECK the FK-nulling pass
+cannot survive fails here as "re-applying over the seeded company failed (datasets/wipe.ts)".
+Before that pass it dates the seeded locations one second after the bootstrap one, because
+`buildCtx` keeps the OLDEST location and inside one transaction `NOW()` gives every row the
+same `createdAt`.
+
After the tiers and before the rollback, `findCoverageShortfalls` (`verify.ts`) counts every
table in `COVERAGE_FLOORS` (`datasets/coverage.ts` — 255 tables, measured across all four
datasets: floor = the smallest count any dataset produced, ×0.8 for volume tables) in one
diff --git a/packages/database/AGENTS.md b/packages/database/AGENTS.md
index 2198cdc187..f49e317f85 100644
--- a/packages/database/AGENTS.md
+++ b/packages/database/AGENTS.md
@@ -16,6 +16,7 @@ DB types, Supabase/Kysely clients, audit config, event system types, rate limiti
select it and a company restore cannot erase it. A migration landing in this package is
also what triggers those scripts to deploy (`.github/workflows/supabase.yml` only fires on
`packages/database/supabase/**`).
+- A new table whose CHECK needs one of several nullable FKs set (an invoice OR a memo, `imagePath` OR `modelUploadId`) and that the demo datasets seed goes in `PARENT_CHECK_TABLES` in `src/datasets/wipe.ts`. The wipe before a re-apply nulls every nullable FK into a table it clears, so that CHECK aborts the whole apply ("new row … violates check constraint"). `pnpm db:check:datasets` re-applies each dataset over itself to catch exactly this.
- Use `fetchAllFromTable` for paginated reads that exceed the 1000-row Supabase limit. It pages
without `count: "exact"` (a `COUNT(*) OVER ()` per page is not free) and fetches the pages past
the first concurrently. `fetchAllRecords` is the same pager over a query FACTORY (`() => builder`)
diff --git a/packages/database/src/datasets/verify.ts b/packages/database/src/datasets/verify.ts
index 9a32785c14..a003a2fdf9 100644
--- a/packages/database/src/datasets/verify.ts
+++ b/packages/database/src/datasets/verify.ts
@@ -115,10 +115,11 @@ export async function verifyDataset(
// dispatch events (pg_net calls would not roll back).
await client.query(`SET LOCAL "app.sync_in_progress" = 'true'`);
- const { companyId } = await seedCompanyReferenceData(client, {
- userId,
- companyName: SCRATCH_COMPANY_NAME
- });
+ const { companyId, locationId: bootstrapLocationId } =
+ await seedCompanyReferenceData(client, {
+ userId,
+ companyName: SCRATCH_COMPANY_NAME
+ });
const timeZone = await resolveCompanyTimeZone(client, companyId);
await applyDatasetTiers(client, {
@@ -139,6 +140,40 @@ export async function verifyDataset(
};
}
+ // Apply it again over itself, as Settings → Demo Data does on a company
+ // that already has data: the wipe nulls every nullable FK into a table it
+ // clears, which trips any CHECK that needs one of them. Only a seeded
+ // company has those rows, so a first apply never reaches that code.
+ //
+ // The wipe keeps the bootstrap location, which `buildCtx` finds as the
+ // oldest. In this one transaction NOW() gave every location the same
+ // `createdAt`; in real use the dataset runs after onboarding committed,
+ // so date the seeded ones after the bootstrap one as they would be.
+ await client.query(
+ `UPDATE location SET "createdAt" = "createdAt" + interval '1 second'
+ WHERE "companyId" = $1 AND id <> $2`,
+ [companyId, bootstrapLocationId]
+ );
+ try {
+ await applyDatasetTiers(client, {
+ companyId,
+ userId,
+ dataset,
+ timeZone,
+ log,
+ wipeFirst: true
+ });
+ } catch (err) {
+ return {
+ key,
+ ok: false,
+ error: `re-applying over the seeded company failed (datasets/wipe.ts): ${
+ err instanceof Error ? err.message : String(err)
+ }`,
+ durationMs: performance.now() - startedAt
+ };
+ }
+
return { key, ok: true, durationMs: performance.now() - startedAt };
} catch (err) {
return {
diff --git a/packages/database/src/datasets/wipe.ts b/packages/database/src/datasets/wipe.ts
index 554ca33c6c..8f8c91b911 100644
--- a/packages/database/src/datasets/wipe.ts
+++ b/packages/database/src/datasets/wipe.ts
@@ -98,6 +98,14 @@ const MODEL_SLIDE_TABLES = [
"quoteOperationStepSlide"
];
+// Other tables whose CHECK needs one of several nullable FKs to stay set, so
+// the FK-nulling pass would empty all of them and trip the CHECK. A new table
+// with a CHECK like that goes here (see .claude/rules/onboarding-company-templates.md).
+const PARENT_CHECK_TABLES = [
+ // customerContractInvoiceLine_parent_check: an invoice OR a credit memo
+ "customerContractInvoiceLine"
+];
+
type ForeignKey = {
child: string;
parent: string;
@@ -383,7 +391,11 @@ export async function wipeCompanyBusinessData(ctx: Ctx): Promise<void> {
// Deleting outright is safe for a dataset apply: MRP regenerates these on
// its next run (the planner's dismissals and assignments on planningAction
// go with the rest of the company's data, as the apply intends).
- for (const t of [...TRANSIENT_MRP_TABLES, ...MODEL_SLIDE_TABLES]) {
+ for (const t of [
+ ...TRANSIENT_MRP_TABLES,
+ ...MODEL_SLIDE_TABLES,
+ ...PARENT_CHECK_TABLES
+ ]) {
if (deleteSet.has(t)) {
await client.query(`DELETE FROM ${quote(t)} WHERE "companyId" = $1`, [
companyId