AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(migrations): build the background track before the schema drift check (#16439)
Othman Abu Ajamiehbd9dbc602fb5
diff --git a/brain/knowledge/data-storage-observability/database.md b/brain/knowledge/data-storage-observability/database.mdindex 236dffb1cc9..d89bcb351d8 100644--- a/brain/knowledge/data-storage-observability/database.md+++ b/brain/knowledge/data-storage-observability/database.md@@ -260,6 +260,7 @@ A background migration that fails every retry stops the app from booting. The gu- **A brand-new hot-path index in the same release as the code that needs it can leave the app on seq scans while background CIC builds.** Only applies when there's *no* prior covering index for the query. Fixes: (a) ship the CIC in a prior release so it's built by the time the code lands (verify via `pendingCount` on `/v1/health/system`), or (b) put the CIC in the blocking track so the index is present when the app starts serving. Reshapes (replacing existing indexes) don't hit this — the old index keeps queries fast while new one builds.- **Same-name migration in both folders would run twice.** The two history tables are independent — a rename or folder move without a seed-row migration double-executes on customers that already ran it in the old track.- **Background migrations run on API pods only.** Workers don't register system-job handlers. Deployments with zero API pods will never finish backfills; not a supported topology, worth flagging if it comes up.+- **`check-migrations` and `db-migration` build their database with `src/migrate.ts`, not `typeorm migration:run`.** The CLI runs only the blocking track, so it reports every entity index that a background migration builds as drift. `migrate.ts` runs `runMigrationsWithCatchup` and then the background track, the boot order, through the same `distributedLock` as boot. It ends with `process.exit`: on memory Redis the lock starts redis-memory-server, whose open connection would keep the step from exiting, and the library's killer process stops `redis-server` once the parent is gone.## Key files@@ -269,6 +270,7 @@ A background migration that fails every retry stops the app from booting. The gu- `packages/server/api/src/app/database/background-migration.ts` — the `BackgroundMigration` interface.- `packages/server/api/src/app/database/migration-helpers.ts` — `assertBackgroundMigrationComplete`, `createIndexConcurrently`.- `packages/server/api/src/app/database/background-migration-runner.ts` — the BullMQ handler.+- `packages/server/api/src/migrate.ts` — both tracks in one process, for `check-migrations` and `db-migration`.- `tools/scripts/check-migration-rollback.ts` — the CI check.## Related pagesdiff --git a/packages/server/api/package.json b/packages/server/api/package.jsonindex 0c11b3ede75..66a7ad1222e 100644--- a/packages/server/api/package.json+++ b/packages/server/api/package.json@@ -150,9 +150,9 @@"test-ce": "export $(cat .env.tests | xargs) && AP_EDITION=ce npm run test-ce-command","test-ee": "export $(cat .env.tests | xargs) && AP_EDITION=ee npm run test-ee-command","test-cloud": "export $(cat .env.tests | xargs) && AP_EDITION=cloud npm run test-cloud-command",- "check-migrations": "echo '🔍 Checking for schema changes...' && export $(cat .env.tests | xargs) && export AP_DEV_PIECES='' && export AP_ENVIRONMENT=dev && AP_EDITION=ce && ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json node_modules/typeorm/cli.js migration:run -d src/app/database/migration-data-source.ts > /dev/null 2>&1 && echo 'Checking for schema drift...' && ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json node_modules/typeorm/cli.js migration:generate -p -d src/app/database/migration-data-source.ts src/app/database/migration/postgres/check --dryrun --check || (echo '❌ ERROR: Schema changes detected without migration!' && exit 1) && echo '✅ No missing migrations detected'",+ "check-migrations": "echo '🔍 Checking for schema changes...' && export $(cat .env.tests | xargs) && export AP_DEV_PIECES='' && export AP_ENVIRONMENT=dev && AP_EDITION=ce && ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json src/migrate.ts && echo 'Checking for schema drift...' && (ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json node_modules/typeorm/cli.js migration:generate -p -d src/app/database/migration-data-source.ts src/app/database/migration/postgres/check --dryrun --check || (echo '❌ ERROR: Schema changes detected without migration!' && exit 1)) && echo '✅ No missing migrations detected'","db": "ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json node_modules/typeorm/cli.js",- "db-migration": "export $(cat .env.tests | xargs) && export AP_DEV_PIECES='' && export AP_ENVIRONMENT=dev && export AP_EDITION=ce && ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json node_modules/typeorm/cli.js migration:generate -p -d src/app/database/migration-data-source.ts",+ "db-migration": "export $(cat .env.tests | xargs) && export AP_DEV_PIECES='' && export AP_ENVIRONMENT=dev && export AP_EDITION=ce && ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json src/migrate.ts && ts-node --transpile-only -r tsconfig-paths/register -P tsconfig.app.json node_modules/typeorm/cli.js migration:generate -p -d src/app/database/migration-data-source.ts","test:docker": "UID=\"$(id -u)\" GID=\"$(id -g)\" docker compose --profile full -f docker-compose.test.yml up --exit-code-from app --attach app","rollback": "node dist/src/rollback.js"}diff --git a/packages/server/api/src/migrate.ts b/packages/server/api/src/migrate.tsnew file mode 100644index 00000000000..41f81af553d--- /dev/null+++ b/packages/server/api/src/migrate.ts@@ -0,0 +1,20 @@+import { backgroundMigrationRunner } from './app/database/background-migration-runner'+import { databaseConnection } from './app/database/database-connection'+import { system } from './app/helper/system/system'++async function main(): Promise<void> {+ process.env.TZ = 'UTC'+ const log = system.globalLogger()+ const dataSource = databaseConnection()+ await dataSource.initialize()+ await backgroundMigrationRunner.runMigrationsWithCatchup({ dataSource, log })+ await backgroundMigrationRunner.run({ dataSource, log })+ await dataSource.destroy()+}++main()+ .then(() => process.exit(0))+ .catch((error: unknown) => {+ system.globalLogger().error({ error }, '[migrate] Failed to apply blocking and background migrations')+ process.exit(1)+ })