Skip to content

A platform for community discussion. Free, open, simple.

DEV: add plugin post-migration generator and migration skill (#39125)

Samd9c0c2da53f3
diff --git a/.skills/discourse-migration/SKILL.md b/.skills/discourse-migration/SKILL.md
new file mode 100644
index 00000000000..3cb0cc415d1
--- /dev/null
+++ b/.skills/discourse-migration/SKILL.md
@@ -0,0 +1,226 @@
+---
+name: discourse-migration
+description: MUST load before writing or reviewing any database migration (db/migrate, db/post_migrate, plugin migrations)
+---
+
+# Discourse Migration Skill
+
+Discourse runs zero-downtime deployments. Migrations are split across two directories:
+
+- **`db/migrate/`** — runs pre-deploy. SafeMigrate (`lib/migration/safe_migrate.rb`) **blocks** dropping/renaming tables or columns, and creating concurrent indexes without first dropping them. Raises `Discourse::InvalidMigration` on violation.
+- **`db/post_migrate/`** — runs post-deploy (skipped when `SKIP_POST_DEPLOYMENT_MIGRATIONS=1`). No safety restrictions — destructive ops go here.
+
+Plugins mirror this: `plugins/<name>/db/migrate/` and `plugins/<name>/db/post_migrate/`.
+
+SafeMigrate is a dev/test guard only (disabled in production).
+
+## Generating migrations
+
+Always use generators — never hand-write timestamps. Manual timestamps like `120000`/`120001` cause collisions.
+
+```bash
+bin/rails g migration CreateWidgets # db/migrate/
+bin/rails g post_migration DropOldColumns # db/post_migrate/
+bin/rails g plugin_migration CreatePluginTable --plugin-name=my-plugin # plugins/<name>/db/migrate/
+bin/rails g plugin_post_migration DropOldPluginCols --plugin-name=my-plugin # plugins/<name>/db/post_migrate/
+```
+
+Use `ActiveRecord::Migration[8.0]` (current version). Always include `# frozen_string_literal: true`.
+
+Use `change` for reversible ops, `up`/`down` for irreversible. Use `raise ActiveRecord::IrreversibleMigration` in `down`. Use `up_only { ... }` for data ops inside an otherwise reversible `change`.
+
+## Safely removing columns
+
+Multi-step process across deployments. Helpers: `lib/migration/column_dropper.rb`, `lib/migration/base_dropper.rb`.
+
+**Step 1 — Mark readonly (regular migration):**
+
+```ruby
+class MarkOldColumnReadonly < ActiveRecord::Migration[8.0]
+ def up
+ change_column_default :my_table, :old_column, nil # MUST drop default first
+ Migration::ColumnDropper.mark_readonly(:my_table, :old_column)
+ end
+
+ def down
+ Migration::ColumnDropper.drop_readonly(:my_table, :old_column)
+ end
+end
+```
+
+Creates a PG trigger rejecting non-null writes. Old code can still read.
+
+**Step 2 — Ignore in model (code change, same PR):**
+
+```ruby
+self.ignored_columns += %i[old_column]
+# TODO(MM-YYYY): Remove this line (calculate 6 months from today)
+```
+
+Use `+=` to append. Without this, dropping the column causes `StatementInvalid`.
+
+**Step 3 — Drop column (post-deploy migration):**
+
+```ruby
+class DropOldColumn < ActiveRecord::Migration[8.0]
+ DROPPED_COLUMNS = { my_table: %i[old_column] }
+
+ def up
+ DROPPED_COLUMNS.each { |table, columns| Migration::ColumnDropper.execute_drop(table, columns) }
+ end
+
+ def down
+ raise ActiveRecord::IrreversibleMigration
+ end
+end
+```
+
+**Step 4** — Remove `ignored_columns` entry after post-deploy migration is promoted.
+
+For deprecation warnings before removal: `include HasDeprecatedColumns` then `deprecate_column :col, drop_from: "3.5"` (see `app/models/concerns/has_deprecated_columns.rb`).
+
+## Safely removing tables
+
+Same pattern via `lib/migration/table_dropper.rb`:
+
+1. Regular migration: `Migration::TableDropper.read_only_table(:old_table)`
+2. Post-deploy migration: `Migration::TableDropper.execute_drop(:old_table)`
+
+If table is already fully unused, just `drop_table` directly in a post-deploy migration.
+
+## Removing site settings
+
+The most common migration type. Use `execute` with `DELETE` or `UPDATE`:
+
+```ruby
+# Removal
+execute "DELETE FROM site_settings WHERE name = 'old_setting_name'"
+
+# Rename
+execute "UPDATE site_settings SET name = 'new_name' WHERE name = 'old_name'"
+```
+
+Always `up`/`down` with `raise ActiveRecord::IrreversibleMigration`.
+
+## Indexing
+
+### Concurrent indexes
+
+Large or busy existing tables require concurrent indexing. Always pair with `disable_ddl_transaction!`. SafeMigrate requires dropping the old index first:
+
+```ruby
+class AddIndexToWidgets < ActiveRecord::Migration[8.0]
+ disable_ddl_transaction!
+
+ def change
+ remove_index :widgets, :user_id, algorithm: :concurrently, if_exists: true
+ add_index :widgets, :user_id, algorithm: :concurrently
+ end
+end
+```
+
+New tables and small/low-traffic existing tables can use regular (non-concurrent) indexes.
+
+### Partial indexes
+
+Use `where:` to reduce index size — common for soft-deletes, nullable uniques, type scoping:
+
+```ruby
+add_index :topic_timers, [:topic_id], where: "deleted_at IS NULL"
+add_index :email_logs, [:bounce_key], unique: true, where: "bounce_key IS NOT NULL"
+add_index :users, [:id], name: "idx_users_admin", where: "admin"
+```
+
+### Composite indexes
+
+Order: equality conditions first, then range/sort. Add both directions for join tables:
+
+```ruby
+add_index :topic_allowed_users, %i[topic_id user_id], unique: true
+add_index :topic_allowed_users, %i[user_id topic_id], unique: true
+```
+
+### GiST indexes
+
+For trigram search: `using: "gist", opclass: :gist_trgm_ops`.
+
+### Naming
+
+Default Rails naming works unless the name exceeds 63 chars (PG limit) — then use a custom `name:`.
+
+## Foreign keys
+
+**Discourse mostly does NOT use foreign keys.** Referential integrity is enforced by application logic and `EnsureDbConsistency` (`app/jobs/scheduled/ensure_db_consistency.rb`), which runs every 12 hours calling `ensure_consistency!` on 18 core models.
+
+**Why:** avoids lock contention, simplifies soft-deletes and bulk ops, prevents unexpected cascading deletes.
+
+**Exceptions:** FKs are used selectively for critical relationships (uploads, security keys). Cascade deletes are rare (2 instances in codebase). Default: don't add FKs.
+
+## Data backfills
+
+Lightweight updates (e.g., nulling `baked_version` for rebake) on large tables are fine unbatched if the WHERE clause limits scope. For heavy data writes on large tables, batch with `disable_ddl_transaction!`:
+
+```ruby
+class BackfillData < ActiveRecord::Migration[8.0]
+ disable_ddl_transaction!
+ BATCH_SIZE = 30_000
+
+ def up
+ loop do
+ count = DB.exec(<<~SQL, batch_size: BATCH_SIZE)
+ WITH cte AS (
+ SELECT id, other_col FROM my_table WHERE new_col IS NULL LIMIT :batch_size
+ )
+ UPDATE my_table SET new_col = cte.other_col FROM cte WHERE my_table.id = cte.id
+ SQL
+ break if count == 0
+ end
+ end
+
+ def down
+ raise ActiveRecord::IrreversibleMigration
+ end
+end
+```
+
+Use `ON CONFLICT` for idempotent inserts. Use parameterized queries (`:param` syntax via `DB.exec`) — not string interpolation.
+
+`execute` is the default for all migration SQL. Only use `DB.exec`/`DB.query` when you need parameterized queries (`:param` syntax) or return values.
+
+## Conditional logic
+
+Use `Migration::Helpers` (`lib/migration/helpers.rb`) for install-vs-upgrade behavior:
+
+```ruby
+if Migration::Helpers.existing_site? # site created > 1 hour ago
+ # e.g., insert a site setting to disable new feature for existing sites
+end
+```
+
+Use `column_exists?`, `table_exists?`, `index_exists?` for idempotency guards.
+
+## NOT NULL constraints
+
+Always clean data first: `DELETE` invalid rows or `UPDATE` nulls to a default, then `change_column_null`.
+
+## Bigint conversions
+
+For large tables (e.g. `notifications.id`), use four migrations:
+1. Add shadow bigint column + insert-mirroring trigger
+2. Batch-copy existing rows (`disable_ddl_transaction!`, ~10k batches)
+3. Swap columns (rename old/new, fix PK/sequences, mark old readonly)
+4. Post-deploy: `execute_drop` the old column
+
+## Review checklist
+
+1. Destructive ops in `db/post_migrate/`, everything else in `db/migrate/`
+2. Timestamp from generator, not hand-written
+3. `# frozen_string_literal: true` and `ActiveRecord::Migration[8.0]`
+4. Concurrent indexes on large/busy tables: `disable_ddl_transaction!` + `remove_index ... if_exists: true` before `add_index`
+5. Column drops: full lifecycle (mark_readonly -> ignored_columns -> execute_drop)
+6. Default dropped before `mark_readonly`
+7. Heavy data writes on large tables batched with `disable_ddl_transaction!`
+8. Idempotent: `IF EXISTS`, `ON CONFLICT`, `column_exists?`, etc.
+9. Rollback: `down` method or `raise ActiveRecord::IrreversibleMigration`
+10. No foreign keys unless strong justification
+11. `execute` for SQL; `DB.exec`/`DB.query` only when param binding or return values needed
diff --git a/AI-AGENTS.md b/AI-AGENTS.md
index 57cb4bf8c92..080040c8a74 100644
--- a/AI-AGENTS.md
+++ b/AI-AGENTS.md
@@ -95,8 +95,7 @@ ALWAYS lint any changes you make
## Database & Performance
- ActiveRecord: use `includes()`/`preload()` (N+1), `find_each()`/`in_batches()` (large sets), `update_all`/`delete_all` (bulk), `exists?` over `present?`
-- Migrations: rollback logic, `algorithm: :concurrently` for large tables, deprecate before removing columns
-- Use `bin/rails generate migration` for new migrations
+- Migrations: use the skill at .skills/discourse-migration before writing or reviewing any migration
- Queries: use `explain`, specify columns, strategic indexing, `counter_cache` for counts
## HTTP Response Codes
diff --git a/lib/generators/rails/plugin_post_migration_generator.rb b/lib/generators/rails/plugin_post_migration_generator.rb
new file mode 100644
index 00000000000..080a9a61074
--- /dev/null
+++ b/lib/generators/rails/plugin_post_migration_generator.rb
@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+require "rails/generators/active_record/migration/migration_generator"
+
+class Rails::PluginPostMigrationGenerator < ActiveRecord::Generators::MigrationGenerator
+ class_option :plugin_name,
+ type: :string,
+ banner: "plugin_name",
+ desc: "The plugin name to generate the post migration into.",
+ required: true
+
+ source_root "#{Gem.loaded_specs["activerecord"].full_gem_path}/lib/rails/generators/active_record/migration/templates"
+
+ private
+
+ def db_migrate_path
+ "plugins/#{options["plugin_name"]}/#{Discourse::DB_POST_MIGRATE_PATH}"
+ end
+end