Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

docs: note the trigram REINDEX after an unaccent change and the pooler

Yudhi Armyndharis27c0dd76c160
diff --git a/docs/05-database-design.md b/docs/05-database-design.md
index fd722561a..d219176dd 100644
--- a/docs/05-database-design.md
+++ b/docs/05-database-design.md
@@ -881,7 +881,7 @@ OpenWA runs **two separate TypeORM connections**, each with its own migrations d
Migrations are hand-authored and idempotent (`IF NOT EXISTS`) so they are safe to adopt on a database originally created by `synchronize`. The two connections differ in how schema is managed:
-- **data** — `synchronize` defaults **off**, so this connection is migration-managed by default. On PostgreSQL `migrationsRun` is hardcoded on, and `DATABASE_SYNCHRONIZE=true` is rejected outright at boot validation (it would drop the migration-created `body_ts` tsvector column that `/search` depends on). On SQLite there is no such rejection and `migrationsRun` is the inverse of `synchronize` — so an opted-in `DATABASE_SYNCHRONIZE=true` switches the data connection to entity-synchronized schema and turns its migration chain **off**; only the deduplication of terminal `webhook_delivery_failures` rows still runs, ahead of the schema synchronization at every boot.
+- **data** — `synchronize` defaults **off**, so this connection is migration-managed by default. On PostgreSQL `migrationsRun` is hardcoded on, and `DATABASE_SYNCHRONIZE=true` is rejected outright at boot validation (it would drop the migration-created `body_ts` tsvector column that `/search` depends on). The PostgreSQL chain runs at boot under a session-level advisory lock, so replicas that start together apply it once (`src/database/pg-boot-migrations.ts`); a PgBouncer in transaction pooling mode runs each transaction on whichever server connection is free, so the lock does not stay with the session that took it: point the data connection at PostgreSQL directly or at a pooler in session mode. On SQLite there is no such rejection and `migrationsRun` is the inverse of `synchronize` — so an opted-in `DATABASE_SYNCHRONIZE=true` switches the data connection to entity-synchronized schema and turns its migration chain **off**; only the deduplication of terminal `webhook_delivery_failures` rows still runs, ahead of the schema synchronization at every boot.
- **main**: migration-managed by default, like data: `migrations-main/` creates and upgrades `api_keys` / `audit_logs` at boot. The chain is idempotent, so a `main.sqlite` an earlier release built with synchronize is adopted in place on the first boot (rows kept, missing columns added, the ledger written). `MAIN_DATABASE_SYNCHRONIZE=true` additionally synchronizes after the chain, for development; it cannot add an entity column that has no migration yet, because the missing-column check below refuses the boot first, so write the migration in `migrations-main/`. Boot logs a warning when it is set under `NODE_ENV=production`. Before any provider reads the file, boot refuses to start (in both modes) when the ledger records a migration this release does not ship (a newer release upgraded the file) or when an entity column is still missing after the chain ran (an older release rebuilt the table, so its values are gone, or the column has no migration yet); the error names the migration or column. For a newer ledger or a rebuilt table, the remedy is restoring `main.sqlite` from the backup taken before the downgrade or upgrade; for a column with no migration yet, it is adding one in `migrations-main/`. For a table rebuilt without `api_keys.allowedChats` and no backup, stop the instance, delete that migration's ledger row from the file the error names (the `MAIN_DATABASE_NAME` path, `data/main.sqlite` by default) and start it. On a source install that is `sqlite3 data/main.sqlite "DELETE FROM migrations WHERE name = 'AddApiKeyAllowedChats1786600000000'"`. On the compose deployment the file lives in the `openwa-data` volume, not on the host, and a stopped container cannot be exec'd into, so run the CLI in a one-off container of the service: `docker compose stop openwa-api && docker compose run --rm --no-deps openwa-api sqlite3 /app/data/main.sqlite "DELETE FROM migrations WHERE name = 'AddApiKeyAllowedChats1786600000000'"` (with `MAIN_DATABASE_NAME` set, use that path under `/app`). On the Helm chart, scale the StatefulSet to 0 and start the `openwa-restore` helper pod on the release's data PVC as [11 - Runbook: Restore from Backup](./11-operational-runbooks.md#runbook-restore-from-backup) does, run `kubectl exec openwa-restore -- sqlite3 /app/data/main.sqlite "DELETE FROM migrations WHERE name = 'AddApiKeyAllowedChats1786600000000'"`, then delete the pod and scale back to 1. Then the chain re-creates `api_keys.allowedChats` empty, which means every chat, so re-apply each key's `allowedChats` before exposing the API. A file whose ledger does not record the lost column's migration (0.23.6 and 0.23.7 wrote the ledger only with `MAIN_DATABASE_SYNCHRONIZE=false`) is not detected: the column comes back empty, which means every chat, so each key's `allowedChats` must be re-applied. Never delete or rename a `migrations-main/` file: its ledger row would read as a newer release's.
### Migration Files
diff --git a/docs/26-global-search.md b/docs/26-global-search.md
index 8bf7bc40c..2dfafb854 100644
--- a/docs/26-global-search.md
+++ b/docs/26-global-search.md
@@ -332,6 +332,21 @@ answering its health probe (a boot that outlasts the Helm startup probe is resta
and rebuilds. The candidate query walks the existing `IDX_messages_createdAt` index, so contains search
adds no other index.
+ The index holds `openwa_unaccent(body)`, which PostgreSQL only indexes because the wrapper is declared
+ IMMUTABLE, while `unaccent` follows its rules file, which a major upgrade may change. After an upgrade
+ whose rules changed (`pg_upgrade` keeps the index as it was built), rebuild the index so it folds
+ messages the way the queries now fold terms; until then a message stored before the upgrade may be
+ missed:
+
+ ```sql
+ REINDEX INDEX CONCURRENTLY idx_messages_body_trgm;
+ ```
+
+ The build lock is a session-level advisory lock, as the boot migration lock is
+ ([05 - Database Design](./05-database-design.md#56-migration-strategy)): a PgBouncer in transaction
+ pooling mode runs each transaction on whichever server connection is free, so neither lock stays with
+ the session that took it. Point OpenWA at PostgreSQL directly or at a pooler in session mode.
+
**Turning it off** later stops `match=contains` (`501` again) but keeps the index, and with it the cost
on every write, so turning it back on needs no rebuild. To reclaim the space, drop it by hand: