Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(infra): keep the database and bucket a bundled container serves
Yudhi Armyndharis05c45efb362e
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 7bb1c943f..b9a34b6b2 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -5691,7 +5691,7 @@ Merge-save infrastructure config to `data/.env.generated` (a `0600` secret file)| ----------------------------------------------------- | ------------------------ | ------------------------ | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ || `database` | object | No | — | DB section (see nested) || `database.type` | `'sqlite' \| 'postgres'` | If none stored or pinned | enum | `sqlite` drops stale postgres keys; `postgres` writes connection keys. Absent keeps the stored selection (else the pinned one); a pin of the other skips the section |-| `database.builtIn` | boolean | No | — | When `true`+postgres, pins host `postgres`, port `5432`, user and database `openwa` and schema `public`, and pushes the `postgres` Docker profile |+| `database.builtIn` | boolean | No | — | When `true`+postgres, pins host `postgres`, port `5432`, user and database `openwa` and schema `public`, or the stored ones (see below), and pushes the `postgres` profile || `database.host` / `.port` / `.username` / `.database` | string | No | `port` is a string | External postgres connection (defaults `localhost`/`5432`/`postgres`/`openwa`) || `database.schema` | string | No | lower-case, no `pg_` prefix (else 400) | Postgres schema, saved as `POSTGRES_SCHEMA`; an empty value writes `public` (also forced to `public` when switching to the built-in DB) || `database.password` | string | No | secret | Empty/omitted keeps the existing stored secret; with built-in postgres, the stored password of the built-in database, else `openwa` |@@ -5703,7 +5703,7 @@ Merge-save infrastructure config to `data/.env.generated` (a `0600` secret file)| `redis.password` | string | No | secret | Empty keeps existing || `queue.enabled` | boolean | No | — | Writes `QUEUE_ENABLED` || `storage.type` | `'local' \| 's3'` | If none stored or pinned | enum | `local` drops stale S3 keys; `s3` drops `STORAGE_LOCAL_PATH`. Absent keeps the stored selection (else the pinned one); a pin of the other skips the section |-| `storage.builtIn` | boolean | No | — | `true`+s3 writes the bundled endpoint and bucket, keeps the stored keys the bundled MinIO uses (else `minioadmin`), pushes `minio` (see below) |+| `storage.builtIn` | boolean | No | — | `true`+s3 writes the bundled endpoint, bucket `openwa` and keys `minioadmin`, or the stored bucket and keys the bundled MinIO already uses (see below); pushes `minio` || `storage.localPath` | string | No | — | Default `./data/media` || `storage.s3Bucket` / `.s3Region` / `.s3Endpoint` | string | No | — | External S3 || `storage.s3AccessKey` / `.s3SecretKey` | string | No | secret | Empty keeps existing |@@ -5712,7 +5712,7 @@ Merge-save infrastructure config to `data/.env.generated` (a `0600` secret file)| `engine.sessionDataPath` | string | No | — | Default `./data/sessions` || `engine.browserArgs` | string | No | — | Saved as `PUPPETEER_ARGS`. Empty saves the default `--no-sandbox --disable-setuid-sandbox --disable-dev-shm-usage --disable-gpu`; a non-empty value replaces that list in full |-Built-in flags: a section sent without `builtIn` stays in the stored mode. A stored `POSTGRES_BUILTIN`, `REDIS_BUILTIN` or `MINIO_BUILTIN` of `true` writes the bundled connection values again, and the fields sent apply only to an external service; send `builtIn: false` with them to switch it, as the dashboard does once a card beside a host the environment pins elsewhere is edited. A built-in Redis asks for the `redis` profile only while something the next boot runs uses Redis (`REDIS_ENABLED`, `QUEUE_ENABLED` or `CACHE_ENABLED`, pinned, sent or stored); a save where nothing uses it, in the file either, writes `REDIS_BUILTIN=false` whatever `builtIn` says and drops a `REDIS_HOST` that names the bundled container. A built-in save keeps the stored database password and S3 keys (`S3_ACCESS_KEY_ID`/`S3_SECRET_ACCESS_KEY`, else the legacy `S3_ACCESS_KEY`/`S3_SECRET_KEY`) when the flag was already `true` or the next boot's host already reaches the bundled container, and writes `openwa` or `minioadmin` otherwise. The bundled container counts in any spelling: its alias (`postgres`, `redis`, `minio`) or its container name (`openwa-postgres`, `openwa-redis`, `openwa-minio`), in any letter case, with or without a trailing dot, and for storage as the host of the endpoint URL.+Built-in flags: a section sent without `builtIn` stays in the stored mode. A stored `POSTGRES_BUILTIN`, `REDIS_BUILTIN` or `MINIO_BUILTIN` of `true` writes the bundled connection values again, and the fields sent apply only to an external service; send `builtIn: false` with them to switch it, as the dashboard does once a card beside a host the environment pins elsewhere is edited. A built-in Redis asks for the `redis` profile only while something the next boot runs uses Redis (`REDIS_ENABLED`, `QUEUE_ENABLED` or `CACHE_ENABLED`, pinned, sent or stored); a save where nothing uses it, in the file either, writes `REDIS_BUILTIN=false` whatever `builtIn` says and drops a `REDIS_HOST` that names the bundled container. A built-in save keeps the stored database password and S3 keys (`S3_ACCESS_KEY_ID`/`S3_SECRET_ACCESS_KEY`, else the legacy `S3_ACCESS_KEY`/`S3_SECRET_KEY`) when the flag was already `true` or the next boot's host already reaches the bundled container, and writes `openwa` or `minioadmin` otherwise. It keeps the stored user, database name and schema (`DATABASE_USERNAME`, `DATABASE_NAME`, `POSTGRES_SCHEMA`), or the stored `S3_BUCKET` and `S3_REGION`, when the next boot already runs PostgreSQL or S3 storage on a host that reaches the bundled container (a file that names that container beside a database or bucket of its own), and writes `openwa`, `openwa` and `public`, or `openwa` and `us-east-1`, otherwise. The bundled container counts in any spelling: its alias (`postgres`, `redis`, `minio`) or its container name (`openwa-postgres`, `openwa-redis`, `openwa-minio`), in any letter case, with or without a trailing dot, and for storage as the host of the endpoint URL.```json{diff --git a/src/modules/infra/bundled-host-decisions.spec.ts b/src/modules/infra/bundled-host-decisions.spec.tsindex 8787bc1ee..7fd134aa7 100644--- a/src/modules/infra/bundled-host-decisions.spec.ts+++ b/src/modules/infra/bundled-host-decisions.spec.ts@@ -43,6 +43,9 @@ const SECRETS = {};const STORED_SECRETS ='DATABASE_PASSWORD=StoredPw789!\nS3_ACCESS_KEY_ID=StoredKey12345\nS3_SECRET_ACCESS_KEY=StoredSecret1234567890\n';+// What an external database or bucket was saved with, beside the host of the case.+const STORED_VALUES =+ 'DATABASE_USERNAME=app\nDATABASE_NAME=appdb\nPOSTGRES_SCHEMA=tenant\nS3_BUCKET=media\nS3_REGION=eu-west-1\n';type Position = 'pinned' | 'file';type Flag = 'true' | 'false' | 'absent';@@ -70,13 +73,13 @@ const label = (c: Case) => `${c.service} ${JSON.stringify(c.host)} ${c.position}let pinnedKeys: string[] = [];let fileText = '';-function arrange(c: Case): void {+function arrange(c: Case, stored = ''): void {const spec = SERVICES[c.service];// A pinned host must outrank the file's, so the file holds a host of the other kind.const fileHost = c.position === 'file' ? c.host : c.reach ? spec.externalHost : spec.bundledHost;const lines = [`${spec.selection}=${spec.selected}`, `${spec.host}=${fileHost}`];if (c.flag !== 'absent') lines.push(`${spec.flag}=${c.flag}`);- fileText = `${lines.join('\n')}\n${STORED_SECRETS}`;+ fileText = `${lines.join('\n')}\n${STORED_SECRETS}${stored}`;const pins: Record<string, string> = { ...SECRETS, ...(c.position === 'pinned' ? { [spec.host]: c.host } : {}) };Object.assign(process.env, pins);pinnedKeys = Object.keys(pins);@@ -199,6 +202,22 @@ describe.each(cases.map(c => [label(c), c] as const))('%s', (_label, c) => {});}+ if (c.service !== 'redis') {+ it('keeps the stored user, database and schema, or bucket and region, of a built-in save only when the bundled container already serves them', () => {+ arrange(c, STORED_VALUES);+ const env = saved(+ c.service === 'database'+ ? { database: { type: 'postgres', builtIn: true } }+ : { storage: { type: 's3', builtIn: true } },+ );+ if (c.service === 'database')+ expect([env.DATABASE_USERNAME, env.DATABASE_NAME, env.POSTGRES_SCHEMA]).toEqual(+ reached ? ['app', 'appdb', 'tenant'] : ['openwa', 'openwa', 'public'],+ );+ else expect([env.S3_BUCKET, env.S3_REGION]).toEqual(reached ? ['media', 'eu-west-1'] : ['openwa', 'us-east-1']);+ });+ }+it('keeps the stored mode of a section sent without its flag, and a stored built-in host with it', () => {arrange(c);const section =diff --git a/src/modules/infra/config-sections.ts b/src/modules/infra/config-sections.tsindex 4c05af6fe..bd24b461c 100644--- a/src/modules/infra/config-sections.ts+++ b/src/modules/infra/config-sections.ts@@ -22,8 +22,9 @@ function setSecret(updates: Record<string, string>, key: string, value: string |if (value) updates[key] = value;}-// The host the next boot dials before this save changes it: one the environment pins, else the stored one.-function nextBootHost(key: string, existing: Record<string, string>): string | undefined {+// What the next boot reads for a key (a host, a selection) before this save changes it: the value the environment+// pins, else the stored one.+function nextBootValue(key: string, existing: Record<string, string>): string | undefined {return isEnvPinned(key) ? process.env[key] : existing[key];}@@ -83,7 +84,13 @@ export function applyDatabaseSection(// Built-in PostgreSQL - use container name as hostupdates.DATABASE_HOST = 'postgres';updates.DATABASE_PORT = '5432';- updates.DATABASE_USERNAME = 'openwa';+ // A database the next boot already dials on the bundled container (by a host that names it in any spelling)+ // keeps the user, database and schema it uses there: such a file reads as built-in, and a save that changes+ // nothing else must not move the instance to another database. From anywhere else they belong to another server.+ const servesStored =+ nextBootValue('DATABASE_TYPE', existing) === 'postgres' &&+ namesBundledContainer('postgres', nextBootValue('DATABASE_HOST', existing));+ updates.DATABASE_USERNAME = (servesStored && existing.DATABASE_USERNAME) || 'openwa';// The bundled credential is only the DEFAULT. Secrets are never echoed back to the form,// so an absent password field means "unchanged", not "reset to 'openwa'" — and the// dashboard ALWAYS sends builtIn, so keying the reset on "explicit builtIn:true" reset a@@ -95,15 +102,15 @@ export function applyDatabaseSection(// external DB's credential and must not be carried into the container.const storedPassword =existing.POSTGRES_BUILTIN === 'true' ||- namesBundledContainer('postgres', nextBootHost('DATABASE_HOST', existing))+ namesBundledContainer('postgres', nextBootValue('DATABASE_HOST', existing))? existing.DATABASE_PASSWORD: undefined;updates.DATABASE_PASSWORD = database.password || storedPassword || 'openwa';- updates.DATABASE_NAME = 'openwa';+ updates.DATABASE_NAME = (servesStored && existing.DATABASE_NAME) || 'openwa';// Built-in Postgres is initialized with the default 'public' schema (see// scripts/postgres-init-schema.sh). Pin it so a later switch from a custom-schema// external DB to built-in doesn't carry a stale POSTGRES_SCHEMA forward.- updates.POSTGRES_SCHEMA = 'public';+ updates.POSTGRES_SCHEMA = (servesStored && existing.POSTGRES_SCHEMA) || 'public';profiles.push('postgres');} else {// External PostgreSQL. Flipping built-in -> external must not carry the bundled@@ -239,13 +246,17 @@ export function applyStorageSection(// password: keys the next boot already dials the bundled endpoint with (the flag already set, or the bundled// endpoint pinned or stored, in any spelling) are the ones that container was created with, and are kept.const keepKeys =- existing.MINIO_BUILTIN === 'true' || namesBundledContainer('minio', nextBootHost('S3_ENDPOINT', existing));+ existing.MINIO_BUILTIN === 'true' || namesBundledContainer('minio', nextBootValue('S3_ENDPOINT', existing));updates.S3_ENDPOINT = 'http://minio:9000';updates.S3_ACCESS_KEY_ID = (keepKeys && (existing.S3_ACCESS_KEY_ID || existing.S3_ACCESS_KEY)) || 'minioadmin';updates.S3_SECRET_ACCESS_KEY =(keepKeys && (existing.S3_SECRET_ACCESS_KEY || existing.S3_SECRET_KEY)) || 'minioadmin';- updates.S3_BUCKET = 'openwa';- updates.S3_REGION = 'us-east-1';+ // As for the database: a bucket the next boot already reaches on the bundled MinIO is kept with its region.+ const servesStored =+ nextBootValue('STORAGE_TYPE', existing) === 's3' &&+ namesBundledContainer('minio', nextBootValue('S3_ENDPOINT', existing));+ updates.S3_BUCKET = (servesStored && existing.S3_BUCKET) || 'openwa';+ updates.S3_REGION = (servesStored && existing.S3_REGION) || 'us-east-1';profiles.push('minio');} else {// External S3/MinIO. Flipping built-in -> external must not carry the bundleddiff --git a/src/modules/infra/infra-config.controller.spec.ts b/src/modules/infra/infra-config.controller.spec.tsindex 90520af25..cd41b266d 100644--- a/src/modules/infra/infra-config.controller.spec.ts+++ b/src/modules/infra/infra-config.controller.spec.ts@@ -1578,6 +1578,98 @@ describe('InfraConfigController.saveConfig built-in/external mode flips and the});});+ // A file naming the bundled container beside an external database or bucket reads as built-in, so the page sends+ // builtIn true for it on a save of any other card. The next boot already dials that container with the stored+ // user, database, schema and bucket, and such a save keeps them.+ describe('built-in save over a database and bucket the bundled containers already serve', () => {+ const FILE = (db: string, s3: string) =>+ `DATABASE_TYPE=postgres\nPOSTGRES_BUILTIN=false\nDATABASE_HOST=${db}\nDATABASE_USERNAME=app\n` ++ 'DATABASE_PASSWORD=AppPw12345!\nDATABASE_NAME=appdb\nPOSTGRES_SCHEMA=tenant\nSTORAGE_TYPE=s3\nMINIO_BUILTIN=false\n' ++ `S3_ENDPOINT=${s3}\nS3_BUCKET=media\nS3_REGION=eu-west-1\nS3_ACCESS_KEY_ID=AppKey1234567\n` ++ 'S3_SECRET_ACCESS_KEY=AppSecret1234567890\n';+ // What the page sends when the operator saves another card: the toggles seeded on, the hidden fields as loaded.+ const pageSave = (db: string, s3: string) => ({+ database: {+ type: 'postgres',+ builtIn: true,+ host: db,+ port: '5432',+ username: 'app',+ password: '',+ database: 'appdb',+ schema: 'tenant',+ poolSize: 10,+ sslEnabled: false,+ sslRejectUnauthorized: true,+ },+ storage: {+ type: 's3',+ builtIn: true,+ localPath: './data/media',+ s3Bucket: 'media',+ s3Region: 'eu-west-1',+ s3AccessKey: '',+ s3SecretKey: '',+ s3Endpoint: s3,+ },+ engine: { headless: false },+ });++ afterEach(() => {+ delete process.env.DATABASE_HOST;+ delete process.env.S3_ENDPOINT;+ recordPinnedEnvKeys({});+ });++ it.each(HOST_SPELLINGS.database.reach.map((db, i) => [db, HOST_SPELLINGS.storage.reach[i]]))(+ 'keeps them under %j and %j',+ (db, s3) => {+ const env = written(pageSave(db, s3), FILE(db, s3));+ for (const line of ['DATABASE_USERNAME=app', 'DATABASE_NAME=appdb', 'POSTGRES_SCHEMA=tenant']) {+ expect(env).toContain(line);+ }+ expect(env).toContain('S3_BUCKET=media');+ expect(env).toContain('S3_REGION=eu-west-1');+ },+ );++ it('writes the bundled ones over another server, as a switch to the built-in container', () => {+ const env = written(+ pageSave('db.example.com', 'https://s3.example.com'),+ FILE('db.example.com', 'https://s3.example.com'),+ );+ for (const line of ['DATABASE_USERNAME=openwa', 'DATABASE_NAME=openwa', 'POSTGRES_SCHEMA=public']) {+ expect(env).toContain(line);+ }+ expect(env).toContain('S3_BUCKET=openwa');+ expect(env).toContain('S3_REGION=us-east-1');+ });++ // A SQLite boot dials no PostgreSQL, and its DATABASE_NAME is a file path.+ it('writes the bundled ones when the next boot runs SQLite beside a bundled host pin', () => {+ process.env.DATABASE_HOST = 'postgres';+ recordPinnedEnvKeys({ DATABASE_HOST: 'postgres' });+ const env = written(+ { database: { type: 'postgres', builtIn: true } },+ 'DATABASE_TYPE=sqlite\nPOSTGRES_BUILTIN=false\nDATABASE_NAME=./data/openwa.sqlite\n',+ );+ expect(env).toContain('DATABASE_NAME=openwa');+ expect(env).not.toContain('openwa.sqlite');+ });++ // Likewise local storage dials no MinIO, whatever bucket an older file left behind.+ it('writes the bundled bucket when the next boot keeps local storage beside a bundled endpoint pin', () => {+ process.env.S3_ENDPOINT = 'http://minio:9000';+ recordPinnedEnvKeys({ S3_ENDPOINT: 'http://minio:9000' });+ const env = written(+ { storage: { type: 's3', builtIn: true } },+ 'STORAGE_TYPE=local\nMINIO_BUILTIN=false\nS3_BUCKET=media\nS3_REGION=eu-west-1\n',+ );+ expect(env).toContain('S3_BUCKET=openwa');+ expect(env).toContain('S3_REGION=us-east-1');+ });+ });+// The guard must evaluate what the next boot would SEE: load-env.ts loads with dotenv// override:false, so a value in the container environment (compose `environment:`) wins over the// saved file. Snapshot + clear every guard-relevant key so these tests are hermetic.