Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
docs(security): cover demoted keys and Kubernetes in admin key recovery (#1820)
Yudhi Armyndharise1092d63f647
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e6a4c81a0..bad41243e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -114,7 +114,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0- Document the memory media messages and parked webhook deliveries hold during a media burst, and the settings that limit it.- The storage migration guide says to check the store size and file count against the import caps before switching backends, and how to re-run an aborted import.- The README and architecture docs describe the Local/S3 storage backend as the live media store, not a backup target.-- Document how to recover a lost admin API key without revoking the other keys.+- Document how to recover a lost admin API key without revoking the other keys, on source, Compose and Helm installs.- The integration docs say `integration_delivery_failures` holds only failed inbound deliveries.- The webhook runbook documents the ADMIN delivery-failure redrive and no longer says failed deliveries retry on their own.diff --git a/docs/04-security-design.md b/docs/04-security-design.mdindex c81def4b1..41ec9c25f 100644--- a/docs/04-security-design.md+++ b/docs/04-security-design.md@@ -106,24 +106,30 @@ Keys are stored as one-way hashes, so a lost key cannot be read back from the dafound where it was saved, or replaced. Try these in order. If every key stopped working at once after`API_KEY_PEPPER` was set or changed, follow the pepper recovery in §4.4 instead.-1. **The bootstrap key file or `API_MASTER_KEY`.** The key seeded on first boot is kept in- `data/.api-key` (or the `BOOTSTRAP_KEY_FILE` path) until that key is no longer active (revoked,- expired or deleted). While the file holds a live key, the startup banner ends with- `(full key in <path>)`. Read it with `cat data/.api-key`, or with- `docker compose exec openwa-api cat /app/data/.api-key` on the compose deployment. If- `API_MASTER_KEY` was set before first boot, the seeded key is that value, and it still works- unless it was revoked since.+1. **The bootstrap key file or `API_MASTER_KEY`.** Whenever `api_keys` is empty (first boot, or+ after step 4 or the §4.4 pepper reset), the gateway seeds an ADMIN key, set to `API_MASTER_KEY`+ when that is set, and keeps it in `data/.api-key` (or the `BOOTSTRAP_KEY_FILE` path) until that+ key is no longer active (revoked, expired or deleted). While the file holds a live key, the+ startup banner ends with `(full key in <path>)`. Read it with `cat data/.api-key`, with+ `docker compose exec openwa-api cat /app/data/.api-key` on the compose deployment, or with+ `kubectl exec openwa-0 -- cat /app/data/.api-key` on the Helm chart (release `openwa`). The key+ manages keys only while it still meets step 2's conditions. Once it is demoted from `admin`,+ given any `allowedSessions` or `allowedChats`, or given `allowedIps` that exclude your address,+ the file and the banner line remain but key management is refused; go on to step 2 or 3.2. **Another ADMIN key.** Any active, unexpired `admin` key without `allowedSessions` or- `allowedChats` can manage keys: mint a replacement with `POST /api/auth/api-keys`- (`{"name": "Admin", "role": "admin"}`; the raw key is returned once, in `apiKey`), then revoke the- lost one with `POST /api/auth/api-keys/:id/revoke`, or do both from the dashboard's API Keys page.+ `allowedChats`, used from an address its `allowedIps` (when set) admits, can manage keys: mint a+ replacement with `POST /api/auth/api-keys` (`{"name": "Admin", "role": "admin"}`; the raw key is+ returned once, in `apiKey`), then revoke the lost one with `POST /api/auth/api-keys/:id/revoke`,+ or do both from the dashboard's API Keys page.3. **Replace the key in the main database.** With no usable ADMIN key left, give the lost key's rowa new key; every other key keeps working. Stop the instance and copy the main database aside. Thengenerate a key and its hash with the gateway's own hash function and configuration (process env,`.env` and `data/.env.generated`, so the same `API_KEY_PEPPER`). Run it from the gateway's install- directory, the one holding `dist/` and `data/`, so it reads that install's configuration:+ directory, the one holding `dist/` and `data/`, so it reads that install's configuration. When a+ service manager (systemd, pm2) sets `API_KEY_PEPPER` only in its unit, export the same value in+ the shell first, or the hash will not match:```bashGEN="require('./dist/config/load-env'); const key = 'owa_k1_' + require('crypto').randomBytes(32).toString('hex'); console.log('key:', key); console.log('prefix:', key.slice(0, 12)); console.log('hash:', require('./dist/modules/auth/api-key-hash').hashApiKey(key, process.env.API_KEY_PEPPER))"@@ -140,6 +146,15 @@ found where it was saved, or replaced. Try these in order. If every key stoppedsqlite3 data/main.sqlite "UPDATE api_keys SET keyHash = '<hash>', keyPrefix = '<prefix>', isActive = 1, expiresAt = NULL, allowedIps = NULL, allowedSessions = NULL, allowedChats = NULL WHERE id = '<id>'"```+ On the Helm chart (release `openwa`), run the generator in the running pod before stopping it,+ since only that pod has the release's Secret and so its `API_KEY_PEPPER`:+ `kubectl exec openwa-0 -- node -e "$GEN"`. Then scale the StatefulSet to 0, start the+ `openwa-restore` helper pod on the release's data PVC as+ [11 - Runbook: Restore from Backup](./11-operational-runbooks.md#runbook-restore-from-backup)+ does, copy the database aside and run both `sqlite3` commands there with+ `kubectl exec openwa-restore --` against `/app/data/main.sqlite`, then delete the pod and scale+ back to 1.+Start the instance and use the new key. The update also clears the row's revoked state, expiry,and IP, session and chat restrictions, so it can manage keys again; narrow it afterwards throughthe API if it carried any. The old key no longer authenticates.@@ -247,7 +262,7 @@ OpenWA serves plain HTTP on its port; terminate **TLS at your reverse proxy / lo**Hardening you can apply today:** set `API_KEY_PEPPER`; restrict the data volume and database to the app's user; and encrypt at the infrastructure layer (LUKS / cloud-provider encrypted volumes / an encrypted managed Postgres) rather than relying on application-level field encryption, which is not implemented.-**Setting or changing `API_KEY_PEPPER` on an existing install invalidates every key, the admin key included.** No stored hash matches any more, minting a key needs a valid ADMIN key, and a key is seeded only into an empty `api_keys` table, so nothing can be re-issued through the API. Set the pepper before first boot. On a running install: stop the instance, set the pepper, empty the table in the main database (the `MAIN_DATABASE_NAME` path, `data/main.sqlite` by default; on a source install `sqlite3 data/main.sqlite "DELETE FROM api_keys"`, on the compose deployment `docker compose run --rm --no-deps openwa-api sqlite3 /app/data/main.sqlite "DELETE FROM api_keys"`), start the instance, take the new ADMIN key from the startup banner or `data/.api-key` (or set `API_MASTER_KEY` beforehand to choose it), then re-issue the other keys.+**Setting or changing `API_KEY_PEPPER` on an existing install invalidates every key, the admin key included.** No stored hash matches any more, minting a key needs a valid ADMIN key, and a key is seeded only into an empty `api_keys` table, so nothing can be re-issued through the API. Set the pepper before first boot. On a running install: stop the instance, set the pepper, empty the table in the main database (the `MAIN_DATABASE_NAME` path, `data/main.sqlite` by default; on a source install `sqlite3 data/main.sqlite "DELETE FROM api_keys"`, on the compose deployment `docker compose run --rm --no-deps openwa-api sqlite3 /app/data/main.sqlite "DELETE FROM api_keys"`), start the instance, take the new ADMIN key from the startup banner or `data/.api-key` (or set `API_MASTER_KEY` beforehand to choose it), then re-issue the other keys. On the Helm chart, empty the table while the StatefulSet is at 0 from the `openwa-restore` helper pod of [11 - Runbook: Restore from Backup](./11-operational-runbooks.md#runbook-restore-from-backup) (`kubectl exec openwa-restore -- sqlite3 /app/data/main.sqlite "DELETE FROM api_keys"`), delete the pod, and only then set the pepper in the release's Secret (`secretEnv.API_KEY_PEPPER`, or the `existingSecret`) and scale back to 1; a `helm upgrade` that sets the pepper also scales back to `replicaCount`. Read the new key with `kubectl exec openwa-0 -- cat /app/data/.api-key`.## 4.5 Input Validationdiff --git a/docs/12-troubleshooting-faq.md b/docs/12-troubleshooting-faq.mdindex 5d4f7262d..243923c76 100644--- a/docs/12-troubleshooting-faq.md+++ b/docs/12-troubleshooting-faq.md@@ -1417,7 +1417,7 @@ retention/encryption to completed archives externally; see the [backup and resto**Q: I lost the admin API key. How do I get back in?**-> A: Keys are stored hashed, so the database cannot give it back. Check `data/.api-key` (or `BOOTSTRAP_KEY_FILE`) and the `API_MASTER_KEY` value the instance was first started with, then use another unscoped ADMIN key to mint a replacement. With neither, stop the instance and write a new key's hash into the lost key's row in the main database, which keeps every other key working. The steps are in [Recovering a lost admin key](./04-security-design.md#recovering-a-lost-admin-key).+> A: Keys are stored hashed, so the database cannot give it back. Try the key in `data/.api-key` (or `BOOTSTRAP_KEY_FILE`), or the `API_MASTER_KEY` value set when the key table was last seeded, which still works while that key is an active, unscoped ADMIN key. Failing that, use another unscoped ADMIN key to mint a replacement. With neither, stop the instance and write a new key's hash into the lost key's row in the main database, which keeps every other key working. The steps are in [Recovering a lost admin key](./04-security-design.md#recovering-a-lost-admin-key).### Webhook Questionsdiff --git a/src/common/docs-governance.spec.ts b/src/common/docs-governance.spec.tsindex 6341b3d7f..816883c8f 100644--- a/src/common/docs-governance.spec.ts+++ b/src/common/docs-governance.spec.ts@@ -297,6 +297,28 @@ describe('governance docs match the repository', () => {);});+ // The bootstrap key file outlives a demotion or a scope change, and the Helm chart's StatefulSet+ // leaves no pod to exec into once it is scaled to 0 for the database write.+ it('gives the admin key recovery its Helm commands and the bootstrap key caveat', () => {+ const security = read('docs/04-security-design.md');+ const recovery = between(security, '### Recovering a lost admin key', '\n## ');+ expect(read('charts/openwa/templates/NOTES.txt')).toContain('default "/app/data/.api-key"');+ expect(recovery).toContain('`kubectl exec openwa-0 -- cat /app/data/.api-key`');+ expect(recovery).toMatch(/demoted from `admin`[\s\S]+go on to step 2 or 3/);+ expect(recovery).toMatch(/given any `allowedSessions` or `allowedChats`, or given `allowedIps` that exclude/);+ expect(recovery).toMatch(/`allowedIps` \(when set\) admits/);+ expect(recovery).toContain('`kubectl exec openwa-0 -- node -e "$GEN"`');+ expect(recovery).toContain('`kubectl exec openwa-restore --`');+ const restore = between(read('docs/11-operational-runbooks.md'), '### Runbook: Restore from Backup', '\n### ');+ expect(restore).toContain('name: openwa-restore');+ expect(restore).toContain('claimName: data-openwa-0');+ const pepper = between(security, '**Setting or changing `API_KEY_PEPPER`', '\n\n');+ expect(pepper).toContain('`kubectl exec openwa-restore -- sqlite3 /app/data/main.sqlite "DELETE FROM api_keys"`');+ const faq = between(read('docs/12-troubleshooting-faq.md'), '**Q: I lost the admin API key.', '\n\n### ');+ expect(faq).not.toMatch(/first started with/);+ expect(faq).toMatch(/last seeded, which still works while that key is an active, unscoped ADMIN key/);+ });+it('keeps the README non-affiliation disclaimer', () => {const section = between(read('README.md'), '## Disclaimer', '\n## ');expect(section).toMatch(/not affiliated/);