Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(infra): refuse a storage path the app cannot write
Yudhi Armyndharise07acdddd0af
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 314c33d3c..403b95c23 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -51,6 +51,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Fixed- Bound each webhook and ingress queue command at 2 s once Redis has connected, so a Redis that stops answering falls back to direct delivery and inline dispatch instead of holding every dispatch and ingress request.+- Refuse to save a local `storage.localPath` the app cannot create or write, where the next start refused to boot and the dashboard that could undo the save was down.- Answer `400` for a send route's or a product send's `chatId`, or a bulk `batchId`, over 255 characters, where PostgreSQL answered `500` once the value passed its index entry limit.- Answer `400` for a chat action's `chatId` over 255 characters, as the send routes do, instead of handing it to the engine.- Use the body hash as the delivery id of an ingress request whose dedup header is longer than 255 characters, where PostgreSQL answered `500` once the value passed its index entry limit.diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 4aacab40d..507b2f9b6 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -5726,11 +5726,12 @@ Merge-save infrastructure config to `data/.env.generated` (a `0600` secret file)}```-Write/IO errors are caught and returned as HTTP `200` with `{ "saved": false, "envPath": "", "profiles": [], "message": "Failed to save configuration: …" }`. DTO validation, an unknown engine type, an invalid `database.schema`, CR/LF injection, a value that cannot be stored so that it reads back unchanged, and a merged config the production boot guard would reject (an empty or default `DATABASE_PASSWORD` or S3 key on an external datastore, or a placeholder `REDIS_PASSWORD`; message starts `Refusing to save a configuration that would be rejected at production boot.`) are real HTTP `400` responses, and nothing is written. `profiles` lists newly-required Docker profiles.+Write/IO errors are caught and returned as HTTP `200` with `{ "saved": false, "envPath": "", "profiles": [], "message": "Failed to save configuration: …" }`. DTO validation, an unknown engine type, an invalid `database.schema`, CR/LF injection, a value that cannot be stored so that it reads back unchanged, and a merged config the production boot guard would reject (an empty or default `DATABASE_PASSWORD` or S3 key on an external datastore, or a placeholder `REDIS_PASSWORD`; message starts `Refusing to save a configuration that would be rejected at production boot.`), and a local `storage.localPath` the app cannot create or write (message starts `Refusing to save the storage path`; the next start would refuse to boot on it, and a path pinned by the host environment or the project `.env` is not judged, as the boot never reads the submitted one) are real HTTP `400` responses, and nothing is written. `profiles` lists newly-required Docker profiles.**Errors:** `400` unknown/mistyped body field, missing nested `database.type`/`storage.type`, unknown`engine.type`, invalid `database.schema`, CR-LF in a value, a value that cannot round-trip through the-env file, or a config the production boot secret guard would reject · `401` · `403`+env file, a config the production boot secret guard would reject, or a local+`storage.localPath` the app cannot create or write · `401` · `403`---diff --git a/openapi.json b/openapi.jsonindex 9eabf2489..f8e30f029 100644--- a/openapi.json+++ b/openapi.json@@ -8283,7 +8283,7 @@}},"400": {- "description": "Rejected before anything was written: an invalid or unknown field, an unknown engine type, a value with a line break or one that cannot be stored, or a configuration that would not boot in production.",+ "description": "Rejected before anything was written: an invalid or unknown field, an unknown engine type, a value with a line break or one that cannot be stored, a configuration that would not boot in production, or a media storage path the app cannot write to.","content": {"application/json": {"schema": {diff --git a/src/modules/infra/infra-config.controller.spec.ts b/src/modules/infra/infra-config.controller.spec.tsindex fe25127c5..05e65425e 100644--- a/src/modules/infra/infra-config.controller.spec.ts+++ b/src/modules/infra/infra-config.controller.spec.ts@@ -1,4 +1,6 @@import * as fs from 'fs';+import * as os from 'os';+import * as path from 'path';// StorageService (imported transitively by the infra controllers) pulls in `archiver`// v8, which is ESM-only and cannot be parsed by ts-jest. The controller logic@@ -22,6 +24,13 @@ jest.mock('fs', () => {};});+// The save guard probes the storage root the way the boot does, which creates the directory. Keep a save+// in a test from making one in the working tree; the cases that need the real probe ask for it.+jest.mock('../../config/storage-root', () => {+ const actual = jest.requireActual<typeof import('../../config/storage-root')>('../../config/storage-root');+ return { ...actual, resolveStorageRoot: jest.fn((options: { configured?: string }) => options.configured ?? '') };+});+import { BadRequestException } from '@nestjs/common';import { plainToInstance } from 'class-transformer';import { validate } from 'class-validator';@@ -29,6 +38,7 @@ import { InfraConfigController } from './infra-config.controller';import { SaveConfigDto } from './dto/save-config.dto';import { AuditAction } from '../audit/entities/audit-log.entity';import { recordOsEnvKeys, recordPinnedEnvKeys } from '../../config/env-precedence';+import { resolveStorageRoot } from '../../config/storage-root';// With no boot snapshot every process.env key counts as host-supplied, so a shell exporting the// CI Postgres or compose Redis settings would otherwise feed the save guard and the pin checks.@@ -110,6 +120,81 @@ describe('InfraConfigController.saveConfig values the next boot refuses', () =>});});+// The next boot refuses to start on a media root the app cannot create or write, before Nest builds, and+// the dashboard that could undo the save is down with it.+describe('InfraConfigController.saveConfig local storage path', () => {+ const newController = () => new InfraConfigController({} as never, {} as never, {} as never);+ const actual = jest.requireActual<typeof import('../../config/storage-root')>('../../config/storage-root');+ const useRealProbe = () => (resolveStorageRoot as jest.Mock).mockImplementationOnce(actual.resolveStorageRoot);+ // A parent that is a file, so the directory cannot be created whoever runs the test.+ const unwritable = '/dev/null/openwa-media';+ const localAt = (localPath: string) => ({ storage: { type: 'local', localPath } }) as never;++ beforeEach(() => {+ (fs.writeFileSync as jest.Mock).mockClear();+ (resolveStorageRoot as jest.Mock).mockClear();+ });++ it('refuses a path the app cannot create, and writes nothing', () => {+ useRealProbe();++ expect(() => newController().saveConfig(localAt(unwritable))).toThrow(BadRequestException);+ expect(fs.writeFileSync).not.toHaveBeenCalled();+ });++ it('names the submitted path, not where it resolves to', () => {+ // Relative, under a regular file (this one), so its absolute form carries the working directory.+ const relative = path.relative(process.cwd(), path.join(__filename, 'media'));+ useRealProbe();++ let message = '';+ try {+ newController().saveConfig(localAt(relative));+ } catch (error) {+ message = (error as Error).message;+ }++ expect(message).toContain(`storage path '${relative}'`);+ expect(message).not.toContain(process.cwd());+ });++ it('saves a path the app can write', () => {+ const root = fs.mkdtempSync(path.join(os.tmpdir(), 'openwa-storage-'));+ try {+ useRealProbe();++ newController().saveConfig(localAt(path.join(root, 'media')));++ const content = ((fs.writeFileSync as jest.Mock).mock.calls as Array<[string, string]>)[0][1];+ expect(content).toContain(`STORAGE_LOCAL_PATH=${path.join(root, 'media')}`);+ } finally {+ fs.rmSync(root, { recursive: true, force: true });+ }+ });++ it('does not judge a path that a value pinned by the host outranks', () => {+ // With no boot snapshot recorded, every key in the environment counts as host-supplied.+ process.env.STORAGE_LOCAL_PATH = '/srv/openwa/media';+ try {+ useRealProbe();++ newController().saveConfig(localAt(unwritable));++ expect(resolveStorageRoot).not.toHaveBeenCalled();+ expect(fs.writeFileSync).toHaveBeenCalledTimes(1);+ } finally {+ delete process.env.STORAGE_LOCAL_PATH;+ }+ });++ it('leaves a save that names no path alone', () => {+ newController().saveConfig({ queue: { enabled: false } });++ expect(resolveStorageRoot).not.toHaveBeenCalled();+ expect(fs.writeFileSync).toHaveBeenCalledTimes(1);+ });+});+describe('InfraConfigController PostgreSQL schema (POSTGRES_SCHEMA)', () => {const newController = () => new InfraConfigController({} as never, {} as never, {} as never);diff --git a/src/modules/infra/infra-config.controller.ts b/src/modules/infra/infra-config.controller.tsindex f01c42e4e..c5cdcc935 100644--- a/src/modules/infra/infra-config.controller.ts+++ b/src/modules/infra/infra-config.controller.ts@@ -25,6 +25,7 @@ import { AuditAction } from '../audit/entities/audit-log.entity';import { SaveConfigDto } from './dto/save-config.dto';import { assertNoDefaultSecretsInProduction } from '../../config/bootstrap-security';import { BLANK_SHADOWED_ENV_KEYS, isEnvPinned, isOsProvidedEnv } from '../../config/env-precedence';+import { resolveStorageRoot } from '../../config/storage-root';import * as fs from 'fs';import * as path from 'path';import * as dotenv from 'dotenv';@@ -192,7 +193,8 @@ export class InfraConfigController {status: 400,description:'Rejected before anything was written: an invalid or unknown field, an unknown engine type, a value with a ' +- 'line break or one that cannot be stored, or a configuration that would not boot in production.',+ 'line break or one that cannot be stored, a configuration that would not boot in production, or a media ' ++ 'storage path the app cannot write to.',})@ApiBody({ description: 'Configuration to save', type: SaveConfigDto })saveConfig(@Body() config: SaveConfigDto): { message: string; saved: boolean; envPath: string; profiles: string[] } {@@ -220,6 +222,7 @@ export class InfraConfigController {this.assertNoLineBreakValues(updates);const merged = this.mergeWithExisting(existing, ctx);this.assertProductionBootable(merged);+ this.assertStorageRootWritable(updates, merged);this.persistGeneratedEnv(envPath, merged);this.auditConfigSaved(config, profiles);@@ -293,30 +296,27 @@ export class InfraConfigController {return merged;}- private assertProductionBootable(merged: Record<string, string>): void {- // Save-time production guard. The file is loaded on the NEXT boot, which may run with- // NODE_ENV=production regardless of this process's environment — so evaluate the merged- // result with the very same boot guard (as production) and refuse the save when that boot- // would refuse to start. This is what stops a built-in->external flip with no fresh- // credentials from persisting a config that crash-loops the next production boot.- // Evaluate what that boot would actually SEE, not just what the file holds: load-env.ts- // loads with dotenv override:false, so a value supplied via the container environment- // (compose `environment:`) wins over this file — the precedence the file header documents.- // Without that, a deployment providing DATABASE_PASSWORD & co. through the environment is- // refused on EVERY save even though its boot passes the guard. A pinned blank is kept as-is:- // clearBlankEnv drops blank host forwards before either snapshot, so a blank that is still- // pinned is a `KEY=` line in ./.env, and boot keeps it. Only without a snapshot (a process that- // never ran load-env) does a blank on a blank-forwarded key count as unset, as clearBlankEnv would.- //- // Only a key from a layer ABOVE the file may win: the host, or the project .env, which load-env- // also loads ahead of data/.env.generated with override:false. load-env merges the generated- // file into process.env too, so reading process.env alone would hand back the very file this- // save is replacing: the guard would then bless a flip by validating the OLD config (a- // built-in -> external switch keeping the bundled 'openwa' password would save cleanly and- // crash-loop the next production boot, the exact case this guard exists for). isEnvPinned's- // snapshot is taken before that file loads; isOsProvidedEnv keeps the no-snapshot default of- // assuming an override.- const bootValue = (key: string): string | undefined => {+ // The value the next boot reads for `key`, so the save-time guards judge what it will see.+ //+ // Evaluate what that boot would actually SEE, not just what the file holds: load-env.ts+ // loads with dotenv override:false, so a value supplied via the container environment+ // (compose `environment:`) wins over this file — the precedence the file header documents.+ // Without that, a deployment providing DATABASE_PASSWORD & co. through the environment is+ // refused on EVERY save even though its boot passes the guard. A pinned blank is kept as-is:+ // clearBlankEnv drops blank host forwards before either snapshot, so a blank that is still+ // pinned is a `KEY=` line in ./.env, and boot keeps it. Only without a snapshot (a process that+ // never ran load-env) does a blank on a blank-forwarded key count as unset, as clearBlankEnv would.+ //+ // Only a key from a layer ABOVE the file may win: the host, or the project .env, which load-env+ // also loads ahead of data/.env.generated with override:false. load-env merges the generated+ // file into process.env too, so reading process.env alone would hand back the very file this+ // save is replacing: the guard would then bless a flip by validating the OLD config (a+ // built-in -> external switch keeping the bundled 'openwa' password would save cleanly and+ // crash-loop the next production boot, the exact case this guard exists for). isEnvPinned's+ // snapshot is taken before that file loads; isOsProvidedEnv keeps the no-snapshot default of+ // assuming an override.+ private bootValueReader(merged: Record<string, string>): (key: string) => string | undefined {+ return (key: string): string | undefined => {if (isEnvPinned(key) && process.env[key] !== undefined) return process.env[key];const envValue = isOsProvidedEnv(key) ? process.env[key] : undefined;if (envValue !== undefined && (envValue.trim() !== '' || !BLANK_SHADOWED_ENV_KEYS.includes(key))) {@@ -324,6 +324,15 @@ export class InfraConfigController {}return merged[key];};+ }++ // Save-time production guard. The file is loaded on the NEXT boot, which may run with+ // NODE_ENV=production regardless of this process's environment — so evaluate the merged+ // result with the very same boot guard (as production) and refuse the save when that boot+ // would refuse to start. This is what stops a built-in->external flip with no fresh+ // credentials from persisting a config that crash-loops the next production boot.+ private assertProductionBootable(merged: Record<string, string>): void {+ const bootValue = this.bootValueReader(merged);try {assertNoDefaultSecretsInProduction({nodeEnv: 'production',@@ -347,6 +356,26 @@ export class InfraConfigController {}}+ // A storage path the app cannot create or write is refused by the next boot before Nest builds, which+ // crash-loops the instance with the dashboard that could undo the save down. Only a path this save+ // names is judged, and only when the boot will read it: a value the host or the project .env pins+ // outranks the file, so the submitted one never reaches the boot. The probe is the boot's own, so the+ // fossil `./uploads` it tolerates is tolerated here too. It names the submitted value only, not the+ // absolute path it resolves to, as the save response does not disclose the host's layout.+ private assertStorageRootWritable(updates: Record<string, string>, merged: Record<string, string>): void {+ const submitted = updates.STORAGE_LOCAL_PATH;+ if (submitted === undefined || this.bootValueReader(merged)('STORAGE_LOCAL_PATH') !== submitted) return;+ try {+ resolveStorageRoot({ configured: submitted });+ } catch {+ throw new BadRequestException(+ `Refusing to save the storage path '${submitted}': the app cannot create or write it, and the next start ` ++ 'would refuse to boot. Use a directory the app user owns, inside the data volume when running in ' ++ 'Docker (for example ./data/media).',+ );+ }+ }+private persistGeneratedEnv(envPath: string, merged: Record<string, string>): void {const unreadable = (key: string) =>new BadRequestException(