Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(ingress): flatten query values, fence ack headers, survive a bad zip
Yudhi Armyndharis1cb2c039e7c3
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 1d9e8d40e..187adc584 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -6693,19 +6693,19 @@ response only; the `verifyToken` is also shown (unchanged).#### GET /api/ingress/:pluginId/:instanceId/:path-**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded+**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded · `503` a route declaring a `session-alive` preflight whose bound session is not connected#### POST /api/ingress/:pluginId/:instanceId/:path-**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded+**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded · `503` a route declaring a `session-alive` preflight whose bound session is not connected#### PUT /api/ingress/:pluginId/:instanceId/:path-**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded+**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded · `503` a route declaring a `session-alive` preflight whose bound session is not connected#### PATCH /api/ingress/:pluginId/:instanceId/:path-**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded+**Errors:** `401` signature verification failed (missing, stale, or wrong per-instance secret) · `413` payload too large · `429` rate limit exceeded · `503` a route declaring a `session-alive` preflight whose bound session is not connected#### DELETE /api/ingress/:pluginId/:instanceId/:path@@ -6724,7 +6724,7 @@ plugin processing. `200` means the `GET` verification-challenge echo, or a routedeclarative `ack` config (doc 25); the plugin itself always runs async. A re-delivery of analready-persisted event is answered with that same ack, byte for byte, and is not enqueued again.-**Errors:** `401` signature verification failed (missing, stale, or wrong secret) · `403` `GET` verification challenge failed (`verifyToken` mismatch) · `404` unknown pluginId/instanceId, or no such claimed route · `413` body over the route's `maxBodyBytes` · `429` rate limit: the per-instance bucket (`INGRESS_INSTANCE_LIMIT`) or the per-client-IP bucket (`INGRESS_IP_LIMIT`), both per `INGRESS_INSTANCE_TTL`; the global per-IP tiers skip this route, so these two are its bounds, and `Retry-After-instance` / `Retry-After-ingress-ip` names the one that shed the request+**Errors:** `401` signature verification failed (missing, stale, or wrong secret) · `403` `GET` verification challenge failed (`verifyToken` mismatch) · `404` unknown pluginId/instanceId, or no such claimed route · `413` body over the route's `maxBodyBytes` · `429` rate limit: the per-instance bucket (`INGRESS_INSTANCE_LIMIT`) or the per-client-IP bucket (`INGRESS_IP_LIMIT`), both per `INGRESS_INSTANCE_TTL`; the global per-IP tiers skip this route, so these two are its bounds, and `Retry-After-instance` / `Retry-After-ingress-ip` names the one that shed the request, alongside a plain `Retry-After` carrying the same delay · `503` a route declaring a `session-alive` preflight whose bound session is not connected: the delivery is not persisted, so the provider's retry is treated as a new delivery, and `Retry-After` carries the delay## 6.5 Real-time API (WebSocket)diff --git a/docs/25-integration-fabric.md b/docs/25-integration-fabric.mdindex 3e1acedc2..f4c841455 100644--- a/docs/25-integration-fabric.md+++ b/docs/25-integration-fabric.md@@ -159,9 +159,11 @@ Four tables live on the data connection, each created by a hand-authored dual-di- **Authentication inversion.** A provider webhook cannot carry an OpenWA API key, so ingress is public tothe API-key guard but validates a **per-instance HMAC (or shared secret)** over the **raw** requestbytes with a constant-time comparison. The raw body is preserved by a verify callback on the body parser- because a re-serialized payload is not byte-identical to what the provider signed. The global rate-limit- guard still applies, and the payload is intentionally not bound to a DTO so strict validation cannot- reject unknown provider fields.+ because a re-serialized payload is not byte-identical to what the provider signed. The route is exempt from+ the global per-IP throttle and bounded by its own guard instead, on two keys, `(pluginId, instanceId)` and+ the client IP: a provider delivering every tenant's webhooks from one egress address would otherwise be+ shed at the global tier before the per-instance bound ever fired. The payload is intentionally not bound+ to a DTO so strict validation cannot reject unknown provider fields.- **Replay and duplication.** A signed-timestamp tolerance rejects stale deliveries, and`(pluginId, instanceId, providerDeliveryId)` deduplication plus a queue job id keyed on the delivery idprovides best-effort de-duplication when the provider supplies a stable delivery id. Standard Webhooks defaultsdiff --git a/openapi.json b/openapi.jsonindex 9c8b1cee0..8680b71c6 100644--- a/openapi.json+++ b/openapi.json@@ -8909,7 +8909,10 @@"description": "Request body exceeds the route maxBodyBytes limit."},"429": {- "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request."+ "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name."+ },+ "503": {+ "description": "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay."}},"tags": [@@ -8967,7 +8970,10 @@"description": "Request body exceeds the route maxBodyBytes limit."},"429": {- "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request."+ "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name."+ },+ "503": {+ "description": "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay."}},"tags": [@@ -9025,7 +9031,10 @@"description": "Request body exceeds the route maxBodyBytes limit."},"429": {- "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request."+ "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name."+ },+ "503": {+ "description": "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay."}},"tags": [@@ -9083,7 +9092,10 @@"description": "Request body exceeds the route maxBodyBytes limit."},"429": {- "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request."+ "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name."+ },+ "503": {+ "description": "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay."}},"tags": [@@ -9141,7 +9153,10 @@"description": "Request body exceeds the route maxBodyBytes limit."},"429": {- "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request."+ "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name."+ },+ "503": {+ "description": "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay."}},"tags": [@@ -9199,7 +9214,10 @@"description": "Request body exceeds the route maxBodyBytes limit."},"429": {- "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request."+ "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name."+ },+ "503": {+ "description": "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay."}},"tags": [@@ -9257,7 +9275,10 @@"description": "Request body exceeds the route maxBodyBytes limit."},"429": {- "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request."+ "description": "Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name."+ },+ "503": {+ "description": "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay."}},"tags": [diff --git a/src/modules/integration/ingress-ack.spec.ts b/src/modules/integration/ingress-ack.spec.tsindex d8396892e..7b7b7e242 100644--- a/src/modules/integration/ingress-ack.spec.ts+++ b/src/modules/integration/ingress-ack.spec.ts@@ -1,4 +1,4 @@-import { ackContentType, renderAck } from './ingress-ack';+import { ackContentType, renderAck, safeAckHeaders } from './ingress-ack';const ctx = { rawBody: '{"a":1}', timestamp: '1700000000', id: 'd1' };@@ -57,3 +57,36 @@ describe('ackContentType', () => {expect(ackContentType({ 'content-type': '' })).toBe('text/plain');});});++describe('safeAckHeaders (a plugin-authored ack cannot undo the app response contract)', () => {+ it('drops the headers that decide how a browser treats the reflected body', () => {+ expect(+ safeAckHeaders({+ 'X-Content-Type-Options': 'nosniff-not',+ 'Content-Security-Policy': 'default-src *',+ 'Set-Cookie': 'a=b',+ 'Access-Control-Allow-Origin': '*',+ 'Content-Type': 'text/html',+ 'X-Provider-Ack': 'ok',+ }),+ ).toEqual({ 'X-Provider-Ack': 'ok' });+ });++ it('is total: no headers, and a non-string value, both answer an object', () => {+ expect(safeAckHeaders(undefined)).toEqual({});+ expect(safeAckHeaders({ 'X-N': 7 as unknown as string })).toEqual({});+ });+});++describe('renderAck / ackContentType stay total on a manifest the loader did not type-check', () => {+ const ctx = { rawBody: '{}', timestamp: '1', id: 'd1' };++ it('ignores a non-string body and a non-number status instead of throwing', () => {+ const spec = { status: '202' as unknown as number, body: 7 as unknown as string };+ expect(renderAck(spec, ctx)).toEqual({ status: 202 });+ });++ it('ignores a non-string declared content type', () => {+ expect(ackContentType({ 'content-type': 7 as unknown as string })).toBe('text/plain');+ });+});diff --git a/src/modules/integration/ingress-ack.ts b/src/modules/integration/ingress-ack.tsindex fbfa73c22..b15ba3593 100644--- a/src/modules/integration/ingress-ack.ts+++ b/src/modules/integration/ingress-ack.ts@@ -17,14 +17,50 @@ export type AckResult = { status: number; body?: string; headers?: Record<string*/export function renderAck(spec: IngressResponseContract['ack'] | undefined, ctx: AckRenderCtx): AckResult {if (!spec) return { status: 202, body: 'accepted' };- const result: AckResult = { status: spec.status ?? 202 };- if (spec.body !== undefined) {+ const result: AckResult = { status: typeof spec.status === 'number' ? spec.status : 202 };+ // Typed as a string, but a manifest is third-party JSON and the loader does not check the type of+ // every leaf, so a number or an object here would throw out of a function documented as total and+ // turn an accepted delivery into a 500.+ if (typeof spec.body === 'string') {result.body = substitute(spec.body, ctx);}- if (spec.headers) result.headers = { ...spec.headers };+ if (spec.headers && typeof spec.headers === 'object') {+ result.headers = Object.fromEntries(Object.entries(spec.headers).filter(([, v]) => typeof v === 'string'));+ }return result;}+/**+ * Header names a route's declared ack may never write.+ *+ * The ack block is plugin-authored and lands on the gateway's own origin, so it must not undo what+ * the app sets for every response: `content-type` is decided by {@link ackContentType} immediately+ * after, and the rest are the browser-facing protections that make a reflected ack body safe in the+ * first place. Anything else a manifest declares still goes out verbatim.+ */+const RESERVED_ACK_HEADERS: ReadonlySet<string> = new Set([+ 'content-type',+ 'content-length',+ 'content-security-policy',+ 'x-content-type-options',+ 'x-frame-options',+ 'strict-transport-security',+ 'referrer-policy',+ 'set-cookie',+ 'access-control-allow-origin',+ 'access-control-allow-credentials',+]);++/** The subset of a declared ack's headers that may reach the wire. Total. */+export function safeAckHeaders(headers: Record<string, string> | undefined): Record<string, string> {+ if (!headers) return {};+ return Object.fromEntries(+ Object.entries(headers).filter(+ ([name, value]) => typeof value === 'string' && !RESERVED_ACK_HEADERS.has(name.toLowerCase()),+ ),+ );+}+/*** Media types a route's declared ack Content-Type may actually put on the wire. Express types a bare* send() as text/html, which would make a reflected body (a GET challenge echo, an ack template@@ -40,7 +76,7 @@ export function ackContentType(headers: Record<string, string> | undefined): strconst declared = headers? Object.entries(headers).find(([name]) => name.toLowerCase() === 'content-type')?.[1]: undefined;- if (!declared) return 'text/plain';+ if (typeof declared !== 'string' || !declared) return 'text/plain';const mediaType = declared.split(';', 1)[0].trim().toLowerCase();return HONORED_ACK_MEDIA_TYPES.has(mediaType) ? declared : 'text/plain';}diff --git a/src/modules/integration/ingress.controller.ts b/src/modules/integration/ingress.controller.tsindex 1a048e7d2..26b7a1dd2 100644--- a/src/modules/integration/ingress.controller.ts+++ b/src/modules/integration/ingress.controller.ts@@ -3,7 +3,7 @@ import { SkipThrottle } from '@nestjs/throttler';import { ApiTags, ApiOkResponse, ApiParam, ApiResponse } from '@nestjs/swagger';import type { Request, Response } from 'express';import { Public } from '../auth/decorators/auth.decorators';-import { ackContentType } from './ingress-ack';+import { ackContentType, safeAckHeaders } from './ingress-ack';import { IngressService } from './ingress.service';import { InstanceThrottlerGuard } from './instance-throttler.guard';@@ -63,7 +63,12 @@ export class IngressController {@ApiResponse({status: 429,description:- 'Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request.',+ 'Rate limit exceeded: the per-instance bucket (INGRESS_INSTANCE_LIMIT) or the per-client-IP bucket (INGRESS_IP_LIMIT). The `Retry-After-instance` / `Retry-After-ingress-ip` header names which one shed the request, and a plain `Retry-After` carries the same delay for a client that reads only the standard name.',+ })+ @ApiResponse({+ status: 503,+ description:+ "A route whose response contract declares a `session-alive` preflight, when the bound session's engine is not connected. The delivery is not persisted, so the provider's retry is treated as a new one; `Retry-After` carries the delay.",})async receive(@Param('pluginId') pluginId: string,@@ -82,6 +87,17 @@ export class IngressController {const headers: Record<string, string> = Object.fromEntries(Object.entries(req.headers).map(([k, v]) => [k.toLowerCase(), Array.isArray(v) ? v.join(',') : String(v ?? '')]),);+ // Express answers a repeated query parameter with an array, and a nested one with an object, so+ // the Record<string, string> the service is typed against is a promise the framework does not+ // keep. The challenge path feeds these straight into a constant-time compare, which throws on+ // anything that is not a string, so `?token=a&token=b` answered 500 rather than failing the+ // challenge. Flattened here, the way the headers above already are.+ const flatQuery: Record<string, string> = Object.fromEntries(+ Object.entries(query as Record<string, unknown>).map(([k, v]) => [+ k,+ Array.isArray(v) ? String(v[0] ?? '') : typeof v === 'string' ? v : '',+ ]),+ );const rawBody = req.rawBody?.toString('utf8') ?? '';const result = await this.ingress.handle({pluginId,@@ -89,10 +105,10 @@ export class IngressController {route,method: req.method,headers,- query,+ query: flatQuery,rawBody,});- if (result.headers) res.set(result.headers);+ if (result.headers) res.set(safeAckHeaders(result.headers));// Both reflections echo provider-controlled strings (hub.challenge, the ack template). Express// types a bare send() as text/html, which turns a reflection into XSS material on this origin, so// only a non-executable declared type survives and everything else is forced to text/plain. Thisdiff --git a/src/modules/plugins/plugin-installer.ts b/src/modules/plugins/plugin-installer.tsindex 96115aa13..8517c46f9 100644--- a/src/modules/plugins/plugin-installer.ts+++ b/src/modules/plugins/plugin-installer.ts@@ -49,14 +49,17 @@ export interface ParsedPackage {* over-size archives). The caller writes the returned entries; this function decides what is safe.*/export function parsePluginPackage(buffer: Buffer, limits: PackageLimits = DEFAULT_PACKAGE_LIMITS): ParsedPackage {- let zip: AdmZip;+ // The constructor reads only the end-of-central-directory record; the central directory itself is+ // parsed lazily by getEntries(), so an archive whose trailer parses and whose directory does not+ // escaped this guard as a plain Error and reached the caller as a 500. Both answer the same+ // question, "is this a readable archive", so both are caught here.+ let files: AdmZip.IZipEntry[];try {- zip = new AdmZip(buffer);+ files = new AdmZip(buffer).getEntries().filter(e => !e.isDirectory);} catch {throw new BadRequestException('Uploaded file is not a valid .zip archive');}- const files = zip.getEntries().filter(e => !e.isDirectory);if (files.length === 0) throw new BadRequestException('The archive is empty');if (files.length > limits.maxEntries) throw new BadRequestException('The archive has too many files');