Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(infra): re-sync plugin instance bindings after import-data
Yudhi Armyndharis8df0a264c032
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 20af2b9a9..012c78ff1 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -5765,6 +5765,8 @@ Because that pre-flight runs _before_ the transaction, its teardown is not coverInside the transaction every migration table is emptied. `webhooks` and `sessions` are DELETEd directly, so a missing table there fails the restore; 13 more go through a tolerant helper where a _genuinely missing_ table is skipped; and `automation_rules` is emptied by the `DELETE FROM sessions` cascade rather than by the helper. Any other DELETE failure propagates to the rollback. Rows are then re-inserted, sessions first. JSON object/array fields are auto-stringified before insert, and the Postgres-form `$N` placeholders are rewritten for SQLite. Two guards return `imported:false` after a rollback: any `warnings`, and a payload that restores **zero** rows in total (a wrong/empty backup would otherwise commit a silent wipe — the response then carries `Backup contained no rows to restore; refused to replace existing data. Check the file.`). On commit the lid→phone mirror is reloaded from the restored rows.+On commit, plugin instance bindings are also re-applied: a session bound only by an instance the backup lacks, or restores disabled, is dropped from that plugin's active sessions and its per-session config is cleared, then every restored enabled instance is bound again. Sessions activated through `PUT /api/plugins/:id/sessions` with no instance behind them are left alone. If that re-sync fails the import still commits, with a notice and `restartRequired: true`.+**Errors:** `400` `tables` absent/not an object, a table whose value is not an array of rows, a row that is not an object (`null`, a bare string, a nested array), a flag spelled as anything but a boolean or exact `true`/`false`, or a property the route does not accept — nothing is written, and field-level detail is suppressed in production unless `VALIDATION_ERROR_DETAIL=true` · `401` · `403` · `409` refused, with the reason in `code` — `IMPORT_WOULD_ORPHAN_ENGINES` (live engines exist for sessions the backup does not contain; retry with `stopOrphans` or `force`), `IMPORT_ALREADY_RUNNING` (another import is running; wait for it), `IMPORT_NESTED_TRANSACTION` (another database transaction holds the connection; retry with nothing else in flight) · `500` unrecoverable DB error> A malformed archive is answered before the restore opens its transaction. Every table present is checked for being an array, and every row in it for being an object — not just `sessions`, since the rest are read inside the transaction where the same mistake would fail mid-restore instead of ahead of it. A hand-edited or truncated backup therefore reports `400` naming the offending `tables.<name>[<index>]`, rather than the `500` that told the operator the server had broken when their file was simply wrong.diff --git a/src/modules/infra/infra-data.controller.spec.ts b/src/modules/infra/infra-data.controller.spec.tsindex 4727f29fa..24912ca41 100644--- a/src/modules/infra/infra-data.controller.spec.ts+++ b/src/modules/infra/infra-data.controller.spec.ts@@ -45,6 +45,9 @@ import { IntegrationDeliveryFailure } from '../integration/entities/integration-import { StatusUpdate } from '../status-store/entities/status-update.entity';import { AutomationRule } from '../automation/entities/automation-rule.entity';import { AuditAction } from '../audit/entities/audit-log.entity';+import { ScopeBindingService } from '../integration/scope-binding.service';+import { PluginInstanceService } from '../integration/plugin-instance.service';+import { PluginLoaderService } from '../../core/plugins/plugin-loader.service';import { BadRequestException } from '@nestjs/common';describe('InfraDataController.importData round-trips export-data (no silent message/batch loss)', () => {@@ -2343,3 +2346,140 @@ describe('InfraDataController.importData rejects a malformed table value', () =>await expect(controller().importData({ tables } as never)).rejects.not.toThrow(/must be an array/);});});++// Plugin runtime bindings (activeSessions, per-session config) are projected from plugin_instances+// rows and kept in the plugin registry, and the boot pass only adds. A restore that drops an instance+// must retire its binding, or the plugin keeps firing on that session with the dropped config.+describe('InfraDataController.importData re-syncs plugin instance bindings', () => {+ let ds: DataSource;+ const cfg = { get: (key: string, def?: unknown) => (key === 'dataDatabase.type' ? 'sqlite' : def) };++ beforeEach(async () => {+ ds = new DataSource({+ type: 'better-sqlite3',+ database: ':memory:',+ entities: [+ Session,+ Webhook,+ Message,+ MessageBatch,+ Template,+ BaileysStoredMessage,+ LidMapping,+ ChatState,+ PluginInstance,+ ConversationMapping,+ IngressEvent,+ WebhookDeliveryFailure,+ WebhookOutboxEvent,+ IntegrationDeliveryFailure,+ StatusUpdate,+ AutomationRule,+ ],+ synchronize: true,+ });+ await ds.initialize();+ });++ afterEach(async () => {+ await ds.destroy();+ });++ const seed = async () => {+ const sessions = ds.getRepository(Session);+ for (const id of ['sess-1', 'sess-2']) {+ await sessions.save(+ sessions.create({ id, name: `session-${id}`, status: SessionStatus.DISCONNECTED, config: {} }),+ );+ }+ const instances = ds.getRepository(PluginInstance);+ for (const [instanceId, sessionScope, token] of [+ ['kept', 'sess-2', 'token-kept'],+ ['dropped', 'sess-1', 'token-dropped'],+ ]) {+ await instances.save(+ instances.create({+ id: `chatwoot:${instanceId}`,+ pluginId: 'chatwoot',+ instanceId,+ sessionScope,+ secret: 's',+ verifyToken: 'v',+ config: { token },+ enabled: true,+ }),+ );+ }+ };++ /** A loader whose runtime state the resync really mutates, as the plugin registry would hold it. */+ const statefulLoader = () => {+ const plugin = {+ manifest: { id: 'chatwoot' },+ activeSessions: ['sess-1', 'sess-2'],+ sessionConfig: { 'sess-1': { token: 'token-dropped' }, 'sess-2': { token: 'token-kept' } } as Record<+ string,+ unknown+ >,+ };+ const loader = {+ getPlugin: (id: string) => (id === 'chatwoot' ? plugin : undefined),+ setPluginSessionConfig: (_id: string, scope: string, config: unknown) => {+ plugin.sessionConfig = { ...plugin.sessionConfig, [scope]: config };+ },+ setPluginSessions: (_id: string, sessions: string[]) => {+ plugin.activeSessions = sessions;+ },+ updatePluginConfig: jest.fn(),+ };+ return { plugin, loader };+ };++ const serviceWith = (moduleRef: unknown) =>+ new InfraDataService(cfg as never, ds, undefined, undefined, undefined, undefined, undefined, moduleRef as never);++ it('retires the binding of an instance the backup does not contain', async () => {+ await seed();+ const dump = await new InfraDataController(serviceWith(undefined)).exportData();+ const tables = {+ ...dump.tables,+ pluginInstances: dump.tables.pluginInstances?.filter(row => row.instanceId !== 'dropped'),+ };+ const { plugin, loader } = statefulLoader();+ const scopeBinding = new ScopeBindingService(+ new PluginInstanceService(ds.getRepository(PluginInstance)),+ loader as unknown as PluginLoaderService,+ { logInfo: jest.fn(), logWarn: jest.fn() } as never,+ ds.getRepository(Session),+ );+ const moduleRef = { get: jest.fn(() => scopeBinding) };++ const res = await new InfraDataController(serviceWith(moduleRef)).importData({ tables });++ expect(res.imported).toBe(true);+ expect(res.restartRequired).toBe(false);+ expect(moduleRef.get).toHaveBeenCalledWith(ScopeBindingService, { strict: false });+ expect(plugin.activeSessions).toEqual(['sess-2']);+ expect(plugin.sessionConfig).toEqual({ 'sess-1': {}, 'sess-2': { token: 'token-kept' } });+ });++ it('commits the import and reports a failed resync as a notice with restartRequired', async () => {+ await seed();+ const dump = await new InfraDataController(serviceWith(undefined)).exportData();+ const resyncAfterImport = jest.fn().mockRejectedValue(new Error('registry write failed'));+ const moduleRef = { get: () => ({ resyncAfterImport }) };++ const res = await new InfraDataController(serviceWith(moduleRef)).importData({ tables: dump.tables });++ expect(res.imported).toBe(true);+ expect(res.restartRequired).toBe(true);+ expect(res.notices.some(n => n.includes('registry write failed'))).toBe(true);+ // The snapshot handed over is the pre-import state, read before the table was cleared.+ expect(resyncAfterImport).toHaveBeenCalledWith(+ expect.arrayContaining([+ expect.objectContaining({ pluginId: 'chatwoot', sessionScope: 'sess-1', enabled: true }),+ expect.objectContaining({ pluginId: 'chatwoot', sessionScope: 'sess-2', enabled: true }),+ ]),+ );+ });+});diff --git a/src/modules/infra/infra-data.service.ts b/src/modules/infra/infra-data.service.tsindex 39c9f32de..46389be25 100644--- a/src/modules/infra/infra-data.service.ts+++ b/src/modules/infra/infra-data.service.ts@@ -1,4 +1,5 @@import { BadRequestException, ConflictException, Injectable, Optional } from '@nestjs/common';+import { ModuleRef } from '@nestjs/core';import { ConfigService } from '@nestjs/config';import { DataSource, QueryRunner } from 'typeorm';import { InjectDataSource } from '@nestjs/typeorm';@@ -10,6 +11,7 @@ import { SessionService } from '../session/session.service';import { LidMappingStoreService } from '../../engine/identity/lid-mapping-store.service';import { ChatStateStoreService } from '../../engine/adapters/baileys-chat-state-store.service';import { SessionOwnershipService } from '../session/session-ownership.service';+import { ScopeBindingService } from '../integration/scope-binding.service';import { Session as SessionEntity, SessionStatus } from '../session/entities/session.entity';import { In } from 'typeorm';import { DateUtils } from 'typeorm/util/DateUtils';@@ -294,6 +296,11 @@ export class InfraDataService {private readonly ownership?: SessionOwnershipService,@Optional()private readonly chatStateStore?: ChatStateStoreService,+ // Resolves ScopeBindingService lazily (strict: false) for the post-import plugin binding resync.+ // A lookup rather than a module import: importing IntegrationModule here would move it deeper in+ // the module graph and reorder its lifecycle hooks relative to the rest of the app.+ @Optional()+ private readonly moduleRef?: ModuleRef,) {}/**@@ -635,6 +642,19 @@ export class InfraDataService {// does not reach it. Without this, a restore onto an instance that already holds chat_states rows// collides on those PKs and the all-or-nothing gate rolls the whole import back.await clearTable('chat_states');+ // The runtime plugin bindings (activeSessions, per-session config) were projected from the rows+ // about to be deleted. Remember which scopes they bound so the post-commit resync can retire+ // the ones the restore drops. Probed first: on PostgreSQL a failed SELECT would abort the+ // transaction, and a missing table is tolerated by clearTable below.+ const previousPluginBindings = (await queryRunner.hasTable('plugin_instances'))+ ? (+ (await queryRunner.query('SELECT "pluginId", "sessionScope", enabled FROM plugin_instances')) as Array<{+ pluginId: string;+ sessionScope: string | null;+ enabled: boolean | number;+ }>+ ).map(row => ({ ...row, enabled: Number(row.enabled) === 1 }))+ : [];// Integration Fabric + both DLQs: none carry an FK constraint to sessions (sessionId is provenance),// so clearing them here before the sessions DELETE keeps the replace-semantics complete.await clearTable('plugin_instances');@@ -809,6 +829,23 @@ export class InfraDataService {await this.lidMappingStore?.reload();await this.chatStateStore?.reload();+ // Plugin runtime bindings are projected from plugin_instances rows and saved to the plugin+ // registry, and the boot pass only ever adds to them. Without this resync, an instance the+ // restore dropped keeps receiving its session's hooks with its old config, even after a+ // restart. Best-effort: the import is already committed.+ if (this.moduleRef) {+ try {+ await this.moduleRef.get(ScopeBindingService, { strict: false }).resyncAfterImport(previousPluginBindings);+ } catch (error) {+ restartRequired = true;+ notices.push(+ `Plugin instance bindings could not be re-applied after the restore ` ++ `(${error instanceof Error ? error.message : String(error)}): check each plugin's active sessions ` ++ `and per-session config (GET /api/plugins/:id, PUT /api/plugins/:id/sessions) against its restored instances.`,+ );+ }+ }+// Audit the destructive replace-all restore, only on the committed-success path (the rollback /// refused-empty branches above return without emitting, since no data actually changed). Any// warnings would have taken the rollback branch, so warnings.length is always 0 here — recorddiff --git a/src/modules/integration/scope-binding.service.spec.ts b/src/modules/integration/scope-binding.service.spec.tsindex 4def6bfce..0a1b61d7e 100644--- a/src/modules/integration/scope-binding.service.spec.ts+++ b/src/modules/integration/scope-binding.service.spec.ts@@ -351,3 +351,119 @@ describe('ScopeBindingService.applyScopeBinding retires an instance without leakexpect(setPluginSessionConfig).toHaveBeenCalledWith('chatwoot', 'sess-1', { baseUrl: 'https://a.example' });});});++// A restore replaces every plugin_instances row, while the runtime bindings projected from the old+// rows live in the plugin registry and the boot pass only adds. resyncAfterImport retires what the+// restore dropped and re-applies what it restored.+describe('ScopeBindingService.resyncAfterImport', () => {+ /** A loader whose activeSessions / sessionConfig actually change, so the end state can be read. */+ function statefulLoader(initial: { activeSessions: string[]; sessionConfig: Record<string, unknown> }) {+ const plugin = { manifest: { id: 'chatwoot' }, ...initial };+ const loader = {+ getPlugin: jest.fn((id: string) => (id === 'chatwoot' ? plugin : undefined)),+ setPluginSessionConfig: jest.fn((_id: string, scope: string, config: Record<string, unknown>) => {+ plugin.sessionConfig = { ...plugin.sessionConfig, [scope]: config };+ }),+ setPluginSessions: jest.fn((_id: string, sessions: string[]) => {+ plugin.activeSessions = sessions;+ }),+ updatePluginConfig: jest.fn(),+ };+ return { plugin, loader };+ }++ function svc(loader: unknown, restored: unknown[]): ScopeBindingService {+ const rowsFor = (pluginId: string) =>+ (restored as Array<{ pluginId: string }>).filter(r => r.pluginId === pluginId);+ return new ScopeBindingService(+ {+ listAll: jest.fn().mockResolvedValue(restored),+ list: jest.fn((pluginId: string) => Promise.resolve(rowsFor(pluginId))),+ } as unknown as PluginInstanceService,+ loader as PluginLoaderService,+ { logInfo: jest.fn(), logWarn: jest.fn() } as unknown as AuditService,+ { findOne: jest.fn().mockResolvedValue({ id: 'x' }) } as unknown as Repository<Session>,+ );+ }++ const row = (+ instanceId: string,+ sessionScope: string | null,+ config: Record<string, unknown> = {},+ enabled = true,+ ) => ({+ pluginId: 'chatwoot',+ instanceId,+ sessionScope,+ config,+ enabled,+ });++ it('retires the scope of an instance the restore dropped, and keeps the restored one', async () => {+ const { plugin, loader } = statefulLoader({+ activeSessions: ['sess-1', 'sess-2'],+ sessionConfig: { 'sess-1': { token: 'dropped' }, 'sess-2': { token: 'old-kept' } },+ });++ await svc(loader, [row('kept', 'sess-2', { token: 'restored' })]).resyncAfterImport([+ { pluginId: 'chatwoot', sessionScope: 'sess-1', enabled: true },+ { pluginId: 'chatwoot', sessionScope: 'sess-2', enabled: true },+ ]);++ expect(plugin.activeSessions).toEqual(['sess-2']);+ expect(plugin.sessionConfig).toEqual({ 'sess-1': {}, 'sess-2': { token: 'restored' } });+ });++ it('retires a scope whose instance was restored DISABLED', async () => {+ const { plugin, loader } = statefulLoader({ activeSessions: ['sess-1'], sessionConfig: { 'sess-1': { t: 1 } } });++ await svc(loader, [row('a', 'sess-1', { t: 1 }, false)]).resyncAfterImport([+ { pluginId: 'chatwoot', sessionScope: 'sess-1', enabled: true },+ ]);++ expect(plugin.activeSessions).toEqual([]);+ expect(plugin.sessionConfig).toEqual({ 'sess-1': {} });+ });++ it('leaves an operator-activated session alone: the teardown set comes from the old rows only', async () => {+ // 'sess-op' was activated through PUT /plugins/:id/sessions with no instance behind it, and '*'+ // too; neither was bound by a pre-import row, so neither may be retired.+ const { plugin, loader } = statefulLoader({ activeSessions: ['*', 'sess-op', 'sess-1'], sessionConfig: {} });++ await svc(loader, []).resyncAfterImport([{ pluginId: 'chatwoot', sessionScope: 'sess-1', enabled: true }]);++ expect(new Set(plugin.activeSessions)).toEqual(new Set(['*', 'sess-op']));+ });++ it('drops "*" when the restore removed the only wildcard instance', async () => {+ const { plugin, loader } = statefulLoader({ activeSessions: ['*'], sessionConfig: {} });++ await svc(loader, []).resyncAfterImport([{ pluginId: 'chatwoot', sessionScope: null, enabled: true }]);++ expect(plugin.activeSessions).toEqual([]);+ });++ it('touches nothing for a pair that was disabled before the import, or a plugin that is not loaded', async () => {+ const { loader } = statefulLoader({ activeSessions: ['sess-1'], sessionConfig: {} });++ await svc(loader, []).resyncAfterImport([+ { pluginId: 'chatwoot', sessionScope: 'sess-1', enabled: false },+ { pluginId: 'not-loaded', sessionScope: 'sess-9', enabled: true },+ ]);++ expect(loader.setPluginSessionConfig).not.toHaveBeenCalled();+ expect(loader.setPluginSessions).not.toHaveBeenCalled();+ });++ it('rejects when the restored rows cannot be listed, so the caller can report it', async () => {+ const { loader } = statefulLoader({ activeSessions: [], sessionConfig: {} });+ const service = new ScopeBindingService(+ { listAll: jest.fn().mockRejectedValue(new Error('db down')) } as unknown as PluginInstanceService,+ loader as unknown as PluginLoaderService,+ { logInfo: jest.fn(), logWarn: jest.fn() } as unknown as AuditService,+ {} as Repository<Session>,+ );++ await expect(service.resyncAfterImport([])).rejects.toThrow('db down');+ });+});diff --git a/src/modules/integration/scope-binding.service.ts b/src/modules/integration/scope-binding.service.tsindex 47b42dac2..e2abac33f 100644--- a/src/modules/integration/scope-binding.service.ts+++ b/src/modules/integration/scope-binding.service.ts@@ -41,13 +41,19 @@ export class ScopeBindingService implements OnApplicationBootstrap {* cannot abort the rest.*/async onApplicationBootstrap(): Promise<void> {- let rows: PluginInstance[];try {- rows = await this.instances.listAll();+ await this.reconcile();} catch (err) {this.logger.error('Scope-binding reconciliation skipped (failed to list instances)', String(err));- return;}+ }++ /**+ * The boot pass itself, shared with {@link resyncAfterImport}. Rejects only when the instance rows+ * cannot be listed; a single binding failure is absorbed by applyScopeBinding.+ */+ async reconcile(): Promise<void> {+ const rows = await this.instances.listAll();// Order-independent reconciliation: listAll() is repo.find() with no ORDER BY, so its row order// is DB/restart-dependent (differs between SQLite and Postgres). applyScopeBinding mutates the@@ -82,6 +88,41 @@ export class ScopeBindingService implements OnApplicationBootstrap {}}+ /**+ * Bring the runtime bindings back in line after a restore replaced every `plugin_instances` row.+ *+ * The boot pass only adds, so on its own it can never retire a binding whose instance the restore+ * dropped (or restored disabled): the plugin would keep firing on that session with the retired+ * instance's config, across restarts. So first tear down every scope that was bound by an enabled+ * row BEFORE the import and is bound by no enabled row now, then re-run the boot pass over the+ * restored rows. The teardown set comes only from the pre-import rows, never from activeSessions,+ * so a session an operator activated through PUT /api/plugins/:id/sessions is left alone. The+ * teardown is additive for the same reason: a restore makes no new narrowing decision, so it drops+ * the retired scope without also retiring an operator's '*'. A retired wildcard still loses its '*'+ * unless a restored wildcard row keeps it. applyScopeBinding runs after the commit, so its sibling+ * checks read the restored rows.+ */+ async resyncAfterImport(+ previous: ReadonlyArray<{ pluginId: string; sessionScope: string | null; enabled: boolean }>,+ ): Promise<void> {+ const scopeKey = (scope: string | null): string => (!scope || scope === '*' ? '*' : scope);+ const bound = new Set(+ (await this.instances.listAll())+ .filter(row => row.enabled)+ .map(row => `${row.pluginId}\u0000${scopeKey(row.sessionScope)}`),+ );+ const retired = new Map<string, { pluginId: string; sessionScope: string | null }>();+ for (const row of previous) {+ const key = `${row.pluginId}\u0000${scopeKey(row.sessionScope)}`;+ if (row.enabled && !bound.has(key)) retired.set(key, row);+ }+ for (const { pluginId, sessionScope } of retired.values()) {+ if (!this.loader.getPlugin(pluginId)) continue;+ await this.applyScopeBinding(pluginId, sessionScope, {}, false, { additive: true });+ }+ await this.reconcile();+ }+/*** Warn when a just-reconciled instance binds a concrete scope no session row matches — a session the* operator deleted, or a scope typed/copied wrong. The plugin is then activated for nothing: thediff --git a/test/integration-instance.e2e-spec.ts b/test/integration-instance.e2e-spec.tsindex 75b615cb1..34faa19ee 100644--- a/test/integration-instance.e2e-spec.ts+++ b/test/integration-instance.e2e-spec.ts@@ -7,6 +7,7 @@ process.env.ALLOW_DEV_API_KEY = 'true';process.env.BASE_URL = 'https://api.example.com';import { INestApplication } from '@nestjs/common';+import { ModuleRef } from '@nestjs/core';import { Test } from '@nestjs/testing';import { getRepositoryToken } from '@nestjs/typeorm';import { Repository } from 'typeorm';@@ -18,6 +19,8 @@ import { PluginLoaderService } from '../src/core/plugins/plugin-loader.service';import { AuthService } from '../src/modules/auth/auth.service';import { ApiKeyRole } from '../src/modules/auth/entities/api-key.entity';import { IntegrationDeliveryFailure } from '../src/modules/integration/entities/integration-delivery-failure.entity';+import { InfraDataService } from '../src/modules/infra/infra-data.service';+import { ScopeBindingService } from '../src/modules/integration/scope-binding.service';// A stub ingress-capable plugin so the capability check passes without a real plugin on disk. The// configSchema carries a NESTED secret field so the masking tests can prove a `secret:true` config@@ -92,6 +95,13 @@ describe('IntegrationInstanceController (e2e)', () => {}});+ // InfraDataService re-syncs plugin bindings after an import through a non-strict lookup of+ // ScopeBindingService, which lives in another module. Proven against the real module graph.+ it('lets InfraDataService resolve the app ScopeBindingService for the post-import resync', () => {+ const infraModuleRef = Reflect.get(app.get(InfraDataService), 'moduleRef') as ModuleRef | undefined;+ expect(infraModuleRef?.get(ScopeBindingService, { strict: false })).toBe(app.get(ScopeBindingService));+ });+it('requires an API key (401 without one)', async () => {await request(app.getHttpServer()).get(`${base}/chatwoot/instances`).expect(401);});