Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
feat(message): add opt-in message retention
Yudhi Armyndharis5b8ec6b1161e
diff --git a/.env.example b/.env.exampleindex 13bc25e4e..3709185fb 100644--- a/.env.example+++ b/.env.example@@ -301,6 +301,14 @@ OPENWA_PIDS_LIMIT=2048# Default 90; set to 0 to keep audit logs forever (disable pruning).# AUDIT_RETENTION_DAYS=90+# Message retention. Stored messages older than this many days, and finished bulk batches, are+# deleted daily (and once at startup). Default 0 keeps them forever; the maximum is 36500. Pruning+# also shrinks stats history, and send pacing then treats a contact silent for longer than the+# window as new (cold).+# Search plugins keep their copies of pruned messages; archived media files are reclaimed by the+# chat-media orphan sweep. History older than the window is not written back by a history sync.+# MESSAGE_RETENTION_DAYS=0+# =============================================================================# DATABASE# =============================================================================diff --git a/docker-compose.dev.yml b/docker-compose.dev.ymlindex 9170c9032..9649b42d2 100644--- a/docker-compose.dev.yml+++ b/docker-compose.dev.yml@@ -279,6 +279,7 @@ services:- UPDATE_CHECK_ENABLED=${UPDATE_CHECK_ENABLED:-}# Misc application tuning.- AUDIT_RETENTION_DAYS=${AUDIT_RETENTION_DAYS:-}+ - MESSAGE_RETENTION_DAYS=${MESSAGE_RETENTION_DAYS:-}- BULK_MAX_CONCURRENT_BATCHES=${BULK_MAX_CONCURRENT_BATCHES:-}- TEMPLATE_RENDER_MAX_CHARS=${TEMPLATE_RENDER_MAX_CHARS:-}- STATS_CACHE_TTL_MS=${STATS_CACHE_TTL_MS:-}diff --git a/docker-compose.yml b/docker-compose.ymlindex 9b5cc22b0..adfb4fc15 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -381,6 +381,7 @@ services:- UPDATE_CHECK_ENABLED=${UPDATE_CHECK_ENABLED:-}# Misc application tuning.- AUDIT_RETENTION_DAYS=${AUDIT_RETENTION_DAYS:-}+ - MESSAGE_RETENTION_DAYS=${MESSAGE_RETENTION_DAYS:-}- BULK_MAX_CONCURRENT_BATCHES=${BULK_MAX_CONCURRENT_BATCHES:-}- TEMPLATE_RENDER_MAX_CHARS=${TEMPLATE_RENDER_MAX_CHARS:-}- STATS_CACHE_TTL_MS=${STATS_CACHE_TTL_MS:-}diff --git a/docs/05-database-design.md b/docs/05-database-design.mdindex 9f6848dbd..30878ae83 100644--- a/docs/05-database-design.md+++ b/docs/05-database-design.md@@ -911,12 +911,12 @@ export class AddMessagesWaMessageIdUnique1781300000000 implements MigrationInter### Retention Policies-Six tables have an automated _time-based_ retention job, across five services: **`audit_logs`**, **`status_updates`**, **`webhook_delivery_failures`**, **`webhook_outbox_events`** (settled rows only), **`ingress_events`** and **`integration_delivery_failures`**. Separately, **`baileys_stored_messages`** is capped per session rather than by age — each write keeps the newest `BAILEYS_MESSAGE_STORE_LIMIT` rows (default 5000) for that session and deletes the rest. Everything else is kept indefinitely (api keys, sessions, webhooks, batches, templates, conversation mappings, plugin instances, lid mappings, chat states, automation rules) and is removed only by user action (e.g. deleting a session) or operational backup/restore — the `messages` history table in particular has no auto-purge and grows without bound.+Six tables have an automated _time-based_ retention job, across five services: **`audit_logs`**, **`status_updates`**, **`webhook_delivery_failures`**, **`webhook_outbox_events`** (settled rows only), **`ingress_events`** and **`integration_delivery_failures`**. Separately, **`baileys_stored_messages`** is capped per session rather than by age: each write keeps the newest `BAILEYS_MESSAGE_STORE_LIMIT` rows (default 5000) for that session and deletes the rest. Everything else is kept indefinitely (api keys, sessions, webhooks, pending and processing batches, templates, conversation mappings, plugin instances, lid mappings, chat states, automation rules) and is removed only by user action (e.g. deleting a session) or operational backup/restore. Two more, `messages` and finished `message_batches`, have an opt-in time-based job (`MESSAGE_RETENTION_DAYS`, off by default); until it is set they are kept indefinitely too.| Data Type | Default Retention | Configurable || ----------------------------- | ----------------- | --------------------------------------------------------------- || Sessions / Webhooks | Indefinite | No |-| Messages / Batches | Indefinite | No (delete a session to drop its data) |+| Messages / finished batches | Indefinite | Yes, `MESSAGE_RETENTION_DAYS` (≤ 0 keeps them, max 36500) || Status updates | 24 hours | No (fixed, matches WhatsApp's own story expiry) || Audit logs | 90 days | Yes — `AUDIT_RETENTION_DAYS` (≤ 0 disables) || Webhook delivery failures | 90 days | Yes — `WEBHOOK_FAILURE_RETENTION_DAYS` (≤ 0 disables) |@@ -951,11 +951,12 @@ async cleanup(olderThanDays = 30): Promise<number> {### Sibling Prune Jobs-The other four interval-based prunes are the same shape — one prune at startup, then a 24-hour `setInterval`, `unref`'d, never a `@Cron`:+The other interval-based prunes are the same shape: one prune at startup, then a 24-hour `setInterval`, `unref`'d, never a `@Cron`:- **`webhook_delivery_failures`** — `WebhookService.onModuleInit()` (`src/modules/webhook/webhook.service.ts`), window `WEBHOOK_FAILURE_RETENTION_DAYS` (default 90; ≤ 0 disables the prune and logs that it is off).- **`webhook_outbox_events`**: `WebhookOutboxService.onModuleInit()` (`src/modules/webhook/webhook-outbox.service.ts`), window `WEBHOOK_OUTBOX_RETENTION_DAYS` (default 7). Only settled rows are deleted; a `pending` row is a delivery that can still be replayed and is never pruned on age. A non-positive value does **not** disable the prune: a settled row carries no payload, so the service warns and falls back to 7 days.- **`ingress_events`** and **`integration_delivery_failures`** — `IntegrationRetentionService` (`src/modules/integration/integration-retention.service.ts`) prunes both in one timer on two independent windows. `INGRESS_DEDUP_RETENTION_DAYS` (default 7) bounds the dedup rows; a non-positive value does **not** disable it — an unpruned dedup table grows without bound for no functional gain, so the service warns and falls back to the 7-day default. `INGRESS_RETENTION_DAYS` (default 90) bounds the DLQ rows, where long retention can be a deliberate operator choice, so ≤ 0 disables that prune (and only that prune).+- **`messages`** and finished **`message_batches`**: `MessageRetentionService` (`src/modules/message/message-retention.service.ts`), window `MESSAGE_RETENTION_DAYS` (default 0, which keeps everything; boot rejects a value above 36500). Messages are deleted by `createdAt` in batches of 500 ids, at most 200 batches per run, so a run deletes at most 100,000 messages; a run that hits that limit logs a warning and is followed by another a minute later until the backlog is cleared. Pending and processing batches are never pruned. Once enabled, send pacing treats a contact silent for longer than the window as a new (cold) contact again, stats totals and search results shrink, archived media files are reclaimed by the chat-media orphan sweep, and a history sync does not write back messages older than the window. Search plugins get no `message:deleted` for pruned rows, so an external index keeps its copies.### Status-Update TTL Sweepdiff --git a/docs/27-plugin-search-providers.md b/docs/27-plugin-search-providers.mdindex 9529d03c0..b3d78a22f 100644--- a/docs/27-plugin-search-providers.md+++ b/docs/27-plugin-search-providers.md@@ -118,6 +118,7 @@ ctx.registerHook('message:deleted', async hookCtx => {Clearing a chat's messages (`DELETE /api/sessions/:sessionId/chats/:chatId/messages`) or deleting a chat(`POST /api/sessions/:sessionId/chats/delete`) also emits `message:deleted` for every stored row it removes, sothe same handler keeps your index in step.+Rows removed in bulk emit no `message:deleted`: deleting a session and message retention (`MESSAGE_RETENTION_DAYS`) leave the plugin's copies in its index until the plugin removes them itself.**Backfill is the plugin's responsibility.** The hook fires only for live traffic. A plugin installed ona deployment with existing message history must perform its own one-time backfill (read the `messages`diff --git a/src/config/env-precedence.ts b/src/config/env-precedence.tsindex abbced8b6..5c3b62f9a 100644--- a/src/config/env-precedence.ts+++ b/src/config/env-precedence.ts@@ -234,6 +234,7 @@ export const BLANK_SHADOWED_ENV_KEYS: string[] = ['PLUGIN_STORAGE_MAX_BYTES','UPDATE_CHECK_ENABLED','AUDIT_RETENTION_DAYS',+ 'MESSAGE_RETENTION_DAYS','BULK_MAX_CONCURRENT_BATCHES','TEMPLATE_RENDER_MAX_CHARS','STATS_CACHE_TTL_MS',diff --git a/src/config/env.validation.spec.ts b/src/config/env.validation.spec.tsindex fd74528dc..abea7c159 100644--- a/src/config/env.validation.spec.ts+++ b/src/config/env.validation.spec.ts@@ -208,6 +208,15 @@ describe('validateEnv', () => {expect(() => validateEnv({ AUDIT_RETENTION_DAYS: '90' })).not.toThrow();});+ it('rejects a non-integer message retention and accepts 0 and negatives as "keep forever"', () => {+ expect(() => validateEnv({ MESSAGE_RETENTION_DAYS: '30d' })).toThrow(/MESSAGE_RETENTION_DAYS/);+ expect(() => validateEnv({ MESSAGE_RETENTION_DAYS: '0' })).not.toThrow();+ expect(() => validateEnv({ MESSAGE_RETENTION_DAYS: '-1' })).not.toThrow();+ expect(() => validateEnv({ MESSAGE_RETENTION_DAYS: '30' })).not.toThrow();+ expect(() => validateEnv({ MESSAGE_RETENTION_DAYS: '36500' })).not.toThrow();+ expect(() => validateEnv({ MESSAGE_RETENTION_DAYS: '36501' })).toThrow(/MESSAGE_RETENTION_DAYS.*36500/);+ });+it('rejects a non-positive / non-integer WEBHOOK_MAX_PAYLOAD_BYTES (0 would reject every dispatch)', () => {expect(() => validateEnv({ WEBHOOK_MAX_PAYLOAD_BYTES: '0' })).toThrow(/WEBHOOK_MAX_PAYLOAD_BYTES/);expect(() => validateEnv({ WEBHOOK_MAX_PAYLOAD_BYTES: 'abc' })).toThrow(/WEBHOOK_MAX_PAYLOAD_BYTES/);diff --git a/src/config/env.validation.ts b/src/config/env.validation.tsindex 28003aa81..e140f0432 100644--- a/src/config/env.validation.ts+++ b/src/config/env.validation.ts@@ -246,9 +246,16 @@ export function validateEnv(config: EnvConfig): EnvConfig {// into a bare checkInt('AUDIT_RETENTION_DAYS') call would silently drop the knob out of that gate.for (const key of ['AUDIT_RETENTION_DAYS', // <= 0 disables retention+ 'MESSAGE_RETENTION_DAYS', // unset or <= 0 keeps messages forever]) {checkInt(key);}+ // A window past about a century gives a cutoff SQLite compares as a later date, deleting every+ // message. Keep in step with MAX_MESSAGE_RETENTION_DAYS in message-retention.service.ts.+ const messageRetentionDays = str('MESSAGE_RETENTION_DAYS');+ if (messageRetentionDays !== undefined && Number(messageRetentionDays) > 36500) {+ errors.push(`MESSAGE_RETENTION_DAYS must be at most 36500 (got "${messageRetentionDays}")`);+ }// BAILEYS_WA_VERSION: optional version pin for the Baileys engine (e.g. 2.3000.1045340097 or 2,3000,1045340097)for (const key of ['BAILEYS_WA_VERSION']) {diff --git a/src/modules/message/message-retention.service.spec.ts b/src/modules/message/message-retention.service.spec.tsnew file mode 100644index 000000000..c9aa25ab8--- /dev/null+++ b/src/modules/message/message-retention.service.spec.ts@@ -0,0 +1,239 @@+import 'reflect-metadata';+import { DataSource, Repository } from 'typeorm';+import { Message, MessageDirection } from './entities/message.entity';+import { BatchStatus, MessageBatch } from './entities/message-batch.entity';+import {+ MessageRetentionService,+ resolveMessageRetentionCutoff,+ resolveMessageRetentionDays,+} from './message-retention.service';+import { AddMessagesFts1782400000000 } from '../../database/migrations/1782400000000-AddMessagesFts';++describe('resolveMessageRetentionDays', () => {+ it.each([+ [undefined, 0],+ ['', 0],+ ['0', 0],+ ['-5', 0],+ ['30d', 0],+ ['abc', 0],+ ['30', 30],+ ['36500', 36500],+ ['36501', 0],+ ['9999999', 0],+ ])('reads %p as %p', (raw, expected) => {+ expect(resolveMessageRetentionDays({ MESSAGE_RETENTION_DAYS: raw })).toBe(expected);+ });++ it('has no cutoff while retention is off', () => {+ expect(resolveMessageRetentionCutoff(new Date(), {})).toBeUndefined();+ expect(resolveMessageRetentionCutoff(new Date('2026-03-31T00:00:00Z'), { MESSAGE_RETENTION_DAYS: '30' })).toEqual(+ new Date('2026-03-01T00:00:00Z'),+ );+ });+});++describe('MessageRetentionService against a real database', () => {+ const NOW = new Date('2026-06-30T12:00:00.000Z');+ const DAY_MS = 86_400_000;++ let ds: DataSource;+ let messages: Repository<Message>;+ let batches: Repository<MessageBatch>;+ let service: MessageRetentionService;+ let prev: string | undefined;++ const daysAgo = (days: number): Date => new Date(NOW.getTime() - days * DAY_MS);++ const addMessages = async (count: number, at: Date, body = 'hello'): Promise<void> => {+ const rows = Array.from({ length: count }, () => ({+ sessionId: 's1',+ chatId: '[email protected]',+ from: '[email protected]',+ to: '[email protected]',+ body,+ direction: MessageDirection.INCOMING,+ createdAt: at,+ }));+ for (let i = 0; i < rows.length; i += 200) {+ await messages.insert(rows.slice(i, i + 200));+ }+ };++ const addBatch = (batchId: string, status: BatchStatus, at: Date): Promise<unknown> =>+ batches.insert({ batchId, sessionId: 's1', status, messages: [], createdAt: at });++ beforeEach(async () => {+ prev = process.env.MESSAGE_RETENTION_DAYS;+ ds = new DataSource({+ type: 'better-sqlite3',+ database: ':memory:',+ entities: [Message, MessageBatch],+ synchronize: true,+ });+ await ds.initialize();+ messages = ds.getRepository(Message);+ batches = ds.getRepository(MessageBatch);+ service = new MessageRetentionService(messages, batches);+ });++ afterEach(async () => {+ service.onModuleDestroy();+ jest.restoreAllMocks();+ if (prev === undefined) delete process.env.MESSAGE_RETENTION_DAYS;+ else process.env.MESSAGE_RETENTION_DAYS = prev;+ await ds.destroy();+ });++ it('deletes nothing and starts no timer while retention is off', async () => {+ delete process.env.MESSAGE_RETENTION_DAYS;+ await addMessages(3, daysAgo(400));+ const interval = jest.spyOn(global, 'setInterval');++ service.onModuleInit();++ expect(interval).not.toHaveBeenCalled();+ expect(await service.prune(NOW)).toEqual({ messages: 0, batches: 0 });+ expect(await messages.count()).toBe(3);+ });++ /**+ * A cutoff tens of thousands of years back is bound on SQLite as a truncated 4-digit year that+ * sorts after today, so an operator asking to keep everything had every row deleted instead.+ */+ it('deletes nothing for a window beyond the supported maximum', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '9999999';+ await addMessages(2, NOW);+ await addBatch('done-now', BatchStatus.COMPLETED, NOW);++ expect(await service.prune(NOW)).toEqual({ messages: 0, batches: 0 });+ expect(await messages.count()).toBe(2);+ expect(await batches.count()).toBe(1);+ });++ it('deletes messages older than the window and keeps newer ones', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ await addMessages(4, daysAgo(31));+ await addMessages(2, daysAgo(29));++ expect(await service.prune(NOW)).toEqual({ messages: 4, batches: 0 });++ expect(await messages.count()).toBe(2);+ });++ it('drains a backlog larger than one delete statement in a single run', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ await addMessages(1203, daysAgo(60));+ const del = jest.spyOn(messages, 'delete');++ expect((await service.prune(NOW)).messages).toBe(1203);++ expect(await messages.count()).toBe(0);+ expect(del).toHaveBeenCalledTimes(3); // 500 + 500 + 203+ });++ it('prunes finished batches past the window but never a pending or processing one', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ await addBatch('done-old', BatchStatus.COMPLETED, daysAgo(40));+ await addBatch('cancelled-old', BatchStatus.CANCELLED, daysAgo(40));+ await addBatch('failed-old', BatchStatus.FAILED, daysAgo(40));+ await addBatch('done-new', BatchStatus.COMPLETED, daysAgo(10));+ await addBatch('pending-old', BatchStatus.PENDING, daysAgo(40));+ await addBatch('processing-old', BatchStatus.PROCESSING, daysAgo(40));++ expect((await service.prune(NOW)).batches).toBe(3);++ const left = (await batches.find()).map(b => b.batchId).sort();+ expect(left).toEqual(['done-new', 'pending-old', 'processing-old']);+ });++ it('removes pruned bodies from the built-in full-text index', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ await new AddMessagesFts1782400000000().up(ds.createQueryRunner());+ await addMessages(1, daysAgo(40), 'ancient walrus');+ await addMessages(1, daysAgo(1), 'recent walrus');++ await service.prune(NOW);++ const hits = await ds.query<{ body: string }[]>(+ `SELECT m."body" FROM messages_fts f JOIN messages m ON m.rowid = f.rowid WHERE messages_fts MATCH 'walrus'`,+ );+ expect(hits.map(h => h.body)).toEqual(['recent walrus']);+ });++ it('runs at startup, then daily on an unref-ed timer that destroy clears', () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ const prune = jest.spyOn(service, 'prune').mockResolvedValue({ messages: 0, batches: 0 });+ const unref = jest.fn();+ const interval = jest+ .spyOn(global, 'setInterval')+ .mockReturnValue({ unref } as unknown as ReturnType<typeof setInterval>);+ const clear = jest.spyOn(global, 'clearInterval');++ service.onModuleInit();++ expect(prune).toHaveBeenCalledTimes(1);+ expect(interval).toHaveBeenCalledWith(expect.any(Function), DAY_MS);+ expect(unref).toHaveBeenCalled();+ service.onModuleDestroy();+ expect(clear).toHaveBeenCalled();+ });++ /**+ * One run deletes at most 100,000 rows. A deployment that takes in more than that a day, or a first+ * enable against a large backlog, never caught up while the next run waited a whole day.+ */+ it('runs again a minute later while a run keeps hitting its cap, then waits a day', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ jest.useFakeTimers();+ try {+ const prune = jest+ .spyOn(service, 'prune')+ .mockResolvedValueOnce({ messages: 100_000, batches: 0 })+ .mockResolvedValueOnce({ messages: 100_000, batches: 0 })+ .mockResolvedValue({ messages: 42, batches: 0 });+ const warn = jest.spyOn(service['logger'], 'warn');++ service.onModuleInit();+ await jest.advanceTimersByTimeAsync(60_000);+ expect(prune).toHaveBeenCalledTimes(2);+ await jest.advanceTimersByTimeAsync(60_000);+ expect(prune).toHaveBeenCalledTimes(3);+ expect(warn).toHaveBeenCalledTimes(2);++ await jest.advanceTimersByTimeAsync(DAY_MS - 120_001);+ expect(prune).toHaveBeenCalledTimes(3);+ await jest.advanceTimersByTimeAsync(1);+ expect(prune).toHaveBeenCalledTimes(4);+ } finally {+ jest.useRealTimers();+ }+ });++ it('drops a pending follow-up run on destroy', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ jest.useFakeTimers();+ try {+ const prune = jest.spyOn(service, 'prune').mockResolvedValue({ messages: 100_000, batches: 0 });++ service.onModuleInit();+ await jest.advanceTimersByTimeAsync(0);+ service.onModuleDestroy();+ await jest.advanceTimersByTimeAsync(60_000);++ expect(prune).toHaveBeenCalledTimes(1);+ } finally {+ jest.useRealTimers();+ }+ });++ it('does not start a second pass while one is running', async () => {+ process.env.MESSAGE_RETENTION_DAYS = '30';+ await addMessages(2, daysAgo(40));++ const [first, second] = await Promise.all([service.prune(NOW), service.prune(NOW)]);++ expect(first.messages + second.messages).toBe(2);+ expect(Math.min(first.messages, second.messages)).toBe(0);+ });+});diff --git a/src/modules/message/message-retention.service.ts b/src/modules/message/message-retention.service.tsnew file mode 100644index 000000000..4ced2f684--- /dev/null+++ b/src/modules/message/message-retention.service.ts@@ -0,0 +1,145 @@+import { Injectable, OnModuleDestroy, OnModuleInit } from '@nestjs/common';+import { InjectRepository } from '@nestjs/typeorm';+import { In, LessThan, Repository } from 'typeorm';+import { Message } from './entities/message.entity';+import { BatchStatus, MessageBatch } from './entities/message-batch.entity';+import { createLogger } from '../../common/services/logger.service';+import { resolveNonNegativeIntEnv } from '../../config/configuration';++const DAY_MS = 24 * 60 * 60 * 1000;++/**+ * Rows deleted per statement. Small enough to stay far below the bind-parameter limit of either+ * dialect and to keep each write short: SQLite has one writer, so a long delete (each row also fires+ * the full-text index trigger) would stall message ingest behind it.+ */+const PRUNE_BATCH_SIZE = 500;++/** Statements per run, so one run deletes at most MAX_BATCHES_PER_RUN * PRUNE_BATCH_SIZE rows. */+const MAX_BATCHES_PER_RUN = 200;++/**+ * Delay before the next run when one stopped at its cap. Without it a deployment that stores more+ * than one run's worth of messages a day, or a first enable against a large backlog, never catches up.+ */+const CAPPED_RUN_FOLLOW_UP_MS = 60_000;++/** Batches in these states are finished; pending and processing ones are never pruned. */+const TERMINAL_BATCH_STATUSES = [BatchStatus.COMPLETED, BatchStatus.CANCELLED, BatchStatus.FAILED];++/**+ * The longest window, about a century. A cutoff far enough back is bound on SQLite as a truncated+ * 4-digit year that sorts after today and matches every row, so a larger value must not reach a query.+ * env.validation.ts rejects it at boot.+ */+export const MAX_MESSAGE_RETENTION_DAYS = 36500;++/**+ * MESSAGE_RETENTION_DAYS: stored messages (and finished bulk batches) older than this many days are+ * deleted. 0, unset, negative, not an integer or above MAX_MESSAGE_RETENTION_DAYS keeps everything,+ * which is the default.+ */+export function resolveMessageRetentionDays(env: NodeJS.ProcessEnv = process.env): number {+ const days = resolveNonNegativeIntEnv(env.MESSAGE_RETENTION_DAYS, 0);+ return days > MAX_MESSAGE_RETENTION_DAYS ? 0 : days;+}++/** The instant before which retention deletes messages, or undefined while retention is off. */+export function resolveMessageRetentionCutoff(+ now: Date = new Date(),+ env: NodeJS.ProcessEnv = process.env,+): Date | undefined {+ const days = resolveMessageRetentionDays(env);+ return days > 0 ? new Date(now.getTime() - days * DAY_MS) : undefined;+}++/**+ * Opt-in age limit for the `messages` table and finished `message_batches` rows. Runs once at startup+ * and then daily on an unref'd timer, like the audit-log prune.+ *+ * Deleting a row needs no other cleanup: the SQLite full-text index follows through its delete+ * trigger, the PostgreSQL one is part of the row, and the chat-media orphan sweep removes archived+ * files whose row is gone. Search plugins are NOT told: no `message:deleted` fires per pruned row,+ * the same as on session delete, so an external index keeps its copies.+ */+@Injectable()+export class MessageRetentionService implements OnModuleInit, OnModuleDestroy {+ private readonly logger = createLogger('MessageRetentionService');+ private timer?: ReturnType<typeof setInterval>;+ private followUp?: ReturnType<typeof setTimeout>;+ private stopped = false;+ private running = false;++ constructor(+ @InjectRepository(Message, 'data') private readonly messages: Repository<Message>,+ @InjectRepository(MessageBatch, 'data') private readonly batches: Repository<MessageBatch>,+ ) {}++ onModuleInit(): void {+ const days = resolveMessageRetentionDays();+ if (days <= 0) {+ this.logger.log('Message retention disabled (MESSAGE_RETENTION_DAYS unset or <= 0)');+ return;+ }+ const run = (): void => {+ this.prune()+ .then(({ messages, batches }) => {+ if (messages > 0 || batches > 0) {+ this.logger.log(+ `Pruned ${messages} message(s) and ${batches} finished batch(es) older than ${days} day(s)`,+ { action: 'messages_pruned', messages, batches },+ );+ }+ if (messages >= MAX_BATCHES_PER_RUN * PRUNE_BATCH_SIZE && !this.stopped) {+ this.logger.warn('Message retention hit its per-run cap; expired rows remain, running again shortly', {+ action: 'messages_prune_capped',+ });+ clearTimeout(this.followUp);+ this.followUp = setTimeout(run, CAPPED_RUN_FOLLOW_UP_MS);+ this.followUp.unref?.();+ }+ })+ .catch(err =>+ this.logger.error('Message retention prune failed', err instanceof Error ? err.stack : String(err)),+ );+ };+ run();+ this.timer = setInterval(run, DAY_MS);+ this.timer.unref?.();+ }++ onModuleDestroy(): void {+ this.stopped = true;+ if (this.timer) clearInterval(this.timer);+ clearTimeout(this.followUp);+ }++ /** One bounded pass. Overlap-guarded, and a no-op while retention is off. */+ async prune(now: Date = new Date()): Promise<{ messages: number; batches: number }> {+ const cutoff = resolveMessageRetentionCutoff(now);+ if (!cutoff || this.running) return { messages: 0, batches: 0 };+ this.running = true;+ try {+ let messages = 0;+ for (let i = 0; i < MAX_BATCHES_PER_RUN; i++) {+ const rows = await this.messages.find({+ select: { id: true },+ where: { createdAt: LessThan(cutoff) },+ order: { createdAt: 'ASC' },+ take: PRUNE_BATCH_SIZE,+ });+ if (rows.length === 0) break;+ await this.messages.delete({ id: In(rows.map(row => row.id)) });+ messages += rows.length;+ if (rows.length < PRUNE_BATCH_SIZE) break;+ }+ const { affected } = await this.batches.delete({+ status: In(TERMINAL_BATCH_STATUSES),+ createdAt: LessThan(cutoff),+ });+ return { messages, batches: affected ?? 0 };+ } finally {+ this.running = false;+ }+ }+}diff --git a/src/modules/message/message.module.ts b/src/modules/message/message.module.tsindex 1b0811e8f..f4afe8145 100644--- a/src/modules/message/message.module.ts+++ b/src/modules/message/message.module.ts@@ -5,6 +5,7 @@ import { MessageSendService } from './message-send.service';import { BulkMessageService } from './bulk-message.service';import { MessageTypeBackfillService } from './message-type-backfill.service';import { PendingMessageReaperService } from './pending-message-reaper.service';+import { MessageRetentionService } from './message-retention.service';import { MessageController } from './message.controller';import { SessionModule } from '../session/session.module';import { TemplateModule } from '../template/template.module';@@ -29,6 +30,7 @@ import { PLUGIN_MESSAGE_PORT } from '../../core/plugins/plugin-host-ports';BulkMessageService,MessageTypeBackfillService,PendingMessageReaperService,+ MessageRetentionService,SendPacingService,// Binds the core-owned plugin capability port to this module's service. The plugin runtime// resolves the token lazily via ModuleRef (PluginHostServices), which keeps its provider cyclediff --git a/src/modules/session/message-history-projector.ts b/src/modules/session/message-history-projector.tsindex 8fe94e426..7a9a7d708 100644--- a/src/modules/session/message-history-projector.ts+++ b/src/modules/session/message-history-projector.ts@@ -6,6 +6,7 @@ import { buildMessageMetadata } from './message-row.mapper';import { resolveFeatureFlags } from '../../config/feature-flags';import { IncomingMessage } from '../../engine/interfaces/whatsapp-engine.interface';import { LoggerService } from '../../common/services/logger.service';+import { resolveMessageRetentionCutoff } from '../message/message-retention.service';/*** Persist pre-connection history into the `messages` table for the chat view, without webhook/hook/ws@@ -20,6 +21,9 @@ export async function persistHistoryMessages(isLive: () => boolean,): Promise<void> {const storeEphemeralMessages = resolveFeatureFlags(configService).storeEphemeralMessages;+ // Rows are stamped with WhatsApp's own time below, so history older than the retention window+ // would be written only for the next prune to delete it again.+ const retentionCutoffMs = resolveMessageRetentionCutoff()?.getTime();const byId = new Map<string, IncomingMessage>();for (const m of messages) {// Need an id to de-dup; chatId/from/to are NOT NULL; status/story posts aren't chats.@@ -32,6 +36,9 @@ export async function persistHistoryMessages(if (!storeEphemeralMessages && (m.ephemeralDuration ?? 0) > 0) {continue;}+ if (retentionCutoffMs !== undefined && m.timestamp && m.timestamp * 1000 < retentionCutoffMs) {+ continue;+ }byId.set(m.id, m);}if (byId.size === 0) {diff --git a/src/modules/session/message-projector.service.spec.ts b/src/modules/session/message-projector.service.spec.tsindex 431300abe..5d8adbdb4 100644--- a/src/modules/session/message-projector.service.spec.ts+++ b/src/modules/session/message-projector.service.spec.ts@@ -280,6 +280,25 @@ describe('MessageProjector', () => {expect(dedupIds(messageRepository.find)).toEqual(['DUP']);});++ it('skips history older than the MESSAGE_RETENTION_DAYS window', async () => {+ const prev = process.env.MESSAGE_RETENTION_DAYS;+ process.env.MESSAGE_RETENTION_DAYS = '30';+ try {+ messageRepository.find.mockResolvedValue([{ waMessageId: 'NEW' }]);+ const nowSec = Math.floor(Date.now() / 1000);++ await projector.persistHistoryMessages('s1', engine, [+ historyMessage({ id: 'OLD', timestamp: nowSec - 31 * 86_400 }),+ historyMessage({ id: 'NEW', timestamp: nowSec - 29 * 86_400 }),+ ]);++ expect(dedupIds(messageRepository.find)).toEqual(['NEW']);+ } finally {+ if (prev === undefined) delete process.env.MESSAGE_RETENTION_DAYS;+ else process.env.MESSAGE_RETENTION_DAYS = prev;+ }+ });});});