Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

52.7 KB
Raw Blame
/**
 * Beeper ingestion sweep (#32, decided on #3 and #12; store shape from #7/#27,
 * Tribe linking from #10/#34, attachment policy from #13).
 *
 * This is the HTTP refetch that makes ingestion CORRECT. It is not a fallback
 * for a dead socket: Beeper's WebSocket is at-most-once with no replay on
 * reconnect, and it drops events during hydration while the connection looks
 * perfectly healthy, so a poll that only ran when the socket was down would be
 * asleep during two of the three loss paths. The socket (#33) makes ingestion
 * PROMPT; this sweep makes it complete, and it runs unconditionally on its own
 * timer.
 *
 * Watermark-bounded, never a full history rebuild. Per account:
 *   1. refresh `beeper_accounts` from the live roster (never `loginID`);
 *   2. page `GET /v1/chats?accountIDs=…` newest-first, sweeping every chat on
 *      the page that is newer than its stored
 *      `beeper_sync_cursors.last_activity`, within the bounded account page
 *      walk. A caught-up chat does not end the walk: a chat whose forward walk was truncated keeps its OLD watermark on
 *      purpose, so it stays eligible while sorting below chats that are
 *      already caught up;
 *   3. for each changed chat: upsert the conversation, upsert participants
 *      (`beeperTribe.upsertParticipant`), page new messages from the stored
 *      opaque cursor with `direction: 'after'`, log daily Tribe touchpoints
 *      (`beeperTribe.logSenderTouchpoints`), then commit message rows,
 *      attachment references and the cursor row in ONE transaction.
 *
 * **The transaction boundary is the point of the whole module.** Message rows,
 * attachment references and the `beeper_sync_cursors` row commit together, so
 * an interrupted sweep leaves the cursor exactly where it was and the next
 * sweep refetches the same window. `imessageSync` implements the same
 * never-skip rule by hand precisely because its store and its cursor live in
 * different places (#12 decision 8); here they share a database, so the
 * guarantee is the transaction's rather than the author's.
 *
 * Everything OUTSIDE that transaction is deliberately idempotent and ordered
 * BEFORE it, so a crash between steps can only cause repeated work, never lost
 * work: the conversation upsert, the participant upserts and the touchpoint
 * writes are all keyed and deduped (touchpoints on `beeper:<YYYY-MM-DD>`), and
 * they all re-run on the next sweep because the cursor never moved.
 *
 * No AI provider call happens anywhere on this path — ingestion is
 * deterministic — so AGENTS.md's "No cold-bootstrap LLM calls" does not gate
 * it, exactly as `imessageScheduler.js` records for the same shape.
 *
 * **Progress (fork issue #80).** `executeSweep` keeps `beeperSweepProgress.js`
 * current as it goes — running/idle, started/finished, accounts done of the
 * total, chats and messages mirrored so far — and `beeperStatus.js` reads it
 * into the status payload the list header and the drawer card render from. It
 * also fires an invalidation frame on `beeperSocketEvents` after the account
 * count is known and after every account finishes, reusing #33's existing
 * push path rather than adding a client-side poller.
 *
 * **A chat's first-page fetch is retried once if it comes back empty (#81).**
 * A brand-new chat's first sweep takes exactly one `direction: 'before'` page
 * (step 3 above). `sweepChat` withholds the cursor and watermark the first
 * time that page comes back with zero items (the same mechanism a truncated
 * forward walk already used), so `chatNeedsSweep` retries the same first page
 * next pass; if the retry is ALSO empty, the real watermark commits (so a
 * genuinely history-less chat still reaches the steady state `chatNeedsSweep`'s
 * docblock describes) and one warn-level line records it.
 *
 * **This retry does not reach every chat #81's live-instance investigation
 * found empty.** `chatNeedsSweep` only re-selects a chat once `chat.lastActivity`
 * is truthy, and the investigation's *entire observed population* was chats
 * Beeper itself never reports any activity for — `chat.lastActivity` null on
 * every sweep, not merely an empty first page. Those chats are swept exactly
 * once, commit `lastActivity: null`, and are never re-selected; the retry
 * above only covers a chat whose `chat.lastActivity` IS truthy but whose
 * first page still came back empty (a transient fetch hiccup, or a
 * history-less chat gaining its first-ever activity right as it enters the
 * mirror). What actually fixed the *reported* symptom — empty rows sorting
 * above threads with recent messages — for the observed population is the
 * ORDER BY in `beeperConversations.js`: it no longer falls back to the
 * mirror row's `created_at` for an activity-less chat (which made a whole
 * batch of them look freshly active because they were freshly SWEPT), so
 * they sort LAST instead of first. See `sweepChat`'s inline comment for the
 * full account of why the two null/null-producing paths are handled
 * identically rather than being told apart.
 */

import { query, withTransaction } from '../lib/db.js';
import {
  BeeperApiError,
  resolveBeeperConfig,
  getJoinedAccounts,
  listChatsPage,
  listMessagesPage,
  getMessage,
  getChat,
  createChat,
} from './beeperClient.js';
import { upsertParticipant, logSenderTouchpoints, loadRosterIndex } from './beeperTribe.js';
import { isInstanceFeatureEnabled } from './instanceFeatures.js';
import { getSettings } from './settings.js';
import { updateBeeperSweepProgress } from './beeperSweepProgress.js';
import { beeperSocketEvents } from './beeperSocketEvents.js';

export const DEFAULT_INTERVAL_MINUTES = 5;

// Safety caps. `GET /v1/chats` has no `limit`, so the page size is the
// server's; these bound one sweep's work rather than one page's.
const MAX_CHAT_PAGES_PER_ACCOUNT = 20;
// Catching up on new messages walks forward from the stored cursor, so this is
// a burst ceiling, not a history depth. A chat that is further behind than this
// keeps its (advanced) cursor AND its old watermark — see `fetchNewMessages`'s
// `truncated` flag — so it stays eligible and finishes catching up on the next
// sweep instead of being skipped as caught-up.
const MAX_MESSAGE_PAGES_PER_CHAT = 10;

const LOG_PREFIX = '🫧 Beeper sweep';

// ---------------------------------------------------------------------------
// Config + arming gate
// ---------------------------------------------------------------------------

/**
 * The `createSyncScheduler` config contract: `{ enabled, intervalMinutes }`.
 * `enabled` is the user's own "Enable scheduled Beeper sync" toggle from the
 * Comms → Beeper card (#30) — the ingestion flag, distinct from the instance
 * FEATURE flag that governs navigation (#11 keeps the two separate). The
 * interval is locked at registration by the factory; changing it needs a
 * restart, identical to the four existing ingestion domains.
 */
export async function getBeeperSyncConfig() {
  const settings = await getSettings().catch(() => ({}));
  const config = settings?.beeper || {};
  return {
    enabled: config.enabled === true,
    intervalMinutes: Number.isFinite(config.intervalMinutes) && config.intervalMinutes >= 1
      ? Math.floor(config.intervalMinutes)
      : DEFAULT_INTERVAL_MINUTES,
  };
}

/**
 * Whether the sweep may be armed at all: the instance feature is on AND a
 * token is configured (#12 decision 2). Deliberately SILENT — the acceptance
 * criterion on #32 is that with the feature off, or with no token, nothing is
 * registered and *nothing logs*; a fresh install that has never opened the
 * Beeper card should not narrate a feature it does not have.
 *
 * **Never gates on Beeper's `app.state`.** Measured on a live instance (#12):
 * the stream reported `initializing` for 105 continuous seconds while nine
 * accounts were `connected` and a chat page returned in 230 ms. Gating on it
 * would stall a healthy install indefinitely.
 *
 * The token is read through `resolveBeeperConfig`, the client's single
 * credential call site, so this works unchanged whether the token still lives
 * in `settings.beeper.token` or behind the encrypted vault (#31), which sits
 * behind that same function. The token value itself is never returned, logged
 * or stored.
 */
export async function isBeeperIngestionArmed() {
  const featureEnabled = await isInstanceFeatureEnabled('beeper').catch(() => false);
  if (!featureEnabled) return false;
  // A vault that cannot be read (corrupt row, rotated key) must not masquerade
  // as "no token": say so once, then decline to arm.
  const { token } = await resolveBeeperConfig().catch((err) => {
    console.error(`❌ Beeper credential unreadable, ingestion not armed: ${err.message}`);
    return { token: null };
  });
  return Boolean(token);
}

// ---------------------------------------------------------------------------
// Normalizers (pure — no I/O, no DB)
// ---------------------------------------------------------------------------

function toIsoOrNull(value) {
  if (value === null || value === undefined || value === '') return null;
  const date = new Date(value);
  return Number.isNaN(date.getTime()) ? null : date.toISOString();
}

function toIntOrNull(value) {
  return Number.isFinite(value) ? Math.trunc(value) : null;
}

/**
 * Account roster row. Fields are picked EXPLICITLY rather than spread — the
 * client already strips the top-level `loginID` (the bridge login credential,
 * effectively the user's phone number), and picking by name means a future
 * field added upstream cannot reach the database without someone naming it
 * here.
 */
export function normalizeAccountRow(account) {
  const accountId = typeof account?.accountID === 'string' ? account.accountID : '';
  return {
    accountId,
    network: String(account?.network ?? ''),
    displayName: String(account?.user?.fullName || account?.user?.username || account?.network || accountId || ''),
    status: String(account?.status || account?.bridgeStatus || ''),
    // account.bridgeId is the join result from getJoinedAccounts (see
    // beeperClient.js joinAccountsWithBridges). account?.bridge is the RAW
    // field /v1/accounts itself carries — live, it is an object
    // { id, type, provider }, never a string, so it must be narrowed to
    // `.id` rather than coerced whole (String() on the object stringifies to
    // the literal text "[object Object]"). The join can also legitimately
    // miss (an account absent from every /v1/bridges items[].accounts[]),
    // which is when this raw fallback is needed at all.
    bridgeId: String(account?.bridgeId || account?.bridge?.id || ''),
  };
}

/**
 * Conversation row. `type` is stored RAW: the schema deliberately carries no
 * enum constraint on it (`db.ddlParity.test.js` forbids one), so a new
 * Beeper chat type never needs a two-file migration to accept. `is_group` is
 * the derived boolean the UI filters on.
 */
export function normalizeChatRow(chat) {
  const type = String(chat?.type || 'single');
  return {
    accountId: String(chat?.accountID ?? ''),
    sourceChatId: String(chat?.id ?? ''),
    network: String(chat?.network ?? ''),
    title: String(chat?.title ?? ''),
    type,
    isGroup: type === 'group',
    isPinned: chat?.isPinned === true,
    isArchived: chat?.isArchived === true,
    isLowPriority: chat?.isLowPriority === true,
    isMuted: chat?.isMuted === true,
    snoozeUntil: toIsoOrNull(chat?.snooze?.snoozeUntil),
    lastActivity: toIsoOrNull(chat?.lastActivity),
    unreadCount: Number.isFinite(chat?.unreadCount) ? Math.max(0, Math.trunc(chat.unreadCount)) : 0,
  };
}

/**
 * Message row. Two mutation shapes ride in on an ordinarily-shaped message
 * rather than as their own events (#7/#13):
 *   - an EDIT is a populated `editedTimestamp`;
 *   - an UNSEND is `isDeleted: true`, a tombstone that keeps the row and its
 *     body. The API reports no deletion timestamp, so `unsent_at` records when
 *     PortOS first OBSERVED the tombstone (`observedAt`) — the upsert below
 *     COALESCEs it, so the first observation is the one that sticks.
 */
export function normalizeMessageRow(message, observedAt) {
  return {
    id: String(message?.id ?? ''),
    senderId: String(message?.senderID ?? ''),
    body: typeof message?.text === 'string' ? message.text : '',
    sentAt: toIsoOrNull(message?.timestamp),
    editedAt: toIsoOrNull(message?.editedTimestamp),
    unsentAt: message?.isDeleted === true ? observedAt : null,
    sortKey: String(message?.sortKey ?? ''),
    // The API's own `isSender`, and the ONLY reliable inbound/outbound signal:
    // `accounts[].user.id` differs from `senderID` on every network (#2), so a
    // comparison against the local user cannot be made. Strict `=== true`, so a
    // bridge that omits the field lands on inbound rather than on `undefined`.
    isSender: message?.isSender === true,
    observedAt,
  };
}

/**
 * Attachment REFERENCES only — metadata, never bytes (#13 puts the lazy byte
 * mirror and its disk budget in its own slice). `srcURL` and `posterImg` are
 * deliberately not persisted: both carry the spec's own "may be temporary or
 * local-only to this device" warning, so storing either would put a decaying
 * value in a durable archive. `sha256` has no source field on the inbound
 * `Attachment` at all and stays NULL until the byte mirror computes one.
 */
export function normalizeAttachmentRows(message) {
  const attachments = Array.isArray(message?.attachments) ? message.attachments : [];
  return attachments.map((attachment, idx) => ({
    idx,
    mxcId: typeof attachment?.id === 'string' && attachment.id ? attachment.id : null,
    // Lowercased (RFC 2045: a media type is case-insensitive) so the stored
    // value matches what `beeperAttachmentPaths.MIRRORED_MIME_TYPES` and the
    // client's `isImage()` check both compare against — a bridge that sends
    // `Image/JPEG` must not silently miss either one.
    mimeType: String(attachment?.mimeType ?? '').toLowerCase(),
    byteLength: toIntOrNull(attachment?.fileSize),
    fileName: String(attachment?.fileName ?? ''),
    width: toIntOrNull(attachment?.size?.width),
    height: toIntOrNull(attachment?.size?.height),
  }));
}

/**
 * Participant roster entries for one chat. `participants` is always an OBJECT
 * (`{ hasMore, items, total }`), never a bare array, and it TRUNCATES at 20 on
 * the list endpoint with no participants endpoint and no cursor to continue on
 * — which is exactly why `beeper_participants.observed_via` exists (#27): a
 * roster row set is always a possible subset.
 */
export function normalizeParticipants(chat) {
  const items = Array.isArray(chat?.participants?.items) ? chat.participants.items : [];
  return items
    .filter((participant) => typeof participant?.id === 'string' && participant.id)
    .map((participant) => ({
      sourceUserId: participant.id,
      displayName: String(participant.fullName || participant.username || ''),
      handle: String(participant.phoneNumber || participant.username || ''),
    }));
}

/**
 * Whether a chat has moved since its stored watermark. Three distinct states,
 * never collapsed (the root AGENTS.md absent-vs-empty rule):
 *   - no cursor row at all → never swept, always sweep;
 *   - a row with no watermark → sweep only once the chat reports activity, so
 *     a genuinely empty chat (a Discord server channel with zero history is
 *     frequently correct) is not re-paged every five minutes forever;
 *   - both present → strictly newer wins.
 */
export function chatNeedsSweep(chat, stored) {
  if (!stored) return true;
  if (!stored.lastActivity) return Boolean(chat?.lastActivity);
  if (!chat?.lastActivity) return false;
  return new Date(chat.lastActivity).getTime() > new Date(stored.lastActivity).getTime();
}

// ---------------------------------------------------------------------------
// Reads
// ---------------------------------------------------------------------------

/** Every stored cursor for one account, as a `Map` keyed on the Beeper chat id. */
async function readAccountCursors(accountId) {
  const result = await query(
    `SELECT chat_id, cursor, last_activity FROM beeper_sync_cursors WHERE account_id = $1`,
    [accountId],
  );
  const rows = Array.isArray(result?.rows) ? result.rows : [];
  return new Map(rows.map((row) => [row.chat_id, {
    cursor: row.cursor || null,
    lastActivity: row.last_activity ? new Date(row.last_activity).toISOString() : null,
  }]));
}

function assertPagedShape(page, what) {
  if (!Array.isArray(page?.items)) {
    throw new BeeperApiError(`Beeper API returned an unexpected ${what} response shape (expected { items: [] })`, {
      status: 502, code: 'MALFORMED_RESPONSE', retryable: false,
    });
  }
}

/**
 * New messages for one chat, plus the opaque cursor to store and whether the
 * forward walk stopped short of the end.
 *
 * With a stored cursor this walks FORWARD (`direction: 'after'`), which the
 * SDK's own auto-pagination never does — it only ever reads `oldestCursor` —
 * so the forward walk is driven by hand here.
 *
 * With NO stored cursor this takes the newest page ONLY and anchors the cursor
 * there. Deep history backfill is a different feature with a different cost
 * profile (history depth varies wildly per network, #3), and silently pulling
 * every message of every chat the first time a user enables the toggle is not
 * a sweep.
 *
 * `truncated` is true whenever a FORWARD walk ends with the server still
 * reporting `hasMore` — i.e. messages exist that this pass did not fetch.
 * That is the page budget running out, and equally a server that answers
 * `hasMore: true` with no usable cursor to continue on (a null one, or the one
 * it was just handed). The caller MUST NOT advance the chat's watermark in that case
 * (see `sweepChat`): the cursor moved forward but the newest messages are
 * still missing, and a watermark equal to the chat's current activity would
 * make `chatNeedsSweep` skip the chat forever. The first-page-only anchor walk
 * is never truncated — it is not trying to reach the end.
 */
async function fetchNewMessages(chatId, storedCursor, clientOptions) {
  const direction = storedCursor ? 'after' : 'before';
  const maxPages = storedCursor ? MAX_MESSAGE_PAGES_PER_CHAT : 1;
  const messages = [];
  let cursor = storedCursor || undefined;
  let anchorCursor = null;
  let truncated = false;

  for (let pageIndex = 0; pageIndex < maxPages; pageIndex++) {
    // eslint-disable-next-line no-await-in-loop -- cursor pagination is inherently sequential
    const page = await listMessagesPage(chatId, { cursor, direction, ...clientOptions });
    assertPagedShape(page, `/v1/chats/{chatID}/messages`);
    messages.push(...page.items);
    if (page.newestCursor) anchorCursor = page.newestCursor;
    if (!page.hasMore) break;
    const nextCursor = direction === 'after' ? page.newestCursor : page.oldestCursor;
    if (!nextCursor || nextCursor === cursor) {
      // The server says there is more and hands back no way to ask for it — a
      // null cursor, or the very cursor it was just given. On a forward walk
      // that is the same state as running out of page budget: messages exist
      // that this pass did not fetch, so the watermark must not commit over
      // them. Reading it as a clean end (which is what falling out of the loop
      // with `truncated === false` did) marked the chat caught up and lost
      // every message beyond the stall.
      truncated = direction === 'after';
      break;
    }
    cursor = nextCursor;
    if (direction === 'after' && pageIndex === maxPages - 1) truncated = true;
  }

  return { messages, cursor: anchorCursor, truncated };
}

// ---------------------------------------------------------------------------
// Writes
// ---------------------------------------------------------------------------

/**
 * Refresh `beeper_accounts` from the live roster. `getJoinedAccounts` joins
 * `/v1/accounts` to `/v1/bridges` by `accountID` (never by `network` — that
 * join returns zero rows) so the status card's `status`/`bridge_id` columns
 * are populated with Beeper Desktop closed. `loginID` never reaches here.
 */
async function refreshAccounts(clientOptions) {
  const accounts = (await getJoinedAccounts(clientOptions)).map(normalizeAccountRow)
    .filter((account) => account.accountId);
  for (const account of accounts) {
    // eslint-disable-next-line no-await-in-loop -- one roster, a handful of rows
    await query(
      `INSERT INTO beeper_accounts (account_id, network, display_name, status, bridge_id, last_seen_at)
       VALUES ($1, $2, $3, $4, $5, NOW())
       ON CONFLICT (account_id) DO UPDATE SET network = EXCLUDED.network,
         display_name = EXCLUDED.display_name, status = EXCLUDED.status,
         bridge_id = EXCLUDED.bridge_id, last_seen_at = NOW(), updated_at = NOW()`,
      [account.accountId, account.network, account.displayName, account.status, account.bridgeId],
    );
  }
  return accounts;
}

/**
 * Upsert the conversation row and return its synthetic UUID. Committed on its
 * own, BEFORE the message transaction, for two reasons: the participant writer
 * (`beeperTribe.upsertParticipant`) runs on the pool and cannot see an
 * uncommitted row, and a conversation row carries no cursor, so re-running it
 * is free. The watermark lives in `beeper_sync_cursors`, never here.
 */
async function upsertConversation(chat) {
  const result = await query(
    `INSERT INTO beeper_conversations (account_id, network, source_chat_id, title, type, is_group,
       is_pinned, is_archived, is_low_priority, is_muted, last_activity, unread_count, snooze_until)
     VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
     ON CONFLICT (account_id, source_chat_id) DO UPDATE SET network = EXCLUDED.network,
       title = EXCLUDED.title, type = EXCLUDED.type, is_group = EXCLUDED.is_group,
       is_pinned = EXCLUDED.is_pinned, is_archived = EXCLUDED.is_archived,
       is_low_priority = EXCLUDED.is_low_priority, is_muted = EXCLUDED.is_muted,
       last_activity = COALESCE(EXCLUDED.last_activity, beeper_conversations.last_activity),
       unread_count = EXCLUDED.unread_count, snooze_until = EXCLUDED.snooze_until, updated_at = NOW()
     RETURNING id`,
    [
      chat.accountId, chat.network, chat.sourceChatId, chat.title, chat.type, chat.isGroup,
      chat.isPinned, chat.isArchived, chat.isLowPriority, chat.isMuted, chat.lastActivity, chat.unreadCount, chat.snoozeUntil,
    ],
  );
  return result.rows[0]?.id ?? null;
}

/** User-triggered direct-chat creation. Reuse the sweep's mirror writer. */
export async function createBeeperConversation({ accountId, participantId }) {
  // Require a mirrored account before a remote write: the conversation FK
  // would otherwise fail after Beeper had already created the chat.
  const account = await query('SELECT account_id FROM beeper_accounts WHERE account_id = $1', [accountId]);
  if (!account.rows.length) throw new BeeperApiError('Unknown Beeper account', { status: 404, code: 'CONVERSATION_NOT_FOUND' });
  const options = await resolveBeeperConfig();
  const created = await createChat(accountId, participantId, options);
  // Older Desktop versions return only chatID; newer ones return the Chat.
  const sourceId = created?.id || created?.chatID;
  if (typeof sourceId !== 'string' || !sourceId) {
    throw new BeeperApiError('Beeper returned no chat ID', { code: 'MALFORMED_RESPONSE' });
  }
  const chat = await getChat(sourceId, options);
  if (chat?.id !== sourceId || chat?.accountID !== accountId) {
    throw new BeeperApiError('Beeper returned a mismatched chat', { code: 'MALFORMED_RESPONSE' });
  }
  const id = await upsertConversation(normalizeChatRow(chat));
  if (!id) throw new BeeperApiError('Could not mirror the created chat', { code: 'MALFORMED_RESPONSE' });
  beeperSocketEvents.emit('invalidate', { kind: 'chats', chatID: sourceId });
  return { id };
}

/** Shared transaction writer: source edit versions outrank fetch-start time.
 * Tombstones are terminal and preserve the archived body and attachments.
 */
export async function upsertMirroredMessage(client, conversationId, message, attachments = []) {
  const result = await client.query(
    `INSERT INTO beeper_messages (id, conversation_id, sender_id, body, sent_at, edited_at, unsent_at, sort_key, is_sender, observed_at)
     VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
     ON CONFLICT (id) DO UPDATE SET
       body = CASE WHEN EXCLUDED.unsent_at IS NOT NULL OR beeper_messages.unsent_at IS NOT NULL
         THEN beeper_messages.body ELSE COALESCE(NULLIF(EXCLUDED.body, ''), beeper_messages.body) END,
       sender_id = COALESCE(NULLIF(EXCLUDED.sender_id, ''), beeper_messages.sender_id),
       sent_at = COALESCE(EXCLUDED.sent_at, beeper_messages.sent_at),
       edited_at = GREATEST(EXCLUDED.edited_at, beeper_messages.edited_at),
       unsent_at = COALESCE(beeper_messages.unsent_at, EXCLUDED.unsent_at),
       sort_key = COALESCE(NULLIF(EXCLUDED.sort_key, ''), beeper_messages.sort_key),
       -- Never downgrades a stored TRUE to FALSE: the field is optional on
       -- the inbound Message, so a later page that omits it must not flip a
       -- message the user actually sent onto the other side of the thread.
       is_sender = beeper_messages.is_sender OR EXCLUDED.is_sender,
       observed_at = EXCLUDED.observed_at, updated_at = NOW()
     WHERE beeper_messages.conversation_id = EXCLUDED.conversation_id
       AND ((EXCLUDED.unsent_at IS NOT NULL AND EXCLUDED.observed_at >= beeper_messages.observed_at) OR
         (beeper_messages.unsent_at IS NULL
          AND COALESCE(EXCLUDED.edited_at, 'epoch'::timestamptz) >= COALESCE(beeper_messages.edited_at, 'epoch'::timestamptz)
          AND (EXCLUDED.edited_at > COALESCE(beeper_messages.edited_at, 'epoch'::timestamptz)
               OR EXCLUDED.observed_at >= beeper_messages.observed_at)))
     RETURNING id`,
    [
      message.id, conversationId, message.senderId, message.body,
      message.sentAt, message.editedAt, message.unsentAt, message.sortKey,
      message.isSender, message.observedAt,
    ],
  );
  // A stale observation or tombstone must not rewrite attachment references.
  if (result.rowCount === 0 || message.unsentAt) return;
  for (const attachment of attachments) {
    // eslint-disable-next-line no-await-in-loop -- same transaction, ordered after its message
    await client.query(
      `INSERT INTO beeper_attachments (conversation_id, message_id, idx, mxc_id, mime_type,
         byte_length, file_name, width, height)
       VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
       ON CONFLICT (conversation_id, message_id, idx) DO UPDATE SET
         mxc_id = COALESCE(EXCLUDED.mxc_id, beeper_attachments.mxc_id),
         mime_type = EXCLUDED.mime_type, byte_length = EXCLUDED.byte_length,
         file_name = EXCLUDED.file_name, width = EXCLUDED.width, height = EXCLUDED.height,
         updated_at = NOW()`,
      [
        conversationId, message.id, attachment.idx, attachment.mxcId, attachment.mimeType,
        attachment.byteLength, attachment.fileName, attachment.width, attachment.height,
      ],
    );
  }
}

async function commitMessages({ conversationId, accountId, sourceChatId, rows, cursor, lastActivity }) {
  return withTransaction(async (client) => {
    for (const { message, attachments } of rows) {
      // eslint-disable-next-line no-await-in-loop -- ordered writes inside one transaction
      await upsertMirroredMessage(client, conversationId, message, attachments);
    }

    // The cursor moves LAST and only inside this transaction, so a failure
    // anywhere above rolls it back with the rows it was going to describe.
    //
    // `WHERE EXISTS` is the purge interlock. `beeper_sync_cursors` carries an FK
    // onto `beeper_accounts` alone, so the conversation's cascade cannot reach
    // it and `purgeConversation` deletes the two by hand in one transaction. A
    // sweep already in flight when that purge commits would otherwise re-insert
    // the cursor here — unconditionally, because a window with ZERO message rows
    // still writes it — carrying the chat's current `last_activity`. That
    // resurrected watermark makes `chatNeedsSweep` skip the chat, so the history
    // the user just purged never comes back, contradicting the typed
    // confirmation's promise that the next sync will mirror it again.
    //
    // The guard is a row check rather than a constraint on purpose: no DDL, no
    // cascade, and the schema's own note that "a cursor can exist before the
    // conversation row does" still holds everywhere else. On this path the
    // conversation is upserted before the walk starts, so a missing row means it
    // was deleted underneath us — exactly the case to decline.
    await client.query(
      `INSERT INTO beeper_sync_cursors (account_id, chat_id, cursor, last_activity, last_swept_at)
       SELECT $1::text, $2::text, $3::text, $4::timestamptz, NOW()
        WHERE EXISTS (SELECT 1 FROM beeper_conversations WHERE id = $5::uuid)
       ON CONFLICT (account_id, chat_id) DO UPDATE SET
         cursor = COALESCE(EXCLUDED.cursor, beeper_sync_cursors.cursor),
         last_activity = COALESCE(EXCLUDED.last_activity, beeper_sync_cursors.last_activity),
         last_swept_at = NOW()`,
      [accountId, sourceChatId, cursor, lastActivity, conversationId],
    );
    return rows.length;
  });
}

// ---------------------------------------------------------------------------
// Sweep
// ---------------------------------------------------------------------------

async function sweepChat({
  chat, stored, clientOptions, observedAt, personIndex,
}) {
  const normalized = normalizeChatRow(chat);
  if (!normalized.accountId || !normalized.sourceChatId) return { messages: 0 };

  const conversationId = await upsertConversation(normalized);
  if (!conversationId) return { messages: 0 };

  const { messages, cursor, truncated } = await fetchNewMessages(
    normalized.sourceChatId, stored?.cursor, clientOptions,
  );

  // #81: a first-page anchor fetch (no stored cursor to walk forward from,
  // so `fetchNewMessages` took the `direction: 'before'` branch) that comes
  // back with zero items is NOT the same as "caught up" — `truncated` is
  // never set on that branch, so falling through to the normal commit below
  // would watermark the chat as done on one empty page and never look again.
  //
  // IMPORTANT — this retry does NOT reach every empty chat. `chatNeedsSweep`
  // only re-selects a chat once `chat.lastActivity` is truthy; a chat Beeper
  // itself never reports any activity for (`chat.lastActivity` stays null
  // forever, which is the *entire observed population* on the live instance
  // #81 investigated) is swept exactly once, commits null/null below, and is
  // then never re-selected — the retry cannot fire because there is nothing
  // for `chatNeedsSweep` to compare a "gained activity" transition against.
  // That population's empty-row-sorts-first symptom is fixed separately, in
  // `beeperConversations.js`'s ORDER BY (an activity-less chat now sorts
  // LAST instead of by its mirror row's mint time). This retry instead
  // covers the case `chat.lastActivity` IS truthy but the first page still
  // came back empty — a transient fetch hiccup, or a chat that gains its
  // first-ever activity right as it enters the mirror.
  //
  // `hadEmptyFirstPageAttempt` recognizes the state a WITHHELD commit leaves
  // behind: a cursor row that exists but carries neither a cursor nor a
  // watermark. That state is NOT exclusive to the withholding branch below —
  // a chat with `chat.lastActivity` null commits exactly the same null/null
  // shape on an ordinary first sweep (see above). The two are handled
  // identically on purpose: whether this null/null row is a prior withhold
  // or a history-less chat that has just gained its first real activity,
  // the right move is the same — one retry, then commit-and-warn if it is
  // still empty — so nothing here needs to (or can, from one sweep's
  // evidence) tell the two apart. No new DB column or migration needed.
  const isFirstPageAttempt = !stored?.cursor;
  const emptyFirstPage = isFirstPageAttempt && messages.length === 0;
  const hadEmptyFirstPageAttempt = Boolean(stored) && !stored.cursor && !stored.lastActivity;

  let commitCursor = cursor || stored?.cursor || null;
  let commitLastActivity = truncated ? null : normalized.lastActivity;

  if (emptyFirstPage) {
    if (hadEmptyFirstPageAttempt) {
      // Still empty after one retry: commit the real watermark so the chat
      // stops being re-swept every pass (mirroring the "genuinely empty
      // chat" steady state `chatNeedsSweep` already protects), and log once
      // so an operator can tell a chat landed here versus simply having no
      // history. Chat id only — never message content.
      console.warn(`${LOG_PREFIX}: chat ${normalized.sourceChatId} still has no mirrored messages after a retry`);
    } else {
      // First time this chat has come back empty. Withhold the cursor and
      // watermark so `chatNeedsSweep` picks it back up next pass and retries
      // the same first-page fetch, exactly once.
      commitCursor = null;
      commitLastActivity = null;
    }
  }

  // Roster first, then senders: a message-sender observation carries a richer
  // handle than a truncated roster entry, and `upsertParticipant` COALESCEs
  // the handle rather than overwriting it, so the better one wins either way.
  //
  // `personIndex` is `executeSweep`'s ONE Tribe-roster load for the whole
  // sweep pass, threaded down here and to the sender loop below — without it
  // `upsertParticipant`'s phone-match fallback would reload and reindex every
  // Tribe person once per participant, in every chat, on every pass.
  for (const participant of normalizeParticipants(chat)) {
    // eslint-disable-next-line no-await-in-loop -- at most 20 per chat (the list endpoint truncates there)
    await upsertParticipant({
      conversationId, ...participant, observedVia: 'participant-list', personIndex,
    });
  }

  const rows = messages
    .map((message) => ({ message: normalizeMessageRow(message, observedAt), raw: message }))
    .filter(({ message }) => message.id)
    .map(({ message, raw }) => ({ message, raw, attachments: normalizeAttachmentRows(raw) }));

  const seenSenders = new Set();
  for (const { message, raw } of rows) {
    if (!message.senderId || seenSenders.has(message.senderId)) continue;
    seenSenders.add(message.senderId);
    // eslint-disable-next-line no-await-in-loop -- one row per distinct sender in this window
    await upsertParticipant({
      conversationId,
      sourceUserId: message.senderId,
      displayName: String(raw?.senderName ?? ''),
      handle: '',
      observedVia: 'message-sender',
      personIndex,
    });
  }

  // Touchpoints are derived from message SENDERS only, never from the
  // participant roster (which truncates, so iterating it would invent contact
  // with people who never messaged). Written BEFORE the commit deliberately:
  // they are day-deduped on `beeper:<YYYY-MM-DD>`, so a rolled-back sweep
  // re-derives exactly the same ones next pass, whereas writing them after the
  // cursor advanced would lose them for good.
  await logSenderTouchpoints(rows
    .filter(({ raw }) => raw?.isSender !== true)
    .map(({ message }) => ({
      conversationId,
      senderId: message.senderId,
      sentAt: message.sentAt,
      network: normalized.network,
    })), { personIndex });

  await commitMessages({
    conversationId,
    accountId: normalized.accountId,
    sourceChatId: normalized.sourceChatId,
    rows,
    // A truncated forward walk advanced the cursor but did NOT reach the
    // chat's newest message, so the watermark must stay where it was: a null
    // here hits `COALESCE(EXCLUDED.last_activity, beeper_sync_cursors.…)` in
    // `commitMessages` and keeps the stored value. `chatNeedsSweep` then still
    // sees the chat as newer than its watermark next pass and resumes from the
    // advanced cursor. Committing `normalized.lastActivity` instead would make
    // the two timestamps equal, end the account walk at this chat, and strand
    // the un-fetched backlog until unrelated new activity arrived. An empty
    // first-page attempt withholds both for the same reason — see
    // `emptyFirstPage` / `hadEmptyFirstPageAttempt` above (#81).
    cursor: commitCursor,
    lastActivity: commitLastActivity,
  });

  return { messages: rows.length };
}

/**
 * One account's newest-first walk. A chat that throws is logged and counted,
 * and the walk CONTINUES — the same blast-radius bound `executeSweep` gives
 * each account. Without it, one chat that fails persistently (a bridge that
 * 502s on its messages, say) sitting at the top of the newest-first list would
 * abort this account's whole walk on every single pass and starve every chat
 * below it indefinitely. Its cursor and watermark are untouched by the failed
 * transaction, so it is retried on a later traversal.
 *
 * A CAUGHT-UP CHAT DOES NOT END THE WALK, and that is the whole reason the
 * truncation contract works. `sweepChat` withholds the watermark from a chat
 * whose forward walk ran out of page budget, precisely so the chat stays
 * eligible next pass — but the list is ordered by last activity, not by
 * eligibility, so that chat sorts wherever its last message puts it. Ending the
 * walk at the first chat that is not newer than its watermark therefore made
 * the two rules cancel each other out: in the steady state, where the chats
 * above are all caught up, a truncated chat sitting below one of them was never
 * reached again and its backlog was stranded until unrelated activity moved it
 * to the top of the list.
 *
 * A caught-up page cannot terminate the walk either: older pages can contain
 * truncated, failed, or purged chats. Persist progress at the page cap and
 * resume next run, reserving one page to refresh the head. End-of-list resets
 * the checkpoint so failed chats remain reachable on subsequent traversals.
 */
async function sweepAccount(account, clientOptions, observedAt, personIndex) {
  await reconcileStoredMessages(account.accountId, clientOptions);
  const cursors = await readAccountCursors(account.accountId);
  const checkpoint = await query('SELECT chat_cursor FROM beeper_accounts WHERE account_id = $1', [account.accountId]);
  let cursor = checkpoint.rows[0]?.chat_cursor || undefined;
  let refreshHead = Boolean(cursor);
  let continuationPending = true;
  let cursorRestarts = 0;
  let chatsSwept = 0;
  let messagesWritten = 0;
  let failedChats = 0;

  for (let pageIndex = 0; pageIndex < MAX_CHAT_PAGES_PER_ACCOUNT; pageIndex++) {
    const requestCursor = refreshHead ? undefined : cursor;
    // eslint-disable-next-line no-await-in-loop -- cursor pagination is inherently sequential
    const page = await listChatsPage({
      cursor: requestCursor, direction: 'before', accountIDs: [account.accountId], ...clientOptions,
    }).catch(async (err) => {
      // A rejected pagination request can be retried from the head. Authentication,
      // transport and server failures must retain the checkpoint, not erase it.
      if (!requestCursor || !(err instanceof BeeperApiError) || ![400, 410, 422].includes(err.status)) throw err;
      await query('UPDATE beeper_accounts SET chat_cursor = $2 WHERE account_id = $1', [account.accountId, null]);
      cursor = undefined;
      cursorRestarts++;
      console.warn(`⚠️ ${LOG_PREFIX}: restarting rejected account pagination`);
      return null;
    });
    if (!page) continue;
    assertPagedShape(page, '/v1/chats');

    for (const chat of page.items) {
      const stored = cursors.get(String(chat?.id ?? ''));
      if (!chatNeedsSweep(chat, stored)) {
        // Snooze state can change without message activity. Refresh it on every
        // enumerated chat without refetching history or rewriting unchanged rows.
        const normalized = normalizeChatRow(chat);
        if (normalized.accountId && normalized.sourceChatId) {
          // eslint-disable-next-line no-await-in-loop -- one mirrored chat at a time
          await query(
            `UPDATE beeper_conversations SET snooze_until = $3, updated_at = NOW()
              WHERE account_id = $1 AND source_chat_id = $2
                AND snooze_until IS DISTINCT FROM $3::timestamptz`,
            [normalized.accountId, normalized.sourceChatId, normalized.snoozeUntil],
          ).catch((err) => {
            failedChats++;
            console.error(`❌ ${LOG_PREFIX}: chat state refresh failed: ${err.message}`);
          });
        }
        continue;
      }
      // eslint-disable-next-line no-await-in-loop -- one chat at a time; each is its own transaction
      const result = await sweepChat({
        chat, stored, clientOptions, observedAt, personIndex,
      }).catch((err) => {
        failedChats++;
        console.error(`❌ ${LOG_PREFIX}: chat ${String(chat?.id ?? 'unknown')} failed: ${err.message}`);
        return null;
      });
      if (!result) continue;
      chatsSwept++;
      messagesWritten += result.messages;
    }

    // The head refresh spends one page of the same budget but never overwrites
    // the historical checkpoint. Failed chats keep their own old watermarks and
    // are retried on a later traversal without starving the rest of the account.
    if (refreshHead) {
      refreshHead = false;
      continue;
    }
    if (page.hasMore && (!page.oldestCursor || page.oldestCursor === cursor)) {
      throw new BeeperApiError('Beeper account pagination stalled without a usable next cursor', {
        code: 'MALFORMED_RESPONSE', status: 502,
      });
    }
    cursor = page.hasMore ? page.oldestCursor : null;
    // Only advance after all work on this page was attempted. A crash or write
    // failure replays this page through the existing idempotent message writes.
    // eslint-disable-next-line no-await-in-loop -- checkpoint follows page work
    await query('UPDATE beeper_accounts SET chat_cursor = $2 WHERE account_id = $1', [account.accountId, cursor]);
    continuationPending = Boolean(page.hasMore);
    if (!continuationPending) break;
  }

  return { chatsSwept, messagesWritten, failedChats, continuationPending, cursorRestarts };
}

// One bounded rotation per account, independent of the forward message cursor.
// A persisted upper bound makes each rotation finite even during new ingestion.
// Advance past failures so one inaccessible message cannot starve the archive;
// a later rotation retries it. No failure is interpreted as a deletion.
const RECONCILE_LIMIT = 20;

async function reconcileStoredRow(row, clientOptions) {
  const observedAt = new Date().toISOString();
  const raw = await getMessage(row.source_chat_id, row.id, clientOptions);
  if (raw?.id !== row.id) throw new Error('Unexpected Beeper message identity');
  await withTransaction(async (client) => {
    // Lock the existing row: purge must not resurrect a message while HTTP was in flight.
    const existing = await client.query(
      'SELECT id FROM beeper_messages WHERE id = $1 AND conversation_id = $2 FOR UPDATE',
      [row.id, row.conversation_id],
    );
    if (!existing.rows.length) return;
    await upsertMirroredMessage(client, row.conversation_id,
      normalizeMessageRow(raw, observedAt), normalizeAttachmentRows(raw));
  });
  beeperSocketEvents.emit('invalidate', {
    kind: 'beeper-reconciled', chatID: row.source_chat_id, ids: [row.id],
    seq: null, ts: observedAt,
  });
}

async function reconcileStoredMessages(accountId, clientOptions) {
  const checkpoint = await query(
    'SELECT message_id, upper_bound FROM beeper_reconcile_cursors WHERE account_id = $1', [accountId],
  );
  let position = checkpoint.rows[0];
  if (!position?.upper_bound) {
    const boundary = await query(
      `SELECT MAX(m.id) AS upper_bound FROM beeper_messages m
       JOIN beeper_conversations c ON c.id = m.conversation_id WHERE c.account_id = $1`, [accountId],
    );
    if (!boundary.rows[0]?.upper_bound) return;
    position = { message_id: '', upper_bound: boundary.rows[0].upper_bound };
    await query(
      `INSERT INTO beeper_reconcile_cursors (account_id, message_id, upper_bound) VALUES ($1, '', $2)
       ON CONFLICT (account_id) DO UPDATE SET message_id = '', upper_bound = EXCLUDED.upper_bound`,
      [accountId, position.upper_bound],
    );
  }
  const result = await query(
    `SELECT m.id, m.conversation_id, c.source_chat_id
       FROM beeper_messages m JOIN beeper_conversations c ON c.id = m.conversation_id
      WHERE c.account_id = $1 AND m.id > $2 AND m.id <= $4 ORDER BY m.id LIMIT $3`,
    [accountId, position.message_id, RECONCILE_LIMIT, position.upper_bound],
  );
  for (const row of result.rows) {
    // eslint-disable-next-line no-await-in-loop -- bounded sequential upstream work
    await reconcileStoredRow(row, clientOptions).catch(() => {
      console.warn('🫧 Beeper reconciliation deferred a message until the next rotation');
    });
    // Progress commits after persistence; a crash before this repeats harmlessly.
    // eslint-disable-next-line no-await-in-loop -- durable checkpoint follows each attempt
    await query(
      `UPDATE beeper_reconcile_cursors SET message_id = $2 WHERE account_id = $1`,
      [accountId, row.id],
    );
  }
  if (result.rows.length === 0) {
    await query('DELETE FROM beeper_reconcile_cursors WHERE account_id = $1', [accountId]);
  }
}

// Bound both the event queue and concurrency. Overflow is repaired by the
// durable rotation; event payloads never authorize ingesting unmirrored history.
const pendingReconciliations = new Map();
let eventReconciliation = null;
export function reconcileBeeperEvent(frame) {
  if (!['message.upserted', 'message.deleted'].includes(frame?.kind) || !frame.chatID) return Promise.resolve();
  for (const id of (frame.ids || []).slice(0, RECONCILE_LIMIT)) {
    if (typeof id === 'string' && pendingReconciliations.size < 100) {
      pendingReconciliations.set(JSON.stringify([frame.chatID, id]), { chatID: frame.chatID, id });
    }
  }
  if (eventReconciliation) return eventReconciliation;
  eventReconciliation = drainReconciliationEvents().finally(() => { eventReconciliation = null; });
  return eventReconciliation;
}

async function drainReconciliationEvents() {
  while (pendingReconciliations.size) {
    const [key, item] = pendingReconciliations.entries().next().value;
    pendingReconciliations.delete(key);
    // eslint-disable-next-line no-await-in-loop -- recheck ingestion consent for queued work
    const config = await getBeeperSyncConfig();
    // eslint-disable-next-line no-await-in-loop -- credential/feature gate
    if (!config.enabled || !await isBeeperIngestionArmed()) {
      pendingReconciliations.clear();
      return;
    }
    // eslint-disable-next-line no-await-in-loop -- one transport request at a time
    await (async () => {
      const result = await query(
        `SELECT m.id, m.conversation_id, c.source_chat_id
           FROM beeper_messages m JOIN beeper_conversations c ON c.id = m.conversation_id
          WHERE m.id = $1 AND c.source_chat_id = $2`, [item.id, item.chatID],
      );
      if (result.rows[0]) await reconcileStoredRow(result.rows[0], await resolveBeeperConfig());
    })().catch(() => console.warn('🫧 Beeper event reconciliation deferred to the periodic rotation'));
  }
}

// A per-run re-entrancy guard: one sweep at a time, process-wide. The timer,
// the manual run-now route and #33's three socket triggers (reconnect, `seq`
// gap, `app.state` recovery) all land here, and two overlapping sweeps would
// page the same chats twice and race on the same cursor rows. A caller that
// arrives mid-sweep is told so rather than queued — the in-flight sweep is
// already fetching everything the newcomer would have.
let inFlightSweep = null;

/**
 * One invalidation frame, reusing the exact shape `beeperSocket.js` emits for
 * a live domain frame (`{ kind, chatID, ids, seq, ts }`, fork issue #33) so
 * the browser side needs no new handling: `useBeeperRealtime` already relays
 * every frame's `kind` untouched, and `BeeperChatSurface`'s debounce treats a
 * `chatID: null` frame as "could be about anything" and refetches the list —
 * exactly right here, since one sweep can touch any number of conversations.
 * This is fork issue #80's "reuse the existing push path instead of adding a
 * client poller": `BeeperTab.jsx` already re-fetches `GET /api/beeper/status`
 * on every invalidation frame, so a frame here is what makes the "Syncing… N
 * of M accounts" strip move and the "Last synced" timestamp update live,
 * without a second polling loop anywhere.
 */
function emitSweepInvalidation() {
  beeperSocketEvents.emit('invalidate', {
    kind: 'beeper-sweep', chatID: null, ids: [], seq: null, ts: new Date().toISOString(),
  });
}

async function executeSweep(reason) {
  updateBeeperSweepProgress({
    running: true, startedAt: new Date().toISOString(), finishedAt: null, reason,
    accountsDone: 0, accountsTotal: null, chats: 0, messages: 0,
  });
  try {
    const clientOptions = await resolveBeeperConfig();
    if (!clientOptions.token) {
      throw new BeeperApiError('Beeper access token is not configured', {
        status: 401, code: 'NOT_CONFIGURED', retryable: false,
      });
    }

    const startedAt = Date.now();
    const observedAt = new Date(startedAt).toISOString();
    const accounts = await refreshAccounts(clientOptions);
    updateBeeperSweepProgress({ accountsTotal: accounts.length });
    emitSweepInvalidation();

    // Loaded ONCE for the whole sweep pass, across every account/chat/participant
    // below — not once per participant. `upsertParticipant`'s phone-match
    // fallback reindexes the entire Tribe roster on a miss, and a sweep touches
    // every participant of every changed chat on every account.
    const personIndex = await loadRosterIndex();

    let chats = 0;
    let messages = 0;
    let accountsDone = 0;
    let failedAccounts = 0;
    let failedChats = 0;
    let unfinishedAccounts = 0;
    let cursorRestarts = 0;
    for (const account of accounts) {
      // eslint-disable-next-line no-await-in-loop -- accounts are swept in order; each owns its own cursors
      const result = await sweepAccount(account, clientOptions, observedAt, personIndex).catch((err) => {
        failedAccounts++;
        console.error(`❌ ${LOG_PREFIX}: account ${account.accountId} failed: ${err.message}`);
        return null;
      });
      if (result) {
        chats += result.chatsSwept;
        messages += result.messagesWritten;
        failedChats += result.failedChats;
        unfinishedAccounts += Number(result.continuationPending);
        cursorRestarts += result.cursorRestarts;
      }
      // #80: one account done, whether it succeeded or failed — "N of M" counts
      // accounts attempted, not accounts that mirrored something.
      accountsDone++;
      updateBeeperSweepProgress({ accountsDone, chats, messages });
      emitSweepInvalidation();
    }

    // Every account failing is a FAILED sweep, not a successful one that wrote
    // nothing. Resolving here would have `eventScheduler` record a green run and
    // `POST /api/beeper/sync` answer 200 while nothing was ingested at all —
    // with only the `failedAccounts` count in the payload to say otherwise.
    // Partial failure still resolves: the accounts that worked kept their pass.
    if (accounts.length > 0 && failedAccounts === accounts.length) {
      throw new BeeperApiError(`Beeper sweep failed for all ${accounts.length} accounts`, {
        status: 502, code: 'SWEEP_FAILED', retryable: true,
      });
    }

    const durationMs = Date.now() - startedAt;
    console.log(`${LOG_PREFIX} (${reason}): ${accounts.length} accounts, ${chats} chats, ${messages} messages in ${durationMs}ms`);
    return {
      skipped: false,
      reason,
      accounts: accounts.length,
      chats,
      messages,
      failedAccounts,
      failedChats,
      unfinishedAccounts,
      enumerationComplete: unfinishedAccounts === 0 && failedAccounts === 0,
      cursorRestarts,
      durationMs,
    };
  } finally {
    // Always runs — a thrown NOT_CONFIGURED or SWEEP_FAILED must flip `running`
    // back to false exactly like a clean resolve, or the status card would show
    // a sweep in progress forever after the first failure.
    updateBeeperSweepProgress({ running: false, finishedAt: new Date().toISOString() });
    emitSweepInvalidation();
  }
}

/**
 * Run one watermark-bounded sweep. Exported for the scheduler, the manual
 * run-now route, and #33's three socket triggers.
 *
 * Throws when the sweep cannot start at all (no token, roster unreachable) or
 * when EVERY account failed — that is a failed run, not a successful one that
 * wrote nothing. Partial failure resolves: a single account is isolated and
 * reported as `failedAccounts` so one broken bridge cannot cost the others
 * their pass, and a single failing chat as `failedChats` so it cannot starve
 * the chats below it in its account's walk.
 */
export async function runBeeperSweep({ reason = 'scheduler' } = {}) {
  if (inFlightSweep) {
    console.log(`${LOG_PREFIX} (${reason}): a sweep is already running — skipping`);
    return {
      skipped: true, reason, accounts: 0, chats: 0, messages: 0, failedAccounts: 0, failedChats: 0, durationMs: 0,
    };
  }
  const run = executeSweep(reason);
  inFlightSweep = run;
  try {
    return await run;
  } finally {
    inFlightSweep = null;
  }
}