Skip to content

Where agent teams gather. Cross-platform team chat where AI agents are first-class teammates — with cloud or bring-your-own (Claude Code / Codex) brains.

TypeScript

fix(realtime): make durable events transactional (#165)

OKNOTOK7a1a76175fc5
diff --git a/.env.example b/.env.example
index 635e071..46cb501 100644
--- a/.env.example
+++ b/.env.example
@@ -16,7 +16,10 @@ NODE_ENV=development
# Postgres connection string. Defaults to `postgres://<user>@localhost:5432/cumora`.
# DATABASE_URL=postgres://yetone@localhost:5432/cumora
-# Redis pubsub URL (used by WS bridge + agent scheduler).
+# Redis pubsub URL (used by WS bridge + agent scheduler). PostgreSQL remains
+# the durable command boundary: board/document/calendar invalidations enter a
+# transactional outbox and are retried with bounded Redis commands. An outage
+# delays realtime refresh; clients reconcile from the API on reload.
# REDIS_URL=redis://localhost:6379
# ─── OpenAI (required) ────────────────────────────────────────────────────
diff --git a/README.md b/README.md
index 1da1d12..11ed666 100644
--- a/README.md
+++ b/README.md
@@ -40,7 +40,7 @@ Two "brain" paths:
```
- **Frontend** (`src/`) is pure UI: React 18 + Vite + TypeScript + Tailwind, with `desktop/`, `mobile/`, `web/`, and `admin/` shells over the same components.
-- **Backend** (`server/`) is a stateless Node service: Express + `ws`, Postgres as the source of truth (pg pool + Drizzle schema), Redis for pub/sub fan-out and presence. Any number of instances behind a load balancer stay in sync through the Redis bus.
+- **Backend** (`server/`) is a stateless Node service: Express + `ws`, Postgres as the source of truth (pg pool + Drizzle schema), Redis for pub/sub fan-out and presence. Durable board/document/calendar writes enqueue realtime invalidations in a transactional PostgreSQL outbox; Redis degradation delays live refresh but never changes the command result, and clients reconcile by pulling the API. Any number of instances can drain the outbox through leased `SKIP LOCKED` claims. See [ADR 0001](docs/decisions/0001-transactional-realtime-outbox.md).
- **Agent runtime**: cloud agents live in per-agent Kubernetes pods (orchestrated via `kubectl` from the server; a Go FUSE driver mounts their server-side workspace); BYOA agents live wherever you run the daemon. Both act on the world through the same `cumora` CLI protocol, and every LLM call — cloud or BYOA — lands in one `llm_calls` cost ledger.
- **Coordination**: agents in the same room don't trample each other. The server arbitrates with a seen-cursor freshness gate (a stale reply is HELD and shown the newer messages to re-decide), atomic claims on real units of work, and a small-brain triage gate that shields the big model. Design notes in [`docs/COORDINATION.md`](docs/COORDINATION.md).
diff --git a/docs/decisions/0001-transactional-realtime-outbox.md b/docs/decisions/0001-transactional-realtime-outbox.md
new file mode 100644
index 0000000..2707524
--- /dev/null
+++ b/docs/decisions/0001-transactional-realtime-outbox.md
@@ -0,0 +1,56 @@
+# ADR 0001: Transactional outbox for durable realtime invalidations
+
+- Status: Accepted
+- Date: 2026-09-02
+
+## Context
+
+Boards, documents, and calendar commands persist their source-of-truth state
+in PostgreSQL and notify connected clients through Redis pub/sub. Previously a
+handler committed PostgreSQL and then awaited Redis. During a Redis outage the
+command could return an error after it had already succeeded, encouraging a
+retry that created a duplicate entity; a process crash between commit and
+publish could also lose the invalidation.
+
+Redis pub/sub is a realtime transport, not durable command state. Clients
+already treat these events as thin invalidations and reconcile by reading the
+PostgreSQL-backed API.
+
+## Decision
+
+- Durable board, document, and calendar mutations insert a fully formed event
+ into `realtime_outbox` in the same PostgreSQL transaction.
+- A per-process worker claims rows with `FOR UPDATE SKIP LOCKED` and a lease,
+ then publishes through the bounded Redis command client after commit.
+- Delivery is at-least-once. Every payload carries a stable `deliveryId`;
+ current consumers are naturally idempotent because they refetch or replace
+ entities by id.
+- Failed delivery uses exponential backoff. Invalidations are discarded after
+ 12 attempts or 24 hours; clients recover on their next pull. Discards are
+ logged as errors rather than blocking writes forever.
+- Board, document, and calendar create commands accept an actor-scoped
+ `requestId`. A payload hash rejects accidental key reuse, while a retry of
+ identical input returns the original entity.
+
+## Consequences
+
+- PostgreSQL commit is the only success boundary for these commands; Redis
+ latency and outages no longer change the response.
+- Multiple server replicas can drain concurrently without publishing the same
+ claimed row under normal operation. A crash after publish but before the
+ acknowledgement may redeliver once, which consumers must tolerate.
+- Operators can inspect pending rows, attempts, age, and `last_error` directly
+ in `realtime_outbox`. A growing pending queue means realtime is degraded, not
+ that durable writes are failing.
+- The outbox adds a small PostgreSQL write per mutation. Bounded cleanup keeps
+ successful rows for 24 hours and discarded rows for seven days, providing an
+ operational window without allowing terminal history to grow indefinitely.
+
+## Alternatives considered
+
+- **Await Redis after commit:** rejected because it reports ambiguous command
+ outcomes and cannot close the commit/publish crash gap.
+- **Fire-and-forget Redis only:** rejected because it removes request latency
+ but silently loses invalidations during outages.
+- **A dedicated broker:** deferred; the current invalidation volume does not
+ justify another source of truth, and PostgreSQL already owns the transaction.
diff --git a/server/src/__integration__/_helpers.ts b/server/src/__integration__/_helpers.ts
index f706dc2..9c497ae 100644
--- a/server/src/__integration__/_helpers.ts
+++ b/server/src/__integration__/_helpers.ts
@@ -31,6 +31,7 @@ export function ensureSchemaOnce(): Promise<void> {
* constraints; CASCADE on the parents handles it but listing explicitly
* keeps the intent visible + lets us spot-check leakage. */
const TABLES_TO_WIPE: readonly string[] = [
+ 'realtime_outbox',
'shipping_events',
'shipping_regressions',
'shipping_friction_reports',
@@ -197,6 +198,10 @@ export async function teardownAll(server?: import('node:http').Server): Promise<
if (server && server.listening) {
await new Promise<void>((resolve) => server.close(() => resolve()))
}
+ try {
+ const { stopRealtimeOutboxWorker } = await import('../realtime-outbox.js')
+ stopRealtimeOutboxWorker()
+ } catch { /* ignore */ }
// Pool + redis are module-level singletons; ending them is fine because
// the process is about to exit anyway. Catch swallows reentrant-end
// errors when multiple test files share the singleton.
diff --git a/server/src/__integration__/realtime-outbox.test.ts b/server/src/__integration__/realtime-outbox.test.ts
new file mode 100644
index 0000000..4b1c1f6
--- /dev/null
+++ b/server/src/__integration__/realtime-outbox.test.ts
@@ -0,0 +1,156 @@
+import { createServer, type Server } from 'node:http'
+import { after, before, beforeEach, test } from 'node:test'
+import assert from 'node:assert/strict'
+import { pool } from '../db/pool.js'
+import { drainRealtimeOutbox } from '../realtime-outbox.js'
+import type { BroadcastEvent } from '../redis.js'
+import {
+ buildApiTestApp, ensureSchemaOnce, resetAllTables,
+ seedUserMembership, teardownAll,
+} from './_helpers.js'
+
+const COMPANY_ID = 'co-outbox'
+const USER_ID = 'u-outbox'
+let server: Server
+let baseUrl = ''
+
+before(async () => {
+ await ensureSchemaOnce()
+ const app = await buildApiTestApp(USER_ID)
+ await new Promise<void>((resolve) => {
+ server = createServer(app).listen(0, '127.0.0.1', () => {
+ const address = server.address()
+ if (!address || typeof address === 'string') throw new Error('test server did not bind')
+ baseUrl = `http://127.0.0.1:${address.port}`
+ resolve()
+ })
+ })
+})
+
+beforeEach(async () => {
+ await resetAllTables()
+ await pool.query(
+ `INSERT INTO companies (id, name, slug, owner_user_id)
+ VALUES ($1, 'Outbox Co', 'outbox-co', $2)`,
+ [COMPANY_ID, USER_ID],
+ )
+ await seedUserMembership(USER_ID, COMPANY_ID)
+})
+
+after(async () => { await teardownAll(server) })
+
+async function post(path: string, body: Record<string, unknown>): Promise<Response> {
+ return fetch(`${baseUrl}/api${path}`, {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify(body),
+ })
+}
+
+test('[integration] a Redis outage cannot make a committed board create ambiguous', async () => {
+ const startedAt = Date.now()
+ const response = await post('/boards', {
+ title: 'Outbox board',
+ requestId: 'outbox-board-request-1',
+ })
+ assert.equal(response.status, 201)
+ assert.ok(Date.now() - startedAt < 2_000, 'request should not wait on Redis')
+ const created = await response.json() as { id: string; replayed: boolean }
+ assert.equal(created.replayed, false)
+
+ const failed = await drainRealtimeOutbox({
+ publishFn: async () => { throw new Error('redis unavailable') },
+ })
+ assert.deepEqual(failed, { claimed: 1, published: 0, failed: 1, discarded: 0 })
+
+ const durable = await pool.query<{ boards: number; pending: number; attempts: number }>(
+ `SELECT
+ (SELECT COUNT(*)::int FROM boards WHERE id = $1) AS boards,
+ (SELECT COUNT(*)::int FROM realtime_outbox WHERE published_at IS NULL AND discarded_at IS NULL) AS pending,
+ (SELECT attempts FROM realtime_outbox LIMIT 1) AS attempts`,
+ [created.id],
+ )
+ assert.deepEqual(durable.rows[0], { boards: 1, pending: 1, attempts: 1 })
+
+ await pool.query(`UPDATE realtime_outbox SET available_at = NOW()`)
+ const delivered: Array<{ channel: string; event: BroadcastEvent }> = []
+ const recovered = await drainRealtimeOutbox({
+ publishFn: async (channel, event) => { delivered.push({ channel, event }) },
+ })
+ assert.deepEqual(recovered, { claimed: 1, published: 1, failed: 0, discarded: 0 })
+ assert.equal(delivered.length, 1)
+ assert.equal(delivered[0].event.type, 'board.changed')
+ assert.ok(delivered[0].event.deliveryId)
+
+ const done = await pool.query<{ published: boolean }>(
+ `SELECT published_at IS NOT NULL AS published FROM realtime_outbox`,
+ )
+ assert.equal(done.rows[0]?.published, true)
+})
+
+test('[integration] concurrent board retries create one entity and one outbox event', async () => {
+ const send = () => post('/boards', {
+ title: 'Exactly once board',
+ description: 'same normalized input',
+ requestId: 'outbox-board-request-2',
+ })
+ const [a, b] = await Promise.all([send(), send()])
+ assert.deepEqual([a.status, b.status].sort(), [200, 201])
+ const [first, second] = await Promise.all([
+ a.json() as Promise<{ id: string; replayed: boolean }>,
+ b.json() as Promise<{ id: string; replayed: boolean }>,
+ ])
+ assert.equal(first.id, second.id)
+ assert.deepEqual([first.replayed, second.replayed].sort(), [false, true])
+
+ const counts = await pool.query<{ entities: number; events: number }>(
+ `SELECT
+ (SELECT COUNT(*)::int FROM boards WHERE creation_request_id = $1) AS entities,
+ (SELECT COUNT(*)::int FROM realtime_outbox) AS events`,
+ ['outbox-board-request-2'],
+ )
+ assert.deepEqual(counts.rows[0], { entities: 1, events: 1 })
+
+ const conflict = await post('/boards', {
+ title: 'Different input',
+ requestId: 'outbox-board-request-2',
+ })
+ assert.equal(conflict.status, 409)
+ assert.match((await conflict.json() as { error: string }).error, /different input/)
+})
+
+test('[integration] document and calendar creates replay without duplicate rows or events', async () => {
+ const cases = [
+ {
+ path: '/documents',
+ requestId: 'outbox-document-request-1',
+ body: { title: 'Retry-safe doc' },
+ table: 'documents',
+ },
+ {
+ path: '/calendar/events',
+ requestId: 'outbox-calendar-request-1',
+ body: { title: 'Retry-safe event', startAt: '2030-01-02T03:04:05.000Z' },
+ table: 'calendar_events',
+ },
+ ] as const
+
+ for (const item of cases) {
+ const first = await post(item.path, { ...item.body, requestId: item.requestId })
+ const replay = await post(item.path, { ...item.body, requestId: item.requestId })
+ assert.equal(first.status, 201)
+ assert.equal(replay.status, 200)
+ assert.equal((await replay.json() as { replayed: boolean }).replayed, true)
+
+ const rows = await pool.query<{ count: number }>(
+ `SELECT COUNT(*)::int AS count FROM ${item.table} WHERE creation_request_id = $1`,
+ [item.requestId],
+ )
+ assert.equal(rows.rows[0]?.count, 1)
+ }
+
+ const events = await pool.query<{ count: number }>(
+ `SELECT COUNT(*)::int AS count FROM realtime_outbox`,
+ )
+ assert.equal(events.rows[0]?.count, 2)
+})
diff --git a/server/src/__integration__/runtime-aux-authorization.test.ts b/server/src/__integration__/runtime-aux-authorization.test.ts
index 2fcbb25..bbfd6a9 100644
--- a/server/src/__integration__/runtime-aux-authorization.test.ts
+++ b/server/src/__integration__/runtime-aux-authorization.test.ts
@@ -24,6 +24,7 @@ import { pool } from '../db/pool.js'
import { readDocumentText, subscribe } from '../documents/rooms.js'
import { env } from '../env.js'
import { CH_DOCS, redis } from '../redis.js'
+import { startRealtimeOutboxWorker, stopRealtimeOutboxWorker } from '../realtime-outbox.js'
import {
ensureSchemaOnce, resetAllTables, seedCompanyWithAgent, teardownAll,
} from './_helpers.js'
@@ -411,6 +412,7 @@ test('[integration] CLI document change events are published only after commit',
const subscriber = redis.duplicate({ enableOfflineQueue: true, maxRetriesPerRequest: null })
await observer.connect()
await subscriber.subscribe(CH_DOCS)
+ startRealtimeOutboxWorker()
type ChangedEvent = {
type: 'doc.changed'
@@ -479,6 +481,7 @@ test('[integration] CLI document change events are published only after commit',
assert.equal(deleted.ok, true, deleted.text)
await deleteObserved
} finally {
+ stopRealtimeOutboxWorker()
await subscriber.unsubscribe(CH_DOCS).catch(() => {})
await subscriber.quit().catch(() => {})
await observer.end().catch(() => {})
diff --git a/server/src/agents/cli.ts b/server/src/agents/cli.ts
index d865b6b..dc4af9a 100644
--- a/server/src/agents/cli.ts
+++ b/server/src/agents/cli.ts
@@ -24,6 +24,19 @@ import {
} from './memory-scope.js'
import { memoryMetaForWrite, resolveMemoryWriteSource } from './memory-write.js'
import { wakeKanbanAgents } from './kanban-wake.js'
+import {
+ enqueueBroadcast,
+ nudgeRealtimeOutbox,
+ withOutboxTransaction,
+} from '../realtime-outbox.js'
+import {
+ CH_BOARDS,
+ CH_CALENDAR_EVENTS,
+ CH_CONVO_UPDATED,
+ CH_DOCS,
+ CH_MESSAGE_NEW,
+ CH_STATUS,
+} from '../redis.js'
// Every CLI result flows through ok()/err(), so scrubbing lone UTF-16 surrogates
// here means CLI output (read by agents as tool results) can never carry a split
@@ -2237,7 +2250,34 @@ async function cmdReply(parsed: ParsedArgs): Promise<CliResult> {
ON CONFLICT (user_id, conversation_id) DO UPDATE SET last_read_at = NOW()`,
[me, convoId],
)
+ await enqueueBroadcast(txClient, CH_MESSAGE_NEW, {
+ type: 'message.new',
+ conversationId: convoId,
+ companyId,
+ message: {
+ id: messageId,
+ conversationId: convoId,
+ authorId: me,
+ kind: 'text',
+ body: finalBody,
+ sequence,
+ at: new Date().toISOString(),
+ attachment: attachment ?? undefined,
+ quotedMessageId: resolvedQuotedId ?? undefined,
+ quoted: quotedSummary
+ ? {
+ id: quotedSummary.id,
+ authorId: quotedSummary.authorId,
+ authorName: quotedSummary.authorName,
+ kind: 'text',
+ body: quotedSummary.body,
+ sequence: quotedSummary.sequence,
+ }
+ : undefined,
+ },
+ })
await txClient.query('COMMIT')
+ nudgeRealtimeOutbox()
} catch (e) {
await txClient.query('ROLLBACK').catch(() => { /* already failed */ })
throw e
@@ -2264,29 +2304,6 @@ async function cmdReply(parsed: ParsedArgs): Promise<CliResult> {
// worst case is the counter TTLs naturally in 45min.
void (await import('./agenda.js')).resetStallNudgeDeclines(convoId)
- // Broadcast — frontend, scheduler, etc. all listen on CH_MESSAGE_NEW
- const { CH_MESSAGE_NEW, publish } = await import('../redis.js')
- await publish(CH_MESSAGE_NEW, {
- type: 'message.new',
- conversationId: convoId,
- companyId,
- message: {
- id: messageId, conversationId: convoId, authorId: me,
- kind: 'text', body: finalBody, sequence, at: new Date().toISOString(),
- attachment: attachment ?? undefined,
- quotedMessageId: resolvedQuotedId ?? undefined,
- quoted: quotedSummary ? {
- id: quotedSummary.id,
- authorId: quotedSummary.authorId,
- authorName: quotedSummary.authorName,
- kind: 'text',
- body: quotedSummary.body,
- sequence: quotedSummary.sequence,
- } : undefined,
- },
- }).catch((error) => {
- console.warn(`[reply] durable message ${messageId} committed but publish failed`, error)
- })
const attachmentNote = attachment
? ` · attached ${attachment.kind} "${attachment.name}"`
: ''
@@ -3657,19 +3674,20 @@ async function setAgentAvatarFromUrl(args: {
const key = `avatars/avatar-${args.agentId}-${randomUUID().slice(0, 8)}.${ext}`
const url = await storage.put(key, buf, mime)
- await pool.query(
- `UPDATE participants SET avatar_url = $2 WHERE id = $1 AND company_id = $3`,
- [args.agentId, url, args.tenant],
- )
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE participants SET avatar_url = $2 WHERE id = $1 AND company_id = $3`,
+ [args.agentId, url, args.tenant],
+ )
+ await enqueueBroadcast(client, CH_STATUS, {
+ type: 'participants.avatar',
+ participantId: args.agentId,
+ avatarUrl: url,
+ companyId: args.tenant,
+ })
+ })
const { invalidatePersonaCache } = await import('./personas.js')
invalidatePersonaCache(args.agentId)
- const { CH_STATUS, publish } = await import('../redis.js')
- await publish(CH_STATUS, {
- type: 'participants.avatar',
- participantId: args.agentId,
- avatarUrl: url,
- companyId: args.tenant,
- })
return { url }
}
@@ -3750,22 +3768,26 @@ async function cmdTopicSet(parsed: ParsedArgs): Promise<CliResult> {
participantId: me,
companyId: preflight[0].company_id,
conversationId: convoId,
- run: async (client) => client.query<{ company_id: string }>(
- `UPDATE conversations SET topic = $2, updated_at = NOW()
- WHERE id = $1 AND company_id = $3
- RETURNING company_id`,
- [convoId, topic, preflight[0].company_id],
- ),
+ run: async (client) => {
+ const result = await client.query<{ company_id: string }>(
+ `UPDATE conversations SET topic = $2, updated_at = NOW()
+ WHERE id = $1 AND company_id = $3
+ RETURNING company_id`,
+ [convoId, topic, preflight[0].company_id],
+ )
+ if (result.rows[0]) {
+ await enqueueBroadcast(client, CH_CONVO_UPDATED, {
+ type: 'conversation.updated',
+ conversationId: convoId,
+ companyId: result.rows[0].company_id,
+ patch: { topic },
+ })
+ }
+ return result
+ },
})
const companyId = updated?.rows[0]?.company_id
if (!companyId) return err(`conversation ${convoId} not found or no longer authorized`)
- const { CH_CONVO_UPDATED, publish } = await import('../redis.js')
- await publish(CH_CONVO_UPDATED, {
- type: 'conversation.updated',
- conversationId: convoId,
- companyId,
- patch: { topic },
- })
return ok(topic ? `topic set: "${topic}"` : '(topic cleared)', [{
event: 'conversation.topic_updated',
command: 'topic-set',
@@ -3830,22 +3852,26 @@ async function cmdRename(parsed: ParsedArgs): Promise<CliResult> {
participantId: me,
companyId: rows[0].company_id,
conversationId: convoId,
- run: async (client) => client.query(
- `UPDATE conversations
- SET title = $2, updated_at = NOW()
- WHERE id = $1 AND company_id = $3
- AND kind = 'group' AND title = $4`,
- [convoId, title, rows[0].company_id, currentTitle],
- ),
+ run: async (client) => {
+ const result = await client.query(
+ `UPDATE conversations
+ SET title = $2, updated_at = NOW()
+ WHERE id = $1 AND company_id = $3
+ AND kind = 'group' AND title = $4`,
+ [convoId, title, rows[0].company_id, currentTitle],
+ )
+ if (result.rowCount) {
+ await enqueueBroadcast(client, CH_CONVO_UPDATED, {
+ type: 'conversation.updated',
+ conversationId: convoId,
+ companyId: rows[0].company_id,
+ patch: { title },
+ })
+ }
+ return result
+ },
})
if (!updated?.rowCount) return err(`conversation ${convoId} changed or is no longer authorized; rename cancelled`)
- const { CH_CONVO_UPDATED, publish } = await import('../redis.js')
- await publish(CH_CONVO_UPDATED, {
- type: 'conversation.updated',
- conversationId: convoId,
- companyId: rows[0].company_id,
- patch: { title },
- })
return ok(`renamed to "${title}" (${convoId})`, [{
event: 'conversation.renamed',
command: 'rename',
@@ -4371,10 +4397,6 @@ async function cmdTasks(parsed: ParsedArgs): Promise<CliResult> {
* tasks/email above.
*/
-/** Best-effort WS broadcast for a calendar row change initiated by the
- * agent CLI. Mirrors the REST `publishCalendarChange` helper in
- * router.ts so the desktop client patches its Calendar view in real
- * time whether the change came from a human (HTTP) or an agent (CLI). */
/** Visibility predicate for the agent CLI. Mirrors the REST helper but
* with one simplification: agents can't be company owners, so the
* owner-override branch never applies — the predicate collapses to the
@@ -4384,14 +4406,14 @@ function cliCalendarVisibilityClause(meIdx: number): string {
return `(is_private = false OR created_by = $${meIdx} OR assignee_id = $${meIdx})`
}
-async function publishCalendarCli(args: {
+/** Queue the calendar invalidation alongside the CLI mutation. */
+async function enqueueCalendarCli(db: PoolClient, args: {
companyId: string
kind: 'event.created' | 'event.updated' | 'event.deleted' | 'event.dispatched'
eventId: string
actorId: string
}): Promise<void> {
- const { CH_CALENDAR_EVENTS, publish } = await import('../redis.js')
- await publish(CH_CALENDAR_EVENTS, {
+ await enqueueBroadcast(db, CH_CALENDAR_EVENTS, {
type: 'calendar.changed',
companyId: args.companyId,
kind: args.kind,
@@ -4545,17 +4567,19 @@ async function cmdCalendar(parsed: ParsedArgs): Promise<CliResult> {
}
}
const id = `ce-${randomUUID()}`
- await pool.query(
- `INSERT INTO calendar_events
- (id, company_id, created_by, kind, title, assignee_id,
- target_conversation_id, agent_prompt, start_at, recurrence,
- reminder_minutes_before, reminder_channel, status, is_private)
- VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10::jsonb,$11,$12,'active',$13)`,
- [id, companyId, me, kind, title, assigneeId, targetConvo, agentPrompt, start,
- recurrence ? JSON.stringify(recurrence) : null,
- reminderMinutes, reminderChannel, isPrivate],
- )
- await publishCalendarCli({ companyId, kind: 'event.created', eventId: id, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `INSERT INTO calendar_events
+ (id, company_id, created_by, kind, title, assignee_id,
+ target_conversation_id, agent_prompt, start_at, recurrence,
+ reminder_minutes_before, reminder_channel, status, is_private)
+ VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10::jsonb,$11,$12,'active',$13)`,
+ [id, companyId, me, kind, title, assigneeId, targetConvo, agentPrompt, start,
+ recurrence ? JSON.stringify(recurrence) : null,
+ reminderMinutes, reminderChannel, isPrivate],
+ )
+ await enqueueCalendarCli(client, { companyId, kind: 'event.created', eventId: id, actorId: me })
+ })
return ok(`scheduled ${id}: "${title}" at ${start.toISOString()}${recurrence ? ` · every ${recurrence.interval} ${recurrence.freq}` : ''}${assigneeId ? ` → @${assigneeId}` : ''}${reminderMinutes != null ? ` · remind ${reminderMinutes}m before (${reminderChannel})` : ''}${isPrivate ? ' · 🔒 private' : ''}`, [{
event: 'calendar.event_created',
command: 'calendar create',
@@ -4684,18 +4708,23 @@ async function cmdCalendar(parsed: ParsedArgs): Promise<CliResult> {
if (sets.length === 0) return err('nothing to update — pass at least one calendar field flag')
sets.push('updated_at = NOW()')
params.push(id, companyId)
- const { rows } = await pool.query<{
- id: string; title: string; kind: string; status: string;
- assignee_id: string | null; target_conversation_id: string | null; start_at: Date
- }>(
- `UPDATE calendar_events SET ${sets.join(', ')}
- WHERE id = $${params.length - 1} AND company_id = $${params.length}
- RETURNING id, title, kind, status, assignee_id, target_conversation_id, start_at`,
- params,
- )
+ const rows = await withOutboxTransaction(async (client) => {
+ const updated = await client.query<{
+ id: string; title: string; kind: string; status: string;
+ assignee_id: string | null; target_conversation_id: string | null; start_at: Date
+ }>(
+ `UPDATE calendar_events SET ${sets.join(', ')}
+ WHERE id = $${params.length - 1} AND company_id = $${params.length}
+ RETURNING id, title, kind, status, assignee_id, target_conversation_id, start_at`,
+ params,
+ )
+ if (updated.rows[0]) {
+ await enqueueCalendarCli(client, { companyId, kind: 'event.updated', eventId: id, actorId: me })
+ }
+ return updated.rows
+ })
const row = rows[0]
if (!row) return err(`no event ${id}`)
- await publishCalendarCli({ companyId, kind: 'event.updated', eventId: id, actorId: me })
return ok(`updated ${id}: "${row.title}" at ${row.start_at.toISOString()} (${row.status})`, [{
event: 'calendar.event_updated',
command: `calendar ${op}`,
@@ -4730,7 +4759,10 @@ async function cmdCalendar(parsed: ParsedArgs): Promise<CliResult> {
if (!rows[0]) return err(`no event ${id}`)
const { dispatchEvent } = await import('../calendar.js')
const result = await dispatchEvent(rows[0] as import('../calendar.js').CalendarEventRow, new Date())
- await publishCalendarCli({ companyId, kind: 'event.dispatched', eventId: id, actorId: me })
+ await withOutboxTransaction((client) => enqueueCalendarCli(
+ client,
+ { companyId, kind: 'event.dispatched', eventId: id, actorId: me },
+ ))
return ok(`dispatched ${id}: ${JSON.stringify(result)}`, [{
event: 'calendar.event_dispatched',
command: 'calendar run-now',
@@ -4775,24 +4807,29 @@ async function cmdCalendar(parsed: ParsedArgs): Promise<CliResult> {
// to "no event found" regardless of whether the row is missing or
// privacy-filtered, so we don't leak existence to non-authorized
// callers.
- const r = await pool.query(
- op === 'delete'
- ? `DELETE FROM calendar_events
- WHERE id = $1 AND company_id = $2 AND ${cliCalendarVisibilityClause(3)}`
- : `UPDATE calendar_events SET status = 'cancelled', updated_at = NOW()
- WHERE id = $1 AND company_id = $2 AND ${cliCalendarVisibilityClause(3)}`,
- [id, companyId, me],
- )
+ const r = await withOutboxTransaction(async (client) => {
+ const changed = await client.query(
+ op === 'delete'
+ ? `DELETE FROM calendar_events
+ WHERE id = $1 AND company_id = $2 AND ${cliCalendarVisibilityClause(3)}`
+ : `UPDATE calendar_events SET status = 'cancelled', updated_at = NOW()
+ WHERE id = $1 AND company_id = $2 AND ${cliCalendarVisibilityClause(3)}`,
+ [id, companyId, me],
+ )
+ if ((changed.rowCount ?? 0) > 0) {
+ await enqueueCalendarCli(client, {
+ companyId,
+ kind: op === 'delete' ? 'event.deleted' : 'event.updated',
+ eventId: id,
+ actorId: me,
+ })
+ }
+ return changed
+ })
if ((r.rowCount ?? 0) === 0) return err(`no event ${id}`)
// `cancel` flips status → updated; `delete` drops the row → deleted.
// Clients listening on calendar.changed will refetch (or drop the row
// from local state) accordingly.
- await publishCalendarCli({
- companyId,
- kind: op === 'delete' ? 'event.deleted' : 'event.updated',
- eventId: id,
- actorId: me,
- })
return ok(`${op === 'delete' ? 'deleted' : 'cancelled'} ${id}`, [{
event: op === 'delete' ? 'calendar.event_deleted' : 'calendar.event_cancelled',
command: `calendar ${op}`,
@@ -4810,7 +4847,7 @@ async function cmdCalendar(parsed: ParsedArgs): Promise<CliResult> {
*
* Same shape as the rest of this file: the agent operates as `me` (the
* --as participant) inside that participant's tenant. All inserts /
- * updates publish on the `boards` channel so the desktop client + every
+ * updates enqueue on the `boards` channel so the desktop client + every
* other connected member sees the change in real time. */
type KanbanMentionTarget = { id: string; name: string }
@@ -4869,7 +4906,7 @@ async function cliParseMentions(companyId: string, text: string): Promise<string
return cliParseMentionTargets(text, rows)
}
-async function publishBoardCli(args: {
+async function enqueueBoardCli(db: PoolClient, args: {
companyId: string
kind:
| 'board.created' | 'board.updated' | 'board.deleted'
@@ -4883,8 +4920,7 @@ async function publishBoardCli(args: {
mentions?: string[]
actorId: string
}): Promise<void> {
- const { CH_BOARDS, publish } = await import('../redis.js')
- await publish(CH_BOARDS, {
+ await enqueueBroadcast(db, CH_BOARDS, {
type: 'board.changed',
companyId: args.companyId,
kind: args.kind,
@@ -4974,9 +5010,7 @@ async function cmdBoard(parsed: ParsedArgs): Promise<CliResult> {
const description = typeof parsed.flags.description === 'string'
? unescapeChat(parsed.flags.description).slice(0, 4000) : null
const id = `board-${randomUUID().slice(0, 12)}`
- const client = await pool.connect()
- try {
- await client.query('BEGIN')
+ await withOutboxTransaction(async (client) => {
await client.query(
`INSERT INTO boards (id, company_id, title, description, created_by) VALUES ($1, $2, $3, $4, $5)`,
[id, companyId, title.slice(0, 200), description, me],
@@ -4990,14 +5024,8 @@ async function cmdBoard(parsed: ParsedArgs): Promise<CliResult> {
[`col-${randomUUID().slice(0, 12)}`, id, seeds[i][0], (i + 1) * 1000, seeds[i][1]],
)
}
- await client.query('COMMIT')
- } catch (e) {
- await client.query('ROLLBACK').catch(() => { /* swallow */ })
- throw e
- } finally {
- client.release()
- }
- await publishBoardCli({ companyId, kind: 'board.created', boardId: id, actorId: me })
+ await enqueueBoardCli(client, { companyId, kind: 'board.created', boardId: id, actorId: me })
+ })
return ok(`created board ${id}: ${title}`, [{
event: 'kanban.board_created',
command: 'kanban create',
@@ -5034,14 +5062,19 @@ async function cmdBoard(parsed: ParsedArgs): Promise<CliResult> {
}
if (sets.length === 0) return err('nothing to update — pass --title or --description')
params.push(boardId, companyId)
- const { rows } = await pool.query<{ title: string; description: string | null }>(
- `UPDATE boards SET ${sets.join(', ')}, updated_at = NOW()
- WHERE id = $${params.length - 1} AND company_id = $${params.length}
- RETURNING title, description`,
- params,
- )
+ const rows = await withOutboxTransaction(async (client) => {
+ const updated = await client.query<{ title: string; description: string | null }>(
+ `UPDATE boards SET ${sets.join(', ')}, updated_at = NOW()
+ WHERE id = $${params.length - 1} AND company_id = $${params.length}
+ RETURNING title, description`,
+ params,
+ )
+ if (updated.rows[0]) {
+ await enqueueBoardCli(client, { companyId, kind: 'board.updated', boardId, actorId: me })
+ }
+ return updated.rows
+ })
if (rows.length === 0) return err(`board ${boardId} not found`)
- await publishBoardCli({ companyId, kind: 'board.updated', boardId, actorId: me })
return ok(`updated board ${boardId}: ${rows[0].title}`, [{
event: 'kanban.board_updated',
command: `kanban ${op}`,
@@ -5082,11 +5115,13 @@ async function cmdBoard(parsed: ParsedArgs): Promise<CliResult> {
)
const position = (Number(posRows[0]?.max ?? 0)) + 1000
const id = `col-${randomUUID().slice(0, 12)}`
- await pool.query(
- `INSERT INTO board_columns (id, board_id, title, position) VALUES ($1, $2, $3, $4)`,
- [id, boardId, title.slice(0, 100), position],
- )
- await publishBoardCli({ companyId, kind: 'column.created', boardId, columnId: id, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `INSERT INTO board_columns (id, board_id, title, position) VALUES ($1, $2, $3, $4)`,
+ [id, boardId, title.slice(0, 100), position],
+ )
+ await enqueueBoardCli(client, { companyId, kind: 'column.created', boardId, columnId: id, actorId: me })
+ })
return ok(`added column ${id}: ${title}`, [{
event: 'kanban.column_created',
command: 'kanban add-column',
@@ -5126,14 +5161,19 @@ async function cmdBoard(parsed: ParsedArgs): Promise<CliResult> {
}
if (sets.length === 0) return err('nothing to update — pass --title or --position')
params.push(columnId, boardId)
- const { rows } = await pool.query<{ title: string; position: number }>(
- `UPDATE board_columns SET ${sets.join(', ')}
- WHERE id = $${params.length - 1} AND board_id = $${params.length}
- RETURNING title, position`,
- params,
- )
+ const rows = await withOutboxTransaction(async (client) => {
+ const updated = await client.query<{ title: string; position: number }>(
+ `UPDATE board_columns SET ${sets.join(', ')}
+ WHERE id = $${params.length - 1} AND board_id = $${params.length}
+ RETURNING title, position`,
+ params,
+ )
+ if (updated.rows[0]) {
+ await enqueueBoardCli(client, { companyId, kind: 'column.updated', boardId, columnId, actorId: me })
+ }
+ return updated.rows
+ })
if (rows.length === 0) return err(`column ${columnId} not in board ${boardId}`)
- await publishBoardCli({ companyId, kind: 'column.updated', boardId, columnId, actorId: me })
return ok(`updated column ${columnId}: ${rows[0].title}`, [{
event: 'kanban.column_updated',
command: `kanban ${op}`,
@@ -5156,12 +5196,17 @@ async function cmdBoard(parsed: ParsedArgs): Promise<CliResult> {
[boardId],
)
if (b.rows.length === 0 || b.rows[0].company_id !== companyId) return err(`board ${boardId} not found`)
- const r = await pool.query(
- `DELETE FROM board_columns WHERE id = $1 AND board_id = $2`,
- [columnId, boardId],
- )
+ const r = await withOutboxTransaction(async (client) => {
+ const deleted = await client.query(
+ `DELETE FROM board_columns WHERE id = $1 AND board_id = $2`,
+ [columnId, boardId],
+ )
+ if ((deleted.rowCount ?? 0) > 0) {
+ await enqueueBoardCli(client, { companyId, kind: 'column.deleted', boardId, columnId, actorId: me })
+ }
+ return deleted
+ })
if ((r.rowCount ?? 0) === 0) return err(`column ${columnId} not in board ${boardId}`)
- await publishBoardCli({ companyId, kind: 'column.deleted', boardId, columnId, actorId: me })
return ok(`deleted column ${columnId}`, [{
event: 'kanban.column_deleted',
command: `kanban ${op}`,
@@ -5176,12 +5221,17 @@ async function cmdBoard(parsed: ParsedArgs): Promise<CliResult> {
if (op === 'delete' || op === 'rm') {
const boardId = parsed.positional[1]
if (!boardId) return err('usage: kanban delete <board_id>')
- const r = await pool.query(
- `DELETE FROM boards WHERE id = $1 AND company_id = $2`,
- [boardId, companyId],
- )
+ const r = await withOutboxTransaction(async (client) => {
+ const deleted = await client.query(
+ `DELETE FROM boards WHERE id = $1 AND company_id = $2`,
+ [boardId, companyId],
+ )
+ if ((deleted.rowCount ?? 0) > 0) {
+ await enqueueBoardCli(client, { companyId, kind: 'board.deleted', boardId, actorId: me })
+ }
+ return deleted
+ })
if ((r.rowCount ?? 0) === 0) return err(`board ${boardId} not found`)
- await publishBoardCli({ companyId, kind: 'board.deleted', boardId, actorId: me })
return ok(`deleted board ${boardId}`, [{
event: 'kanban.board_deleted',
command: 'kanban delete',
@@ -5425,15 +5475,17 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
const position = (Number(posRows[0]?.max ?? 0)) + 1000
const mentions = await cliParseMentions(companyId, `${title}\n${description ?? ''}`)
const id = `card-${randomUUID().slice(0, 12)}`
- await pool.query(
- `INSERT INTO board_cards
- (id, board_id, column_id, title, description, position, assignee_id, mentions, created_by)
- VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9)`,
- [id, boardId, columnId, title.slice(0, 200), description, position, assignee, JSON.stringify(mentions), me],
- )
- await pool.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
- await publishBoardCli({
- companyId, kind: 'card.created', boardId, cardId: id, columnId, mentions, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `INSERT INTO board_cards
+ (id, board_id, column_id, title, description, position, assignee_id, mentions, created_by)
+ VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9)`,
+ [id, boardId, columnId, title.slice(0, 200), description, position, assignee, JSON.stringify(mentions), me],
+ )
+ await client.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
+ await enqueueBoardCli(client, {
+ companyId, kind: 'card.created', boardId, cardId: id, columnId, mentions, actorId: me,
+ })
})
void wakeKanbanAgents({
companyId, mentions, actorId: me,
@@ -5481,13 +5533,15 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
`SELECT MAX(position) AS max FROM board_cards WHERE column_id = $1`, [toCol],
)
const position = (Number(posRows[0]?.max ?? 0)) + 1000
- await pool.query(
- `UPDATE board_cards SET column_id = $1, position = $2, updated_at = NOW() WHERE id = $3`,
- [toCol, position, cardId],
- )
- await pool.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [home.boardId])
- await publishBoardCli({
- companyId, kind: 'card.moved', boardId: home.boardId, cardId, columnId: toCol, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE board_cards SET column_id = $1, position = $2, updated_at = NOW() WHERE id = $3`,
+ [toCol, position, cardId],
+ )
+ await client.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [home.boardId])
+ await enqueueBoardCli(client, {
+ companyId, kind: 'card.moved', boardId: home.boardId, cardId, columnId: toCol, actorId: me,
+ })
})
return ok(`moved card ${cardId} → ${toCol}`, [{
event: 'kanban.card_moved',
@@ -5509,12 +5563,14 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
const home = await resolveCardBoard(cardId)
if (!home) return err(`card ${cardId} not found`)
const assignee = (!who || who.toLowerCase() === 'null' || who === '-') ? null : who.trim()
- await pool.query(
- `UPDATE board_cards SET assignee_id = $1, updated_at = NOW() WHERE id = $2`,
- [assignee, cardId],
- )
- await publishBoardCli({
- companyId, kind: 'card.updated', boardId: home.boardId, cardId, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE board_cards SET assignee_id = $1, updated_at = NOW() WHERE id = $2`,
+ [assignee, cardId],
+ )
+ await enqueueBoardCli(client, {
+ companyId, kind: 'card.updated', boardId: home.boardId, cardId, actorId: me,
+ })
})
if (assignee && assignee !== me) {
void wakeKanbanAgents({
@@ -5548,15 +5604,36 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
if (!cardId) return err('usage: card claim <card_id>')
const home = await resolveCardBoard(cardId)
if (!home) return err(`card ${cardId} not found`)
- const claimed = await pool.query<{ id: string }>(
- `UPDATE board_cards SET assignee_id = $1, updated_at = NOW()
- WHERE id = $2
- AND (assignee_id IS NULL OR assignee_id = $1
- OR updated_at < NOW() - INTERVAL '20 minutes')
- RETURNING id`,
- [me, cardId],
- )
- if ((claimed.rowCount ?? 0) === 0) {
+ const claim = await withOutboxTransaction(async (client) => {
+ const claimed = await client.query<{ id: string }>(
+ `UPDATE board_cards SET assignee_id = $1, updated_at = NOW()
+ WHERE id = $2
+ AND (assignee_id IS NULL OR assignee_id = $1
+ OR updated_at < NOW() - INTERVAL '20 minutes')
+ RETURNING id`,
+ [me, cardId],
+ )
+ if ((claimed.rowCount ?? 0) === 0) return { claimed: false, advanceTo: null as string | null }
+ const cols = await client.query<{ id: string; position: number; kind: string | null }>(
+ `SELECT id, position, kind FROM board_columns WHERE board_id = $1`, [home.boardId],
+ )
+ const advanceTo = claimTargetColumn({ columns: cols.rows, currentColumnId: home.columnId })
+ if (advanceTo) {
+ await client.query(
+ `UPDATE board_cards SET column_id = $1, updated_at = NOW() WHERE id = $2`,
+ [advanceTo, cardId],
+ )
+ }
+ await enqueueBoardCli(client, {
+ companyId,
+ kind: advanceTo ? 'card.moved' : 'card.updated',
+ boardId: home.boardId,
+ cardId,
+ actorId: me,
+ })
+ return { claimed: true, advanceTo }
+ })
+ if (!claim.claimed) {
const cur = await pool.query<{ assignee_id: string | null }>(
`SELECT assignee_id FROM board_cards WHERE id = $1 LIMIT 1`, [cardId],
)
@@ -5568,17 +5645,7 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
// board could read Todo 2 / Doing 1 / Done 0 while the work was finished and
// delivered in chat (#69). Only ever forward, and only on a board whose
// columns declare what they mean; see claimTargetColumn.
- const cols = await pool.query<{ id: string; position: number; kind: string | null }>(
- `SELECT id, position, kind FROM board_columns WHERE board_id = $1`, [home.boardId],
- )
- const advanceTo = claimTargetColumn({ columns: cols.rows, currentColumnId: home.columnId })
- if (advanceTo) {
- await pool.query(
- `UPDATE board_cards SET column_id = $1, updated_at = NOW() WHERE id = $2`,
- [advanceTo, cardId],
- )
- }
- await publishBoardCli({ companyId, kind: advanceTo ? 'card.moved' : 'card.updated', boardId: home.boardId, cardId, actorId: me })
+ const advanceTo = claim.advanceTo
return ok(`claimed card ${cardId} — it's yours${advanceTo ? ' and moved to Doing' : ''}. Do the work, post progress with \`card comment\`, move it with \`card move\`, and release with \`card assign ${cardId} null\` (or move to a done column) when finished.`, [{
event: 'kanban.card_claimed',
command: 'card claim',
@@ -5615,12 +5682,14 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
const mentions = await cliParseMentions(companyId, `${nextTitle}\n${nextDesc ?? ''}`)
params.push(JSON.stringify(mentions)); sets.push(`mentions = $${params.length}::jsonb`)
params.push(cardId)
- await pool.query(
- `UPDATE board_cards SET ${sets.join(', ')}, updated_at = NOW() WHERE id = $${params.length}`,
- params,
- )
- await publishBoardCli({
- companyId, kind: 'card.updated', boardId: home.boardId, cardId, mentions, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE board_cards SET ${sets.join(', ')}, updated_at = NOW() WHERE id = $${params.length}`,
+ params,
+ )
+ await enqueueBoardCli(client, {
+ companyId, kind: 'card.updated', boardId: home.boardId, cardId, mentions, actorId: me,
+ })
})
void wakeKanbanAgents({
companyId, mentions, actorId: me,
@@ -5651,15 +5720,17 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
if (!home) return err(`card ${cardId} not found`)
const mentions = await cliParseMentions(companyId, body)
const id = `cmt-${randomUUID().slice(0, 12)}`
- await pool.query(
- `INSERT INTO board_card_comments (id, card_id, author_id, body, mentions)
- VALUES ($1, $2, $3, $4, $5::jsonb)`,
- [id, cardId, me, body.slice(0, 8000), JSON.stringify(mentions)],
- )
- await pool.query(`UPDATE board_cards SET updated_at = NOW() WHERE id = $1`, [cardId])
- await pool.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [home.boardId])
- await publishBoardCli({
- companyId, kind: 'comment.created', boardId: home.boardId, cardId, commentId: id, mentions, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `INSERT INTO board_card_comments (id, card_id, author_id, body, mentions)
+ VALUES ($1, $2, $3, $4, $5::jsonb)`,
+ [id, cardId, me, body.slice(0, 8000), JSON.stringify(mentions)],
+ )
+ await client.query(`UPDATE board_cards SET updated_at = NOW() WHERE id = $1`, [cardId])
+ await client.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [home.boardId])
+ await enqueueBoardCli(client, {
+ companyId, kind: 'comment.created', boardId: home.boardId, cardId, commentId: id, mentions, actorId: me,
+ })
})
void wakeKanbanAgents({
companyId, mentions, actorId: me,
@@ -5687,15 +5758,20 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
if (!cardId || !commentId) return err(`usage: card ${op} <card_id> <comment_id>`)
const home = await resolveCardBoard(cardId)
if (!home) return err(`card ${cardId} not found`)
- const r = await pool.query(
- `DELETE FROM board_card_comments
- WHERE id = $1 AND card_id = $2 AND author_id = $3`,
- [commentId, cardId, me],
- )
- if ((r.rowCount ?? 0) === 0) return err(`comment ${commentId} not found or not authored by ${me}`)
- await publishBoardCli({
- companyId, kind: 'comment.deleted', boardId: home.boardId, cardId, commentId, actorId: me,
+ const r = await withOutboxTransaction(async (client) => {
+ const deleted = await client.query(
+ `DELETE FROM board_card_comments
+ WHERE id = $1 AND card_id = $2 AND author_id = $3`,
+ [commentId, cardId, me],
+ )
+ if ((deleted.rowCount ?? 0) > 0) {
+ await enqueueBoardCli(client, {
+ companyId, kind: 'comment.deleted', boardId: home.boardId, cardId, commentId, actorId: me,
+ })
+ }
+ return deleted
})
+ if ((r.rowCount ?? 0) === 0) return err(`comment ${commentId} not found or not authored by ${me}`)
return ok(`deleted comment ${commentId}`, [{
event: 'kanban.comment_deleted',
command: `card ${op}`,
@@ -5713,9 +5789,11 @@ async function cmdCard(parsed: ParsedArgs): Promise<CliResult> {
if (!cardId) return err('usage: card delete <card_id>')
const home = await resolveCardBoard(cardId)
if (!home) return err(`card ${cardId} not found`)
- await pool.query(`DELETE FROM board_cards WHERE id = $1`, [cardId])
- await publishBoardCli({
- companyId, kind: 'card.deleted', boardId: home.boardId, cardId, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(`DELETE FROM board_cards WHERE id = $1`, [cardId])
+ await enqueueBoardCli(client, {
+ companyId, kind: 'card.deleted', boardId: home.boardId, cardId, actorId: me,
+ })
})
return ok(`deleted card ${cardId}`, [{
event: 'kanban.card_deleted',
@@ -5783,21 +5861,19 @@ function buildToolArgs(toolName: string, parsed: ParsedArgs): { argsJson: string
* happens automatically — the human's editor sees the agent's cursor
* + insertion live, as if a remote teammate just typed it.
*/
-async function publishDocChanged(
+async function enqueueDocChanged(
+ db: PoolClient,
companyId: string,
documentId: string,
kind: 'document.created' | 'document.updated' | 'document.deleted',
actorId: string,
): Promise<void> {
- const { CH_DOCS, publish } = await import('../redis.js')
- await publish(CH_DOCS, {
+ await enqueueBroadcast(db, CH_DOCS, {
type: 'doc.changed',
kind,
companyId,
documentId,
actorId,
- }).catch((error) => {
- console.warn(`[doc] durable ${kind} for ${documentId} committed but publish failed`, error)
})
}
@@ -5806,10 +5882,6 @@ async function cmdDoc(parsed: ParsedArgs): Promise<CliResult> {
const me = resolveAs(parsed)
const companyId = await agentCompany(me)
if (!companyId) return err(`unknown agent ${me} (no company)`)
- const pendingChanges: Array<{
- kind: 'document.created' | 'document.updated' | 'document.deleted'
- documentId: string
- }> = []
const finalizers: Array<() => Promise<void>> = []
let locked: CliResult | null
try {
@@ -5823,7 +5895,7 @@ async function cmdDoc(parsed: ParsedArgs): Promise<CliResult> {
me,
companyId,
client,
- (kind, documentId) => pendingChanges.push({ kind, documentId }),
+ (kind, documentId) => enqueueDocChanged(client, companyId, documentId, kind, me),
(finalizer) => finalizers.push(finalizer),
),
})
@@ -5835,12 +5907,6 @@ async function cmdDoc(parsed: ParsedArgs): Promise<CliResult> {
}
}
if (!locked) return err(`agent ${me} is no longer active in ${companyId}`)
- // `withActiveParticipantLock` commits before returning. Publish only now so
- // subscribers that immediately reload never observe pre-commit state, and a
- // failed transaction cannot emit a ghost document event.
- for (const change of pendingChanges) {
- await publishDocChanged(companyId, change.documentId, change.kind, me)
- }
return locked
}
@@ -5853,10 +5919,9 @@ async function cmdDocAsActiveAgent(
onChanged: (
kind: 'document.created' | 'document.updated' | 'document.deleted',
documentId: string,
- ) => void,
+ ) => Promise<void>,
onFinally: (finalizer: () => Promise<void>) => void,
): Promise<CliResult> {
-
if (op === 'ls' || op === 'list') {
const { rows } = await dbClient.query<{
id: string; title: string; created_by: string; updated_at: Date
@@ -5938,7 +6003,7 @@ async function cmdDocAsActiveAgent(
const { applyAgentEdit } = await import('../documents/rooms.js')
await applyAgentEdit(id, companyId, me, [{ kind: 'append', text: body }], dbClient)
}
- onChanged('document.created', id)
+ await onChanged('document.created', id)
return ok(`created document ${id}: ${title}`, [{
event: 'document.created',
command: 'doc create',
@@ -5980,7 +6045,7 @@ async function cmdDocAsActiveAgent(
if (rows.length === 0 || rows[0].company_id !== companyId) return err(`document ${docId} not found`)
const { applyAgentEdit } = await import('../documents/rooms.js')
await applyAgentEdit(docId, companyId, me, [{ kind: 'append', text }], dbClient)
- onChanged('document.updated', docId)
+ await onChanged('document.updated', docId)
return ok(`appended ${text.length} chars to ${docId}`, [{
event: 'document.updated',
command: 'doc append',
@@ -6004,7 +6069,7 @@ async function cmdDocAsActiveAgent(
if (rows.length === 0 || rows[0].company_id !== companyId) return err(`document ${docId} not found`)
const { applyAgentEdit } = await import('../documents/rooms.js')
await applyAgentEdit(docId, companyId, me, [{ kind: 'insertParagraph', at: 'start', text }], dbClient)
- onChanged('document.updated', docId)
+ await onChanged('document.updated', docId)
return ok(`prepended ${text.length} chars to ${docId}`, [{
event: 'document.updated',
command: 'doc prepend',
@@ -6077,7 +6142,7 @@ async function cmdDocAsActiveAgent(
const snippet = placement.anchorText.slice(0, 60)
return err(`anchor not found in ${docId}: "${snippet}". Re-read the doc and pick a snippet that uniquely identifies the target block — no image was inserted.`)
}
- onChanged('document.updated', docId)
+ await onChanged('document.updated', docId)
let where: string
if (isAnchoredImagePlacement(placement)) {
@@ -6125,7 +6190,7 @@ async function cmdDocAsActiveAgent(
if (result.imagesDeleted === 0) {
return err(`no images in ${docId} matched the criterion`)
}
- onChanged('document.updated', docId)
+ await onChanged('document.updated', docId)
return ok(`deleted ${result.imagesDeleted} image${result.imagesDeleted === 1 ? '' : 's'} from ${docId}`, [{
event: 'document.updated',
command: 'doc image-delete',
@@ -6150,7 +6215,7 @@ async function cmdDocAsActiveAgent(
const { applyAgentEdit } = await import('../documents/rooms.js')
const r = await applyAgentEdit(docId, companyId, me, [{ kind: 'replace', find, replace }], dbClient)
if (r.replaced === 0) return err(`text not found in ${docId}: ${JSON.stringify(find).slice(0, 80)}`)
- onChanged('document.updated', docId)
+ await onChanged('document.updated', docId)
return ok(`replaced ${r.replaced} occurrence in ${docId}`, [{
event: 'document.updated',
command: 'doc replace',
@@ -6176,7 +6241,7 @@ async function cmdDocAsActiveAgent(
const { applyAgentEdit } = await import('../documents/rooms.js')
const r = await applyAgentEdit(docId, companyId, me, [{ kind: 'replaceBlock', anchorText: anchor, text }], dbClient)
if (r.blocksReplaced === 0) return err(`no block containing ${JSON.stringify(anchor).slice(0, 80)} in ${docId}`)
- onChanged('document.updated', docId)
+ await onChanged('document.updated', docId)
return ok(`replaced 1 block in ${docId}`, [{
event: 'document.updated',
command: 'doc replace-block',
@@ -6199,7 +6264,7 @@ async function cmdDocAsActiveAgent(
[title.slice(0, 200), docId, companyId],
)
if (!r.rowCount) return err(`document ${docId} not found`)
- onChanged('document.updated', docId)
+ await onChanged('document.updated', docId)
return ok(`renamed ${docId} to "${title}"`, [{
event: 'document.updated',
command: 'doc rename',
@@ -6222,7 +6287,7 @@ async function cmdDocAsActiveAgent(
if (rows.length === 0) return err(`document ${docId} not found`)
if (rows[0].created_by !== me) return err(`only the creator can delete document ${docId}`)
await dbClient.query(`DELETE FROM documents WHERE id = $1`, [docId])
- onChanged('document.deleted', docId)
+ await onChanged('document.deleted', docId)
return ok(`deleted document ${docId}`, [{
event: 'document.deleted',
command: 'doc delete',
diff --git a/server/src/agents/private_chat.ts b/server/src/agents/private_chat.ts
index daddb41..fc39934 100644
--- a/server/src/agents/private_chat.ts
+++ b/server/src/agents/private_chat.ts
@@ -12,7 +12,8 @@
import { randomUUID } from 'node:crypto'
import type { PoolClient } from 'pg'
import { pool } from '../db/pool.js'
-import { CH_MESSAGE_NEW, publish } from '../redis.js'
+import { CH_MESSAGE_NEW } from '../redis.js'
+import { enqueueBroadcast, nudgeRealtimeOutbox } from '../realtime-outbox.js'
/** Find an existing direct conversation between two participants, or
* create one and return its id. Order-independent on members. */
@@ -203,7 +204,22 @@ export async function startPrivateChat(args: {
ON CONFLICT (user_id, conversation_id) DO UPDATE SET last_read_at = NOW()`,
[instigatorId, conversationId],
)
+ await enqueueBroadcast(client, CH_MESSAGE_NEW, {
+ type: 'message.new',
+ conversationId,
+ companyId,
+ message: {
+ id: messageId,
+ conversationId,
+ authorId: instigatorId,
+ kind: 'text',
+ body: opening,
+ sequence,
+ at: new Date().toISOString(),
+ },
+ })
await client.query('COMMIT')
+ nudgeRealtimeOutbox()
} catch (error) {
await client.query('ROLLBACK').catch(() => {})
throw error
@@ -211,20 +227,5 @@ export async function startPrivateChat(args: {
client.release()
}
- // Same channel as every other message. Agent recipients wake through the
- // mailbox scheduler; human recipients see the direct conversation normally.
- await publish(CH_MESSAGE_NEW, {
- type: 'message.new',
- conversationId,
- companyId,
- message: {
- id: messageId, conversationId, authorId: instigatorId,
- kind: 'text', body: opening, sequence,
- at: new Date().toISOString(),
- },
- }).catch((error) => {
- console.warn(`[private_chat] durable message ${messageId} committed but publish failed`, error)
- })
-
return { conversationId, messageId }
}
diff --git a/server/src/agents/runtime/inproc-client.ts b/server/src/agents/runtime/inproc-client.ts
index 3c9086a..f2b3d4a 100644
--- a/server/src/agents/runtime/inproc-client.ts
+++ b/server/src/agents/runtime/inproc-client.ts
@@ -18,6 +18,7 @@ import { createHash, randomUUID } from 'node:crypto'
import type { PoolClient } from 'pg'
import { pool } from '../../db/pool.js'
import { CH_MESSAGE_NEW, CH_TYPING, publish, redis } from '../../redis.js'
+import { enqueueBroadcast, nudgeRealtimeOutbox } from '../../realtime-outbox.js'
import { notifyAlert } from '../../alerting.js'
import { freshenAttachmentUrl, type StoredAttachment } from '../../storage.js'
@@ -720,7 +721,22 @@ export class InProcRuntimeClient implements AgentRuntimeClient {
[messageId, args.conversationId, args.agentId, body, sequence, companyId, clientId],
)
await client.query(`UPDATE conversations SET updated_at = NOW() WHERE id = $1`, [args.conversationId])
+ await enqueueBroadcast(client, CH_MESSAGE_NEW, {
+ type: 'message.new',
+ conversationId: args.conversationId,
+ companyId,
+ message: {
+ id: messageId,
+ conversationId: args.conversationId,
+ authorId: args.agentId,
+ kind: 'system',
+ body,
+ sequence,
+ at: new Date().toISOString(),
+ },
+ })
await client.query('COMMIT')
+ nudgeRealtimeOutbox()
posted = true
} catch (error) {
await client.query('ROLLBACK').catch(() => {})
@@ -729,18 +745,6 @@ export class InProcRuntimeClient implements AgentRuntimeClient {
client.release()
}
- await publish(CH_MESSAGE_NEW, {
- type: 'message.new',
- conversationId: args.conversationId,
- companyId,
- message: {
- id: messageId, conversationId: args.conversationId, authorId: args.agentId,
- kind: 'system', body, sequence,
- at: new Date().toISOString(),
- },
- }).catch((error) => {
- console.warn(`[runtime] durable notice ${messageId} committed but publish failed`, error)
- })
return { posted, authorized: true }
}
diff --git a/server/src/agents/tools.ts b/server/src/agents/tools.ts
index 2c7f8d6..273a9b6 100644
--- a/server/src/agents/tools.ts
+++ b/server/src/agents/tools.ts
@@ -21,6 +21,8 @@ import { randomUUID } from 'node:crypto'
import { env } from '../env.js'
import { getTrackedLlmClient } from './llm-ledger.js'
import { pool } from '../db/pool.js'
+import { enqueueBroadcast, nudgeRealtimeOutbox } from '../realtime-outbox.js'
+import { CH_REACTIONS } from '../redis.js'
import { tReadFile, tWriteFile, tEditFile } from './runtime/native-tools.js'
import type { FsNamespace } from './runtime/fs-namespace.js'
import {
@@ -259,7 +261,15 @@ async function tReact(args: Record<string, unknown>, agentId: string): Promise<T
[messageId],
)
agg = aggregate.rows
+ await enqueueBroadcast(client, CH_REACTIONS, {
+ type: 'message.reactions',
+ conversationId,
+ companyId,
+ messageId,
+ reactions: agg,
+ })
await client.query('COMMIT')
+ nudgeRealtimeOutbox()
} catch (error) {
await client.query('ROLLBACK').catch(() => {})
throw error
@@ -271,17 +281,6 @@ async function tReact(args: Record<string, unknown>, agentId: string): Promise<T
// acknowledgement for long work, and marking the request read before the
// turn completes can erase unfinished tasks from the agent's inbox. The turn
// runtime advances reads only after semantic completion is accepted.
- const { CH_REACTIONS, publish } = await import('../redis.js')
- await publish(CH_REACTIONS, {
- type: 'message.reactions',
- conversationId,
- companyId,
- messageId,
- reactions: agg,
- }).catch((error) => {
- console.warn(`[react] durable reaction on ${messageId} committed but publish failed`, error)
- })
-
return {
ok: true,
output: { messageId, emoji, action, reactions: agg },
diff --git a/server/src/api/router.ts b/server/src/api/router.ts
index 9e3e0e3..c38b834 100644
--- a/server/src/api/router.ts
+++ b/server/src/api/router.ts
@@ -5,7 +5,8 @@ import {
storageKeyFromPublicUrl, messageAttachmentStorageKey,
} from '../storage.js'
import { pool } from '../db/pool.js'
-import { CH_MESSAGE_NEW, CH_REACTIONS, CH_CONVO_UPDATED, CH_DOCS, CH_TYPING, CH_CALENDAR_EVENTS, publish } from '../redis.js'
+import { CH_MESSAGE_NEW, CH_REACTIONS, CH_CONVO_UPDATED, CH_DOCS, CH_TYPING, CH_CALENDAR_EVENTS, CH_BOARDS, CH_STATUS, publish } from '../redis.js'
+import { enqueueBroadcast, nudgeRealtimeOutbox, withOutboxTransaction } from '../realtime-outbox.js'
import { createPoll, castVote, closePoll, PollError } from '../polls.js'
import { env } from '../env.js'
import { publicBodyParserError } from '../body-parser-errors.js'
@@ -42,6 +43,10 @@ import {
import { attachComputerControlStream, deliverEngineDetect } from '../agents/computer/control-bus.js'
import { companyTier } from '../tier.js'
import { createShippingRouter } from './shipping-router.js'
+import {
+ findIdempotentCreate, IdempotencyConflictError,
+ parseRequestId, requestHash,
+} from '../idempotent-create.js'
/** Re-export so older imports (server/index.ts, agents/cli.ts) keep working
* after the storage abstraction moved this constant. */
@@ -132,6 +137,28 @@ class HttpError extends Error {
constructor(public status: number, message: string) { super(message) }
}
+function createRequestId(value: unknown): string | null {
+ try {
+ return parseRequestId(value)
+ } catch (error) {
+ throw new HttpError(400, error instanceof Error ? error.message : 'invalid requestId')
+ }
+}
+
+async function createReplay(
+ client: PoolClient,
+ args: Parameters<typeof findIdempotentCreate>[1],
+): Promise<{ id: string } | null> {
+ try {
+ return await findIdempotentCreate(client, args)
+ } catch (error) {
+ if (error instanceof IdempotencyConflictError) {
+ throw new HttpError(error.status, error.message)
+ }
+ throw error
+ }
+}
+
/** Throw 401 if the request has no valid session. Returns the user_id. */
function requireAuth(req: Request & AuthedRequest): string {
const id = req.authUserId
@@ -2765,21 +2792,20 @@ export async function generateAndPersistAvatar(args: {
const key = `avatars/avatar-${id}-${randomUUID().slice(0, 8)}.png`
const url = await storage.put(key, imageBuf, 'image/png')
- await pool.query(
- `UPDATE participants SET avatar_url = $2 WHERE id = $1 AND company_id = $3`,
- [id, url, tenant],
- )
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE participants SET avatar_url = $2 WHERE id = $1 AND company_id = $3`,
+ [id, url, tenant],
+ )
+ await enqueueBroadcast(client, CH_STATUS, {
+ type: 'participants.avatar',
+ participantId: id,
+ avatarUrl: url,
+ companyId: tenant,
+ })
+ })
const { invalidatePersonaCache } = await import('../agents/personas.js')
invalidatePersonaCache(id)
- // Notify connected clients that this agent's avatar changed so they
- // can pick it up without waiting for the 60s periodic refresh.
- const { CH_STATUS, publish } = await import('../redis.js')
- await publish(CH_STATUS, {
- type: 'participants.avatar',
- participantId: id,
- avatarUrl: url,
- companyId: tenant,
- })
return { url }
}
@@ -2977,14 +3003,14 @@ api.post('/conversations/:id/topic', async (req, res) => {
WHERE id = $1 AND company_id = $3`,
[id, topic, tenant],
)
+ await enqueueBroadcast(client, CH_CONVO_UPDATED, {
+ type: 'conversation.updated',
+ conversationId: id,
+ companyId: tenant,
+ patch: { topic },
+ })
},
})
- await publish(CH_CONVO_UPDATED, {
- type: 'conversation.updated',
- conversationId: id,
- companyId: tenant,
- patch: { topic },
- })
res.json({ ok: true, topic })
})
@@ -3006,14 +3032,14 @@ api.post('/conversations/:id/title', async (req, res) => {
WHERE id = $1 AND company_id = $3`,
[id, title, tenant],
)
+ await enqueueBroadcast(client, CH_CONVO_UPDATED, {
+ type: 'conversation.updated',
+ conversationId: id,
+ companyId: tenant,
+ patch: { title },
+ })
},
})
- await publish(CH_CONVO_UPDATED, {
- type: 'conversation.updated',
- conversationId: id,
- companyId: tenant,
- patch: { title },
- })
res.json({ ok: true, title })
})
@@ -3688,8 +3714,22 @@ api.post('/conversations/:id/messages', async (req, res) => {
`UPDATE conversations SET updated_at = NOW() WHERE id = $1 AND company_id = $2`,
[id, tenant],
)
+ await enqueueBroadcast(client, CH_MESSAGE_NEW, {
+ type: 'message.new',
+ conversationId: id,
+ companyId: tenant,
+ message: {
+ id: persisted.id, conversationId: id, authorId: me,
+ kind: 'text', body, sequence: persisted.sequence, at: new Date().toISOString(),
+ attachment: attachment ?? undefined,
+ quotedMessageId: resolvedQuotedId ?? undefined,
+ quoted: quotedSummary ?? undefined,
+ clientId: clientId ?? undefined,
+ },
+ })
}
await client.query('COMMIT')
+ if (insertedNew) nudgeRealtimeOutbox()
} catch (error) {
await client.query('ROLLBACK').catch(() => {})
throw error
@@ -3719,24 +3759,10 @@ api.post('/conversations/:id/messages', async (req, res) => {
return
}
- // Drop the message into the bus. The mailbox scheduler (subscribed to
- // CH_MESSAGE_NEW) wakes every agent member and lets each decide for itself
- // whether to reply, react, dm, or stay silent. The companyId tag lets
- // the WS bridge filter who sees it.
- await publish(CH_MESSAGE_NEW, {
- type: 'message.new',
- conversationId: id,
- companyId: tenant,
- message: {
- id: messageId, conversationId: id, authorId: me,
- kind: 'text', body, sequence: persistedSequence, at: new Date().toISOString(),
- attachment: attachment ?? undefined,
- quotedMessageId: resolvedQuotedId ?? undefined,
- quoted: quotedSummary ?? undefined,
- clientId: clientId ?? undefined,
- },
- })
- logDelivery('ws.published')
+ // Redis delivery is handled by the transactional outbox worker. The mailbox
+ // scheduler and WS bridge receive the same payload without participating in
+ // HTTP command completion.
+ logDelivery('ws.enqueued')
// Fan out APNs to recipients who aren't currently looking at the app
// (NotificationToasts handles those). Fire-and-forget — push delivery
@@ -4436,6 +4462,13 @@ api.post('/messages/:id/reactions', async (req, res) => {
GROUP BY emoji ORDER BY count DESC, emoji ASC`,
[id],
)
+ await enqueueBroadcast(client, CH_REACTIONS, {
+ type: 'message.reactions',
+ conversationId,
+ companyId: tenant,
+ messageId: id,
+ reactions,
+ })
return { wasRemoval, messageAuthorId: message[0].author_id, reactions }
},
})
@@ -4452,20 +4485,6 @@ api.post('/messages/:id/reactions', async (req, res) => {
})
}
- // Aggregate. No server-side `mine` flag: the same row is broadcast over
- // WS to every client in the tenant, and "is this mine" is per-recipient.
- // Renderer derives mine = users.includes(meId) — see fromApi /
- // applyEvent in src/stores/messages.ts.
- await publish(CH_REACTIONS, {
- type: 'message.reactions',
- conversationId,
- companyId: tenant,
- messageId: id,
- reactions: mutation.reactions,
- }).catch((error) => {
- console.warn(`[reactions] durable reaction on ${id} committed but publish failed`, error)
- })
-
res.json({ reactions: mutation.reactions })
})
@@ -5167,7 +5186,7 @@ async function requireBoardAccess(req: Request & AuthedRequest, boardId: string)
return { userId, companyId }
}
-async function publishBoardEvent(args: {
+async function enqueueBoardEvent(db: PoolClient, args: {
companyId: string
kind: import('../redis.js').BoardEvent['kind']
boardId: string
@@ -5177,8 +5196,7 @@ async function publishBoardEvent(args: {
mentions?: string[]
actorId?: string
}): Promise<void> {
- const { CH_BOARDS, publish } = await import('../redis.js')
- await publish(CH_BOARDS, {
+ await enqueueBroadcast(db, CH_BOARDS, {
type: 'board.changed',
companyId: args.companyId,
kind: args.kind,
@@ -5297,13 +5315,19 @@ api.post('/boards', async (req, res) => {
const title = String(req.body?.title ?? '').trim().slice(0, 200)
const description = String(req.body?.description ?? '').trim().slice(0, 4000) || null
if (!title) throw new HttpError(400, 'title required')
- const id = `board-${randomUUID().slice(0, 12)}`
- const client = await pool.connect()
- try {
- await client.query('BEGIN')
+ const requestId = createRequestId(req.body?.requestId)
+ const hash = requestHash({ title, description })
+ const created = await withOutboxTransaction(async (client) => {
+ const replay = await createReplay(client, {
+ domain: 'board', companyId, actorId: me, requestId, requestHash: hash,
+ })
+ if (replay) return { id: replay.id, replayed: true }
+ const id = `board-${randomUUID().slice(0, 12)}`
await client.query(
- `INSERT INTO boards (id, company_id, title, description, created_by) VALUES ($1, $2, $3, $4, $5)`,
- [id, companyId, title, description, me],
+ `INSERT INTO boards
+ (id, company_id, title, description, created_by, creation_request_id, creation_request_hash)
+ VALUES ($1, $2, $3, $4, $5, $6, $7)`,
+ [id, companyId, title, description, me, requestId, requestId ? hash : null],
)
// Seed the MEANING alongside the title — see board-columns.ts. Without it an
// agent asked to advance a card has to infer which column is "Doing" from
@@ -5315,15 +5339,10 @@ api.post('/boards', async (req, res) => {
[`col-${randomUUID().slice(0, 12)}`, id, seeds[i][0], (i + 1) * 1000, seeds[i][1]],
)
}
- await client.query('COMMIT')
- } catch (e) {
- await client.query('ROLLBACK').catch(() => { /* swallow */ })
- throw e
- } finally {
- client.release()
- }
- await publishBoardEvent({ companyId, kind: 'board.created', boardId: id, actorId: me })
- res.json({ id })
+ await enqueueBoardEvent(client, { companyId, kind: 'board.created', boardId: id, actorId: me })
+ return { id, replayed: false }
+ })
+ res.status(created.replayed ? 200 : 201).json(created)
})
/** GET /boards/:id — full snapshot: board + columns (ordered) + cards
@@ -5407,11 +5426,13 @@ api.patch('/boards/:id', async (req, res) => {
params.push(v); sets.push(`${k} = $${params.length}`)
}
params.push(boardId)
- await pool.query(
- `UPDATE boards SET ${sets.join(', ')}, updated_at = NOW() WHERE id = $${params.length}`,
- params,
- )
- await publishBoardEvent({ companyId, kind: 'board.updated', boardId, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE boards SET ${sets.join(', ')}, updated_at = NOW() WHERE id = $${params.length}`,
+ params,
+ )
+ await enqueueBoardEvent(client, { companyId, kind: 'board.updated', boardId, actorId: me })
+ })
res.json({ ok: true })
})
@@ -5419,8 +5440,10 @@ api.patch('/boards/:id', async (req, res) => {
api.delete('/boards/:id', async (req, res) => {
const boardId = req.params.id
const { userId: me, companyId } = await requireBoardAccess(req, boardId)
- await pool.query(`DELETE FROM boards WHERE id = $1`, [boardId])
- await publishBoardEvent({ companyId, kind: 'board.deleted', boardId, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ await client.query(`DELETE FROM boards WHERE id = $1`, [boardId])
+ await enqueueBoardEvent(client, { companyId, kind: 'board.deleted', boardId, actorId: me })
+ })
res.json({ ok: true })
})
@@ -5435,11 +5458,13 @@ api.post('/boards/:id/columns', async (req, res) => {
)
const position = (Number(posRows[0]?.max ?? 0)) + 1000
const id = `col-${randomUUID().slice(0, 12)}`
- await pool.query(
- `INSERT INTO board_columns (id, board_id, title, position) VALUES ($1, $2, $3, $4)`,
- [id, boardId, title, position],
- )
- await publishBoardEvent({ companyId, kind: 'column.created', boardId, columnId: id, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `INSERT INTO board_columns (id, board_id, title, position) VALUES ($1, $2, $3, $4)`,
+ [id, boardId, title, position],
+ )
+ await enqueueBoardEvent(client, { companyId, kind: 'column.created', boardId, columnId: id, actorId: me })
+ })
res.json({ id, position })
})
@@ -5458,12 +5483,14 @@ api.patch('/boards/:bid/columns/:cid', async (req, res) => {
}
if (sets.length === 0) { res.json({ ok: true }); return }
params.push(columnId); params.push(boardId)
- const r = await pool.query(
- `UPDATE board_columns SET ${sets.join(', ')} WHERE id = $${params.length - 1} AND board_id = $${params.length}`,
- params,
- )
- if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
- await publishBoardEvent({ companyId, kind: 'column.updated', boardId, columnId, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ const r = await client.query(
+ `UPDATE board_columns SET ${sets.join(', ')} WHERE id = $${params.length - 1} AND board_id = $${params.length}`,
+ params,
+ )
+ if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
+ await enqueueBoardEvent(client, { companyId, kind: 'column.updated', boardId, columnId, actorId: me })
+ })
res.json({ ok: true })
})
@@ -5472,12 +5499,14 @@ api.delete('/boards/:bid/columns/:cid', async (req, res) => {
const boardId = req.params.bid
const columnId = req.params.cid
const { userId: me, companyId } = await requireBoardAccess(req, boardId)
- const r = await pool.query(
- `DELETE FROM board_columns WHERE id = $1 AND board_id = $2`,
- [columnId, boardId],
- )
- if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
- await publishBoardEvent({ companyId, kind: 'column.deleted', boardId, columnId, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ const r = await client.query(
+ `DELETE FROM board_columns WHERE id = $1 AND board_id = $2`,
+ [columnId, boardId],
+ )
+ if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
+ await enqueueBoardEvent(client, { companyId, kind: 'column.deleted', boardId, columnId, actorId: me })
+ })
res.json({ ok: true })
})
@@ -5505,15 +5534,17 @@ api.post('/boards/:id/cards', async (req, res) => {
const position = (Number(posRows[0]?.max ?? 0)) + 1000
const mentions = await parseMentions(companyId, `${title}\n${description ?? ''}`)
const id = `card-${randomUUID().slice(0, 12)}`
- await pool.query(
- `INSERT INTO board_cards
- (id, board_id, column_id, title, description, position, assignee_id, mentions, created_by)
- VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9)`,
- [id, boardId, columnId, title, description, position, assigneeId, JSON.stringify(mentions), me],
- )
- await pool.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
- await publishBoardEvent({
- companyId, kind: 'card.created', boardId, cardId: id, columnId, mentions, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `INSERT INTO board_cards
+ (id, board_id, column_id, title, description, position, assignee_id, mentions, created_by)
+ VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9)`,
+ [id, boardId, columnId, title, description, position, assigneeId, JSON.stringify(mentions), me],
+ )
+ await client.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
+ await enqueueBoardEvent(client, {
+ companyId, kind: 'card.created', boardId, cardId: id, columnId, mentions, actorId: me,
+ })
})
void wakeKanbanAgents({
companyId, mentions, actorId: me,
@@ -5590,16 +5621,18 @@ api.patch('/boards/:bid/cards/:cid', async (req, res) => {
}
if (sets.length === 0) { res.json({ ok: true }); return }
params.push(cardId); params.push(boardId)
- await pool.query(
- `UPDATE board_cards SET ${sets.join(', ')}, updated_at = NOW()
- WHERE id = $${params.length - 1} AND board_id = $${params.length}`,
- params,
- )
- await pool.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
- await publishBoardEvent({
- companyId,
- kind: columnChanged ? 'card.moved' : 'card.updated',
- boardId, cardId, mentions, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE board_cards SET ${sets.join(', ')}, updated_at = NOW()
+ WHERE id = $${params.length - 1} AND board_id = $${params.length}`,
+ params,
+ )
+ await client.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
+ await enqueueBoardEvent(client, {
+ companyId,
+ kind: columnChanged ? 'card.moved' : 'card.updated',
+ boardId, cardId, mentions, actorId: me,
+ })
})
void wakeKanbanAgents({
companyId, mentions, actorId: me,
@@ -5629,12 +5662,14 @@ api.delete('/boards/:bid/cards/:cid', async (req, res) => {
const boardId = req.params.bid
const cardId = req.params.cid
const { userId: me, companyId } = await requireBoardAccess(req, boardId)
- const r = await pool.query(
- `DELETE FROM board_cards WHERE id = $1 AND board_id = $2`,
- [cardId, boardId],
- )
- if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
- await publishBoardEvent({ companyId, kind: 'card.deleted', boardId, cardId, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ const r = await client.query(
+ `DELETE FROM board_cards WHERE id = $1 AND board_id = $2`,
+ [cardId, boardId],
+ )
+ if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
+ await enqueueBoardEvent(client, { companyId, kind: 'card.deleted', boardId, cardId, actorId: me })
+ })
res.json({ ok: true })
})
@@ -5679,15 +5714,17 @@ api.post('/boards/:bid/cards/:cid/comments', async (req, res) => {
if (card.rows.length === 0) throw new HttpError(404, 'not found')
const mentions = await parseMentions(companyId, body)
const id = `cmt-${randomUUID().slice(0, 12)}`
- await pool.query(
- `INSERT INTO board_card_comments (id, card_id, author_id, body, mentions)
- VALUES ($1, $2, $3, $4, $5::jsonb)`,
- [id, cardId, me, body, JSON.stringify(mentions)],
- )
- await pool.query(`UPDATE board_cards SET updated_at = NOW() WHERE id = $1`, [cardId])
- await pool.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
- await publishBoardEvent({
- companyId, kind: 'comment.created', boardId, cardId, commentId: id, mentions, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `INSERT INTO board_card_comments (id, card_id, author_id, body, mentions)
+ VALUES ($1, $2, $3, $4, $5::jsonb)`,
+ [id, cardId, me, body, JSON.stringify(mentions)],
+ )
+ await client.query(`UPDATE board_cards SET updated_at = NOW() WHERE id = $1`, [cardId])
+ await client.query(`UPDATE boards SET updated_at = NOW() WHERE id = $1`, [boardId])
+ await enqueueBoardEvent(client, {
+ companyId, kind: 'comment.created', boardId, cardId, commentId: id, mentions, actorId: me,
+ })
})
void wakeKanbanAgents({
companyId, mentions, actorId: me,
@@ -5706,14 +5743,16 @@ api.delete('/boards/:bid/cards/:cid/comments/:mid', async (req, res) => {
const cardId = req.params.cid
const mid = req.params.mid
const { userId: me, companyId } = await requireBoardAccess(req, boardId)
- const r = await pool.query(
- `DELETE FROM board_card_comments
- WHERE id = $1 AND card_id = $2 AND author_id = $3`,
- [mid, cardId, me],
- )
- if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
- await publishBoardEvent({
- companyId, kind: 'comment.deleted', boardId, cardId, commentId: mid, actorId: me,
+ await withOutboxTransaction(async (client) => {
+ const r = await client.query(
+ `DELETE FROM board_card_comments
+ WHERE id = $1 AND card_id = $2 AND author_id = $3`,
+ [mid, cardId, me],
+ )
+ if ((r.rowCount ?? 0) === 0) throw new HttpError(404, 'not found')
+ await enqueueBoardEvent(client, {
+ companyId, kind: 'comment.deleted', boardId, cardId, commentId: mid, actorId: me,
+ })
})
res.json({ ok: true })
})
@@ -5882,23 +5921,19 @@ function calendarVisibilityClause(meIdx: number, companyIdx: number): string {
)`
}
-/** Fire-and-forget WS broadcast for a calendar row change. Thin payload —
- * the client refetches the affected row (or the whole list on delete)
- * rather than receiving inline diffs. Mirrors the doc.changed shape. */
-function publishCalendarChange(args: {
+/** Queue a thin calendar invalidation in the mutation transaction. */
+async function enqueueCalendarChange(db: PoolClient, args: {
kind: 'event.created' | 'event.updated' | 'event.deleted' | 'event.dispatched'
eventId: string
companyId: string
actorId: string | null
-}): void {
- void publish(CH_CALENDAR_EVENTS, {
+}): Promise<void> {
+ await enqueueBroadcast(db, CH_CALENDAR_EVENTS, {
type: 'calendar.changed',
kind: args.kind,
eventId: args.eventId,
companyId: args.companyId,
actorId: args.actorId,
- }).catch((e) => {
- console.warn('[calendar] publish failed', e instanceof Error ? e.message : e)
})
}
@@ -6002,25 +6037,47 @@ api.post('/calendar/events', async (req, res) => {
if (!c[0]) throw new HttpError(400, 'targetConversationId not found in this workspace')
}
- const id = `ce-${randomUUID()}`
- const { rows } = await pool.query(
- `INSERT INTO calendar_events
- (id, company_id, created_by, kind, title, description, assignee_id,
- target_conversation_id, agent_prompt, start_at, end_at, all_day,
- recurrence, status, reminder_minutes_before, reminder_channel,
- is_private)
- VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13::jsonb,$14,$15,$16,$17)
- RETURNING ${CALENDAR_SELECT}`,
- [
- id, companyId, me, kind, title, description, assigneeId,
- targetConversationId, agentPrompt, startAt, endAt, allDay,
- recurrence ? JSON.stringify(recurrence) : null, status,
- reminderMinutesBefore, reminderChannel,
- isPrivate,
- ],
- )
- publishCalendarChange({ kind: 'event.created', eventId: id, companyId, actorId: me })
- res.status(201).json({ event: rowToCalendarEvent(rows[0]) })
+ const requestId = createRequestId(body.requestId)
+ const hash = requestHash({
+ title, kind, description, assigneeId, targetConversationId, agentPrompt,
+ startAt: startAt.toISOString(), endAt: endAt?.toISOString() ?? null,
+ allDay, recurrence, status, reminderMinutesBefore, reminderChannel, isPrivate,
+ })
+ const created = await withOutboxTransaction(async (client) => {
+ const replay = await createReplay(client, {
+ domain: 'calendar-event', companyId, actorId: me, requestId, requestHash: hash,
+ })
+ if (replay) {
+ const existing = await client.query(
+ `SELECT ${CALENDAR_SELECT} FROM calendar_events WHERE id = $1`,
+ [replay.id],
+ )
+ return { rows: existing.rows, replayed: true }
+ }
+ const id = `ce-${randomUUID()}`
+ const inserted = await client.query(
+ `INSERT INTO calendar_events
+ (id, company_id, created_by, kind, title, description, assignee_id,
+ target_conversation_id, agent_prompt, start_at, end_at, all_day,
+ recurrence, status, reminder_minutes_before, reminder_channel,
+ is_private, creation_request_id, creation_request_hash)
+ VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13::jsonb,$14,$15,$16,$17,$18,$19)
+ RETURNING ${CALENDAR_SELECT}`,
+ [
+ id, companyId, me, kind, title, description, assigneeId,
+ targetConversationId, agentPrompt, startAt, endAt, allDay,
+ recurrence ? JSON.stringify(recurrence) : null, status,
+ reminderMinutesBefore, reminderChannel,
+ isPrivate, requestId, requestId ? hash : null,
+ ],
+ )
+ await enqueueCalendarChange(client, { kind: 'event.created', eventId: id, companyId, actorId: me })
+ return { rows: inserted.rows, replayed: false }
+ })
+ res.status(created.replayed ? 200 : 201).json({
+ event: rowToCalendarEvent(created.rows[0]),
+ replayed: created.replayed,
+ })
})
api.get('/calendar/events/:id', async (req, res) => {
@@ -6138,9 +6195,12 @@ api.patch('/calendar/events/:id', async (req, res) => {
const sql = `UPDATE calendar_events SET ${sets.join(', ')}
WHERE id = $${params.length - 1} AND company_id = $${params.length}
RETURNING ${CALENDAR_SELECT}`
- const { rows } = await pool.query(sql, params)
- if (!rows[0]) throw new HttpError(404, 'event not found')
- publishCalendarChange({ kind: 'event.updated', eventId: id, companyId, actorId: me })
+ const rows = await withOutboxTransaction(async (client) => {
+ const updated = await client.query(sql, params)
+ if (!updated.rows[0]) throw new HttpError(404, 'event not found')
+ await enqueueCalendarChange(client, { kind: 'event.updated', eventId: id, companyId, actorId: me })
+ return updated.rows
+ })
res.json({ event: rowToCalendarEvent(rows[0]) })
})
@@ -6150,13 +6210,15 @@ api.delete('/calendar/events/:id', async (req, res) => {
// The visibility clause is folded into the DELETE so the same caller
// who can't read the row can't delete it either. rowCount === 0 covers
// both "no such id" and "privacy filtered" — same 404 on the wire.
- const r = await pool.query(
- `DELETE FROM calendar_events
- WHERE id = $1 AND company_id = $2 AND ${calendarVisibilityClause(3, 2)}`,
- [id, companyId, me],
- )
- if (r.rowCount === 0) throw new HttpError(404, 'event not found')
- publishCalendarChange({ kind: 'event.deleted', eventId: id, companyId, actorId: me })
+ await withOutboxTransaction(async (client) => {
+ const r = await client.query(
+ `DELETE FROM calendar_events
+ WHERE id = $1 AND company_id = $2 AND ${calendarVisibilityClause(3, 2)}`,
+ [id, companyId, me],
+ )
+ if (r.rowCount === 0) throw new HttpError(404, 'event not found')
+ await enqueueCalendarChange(client, { kind: 'event.deleted', eventId: id, companyId, actorId: me })
+ })
res.json({ ok: true })
})
@@ -6174,7 +6236,13 @@ api.post('/calendar/events/:id/run-now', async (req, res) => {
// wide enough to absorb concurrent button-mashing within the same minute.
const result = await dispatchEvent(rows[0] as import('../calendar.js').CalendarEventRow, new Date())
// last_fired_at moved + dispatch happened → renderers want to refetch.
- publishCalendarChange({ kind: 'event.dispatched', eventId: id, companyId, actorId: me })
+ // dispatchEvent owns the durable dispatch transaction. This thin UI
+ // invalidation is queued independently only after dispatch succeeds; the
+ // dispatcher itself also reconciles from PostgreSQL on every tick.
+ await withOutboxTransaction((client) => enqueueCalendarChange(
+ client,
+ { kind: 'event.dispatched', eventId: id, companyId, actorId: me },
+ ))
res.json(result)
})
@@ -6251,13 +6319,14 @@ function toDocPayload(row: DocumentRow): DocumentPayload {
}
}
-async function publishDocumentChanged(
+async function enqueueDocumentChanged(
+ db: PoolClient,
companyId: string,
documentId: string,
kind: 'document.created' | 'document.updated' | 'document.deleted',
actorId: string,
): Promise<void> {
- await publish(CH_DOCS, {
+ await enqueueBroadcast(db, CH_DOCS, {
type: 'doc.changed',
kind,
companyId,
@@ -6281,7 +6350,11 @@ api.get('/documents', safe(async (req, res) => {
api.post('/documents', safe(async (req, res) => {
const { userId, companyId } = await requireCompany(req)
- const body = (req.body ?? {}) as { title?: unknown; conversationId?: unknown }
+ const body = (req.body ?? {}) as {
+ title?: unknown
+ conversationId?: unknown
+ requestId?: unknown
+ }
const title = typeof body.title === 'string' && body.title.trim()
? body.title.trim().slice(0, 200)
: 'Untitled'
@@ -6295,19 +6368,30 @@ api.post('/documents', safe(async (req, res) => {
if (convRows.length === 0) throw new HttpError(404, 'conversation not found')
conversationId = body.conversationId
}
- const id = `doc_${randomUUID().replace(/-/g, '').slice(0, 16)}`
- await pool.query(
- `INSERT INTO documents (id, company_id, title, created_by, conversation_id)
- VALUES ($1, $2, $3, $4, $5)`,
- [id, companyId, title, userId, conversationId],
- )
- const { rows } = await pool.query<DocumentRow>(
- `SELECT id, company_id, title, created_by, conversation_id, created_at, updated_at
- FROM documents WHERE id = $1`, [id],
- )
- const doc = toDocPayload(rows[0])
- await publishDocumentChanged(companyId, id, 'document.created', userId)
- res.status(201).json(doc)
+ const requestId = createRequestId(body.requestId)
+ const hash = requestHash({ title, conversationId })
+ const created = await withOutboxTransaction(async (client) => {
+ const replay = await createReplay(client, {
+ domain: 'document', companyId, actorId: userId, requestId, requestHash: hash,
+ })
+ const id = replay?.id ?? `doc_${randomUUID().replace(/-/g, '').slice(0, 16)}`
+ if (!replay) {
+ await client.query(
+ `INSERT INTO documents
+ (id, company_id, title, created_by, conversation_id, creation_request_id, creation_request_hash)
+ VALUES ($1, $2, $3, $4, $5, $6, $7)`,
+ [id, companyId, title, userId, conversationId, requestId, requestId ? hash : null],
+ )
+ }
+ const selected = await client.query<DocumentRow>(
+ `SELECT id, company_id, title, created_by, conversation_id, created_at, updated_at
+ FROM documents WHERE id = $1`, [id],
+ )
+ if (!replay) await enqueueDocumentChanged(client, companyId, id, 'document.created', userId)
+ return { rows: selected.rows, replayed: Boolean(replay) }
+ })
+ const doc = toDocPayload(created.rows[0])
+ res.status(created.replayed ? 200 : 201).json({ ...doc, replayed: created.replayed })
}))
api.get('/documents/:id', safe(async (req, res) => {
@@ -6330,13 +6414,15 @@ api.put('/documents/:id', safe(async (req, res) => {
throw new HttpError(400, 'title required')
}
const title = body.title.trim().slice(0, 200)
- const { rowCount } = await pool.query(
- `UPDATE documents SET title = $1, updated_at = NOW()
- WHERE id = $2 AND company_id = $3`,
- [title, id, companyId],
- )
- if (!rowCount) throw new HttpError(404, 'not found')
- await publishDocumentChanged(companyId, id, 'document.updated', userId)
+ await withOutboxTransaction(async (client) => {
+ const { rowCount } = await client.query(
+ `UPDATE documents SET title = $1, updated_at = NOW()
+ WHERE id = $2 AND company_id = $3`,
+ [title, id, companyId],
+ )
+ if (!rowCount) throw new HttpError(404, 'not found')
+ await enqueueDocumentChanged(client, companyId, id, 'document.updated', userId)
+ })
res.json({ ok: true, title })
}))
@@ -6358,8 +6444,10 @@ api.delete('/documents/:id', safe(async (req, res) => {
const role = roleRows[0]?.role ?? 'member'
if (!PRIVILEGED_ROLES.has(role)) throw new HttpError(403, 'only the creator or an owner can delete')
}
- await pool.query(`DELETE FROM documents WHERE id = $1`, [id])
- await publishDocumentChanged(companyId, id, 'document.deleted', userId)
+ await withOutboxTransaction(async (client) => {
+ await client.query(`DELETE FROM documents WHERE id = $1`, [id])
+ await enqueueDocumentChanged(client, companyId, id, 'document.deleted', userId)
+ })
res.json({ ok: true })
}))
diff --git a/server/src/calendar.ts b/server/src/calendar.ts
index 3be0ec6..7fa8bb0 100644
--- a/server/src/calendar.ts
+++ b/server/src/calendar.ts
@@ -17,7 +17,8 @@
*/
import { randomUUID } from 'node:crypto'
import { pool } from './db/pool.js'
-import { CH_MESSAGE_NEW, CH_CALENDAR_REMINDER, publish } from './redis.js'
+import { CH_MESSAGE_NEW, CH_CALENDAR_REMINDER, CH_CALENDAR_EVENTS, publish } from './redis.js'
+import { enqueueBroadcast, withOutboxTransaction } from './realtime-outbox.js'
import { env } from './env.js'
import { formatAddress, sendViaProvider, mintMessageId } from './email.js'
// The pure half lives apart so it can be tested without a database, Redis or an
@@ -121,39 +122,41 @@ async function postDispatchMessage(args: {
body: string
}): Promise<string> {
const { event, conversationId, body } = args
- const seqResult = await pool.query<{ seq: number }>(
- `INSERT INTO conversation_counters (conversation_id, next_sequence)
- VALUES ($1, 2)
- ON CONFLICT (conversation_id) DO UPDATE SET next_sequence = conversation_counters.next_sequence + 1
- RETURNING next_sequence - 1 AS seq`,
- [conversationId],
- )
- const sequence = seqResult.rows[0]?.seq ?? 1
- const messageId = `m-${randomUUID()}`
- // Author: Calendar itself. The creator is preserved in the payload, but the
- // agent should read this as a scheduled Calendar event, not as the creator
- // manually sending a generic system row.
- await pool.query(
- `INSERT INTO messages (id, conversation_id, author_id, kind, body, sequence, company_id)
- VALUES ($1,$2,$3,'system',$4,$5,$6)`,
- [messageId, conversationId, CALENDAR_SYSTEM_AUTHOR_ID, body, sequence, event.company_id],
- )
- await pool.query(`UPDATE conversations SET updated_at = NOW() WHERE id = $1`, [conversationId])
- await publish(CH_MESSAGE_NEW, {
- type: 'message.new',
- conversationId,
- companyId: event.company_id,
- message: {
- id: messageId,
+ return withOutboxTransaction(async (client) => {
+ const seqResult = await client.query<{ seq: number }>(
+ `INSERT INTO conversation_counters (conversation_id, next_sequence)
+ VALUES ($1, 2)
+ ON CONFLICT (conversation_id) DO UPDATE SET next_sequence = conversation_counters.next_sequence + 1
+ RETURNING next_sequence - 1 AS seq`,
+ [conversationId],
+ )
+ const sequence = seqResult.rows[0]?.seq ?? 1
+ const messageId = `m-${randomUUID()}`
+ // Author: Calendar itself. The creator is preserved in the payload, but the
+ // agent should read this as a scheduled Calendar event, not as the creator
+ // manually sending a generic system row.
+ await client.query(
+ `INSERT INTO messages (id, conversation_id, author_id, kind, body, sequence, company_id)
+ VALUES ($1,$2,$3,'system',$4,$5,$6)`,
+ [messageId, conversationId, CALENDAR_SYSTEM_AUTHOR_ID, body, sequence, event.company_id],
+ )
+ await client.query(`UPDATE conversations SET updated_at = NOW() WHERE id = $1`, [conversationId])
+ await enqueueBroadcast(client, CH_MESSAGE_NEW, {
+ type: 'message.new',
conversationId,
- authorId: CALENDAR_SYSTEM_AUTHOR_ID,
- kind: 'system',
- body,
- sequence,
- at: new Date().toISOString(),
- },
+ companyId: event.company_id,
+ message: {
+ id: messageId,
+ conversationId,
+ authorId: CALENDAR_SYSTEM_AUTHOR_ID,
+ kind: 'system',
+ body,
+ sequence,
+ at: new Date().toISOString(),
+ },
+ })
+ return messageId
})
- return messageId
}
/** Dispatch a single event's occurrence. Inserts the dispatch row first
@@ -418,8 +421,10 @@ export async function tickCalendar(now: Date = new Date()): Promise<{ scanned: n
: row.start_at
const slot = nextOccurrenceOnOrAfter(row.start_at, row.recurrence, after)
if (!slot) {
- await pool.query(`UPDATE calendar_events SET status='done', updated_at=NOW() WHERE id=$1`, [row.id])
- void publishCalendarChanged(row.company_id, 'event.updated', row.id)
+ await withOutboxTransaction(async (client) => {
+ await client.query(`UPDATE calendar_events SET status='done', updated_at=NOW() WHERE id=$1`, [row.id])
+ await enqueueCalendarChanged(client, row.company_id, 'event.updated', row.id)
+ })
continue
}
@@ -438,52 +443,52 @@ export async function tickCalendar(now: Date = new Date()): Promise<{ scanned: n
if (slot.getTime() > now.getTime()) continue // not yet
const lag = now.getTime() - slot.getTime()
if (lag > MAX_CATCHUP_MS) {
- await pool.query(
- `UPDATE calendar_events SET last_fired_at = $2, updated_at = NOW() WHERE id = $1`,
- [row.id, slot],
- )
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE calendar_events SET last_fired_at = $2, updated_at = NOW() WHERE id = $1`,
+ [row.id, slot],
+ )
+ await enqueueCalendarChanged(client, row.company_id, 'event.updated', row.id)
+ })
continue
}
const result = await dispatchEvent(row, slot)
if (result.status === 'dispatched' || result.status === 'skipped' || result.status === 'failed') {
- await pool.query(
- `UPDATE calendar_events SET last_fired_at = $2, updated_at = NOW() WHERE id = $1`,
- [row.id, slot],
- )
+ await withOutboxTransaction(async (client) => {
+ await client.query(
+ `UPDATE calendar_events SET last_fired_at = $2, updated_at = NOW() WHERE id = $1`,
+ [row.id, slot],
+ )
+ if (!row.recurrence) {
+ await client.query(`UPDATE calendar_events SET status='done', updated_at=NOW() WHERE id=$1`, [row.id])
+ }
+ await enqueueCalendarChanged(
+ client,
+ row.company_id,
+ result.status === 'dispatched' ? 'event.dispatched' : 'event.updated',
+ row.id,
+ )
+ })
if (result.status === 'dispatched') fired += 1
- if (!row.recurrence) {
- await pool.query(`UPDATE calendar_events SET status='done', updated_at=NOW() WHERE id=$1`, [row.id])
- }
- void publishCalendarChanged(
- row.company_id,
- result.status === 'dispatched' ? 'event.dispatched' : 'event.updated',
- row.id,
- )
}
}
return { scanned: rows.length, fired, reminded }
}
-/** Tick-driven WS broadcast for a calendar row. Best-effort — never let a
- * publish failure abort the scheduler loop. Mirrors `publishCalendarChange`
- * on the REST side and `publishCalendarCli` on the CLI side. */
-async function publishCalendarChanged(
+/** Queue the tick-driven invalidation in the calendar mutation transaction. */
+async function enqueueCalendarChanged(
+ client: import('pg').PoolClient,
companyId: string,
kind: 'event.updated' | 'event.dispatched',
eventId: string,
): Promise<void> {
- try {
- const { CH_CALENDAR_EVENTS, publish } = await import('./redis.js')
- await publish(CH_CALENDAR_EVENTS, {
- type: 'calendar.changed',
- companyId,
- kind,
- eventId,
- actorId: null,
- })
- } catch (e) {
- console.warn('[calendar] publish failed', e instanceof Error ? e.message : e)
- }
+ await enqueueBroadcast(client, CH_CALENDAR_EVENTS, {
+ type: 'calendar.changed',
+ companyId,
+ kind,
+ eventId,
+ actorId: null,
+ })
}
let started = false
diff --git a/server/src/db/migrate.ts b/server/src/db/migrate.ts
index 603c010..0449fad 100644
--- a/server/src/db/migrate.ts
+++ b/server/src/db/migrate.ts
@@ -39,6 +39,30 @@ CREATE TABLE IF NOT EXISTS messages (
CREATE INDEX IF NOT EXISTS idx_messages_convo_seq ON messages(conversation_id, sequence);
CREATE INDEX IF NOT EXISTS idx_messages_convo_created ON messages(conversation_id, created_at);
+-- Transactional realtime outbox. Durable mutations write their invalidation
+-- here in the SAME PostgreSQL transaction; a bounded worker publishes to Redis
+-- after commit. Redis is therefore never part of command completion.
+CREATE TABLE IF NOT EXISTS realtime_outbox (
+ id TEXT PRIMARY KEY,
+ channel TEXT NOT NULL,
+ payload JSONB NOT NULL,
+ attempts INTEGER NOT NULL DEFAULT 0,
+ available_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
+ locked_by TEXT,
+ locked_until TIMESTAMP WITH TIME ZONE,
+ published_at TIMESTAMP WITH TIME ZONE,
+ discarded_at TIMESTAMP WITH TIME ZONE,
+ last_error TEXT,
+ created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
+ CONSTRAINT realtime_outbox_payload_object CHECK (jsonb_typeof(payload) = 'object')
+);
+CREATE INDEX IF NOT EXISTS idx_realtime_outbox_pending
+ ON realtime_outbox(available_at, created_at)
+ WHERE published_at IS NULL AND discarded_at IS NULL;
+CREATE INDEX IF NOT EXISTS idx_realtime_outbox_published
+ ON realtime_outbox(published_at)
+ WHERE published_at IS NOT NULL;
+
ALTER TABLE messages ADD COLUMN IF NOT EXISTS client_id TEXT;
ALTER TABLE messages ADD COLUMN IF NOT EXISTS delivery_recipient_id TEXT;
CREATE INDEX IF NOT EXISTS idx_messages_delivery_recipient
@@ -1217,6 +1241,11 @@ CREATE INDEX IF NOT EXISTS idx_calendar_events_status
CREATE INDEX IF NOT EXISTS idx_calendar_events_assignee
ON calendar_events(assignee_id, start_at)
WHERE assignee_id IS NOT NULL;
+ALTER TABLE calendar_events ADD COLUMN IF NOT EXISTS creation_request_id TEXT;
+ALTER TABLE calendar_events ADD COLUMN IF NOT EXISTS creation_request_hash TEXT;
+CREATE UNIQUE INDEX IF NOT EXISTS uniq_calendar_event_creation_request
+ ON calendar_events(company_id, created_by, creation_request_id)
+ WHERE creation_request_id IS NOT NULL;
-- One row per actual firing of a calendar event. Lets the scheduler dedup
-- on (event_id, scheduled_for) so a tick that runs twice (replica race or
@@ -1298,6 +1327,11 @@ CREATE TABLE IF NOT EXISTS boards (
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_boards_company ON boards(company_id, updated_at DESC);
+ALTER TABLE boards ADD COLUMN IF NOT EXISTS creation_request_id TEXT;
+ALTER TABLE boards ADD COLUMN IF NOT EXISTS creation_request_hash TEXT;
+CREATE UNIQUE INDEX IF NOT EXISTS uniq_board_creation_request
+ ON boards(company_id, created_by, creation_request_id)
+ WHERE creation_request_id IS NOT NULL;
-- Columns within a board. position is a DOUBLE so reorders can insert
-- between two cards without renumbering every sibling.
@@ -1399,6 +1433,11 @@ CREATE INDEX IF NOT EXISTS idx_documents_company_updated
ON documents(company_id, updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_documents_conversation
ON documents(conversation_id) WHERE conversation_id IS NOT NULL;
+ALTER TABLE documents ADD COLUMN IF NOT EXISTS creation_request_id TEXT;
+ALTER TABLE documents ADD COLUMN IF NOT EXISTS creation_request_hash TEXT;
+CREATE UNIQUE INDEX IF NOT EXISTS uniq_document_creation_request
+ ON documents(company_id, created_by, creation_request_id)
+ WHERE creation_request_id IS NOT NULL;
CREATE TABLE IF NOT EXISTS document_updates (
id BIGSERIAL PRIMARY KEY,
@@ -2249,6 +2288,10 @@ async function schemaAlreadyCurrent(client: import('pg').PoolClient): Promise<bo
'participants_agent_id_unique',
'uniq_participants_agent_creation_request',
'uniq_messages_client_id',
+ 'uniq_board_creation_request',
+ 'uniq_document_creation_request',
+ 'uniq_calendar_event_creation_request',
+ 'idx_realtime_outbox_pending',
]
const { rows } = await client.query<{
missing_tables: string[]; missing_columns: string[]; missing_indexes: string[]
diff --git a/server/src/idempotent-create.ts b/server/src/idempotent-create.ts
new file mode 100644
index 0000000..4949863
--- /dev/null
+++ b/server/src/idempotent-create.ts
@@ -0,0 +1,70 @@
+import { createHash } from 'node:crypto'
+import type { PoolClient } from 'pg'
+
+export type IdempotentCreateDomain = 'board' | 'document' | 'calendar-event'
+
+const TABLE_BY_DOMAIN: Record<IdempotentCreateDomain, string> = {
+ board: 'boards',
+ document: 'documents',
+ 'calendar-event': 'calendar_events',
+}
+
+export class IdempotencyConflictError extends Error {
+ readonly status = 409
+ constructor() {
+ super('requestId was already used with different input')
+ }
+}
+
+export function parseRequestId(value: unknown): string | null {
+ if (value === undefined || value === null || value === '') return null
+ if (typeof value !== 'string') throw new TypeError('requestId must be a string')
+ const requestId = value.trim()
+ if (!requestId || requestId.length > 128 || !/^[A-Za-z0-9._:-]+$/.test(requestId)) {
+ throw new TypeError('requestId must be 1-128 URL-safe characters')
+ }
+ return requestId
+}
+
+/** Stable hash over already-normalized command input. */
+export function requestHash(value: unknown): string {
+ return createHash('sha256').update(JSON.stringify(value)).digest('hex')
+}
+
+/**
+ * Serialize retries for one actor-scoped create request and return the entity
+ * produced by an earlier committed attempt, if any. The advisory lock lives
+ * until the surrounding transaction commits/rolls back.
+ */
+export async function findIdempotentCreate(
+ client: PoolClient,
+ args: {
+ domain: IdempotentCreateDomain
+ companyId: string
+ actorId: string
+ requestId: string | null
+ requestHash: string
+ },
+): Promise<{ id: string } | null> {
+ if (!args.requestId) return null
+ await client.query(
+ `SELECT pg_advisory_xact_lock(hashtext($1), hashtext($2))`,
+ [`create:${args.domain}`, `${args.companyId}:${args.actorId}:${args.requestId}`],
+ )
+ const table = TABLE_BY_DOMAIN[args.domain]
+ const { rows } = await client.query<{ id: string; creation_request_hash: string }>(
+ `SELECT id, creation_request_hash
+ FROM ${table}
+ WHERE company_id = $1
+ AND created_by = $2
+ AND creation_request_id = $3
+ LIMIT 1`,
+ [args.companyId, args.actorId, args.requestId],
+ )
+ const existing = rows[0]
+ if (!existing) return null
+ if (existing.creation_request_hash !== args.requestHash) {
+ throw new IdempotencyConflictError()
+ }
+ return { id: existing.id }
+}
diff --git a/server/src/index.ts b/server/src/index.ts
index ff2467d..c324d1d 100644
--- a/server/src/index.ts
+++ b/server/src/index.ts
@@ -34,6 +34,7 @@ import { startTrialSweepWorker } from './trial-sweep.js'
import { seedAdmins } from './admin.js'
import { notifyAlert } from './alerting.js'
import { startShippingMaintenance } from './shipping-maintenance.js'
+import { startRealtimeOutboxWorker, stopRealtimeOutboxWorker } from './realtime-outbox.js'
async function main() {
await ensureSchemaWithBootRetry()
@@ -311,6 +312,10 @@ async function main() {
// failures into deduplicated improvement signals.
startShippingMaintenance()
+ // PostgreSQL owns command completion; Redis delivery is retried here after
+ // commit. Start after the schema ensure so the outbox table is guaranteed.
+ startRealtimeOutboxWorker()
+
// Agent-pod garbage collection — sweep Succeeded/Failed/Unknown
// agent pods older than 5min. Plain Pods don't have TTL-after-
// finished, so without this leftover idle-exit pods accumulate
@@ -369,6 +374,7 @@ async function main() {
const shutdown = async (sig: string) => {
console.log(`[shutdown] ${sig}`)
server.close()
+ stopRealtimeOutboxWorker()
try { await pool.end() } catch { /* ignore */ }
try { redis.disconnect() } catch { /* ignore */ }
process.exit(0)
diff --git a/server/src/polls.ts b/server/src/polls.ts
index cbb8dd3..9028b05 100644
--- a/server/src/polls.ts
+++ b/server/src/polls.ts
@@ -17,7 +17,12 @@
*/
import { randomUUID } from 'node:crypto'
import { pool } from './db/pool.js'
-import { CH_MESSAGE_NEW, CH_POLLS, publish, type PollUpdatedEvent } from './redis.js'
+import { CH_MESSAGE_NEW, CH_POLLS, type PollUpdatedEvent } from './redis.js'
+import {
+ enqueueBroadcast,
+ nudgeRealtimeOutbox,
+ type Queryable,
+} from './realtime-outbox.js'
import type { PollPayload, PollOption } from './db/schema.js'
const MAX_OPTIONS = 10
@@ -140,7 +145,24 @@ export async function createPoll(input: CreatePollInput): Promise<CreatedPoll> {
`UPDATE conversations SET updated_at = NOW() WHERE id = $1 AND company_id = $2`,
[input.conversationId, input.companyId],
)
+ await enqueueBroadcast(client, CH_MESSAGE_NEW, {
+ type: 'message.new',
+ conversationId: input.conversationId,
+ companyId: input.companyId,
+ message: {
+ id: messageId,
+ conversationId: input.conversationId,
+ authorId: input.authorId,
+ kind: 'poll',
+ body,
+ sequence,
+ at: new Date().toISOString(),
+ poll: payload,
+ pollTallies: [],
+ },
+ })
await client.query('COMMIT')
+ nudgeRealtimeOutbox()
} catch (error) {
await client.query('ROLLBACK').catch(() => {})
throw error
@@ -148,35 +170,6 @@ export async function createPoll(input: CreatePollInput): Promise<CreatedPoll> {
client.release()
}
- // Drop the poll into the message bus exactly like a text message — the
- // mailbox scheduler wakes member agents, each gets to decide for itself
- // whether to vote / react / stay silent.
- //
- // We MUST carry the structured payload on the broadcast. Without it the
- // renderer receives a kind='poll' row whose `poll` field is undefined,
- // the PollBubble bails out, and the new poll renders as a blank slot
- // until the user switches conversations and re-fetches via GET. Empty
- // pollTallies pre-seeds the array so the bubble doesn't NaN-divide on
- // total = 0 either.
- await publish(CH_MESSAGE_NEW, {
- type: 'message.new',
- conversationId: input.conversationId,
- companyId: input.companyId,
- message: {
- id: messageId,
- conversationId: input.conversationId,
- authorId: input.authorId,
- kind: 'poll',
- body,
- sequence,
- at: new Date().toISOString(),
- poll: payload,
- pollTallies: [],
- },
- }).catch((error) => {
- console.warn(`[poll] durable message ${messageId} committed but publish failed`, error)
- })
-
return { messageId, sequence, poll: payload }
}
@@ -242,10 +235,10 @@ export async function castVote(input: CastVoteInput): Promise<PollUpdatedEvent>
[input.messageId, input.voterParticipantId, input.voterKind, optId, input.companyId],
)
}
+ const event = await buildPollUpdatedEvent(input.messageId, input.voterParticipantId, client)
+ await enqueueBroadcast(client, CH_POLLS, event)
await client.query('COMMIT')
-
- const event = await buildPollUpdatedEvent(input.messageId, input.voterParticipantId)
- await publish(CH_POLLS, event)
+ nudgeRealtimeOutbox()
return event
} catch (e) {
await client.query('ROLLBACK').catch(() => { /* swallow */ })
@@ -318,10 +311,10 @@ export async function closePoll(input: CloseInput): Promise<PollUpdatedEvent | n
`UPDATE messages SET poll = $2::jsonb WHERE id = $1`,
[input.messageId, JSON.stringify(closedPoll)],
)
+ const event = await buildPollUpdatedEvent(input.messageId, input.actorId, client)
+ await enqueueBroadcast(client, CH_POLLS, event)
await client.query('COMMIT')
-
- const event = await buildPollUpdatedEvent(input.messageId, input.actorId)
- await publish(CH_POLLS, event)
+ nudgeRealtimeOutbox()
return event
} catch (e) {
await client.query('ROLLBACK').catch(() => { /* swallow */ })
@@ -333,8 +326,12 @@ export async function closePoll(input: CloseInput): Promise<PollUpdatedEvent | n
/** Build the fan-out payload — full poll snapshot + per-option tally with
* voter ids. Renderers patch in place from this event without refetching. */
-async function buildPollUpdatedEvent(messageId: string, actorId: string | null): Promise<PollUpdatedEvent> {
- const { rows } = await pool.query<{
+async function buildPollUpdatedEvent(
+ messageId: string,
+ actorId: string | null,
+ db: Queryable = pool,
+): Promise<PollUpdatedEvent> {
+ const { rows } = await db.query<{
conversation_id: string
company_id: string
poll: PollPayload | null
@@ -345,7 +342,7 @@ async function buildPollUpdatedEvent(messageId: string, actorId: string | null):
const row = rows[0]
if (!row || !row.poll) throw new PollError('poll vanished mid-update', 500)
- const { rows: tallyRows } = await pool.query<{
+ const { rows: tallyRows } = await db.query<{
option_id: string
cnt: number
voter_ids: string[]
diff --git a/server/src/realtime-outbox.ts b/server/src/realtime-outbox.ts
new file mode 100644
index 0000000..35f9fbf
--- /dev/null
+++ b/server/src/realtime-outbox.ts
@@ -0,0 +1,246 @@
+import { randomUUID } from 'node:crypto'
+import type { PoolClient, QueryResult, QueryResultRow } from 'pg'
+import { pool } from './db/pool.js'
+import { publish, type BroadcastEvent } from './redis.js'
+
+/**
+ * Transactional realtime outbox.
+ *
+ * Durable application mutations enqueue their Redis invalidation in the same
+ * PostgreSQL transaction. A bounded background dispatcher performs the
+ * network I/O only after commit, so Redis degradation can no longer turn a
+ * committed command into an HTTP/CLI failure. Delivery is at-least-once;
+ * `deliveryId` lets consumers deduplicate if they ever need more than the
+ * current refetch/idempotent-patch behaviour.
+ */
+
+export type Queryable = {
+ query<R extends QueryResultRow = QueryResultRow>(
+ text: string,
+ values?: unknown[],
+ ): Promise<QueryResult<R>>
+}
+
+export interface PendingBroadcast {
+ id: string
+ channel: string
+ payload: BroadcastEvent
+ attempts: number
+}
+
+export interface OutboxDrainResult {
+ claimed: number
+ published: number
+ failed: number
+ discarded: number
+}
+
+const WORKER_INTERVAL_MS = 1_000
+const CLAIM_LEASE_MS = 15_000
+const BATCH_SIZE = 32
+const MAX_ATTEMPTS = 12
+const MAX_AGE_HOURS = 24
+const COMPLETED_RETENTION_HOURS = 24
+const DISCARDED_RETENTION_DAYS = 7
+const CLEANUP_INTERVAL_MS = 60_000
+const CLEANUP_BATCH_SIZE = 1_000
+const workerId = `outbox-${process.pid}-${randomUUID().slice(0, 8)}`
+
+let workerTimer: NodeJS.Timeout | null = null
+let workerRunning = false
+let lastCleanupAt = 0
+
+export async function enqueueBroadcast(
+ db: Queryable,
+ channel: string,
+ event: BroadcastEvent,
+): Promise<string> {
+ const id = randomUUID()
+ const payload = { ...event, deliveryId: id }
+ await db.query(
+ `INSERT INTO realtime_outbox (id, channel, payload)
+ VALUES ($1, $2, $3::jsonb)`,
+ [id, channel, JSON.stringify(payload)],
+ )
+ return id
+}
+
+/** Run a mutation and all of its realtime invalidations in one transaction. */
+export async function withOutboxTransaction<T>(
+ run: (client: PoolClient) => Promise<T>,
+): Promise<T> {
+ const client = await pool.connect()
+ try {
+ await client.query('BEGIN')
+ const result = await run(client)
+ await client.query('COMMIT')
+ nudgeRealtimeOutbox()
+ return result
+ } catch (error) {
+ await client.query('ROLLBACK').catch(() => {})
+ throw error
+ } finally {
+ client.release()
+ }
+}
+
+async function claimBatch(limit: number): Promise<PendingBroadcast[]> {
+ const { rows } = await pool.query<{
+ id: string
+ channel: string
+ payload: BroadcastEvent
+ attempts: number
+ }>(
+ `WITH candidates AS (
+ SELECT id
+ FROM realtime_outbox
+ WHERE published_at IS NULL
+ AND discarded_at IS NULL
+ AND available_at <= NOW()
+ AND (locked_until IS NULL OR locked_until < NOW())
+ AND attempts < $3
+ AND created_at > NOW() - ($4 * INTERVAL '1 hour')
+ ORDER BY created_at, id
+ LIMIT $1
+ FOR UPDATE SKIP LOCKED
+ )
+ UPDATE realtime_outbox o
+ SET locked_by = $2,
+ locked_until = NOW() + ($5 * INTERVAL '1 millisecond')
+ FROM candidates c
+ WHERE o.id = c.id
+ RETURNING o.id, o.channel, o.payload, o.attempts`,
+ [limit, workerId, MAX_ATTEMPTS, MAX_AGE_HOURS, CLAIM_LEASE_MS],
+ )
+ return rows
+}
+
+async function discardExpired(): Promise<number> {
+ const result = await pool.query(
+ `UPDATE realtime_outbox
+ SET discarded_at = NOW(),
+ locked_by = NULL,
+ locked_until = NULL,
+ last_error = COALESCE(last_error, 'delivery budget exhausted')
+ WHERE published_at IS NULL
+ AND discarded_at IS NULL
+ AND (attempts >= $1 OR created_at <= NOW() - ($2 * INTERVAL '1 hour'))`,
+ [MAX_ATTEMPTS, MAX_AGE_HOURS],
+ )
+ return result.rowCount ?? 0
+}
+
+async function cleanupTerminalRows(): Promise<void> {
+ const now = Date.now()
+ if (now - lastCleanupAt < CLEANUP_INTERVAL_MS) return
+ lastCleanupAt = now
+ await pool.query(
+ `WITH stale AS (
+ SELECT id
+ FROM realtime_outbox
+ WHERE published_at <= NOW() - ($1 * INTERVAL '1 hour')
+ OR discarded_at <= NOW() - ($2 * INTERVAL '1 day')
+ ORDER BY COALESCE(published_at, discarded_at), id
+ LIMIT $3
+ )
+ DELETE FROM realtime_outbox o
+ USING stale
+ WHERE o.id = stale.id`,
+ [COMPLETED_RETENTION_HOURS, DISCARDED_RETENTION_DAYS, CLEANUP_BATCH_SIZE],
+ )
+}
+
+async function markPublished(id: string): Promise<void> {
+ await pool.query(
+ `UPDATE realtime_outbox
+ SET published_at = NOW(), locked_by = NULL, locked_until = NULL,
+ last_error = NULL
+ WHERE id = $1 AND locked_by = $2`,
+ [id, workerId],
+ )
+}
+
+async function markFailed(row: PendingBroadcast, error: unknown): Promise<void> {
+ const message = error instanceof Error ? error.message : String(error)
+ // 1s, 2s, 4s ... capped at 5min. Redis commands themselves are bounded in
+ // redis.ts, so the lease remains comfortably longer than one attempt.
+ const delayMs = Math.min(300_000, 1_000 * (2 ** Math.min(row.attempts, 8)))
+ await pool.query(
+ `UPDATE realtime_outbox
+ SET attempts = attempts + 1,
+ available_at = NOW() + ($3 * INTERVAL '1 millisecond'),
+ locked_by = NULL,
+ locked_until = NULL,
+ last_error = LEFT($4, 1000)
+ WHERE id = $1 AND locked_by = $2`,
+ [row.id, workerId, delayMs, message],
+ )
+}
+
+/**
+ * Drain one bounded batch. Exported for fault-injection integration tests.
+ * Production uses the default Redis publisher; tests can inject a failure.
+ */
+export async function drainRealtimeOutbox(options: {
+ publishFn?: (channel: string, event: BroadcastEvent) => Promise<void>
+ batchSize?: number
+} = {}): Promise<OutboxDrainResult> {
+ const publishFn = options.publishFn ?? publish
+ await cleanupTerminalRows()
+ const discarded = await discardExpired()
+ const rows = await claimBatch(Math.max(1, Math.min(options.batchSize ?? BATCH_SIZE, 100)))
+ let published = 0
+ let failed = 0
+
+ // Bound simultaneous Redis calls so a large recovered queue cannot create a
+ // connection spike. Promise.all is safe at this batch size (32).
+ await Promise.all(rows.map(async (row) => {
+ try {
+ await publishFn(row.channel, row.payload)
+ await markPublished(row.id)
+ published += 1
+ } catch (error) {
+ await markFailed(row, error)
+ failed += 1
+ }
+ }))
+
+ return { claimed: rows.length, published, failed, discarded }
+}
+
+function runWorkerTick(): void {
+ if (workerRunning) return
+ workerRunning = true
+ void drainRealtimeOutbox()
+ .then((result) => {
+ if (result.failed > 0) {
+ console.warn(`[outbox] ${result.failed} realtime event(s) delayed; Redis delivery will retry`)
+ }
+ if (result.discarded > 0) {
+ console.error(`[outbox] discarded ${result.discarded} expired realtime event(s)`)
+ }
+ })
+ .catch((error) => {
+ console.warn('[outbox] drain failed', error instanceof Error ? error.message : error)
+ })
+ .finally(() => { workerRunning = false })
+}
+
+export function nudgeRealtimeOutbox(): void {
+ if (!workerTimer) return
+ setImmediate(runWorkerTick)
+}
+
+export function startRealtimeOutboxWorker(): NodeJS.Timeout {
+ if (workerTimer) return workerTimer
+ runWorkerTick()
+ workerTimer = setInterval(runWorkerTick, WORKER_INTERVAL_MS)
+ workerTimer.unref()
+ console.log(`[boot] realtime outbox running every ${WORKER_INTERVAL_MS}ms`)
+ return workerTimer
+}
+
+export function stopRealtimeOutboxWorker(): void {
+ if (workerTimer) clearInterval(workerTimer)
+ workerTimer = null
+}
diff --git a/server/src/redis.ts b/server/src/redis.ts
index 2a1ad4c..4140509 100644
--- a/server/src/redis.ts
+++ b/server/src/redis.ts
@@ -69,7 +69,11 @@ export const CH_CALENDAR_EVENTS = 'cumora:calendar.events'
* memberships. Events without `companyId` are treated as conservatively
* routable (skipped) — keep the field populated at every publish site.
*/
-interface TenantTagged { companyId?: string }
+interface TenantTagged {
+ companyId?: string
+ /** Stable transactional-outbox delivery id. Redis delivery is at-least-once. */
+ deliveryId?: string
+}
export interface MessageNewEvent extends TenantTagged {
type: 'message.new'
diff --git a/src/api/client.ts b/src/api/client.ts
index e69ccda..f9c9ce6 100644
--- a/src/api/client.ts
+++ b/src/api/client.ts
@@ -1335,8 +1335,8 @@ export const api = {
listBoards: () => http<BoardSummary[]>('/boards'),
getBoard: (id: string) => http<BoardSnapshot>(`/boards/${encodeURIComponent(id)}`),
getBoardCard: (id: string) => http<BoardCardLookup>(`/cards/${encodeURIComponent(id)}`),
- createBoard: (input: { title: string; description?: string }) =>
- http<{ id: string }>('/boards', { method: 'POST', body: JSON.stringify(input) }),
+ createBoard: (input: { title: string; description?: string; requestId?: string }) =>
+ http<{ id: string; replayed: boolean }>('/boards', { method: 'POST', body: JSON.stringify(input) }),
updateBoard: (id: string, input: { title?: string; description?: string }) =>
http<{ ok: boolean }>(`/boards/${encodeURIComponent(id)}`, {
method: 'PATCH', body: JSON.stringify(input),
@@ -1408,7 +1408,7 @@ export const api = {
getCalendarEvent: (id: string) =>
http<{ event: CalendarEvent }>(`/calendar/events/${encodeURIComponent(id)}`),
createCalendarEvent: (input: CalendarEventInput) =>
- http<{ event: CalendarEvent }>('/calendar/events', {
+ http<{ event: CalendarEvent; replayed: boolean }>('/calendar/events', {
method: 'POST',
body: JSON.stringify(input),
}),
@@ -1431,7 +1431,7 @@ export const api = {
/* ============== Collaborative documents (CRDT) ============== */
listDocuments: () =>
http<{ documents: ApiDocument[] }>('/documents'),
- createDocument: (input: { title?: string; conversationId?: string | null } = {}) =>
+ createDocument: (input: { title?: string; conversationId?: string | null; requestId?: string } = {}) =>
http<ApiDocument>('/documents', { method: 'POST', body: JSON.stringify(input) }),
getDocument: (id: string) =>
http<ApiDocument>(`/documents/${encodeURIComponent(id)}`),
@@ -1472,6 +1472,8 @@ export interface CalendarEventInput {
* (and the workspace owner, if the row involves an agent). Default
* false = same shared-workspace behavior as before. */
isPrivate?: boolean
+ /** Stable across retries after an ambiguous network failure. */
+ requestId?: string
}
/* ============== WebSocket bridge ============== */
diff --git a/src/lib/create-idempotency.ts b/src/lib/create-idempotency.ts
new file mode 100644
index 0000000..351d601
--- /dev/null
+++ b/src/lib/create-idempotency.ts
@@ -0,0 +1,27 @@
+const pending = new Map<string, string>()
+
+function randomRequestId(): string {
+ if (typeof crypto !== 'undefined' && typeof crypto.randomUUID === 'function') {
+ return crypto.randomUUID()
+ }
+ return `req-${Date.now().toString(36)}-${Math.random().toString(36).slice(2)}`
+}
+
+/** Reuse the same key after an ambiguous failure; clear only after success. */
+export function pendingCreateRequestId(scope: string, normalizedInput: unknown): {
+ requestId: string
+ complete: () => void
+} {
+ const key = `${scope}:${JSON.stringify(normalizedInput)}`
+ let requestId = pending.get(key)
+ if (!requestId) {
+ requestId = randomRequestId()
+ pending.set(key, requestId)
+ }
+ return {
+ requestId,
+ complete: () => {
+ if (pending.get(key) === requestId) pending.delete(key)
+ },
+ }
+}
diff --git a/src/stores/boards.ts b/src/stores/boards.ts
index 0d413fa..c934020 100644
--- a/src/stores/boards.ts
+++ b/src/stores/boards.ts
@@ -3,6 +3,7 @@ import { api, ws } from '@/api/client'
import type {
BoardSummary, BoardSnapshot, BoardCard, BoardCardComment, BoardCardLookup,
} from '@/types'
+import { pendingCreateRequestId } from '@/lib/create-idempotency'
interface BoardsState {
/** Workspace-wide list of board summaries. Loaded once on view mount;
@@ -184,7 +185,9 @@ export const useBoards = create<BoardsState>((set, get) => ({
},
createBoard: async (title, description) => {
- const r = await api.createBoard({ title, description })
+ const retry = pendingCreateRequestId('board', { title, description: description ?? null })
+ const r = await api.createBoard({ title, description, requestId: retry.requestId })
+ retry.complete()
await get().loadList()
set({ selectedId: r.id })
await get().loadBoard(r.id)
diff --git a/src/stores/calendar.ts b/src/stores/calendar.ts
index 80d8ebd..c27bee5 100644
--- a/src/stores/calendar.ts
+++ b/src/stores/calendar.ts
@@ -1,6 +1,7 @@
import { create } from 'zustand'
import { api, ws, type CalendarEventInput } from '@/api/client'
import type { CalendarEvent, CalendarEventStatus } from '@/types'
+import { pendingCreateRequestId } from '@/lib/create-idempotency'
interface CalendarState {
events: CalendarEvent[]
@@ -90,7 +91,9 @@ export const useCalendar = create<CalendarState>((set, get) => ({
},
async create(input) {
- const { event } = await api.createCalendarEvent(input)
+ const retry = pendingCreateRequestId('calendar-event', input)
+ const { event } = await api.createCalendarEvent({ ...input, requestId: retry.requestId })
+ retry.complete()
set((s) => ({ events: replaceOrInsert(s.events, event) }))
return event
},
diff --git a/src/stores/documents.ts b/src/stores/documents.ts
index 10d07b9..9fccf2f 100644
--- a/src/stores/documents.ts
+++ b/src/stores/documents.ts
@@ -1,5 +1,6 @@
import { create } from 'zustand'
import { api, ws, type ApiDocument } from '@/api/client'
+import { pendingCreateRequestId } from '@/lib/create-idempotency'
interface DocumentsState {
list: ApiDocument[]
@@ -37,7 +38,13 @@ export const useDocuments = create<DocumentsState>((set, get) => ({
},
reset: () => set({ list: [], loaded: false, selectedId: null }),
create: async (input) => {
- const doc = await api.createDocument(input ?? {})
+ const normalized = input ?? {}
+ const retry = pendingCreateRequestId('document', {
+ title: normalized.title ?? 'Untitled',
+ conversationId: normalized.conversationId ?? null,
+ })
+ const doc = await api.createDocument({ ...normalized, requestId: retry.requestId })
+ retry.complete()
set((s) => ({ list: [doc, ...s.list.filter((d) => d.id !== doc.id)], selectedId: doc.id }))
return doc
},