Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

backend: per-pod Swarm cache keyed by a Redis stamp; reads outside transactions; reassemble once (#1907)

0xken15f8d210b9ca59
diff --git a/backend/integration/bench/bench.ts b/backend/integration/bench/bench.ts
index 91439cd32..0f339e076 100644
--- a/backend/integration/bench/bench.ts
+++ b/backend/integration/bench/bench.ts
@@ -11,6 +11,7 @@ import { legacyDriver } from './legacyDriver.js'
import { partsDriver } from './drivers/parts.js'
import { runLoad } from './load.js'
import { plan, planDigest } from './plan.js'
+import { cpuMicros, isolatedKpis, loadKpis, serverOperations } from './kpi.js'
import { isolatedPolls } from './probe.js'
import type { BenchReport } from './report.js'
import { seedSwarm } from './seed.js'
@@ -43,9 +44,16 @@ export async function runBench(config: BenchConfig, driverName: string, db: Pris
const { swarm, calls } = await seedSwarm(driver, config, p, people)
const setupMs = performance.now() - setupStart
const afterSetup = await driver.stateSize(swarm.tabId)
+ const [operationsBefore, cpuBefore] = [await serverOperations(db), cpuMicros()]
const load = await runLoad(driver, counter, config, p, swarm)
+ const [operations, cpu] = [await serverOperations(db) - operationsBefore, cpuMicros() - cpuBefore]
const afterLoad = await driver.stateSize(swarm.tabId)
const polls = await isolatedPolls(db, load.readers)
+ const desktop = load.readers.find(r => r.label.startsWith('desktop:'))
+ const kpis = {
+ load: loadKpis({ ms: load.actualMs, operations, cpuMicros: cpu, failedAttempts: load.recorder.byOp().reduce((sum, row) => sum + row.conflicts, 0) }),
+ isolated: desktop ? await isolatedKpis(db, driver, swarm, desktop) : null,
+ }
const readerLabels = new Set(load.readers.map(r => r.label))
return {
benchmark: 'swarm-state', version: 1, driver: driver.name, startedAt, commit: commit(),
@@ -57,6 +65,7 @@ export async function runBench(config: BenchConfig, driverName: string, db: Pris
writers: load.recorder.byWriter().filter(w => !readerLabels.has(w.key)), readers: load.recorder.byWriter().filter(w => readerLabels.has(w.key)) },
state: { afterSetup, afterLoad },
isolatedPolls: polls,
+ kpis,
}
} finally {
await driver.close()
diff --git a/backend/integration/bench/config.test.ts b/backend/integration/bench/config.test.ts
index bc39efd44..eac519034 100644
--- a/backend/integration/bench/config.test.ts
+++ b/backend/integration/bench/config.test.ts
@@ -35,3 +35,12 @@ describe('parseArgs', () => {
expect(() => parseArgs(['--colour', 'red'])).toThrow(/--colour/)
})
})
+
+describe('the read cache the bench runs with', () => {
+ it('keeps stamps in this process by default, or in a Redis it is given, or runs without a cache', () => {
+ expect(parseArgs([]).cache).toBe('memory')
+ expect(parseArgs(['--cache', 'off']).cache).toBe('off')
+ expect(parseArgs(['--cache', 'redis://127.0.0.1:6379/15']).cache).toBe('redis://127.0.0.1:6379/15')
+ expect(() => parseArgs(['--cache', 'disk'])).toThrow(/--cache/)
+ })
+})
diff --git a/backend/integration/bench/config.ts b/backend/integration/bench/config.ts
index c3ef7b0ad..96e1a15cb 100644
--- a/backend/integration/bench/config.ts
+++ b/backend/integration/bench/config.ts
@@ -57,7 +57,8 @@ export const DEFAULT_CONFIG: BenchConfig = {
text: { briefChars: 3_000, resultChars: 600, messageChars: 300 },
}
-export interface Args { config: BenchConfig; out: string | null; driver: string }
+/** The read cache's stamps (src/superTabs/cache.ts): in this process ('memory', one backend process), a Redis URL, or 'off'. */
+export interface Args { config: BenchConfig; out: string | null; driver: string; cache: string }
const NUMBERS: Record<string, (config: BenchConfig, value: number) => void> = {
'--seed': (c, v) => { c.seed = v },
@@ -83,7 +84,7 @@ function pairs<T extends object>(flag: string, text: string, known: T): Partial<
export function parseArgs(argv: readonly string[]): Args {
const config: BenchConfig = structuredClone(DEFAULT_CONFIG)
- let out: string | null = null, driver = 'parts'
+ let out: string | null = null, driver = 'parts', cache = 'memory'
for (let i = 0; i < argv.length; i += 2) {
const flag = argv[i]!, value = argv[i + 1]
if (value === undefined) throw new Error(`${flag} needs a value`)
@@ -97,10 +98,13 @@ export function parseArgs(argv: readonly string[]): Args {
config.lane = value as Lane
} else if (flag === '--out') out = value
else if (flag === '--driver') driver = value
- else throw new Error(`Unknown option ${flag}`)
+ else if (flag === '--cache') {
+ if (value !== 'memory' && value !== 'off' && !value.startsWith('redis://')) throw new Error('--cache is memory, off or a redis:// URL')
+ cache = value
+ } else throw new Error(`Unknown option ${flag}`)
}
validate(config)
- return { config, out, driver }
+ return { config, out, driver, cache }
}
export function activeCount(config: BenchConfig): number {
diff --git a/backend/integration/bench/kpi.test.ts b/backend/integration/bench/kpi.test.ts
new file mode 100644
index 000000000..0ab22762f
--- /dev/null
+++ b/backend/integration/bench/kpi.test.ts
@@ -0,0 +1,16 @@
+import { describe, expect, it } from 'vitest'
+import { loadKpis, mongoOperations } from './kpi.js'
+
+describe('mongoOperations', () => {
+ it('sums every operation MongoDB served, as serverStatus reports them through Prisma', () => {
+ const status = { opcounters: { insert: { $numberLong: '10' }, query: 200, update: { $numberInt: '30' }, delete: 4, getmore: 6, command: { $numberLong: '50' } } }
+ expect(mongoOperations(status)).toBe(300)
+ })
+})
+
+describe('loadKpis', () => {
+ it('turns the counters around the load into the roadmap numbers', () => {
+ expect(loadKpis({ ms: 120_000, operations: 36_000, cpuMicros: 30_000_000, failedAttempts: 6 }))
+ .toEqual({ mongoOpsPerSecond: 300, cpuPercent: 25, retriesPerMinute: 3 })
+ })
+})
diff --git a/backend/integration/bench/kpi.ts b/backend/integration/bench/kpi.ts
new file mode 100644
index 000000000..e0cd884f7
--- /dev/null
+++ b/backend/integration/bench/kpi.ts
@@ -0,0 +1,62 @@
+import { performance } from 'node:perf_hooks'
+import type { PrismaClient } from '@prisma/client'
+import type { AuthUser } from '../../src/lib/ssoAuth.js'
+import type { Reader, SwarmDriver } from './driver.js'
+import { percentile } from './metrics.js'
+import { count } from './probe.js'
+import type { Swarm } from './seed.js'
+
+/** The Stability Roadmap's numbers for one run (harness-stability-roadmap.md 2.2 and 2.6). */
+export interface Kpis {
+ load: LoadKpis
+ /** One write and one up-to-date Desktop poll at a time, after the load, with nothing else running. */
+ isolated: { writes: number; cpuMsPerWrite: number; mongoOpsPerWrite: number; polls: number; pollP50Ms: number; pollP95Ms: number; mongoOpsPerPoll: number }
+}
+export interface LoadKpis { mongoOpsPerSecond: number; cpuPercent: number; retriesPerMinute: number }
+
+/** Every operation MongoDB served since it started: reads, writes and commands (a transaction's commit is one). */
+export function mongoOperations(status: unknown): number {
+ const counters = (status as { opcounters: Record<string, unknown> }).opcounters
+ return ['insert', 'query', 'update', 'delete', 'getmore', 'command'].reduce((sum, key) => sum + count(counters[key]), 0)
+}
+
+export async function serverOperations(db: PrismaClient): Promise<number> {
+ return mongoOperations(await db.$runCommandRaw({ serverStatus: 1 }))
+}
+
+/** This process's CPU time, user and system, in microseconds: the backend under test runs in it. */
+export const cpuMicros = () => { const { user, system } = process.cpuUsage(); return user + system }
+
+export function loadKpis(run: { ms: number; operations: number; cpuMicros: number; failedAttempts: number }): LoadKpis {
+ const seconds = run.ms / 1000
+ return { mongoOpsPerSecond: run.operations / seconds, cpuPercent: run.cpuMicros / 1000 / run.ms * 100, retriesPerMinute: run.failedAttempts / (seconds / 60) }
+}
+
+/**
+ * Writes and polls one at a time: a worker's post in its own group, then an up-to-date Desktop window's poll. The status
+ * calls' own operations are measured and taken off.
+ */
+export async function isolatedKpis(db: PrismaClient, driver: SwarmDriver, swarm: Swarm, reader: Reader, n = 40): Promise<Kpis['isolated']> {
+ const before = await serverOperations(db)
+ const ownCall = await serverOperations(db) - before
+ const worker = swarm.workers[0]!, host: AuthUser = swarm.people[worker.machine]!
+ let ops = await serverOperations(db), cpu = cpuMicros()
+ for (let i = 0; i < n; i++) {
+ const reply = await driver.execute(swarm.tabId, host, `bench-kpi-write-${i}`,
+ { type: 'message.send', conversationId: worker.group, text: `KPI write ${i}`, replyTo: null, recipientIds: [] }, worker.id)
+ if (reply.code) throw new Error(`KPI write failed: ${reply.code}`)
+ }
+ const cpuMsPerWrite = (cpuMicros() - cpu) / 1000 / n, mongoOpsPerWrite = (await serverOperations(db) - ops - ownCall) / n
+ // Up to date: the first poll takes in the writes above.
+ await reader.poll()
+ const ms: number[] = []
+ ops = await serverOperations(db)
+ for (let i = 0; i < n; i++) {
+ const started = performance.now()
+ const reply = await reader.poll()
+ ms.push(performance.now() - started)
+ if (reply.code) throw new Error(`KPI poll failed: ${reply.code}`)
+ }
+ const mongoOpsPerPoll = (await serverOperations(db) - ops - ownCall) / n
+ return { writes: n, cpuMsPerWrite, mongoOpsPerWrite, polls: n, pollP50Ms: percentile(ms, 50)!, pollP95Ms: percentile(ms, 95)!, mongoOpsPerPoll }
+}
diff --git a/backend/integration/bench/probe.ts b/backend/integration/bench/probe.ts
index 1f26acb07..8925277e0 100644
--- a/backend/integration/bench/probe.ts
+++ b/backend/integration/bench/probe.ts
@@ -11,7 +11,7 @@ export interface IsolatedPoll {
}
/** A raw command's number: plain, or Extended JSON's `$numberLong` / `$numberInt` as Prisma returns some of them. */
-function count(value: unknown): number {
+export function count(value: unknown): number {
if (typeof value === 'number') return value
if (value && typeof value === 'object') {
const wrapped = (value as Record<string, unknown>).$numberLong ?? (value as Record<string, unknown>).$numberInt
diff --git a/backend/integration/bench/report.ts b/backend/integration/bench/report.ts
index a5e4ab782..2737a3d4a 100644
--- a/backend/integration/bench/report.ts
+++ b/backend/integration/bench/report.ts
@@ -1,5 +1,6 @@
import type { BenchConfig } from './config.js'
import type { Row, WriterRow } from './metrics.js'
+import type { Kpis, LoadKpis } from './kpi.js'
import type { IsolatedPoll } from './probe.js'
export interface Size { bytes: number; documents: number }
@@ -18,6 +19,10 @@ export interface BenchReport {
load: { plannedMs: number; actualMs: number; skipped: Record<string, number>; ops: Row[]; writerKinds: Row[]; writers: WriterRow[]; readers: Row[] }
state: { afterSetup: Size; afterLoad: Size }
isolatedPolls: IsolatedPoll[]
+ /** The Stability Roadmap's numbers (kpi.ts); absent from reports made before they were measured. */
+ kpis?: { load: LoadKpis; isolated: Kpis['isolated'] | null }
+ /** The read cache it ran with (stamps.ts); absent from reports made before there was one. */
+ cache?: string
}
const ms = (value: number | null) => value === null ? '–' : value.toFixed(1)
@@ -41,7 +46,7 @@ export function renderMarkdown(report: BenchReport): string {
['Machine', `${report.machine.cpu}, ${report.machine.cpus} cores, ${report.machine.memoryGb} GB, ${report.machine.platform} ${report.machine.release} ${report.machine.arch}`],
['Runtime', `Node ${report.machine.node}, MongoDB ${report.machine.mongo} (local single-node replica set)`],
['Lane', config.lane === 'cross-group' ? 'cross-group (one writer per group: no Lead messages, no delivery passes)' : 'mixed'],
- ['Seed', String(config.seed)], ['Plan', `sha256 ${report.planDigest.slice(0, 16)}`],
+ ['Read cache', report.cache ?? 'none (before the cache)'], ['Seed', String(config.seed)], ['Plan', `sha256 ${report.planDigest.slice(0, 16)}`],
['Swarm', `${config.groups} groups (${setup.doneGroups} done, ${setup.activeGroups} active), ${setup.workers} workers and a Lead, ${config.machines} machines`],
['Readers', `${config.desktopReaders} Desktop every ${config.every.desktopPollMs} ms, ${config.hostReaders} hosts every ${config.every.hostTickMs} ms`],
['Cadence', Object.entries(config.every).map(([k, v]) => `${k} ${v}`).join(', ')],
@@ -64,7 +69,19 @@ export function renderMarkdown(report: BenchReport): string {
return [row.key, String(row.count), ms(row.p50Ms), ms(row.p95Ms), row.bytes ? kb(row.bytes.mean) : '–', row.bytes ? kb(row.bytes.p95) : '–',
alone ? kb(alone.responseBytes) : '–', alone ? kb(alone.databaseBytesOut) : '–']
})), '',
+ ...(report.kpis ? kpiSection(report.kpis) : []),
'## State', '', table(['When', 'Documents', 'KB'], [
['After setup', String(state.afterSetup.documents), kb(state.afterSetup.bytes)], ['After the load', String(state.afterLoad.documents), kb(state.afterLoad.bytes)]]), '',
].join('\n')
}
+
+function kpiSection(kpis: NonNullable<BenchReport['kpis']>): string[] {
+ const { load, isolated } = kpis
+ const rows = [['Mongo operations per second, during the load', load.mongoOpsPerSecond.toFixed(1)],
+ ['Backend CPU during the load (% of one core)', load.cpuPercent.toFixed(1)], ['Transaction retries per minute, during the load', load.retriesPerMinute.toFixed(2)],
+ ...isolated ? [['CPU per write, alone (ms)', isolated.cpuMsPerWrite.toFixed(2)], ['Mongo operations per write, alone', isolated.mongoOpsPerWrite.toFixed(1)],
+ ['Up-to-date Desktop poll, alone: p50 / p95 ms', `${ms(isolated.pollP50Ms)} / ${ms(isolated.pollP95Ms)}`],
+ ['Mongo operations per Desktop poll, alone', isolated.mongoOpsPerPoll.toFixed(1)]] : []]
+ return ['## Stability KPIs', '', `Load figures cover the whole load; the isolated ones run ${isolated?.writes ?? 0} writes (a worker's post) and ${isolated?.polls ?? 0} up-to-date Desktop polls one at a time after it.`, '',
+ table(['', ''], rows), '']
+}
diff --git a/backend/integration/bench/run.ts b/backend/integration/bench/run.ts
index 34e696b3f..77158450a 100644
--- a/backend/integration/bench/run.ts
+++ b/backend/integration/bench/run.ts
@@ -4,14 +4,16 @@ import { runBench } from './bench.js'
import { parseArgs } from './config.js'
import { localDatabase } from './mongo.js'
import { renderMarkdown } from './report.js'
+import { benchStamps } from './stamps.js'
/** `npm run bench:swarm-state -- [options]` (README.md). Writes `<driver>-<time>.json` and `.md` to the results folder. */
async function main() {
- const { config, out, driver } = parseArgs(process.argv.slice(2))
+ const { config, out, driver, cache } = parseArgs(process.argv.slice(2))
const folder = out ?? join(process.cwd(), 'integration/bench/results')
const database = await localDatabase()
+ const stamps = await benchStamps(cache)
try {
- const report = await runBench(config, driver, database.db)
+ const report = { ...await runBench(config, driver, database.db), cache: stamps.label }
const name = `${report.driver}-${report.startedAt.replaceAll(':', '-').replace(/\.\d+Z$/, 'Z')}`
const markdown = renderMarkdown(report)
await mkdir(folder, { recursive: true })
@@ -19,6 +21,7 @@ async function main() {
await writeFile(join(folder, `${name}.md`), markdown)
process.stdout.write(`${markdown}\nWrote ${join(folder, name)}.json and .md\n`)
} finally {
+ await stamps.close()
await database.stop()
}
}
diff --git a/backend/integration/bench/stamps.ts b/backend/integration/bench/stamps.ts
new file mode 100644
index 000000000..2824ed74a
--- /dev/null
+++ b/backend/integration/bench/stamps.ts
@@ -0,0 +1,32 @@
+import { Redis } from 'ioredis'
+import { configureSwarmStamps, redisStamps, type StampScope } from '../../src/superTabs/cache.js'
+import { MemoryStamps } from '../../src/superTabs/cache.test-support.js'
+
+/** Sets up the read cache's stamps for the run (config.ts `--cache`); returns what the report names and how to close it. */
+export async function benchStamps(cache: string): Promise<{ label: string; close(): Promise<void> }> {
+ if (cache === 'off') {
+ configureSwarmStamps(new MemoryStamps(), { reads: false })
+ return { label: 'off (SWARM_READ_CACHE=off: every read goes to MongoDB)', close: async () => configureSwarmStamps(null) }
+ }
+ if (cache === 'memory') {
+ configureSwarmStamps(new MemoryStamps())
+ return { label: 'on, stamps in this process', close: async () => configureSwarmStamps(null) }
+ }
+ const pub = new Redis(cache, { maxRetriesPerRequest: 1, enableOfflineQueue: false }), sub = new Redis(cache)
+ await Promise.all([pub, sub].map(r => new Promise(resolve => r.once('ready', resolve))))
+ const channel = 'swarm-stamp'
+ await sub.subscribe(channel)
+ configureSwarmStamps(redisStamps(pub, {
+ publish: (tabId, scopes) => pub.publish(channel, JSON.stringify({ tabId, scopes })),
+ subscribe: listener => {
+ const heard = (from: string, payload: string) => {
+ if (from !== channel) return
+ const { tabId, scopes } = JSON.parse(payload) as { tabId: string; scopes: StampScope[] }
+ listener(tabId, scopes)
+ }
+ sub.on('message', heard)
+ return () => { sub.off('message', heard) }
+ },
+ }))
+ return { label: `on, stamps in Redis (${new URL(cache).host})`, close: async () => { configureSwarmStamps(null); await Promise.all([pub.quit(), sub.quit()]) } }
+}
diff --git a/backend/integration/cacheOn.test-support.ts b/backend/integration/cacheOn.test-support.ts
new file mode 100644
index 000000000..62db2e63c
--- /dev/null
+++ b/backend/integration/cacheOn.test-support.ts
@@ -0,0 +1,6 @@
+// Every super-tabs integration suite runs with the read cache on (superTabs/cache.ts): one process's cache over stamps in
+// memory, as Redis would hold them. A test that writes parts outside the store uses swarmTransaction, as every writer does.
+import { configureSwarmStamps } from '../src/superTabs/cache.js'
+import { MemoryStamps } from '../src/superTabs/cache.test-support.js'
+
+configureSwarmStamps(new MemoryStamps())
diff --git a/backend/integration/superTabCache.test.ts b/backend/integration/superTabCache.test.ts
new file mode 100644
index 000000000..10ea082eb
--- /dev/null
+++ b/backend/integration/superTabCache.test.ts
@@ -0,0 +1,349 @@
+import { randomUUID } from 'node:crypto'
+import { execFile } from 'node:child_process'
+import { promisify } from 'node:util'
+import { join } from 'node:path'
+import { tmpdir } from 'node:os'
+import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
+import { MongoMemoryReplSet } from 'mongodb-memory-server-core'
+import { PrismaClient } from '@prisma/client'
+import type { AuthUser } from '../src/lib/ssoAuth.js'
+import { loadRow, SuperTabStore } from '../src/superTabs/store.js'
+import { configureSwarmStamps, swarmReadCache, swarmTransaction } from '../src/superTabs/cache.js'
+import { MemoryStamps } from '../src/superTabs/cache.test-support.js'
+import * as parts from '../src/superTabs/parts/index.js'
+import { guardedWrite } from '../src/superTabs/parts/write.js'
+import type { HeadData, MachineData } from '../src/superTabs/parts/index.js'
+import { sweepSwarms } from '../src/superTabs/sweeps.js'
+import { SwarmCache } from '../src/superTabs/cache.js'
+import { readChanged, readLastChangedAt, readParts } from '../src/superTabs/cachedReads.js'
+import { SwarmReads } from '../src/superTabs/read/service.js'
+import { migrateSwarm } from '../src/superTabs/migration/migrate.js'
+import type { SuperTabState } from '../src/superTabs/model.js'
+import { writeLegacySwarm } from '../scripts/swarm-parts/legacy.js'
+import { outsider as outsiderActor, owner as ownerActor, peer as peerActor, richSwarm } from '../src/superTabs/read/fixture.test-support.js'
+
+// The per-process Swarm cache (superTabs/cache.ts, Stability Roadmap 2.2) on a fresh loopback replica set; Redis is
+// MemoryStamps, one store of stamps every "pod" here shares. This suite never reads DATABASE_URL or an existing database.
+let replica: MongoMemoryReplSet
+let db: PrismaClient
+let stamps: MemoryStamps
+const user = (n: number): AuthUser => ({ sub: `6a00000000000000000000${String(n).padStart(2, '0')}`, email: `cache-${n}@example.com`, autonomousEnv: 'prod', role: 'user' })
+const [owner] = [user(0)]
+const person = (actor: { userId: string; email: string }): AuthUser => ({ sub: actor.userId, email: actor.email, autonomousEnv: 'prod', role: 'user' })
+const people = { owner: person(ownerActor), peer: person(peerActor), outsider: person(outsiderActor) }
+
+beforeAll(async () => {
+ replica = await MongoMemoryReplSet.create({ binary: { version: '7.0.24', downloadDir: join(tmpdir(), 'harness-super-tab-mongodb-binaries') },
+ replSet: { count: 1, ip: '127.0.0.1', storageEngine: 'wiredTiger' } })
+ const uri = replica.getUri(`swarm_cache_test_${randomUUID().replaceAll('-', '_')}`)
+ await promisify(execFile)(join(process.cwd(), 'node_modules/.bin/prisma'), ['db', 'push', '--skip-generate'], {
+ env: { ...process.env, DATABASE_URL: uri }, timeout: 60000,
+ })
+ db = new PrismaClient({ datasources: { db: { url: uri } } })
+ for (const each of [owner, people.owner, people.peer, people.outsider]) await db.user.create({ data: { id: each.sub, externalId: `fixture-${each.sub}`, email: each.email, autonomousEnv: 'prod' } })
+})
+afterAll(async () => {
+ configureSwarmStamps(null)
+ await db?.$disconnect()
+ await replica?.stop()
+})
+beforeEach(() => { stamps = new MemoryStamps(); configureSwarmStamps(stamps) })
+afterEach(() => configureSwarmStamps(null))
+
+async function newSwarm(store = new SuperTabStore(db)) {
+ const { id } = await store.create(owner, `create-${randomUUID()}`, 'Cache')
+ const { state } = await store.read(owner, id)
+ return { store, tabId: id, lead: state.conversations.find(c => !c.kind)!.id }
+}
+const bumpsOf = (tabId: string) => stamps.bumps.filter(b => b.tabId === tabId).map(b => b.scopes)
+const send = (store: SuperTabStore, tabId: string, conversationId: string, text = 'hello') =>
+ store.execute(owner, tabId, { operationId: `send-${randomUUID()}`, command: { type: 'message.send', conversationId, text, replyTo: null, recipientIds: [] } })
+
+describe('stamps after commit', () => {
+ it('bumps a Swarm\'s main stamp once per command, after the command committed', async () => {
+ const { store, tabId, lead } = await newSwarm()
+ expect(bumpsOf(tabId)).toEqual([['main']])
+ const seen: Array<Promise<number>> = []
+ stamps.subscribe(id => { if (id === tabId) seen.push(db.superTabMessage.count({ where: { tabId, text: 'after commit' } })) })
+ await send(store, tabId, lead, 'after commit')
+ expect(bumpsOf(tabId)).toEqual([['main'], ['main']])
+ expect(await Promise.all(seen)).toEqual([1])
+ })
+
+ it('bumps nothing for a command that was refused', async () => {
+ const { store, tabId } = await newSwarm()
+ await expect(send(store, tabId, 'no-such-conversation')).rejects.toThrow()
+ expect(bumpsOf(tabId)).toEqual([['main']])
+ })
+
+ it('moves only the machine stamp for a machine part whose reading alone changed, and the main one for its settings', async () => {
+ const { tabId } = await newSwarm()
+ const machine: MachineData = { machineId: 'm1', contribution: null, reservations: [], released: [], closingSessions: [], cleanups: [] }
+ await swarmTransaction(db, tx => guardedWrite(tx, tabId, { op: 'insert', kind: 'machine', key: 'm1', data: machine, ordinal: 1 }, parts.clock.next()))
+ await swarmTransaction(db, tx => guardedWrite(tx, tabId, { op: 'update', kind: 'machine', key: 'm1', seen: 1, data: machine }, parts.clock.next()))
+ await swarmTransaction(db, tx => guardedWrite(tx, tabId, { op: 'update', kind: 'machine', key: 'm1', seen: 2, data: machine,
+ host: { hostChangedAt: new Date() } }, parts.clock.next()))
+ expect(bumpsOf(tabId).slice(1)).toEqual([['main'], ['machine'], ['main']])
+ })
+
+ it('bumps nothing for a touch alone', async () => {
+ const { tabId } = await newSwarm()
+ const head = (await db.swarmHead.findUniqueOrThrow({ where: { id: `${tabId}:head` } }))
+ await swarmTransaction(db, tx => guardedWrite(tx, tabId, { op: 'touch', kind: 'head', key: 'head', seen: head.version }, parts.clock.next()))
+ expect(bumpsOf(tabId)).toEqual([['main']])
+ })
+
+ it('bumps the Swarms an expiry sweep changed', async () => {
+ const { tabId } = await newSwarm()
+ const head = await db.swarmHead.findUniqueOrThrow({ where: { id: `${tabId}:head` } })
+ const data = head.data as unknown as HeadData
+ await swarmTransaction(db, tx => guardedWrite(tx, tabId, { op: 'update', kind: 'head', key: 'head', seen: head.version,
+ data: { ...data, invitations: [{ id: 'i1', email: '[email protected]', role: 'viewer', conversationId: 'lead', expiresAt: 1 }] },
+ head: { groupCounter: head.groupCounter, visibilityEpoch: head.visibilityEpoch, sequenceBase: head.sequenceBase } }, parts.clock.next()))
+ await sweepSwarms(db)
+ expect(bumpsOf(tabId)).toEqual([['main'], ['main'], ['main']])
+ })
+
+ it('keeps a command that committed when its bump fails, and forgets the Swarm in this process', async () => {
+ const { store, tabId, lead } = await newSwarm()
+ await (await swarmReadCache()!.at([tabId])).get(tabId, 'probe', 'main', async () => ({ held: true }))
+ expect(swarmReadCache()!.size.entries).toBeGreaterThan(0)
+ const bump = stamps.bump.bind(stamps)
+ stamps.bump = async () => { throw new Error('Redis is unreachable') }
+ await expect(send(store, tabId, lead, 'committed anyway')).resolves.toMatchObject({ result: { messageId: expect.any(String) } })
+ expect(await db.superTabMessage.count({ where: { tabId, text: 'committed anyway' } })).toBe(1)
+ expect(swarmReadCache()!.size.entries).toBe(0)
+ stamps.bump = bump
+ // Owed, and paid once Redis answers again: every process then sees the commit.
+ await vi.waitFor(() => expect(bumpsOf(tabId)).toEqual([['main'], ['main']]), { timeout: 5_000 })
+ })
+})
+
+/** One backend process: its own cache over the stamps every process shares. */
+function pod(options: { cache?: SwarmCache | null } = {}) {
+ const cache = options.cache === undefined ? new SwarmCache(stamps) : options.cache
+ const store = new SuperTabStore(db, Date.now, null, { readCache: cache })
+ return { store, reads: new SwarmReads(store), cache }
+}
+
+/** The rich fixture Swarm in parts, with a conversation private to `peer` (and the project's Lead). */
+async function privateScene() {
+ const s = richSwarm(Date.now())
+ const privateId = s.group('Peer and PM')
+ s.edit(state => {
+ const generation = state.people.find(p => p.id === peerActor.userId)!.generation
+ state.conversations.find(c => c.id === privateId)!.audience = { people: [{ id: peerActor.userId, generation }] }
+ })
+ const tabId = `tab-${randomUUID()}`
+ await writeLegacySwarm(db, { state: { ...structuredClone(s.state), id: tabId } as SuperTabState, revision: 10 })
+ const report = await migrateSwarm(db, tabId)
+ if (!report.ok) throw new Error(report.error)
+ return { tabId, privateId, secret: s.secret, secretGroup: s.secretGroup, web: s.web, lead: s.lead }
+}
+const groupKeys = (read: { groups: Array<{ key: string }> }) => read.groups.map(g => g.key)
+
+/** The part collections' reads MongoDB served while `run` ran (its profiler, every operation). */
+async function partReads<T>(run: () => Promise<T>): Promise<{ value: T; reads: number }> {
+ await db.$runCommandRaw({ profile: 2 })
+ const since = new Date()
+ try {
+ const value = await run()
+ const rows = await db.$runCommandRaw({ find: 'system.profile', filter: { ts: { $gte: { $date: since.toISOString() } },
+ ns: { $regex: '\\.Swarm(Head|Member|Agent|Group|Machine|Start|Progress)$' } } }) as { cursor: { firstBatch: unknown[] } }
+ return { value, reads: rows.cursor.firstBatch.length }
+ } finally {
+ await db.$runCommandRaw({ profile: 0 })
+ }
+}
+
+describe('reads through the cache', () => {
+ it('reads a Swarm whose stamp did not move without asking Mongo for its parts', async () => {
+ const { tabId } = await privateScene()
+ const { reads } = pod()
+ await reads.firstLoad(people.owner, tabId)
+ const second = await partReads(() => reads.firstLoad(people.owner, tabId))
+ expect(second.reads).toBe(0)
+ const first = await partReads(() => pod().reads.firstLoad(people.owner, tabId))
+ expect(first.reads).toBeGreaterThan(0)
+ })
+
+ it('shows a commit on one process to the next read on another (stale after commit, two pods)', async () => {
+ const { tabId } = await privateScene()
+ const a = pod(), b = pod()
+ const before = await b.reads.firstLoad(people.owner, tabId)
+ const { result } = await a.store.execute(people.owner, tabId, { operationId: `create-${randomUUID()}`,
+ command: { type: 'conversation.create', title: 'Made on A', brief: '', checks: [] } })
+ const created = result.conversationId as string
+ expect(groupKeys(await b.reads.firstLoad(people.owner, tabId))).toContain(created)
+ const changes = await b.reads.changes(people.owner, tabId, before.cursor)
+ expect('reload' in changes && changes.reload).toBe(false)
+ expect(groupKeys(changes as { groups: Array<{ key: string }> })).toContain(created)
+ // And the other way: a message on B's Swarm, read on A.
+ await b.store.execute(people.owner, tabId, { operationId: `send-${randomUUID()}`,
+ command: { type: 'message.send', conversationId: created, text: 'from B', replyTo: null, recipientIds: [] } })
+ const messages = await a.store.messages(people.owner, tabId, created, {})
+ expect(messages.messages.map(m => m.text)).toEqual(['from B'])
+ })
+
+ it('shows a removal on one process to the removed person\'s next read on another: from a project, then the Swarm', async () => {
+ const { tabId, privateId, secret, secretGroup } = await privateScene()
+ const a = pod(), b = pod()
+ const sharing = (memberIds: string[]) => a.store.execute(people.owner, tabId, { operationId: `sharing-${randomUUID()}`,
+ command: { type: 'project.sharing', projectId: secret, visibility: 'private', memberIds } })
+ await sharing([peerActor.userId])
+ const before = await b.reads.firstLoad(people.peer, tabId)
+ expect(groupKeys(before)).toContain(secretGroup)
+ expect((await b.store.messages(people.peer, tabId, secretGroup, {})).messages).toBeDefined()
+ await sharing([])
+ expect(groupKeys(await b.reads.firstLoad(people.peer, tabId))).not.toContain(secretGroup)
+ // Who may read what changed: the change feed sends the reader back to a full read rather than a delta.
+ expect(await b.reads.changes(people.peer, tabId, before.cursor)).toEqual({ reload: true })
+ await expect(b.store.messages(people.peer, tabId, secretGroup, {})).rejects.toMatchObject({ code: 'NOT_FOUND' })
+ // Warm again, then removed from the Swarm on A: every read on B refuses at once.
+ const last = await b.reads.firstLoad(people.peer, tabId)
+ expect(groupKeys(last)).toContain(privateId)
+ await a.store.execute(people.owner, tabId, { operationId: `remove-${randomUUID()}`, command: { type: 'member.remove', userId: peerActor.userId } })
+ await expect(b.reads.firstLoad(people.peer, tabId)).rejects.toMatchObject({ code: expect.stringMatching(/NOT_FOUND|FORBIDDEN/) })
+ await expect(b.reads.changes(people.peer, tabId, last.cursor)).rejects.toMatchObject({ code: expect.stringMatching(/NOT_FOUND|FORBIDDEN/) })
+ await expect(b.store.messages(people.peer, tabId, privateId, {})).rejects.toMatchObject({ code: expect.stringMatching(/NOT_FOUND|FORBIDDEN/) })
+ })
+
+ it('never shows one reader what only another may read, whichever reads first', async () => {
+ const { tabId, privateId, secretGroup } = await privateScene()
+ for (const order of [[people.peer, people.outsider], [people.outsider, people.peer]]) {
+ const { reads } = pod()
+ const seen = new Map<string, string[]>()
+ for (const reader of order) seen.set(reader.sub, groupKeys(await reads.firstLoad(reader, tabId)))
+ expect(seen.get(people.peer.sub)).toContain(privateId)
+ expect(seen.get(people.outsider.sub)).not.toContain(privateId)
+ expect(seen.get(people.outsider.sub)).not.toContain(secretGroup)
+ expect(seen.get(people.peer.sub)).not.toContain(secretGroup)
+ }
+ const { reads } = pod()
+ await reads.firstLoad(people.owner, tabId)
+ const owners = await reads.firstLoad(people.owner, tabId)
+ const outsiders = await reads.firstLoad(people.outsider, tabId)
+ expect(groupKeys(owners)).toContain(secretGroup)
+ expect(groupKeys(outsiders)).not.toContain(secretGroup)
+ await expect(pod().store.messages(people.outsider, tabId, privateId, {})).rejects.toMatchObject({ code: 'NOT_FOUND' })
+ })
+
+ it('reads Mongo directly while Redis is down, and stays right across writes', async () => {
+ const { tabId } = await privateScene()
+ const { reads, store, cache } = pod()
+ await reads.firstLoad(people.owner, tabId)
+ stamps.down = true
+ const { result } = await store.execute(people.owner, tabId, { operationId: `create-${randomUUID()}`,
+ command: { type: 'conversation.create', title: 'Redis down', brief: '', checks: [] } })
+ const read = await partReads(() => reads.firstLoad(people.owner, tabId))
+ expect(groupKeys(read.value)).toContain(result.conversationId)
+ expect(read.reads).toBeGreaterThan(0)
+ // This process held the Swarm from before the write, whose bump Redis missed: it is paid once Redis is back.
+ expect(cache!.size.entries).toBeGreaterThan(0)
+ stamps.down = false
+ await vi.waitFor(async () => expect(groupKeys(await reads.firstLoad(people.owner, tabId))).toContain(result.conversationId), { timeout: 5_000 })
+ })
+
+ it('reads directly with the read cache switched off', async () => {
+ const { tabId } = await privateScene()
+ const { reads } = pod({ cache: null })
+ await reads.firstLoad(people.owner, tabId)
+ expect((await partReads(() => reads.firstLoad(people.owner, tabId))).reads).toBeGreaterThan(0)
+ })
+})
+
+describe('composing a read from what the cache holds', () => {
+ it('gives the state loadActive gives, for every way a read names what it needs', async () => {
+ const { tabId, web, lead } = await privateScene()
+ // A quiet conversation with the Lead in it, and a finished start.
+ const quiet = (await db.swarmGroup.findFirstOrThrow({ where: { tabId, key: { notIn: [web, 'lead'] }, kind: null, dormant: false } })).key
+ await db.swarmGroup.update({ where: { id: `${tabId}:${quiet}` }, data: { dormant: true } })
+ const asLoaded = (p: parts.SwarmParts) => {
+ const state = parts.reassemble(p)
+ return { ...state, conversations: [...state.conversations].sort((a, b) => a.id.localeCompare(b.id)) }
+ }
+ const cache = new SwarmCache(stamps)
+ for (const named of [{}, { includeRecentStarts: true }, { groupIds: [quiet] }, { groupIds: [quiet, 'missing'] }, { agentIds: [lead] },
+ { projectIds: ['lead'] }, { allGroups: true }, { startIds: ['none'] }]) {
+ const direct = await db.$transaction(tx => parts.loadActive(tx, tabId, named))
+ const view = await cache.at([tabId])
+ expect(asLoaded(await readParts(db, view, tabId, named)), JSON.stringify(named)).toEqual(asLoaded(direct))
+ }
+ })
+})
+
+describe('the change feed', () => {
+ it('gives what loadChanged gives for any cursor, and serves a later cursor at the same stamp from memory', async () => {
+ const { tabId, web } = await privateScene()
+ const t = Date.now()
+ await swarmTransaction(db, async tx => {
+ const group = (await parts.loadActive(tx, tabId, { groupIds: [web] })).groups.find(g => g.key === web)!
+ await guardedWrite(tx, tabId, { op: 'update', kind: 'group', key: web, seen: group.version, data: { ...group.data!, title: 'Web 2' } }, t)
+ })
+ const direct = async (since: number) => (await db.$transaction(tx => parts.loadChanged(tx, [tabId], new Date(since)))).get(tabId)!
+ const keys = (list: parts.Part[]) => list.map(p => `${p.kind}:${p.key}:${p.version}`).sort()
+ const cache = new SwarmCache(stamps)
+ const read = async (since: number) => readChanged(db, await cache.at([tabId]), tabId, since)
+ expect(keys(await read(t - 30_000))).toEqual(keys(await direct(t - 30_000)))
+ const later = await partReads(() => read(t - 10_000))
+ expect(later.reads).toBe(0)
+ expect(keys(later.value)).toEqual(keys(await direct(t - 10_000)))
+ expect(keys(await read(t))).toEqual([])
+ expect(keys(await read(t - 3_600_000))).toEqual(keys(await direct(t - 3_600_000)))
+ })
+})
+
+describe('a command\'s load', () => {
+ it('reassembles once: its state shares the loaded records, which are frozen so nothing changes them in place', async () => {
+ const { tabId, web } = await privateScene()
+ const loaded = await db.$transaction(async tx => loadRow(tx, await tx.superTab.findFirstOrThrow({ where: { id: tabId } })))
+ const group = loaded.parts.groups.find(g => g.key === web)!
+ expect(Object.isFrozen(group.data)).toBe(true)
+ expect(loaded.state.conversations.find(c => c.id === web)).toBe(group.data)
+ expect(() => { loaded.state.conversations.find(c => c.id === web)!.title = 'changed in place' }).toThrow(TypeError)
+ })
+})
+
+describe('a machine\'s own reading', () => {
+ async function withMachine() {
+ const { tabId } = await privateScene()
+ const machine: MachineData = { machineId: 'm1', contribution: null, reservations: [], released: [], closingSessions: [], cleanups: [] }
+ await swarmTransaction(db, tx => guardedWrite(tx, tabId, { op: 'insert', kind: 'machine', key: 'm1', data: machine, ordinal: 1 }, Date.now() - 60_000))
+ const write = async (at: number, change: Partial<MachineData>, settings: boolean) => swarmTransaction(db, async tx => {
+ const row = await tx.swarmMachine.findUniqueOrThrow({ where: { id: `${tabId}:m1` } })
+ await guardedWrite(tx, tabId, { op: 'update', kind: 'machine', key: 'm1', seen: row.version, data: { ...row.data as unknown as MachineData, ...change },
+ ...settings ? { host: { hostChangedAt: new Date(at) } } : {} }, at)
+ })
+ return { tabId, write }
+ }
+ const newest = async (tabId: string) => (await parts.lastChangedAt(db, [tabId])).get(tabId)
+
+ it('leaves the Swarm\'s newest change where it was, and still reaches readers of the change feed', async () => {
+ const { tabId, write } = await withMachine()
+ const before = await newest(tabId)
+ const at = Date.now()
+ await write(at, { released: ['reading-only'] }, false)
+ expect(await newest(tabId)).toBe(before)
+ const changed = (await parts.loadChanged(db, [tabId], new Date(at - 1))).get(tabId)!
+ expect(changed.filter(p => p.kind === 'machine').map(p => (p.data as MachineData).released)).toEqual([['reading-only']])
+ const cache = new SwarmCache(stamps)
+ expect((await readChanged(db, await cache.at([tabId]), tabId, at - 1)).filter(p => p.kind === 'machine').length).toBe(1)
+ expect(await readChanged(db, await cache.at([tabId]), tabId, at)).toEqual([])
+ })
+
+ it('moves the newest change when the machine\'s settings change', async () => {
+ const { tabId, write } = await withMachine()
+ const at = Date.now() + 5_000
+ await write(at, { released: ['settings'] }, true)
+ expect(await newest(tabId)).toBe(at)
+ })
+
+ it('keeps the newest change held over a machine report', async () => {
+ const { tabId, write } = await withMachine()
+ const cache = new SwarmCache(stamps)
+ const first = await readLastChangedAt(db, await cache.at([tabId]), tabId)
+ await write(Date.now(), { released: ['again'] }, false)
+ const again = await partReads(async () => readLastChangedAt(db, await cache.at([tabId]), tabId))
+ expect(again).toEqual({ value: first, reads: 0 })
+ })
+})
diff --git a/backend/integration/superTabRead.test.ts b/backend/integration/superTabRead.test.ts
index b3b76028c..621e6a205 100644
--- a/backend/integration/superTabRead.test.ts
+++ b/backend/integration/superTabRead.test.ts
@@ -12,6 +12,7 @@ import { SuperTabStore } from '../src/superTabs/store.js'
import { superTabRoutes } from '../src/routes/superTabs.js'
import { errorHandler } from '../src/middlewares/errorHandler.js'
import * as parts from '../src/superTabs/parts/index.js'
+import { noteChange, swarmTransaction } from '../src/superTabs/cache.js'
import { guardedWrite } from '../src/superTabs/parts/write.js'
import { migrateSwarm } from '../src/superTabs/migration/migrate.js'
import { SwarmReads } from '../src/superTabs/read/service.js'
@@ -98,7 +99,7 @@ const get = async (url: string, as: AuthUser = owner) => {
const keys = (list: Array<{ key: string }>) => list.map(p => p.key)
/** A later write of one group by a command, as group 2's store does it, stamped `at`. */
async function writeGroup(tabId: string, key: string, at: number, change: (group: Conversation) => void, op: 'update' | 'remove' = 'update') {
- await db.$transaction(async tx => {
+ await swarmTransaction(db, async tx => {
const current = (await parts.loadActive(tx, tabId, { groupIds: [key] })).groups.find(g => g.key === key)!
const data = structuredClone(current.data!) as Conversation
change(data)
@@ -276,7 +277,7 @@ describe('changes since a cursor (design 4.2, 4.4)', () => {
const s = await scene()
const cursor = await firstLoad(s.tabId, peer, NOW)
expect((await changes(s.tabId, cursor, peer, NOW - 30_000 + 7 * DAY)).body.data).toEqual({ reload: true })
- await db.swarmHead.updateMany({ where: { tabId: s.tabId }, data: { visibilityEpoch: 2 } })
+ await swarmTransaction(db, async tx => { noteChange(tx, s.tabId, 'main'); await tx.swarmHead.updateMany({ where: { tabId: s.tabId }, data: { visibilityEpoch: 2 } }) })
expect((await changes(s.tabId, cursor, peer, NOW + 1_000)).body.data).toEqual({ reload: true })
expect((await changes(s.tabId, 'not-a-cursor', peer, NOW + 1_000)).body.data).toEqual({ reload: true })
})
diff --git a/backend/integration/superTabs.test.ts b/backend/integration/superTabs.test.ts
index 85543f623..baed01018 100644
--- a/backend/integration/superTabs.test.ts
+++ b/backend/integration/superTabs.test.ts
@@ -19,6 +19,7 @@ import { stableId, type SuperTabState } from '../src/superTabs/model.js'
import { ensureTerminalExpiry, TERMINAL_EXPIRY_MS } from '../src/superTabs/terminal.js'
import { ensureMentionExpiry, MENTION_EXPIRY_MS } from '../src/superTabs/mentions.js'
import * as parts from '../src/superTabs/parts/index.js'
+import { swarmTransaction } from '../src/superTabs/cache.js'
import { guardedWrite, PartConflict } from '../src/superTabs/parts/write.js'
import { ensurePartExpiry, sweepSwarms } from '../src/superTabs/sweeps.js'
import { assignmentRows, poolRows, rosterRows } from './hostRows.test-support.js'
@@ -71,7 +72,7 @@ async function storedState(tab: string): Promise<SuperTabState> {
}
/** Replaces a Swarm's parts with `state`, as an older release or the migration would have left them. Versions keep rising. */
async function saveState(tab: string, state: SuperTabState) {
- await db.$transaction(async tx => {
+ await swarmTransaction(db, async tx => {
const current = await parts.loadAll(tx, tab)
const version = Math.max(current.head.version, ...[...current.members, ...current.agents, ...current.groups, ...current.machines,
...current.starts, ...current.progress].map(p => p.version)) + 1
diff --git a/backend/prisma/schema.prisma b/backend/prisma/schema.prisma
index a697123b5..ef79a0351 100644
--- a/backend/prisma/schema.prisma
+++ b/backend/prisma/schema.prisma
@@ -1508,9 +1508,13 @@ model SwarmMachine {
cleanupOwnerIds String[]
/// The last change to its settings or ledger, never its own reading (host reads wake on this, not on changedAt).
hostChangedAt DateTime?
+ /// The last write of its own reading alone (a report every 15 s): readers of changes see it, while `changedAt`, and so the
+ /// Swarm's newest change, stays at the last change of anything else (Stability Roadmap 2.2).
+ readAt DateTime?
@@unique([tabId, key])
@@index([tabId, changedAt])
+ @@index([tabId, readAt])
@@index([machineId, deleted])
}
diff --git a/backend/src/config/env.ts b/backend/src/config/env.ts
index 63f1d0652..b1d9bfe69 100644
--- a/backend/src/config/env.ts
+++ b/backend/src/config/env.ts
@@ -22,6 +22,11 @@ const envSchema = z.object({
// manager socket may land on different backend instances; Redis pub/sub bridges them
// (channels up:{machineId} / down:{machineId}) plus an agent→manager presence key. See lib/bus.ts.
REDIS_URL: z.string().default('redis://localhost:6379'),
+ // What each process keeps of the Swarms it read, by their stamps in Redis (superTabs/cache.ts). 'off' is the kill switch:
+ // every read goes to Mongo again, while writes still bump the stamps, so processes that keep theirs on stay right.
+ SWARM_READ_CACHE: z.enum(['on', 'off']).default('on'),
+ // Its size per process, counted as the JSON size of what it holds.
+ SWARM_CACHE_MB: z.string().default('64').transform(Number),
// This backend instance's stable id (used to route manager-command replies via Redis in a
// multi-instance deploy). Defaults to the pm2 instance index or a per-process fallback.
diff --git a/backend/src/lib/bus.test.ts b/backend/src/lib/bus.test.ts
index 3126bb45f..b8bdde064 100644
--- a/backend/src/lib/bus.test.ts
+++ b/backend/src/lib/bus.test.ts
@@ -219,3 +219,26 @@ describe('redisCount', () => {
await expect(bus.redisCount('k', 1_000)).rejects.toThrow('redis count failed')
})
})
+
+describe('Swarm stamps', () => {
+ beforeEach(() => { pub.publish.mockClear() })
+
+ it('broadcasts which Swarm moved and which stamps, and drops the broadcast while Redis is down', async () => {
+ await bus.publishSwarmStamp('tab-1', ['main', 'machine'])
+ expect(pub.publish.mock.calls).toEqual([['swarm-stamp', JSON.stringify({ tabId: 'tab-1', scopes: ['main', 'machine'] })]])
+ pub.publish.mockRejectedValueOnce(new Error('ECONNREFUSED'))
+ await expect(bus.publishSwarmStamp('tab-1', ['main'])).resolves.toBe(0)
+ })
+
+ it('hands each broadcast to the listener, and ignores one that is not a stamp', async () => {
+ const sub = state.instances[1] as unknown as { on: MockFn; subscribe: MockFn }
+ const heard: unknown[] = []
+ await bus.subscribeSwarmStamps((tabId, scopes) => heard.push([tabId, scopes]))
+ expect(sub.subscribe).toHaveBeenCalledWith('swarm-stamp')
+ const deliver = sub.on.mock.calls.find((c: unknown[]) => c[0] === 'message')![1] as (channel: string, payload: string) => void
+ deliver('swarm-stamp', JSON.stringify({ tabId: 'tab-2', scopes: ['machine'] }))
+ deliver('swarm-stamp', JSON.stringify({ tabId: 7 }))
+ deliver('swarm-stamp', JSON.stringify({ tabId: 'tab-3', scopes: ['everything'] }))
+ expect(heard).toEqual([['tab-2', ['machine']]])
+ })
+})
diff --git a/backend/src/lib/bus.ts b/backend/src/lib/bus.ts
index 3414faca3..f2d91c2db 100644
--- a/backend/src/lib/bus.ts
+++ b/backend/src/lib/bus.ts
@@ -19,6 +19,7 @@ import { Redis } from 'ioredis'
import { env } from '../config/env.js'
import { logger } from '../utils/logger.js'
import type { Frame, UpBusMsg, DownBusMsg } from './tunnel.js'
+import { redisStamps, type SwarmStamps } from '../superTabs/cache.js'
// A connection in subscriber mode must never fail its (re)SUBSCRIBE commands, so the sub side keeps
// ioredis' "retry forever" setting.
@@ -234,6 +235,29 @@ export function publishTabRelay(share: string, machineId: string, msg: unknown):
return safePublish(`amsg:${share}:${machineId}`, JSON.stringify(msg))
}
+/** A Swarm's stamp moved (superTabs/cache.ts): every instance frees what it holds of the Swarm at the old stamp. One
+ * channel for every Swarm; a broadcast lost while Redis is down costs nothing, since each read checks the stamp itself. */
+const SWARM_STAMP = 'swarm-stamp'
+type StampScopes = Array<'main' | 'machine'>
+export function publishSwarmStamp(tabId: string, scopes: readonly ('main' | 'machine')[]): Promise<number> {
+ return safePublish(SWARM_STAMP, JSON.stringify({ tabId, scopes }))
+}
+export function subscribeSwarmStamps(cb: (tabId: string, scopes: StampScopes) => void): Promise<() => void> {
+ return addSub(SWARM_STAMP, (msg: unknown) => {
+ const { tabId, scopes } = (msg ?? {}) as { tabId?: unknown; scopes?: unknown }
+ if (typeof tabId !== 'string' || !Array.isArray(scopes) || !scopes.every(s => s === 'main' || s === 'machine')) return
+ cb(tabId, scopes as StampScopes)
+ })
+}
+
+/** The Swarm stamps on this Redis (superTabs/cache.ts), their bumps broadcast on the bus. */
+export function redisSwarmStamps(): SwarmStamps {
+ return redisStamps(pub, { publish: publishSwarmStamp, subscribe: listener => {
+ const subscribed = subscribeSwarmStamps(listener).catch((err: unknown) => { logger.error('[bus] Swarm stamp subscribe failed', err); return () => {} })
+ return () => { void subscribed.then(stop => stop()) }
+ } })
+}
+
/** A machine's sharing changed (routes/machineShares.ts): a session another account holds to it on any
* instance asks again whether it may stay (lib/webWs.ts). No content on this channel. */
export function subscribeMachineShareChanged(machineId: string, cb: () => void): Promise<() => void> {
diff --git a/backend/src/server.ts b/backend/src/server.ts
index 6677faa06..8ed6b7ce4 100644
--- a/backend/src/server.ts
+++ b/backend/src/server.ts
@@ -19,9 +19,10 @@ import { handleAdapterUpgrade } from './lib/adapterWs.js'
import { logger } from './utils/logger.js'
import { registerSecurityHeaders } from './middlewares/securityHeaders.js'
import { startTurnCredentialRefresh } from './lib/turnCredentials.js'
-import { closeBus, pub, redisCount, redisSsoProfileStore } from './lib/bus.js'
+import { closeBus, pub, redisCount, redisSsoProfileStore, redisSwarmStamps } from './lib/bus.js'
import { prisma } from './lib/prisma.js'
import { backendReadinessChecks, createReadiness, drainBeforeClosing, processDrain } from './lib/readiness.js'
+import { configureSwarmStamps } from './superTabs/cache.js'
import { useSharedSsoProfileStore } from './lib/ssoAuth.js'
import { selfProfileFromEnv } from './lib/selfProfile.js'
import { drainAllSockets, openSocketCount, RELEASE_UPGRADE_SLOT, type SlotSocket } from './lib/wsServer.js'
@@ -143,6 +144,7 @@ async function start(): Promise<void> {
// Every process shares one memory of validated tokens; without it each worker asks the profile API
// for itself. Attached before anything can authenticate — the REST gate below and the socket upgrades.
useSharedSsoProfileStore(redisSsoProfileStore)
+ configureSwarmStamps(redisSwarmStamps(), { reads: env.SWARM_READ_CACHE === 'on', maxBytes: env.SWARM_CACHE_MB * 2 ** 20 })
// SSO access-token gate for the control API (data-plane auth happens before Fastify).
registerAuthMiddleware(app)
diff --git a/backend/src/superTabs/cache.test-support.ts b/backend/src/superTabs/cache.test-support.ts
new file mode 100644
index 000000000..d6ef009b3
--- /dev/null
+++ b/backend/src/superTabs/cache.test-support.ts
@@ -0,0 +1,33 @@
+import type { Stamp, StampScope, SwarmStamps } from './cache.js'
+
+/**
+ * Redis's stamps (cache.ts redisStamps) in this process, for tests: one store of stamps that every "pod" shares, as they
+ * share Redis. `down` makes every call fail, as an unreachable Redis does; `slowMs` makes reads answer late.
+ */
+export class MemoryStamps implements SwarmStamps {
+ down = false
+ slowMs = 0
+ readonly bumps: Array<{ tabId: string; scopes: StampScope[] }> = []
+ private readonly stamps = new Map<string, Stamp>()
+ private readonly listeners = new Set<(tabId: string, scopes: readonly StampScope[]) => void>()
+
+ async read(tabIds: readonly string[]): Promise<Map<string, Stamp>> {
+ if (this.down) throw new Error('Redis is unreachable')
+ if (this.slowMs) await new Promise(resolve => setTimeout(resolve, this.slowMs))
+ return new Map(tabIds.map(id => [id, { ...this.stamps.get(id) ?? { main: 0, machine: 0 } }]))
+ }
+
+ async bump(tabId: string, scopes: readonly StampScope[]): Promise<void> {
+ if (this.down) throw new Error('Redis is unreachable')
+ const stamp = { ...this.stamps.get(tabId) ?? { main: 0, machine: 0 } }
+ for (const scope of scopes) stamp[scope]++
+ this.stamps.set(tabId, stamp)
+ this.bumps.push({ tabId, scopes: [...scopes] })
+ for (const listener of this.listeners) listener(tabId, scopes)
+ }
+
+ subscribe(listener: (tabId: string, scopes: readonly StampScope[]) => void): () => void {
+ this.listeners.add(listener)
+ return () => this.listeners.delete(listener)
+ }
+}
diff --git a/backend/src/superTabs/cache.test.ts b/backend/src/superTabs/cache.test.ts
new file mode 100644
index 000000000..1544f15a1
--- /dev/null
+++ b/backend/src/superTabs/cache.test.ts
@@ -0,0 +1,142 @@
+import { describe, expect, it } from 'vitest'
+import { SwarmCache } from './cache.js'
+import { MemoryStamps } from './cache.test-support.js'
+
+/** A loader that counts its calls and returns a fresh object each time, numbered. */
+function counted<T extends object>(make: (n: number) => T) {
+ const loader = async () => make(++loader.calls)
+ loader.calls = 0
+ return loader
+}
+
+describe('SwarmCache', () => {
+ it('answers a read at an unchanged stamp from memory', async () => {
+ const cache = new SwarmCache(new MemoryStamps())
+ const load = counted(n => ({ n }))
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ n: 1 })
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ n: 1 })
+ expect(load.calls).toBe(1)
+ })
+
+ it('loads again once the Swarm was bumped, and keeps other Swarms', async () => {
+ const stamps = new MemoryStamps(), cache = new SwarmCache(stamps)
+ const one = counted(n => ({ n })), two = counted(n => ({ n }))
+ await (await cache.at(['t1'])).get('t1', 'base', 'main', one)
+ await (await cache.at(['t2'])).get('t2', 'base', 'main', two)
+ await stamps.bump('t1', ['main'])
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', one)).toEqual({ n: 2 })
+ expect(await (await cache.at(['t2'])).get('t2', 'base', 'main', two)).toEqual({ n: 1 })
+ })
+
+ it('keeps a load under the stamp read before it, so a bump during the load is not missed', async () => {
+ const stamps = new MemoryStamps(), cache = new SwarmCache(stamps)
+ let calls = 0
+ const load = async () => { calls++; if (calls === 1) await stamps.bump('t1', ['main']); return { calls } }
+ await (await cache.at(['t1'])).get('t1', 'base', 'main', load)
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ calls: 2 })
+ })
+
+ it('reads the Swarm directly, and keeps nothing, while the stamps cannot be read', async () => {
+ const stamps = new MemoryStamps(), cache = new SwarmCache(stamps)
+ const load = counted(n => ({ n }))
+ stamps.down = true
+ const view = await cache.at(['t1'])
+ expect(view.available).toBe(false)
+ expect(await view.get('t1', 'base', 'main', load)).toEqual({ n: 1 })
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ n: 2 })
+ expect(cache.size).toEqual({ entries: 0, bytes: 0 })
+ stamps.down = false
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ n: 3 })
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ n: 3 })
+ })
+
+ it('reads directly when the stamps answer too late', async () => {
+ const stamps = new MemoryStamps(), cache = new SwarmCache(stamps, { stampTimeoutMs: 20 })
+ stamps.slowMs = 200
+ expect((await cache.at(['t1'])).available).toBe(false)
+ })
+
+ it('keeps a main-scoped entry over a machine report, and reloads one that covers machines', async () => {
+ const stamps = new MemoryStamps(), cache = new SwarmCache(stamps)
+ const main = counted(n => ({ n })), all = counted(n => ({ n }))
+ const read = async () => { const view = await cache.at(['t1']); return [await view.get('t1', 'head', 'main', main), await view.get('t1', 'machines', 'all', all)] }
+ await read()
+ await stamps.bump('t1', ['machine'])
+ expect(await read()).toEqual([{ n: 1 }, { n: 2 }])
+ await stamps.bump('t1', ['main'])
+ expect(await read()).toEqual([{ n: 2 }, { n: 3 }])
+ })
+
+ it('forgets an entry older than its maximum age, whatever the stamp', async () => {
+ let now = 1_000
+ const cache = new SwarmCache(new MemoryStamps(), { maxAgeMs: 60_000, now: () => now })
+ const load = counted(n => ({ n }))
+ await (await cache.at(['t1'])).get('t1', 'base', 'main', load)
+ now += 59_999
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ n: 1 })
+ now += 1
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ n: 2 })
+ })
+
+ it('stays within its byte budget by dropping the least recently used entry', async () => {
+ // Each value is about 1 KB as JSON.
+ const cache = new SwarmCache(new MemoryStamps(), { maxBytes: 2_500 })
+ const value = (n: number) => ({ text: 'x'.repeat(1_000), n })
+ const loads = { a: counted(value), b: counted(value), c: counted(value) }
+ const get = async (tab: 'a' | 'b' | 'c') => (await cache.at([tab])).get(tab, 'base', 'main', loads[tab])
+ await get('a'); await get('b'); await get('a'); await get('c')
+ expect(cache.size.entries).toBe(2)
+ expect(cache.size.bytes).toBeLessThanOrEqual(2_500)
+ await get('a'); await get('c'); await get('b')
+ expect([loads.a.calls, loads.b.calls, loads.c.calls]).toEqual([1, 2, 1])
+ })
+
+ it('keeps nothing larger than its whole budget', async () => {
+ const cache = new SwarmCache(new MemoryStamps(), { maxBytes: 100 })
+ await (await cache.at(['t1'])).get('t1', 'base', 'main', async () => ({ text: 'x'.repeat(1_000) }))
+ expect(cache.size).toEqual({ entries: 0, bytes: 0 })
+ })
+
+ it('hands every reader the same frozen value, so no reader can change what another reads', async () => {
+ const cache = new SwarmCache(new MemoryStamps())
+ const load = async () => ({ people: [{ id: 'p1', role: 'member' }] })
+ const first = await (await cache.at(['t1'])).get('t1', 'base', 'main', load)
+ expect(() => { (first.people[0] as { role: string }).role = 'owner' }).toThrow(TypeError)
+ expect(() => { first.people.push({ id: 'p2', role: 'owner' }) }).toThrow(TypeError)
+ expect(await (await cache.at(['t1'])).get('t1', 'base', 'main', load)).toEqual({ people: [{ id: 'p1', role: 'member' }] })
+ })
+
+ it('loads once for readers that miss together', async () => {
+ const cache = new SwarmCache(new MemoryStamps())
+ let calls = 0
+ const load = async () => { calls++; await new Promise(resolve => setTimeout(resolve, 10)); return { calls } }
+ const views = await Promise.all([cache.at(['t1']), cache.at(['t1'])])
+ expect(await Promise.all(views.map(view => view.get('t1', 'base', 'main', load)))).toEqual([{ calls: 1 }, { calls: 1 }])
+ })
+
+ it('frees what it holds of a Swarm when any pod bumps it', async () => {
+ const stamps = new MemoryStamps(), cache = new SwarmCache(stamps)
+ await (await cache.at(['t1'])).get('t1', 'base', 'main', async () => ({ text: 'x'.repeat(100) }))
+ expect(cache.size.entries).toBe(1)
+ await stamps.bump('t1', ['main'])
+ expect(cache.size).toEqual({ entries: 0, bytes: 0 })
+ })
+
+ it('reads the stamps of several Swarms at once', async () => {
+ const stamps = new MemoryStamps(), cache = new SwarmCache(stamps)
+ const one = counted(n => ({ n })), two = counted(n => ({ n }))
+ let view = await cache.at(['t1', 't2'])
+ await view.get('t1', 'inputs', 'main', one); await view.get('t2', 'inputs', 'main', two)
+ await stamps.bump('t2', ['main'])
+ view = await cache.at(['t1', 't2'])
+ expect([await view.get('t1', 'inputs', 'main', one), await view.get('t2', 'inputs', 'main', two)]).toEqual([{ n: 1 }, { n: 2 }])
+ })
+
+ it('reads directly for a Swarm the view did not read the stamp of', async () => {
+ const cache = new SwarmCache(new MemoryStamps())
+ const load = counted(n => ({ n }))
+ const view = await cache.at(['t1'])
+ await view.get('t2', 'base', 'main', load); await view.get('t2', 'base', 'main', load)
+ expect(load.calls).toBe(2)
+ })
+})
diff --git a/backend/src/superTabs/cache.ts b/backend/src/superTabs/cache.ts
new file mode 100644
index 000000000..1553676a9
--- /dev/null
+++ b/backend/src/superTabs/cache.ts
@@ -0,0 +1,291 @@
+/**
+ * What each backend process keeps of the Swarms it read, by stamp (Stability Roadmap 2.2; Design in
+ * docs/design/2026-10-28-swarm-read-cache.md). Every commit that changes a Swarm's parts bumps its stamp in Redis after it
+ * commits; a read takes the stamp before it loads, and serves what it holds at that stamp without asking Mongo. A
+ * machine's own reading moves only the machine stamp, so a report every 15 s leaves the rest of the Swarm cached.
+ *
+ * What is held is the stored state, for every reader alike: each read applies its reader's permissions after the cache
+ * (read/views.ts readerView), and what is held is frozen so no reader can change what the next one reads.
+ */
+
+import type { Prisma, PrismaClient } from '@prisma/client'
+import { logger } from '../utils/logger.js'
+
+/** The stamps of one Swarm. A stamp only grows; one that never moved is 0. */
+export interface Stamp { main: number; machine: number }
+/** `main`: any change of a Swarm's parts. `machine`: a change of a machine part's own reading only. */
+export type StampScope = keyof Stamp
+
+/** Where every process reads and bumps the stamps: Redis in production (redisStamps). */
+export interface SwarmStamps {
+ read(tabIds: readonly string[]): Promise<Map<string, Stamp>>
+ bump(tabId: string, scopes: readonly StampScope[]): Promise<void>
+ /** Called back when any process bumps a Swarm, so this one frees what it holds of it early. */
+ subscribe(listener: (tabId: string, scopes: readonly StampScope[]) => void): () => void
+}
+
+interface Entry { stamp: Stamp; scope: StampScope | 'all'; at: number; bytes: number; value: unknown }
+
+export interface CacheOptions {
+ /** What the held values may take, counted as their JSON size. */
+ maxBytes?: number
+ /** A safety net: an entry is read again after this long whatever its stamp (a bump lost to a crash after commit). */
+ maxAgeMs?: number
+ /** A read that cannot learn the stamps in this time reads Mongo itself. */
+ stampTimeoutMs?: number
+ now?: () => number
+}
+
+export const CACHE_MAX_BYTES = 64 * 1024 * 1024
+export const CACHE_MAX_AGE_MS = 60_000
+export const STAMP_TIMEOUT_MS = 200
+
+const key = (tabId: string, name: string) => `${tabId}\u0000${name}`
+
+/** Freezes a loaded value all through and tells its size as JSON, in one pass. */
+export function freeze(value: unknown): number {
+ if (value === null || value === undefined) return 4
+ switch (typeof value) {
+ case 'string': return value.length + 2
+ case 'number': case 'boolean': case 'bigint': return 8
+ case 'object': {
+ if (value instanceof Date) return 26
+ let bytes = 2
+ if (Array.isArray(value)) for (const item of value) bytes += freeze(item) + 1
+ else for (const [name, item] of Object.entries(value)) bytes += name.length + 4 + freeze(item)
+ Object.freeze(value)
+ return bytes
+ }
+ default: return 8
+ }
+}
+
+export class SwarmCache {
+ private readonly entries = new Map<string, Entry>()
+ private readonly loading = new Map<string, Promise<unknown>>()
+ private bytes = 0
+ private readonly maxBytes: number
+ private readonly maxAgeMs: number
+ private readonly stampTimeoutMs: number
+ private readonly now: () => number
+
+ constructor(private readonly stamps: SwarmStamps, options: CacheOptions = {}) {
+ this.maxBytes = options.maxBytes ?? CACHE_MAX_BYTES
+ this.maxAgeMs = options.maxAgeMs ?? CACHE_MAX_AGE_MS
+ this.stampTimeoutMs = options.stampTimeoutMs ?? STAMP_TIMEOUT_MS
+ this.now = options.now ?? Date.now
+ stamps.subscribe((tabId, scopes) => this.drop(tabId, scopes))
+ }
+
+ get size() { return { entries: this.entries.size, bytes: this.bytes } }
+
+ /** One read's view of these Swarms: their stamps, read once, before anything is loaded. */
+ async at(tabIds: readonly string[]): Promise<CacheView> {
+ let timer: NodeJS.Timeout | undefined
+ const late = new Promise<null>(resolve => { timer = setTimeout(() => resolve(null), this.stampTimeoutMs) })
+ try {
+ return new CacheView(this, await Promise.race([this.stamps.read(tabIds), late]))
+ } catch {
+ return new CacheView(this, null)
+ } finally {
+ clearTimeout(timer)
+ }
+ }
+
+ /** Forgets what this process holds of a Swarm that these stamps' moves make stale; everything, by default. */
+ drop(tabId: string, scopes: readonly StampScope[] = ['main']): void {
+ const prefix = key(tabId, ''), main = scopes.includes('main')
+ for (const [name, entry] of this.entries) {
+ if (!name.startsWith(prefix) || !main && entry.scope === 'main') continue
+ this.entries.delete(name)
+ this.bytes -= entry.bytes
+ }
+ }
+
+ /** @internal CacheView's lookup. */
+ held(tabId: string, name: string, stamp: Stamp): unknown {
+ const entry = this.entries.get(key(tabId, name))
+ if (!entry) return undefined
+ const fresh = entry.stamp.main === stamp.main && (entry.scope === 'main' || entry.stamp.machine === stamp.machine) && this.now() - entry.at < this.maxAgeMs
+ if (!fresh) {
+ this.entries.delete(key(tabId, name))
+ this.bytes -= entry.bytes
+ return undefined
+ }
+ // Most recently used last.
+ this.entries.delete(key(tabId, name))
+ this.entries.set(key(tabId, name), entry)
+ return entry.value
+ }
+
+ /** @internal CacheView's load: one load for every reader that misses the same entry at the same stamp. */
+ async load<T>(tabId: string, name: string, scope: StampScope | 'all', stamp: Stamp, load: () => Promise<T>): Promise<T> {
+ const flight = `${key(tabId, name)}\u0000${stamp.main}\u0000${scope === 'main' ? '' : stamp.machine}`
+ const running = this.loading.get(flight) as Promise<T> | undefined
+ if (running) return running
+ const at = this.now()
+ const loaded = load().then(value => {
+ const bytes = freeze(value)
+ if (bytes <= this.maxBytes) {
+ this.drop1(key(tabId, name))
+ this.entries.set(key(tabId, name), { stamp, scope, at, bytes, value })
+ this.bytes += bytes
+ for (const [oldest, entry] of this.entries) {
+ if (this.bytes <= this.maxBytes) break
+ this.entries.delete(oldest)
+ this.bytes -= entry.bytes
+ }
+ }
+ return value
+ }).finally(() => this.loading.delete(flight))
+ this.loading.set(flight, loaded)
+ return loaded
+ }
+
+ private drop1(name: string) {
+ const entry = this.entries.get(name)
+ if (!entry) return
+ this.entries.delete(name)
+ this.bytes -= entry.bytes
+ }
+}
+
+/** One read's stamps. Without them (Redis unreachable or late) every get reads Mongo, and nothing is kept. */
+export class CacheView {
+ constructor(private readonly cache: SwarmCache | null, private readonly stamps: Map<string, Stamp> | null) {}
+
+ get available(): boolean { return this.stamps !== null }
+
+ /** The value `load` gives for this Swarm at the view's stamp: held, or loaded and held. `scope` says which stamps it
+ * depends on: 'main' for what a machine's reading never changes, 'all' for anything holding machine parts. */
+ async get<T>(tabId: string, name: string, scope: StampScope | 'all', load: () => Promise<T>): Promise<T> {
+ const stamp = this.stamps?.get(tabId)
+ if (!stamp || !this.cache) return load()
+ const held = this.cache.held(tabId, name, stamp)
+ if (held !== undefined) return held as T
+ return this.cache.load(tabId, name, scope, stamp, load)
+ }
+}
+
+/** A stamp kept this long after its Swarm's last change; a Swarm read after that reads 0, then starts again from the time. */
+export const STAMP_TTL_MS = 30 * 86_400_000
+const stampKey = (tabId: string) => `swarm:stamp:${tabId}`
+/**
+ * A Swarm's first bump starts both stamps from Redis's clock in ms, then moves the named ones by one. Redis losing the key
+ * (a restart, the expiry) therefore never hands out a stamp a process may still hold: it starts again above it, as long as
+ * a Swarm has fewer commits than milliseconds between its first and its loss.
+ */
+const BUMP = `if redis.call('exists', KEYS[1]) == 0 then
+ local t = redis.call('time')
+ local base = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)
+ redis.call('hset', KEYS[1], 'main', base - 1, 'machine', base - 1)
+end
+for i = 2, #ARGV do redis.call('hincrby', KEYS[1], ARGV[i], 1) end
+redis.call('pexpire', KEYS[1], ARGV[1])
+return 1`
+
+/** What redisStamps needs of an ioredis connection that is not in subscribe mode (lib/bus.ts `pub`). */
+export interface StampRedis {
+ eval(script: string, keys: number, ...args: Array<string | number>): Promise<unknown>
+ hmget(key: string, ...fields: string[]): Promise<Array<string | null>>
+ pipeline(): { hmget(key: string, ...fields: string[]): unknown; exec(): Promise<Array<[Error | null, unknown]> | null> }
+}
+/** The bus's broadcast of bumps (lib/bus.ts publishSwarmStamp, subscribeSwarmStamps). */
+export interface StampBus {
+ publish(tabId: string, scopes: readonly StampScope[]): Promise<unknown>
+ subscribe(listener: (tabId: string, scopes: readonly StampScope[]) => void): () => void
+}
+
+const stampOf = (fields: Array<string | null>): Stamp => ({ main: Number(fields[0] ?? 0), machine: Number(fields[1] ?? 0) })
+
+export function redisStamps(redis: StampRedis, bus: StampBus): SwarmStamps {
+ return {
+ async read(tabIds) {
+ if (tabIds.length === 1) return new Map([[tabIds[0]!, stampOf(await redis.hmget(stampKey(tabIds[0]!), 'main', 'machine'))]])
+ const pipeline = redis.pipeline()
+ for (const id of tabIds) pipeline.hmget(stampKey(id), 'main', 'machine')
+ const replies = await pipeline.exec() ?? []
+ return new Map(tabIds.map((id, i) => {
+ const [error, fields] = replies[i] ?? [new Error('No reply')]
+ if (error) throw error
+ return [id, stampOf(fields as Array<string | null>)]
+ }))
+ },
+ async bump(tabId, scopes) {
+ await redis.eval(BUMP, 1, stampKey(tabId), STAMP_TTL_MS, ...scopes)
+ await bus.publish(tabId, scopes)
+ },
+ subscribe: listener => bus.subscribe(listener),
+ }
+}
+
+// ── The process's stamps ─────────────────────────────────────────────────────────────────────────────────────────────
+
+let stamps: SwarmStamps | null = null
+let readCache: SwarmCache | null = null
+
+/**
+ * The stamps every Swarm write of this process bumps, and the read cache over them unless `reads` is false (the kill
+ * switch, SWARM_READ_CACHE=off: reads go to Mongo, writes still bump, so other processes' caches stay right). Without
+ * stamps (null) nothing is bumped or cached.
+ */
+export function configureSwarmStamps(next: SwarmStamps | null, options: CacheOptions & { reads?: boolean } = {}): void {
+ stamps = next
+ readCache = next && options.reads !== false ? new SwarmCache(next, options) : null
+}
+export const swarmReadCache = (): SwarmCache | null => readCache
+
+/** The Swarms a transaction changed, by stamp scope: bumped once it commits (`swarmTransaction`). */
+type Changes = Map<string, Set<StampScope>>
+const changesOf = new WeakMap<object, Changes>()
+
+/** A part write in this transaction changed this Swarm (parts/write.ts). */
+export function noteChange(tx: object, tabId: string, scope: StampScope): void {
+ const changes = changesOf.get(tx)
+ if (!changes) return
+ const scopes = changes.get(tabId) ?? new Set<StampScope>()
+ scopes.add(scope)
+ changes.set(tabId, scopes)
+}
+
+/**
+ * One database transaction that writes Swarm parts: once it commits, every Swarm it changed is bumped, before the caller
+ * answers, so any read after the answer, on any process, sees the change. A bump Redis cannot take is owed: this process
+ * forgets the Swarm and tries again every second until Redis takes it (other processes hold the Swarm until then, at most
+ * CACHE_MAX_AGE_MS). A transaction that fails bumps nothing.
+ */
+export async function swarmTransaction<T>(db: Pick<PrismaClient, '$transaction'>, body: (tx: Prisma.TransactionClient) => Promise<T>,
+ options?: { maxWait?: number; timeout?: number }): Promise<T> {
+ const changes: Changes = new Map()
+ const result = await db.$transaction(tx => { changesOf.set(tx, changes); return body(tx) }, options)
+ await announce(changes)
+ return result
+}
+
+const owed: Changes = new Map()
+let paying: NodeJS.Timeout | null = null
+export const OWED_RETRY_MS = 1_000
+
+async function announce(changes: Changes): Promise<void> {
+ if (!stamps || !changes.size) return
+ const current = stamps
+ await Promise.all([...changes].map(async ([tabId, scopes]) => {
+ try { await current.bump(tabId, [...scopes]) } catch (error) {
+ readCache?.drop(tabId)
+ if (!owed.size) logger.error('[swarm-cache] stamp bump failed after commit; retrying every second', error, { tabId })
+ const due = owed.get(tabId) ?? new Set<StampScope>()
+ for (const scope of scopes) due.add(scope)
+ owed.set(tabId, due)
+ paying ??= setTimeout(payOwed, OWED_RETRY_MS)
+ paying.unref?.()
+ }
+ }))
+}
+
+async function payOwed(): Promise<void> {
+ paying = null
+ const due = [...owed]
+ owed.clear()
+ await announce(new Map(due))
+ if (!owed.size && due.length) logger.info('[swarm-cache] owed stamp bumps paid', { swarms: due.length })
+}
diff --git a/backend/src/superTabs/cacheRedis.test.ts b/backend/src/superTabs/cacheRedis.test.ts
new file mode 100644
index 000000000..9317a288a
--- /dev/null
+++ b/backend/src/superTabs/cacheRedis.test.ts
@@ -0,0 +1,75 @@
+import { randomUUID } from 'node:crypto'
+import { Redis } from 'ioredis'
+import { afterAll, beforeAll, describe, expect, it } from 'vitest'
+import { redisStamps, STAMP_TTL_MS, type StampScope } from './cache.js'
+
+// A real Redis: REDIS_TEST_URL, e.g. redis://127.0.0.1:6379/15 (CI has none, so this file is skipped there). Only keys of
+// this run are touched, and they are removed afterwards.
+const url = process.env.REDIS_TEST_URL
+
+describe.skipIf(!url)('redisStamps', () => {
+ let redis: Redis
+ const tab = () => `stamp-test-${randomUUID()}`
+ const used: string[] = []
+ const published: Array<{ tabId: string; scopes: readonly StampScope[] }> = []
+ const bus = { publish: async (tabId: string, scopes: readonly StampScope[]) => { published.push({ tabId, scopes }) }, subscribe: () => () => {} }
+
+ beforeAll(async () => {
+ redis = new Redis(url!, { maxRetriesPerRequest: 1, enableOfflineQueue: false })
+ await new Promise(resolve => redis.once('ready', resolve))
+ })
+ afterAll(async () => {
+ if (used.length) await redis.del(...used.map(id => `swarm:stamp:${id}`))
+ await redis.quit()
+ })
+
+ it('reads 0 for a Swarm never bumped', async () => {
+ const id = tab()
+ expect(await redisStamps(redis, bus).read([id])).toEqual(new Map([[id, { main: 0, machine: 0 }]]))
+ })
+
+ it('starts a Swarm from the time, so a Redis that lost its keys never hands out an old stamp again', async () => {
+ const stamps = redisStamps(redis, bus), id = tab()
+ used.push(id)
+ const before = Date.now()
+ await stamps.bump(id, ['main'])
+ const first = (await stamps.read([id])).get(id)!
+ expect(first.main).toBeGreaterThan(before - 60_000)
+ expect(first.machine).toBe(first.main - 1)
+ await stamps.bump(id, ['main']); await stamps.bump(id, ['main'])
+ const old = (await stamps.read([id])).get(id)!.main
+ await redis.del(`swarm:stamp:${id}`)
+ await stamps.bump(id, ['main'])
+ expect((await stamps.read([id])).get(id)!.main).toBeGreaterThan(old)
+ })
+
+ it('moves only the scopes named, by one each', async () => {
+ const stamps = redisStamps(redis, bus), id = tab()
+ used.push(id)
+ await stamps.bump(id, ['main'])
+ const start = (await stamps.read([id])).get(id)!
+ await stamps.bump(id, ['machine'])
+ expect((await stamps.read([id])).get(id)).toEqual({ main: start.main, machine: start.machine + 1 })
+ await stamps.bump(id, ['main', 'machine'])
+ expect((await stamps.read([id])).get(id)).toEqual({ main: start.main + 1, machine: start.machine + 2 })
+ })
+
+ it('reads several Swarms in one call, and lets a quiet Swarm\'s stamp expire', async () => {
+ const stamps = redisStamps(redis, bus), a = tab(), b = tab()
+ used.push(a, b)
+ await stamps.bump(a, ['main'])
+ const read = await stamps.read([a, b])
+ expect(read.get(a)!.main).toBeGreaterThan(0)
+ expect(read.get(b)).toEqual({ main: 0, machine: 0 })
+ const ttl = await redis.pttl(`swarm:stamp:${a}`)
+ expect(ttl).toBeGreaterThan(STAMP_TTL_MS - 60_000)
+ expect(ttl).toBeLessThanOrEqual(STAMP_TTL_MS)
+ })
+
+ it('tells the other processes which Swarm moved, and which stamps', async () => {
+ const stamps = redisStamps(redis, bus), id = tab()
+ used.push(id)
+ await stamps.bump(id, ['machine'])
+ expect(published.at(-1)).toEqual({ tabId: id, scopes: ['machine'] })
+ })
+})
diff --git a/backend/src/superTabs/cachedReads.ts b/backend/src/superTabs/cachedReads.ts
new file mode 100644
index 000000000..707853903
--- /dev/null
+++ b/backend/src/superTabs/cachedReads.ts
@@ -0,0 +1,84 @@
+/**
+ * Reads of a Swarm's parts through the per-process cache (cache.ts, Stability Roadmap 2.2): each read gives what the
+ * loader it replaces gives (parts/load.ts loadActive, parts/light.ts loadLight, loadChanged, lastChangedAt), built from
+ * what the cache holds at the read's stamps. Without stamps (Redis down, the cache off) each piece is read from Mongo.
+ * Everything here is the stored state, the same for every reader: permissions are applied after (read/views.ts).
+ */
+import type { Prisma, PrismaClient } from '@prisma/client'
+import { stableId, type AgentStart } from './model.js'
+import { CacheView } from './cache.js'
+import * as store from './parts/index.js'
+import type { GroupData, Part, SwarmParts } from './parts/index.js'
+import { startFinished } from './parts/write.js'
+
+type Db = Pick<PrismaClient, '$transaction'> & Prisma.TransactionClient
+type LightOptions = NonNullable<Parameters<typeof store.loadLight>[2]>
+
+/** What a read names beyond a Swarm's active parts, as loadActive takes it. */
+export interface ReadNamed { groupIds?: string[]; agentIds?: string[]; projectIds?: string[]; startIds?: string[]; startAgentIds?: string[]
+ allGroups?: boolean; includeRecentStarts?: boolean }
+
+/** A view with no stamps: every read goes to Mongo, nothing is kept. */
+export const DIRECT = new CacheView(null, null)
+
+/** A miss loads the feed from this long before its reader's cursor, so readers polling a few seconds apart share it. */
+const FEED_SLACK_MS = 15_000
+
+/** A Swarm's live machine parts, held at both stamps: a machine's reading moves only the machine stamp. */
+export const readMachines = (db: Prisma.TransactionClient, view: CacheView, tabId: string) =>
+ view.get(tabId, 'machines', 'all', () => store.loadMachines(db, tabId))
+
+/** The active parts, then the machines: one read of the base in a transaction, so its parts are one snapshot. */
+const base = (db: Db, view: CacheView, tabId: string) => Promise.all([
+ view.get(tabId, 'parts', 'main', () => db.$transaction(tx => store.loadReadBase(tx, tabId))),
+ readMachines(db, view, tabId),
+])
+
+/** The parts loadActive gives for `named`, as one Swarm read (design section 3.1). */
+export async function readParts(db: Db, view: CacheView, tabId: string, named: ReadNamed = {}): Promise<SwarmParts> {
+ const [active, machines] = await base(db, view, tabId)
+ const held = new Set(active.groups.map(g => g.key))
+ const extra: Part<GroupData>[] = []
+ const add = (groups: Part<GroupData>[]) => { for (const g of groups) if (!held.has(g.key)) { held.add(g.key); extra.push(g) } }
+ if (named.allGroups || named.agentIds?.length || named.projectIds?.length) {
+ const dormant = await view.get(tabId, 'dormant', 'main', () => store.loadGroups(db, tabId, { dormant: true }))
+ const agents = new Set(named.agentIds ?? []), projects = new Set(named.projectIds ?? [])
+ // A group stored with no project belongs to the default one (projects.ts groupProject).
+ add(named.allGroups ? dormant : dormant.filter(g => g.data!.agentIds.some(id => agents.has(id)) || projects.has(g.data!.projectId ?? 'lead')))
+ }
+ for (const id of named.groupIds ?? []) {
+ if (!held.has(id)) add(await view.get(tabId, `group:${id}`, 'main', () => store.loadGroups(db, tabId, { key: id })))
+ }
+ const startIds = new Set(named.startIds ?? []), startAgents = new Set(named.startAgentIds ?? [])
+ const agentOf = (start: AgentStart) => start.harnessId ? stableId(start.request.machineId, start.harnessId) : null
+ const starts = named.includeRecentStarts ? active.starts : active.starts.filter(s => !startFinished(s.data!) || startIds.has(s.key)
+ || startAgents.has(agentOf(s.data!) ?? ''))
+ return { ...active, machines, groups: extra.length ? [...active.groups, ...extra] : active.groups, starts }
+}
+
+/** The parts loadLight gives for `opts`, held whole by what it selects. */
+export async function readLight(db: Db, view: CacheView, tabId: string, opts: Omit<LightOptions, 'machines'>): Promise<SwarmParts> {
+ const name = `light:${JSON.stringify([opts.groups ?? 'active', !!opts.includeRecentStarts, ...(['groupIds', 'agentIds', 'projectIds'] as const)
+ .map(key => [...opts[key] ?? []].sort())])}`
+ const [light, machines] = await Promise.all([
+ view.get(tabId, name, 'main', () => db.$transaction(tx => store.loadLight(tx, tabId, { ...opts, machines: false }))),
+ readMachines(db, view, tabId),
+ ])
+ return { ...light, machines }
+}
+
+/** The Swarm's newest data change (parts/load.ts lastChangedAt), held at its main stamp: a machine's reading leaves it. */
+export async function readLastChangedAt(db: Db, view: CacheView, tabId: string): Promise<number | null> {
+ return view.get(tabId, 'lastChangedAt', 'main', async () => (await store.lastChangedAt(db, [tabId])).get(tabId) ?? null)
+}
+
+/** The Swarm's parts changed after `since` (parts/load.ts loadChanged): from the held feed when it reaches back that far, else read. */
+export async function readChanged(db: Db, view: CacheView, tabId: string, since: number): Promise<Part[]> {
+ const direct = async () => (await store.loadChanged(db, [tabId], new Date(since))).get(tabId) ?? []
+ if (!view.available) return direct()
+ const feed = await view.get(tabId, 'feed', 'all', async () => {
+ const from = since - FEED_SLACK_MS
+ return { from, parts: (await store.loadChanged(db, [tabId], new Date(from))).get(tabId) ?? [] }
+ })
+ return since >= feed.from ? feed.parts.filter(p => Math.max(p.changedAt, p.readAt ?? 0) > since) : direct()
+}
diff --git a/backend/src/superTabs/migration/migrate.ts b/backend/src/superTabs/migration/migrate.ts
index b53e9b390..93976dbec 100644
--- a/backend/src/superTabs/migration/migrate.ts
+++ b/backend/src/superTabs/migration/migrate.ts
@@ -6,6 +6,7 @@
import type { Prisma, PrismaClient } from '@prisma/client'
import type { SuperTabState } from '../model.js'
import * as parts from '../parts/index.js'
+import { swarmTransaction } from '../cache.js'
import { fingerprint, fingerprintDifference, lastActivity } from './fingerprint.js'
import { restorePullRequestUrls } from './links.js'
import { normalize } from './normalize.js'
@@ -37,7 +38,7 @@ export async function migrateSwarm(db: PrismaClient, tabId: string, { now = Date
const started = Date.now()
const empty = { tabId, bytes: 0, groups: 0, dormant: 0, parts: 0, messages: 0 }
try {
- return await db.$transaction(tx => migrateIn(tx, tabId, now(), started), MIGRATION_TRANSACTION)
+ return await swarmTransaction(db, tx => migrateIn(tx, tabId, now(), started), MIGRATION_TRANSACTION)
} catch (error) {
return { ...empty, ok: false, error: error instanceof Error ? error.message : String(error), ms: Date.now() - started }
}
diff --git a/backend/src/superTabs/migration/rollback.ts b/backend/src/superTabs/migration/rollback.ts
index f4314c682..e299c57fa 100644
--- a/backend/src/superTabs/migration/rollback.ts
+++ b/backend/src/superTabs/migration/rollback.ts
@@ -7,6 +7,7 @@
import type { Prisma, PrismaClient } from '@prisma/client'
import { nextConversationNumber, type Conversation, type SuperTabState } from '../model.js'
import * as parts from '../parts/index.js'
+import { noteChange, swarmTransaction } from '../cache.js'
export interface RollbackReport { tabId: string; ok: boolean; error?: string }
@@ -17,7 +18,8 @@ const fail = (message: string): never => { throw new RollbackFailed(message) }
async function run(db: PrismaClient, tabId: string, body: (tx: Prisma.TransactionClient) => Promise<void>): Promise<RollbackReport> {
try {
- await db.$transaction(body, ROLLBACK_TRANSACTION)
+ // The Swarm leaves its parts: what any process holds of it is stale.
+ await swarmTransaction(db, tx => { noteChange(tx, tabId, 'main'); return body(tx) }, ROLLBACK_TRANSACTION)
return { tabId, ok: true }
} catch (error) {
return { tabId, ok: false, error: error instanceof Error ? error.message : String(error) }
diff --git a/backend/src/superTabs/migration/workspaces.ts b/backend/src/superTabs/migration/workspaces.ts
index 67809d6d0..dbac4bd02 100644
--- a/backend/src/superTabs/migration/workspaces.ts
+++ b/backend/src/superTabs/migration/workspaces.ts
@@ -11,6 +11,7 @@
*/
import type { Prisma, PrismaClient } from '@prisma/client'
import { clock, guardedWrite, PartConflict } from '../parts/index.js'
+import { swarmTransaction } from '../cache.js'
import { addMember, personalWorkspaceId } from '../workspaceStore.js'
import type { Person } from '../model.js'
@@ -94,7 +95,7 @@ export async function migrateWorkspaces(db: PrismaClient, opts: { apply: boolean
report.swarms += swarms.length
if (!opts.apply) continue
try {
- await db.$transaction(async tx => {
+ await swarmTransaction(db, async tx => {
if (!await tx.swarmWorkspace.findUnique({ where: { id: workspaceId } })) {
await tx.swarmWorkspace.create({ data: { id: workspaceId, autonomousEnv: first.autonomousEnv, name: first.name || 'Workspace', ownerId: first.ownerId, deletedAt: null, purgeAt: null } })
}
@@ -109,7 +110,7 @@ export async function migrateWorkspaces(db: PrismaClient, opts: { apply: boolean
}
for (const swarm of swarms) {
try {
- await db.$transaction(tx => stampSwarm(tx, swarm.id, swarm.headVersion, swarm.head, swarm.headFields, workspaceId), TRANSACTION)
+ await swarmTransaction(db, tx => stampSwarm(tx, swarm.id, swarm.headVersion, swarm.head, swarm.headFields, workspaceId), TRANSACTION)
} catch (error) {
report.failed.push({ tabId: swarm.id, error: error instanceof Error ? error.message : String(error) })
}
@@ -141,7 +142,7 @@ export async function rollbackWorkspaces(db: PrismaClient, opts: { apply: boolea
if (!opts.apply) return report
for (const row of rows) {
try {
- await db.$transaction(async tx => {
+ await swarmTransaction(db, async tx => {
const head = await tx.swarmHead.findUniqueOrThrow({ where: { id: `${row.id}:head` } })
await stampSwarm(tx, row.id, head.version, head.data as Prisma.JsonObject,
{ groupCounter: head.groupCounter, visibilityEpoch: head.visibilityEpoch, sequenceBase: head.sequenceBase }, null)
diff --git a/backend/src/superTabs/model.test.ts b/backend/src/superTabs/model.test.ts
new file mode 100644
index 000000000..150d2e936
--- /dev/null
+++ b/backend/src/superTabs/model.test.ts
@@ -0,0 +1,23 @@
+import { describe, expect, it } from 'vitest'
+import { createSuperTab, syncLeads, type Agent, type SuperTabState } from './model.js'
+
+const deepFreeze = <T>(value: T): T => {
+ if (value && typeof value === 'object') { Object.values(value).forEach(deepFreeze); Object.freeze(value) }
+ return value
+}
+
+describe('syncLeads', () => {
+ it('lists the Leads in the Leads conversation without changing the conversation it was given', () => {
+ const base = createSuperTab('tab1', 'Pilot', { id: 'u1', email: '[email protected]' }, 'prod')
+ syncLeads(base)
+ const lead = base.conversations.find(c => !c.kind)!
+ lead.leadId = 'a1'
+ base.agents.push({ id: 'a1' } as Agent)
+ const stored = deepFreeze(structuredClone(base))
+ // A read's state is a new object over frozen stored records (parts/shape.ts reassemble with `share`).
+ const state: SuperTabState = { ...stored, conversations: [...stored.conversations] }
+ syncLeads(state)
+ expect(state.conversations.find(c => c.kind === 'leads')!.agentIds).toEqual(['a1'])
+ expect(stored.conversations.find(c => c.kind === 'leads')!.agentIds).toEqual([])
+ })
+})
diff --git a/backend/src/superTabs/model.ts b/backend/src/superTabs/model.ts
index da93af657..d4af433d5 100644
--- a/backend/src/superTabs/model.ts
+++ b/backend/src/superTabs/model.ts
@@ -435,8 +435,9 @@ export function syncLeads(state: SuperTabState): void {
const id = state.leadsConversationId ??= stableId(state.id, 'leads')
const group = state.conversations.find(c => c.id === id) ?? { id, kind: 'leads', title: 'Leads', leadId: null, agentIds: [],
archived: false, brief: '', checks: [], assignment: null, lastMessageSequence: 0 }
- state.conversations = [...state.conversations.filter(c => c !== group), group]
- group.agentIds = state.agents.filter(a => ledProject(state, a.id)).map(a => a.id)
+ // A new record, not the one found: a read's state shares the stored ones (parts/shape.ts reassemble).
+ state.conversations = [...state.conversations.filter(c => c !== group),
+ { ...group, agentIds: state.agents.filter(a => ledProject(state, a.id)).map(a => a.id) }]
}
/** Legacy groups keep their append order, including archived groups. Persisted
diff --git a/backend/src/superTabs/parts/commit.test.ts b/backend/src/superTabs/parts/commit.test.ts
index 10cdeffb9..9cb3c6a88 100644
--- a/backend/src/superTabs/parts/commit.test.ts
+++ b/backend/src/superTabs/parts/commit.test.ts
@@ -1,6 +1,6 @@
import { describe, expect, it } from 'vitest'
import { createSuperTab } from '../model.js'
-import { capReleased, departedAgents, hostMarker, visibilityChanged } from './commit.js'
+import { capReleased, departedAgents, hostMarker, sameData, visibilityChanged } from './commit.js'
describe('writing a Swarm\'s parts', () => {
it('keeps the newest 200 released harnesses of each machine and drops the oldest', () => {
@@ -86,3 +86,50 @@ describe('agents a group write lets go (review round 2)', () => {
expect(departedAgents(undefined, group(['lead']))).toEqual([])
})
})
+
+describe('whether a part\'s data changed', () => {
+ /** What a stored record says, as JSON with sorted keys: the comparison commits made before (the oracle). */
+ const sorted = (value: unknown) => JSON.stringify(value, (_key, item: unknown) => item && typeof item === 'object' && !Array.isArray(item)
+ ? Object.fromEntries(Object.entries(item as Record<string, unknown>).sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0)) : item)
+
+ it('ignores the order keys were built in, and fields left undefined', () => {
+ expect(sameData({ a: 1, b: { c: [1, 2], d: 'x' } }, { b: { d: 'x', c: [1, 2] }, a: 1 })).toBe(true)
+ expect(sameData({ a: 1, gone: undefined }, { a: 1 })).toBe(true)
+ expect(sameData({ when: new Date(5) }, { when: new Date(5).toISOString() })).toBe(true)
+ })
+
+ it('sees a changed value, a null where nothing was, a reordered list, and a longer one', () => {
+ expect(sameData({ a: { b: 1 } }, { a: { b: 2 } })).toBe(false)
+ expect(sameData({ a: null }, {})).toBe(false)
+ expect(sameData({ list: [1, 2] }, { list: [2, 1] })).toBe(false)
+ expect(sameData({ list: [1] }, { list: [1, undefined] })).toBe(false)
+ expect(sameData({ n: 1 }, { n: '1' })).toBe(false)
+ expect(sameData([], {})).toBe(false)
+ })
+
+ it('agrees with comparing sorted JSON on many random records', () => {
+ let seed = 1726
+ const next = () => (seed = (seed * 1103515245 + 12345) % 2 ** 31) / 2 ** 31
+ const leaf = () => [null, undefined, 0, 1, 'a', 'b', true, false, Number.NaN][Math.floor(next() * 9)]
+ const make = (depth: number): unknown => {
+ const r = next()
+ if (depth > 3 || r < 0.4) return leaf()
+ if (r < 0.7) return Array.from({ length: Math.floor(next() * 3) }, () => make(depth + 1))
+ return Object.fromEntries(Array.from({ length: Math.floor(next() * 4) }, () => [['x', 'y', 'z', 'w'][Math.floor(next() * 4)]!, make(depth + 1)]))
+ }
+ const mutate = (value: unknown): unknown => {
+ if (next() < 0.3) return make(2)
+ if (Array.isArray(value)) return value.map(mutate)
+ if (value && typeof value === 'object') return Object.fromEntries(Object.entries(value).reverse().map(([k, v]) => [k, mutate(v)]))
+ return value
+ }
+ let differ = 0
+ for (let i = 0; i < 5_000; i++) {
+ const a = make(0), b = next() < 0.5 ? structuredClone(a) : mutate(a)
+ const expected = sorted(a) === sorted(b)
+ if (!expected) differ++
+ expect(sameData(a, b), `${sorted(a)} vs ${sorted(b)}`).toBe(expected)
+ }
+ expect(differ).toBeGreaterThan(500)
+ })
+})
diff --git a/backend/src/superTabs/parts/commit.ts b/backend/src/superTabs/parts/commit.ts
index 3c91f886c..0bf3c39cd 100644
--- a/backend/src/superTabs/parts/commit.ts
+++ b/backend/src/superTabs/parts/commit.ts
@@ -24,6 +24,39 @@ function canonical(value: unknown): string {
? Object.fromEntries(Object.entries(item as Record<string, unknown>).sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0)) : item)
}
+/** A value as JSON writes it: a Date as its string, a number JSON cannot hold as null. */
+const asJson = (value: unknown): unknown => {
+ const plain = value && typeof value === 'object' && typeof (value as { toJSON?: unknown }).toJSON === 'function' ? (value as { toJSON(): unknown }).toJSON() : value
+ return typeof plain === 'number' && !Number.isFinite(plain) ? null : plain
+}
+/** What JSON leaves out of an object, and writes as null in a list. */
+const absent = (value: unknown) => value === undefined || typeof value === 'function' || typeof value === 'symbol'
+
+/**
+ * Whether two records hold the same data as stored, whatever order their keys were built in: what comparing their
+ * `canonical` JSON says, without writing either out (Stability Roadmap 2.6: a write compared every part this way, twice).
+ * It stops at the first difference.
+ */
+export function sameData(a: unknown, b: unknown): boolean {
+ const x = asJson(a), y = asJson(b)
+ if (x === y) return true
+ if (absent(x) || absent(y)) return absent(x) && absent(y)
+ if (!x || !y || typeof x !== 'object' || typeof y !== 'object' || Array.isArray(x) !== Array.isArray(y)) return false
+ if (Array.isArray(x)) {
+ const list = y as unknown[]
+ return x.length === list.length && x.every((item, i) => sameData(absent(item) ? null : item, absent(list[i]) ? null : list[i]))
+ }
+ const left = x as Record<string, unknown>, right = y as Record<string, unknown>
+ let count = 0
+ for (const key in left) {
+ if (!Object.hasOwn(left, key) || absent(left[key])) continue
+ count++
+ if (!Object.hasOwn(right, key) || absent(right[key]) || !sameData(left[key], right[key])) return false
+ }
+ for (const key in right) if (Object.hasOwn(right, key) && !absent(right[key])) count--
+ return count === 0
+}
+
/**
* Who sees what changed (design section 4.2): a project's visibility or members, a person removed or rejoined, the set of
* Leads (the Leads conversation's members), a conversation's audience, or a project removed. Readers reload in full.
@@ -103,7 +136,7 @@ const withoutReading = (data: unknown) => {
* group's write concerned, by its agents and its work before and after, so an agent leaving its last group wakes its host.
*/
export function hostMarker(kind: PartKind, before: unknown, after: unknown, machineOf: ReadonlyMap<string, string>, stamp: number): { host?: HostColumns } {
- if (kind === 'machine') return canonical(withoutReading(before)) === canonical(withoutReading(after)) ? {} : { host: { hostChangedAt: new Date(stamp) } }
+ if (kind === 'machine') return sameData(withoutReading(before), withoutReading(after)) ? {} : { host: { hostChangedAt: new Date(stamp) } }
if (kind !== 'group') return {}
const machines = ([before, after] as Array<GroupData | undefined>).flatMap(g => g
? [...g.agentIds.flatMap(id => machineOf.get(id) ?? []), ...(g.assignment ? [g.assignment.execution.machineId] : [])] : [])
@@ -138,7 +171,7 @@ export async function commitParts(tx: Transaction, loaded: { parts: SwarmParts;
if (!prior) {
writes.push({ op: 'insert', kind, key, data: part.data, ordinal: stamp + added++ / 1000, first, ...host(kind, undefined, part.data) })
if (kind !== 'group' && kind !== 'start' && kind !== 'progress') referencesChanged = true
- } else if (canonical(prior.data) !== canonical(part.data)) {
+ } else if (!sameData(prior.data, part.data)) {
writes.push({ op: 'update', kind, key, seen: prior.version, data: part.data, ...host(kind, prior.data, part.data) })
}
}
@@ -159,7 +192,7 @@ export async function commitParts(tx: Transaction, loaded: { parts: SwarmParts;
}
const epoch = parts.visibilityEpoch + (visibilityChanged(loaded.state, next) ? 1 : 0)
const head: HeadColumns = { groupCounter: Math.max(parts.groupCounter, next.groupCounter ?? 0), visibilityEpoch: epoch, sequenceBase: parts.sequenceBase }
- if (canonical(parts.head.data) !== canonical(after.head.data) || head.groupCounter !== parts.groupCounter || epoch !== parts.visibilityEpoch) {
+ if (!sameData(parts.head.data, after.head.data) || head.groupCounter !== parts.groupCounter || epoch !== parts.visibilityEpoch) {
writes.push({ op: 'update', kind: 'head', key: 'head', seen: parts.head.version, data: after.head.data, head })
}
if (canonical(indexesOf(loaded.state)) !== canonical(indexesOf(next))) referencesChanged = true
diff --git a/backend/src/superTabs/parts/index.ts b/backend/src/superTabs/parts/index.ts
index dd2c031a5..4467e3182 100644
--- a/backend/src/superTabs/parts/index.ts
+++ b/backend/src/superTabs/parts/index.ts
@@ -7,6 +7,6 @@ export * from './types.js'
export { clock, createClock } from './clock.js'
export { knownKind, reassemble, split, toLegacyState } from './shape.js'
-export { insertParts, lastChangedAt, loadActive, loadAll, loadChanged, loadDormant } from './load.js'
+export { insertParts, lastChangedAt, loadActive, loadAll, loadChanged, loadDormant, loadGroups, loadMachines, loadReadBase } from './load.js'
export { loadLight } from './light.js'
export { guardedWrite, PartConflict, type PartWrite } from './write.js'
diff --git a/backend/src/superTabs/parts/light.ts b/backend/src/superTabs/parts/light.ts
index cc3d0bcfd..f245a2da8 100644
--- a/backend/src/superTabs/parts/light.ts
+++ b/backend/src/superTabs/parts/light.ts
@@ -78,7 +78,9 @@ const toPart = <T>(kind: PartKind, row: PartRow): Part<T> => ({ kind, key: row.k
* dormant ones too, and the active ones of the named projects. Starts: unfinished, or every live start with `includeRecentStarts`.
*/
export async function loadLight(tx: Transaction, tabId: string,
- opts: { groupIds?: string[]; agentIds?: string[]; projectIds?: string[]; includeRecentStarts?: boolean; groups?: 'active' | 'working' } = {}): Promise<SwarmParts> {
+ opts: { groupIds?: string[]; agentIds?: string[]; projectIds?: string[]; includeRecentStarts?: boolean; groups?: 'active' | 'working'
+ /** False leaves machines out: the read cache keeps them at their own stamp (cache.ts). */
+ machines?: boolean } = {}): Promise<SwarmParts> {
const head = await tx.swarmHead.findUnique({ where: { id: partId(tabId, 'head') } })
if (!head) throw Object.assign(new Error(`Swarm ${tabId} has no parts`), { code: 'PARTS_MISSING' })
const live = { tabId, deleted: false }
@@ -96,7 +98,8 @@ export async function loadLight(tx: Transaction, tabId: string,
const lightGroups = async (match: Record<string, unknown>) => (await tx.swarmGroup.aggregateRaw({ pipeline: [{ $match: { tabId, deleted: false, ...match } },
{ $project: LIGHT_GROUP }] }) as unknown as Array<Record<string, unknown>>).map(lightGroup)
const [members, agents, machines, progress, groups] = await Promise.all([
- ...(['member', 'agent', 'machine'] as const).map(kind => delegate(tx, kind).findMany({ where: live }).then(rows => rows.map(row => toPart(kind, row)))),
+ ...(['member', 'agent', 'machine'] as const).map(kind => kind === 'machine' && opts.machines === false ? Promise.resolve([])
+ : delegate(tx, kind).findMany({ where: live }).then(rows => rows.map(row => toPart(kind, row)))),
delegate(tx, 'progress').findMany({ where: live }).then(rows => rows.map(row => toPart('progress', row))),
lightGroups({ $or: named }),
])
diff --git a/backend/src/superTabs/parts/load.ts b/backend/src/superTabs/parts/load.ts
index 0ad9c3b4a..8c327d084 100644
--- a/backend/src/superTabs/parts/load.ts
+++ b/backend/src/superTabs/parts/load.ts
@@ -1,7 +1,8 @@
import type { Prisma } from '@prisma/client'
import { delegate, extracted, partId, type PartRow } from './write.js'
-import type { GroupData, HeadData, Part, PartKind, SwarmParts } from './types.js'
+import type { GroupData, HeadData, MachineData, Part, PartKind, SwarmParts } from './types.js'
import { PART_KINDS } from './types.js'
+import { noteChange } from '../cache.js'
import { ms } from './light.js'
type Transaction = Prisma.TransactionClient
@@ -10,11 +11,12 @@ type Where = Record<string, unknown>
const toPart = <T>(kind: PartKind, row: PartRow): Part<T> => ({ kind, key: row.key, version: row.version, changedAt: row.changedAt.getTime(),
ordinal: row.ordinal, data: row.data as T, ...(row.deleted ? { deleted: true as const } : {}), ...(row.dormant ? { dormant: true as const } : {}) })
-async function assemble(tx: Transaction, tabId: string, where: Record<Exclude<PartKind, 'head'>, Where>): Promise<SwarmParts> {
+/** `null` for a kind leaves it out: none are loaded. */
+async function assemble(tx: Transaction, tabId: string, where: Record<Exclude<PartKind, 'head'>, Where | null>): Promise<SwarmParts> {
const head = await tx.swarmHead.findUnique({ where: { id: partId(tabId, 'head') } })
if (!head) throw Object.assign(new Error(`Swarm ${tabId} has no parts`), { code: 'PARTS_MISSING' })
const [members, agents, groups, machines, starts, progress] = await Promise.all((['member', 'agent', 'group', 'machine', 'start', 'progress'] as const)
- .map(kind => delegate(tx, kind).findMany({ where: { tabId, ...where[kind] } }).then(rows => rows.map(row => toPart(kind, row)))))
+ .map(kind => where[kind] === null ? Promise.resolve([]) : delegate(tx, kind).findMany({ where: { tabId, ...where[kind] } }).then(rows => rows.map(row => toPart(kind, row)))))
return { tabId, head: toPart<HeadData>('head', head as unknown as PartRow), members, agents, groups, machines, starts, progress,
sequenceBase: head.sequenceBase, visibilityEpoch: head.visibilityEpoch, groupCounter: head.groupCounter } as SwarmParts
}
@@ -43,6 +45,23 @@ export function loadActive(tx: Transaction, tabId: string,
})
}
+/** What the read cache keeps of a Swarm at its main stamp (cache.ts): `loadActive`'s parts with every live start, and no
+ * machines, which it keeps at their own stamp (`loadMachines`). Dormant groups and their like come by name (`loadGroups`). */
+export function loadReadBase(tx: Transaction, tabId: string): Promise<SwarmParts> {
+ const live = { deleted: false }
+ return assemble(tx, tabId, { member: live, agent: live, group: { ...live, dormant: false }, machine: null, start: live, progress: live })
+}
+
+/** A Swarm's live machine parts. */
+export async function loadMachines(tx: Transaction, tabId: string): Promise<Part<MachineData>[]> {
+ return (await delegate(tx, 'machine').findMany({ where: { tabId, deleted: false } })).map(row => toPart<MachineData>('machine', row))
+}
+
+/** A Swarm's live groups that match `where` (by key, or the dormant ones). */
+export async function loadGroups(tx: Transaction, tabId: string, where: Where): Promise<Part<GroupData>[]> {
+ return (await delegate(tx, 'group').findMany({ where: { tabId, deleted: false, ...where } })).map(row => toPart<GroupData>('group', row))
+}
+
/** Every part of one Swarm, tombstones included with their last data (migration verification and rollback). */
export function loadAll(tx: Transaction, tabId: string): Promise<SwarmParts> {
return assemble(tx, tabId, { member: {}, agent: {}, group: {}, machine: {}, start: {}, progress: {} })
@@ -51,6 +70,7 @@ export function loadAll(tx: Transaction, tabId: string): Promise<SwarmParts> {
/** Inserts a Swarm's parts as they are (the migration, a new Swarm). An existing part fails with P2002. */
export async function insertParts(tx: Transaction, parts: SwarmParts): Promise<void> {
const { tabId } = parts
+ noteChange(tx, tabId, 'main')
const rows = (kind: PartKind, list: Part[]) => list.map(part => ({ id: partId(tabId, part.key), tabId, key: part.key, version: part.version,
changedAt: new Date(part.changedAt), deleted: !!part.deleted, ordinal: part.ordinal, data: part.data as Prisma.InputJsonValue,
...extracted(kind, part.data, part.changedAt),
@@ -61,14 +81,16 @@ export async function insertParts(tx: Transaction, parts: SwarmParts): Promise<v
}
}
-/** Parts of these Swarms whose data changed after `since`, by tab, each kind in turn. Tombstones and dormant groups come
- * with their last data and `deleted`/`dormant` set, so the reader can check who may see them before sending anything. */
+/** Parts of these Swarms whose data changed after `since`, by tab, each kind in turn, machines also by their own reading
+ * (`readAt`). Tombstones and dormant groups come with their last data and `deleted`/`dormant` set, so the reader can
+ * check who may see them before sending anything. */
export async function loadChanged(tx: Transaction, tabIds: string[], since: Date, kinds: readonly PartKind[] = PART_KINDS): Promise<Map<string, Part[]>> {
const changed = new Map<string, Part[]>(tabIds.map(id => [id, []]))
if (!tabIds.length) return changed
for (const kind of kinds) {
- for (const row of await delegate(tx, kind).findMany({ where: { tabId: { in: tabIds }, changedAt: { gt: since } } })) {
- changed.get(row.tabId)!.push(toPart(kind, row))
+ const after = kind === 'machine' ? { OR: [{ changedAt: { gt: since } }, { readAt: { gt: since } }] } : { changedAt: { gt: since } }
+ for (const row of await delegate(tx, kind).findMany({ where: { tabId: { in: tabIds }, ...after } })) {
+ changed.get(row.tabId)!.push({ ...toPart(kind, row), ...row.readAt ? { readAt: row.readAt.getTime() } : {} })
}
}
return changed
diff --git a/backend/src/superTabs/parts/shape.test.ts b/backend/src/superTabs/parts/shape.test.ts
index 7a4531a74..b8bf85d92 100644
--- a/backend/src/superTabs/parts/shape.test.ts
+++ b/backend/src/superTabs/parts/shape.test.ts
@@ -110,3 +110,26 @@ describe('reassembling a Swarm from its parts', () => {
since: 111, requestedAt: 222, requestMessageId: 'm1' })
})
})
+
+describe('reassembling for a read', () => {
+ const deepFreeze = (value: unknown): void => {
+ if (value && typeof value === 'object') { Object.values(value).forEach(deepFreeze); Object.freeze(value) }
+ }
+
+ it('gives the same state as a command\'s reassembly, sharing the parts\' records instead of copying them', () => {
+ const parts = split(legacy(), { tabId: 'tab1', version: 1, now: 123 })
+ deepFreeze(parts)
+ const shared = reassemble(parts, { share: true })
+ expect(shared).toEqual(reassemble(parts))
+ expect(shared.people[0]).toBe(parts.members[0]!.data)
+ expect(shared.conversations[1]).toBe(parts.groups[1]!.data)
+ expect(shared.machines![0]).toBe(parts.machines[0]!.data!.contribution)
+ })
+
+ it('still copies for a command, which changes what it reassembled', () => {
+ const parts = split(legacy(), { tabId: 'tab1', version: 1, now: 123 })
+ const state = reassemble(parts)
+ state.people[0]!.role = 'viewer'
+ expect(parts.members[0]!.data!.role).toBe('owner')
+ })
+})
diff --git a/backend/src/superTabs/parts/shape.ts b/backend/src/superTabs/parts/shape.ts
index a226121bd..b1f4d30c8 100644
--- a/backend/src/superTabs/parts/shape.ts
+++ b/backend/src/superTabs/parts/shape.ts
@@ -91,19 +91,24 @@ export function split(state: SuperTabState, opts: { tabId: string; version: numb
const live = <T>(parts: Part<T>[]): T[] => parts.filter(p => !p.deleted && p.data).sort((a, b) => a.ordinal - b.ordinal).map(p => p.data!)
-/** The state commands run on: `SuperTab.state`'s shape, each list in part order, with the part-only group fields kept. */
-export function reassemble(parts: SwarmParts): SuperTabState {
+/**
+ * The state commands run on: `SuperTab.state`'s shape, each list in part order, with the part-only group fields kept. A
+ * command changes what it reassembled, so each record is a copy; a read (`share`) changes nothing and takes the parts'
+ * own records, which the read cache holds frozen (cache.ts).
+ */
+export function reassemble(parts: SwarmParts, { share = false }: { share?: boolean } = {}): SuperTabState {
+ const copy = share ? <T>(value: T): T => value : structuredClone
const groups = live(parts.groups)
const machines = live(parts.machines)
const progress = new Map(parts.progress.filter(p => !p.deleted && p.data).map(p => [p.key, p.data!]))
const flat = <T>(each: (machine: MachineData) => T[]) => machines.flatMap(each)
const state: SuperTabState = {
- ...structuredClone(parts.head.data!),
- ...(parts.head.data!.projects ? { projects: parts.head.data!.projects.map(project => ({ ...structuredClone(project),
- ...(progress.has(project.id) ? { progress: structuredClone(progress.get(project.id)!) as unknown as SwarmProject['progress'] } : {}) })) } : {}),
- people: structuredClone(live(parts.members)),
- agents: structuredClone(live(parts.agents)),
- conversations: structuredClone(groups.filter(knownKind)),
+ ...copy(parts.head.data!),
+ ...(parts.head.data!.projects ? { projects: parts.head.data!.projects.map(project => ({ ...copy(project),
+ ...(progress.has(project.id) ? { progress: copy(progress.get(project.id)!) as unknown as SwarmProject['progress'] } : {}) })) } : {}),
+ people: copy(live(parts.members)),
+ agents: copy(live(parts.agents)),
+ conversations: copy(groups.filter(knownKind)),
groupCounter: parts.groupCounter,
sequenceBase: parts.sequenceBase,
}
@@ -117,7 +122,7 @@ export function reassemble(parts: SwarmParts): SuperTabState {
const closing = flat(m => m.closingSessions.map(c => ({ agentId: c.agentId, machineId: m.machineId, harnessId: c.harnessId, machineOwnerId: c.machineOwnerId })))
const cleanups: MachineCleanup[] = flat(m => m.cleanups.map(c => ({ machineId: m.machineId, removalId: c.removalId, machineOwnerId: c.machineOwnerId })))
const list = <K extends typeof LISTS[number]>(key: K, items: NonNullable<SuperTabState[K]>) => {
- if (items.length || present.has(key)) state[key] = structuredClone(items)
+ if (items.length || present.has(key)) state[key] = copy(items)
}
list('parkedConversations', parked)
list('starts', starts)
diff --git a/backend/src/superTabs/parts/types.ts b/backend/src/superTabs/parts/types.ts
index 417e3af8b..c00e2d0c5 100644
--- a/backend/src/superTabs/parts/types.ts
+++ b/backend/src/superTabs/parts/types.ts
@@ -12,8 +12,10 @@ export interface Part<T = unknown> {
key: string
/** Moves on every write and every guarded touch; never resets. */
version: number
- /** Epoch ms of the last data change (or tombstone). A touch leaves it. */
+ /** Epoch ms of the last data change (or tombstone). A touch leaves it, and so does a machine's own reading (`readAt`). */
changedAt: number
+ /** A machine part's last write of its own reading alone, when there was one (change reads only). */
+ readAt?: number
deleted?: true
dormant?: true
ordinal: number
diff --git a/backend/src/superTabs/parts/write.ts b/backend/src/superTabs/parts/write.ts
index 448331b03..6e161ec2a 100644
--- a/backend/src/superTabs/parts/write.ts
+++ b/backend/src/superTabs/parts/write.ts
@@ -3,6 +3,7 @@ import type { Prisma } from '@prisma/client'
import { activeWork, stableId, type AgentStart, type Agent } from '../model.js'
import type { GroupData, MachineData, PartKind } from './types.js'
import { RETENTION_MS } from './types.js'
+import { noteChange } from '../cache.js'
type Transaction = Prisma.TransactionClient
const json = (value: unknown): Prisma.InputJsonValue => JSON.parse(JSON.stringify(value)) as Prisma.InputJsonValue
@@ -42,7 +43,7 @@ export function delegate(tx: Transaction, kind: PartKind) {
aggregateRaw(args: { pipeline: Record<string, unknown>[] }): Promise<unknown>
}
}
-export interface PartRow { id: string; tabId: string; key: string; version: number; changedAt: Date; deleted: boolean; ordinal: number; data: unknown
+export interface PartRow { id: string; tabId: string; key: string; version: number; changedAt: Date; readAt?: Date | null; deleted: boolean; ordinal: number; data: unknown
dormant?: boolean; groupCounter?: number; visibilityEpoch?: number; sequenceBase?: number }
export const partId = (tabId: string, key: string) => `${tabId}:${key}`
@@ -101,6 +102,10 @@ async function keepFinishedAt(tx: Transaction, id: string, fields: Record<string
*/
export async function guardedWrite(tx: Transaction, tabId: string, write: PartWrite, stamp: number): Promise<number> {
const model = delegate(tx, write.kind), id = partId(tabId, write.key)
+ // A machine's own reading leaves what its host hears of unmarked (commit.ts hostMarker): only the machine stamp moves, and
+ // it is dated `readAt`, not `changedAt`, so the Swarm's newest change stays (load.ts lastChangedAt).
+ const reading = write.kind === 'machine' && write.op === 'update' && !write.host
+ if (write.op !== 'touch') noteChange(tx, tabId, reading ? 'machine' : 'main')
if (write.op === 'insert') {
let fields = { ...extracted(write.kind, write.data, stamp), ...write.head, ...write.host }
// A part added again under the same key continues from its tombstone's version (spec ruling 6).
@@ -120,7 +125,7 @@ export async function guardedWrite(tx: Transaction, tabId: string, write: PartWr
}
const data = write.op === 'touch' ? {}
: write.op === 'remove' ? { changedAt: new Date(stamp), deleted: true, purgeAt: new Date(stamp + RETENTION_MS), data: json(write.data), ...write.host }
- : { changedAt: new Date(stamp), data: json(write.data), ...(write.kind === 'start'
+ : { ...reading ? { readAt: new Date(stamp) } : { changedAt: new Date(stamp) }, data: json(write.data), ...(write.kind === 'start'
? await keepFinishedAt(tx, id, extracted(write.kind, write.data, stamp)) : extracted(write.kind, write.data, stamp)), ...write.head, ...write.host }
const updated = await model.updateMany({ where: { id, version: write.seen, deleted: false }, data: { version: write.seen + 1, ...data } })
if (!updated.count) throw new PartConflict(write.kind, write.key)
diff --git a/backend/src/superTabs/read/host.ts b/backend/src/superTabs/read/host.ts
index 57b5ea716..b7c6dc573 100644
--- a/backend/src/superTabs/read/host.ts
+++ b/backend/src/superTabs/read/host.ts
@@ -10,7 +10,9 @@ import { machineAlive } from '../../lib/prisma.js'
import { isTeamSession } from '../../lib/teamSession.js'
import { machineBillingAllowsDataPlane } from '../../lib/billingState.js'
import { fail, type AgentStart, type SuperTabState } from '../model.js'
-import { transaction, type SuperTabStore } from '../store.js'
+import type { SuperTabStore } from '../store.js'
+import type { CacheView } from '../cache.js'
+import { readMachines } from '../cachedReads.js'
import * as parts from '../parts/index.js'
import type { MachineData, Part } from '../parts/index.js'
import { READ_FORMAT, type HostChanges, type HostChangesRequest } from './contract.js'
@@ -47,38 +49,39 @@ export class HostReads {
async changes(user: AuthUser, machineId: string, request: HostChangesRequest): Promise<HostChanges> {
const t0 = this.now()
const from = request.cursor && !request.full ? readCursor(request.cursor, 0, t0) : { reload: true as const }
- return transaction(this.db, async tx => {
- await this.store.hostFor(tx, user, machineId)
- const owned = await tx.machine.findFirst({ where: { machineId, userId: user.sub, autonomousEnv: user.autonomousEnv, ...machineAlive } })
- if (!owned || !machineBillingAllowsDataPlane(owned)) fail('HOST_UNAVAILABLE', 'This execution host is unavailable.', 403)
- // Swarms still in one row are migrated when first opened; the job migrates every one before the new release serves.
- const ids = (await tx.superTab.findMany({ where: { autonomousEnv: user.autonomousEnv, format: 2,
- OR: [{ machineIds: { has: machineId } }, { cleanupMachineIds: { has: machineId } }] },
- orderBy: { updatedAt: 'desc' }, take: FULL_TABS, select: { id: true } })).map(r => r.id)
- const teams = isTeamSession(user) || this.options.teamAccountsIdle ? [] : await tx.machine.findMany({ where: { userId: user.sub,
- hostMachineId: machineId, sessionKind: 'team', autonomousEnv: user.autonomousEnv, ...machineAlive }, select: { machineId: true } })
- const full = 'reload' in from
- const woken = full ? null : await this.woken(tx, ids, machineId, from.since)
- // Only for the Swarms answered for: this runs every second on every host, and seven groupBy over all of a Swarm's parts
- // are no use for one that has nothing to say (a production backend at 5 pods, 2026-10-27).
- const newest = await parts.lastChangedAt(tx, woken ? ids.filter(id => woken.has(id)) : ids)
- const tabs = []
- for (const tabId of ids) {
- if (woken && !woken.has(tabId)) continue
- const lastChangedAt = newest.get(tabId) ?? null, held = request.held?.[tabId]
- // The etag is the rows' hash and then the signature of what they were built from. A matching signature answers
- // 'unchanged' from version columns alone. Never on a full read: grants and billing live outside the parts.
- const inputs = await this.inputs(tx, tabId)
- if (!full && held && inputs && held.slice(ROW_HASH) === inputs) { tabs.push({ tabId, lastChangedAt, etag: held, unchanged: true as const }); continue }
- const state = await this.tabState(tx, tabId, machineId)
- const rows = await hostTab(tabId, state, machineId, user, lastChangedAt, await this.lookups(tx, state, teams))
- const etag = rows.etag + (inputs ?? '')
- // The host already holds these rows: say so instead of sending them again.
- tabs.push(held?.slice(0, ROW_HASH) === rows.etag ? { tabId, lastChangedAt, etag, unchanged: true as const } : { ...rows, etag })
- }
- return { format: READ_FORMAT, full, cursor: cursorFor(t0, 0), tabs,
- removedTabIds: full ? [] : await this.removed(tx, user.autonomousEnv, machineId, from.since, new Set(ids)) }
- })
+ // Outside any transaction: each Swarm's signature and state come through the read cache (cache.ts), at stamps read once.
+ const tx = this.db
+ await this.store.hostFor(tx, user, machineId)
+ const owned = await tx.machine.findFirst({ where: { machineId, userId: user.sub, autonomousEnv: user.autonomousEnv, ...machineAlive } })
+ if (!owned || !machineBillingAllowsDataPlane(owned)) fail('HOST_UNAVAILABLE', 'This execution host is unavailable.', 403)
+ // Swarms still in one row are migrated when first opened; the job migrates every one before the new release serves.
+ const ids = (await tx.superTab.findMany({ where: { autonomousEnv: user.autonomousEnv, format: 2,
+ OR: [{ machineIds: { has: machineId } }, { cleanupMachineIds: { has: machineId } }] },
+ orderBy: { updatedAt: 'desc' }, take: FULL_TABS, select: { id: true } })).map(r => r.id)
+ const teams = isTeamSession(user) || this.options.teamAccountsIdle ? [] : await tx.machine.findMany({ where: { userId: user.sub,
+ hostMachineId: machineId, sessionKind: 'team', autonomousEnv: user.autonomousEnv, ...machineAlive }, select: { machineId: true } })
+ const full = 'reload' in from
+ const woken = full ? null : await this.woken(tx, ids, machineId, from.since)
+ // Only for the Swarms answered for: this runs every second on every host, and seven groupBy over all of a Swarm's parts
+ // are no use for one that has nothing to say (a production backend at 5 pods, 2026-10-27).
+ const newest = await parts.lastChangedAt(tx, woken ? ids.filter(id => woken.has(id)) : ids)
+ const tabs = []
+ const cached = await this.store.view(woken ? ids.filter(id => woken.has(id)) : ids)
+ for (const tabId of ids) {
+ if (woken && !woken.has(tabId)) continue
+ const lastChangedAt = newest.get(tabId) ?? null, held = request.held?.[tabId]
+ // The etag is the rows' hash and then the signature of what they were built from. A matching signature answers
+ // 'unchanged' from version columns alone. Never on a full read: grants and billing live outside the parts.
+ const inputs = await cached.get(tabId, 'host:inputs', 'main', () => this.inputs(tx, tabId))
+ if (!full && held && inputs && held.slice(ROW_HASH) === inputs) { tabs.push({ tabId, lastChangedAt, etag: held, unchanged: true as const }); continue }
+ const state = await this.tabState(tx, cached, tabId, machineId)
+ const rows = await hostTab(tabId, state, machineId, user, lastChangedAt, await this.lookups(tx, state, teams))
+ const etag = rows.etag + (inputs ?? '')
+ // The host already holds these rows: say so instead of sending them again.
+ tabs.push(held?.slice(0, ROW_HASH) === rows.etag ? { tabId, lastChangedAt, etag, unchanged: true as const } : { ...rows, etag })
+ }
+ return { format: READ_FORMAT, full, cursor: cursorFor(t0, 0), tabs,
+ removedTabIds: full ? [] : await this.removed(tx, user.autonomousEnv, machineId, from.since, new Set(ids)) }
}
/**
@@ -107,7 +110,15 @@ export class HostReads {
/** The Swarm's active parts plus, dormant ones included, every group its agents on this machine are in (a finished
* group that is not archived still keeps its agent from automatic cleanup) and every group of a start it holds (the
* host still settles a queued start whose group went quiet). */
- private async tabState(tx: Transaction, tabId: string, machineId: string): Promise<SuperTabState> {
+ private async tabState(tx: Transaction, cached: CacheView, tabId: string, machineId: string): Promise<SuperTabState> {
+ const [light, machines] = await Promise.all([cached.get(tabId, `host:state:${machineId}`, 'main',
+ // On a miss, one snapshot of what the host's rows are built from.
+ () => this.db.$transaction(t => this.hostParts(t, tabId, machineId))),
+ readMachines(tx, cached, tabId)])
+ return parts.reassemble({ ...light, machines }, { share: true })
+ }
+
+ private async hostParts(tx: Transaction, tabId: string, machineId: string) {
const [agents, starts] = await Promise.all([
tx.swarmAgent.findMany({ where: { tabId, machineId, deleted: false }, select: { key: true } }),
tx.swarmStart.findMany({ where: { tabId, machineId, deleted: false, finishedAt: null }, select: { conversationId: true } }),
@@ -118,7 +129,7 @@ export class HostReads {
.map(g => g.projectId ?? 'lead'))] : []
// Light conversations (parts/light.ts), and only those host rows read: its agents' (dormant ones too, for `active`),
// the Lead conversations, running work (assignments, attention, stop requests), and its starts' and their projects'.
- return parts.reassemble(await parts.loadLight(tx, tabId, { groups: 'working', agentIds: agents.map(a => a.key), groupIds, projectIds }))
+ return parts.loadLight(tx, tabId, { groups: 'working', agentIds: agents.map(a => a.key), groupIds, projectIds, machines: false })
}
/**
diff --git a/backend/src/superTabs/read/service.ts b/backend/src/superTabs/read/service.ts
index 1d8bf6557..28b1ab065 100644
--- a/backend/src/superTabs/read/service.ts
+++ b/backend/src/superTabs/read/service.ts
@@ -2,13 +2,16 @@
* The read API by parts (design section 4; shapes in contract.ts). Every read takes its time `T0` before any query and
* makes the next cursor from it, so a write that commits late, inside the overlap, is still seen by the next poll.
*/
-import type { Prisma, PrismaClient } from '@prisma/client'
+import type { PrismaClient } from '@prisma/client'
import type { AuthUser } from '../../lib/ssoAuth.js'
import { isTeamSession, TEAM_SESSION_LIMITED, TEAM_SESSION_MESSAGE } from '../../lib/teamSession.js'
import { author, fail, isLeads, seenMachineIds, visibleState, type Actor, type AgentStart } from '../model.js'
import { mentionsWaiting } from '../mentions.js'
import { decisionsWaiting } from '../decisions.js'
-import { transaction, type SuperTabStore } from '../store.js'
+import type { SuperTabStore } from '../store.js'
+import { transaction } from '../store.js'
+import type { CacheView } from '../cache.js'
+import { readChanged, readLastChangedAt, readLight, readParts } from '../cachedReads.js'
import { migrateSwarm } from '../migration/migrate.js'
import * as parts from '../parts/index.js'
import type { SwarmParts } from '../parts/index.js'
@@ -26,8 +29,8 @@ import { changesFor } from './changes.js'
import { envelope, groupView, readerView, seenIn, type ReaderView } from './views.js'
import { decodeDormantPage, encodeDormantPage } from './dormantPage.js'
-type Transaction = Prisma.TransactionClient
type LoadOptions = NonNullable<Parameters<typeof parts.loadActive>[2]> & Pick<NonNullable<Parameters<typeof parts.loadLight>[2]>, 'groups'>
+type Loader = (view: CacheView, tabId: string, opts: LoadOptions) => Promise<SwarmParts>
interface Read { parts: SwarmParts; actor: Actor; view: ReaderView; lastChangedAt: number | null }
export class SwarmReads {
@@ -43,8 +46,8 @@ export class SwarmReads {
async firstLoad(user: AuthUser, tabId: string, asAgentId?: string): Promise<FirstLoad> {
const t0 = this.now()
- return this.read(user, tabId, async tx => {
- const { parts: loaded, view, lastChangedAt } = await this.load(tx, user, tabId, asAgentId, t0, { includeRecentStarts: true })
+ return this.read(user, tabId, async (tx, cached) => {
+ const { parts: loaded, view, lastChangedAt } = await this.load(tx, cached, user, tabId, asAgentId, t0, { includeRecentStarts: true })
return { format: READ_FORMAT, tabId, cursor: cursorFor(t0, loaded.visibilityEpoch), serverTime: t0, lastChangedAt, epoch: loaded.visibilityEpoch,
...(this.options.githubApp ? { githubApp: true as const } : {}), ...view.parts, derived: await this.withAvatars(view, t0) }
})
@@ -60,30 +63,30 @@ export class SwarmReads {
async changes(user: AuthUser, tabId: string, cursor: string, asAgentId?: string): Promise<Changes> {
const t0 = this.now()
- return this.read(user, tabId, async tx => {
+ return this.read(user, tabId, async (tx, cached) => {
// Light (parts/light.ts): changed parts come whole from the change feed; the rest is read only to check visibility and
// derive values: the Lead conversations, those with running work, and those a person's mentions are in (their count).
// Open decisions count their conversations themselves (decisions.ts).
const mentioned = asAgentId ? [] : (await tx.superTabMention.findMany({ where: { tabId, personId: user.sub }, select: { conversationId: true },
distinct: ['conversationId'] })).map(m => m.conversationId)
const light = { includeRecentStarts: true, groups: 'working' as const, groupIds: mentioned }
- let read = await this.load(tx, user, tabId, asAgentId, t0, light, parts.loadLight)
+ const lightly: Loader = (view, id, opts) => readLight(tx, view, id, opts)
+ let read = await this.load(tx, cached, user, tabId, asAgentId, t0, light, lightly)
const from = readCursor(cursor, read.parts.visibilityEpoch, t0)
if ('reload' in from) return from
- const changed = (await parts.loadChanged(tx, [tabId], new Date(from.since))).get(tabId) ?? []
+ const changed = await readChanged(tx, cached, tabId, from.since)
// A changed start whose group went dormant is read against that group, so its readers alone learn of it.
const held = new Set(read.parts.groups.map(g => g.key))
const dormant = [...new Set(changed.flatMap(p => p.kind === 'start' ? [(p.data as AgentStart).request.conversationId] : []))].filter(id => !held.has(id))
- if (dormant.length) read = await this.load(tx, user, tabId, asAgentId, t0, { ...light, groupIds: [...mentioned, ...dormant] }, parts.loadLight)
+ if (dormant.length) read = await this.load(tx, cached, user, tabId, asAgentId, t0, { ...light, groupIds: [...mentioned, ...dormant] }, lightly)
return { format: READ_FORMAT, reload: false, cursor: cursorFor(t0, read.parts.visibilityEpoch), serverTime: t0, lastChangedAt: read.lastChangedAt,
epoch: read.parts.visibilityEpoch, ...changesFor(read.view, changed, read.actor), derived: await this.withAvatars(read.view, t0) }
})
}
async dormant(user: AuthUser, tabId: string, page: { projectId?: string; before?: string }, asAgentId?: string): Promise<DormantGroups> {
- return this.read(user, tabId, async tx => {
- const loaded = await parts.loadActive(tx, tabId)
- const state = parts.reassemble(loaded)
+ return this.read(user, tabId, async (tx, cached) => {
+ const state = parts.reassemble(await readParts(tx, cached, tabId), { share: true })
const actor = await this.store.actor(tx, user, state, asAgentId)
author(state, actor)
const before = page.before ? decodeDormantPage(page.before) : undefined
@@ -97,8 +100,8 @@ export class SwarmReads {
async group(user: AuthUser, tabId: string, conversationId: string, asAgentId?: string): Promise<OneGroup> {
const t0 = this.now()
- return this.read(user, tabId, async tx => {
- const { view, parts: loaded, actor } = await this.load(tx, user, tabId, asAgentId, t0, { groupIds: [conversationId] })
+ return this.read(user, tabId, async (tx, cached) => {
+ const { view, parts: loaded, actor } = await this.load(tx, cached, user, tabId, asAgentId, t0, { groupIds: [conversationId] })
const active = view.parts.groups.find(g => g.key === conversationId)
if (active) return { group: active }
// A dormant group is in no whole-Swarm view; it passes the same read check as on the dormant list.
@@ -115,8 +118,8 @@ export class SwarmReads {
async leaderboard(user: AuthUser, tabId: string, projectId: string, week?: number): Promise<Leaderboard> {
if (isTeamSession(user)) fail(TEAM_SESSION_LIMITED, TEAM_SESSION_MESSAGE, 403)
const now = this.now(), start = readableWeek(week, now)
- return this.read(user, tabId, async tx => {
- const state = parts.reassemble(await parts.loadActive(tx, tabId))
+ return this.read(user, tabId, async (tx, cached) => {
+ const state = parts.reassemble(await readParts(tx, cached, tabId), { share: true })
const actor = await this.store.actor(tx, user, state)
readProject(state, actor, projectId)
const dormant = await recentDormantGroups(tx, tabId, projectId, now - LEDGER_WEEKS * WEEK_MS)
@@ -146,29 +149,30 @@ export class SwarmReads {
}), next: rows.length > 100 ? rows[99]!.id : null }
}
- /** Runs a read; a Swarm still in one row is migrated first, as a safety net for one the job did not see (design 6.1),
- * and answers 503 while that cannot be done. The store's own loader refuses a Swarm not in parts. */
- private async read<T>(user: AuthUser, tabId: string, body: (tx: Transaction) => Promise<T>): Promise<T> {
+ /** Runs a read, outside any transaction: its parts come through the read cache at the stamps `body` is given (cache.ts),
+ * and only the read of the base parts on a miss is a transaction (cachedReads.ts). A Swarm still in one row is migrated
+ * first, as a safety net for one the job did not see (design 6.1), and answers 503 while that cannot be done. */
+ private async read<T>(user: AuthUser, tabId: string, body: (tx: PrismaClient, view: CacheView) => Promise<T>): Promise<T> {
// The row's format only: a migrated row still holds its old state, as large as the Swarm, until the owner drops it.
const row = await this.db.superTab.findFirst({ where: { id: tabId, autonomousEnv: user.autonomousEnv }, select: { format: true } })
if (!row) fail('NOT_FOUND', 'This Super Tab is unavailable.', 404)
if (row.format !== 2 && !(await migrateSwarm(this.db, tabId)).ok) fail(SWARM_MIGRATING.code, SWARM_MIGRATING.message, SWARM_MIGRATING.status)
- return transaction(this.db, body)
+ return body(this.db, await this.store.view([tabId]))
}
- private async load(tx: Transaction, user: AuthUser, tabId: string, asAgentId: string | undefined, t0: number, opts: LoadOptions,
- loader: (tx: Transaction, tabId: string, opts: LoadOptions) => Promise<SwarmParts> = parts.loadActive): Promise<Read> {
- const loaded = await loader(tx, tabId, opts)
- const state = parts.reassemble(loaded)
+ private async load(tx: PrismaClient, cached: CacheView, user: AuthUser, tabId: string, asAgentId: string | undefined, t0: number, opts: LoadOptions,
+ loader: Loader = (view, id, named) => readParts(tx, view, id, named)): Promise<Read> {
+ const loaded = await loader(cached, tabId, opts)
+ const state = parts.reassemble(loaded, { share: true })
const actor = await this.store.actor(tx, user, state, asAgentId)
const [teamMachines, otherWorkspaceMachines, counts, newest] = await Promise.all([
state.machines?.length ? this.store.teamMachines(tx, state) : [],
state.machines?.length ? this.store.otherWorkspaceMachines(tx, state) : [],
actor.kind === 'person' ? Promise.all([mentionsWaiting(tx, tabId, state, actor, t0), decisionsWaiting(tx, tabId, state, actor)]) : undefined,
- parts.lastChangedAt(tx, [tabId]),
+ readLastChangedAt(tx, cached, tabId),
])
- return { parts: loaded, actor, lastChangedAt: newest.get(tabId) ?? null,
+ return { parts: loaded, actor, lastChangedAt: newest,
view: readerView(loaded, actor, t0, { teamMachines, otherWorkspaceMachines,
- ...(counts ? { mentionsWaiting: counts[0], decisionsWaiting: counts[1].count, decisionsDigest: counts[1].digest } : {}) }) }
+ ...(counts ? { mentionsWaiting: counts[0], decisionsWaiting: counts[1].count, decisionsDigest: counts[1].digest } : {}) }, state) }
}
}
diff --git a/backend/src/superTabs/read/views.test.ts b/backend/src/superTabs/read/views.test.ts
index 8be19383a..e296c84b9 100644
--- a/backend/src/superTabs/read/views.test.ts
+++ b/backend/src/superTabs/read/views.test.ts
@@ -117,3 +117,22 @@ describe('reader view of the parts (design 4.1, 4.3)', () => {
expect(() => readerView(parts, { kind: 'person', userId: 'nobody', email: '[email protected]' }, NOW, { teamMachines: [] })).toThrow(/no longer have access/)
})
})
+
+describe('reader view over what the read cache holds', () => {
+ const deepFreeze = <T>(value: T): T => {
+ if (value && typeof value === 'object') { Object.values(value).forEach(deepFreeze); Object.freeze(value) }
+ return value
+ }
+
+ it('builds from the state it is given, over frozen parts, as it would from the parts alone', () => {
+ const { parts } = scene()
+ const expected = readerView(structuredClone(parts), peer, NOW, { teamMachines: [] })
+ const frozen = deepFreeze(structuredClone(parts))
+ const state = store.reassemble(frozen, { share: true })
+ for (const actor of [peer, owner, outsider, agentActor(richSwarm().lead)]) {
+ const view = readerView(frozen, actor, NOW, { teamMachines: [] }, state)
+ expect(view.state).toBe(state)
+ if (actor === peer) expect({ parts: view.parts, derived: view.derived }).toEqual({ parts: expected.parts, derived: expected.derived })
+ }
+ })
+})
diff --git a/backend/src/superTabs/read/views.ts b/backend/src/superTabs/read/views.ts
index a44a0dff0..8d204a066 100644
--- a/backend/src/superTabs/read/views.ts
+++ b/backend/src/superTabs/read/views.ts
@@ -67,8 +67,8 @@ export function agentView(agent: Agent): AgentView {
return withAvatarView(rest) as AgentView
}
-export function readerView(parts: SwarmParts, actor: Actor, now: number, extras: ReadExtras): ReaderView {
- const state = store.reassemble(parts)
+/** `state` is the parts already reassembled, when the caller has it (a read reassembles once). */
+export function readerView(parts: SwarmParts, actor: Actor, now: number, extras: ReadExtras, state = store.reassemble(parts, { share: true })): ReaderView {
const visible = visibleState(state, actor, now)
const groups = visible.conversations.filter(group => canReadGroup(state, actor, group))
const readable = new Set(groups.map(g => g.id))
diff --git a/backend/src/superTabs/store.ts b/backend/src/superTabs/store.ts
index 8d47c90cc..72e80f202 100644
--- a/backend/src/superTabs/store.ts
+++ b/backend/src/superTabs/store.ts
@@ -23,9 +23,11 @@ import { answersRequest, dirtyGroups, noteProjectReports, projectUpdateRequest }
import { digestDue, digestRequest, digestSince } from './digest.js'
import { judgedMachines, loseTasks, sweepLiveness, SWEEP_EVERY_MS } from './liveness.js'
import { archivable, archiveLine, stopIdleWorkers, stoppedMachines } from './tidy.js'
-import { clock, insertParts, lastChangedAt, loadActive, reassemble, split, type HeadData, type SwarmParts } from './parts/index.js'
+import { clock, insertParts, loadActive, reassemble, split, type HeadData, type SwarmParts } from './parts/index.js'
import { commitParts, indexesOf, numberMessages, type Touch, type Written } from './parts/commit.js'
import { touchesFor } from './parts/touches.js'
+import { freeze, swarmReadCache, swarmTransaction, type CacheView, type SwarmCache } from './cache.js'
+import { DIRECT, readLastChangedAt, readParts, type ReadNamed } from './cachedReads.js'
import { revokeNativeContexts, SuperTabNativeStore } from './native.js'
import { activeMember, addMember, hasWorkspaceAccess, listWorkspaces, memberId, WORKSPACE_ACCESS_REVOKED, workspaceForNewSwarm, requireWorkspaceAccess, requireWorkspaceRole, SwarmWorkspaces } from './workspaceStore.js'
import { WORKSPACE_LABEL, type WorkspaceRole } from './workspaces.js'
@@ -115,10 +117,12 @@ function deliveryTouches(delivery: SuperTabDelivery): Touch[] {
}
/** Loads one Swarm's parts and the state commands run on. Reads never write (design ruling R8); the migration rebuilt
- * every group's link list from its history. */
+ * every group's link list from its history. The state shares the loaded records, frozen: a command changes its own copy
+ * (commands.ts applyCommand), and the parts stay what the write is compared with (parts/commit.ts). */
export async function loadRow(tx: Transaction, row: SwarmRow, named: Named = {}): Promise<Loaded> {
const parts = await loadActive(tx, row.id, named)
- const state = reassemble(parts)
+ freeze(parts)
+ const state = reassemble(parts, { share: true })
parkLaterConversations(state)
syncLeads(state)
return { row, parts, state }
@@ -139,7 +143,8 @@ export async function transaction<T>(db: Pick<PrismaClient, '$transaction'>, bod
for (let attempt = 1; ; attempt++) {
let committed: Array<() => void> = []
try {
- const result = await db.$transaction(tx => { committed = []; pendingCommits.set(tx, committed); return body(tx) }, TRANSACTION_OPTIONS)
+ // The cache's stamps are bumped, and awaited, once the attempt commits (cache.ts); after-commit work runs after them.
+ const result = await swarmTransaction(db, tx => { committed = []; pendingCommits.set(tx, committed); return body(tx) }, TRANSACTION_OPTIONS)
for (const callback of committed) {
try { callback() } catch (error) { logger.warn('[super-tab] after-commit callback failed', { error: error instanceof Error ? error.message : String(error) }) }
}
@@ -213,6 +218,8 @@ export interface SuperTabStoreHooks {
/** No Harness GitHub App on this server: a Mac's Harness Team account joins no Swarm and takes no teammates' work, since
* it has no git login to fetch or push with (commands.ts `teamAccountsIdle`). */
teamAccountsIdle?: boolean
+ /** The read cache this store reads through (cache.ts): the process's by default, none when null. */
+ readCache?: SwarmCache | null
/** When each caller's full delivery poll last succeeded, shared by every server (Redis in production, pollMemory.ts). Left
* out, this server remembers alone. */
polls?: PollMemory
@@ -402,6 +409,25 @@ export class SuperTabStore {
return loadRow(tx, await this.row(tx, user, tabId), named)
}
+ /** One read's view of these Swarms' stamps (cache.ts), or none: reads then go to Mongo. */
+ async view(tabIds: string[]): Promise<CacheView> {
+ const cache = this.hooks.readCache !== undefined ? this.hooks.readCache : swarmReadCache()
+ return cache ? cache.at(tabIds) : DIRECT
+ }
+
+ /**
+ * A Swarm as a read works on it, outside any transaction: its parts through the read cache, in `loadRow`'s shape, sharing
+ * the stored records (frozen when held). Only for reads: a command changes what it loads, and loads in its transaction.
+ */
+ async readLoad(user: AuthUser, tabId: string, named: ReadNamed = {}, view?: CacheView): Promise<Loaded> {
+ const row = await this.row(this.db, user, tabId)
+ const parts = await readParts(this.db, view ?? await this.view([tabId]), tabId, named)
+ const state = reassemble(parts, { share: true })
+ parkLaterConversations(state)
+ syncLeads(state)
+ return { row, parts, state }
+ }
+
/** Loads again with more named groups, agents or starts when the first load lacks one a command needs. */
private async ensure(tx: Transaction, loaded: Loaded, named: Named): Promise<Loaded> {
if (named.allGroups) return loadRow(tx, loaded.row, named)
@@ -700,24 +726,23 @@ export class SuperTabStore {
}
async read(user: AuthUser, tabId: string, asAgentId?: string) {
- return transaction(this.db, async tx => {
- // A first load shows finished starts until they expire (design section 4.3).
- const { state } = await this.load(tx, user, tabId, { includeRecentStarts: true })
- const actor = await this.actor(tx, user, state, asAgentId)
- const now = this.now()
- const visible = visibleState(state, actor, now)
- const mentions = actor.kind === 'person' ? { mentionsWaiting: await mentionsWaiting(tx, tabId, state, actor, now) } : {}
- const teamMachines = visible.machines?.length ? await this.teamMachines(tx, state) : []
- const otherWorkspace = visible.machines?.length ? await this.otherWorkspaceMachines(tx, state) : []
- return { lastChangedAt: (await lastChangedAt(tx, [tabId])).get(tabId) ?? 0, state: { ...visible, ...mentions, machines: visible.machines?.map(pool => {
- const stored = state.machines!.find(m => m.machineId === pool.machineId)!
- // Whether it takes teammates' projects now, and if not why (machines.ts teamWorkRefusal): what routing leaves it
- // out for, and what Desktop can show on the machine.
- const refusal = teamWorkRefusal(stored, now, teamMachines.includes(pool.machineId), teamAccountRequired(state), otherWorkspace.includes(pool.machineId))
- return { ...pool, availableSlots: Math.max(0, pool.maxAgents - machineSlots(state, stored)),
- teamWork: refusal ? { allowed: false as const, ...refusal } : { allowed: true as const } }
- }) } }
- })
+ const tx = this.db, view = await this.view([tabId])
+ // A first load shows finished starts until they expire (design section 4.3).
+ const { state } = await this.readLoad(user, tabId, { includeRecentStarts: true }, view)
+ const actor = await this.actor(tx, user, state, asAgentId)
+ const now = this.now()
+ const visible = visibleState(state, actor, now)
+ const mentions = actor.kind === 'person' ? { mentionsWaiting: await mentionsWaiting(tx, tabId, state, actor, now) } : {}
+ const teamMachines = visible.machines?.length ? await this.teamMachines(tx, state) : []
+ const otherWorkspace = visible.machines?.length ? await this.otherWorkspaceMachines(tx, state) : []
+ return { lastChangedAt: await readLastChangedAt(tx, view, tabId) ?? 0, state: { ...visible, ...mentions, machines: visible.machines?.map(pool => {
+ const stored = state.machines!.find(m => m.machineId === pool.machineId)!
+ // Whether it takes teammates' projects now, and if not why (machines.ts teamWorkRefusal): what routing leaves it
+ // out for, and what Desktop can show on the machine.
+ const refusal = teamWorkRefusal(stored, now, teamMachines.includes(pool.machineId), teamAccountRequired(state), otherWorkspace.includes(pool.machineId))
+ return { ...pool, availableSlots: Math.max(0, pool.maxAgents - machineSlots(state, stored)),
+ teamWork: refusal ? { allowed: false as const, ...refusal } : { allowed: true as const } }
+ }) } }
}
async start(user: AuthUser, tabId: string, startId: string, asAgentId?: string, hostMachineId?: string) {
@@ -1541,45 +1566,42 @@ export class SuperTabStore {
* a sequence, which is enough to poll for new messages. */
async messages(user: AuthUser, tabId: string, conversationId: string,
cursor: { after?: number; afterId?: string; before?: number; beforeId?: string; messageId?: string }, asAgentId?: string) {
- return transaction(this.db, async tx => {
- const { state } = await this.load(tx, user, tabId, { groupIds: [conversationId] })
- const actor = await this.actor(tx, user, state, asAgentId)
- const group = readConversation(state, actor, conversationId)
- const forward = cursor.after !== undefined
- const past = (sequence: number, id: string | undefined, direction: 'gt' | 'lt') => id === undefined ? { sequence: { [direction]: sequence } }
- : { OR: [{ sequence: { [direction]: sequence } }, { sequence, id: { [direction]: id } }] }
- const where = { tabId, conversationId,
- ...(cursor.messageId ? { id: cursor.messageId } : {}),
- ...(forward ? past(cursor.after!, cursor.afterId, 'gt') : cursor.before !== undefined ? past(cursor.before, cursor.beforeId, 'lt') : {}) }
- // An agent-only notice is listed for its own recipient alone. Hidden rows never shorten a page: Desktop reads on from the
- // last row it shows, so the page reads past them until it has 101 rows this reader may see, or reaches the end.
- const rows: SuperTabMessage[] = []
- for (let skip = 0; rows.length <= 100; skip += 101) {
- const order = forward ? 'asc' as const : 'desc' as const
- const batch = await tx.superTabMessage.findMany({ where, orderBy: [{ sequence: order }, { id: order }], skip, take: 101 })
- rows.push(...batch.filter(m => !agentOnly(m.source) || (actor.kind === 'agent' && m.recipientIds.includes(actor.agentId))))
- if (batch.length < 101) break
- }
- const page = rows.slice(0, 100)
- const next = rows.length > 100 ? { sequence: page.at(-1)!.sequence, id: page.at(-1)!.id } : null
- if (!forward) page.reverse()
- const reactions = page.length ? reactionCounts(state, await pageReactions(tx, tabId, page.map(m => m.id))) : new Map()
- const deliveries = page.length ? await pageDeliveries(tx, tabId, page.map(m => m.id)) : []
- return { messages: page.map(m => ({ ...m, mentions: m.mentions ?? [], reactions: reactions.get(m.id) ?? [],
- deliveries: deliveries.filter(d => d.messageId === m.id) })), next,
- // A poll for new messages learns here that a reaction changed on a message it already shows (reactions.ts).
- reactionSeq: group.reactionSeq ?? 0 }
- })
+ const tx = this.db
+ const { state } = await this.readLoad(user, tabId, { groupIds: [conversationId] })
+ const actor = await this.actor(tx, user, state, asAgentId)
+ const group = readConversation(state, actor, conversationId)
+ const forward = cursor.after !== undefined
+ const past = (sequence: number, id: string | undefined, direction: 'gt' | 'lt') => id === undefined ? { sequence: { [direction]: sequence } }
+ : { OR: [{ sequence: { [direction]: sequence } }, { sequence, id: { [direction]: id } }] }
+ const where = { tabId, conversationId,
+ ...(cursor.messageId ? { id: cursor.messageId } : {}),
+ ...(forward ? past(cursor.after!, cursor.afterId, 'gt') : cursor.before !== undefined ? past(cursor.before, cursor.beforeId, 'lt') : {}) }
+ // An agent-only notice is listed for its own recipient alone. Hidden rows never shorten a page: Desktop reads on from the
+ // last row it shows, so the page reads past them until it has 101 rows this reader may see, or reaches the end.
+ const rows: SuperTabMessage[] = []
+ for (let skip = 0; rows.length <= 100; skip += 101) {
+ const order = forward ? 'asc' as const : 'desc' as const
+ const batch = await tx.superTabMessage.findMany({ where, orderBy: [{ sequence: order }, { id: order }], skip, take: 101 })
+ rows.push(...batch.filter(m => !agentOnly(m.source) || (actor.kind === 'agent' && m.recipientIds.includes(actor.agentId))))
+ if (batch.length < 101) break
+ }
+ const page = rows.slice(0, 100)
+ const next = rows.length > 100 ? { sequence: page.at(-1)!.sequence, id: page.at(-1)!.id } : null
+ if (!forward) page.reverse()
+ const reactions = page.length ? reactionCounts(state, await pageReactions(tx, tabId, page.map(m => m.id))) : new Map()
+ const deliveries = page.length ? await pageDeliveries(tx, tabId, page.map(m => m.id)) : []
+ return { messages: page.map(m => ({ ...m, mentions: m.mentions ?? [], reactions: reactions.get(m.id) ?? [],
+ deliveries: deliveries.filter(d => d.messageId === m.id) })), next,
+ // A poll for new messages learns here that a reaction changed on a message it already shows (reactions.ts).
+ reactionSeq: group.reactionSeq ?? 0 }
}
/** The caller's own tags in this Swarm, newest first: still waiting, or answered and marked done. */
async mentions(user: AuthUser, tabId: string, state: 'waiting' | 'earlier') {
this.personOnly(user)
- return transaction(this.db, async tx => {
- const { state: tab } = await this.load(tx, user, tabId, { allGroups: true })
- await this.actor(tx, user, tab)
- return { mentions: await listMentions(tx, tabId, tab, { kind: 'person', userId: user.sub, email: user.email }, this.now(), state === 'waiting') }
- })
+ const { state: tab } = await this.readLoad(user, tabId, { allGroups: true })
+ await this.actor(this.db, user, tab)
+ return { mentions: await listMentions(this.db, tabId, tab, { kind: 'person', userId: user.sub, email: user.email }, this.now(), state === 'waiting') }
}
/** The caller's decisions in this Swarm, newest first: a person's asked of them, an agent's asked by it (a Lead's also
@@ -1769,17 +1791,16 @@ export class SuperTabStore {
// Watching is a person's: a team session has no person to watch as.
if (isTeamSession(user)) return false
try {
- return await transaction(this.db, async tx => {
- const { state } = await this.load(tx, user, tabId)
- person(state, user.sub)
- await requireWorkspaceAccess(tx, state, user.sub)
- const target = state.agents.find(a => a.machineId === machineId && a.harnessId === harnessId && a.machineOwnerId === machineOwnerId)
- if (!target) return false
- agent(state, target.id)
- if (!canWatchAgent(state, { kind: 'person', userId: user.sub, email: user.email }, target, await this.confidential(tx, target.id))) return false
- await this.agentGrant(tx, state, target)
- return true
- })
+ const tx = this.db
+ const { state } = await this.readLoad(user, tabId)
+ person(state, user.sub)
+ await requireWorkspaceAccess(tx, state, user.sub)
+ const target = state.agents.find(a => a.machineId === machineId && a.harnessId === harnessId && a.machineOwnerId === machineOwnerId)
+ if (!target) return false
+ agent(state, target.id)
+ if (!canWatchAgent(state, { kind: 'person', userId: user.sub, email: user.email }, target, await this.confidential(tx, target.id))) return false
+ await this.agentGrant(tx, state, target)
+ return true
} catch (error) { if (error instanceof SuperTabError) return false; throw error }
}
@@ -1791,7 +1812,7 @@ export class SuperTabStore {
/** Whether a terminal share may open, wherever it came from (routes/harnessShares.ts): a confidential session's only for
* its machine's person, even an ordinary share made outside any Swarm; a Swarm's share only while that Swarm allows it. */
async mayOpenShare(user: AuthUser, share: { superTabId: string | null; machineId: string; agentId: string; ownerId: string }): Promise<boolean> {
- if (user.sub !== share.ownerId && await transaction(this.db, tx => this.confidential(tx, stableId(share.machineId, share.agentId)))) return false
+ if (user.sub !== share.ownerId && await this.confidential(this.db, stableId(share.machineId, share.agentId))) return false
return !share.superTabId || this.mayObserve(user, share.superTabId, share.machineId, share.agentId, share.ownerId)
}
diff --git a/backend/src/superTabs/sweeps.ts b/backend/src/superTabs/sweeps.ts
index b14d85b8a..a513a240e 100644
--- a/backend/src/superTabs/sweeps.ts
+++ b/backend/src/superTabs/sweeps.ts
@@ -3,6 +3,7 @@ import type { AgentStart, Conversation } from './model.js'
import { clock, guardedWrite, PartConflict, type HeadData, type ProgressData } from './parts/index.js'
import { DORMANT_AFTER_MS, RETENTION_MS } from './parts/types.js'
import { logger } from '../utils/logger.js'
+import { swarmTransaction } from './cache.js'
/** Parts whose key is never used twice (conversations, starts, projects' progress) lose their tombstone at `purgeAt`.
* A person, agent or machine can come back under the same key, so its tombstone is kept for good, compacted to its
@@ -34,7 +35,7 @@ const BATCH = 500
/** One guarded write in its own transaction. A part that changed since it was read is left for the next sweep. */
async function guarded(db: PrismaClient, write: Parameters<typeof guardedWrite>[2] & { tabId: string }): Promise<boolean> {
try {
- await db.$transaction(tx => guardedWrite(tx, write.tabId, write, clock.next()))
+ await swarmTransaction(db, tx => guardedWrite(tx, write.tabId, write, clock.next()))
return true
} catch (error) {
if (error instanceof PartConflict) return false
@@ -102,7 +103,7 @@ async function expireInvitations(db: PrismaClient, now: number): Promise<number>
const kept = data.invitations.filter(i => i.expiresAt > now)
if (kept.length === data.invitations.length) continue
try {
- await db.$transaction(async tx => {
+ await swarmTransaction(db, async tx => {
await guardedWrite(tx, head.tabId, { op: 'update', kind: 'head', key: 'head', seen: head.version, data: { ...data, invitations: kept },
head: { groupCounter: head.groupCounter, visibilityEpoch: head.visibilityEpoch, sequenceBase: head.sequenceBase } }, clock.next())
const row = await tx.superTab.findUniqueOrThrow({ where: { id: head.tabId }, select: { revision: true } })
diff --git a/backend/src/superTabs/workspaceStore.ts b/backend/src/superTabs/workspaceStore.ts
index 300aa480e..a3f5d40cf 100644
--- a/backend/src/superTabs/workspaceStore.ts
+++ b/backend/src/superTabs/workspaceStore.ts
@@ -29,6 +29,9 @@ const checked = new WeakMap<object, Map<string, boolean>>()
* per person per transaction, which reads one snapshot. */
export async function hasWorkspaceAccess(tx: Transaction, state: Pick<SuperTabState, 'workspaceId'>, userId: string): Promise<boolean> {
if (!state.workspaceId) return !workspacesRequired()
+ // Remembered only within a transaction. A read outside one (the read cache's, cache.ts) runs on the process's own client,
+ // which lives as long as the process: an answer kept on it would outlive the person's removal.
+ if (typeof (tx as { $transaction?: unknown }).$transaction === 'function') return !!await activeMember(tx, state.workspaceId, userId)
const answers = checked.get(tx) ?? new Map<string, boolean>()
checked.set(tx, answers)
const key = memberId(state.workspaceId, userId)
diff --git a/backend/src/worker.ts b/backend/src/worker.ts
index cf633c5a4..e58acb09c 100644
--- a/backend/src/worker.ts
+++ b/backend/src/worker.ts
@@ -1,5 +1,6 @@
import 'dotenv/config'
-import { closeBus } from './lib/bus.js'
+import { closeBus, redisSwarmStamps } from './lib/bus.js'
+import { configureSwarmStamps } from './superTabs/cache.js'
import { reconcileCommunityLikeCounts } from './lib/db/communityLikeCounts.js'
import { backfillMachineFreeEntitlements } from './lib/db/freeEntitlement.js'
import {
@@ -72,6 +73,8 @@ async function start(): Promise<void> {
machineBillingWorkerService.start()
productUsagePeopleRollupService.start()
+ // The worker's sweeps and archive write Swarms: they bump the stamps the API processes read by. It reads too little to keep a cache.
+ configureSwarmStamps(redisSwarmStamps(), { reads: false })
// Archive notes and Workspace removals write messages too: their recipients' hosts hear of them at once.
const swarms = new SuperTabStore(prisma, Date.now, null, { deliveriesWaiting: announceSwarmDeliveries })
stopSweeps = startSwarmSweeps(prisma, undefined, () => swarms.workspaces.finishRemovals())
diff --git a/backend/vitest.super-tabs.config.ts b/backend/vitest.super-tabs.config.ts
index 09463952c..cb5657057 100644
--- a/backend/vitest.super-tabs.config.ts
+++ b/backend/vitest.super-tabs.config.ts
@@ -3,5 +3,5 @@ import { defineConfig } from 'vitest/config'
export default defineConfig({ test: { include: ['integration/superTabs.test.ts', 'integration/superTabSharing.test.ts', 'integration/superTabGithub.test.ts', 'integration/superTabTeam.test.ts', 'integration/superTabWorkspaces.test.ts',
'integration/superTabRead.test.ts', 'integration/superTabLeaderboard.test.ts', 'integration/superTabMigration.test.ts', 'integration/superTabLiveness.test.ts', 'integration/superTabTidy.test.ts',
'integration/superTabProfile.test.ts', 'integration/superTabLegacyDrop.test.ts', 'integration/shareRaces.test.ts',
- 'integration/superTabIndexes.test.ts'], hookTimeout: 120000, testTimeout: 30000,
+ 'integration/superTabIndexes.test.ts', 'integration/superTabCache.test.ts'], hookTimeout: 120000, testTimeout: 30000, setupFiles: ['integration/cacheOn.test-support.ts'],
fileParallelism: false, env: { NODE_ENV: 'test', HARNESS_CREDENTIAL_ENCRYPTION_KEY: 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=' } } })
diff --git a/docs/design/2026-10-28-swarm-read-cache.md b/docs/design/2026-10-28-swarm-read-cache.md
new file mode 100644
index 000000000..3f5cada71
--- /dev/null
+++ b/docs/design/2026-10-28-swarm-read-cache.md
@@ -0,0 +1,73 @@
+# Design: Swarm read cache by stamp (Stability Roadmap 2.2 + 2.6)
+
+Every backend process keeps what it read of a Swarm's parts, keyed by that Swarm's stamp in Redis. A read with an
+unchanged stamp does not ask MongoDB for the Swarm's parts. Code: `backend/src/superTabs/cache.ts` (stamps, cache,
+bumps), `backend/src/superTabs/cachedReads.ts` (reads built from what the cache holds).
+
+## How it works
+
+- **Stamps.** Redis hash `swarm:stamp:{tabId}` with two counters:
+ - `main` moves on any commit that changes the Swarm's parts.
+ - `machine` moves when a machine part's own reading is the only change (a report every 15 s). What a machine's host
+ hears of is unmarked by such a write (`parts/commit.ts hostMarker`), so nothing else depends on it. Such a write
+ dates the part's `readAt`, not its `changedAt`. The change feed reads both. The Swarm's `lastChangedAt` (max
+ `changedAt`) therefore no longer moves every 15 s for every machine. Clients keep the part with the higher
+ version and never read `changedAt`.
+ - Touches change no data and move nothing.
+- **Bump after commit.** Every part write records its Swarm in the transaction (`parts/write.ts guardedWrite`,
+ `parts/load.ts insertParts`). `swarmTransaction` bumps those Swarms after the commit, before the caller answers. Any
+ read after the answer, on any process, therefore sees the change. This covers the store's commands, sweeps,
+ migration and rollback. Each bump is also broadcast on the bus (`swarm-stamp`), so other processes free their
+ entries early.
+- **Read.** A read takes the stamps first, then serves what it holds at those stamps, or loads and holds the result
+ under the stamps it read. A commit during the load moves the stamp, so the next read loads again.
+- **What is held.** The stored state, the same for every reader:
+ - the active parts, without machines (`main`);
+ - the machine parts (`main` + `machine`);
+ - quiet groups named by id or all of them (`main`);
+ - the light selections the change and host reads use (`main`);
+ - the change feed from 15 s before the cursor of the read that missed (`main` + `machine`);
+ - `lastChangedAt`, computed by `parts/load.ts lastChangedAt` once per main stamp (`main`). It is not written to a
+ stored field: a field every commit writes is the hotspot the parts design rejected (its section 3.7), and a write
+ after the commit could be lost.
+ - the host form's signature and state (`main`).
+- **Permissions after the cache.** Every reader's view is built from the held state by `read/views.ts readerView`, the
+ only path to a client, as before. Held values are frozen, so no reader can change what the next one reads.
+- **Reassemble once.** A read reassembles its parts once, sharing the stored records (`reassemble(parts, { share: true })`),
+ and `readerView` takes that state instead of reassembling again. Commands still copy what they load.
+- **No transactions for reads.** Reads run outside interactive transactions. Only a miss's load of the base parts (or
+ of a light selection, or the host state) runs in one read transaction, so it is one snapshot. Nothing is retried.
+
+## Design
+
+- **Growth:** the LRU's byte budget (`SWARM_CACHE_MB`, default 64 MB per process, counted as JSON size) drops the least
+ recently used entries. Entries also expire after 60 s, and a bump drops the Swarm's entries on every process.
+ Redis keys expire 30 days after a Swarm's last change.
+- **Concurrency:** there are no locks. A read keeps its result under the stamps it read before loading. Concurrent
+ misses of one entry share one load. Commands never read from the cache; they load in their own transaction as before.
+- **Read cost:** a hit costs one Redis `HMGET` (a pipeline for a host's Swarms). A miss reloads only the entries whose
+ stamps moved: a machine report reloads only the machine parts.
+- **Scale:** at 10x Swarms or processes, Redis does one HMGET per read and one script call plus one publish per
+ commit. The first warning would be the LRU's evictions (hit rate falling) and Redis CPU. Watch the miss logs and the
+ roadmap 1.1 Mongo query rate.
+- **Compatibility:** responses are unchanged, and old CLI, Desktop and phone clients read the same shapes. During a
+ rolling deploy, old processes don't bump. An entry on a new process is then stale for at most 60 s after an old
+ process's write. An old process also misses machine readings dated by `readAt` until the machine's next settings
+ change. Recommended rollout: deploy with `SWARM_READ_CACHE=off`, then turn it on once every pod runs the release.
+ Once every process is new, a commit is visible to the next read everywhere.
+- **Migration + rollback:** nothing is stored outside Redis keys, which are disposable. Roll back by deploying the
+ previous release, or with `SWARM_READ_CACHE=off`. Writes still bump with the switch off, so processes that keep the
+ cache on stay right.
+- **Audience:** the server applies every permission after the cache, from the same `readerView` and `visibleState` as
+ before. The integration test reads a private conversation as two readers, in both orders, on one cache.
+- **Identity:** stamps are per Swarm; nothing in the cache is keyed by, or sent as, the machine owner's login.
+- **Failure:** Redis down or slower than 200 ms means reads go straight to MongoDB and nothing is held. A failed
+ bump is logged. The process forgets the Swarm and retries the bump every second until Redis takes it. Other
+ processes keep the old entry until then, at most 60 s. A process killed between commit and bump leaves other
+ processes stale for at most 60 s.
+- **Platforms:** backend only (Node, Linux pods). There is no client change.
+
+## Measured
+
+See the PR for the benchmark before and after (`npm run bench:swarm-state -- --cache redis://…`), against the
+roadmap's KPIs.