Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Merge batch 11-02 b2110: #2613 #2612 #2609 #2611 (#2620)

0xken15c9276c6fd658
diff --git a/backend/integration/superTabCommandLoads.test.ts b/backend/integration/superTabCommandLoads.test.ts
new file mode 100644
index 000000000..0ed82f981
--- /dev/null
+++ b/backend/integration/superTabCommandLoads.test.ts
@@ -0,0 +1,114 @@
+import { randomUUID } from 'node:crypto'
+import { execFile } from 'node:child_process'
+import { promisify } from 'node:util'
+import { join } from 'node:path'
+import { tmpdir } from 'node:os'
+import { afterAll, beforeAll, describe, expect, it } from 'vitest'
+import { MongoMemoryReplSet } from 'mongodb-memory-server-core'
+import { PrismaClient, type Prisma } from '@prisma/client'
+import type { AuthUser } from '../src/lib/ssoAuth.js'
+import { SuperTabStore } from '../src/superTabs/store.js'
+import type { Command } from '../src/superTabs/commands.js'
+import type { Conversation } from '../src/superTabs/model.js'
+
+// Ken's prod DB profile, 2026-11-02: a command naming the PM read every group the PM was ever in, 1,628 of them and growing with
+// every task, up to 1.66 s a read. Those that write its groups need them; those that only ask about it do not.
+let replica: MongoMemoryReplSet
+let db: PrismaClient
+const owner: AuthUser = { sub: '220000000000000000000001', email: '[email protected]', autonomousEnv: 'prod', role: 'user' }
+const config = { engine: 'codex', model: 'fixture-model', effort: 'high', contextWindow: 200000, engineVersion: 'fixture' }
+
+beforeAll(async () => {
+ replica = await MongoMemoryReplSet.create({ binary: { version: '7.0.24', downloadDir: join(tmpdir(), 'harness-super-tab-mongodb-binaries') },
+ replSet: { count: 1, ip: '127.0.0.1', storageEngine: 'wiredTiger' } })
+ const uri = replica.getUri(`swarm_command_loads_test_${randomUUID().replaceAll('-', '_')}`)
+ await promisify(execFile)(join(process.cwd(), 'node_modules/.bin/prisma'), ['db', 'push', '--skip-generate'], {
+ env: { ...process.env, DATABASE_URL: uri }, timeout: 60000,
+ })
+ db = new PrismaClient({ datasources: { db: { url: uri } } })
+ await db.user.create({ data: { id: owner.sub, externalId: `fixture-${owner.sub}`, email: owner.email, autonomousEnv: owner.autonomousEnv } })
+ await db.machine.create({ data: { machineId: 'mac', userId: owner.sub, apiKey: 'fixture-mac', managerId: 'fixture', autonomousEnv: 'prod', authMode: 'remote' } })
+}, 120_000)
+afterAll(async () => {
+ await db?.$disconnect()
+ await replica?.stop()
+})
+
+/** A Swarm with a coordinator, a worker and [dormant] archived-free dormant groups holding the coordinator. */
+async function swarm(dormant = 0) {
+ const store = new SuperTabStore(db)
+ const tab = (await store.create(owner, randomUUID(), 'Loads', 'public')).id
+ const execute = (command: Command, asAgentId?: string) => store.execute(owner, tab, { operationId: randomUUID(), command }, asAgentId)
+ await execute({ type: 'machine.contribute', machineId: 'mac', name: 'Mac', cwd: '/repo', engines: ['codex'], maxAgents: 8 })
+ const harness = { lead: randomUUID(), worker: randomUUID() }
+ const lead = String((await execute({ type: 'agent.add', conversationId: 'lead', machineId: 'mac', harnessId: harness.lead, name: 'Lead', configuration: config, lead: true })).result.agentId)
+ const group = String((await execute({ type: 'conversation.create', title: 'Ship the parser', brief: 'Implement it', checks: [] })).result.conversationId)
+ const worker = String((await execute({ type: 'agent.add', conversationId: group, machineId: 'mac', harnessId: harness.worker, name: 'Worker', configuration: config, lead: false })).result.agentId)
+ const part = await db.swarmGroup.findFirstOrThrow({ where: { tabId: tab, key: group } })
+ const { id: _id, ...shape } = part
+ const rows: Prisma.SwarmGroupCreateManyInput[] = Array.from({ length: dormant }, (_, i) => {
+ const key = `old-${i}`
+ return { ...shape, id: `${tab}:group:${key}`, key, dormant: true, agentIds: [lead], executionMachineId: null,
+ data: { ...part.data as object, id: key, title: `Old ${i}`, agentIds: [lead], leadId: lead, assignment: null, dormant: true } } as Prisma.SwarmGroupCreateManyInput
+ })
+ for (let i = 0; i < rows.length; i += 200) await db.swarmGroup.createMany({ data: rows.slice(i, i + 200) })
+ return { store, tab, execute, lead, worker, group, harness }
+}
+
+/** The SwarmGroup rows Mongo returned for [run], whichever read asked. */
+async function groupsRead(run: () => Promise<unknown>): Promise<number> {
+ await db.$runCommandRaw({ profile: 2 })
+ await run()
+ await db.$runCommandRaw({ profile: 0 })
+ const found = await db.$runCommandRaw({ find: 'system.profile', filter: { ns: { $regex: '\\.SwarmGroup$' } }, projection: { nreturned: 1 }, batchSize: 100000 }) as
+ unknown as { cursor: { firstBatch: Array<{ nreturned?: number }> } }
+ await db.$runCommandRaw({ drop: 'system.profile' })
+ return found.cursor.firstBatch.reduce((sum, row) => sum + (row.nreturned ?? 0), 0)
+}
+
+const FEW = 100
+
+describe('a command naming the coordinator', () => {
+ it('agent.alert reads none of its 1,000 dormant groups, and its row still lands', async () => {
+ const f = await swarm(1000)
+ const read = await groupsRead(() => f.execute({ type: 'agent.alert', agentId: f.lead, alert: 'prompt', active: true }, f.lead))
+ expect(read).toBeLessThan(FEW)
+ const row = (await db.superTabMessage.findMany({ where: { tabId: f.tab } })).find(m => (m.source as { kind?: string } | null)?.kind === 'terminal-alert')
+ expect(row?.conversationId).toBe('lead')
+ })
+
+ it('agent.rebind reads none of them', async () => {
+ const f = await swarm(1000)
+ const read = await groupsRead(() => f.execute({ type: 'agent.rebind', agentId: f.lead, machineId: 'mac', projectId: 'lead', harnessId: randomUUID(), previousHarnessId: f.harness.lead, configuration: config } as Command, f.lead))
+ expect(read).toBeLessThan(FEW)
+ })
+
+ it('agent.input reads none of them, and its terminal stays closed to another agent', async () => {
+ const f = await swarm(1000)
+ let refused: unknown
+ const read = await groupsRead(() => f.execute({ type: 'agent.input', agentId: f.lead, text: 'hello' } as Command, f.worker).catch(error => { refused = error }))
+ expect(read).toBeLessThan(FEW)
+ expect(refused).toMatchObject({ code: expect.stringMatching(/TERMINAL_PRIVATE|FORBIDDEN|CONFIDENTIAL/) })
+ })
+})
+
+describe('what still reads an agent\'s dormant groups', () => {
+ it('a worker\'s alert finds its only group, asleep', async () => {
+ const f = await swarm()
+ const part = await db.swarmGroup.findFirstOrThrow({ where: { tabId: f.tab, key: f.group } })
+ await db.swarmGroup.update({ where: { id: part.id }, data: { dormant: true, data: { ...part.data as object, dormant: true } } })
+ await f.execute({ type: 'agent.alert', agentId: f.worker, alert: 'prompt', active: true }, f.worker)
+ const row = (await db.superTabMessage.findMany({ where: { tabId: f.tab } })).find(m => (m.source as { kind?: string } | null)?.kind === 'terminal-alert')
+ expect(row?.conversationId).toBe(f.group)
+ })
+
+ it('removing a worker takes it out of a dormant group too', async () => {
+ const f = await swarm()
+ const part = await db.swarmGroup.findFirstOrThrow({ where: { tabId: f.tab, key: f.group } })
+ await db.swarmGroup.update({ where: { id: part.id }, data: { dormant: true, data: { ...part.data as object, dormant: true } } })
+ await f.execute({ type: 'agent.remove', agentId: f.worker })
+ const after = await db.swarmGroup.findFirstOrThrow({ where: { tabId: f.tab, key: f.group } })
+ expect((after.data as unknown as Conversation).agentIds).not.toContain(f.worker)
+ expect(after.agentIds).not.toContain(f.worker)
+ })
+})
diff --git a/backend/integration/superTabHostGroups.test.ts b/backend/integration/superTabHostGroups.test.ts
new file mode 100644
index 000000000..d61406cc6
--- /dev/null
+++ b/backend/integration/superTabHostGroups.test.ts
@@ -0,0 +1,259 @@
+import { createHash, randomUUID } from 'node:crypto'
+import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
+import type { MongoMemoryReplSet } from 'mongodb-memory-server-core'
+import type { PrismaClient } from '@prisma/client'
+import type { AuthUser } from '../src/lib/ssoAuth.js'
+import { SuperTabStore } from '../src/superTabs/store.js'
+import { configureSwarmStamps, swarmTransaction, SwarmCache } from '../src/superTabs/cache.js'
+import { MemoryStamps } from '../src/superTabs/cache.test-support.js'
+import * as parts from '../src/superTabs/parts/index.js'
+import { guardedWrite } from '../src/superTabs/parts/write.js'
+import { listNamedIndexes } from '../src/superTabs/parts/indexed.js'
+import { DIRECT, readDelta } from '../src/superTabs/cachedReads.js'
+import { hostInputs } from '../src/superTabs/read/hostInputs.js'
+import { migrateSwarm } from '../src/superTabs/migration/migrate.js'
+import type { SuperTabState } from '../src/superTabs/model.js'
+import { writeLegacySwarm } from '../scripts/swarm-parts/legacy.js'
+import { owner as ownerActor, richSwarm, swarm } from '../src/superTabs/read/fixture.test-support.js'
+import { HostReads } from '../src/superTabs/read/host.js'
+import { planDatabase, profiled, reads } from './plans.test-support.js'
+
+// The host signature's conversations (read/hostInputs.ts) through the patched read cache, against the `$group` pipeline it
+// replaced (prod's slow log, 2026-11-02: 76 slow, 24 s): the same string after every kind of write, changing exactly when a
+// host-relevant conversation does, and a check after one change reading that one conversation, not the Swarm's. A fresh
+// loopback replica set; never DATABASE_URL.
+let replica: MongoMemoryReplSet
+let db: PrismaClient
+let stamps: MemoryStamps
+const owner: AuthUser = { sub: ownerActor.userId, email: ownerActor.email, autonomousEnv: 'prod', role: 'user' }
+/** Conversations added to the fixture's few, so a read of all of them is plainly more than a read of one. */
+const EXTRA = 150
+
+beforeAll(async () => {
+ ;({ replica, db } = await planDatabase('swarm_host_groups_test'))
+ await db.user.create({ data: { id: owner.sub, externalId: `fixture-${owner.sub}`, email: owner.email, autonomousEnv: 'prod' } })
+ // As each server does at startup (server.ts): reads name only the indexes the database has.
+ await listNamedIndexes(db)
+})
+afterAll(async () => {
+ configureSwarmStamps(null)
+ await db?.$disconnect()
+ await replica?.stop()
+})
+beforeEach(() => { stamps = new MemoryStamps(); configureSwarmStamps(stamps) })
+afterEach(() => configureSwarmStamps(null))
+
+async function scene() {
+ const s = richSwarm(Date.now())
+ const tabId = `tab-${randomUUID()}`
+ await writeLegacySwarm(db, { state: { ...structuredClone(s.state), id: tabId } as SuperTabState, revision: 10 })
+ const report = await migrateSwarm(db, tabId)
+ if (!report.ok) throw new Error(report.error)
+ await swarmTransaction(db, async tx => {
+ const all = await parts.loadAll(tx, tabId)
+ const model = all.groups.find(g => g.key === s.web)!
+ for (let i = 0; i < EXTRA; i++) {
+ const key = `extra-${i}`
+ await guardedWrite(tx, tabId, { op: 'insert', kind: 'group', key, ordinal: 10_000 + i, data: { ...model.data!, id: key, title: `Extra ${i}`, assignment: null } }, Date.now())
+ }
+ })
+ await age(tabId)
+ return { tabId, web: s.web }
+}
+
+/** Every conversation last changed ten minutes ago, as a Swarm's quiet ones are: the cache's overlap re-reads the newest few seconds. */
+const age = (tabId: string) => db.swarmGroup.updateMany({ where: { tabId }, data: { changedAt: new Date(Date.now() - 600_000) } })
+
+/** One write, committed as a command commits: in a Swarm transaction, guarded by the version read in it. */
+async function write(tabId: string, pick: (all: parts.SwarmParts) => parts.PartWrite | null, stamp = Date.now()): Promise<boolean> {
+ return swarmTransaction(db, async tx => {
+ const change = pick(await parts.loadAll(tx, tabId))
+ if (!change) return false
+ await guardedWrite(tx, tabId, change, stamp)
+ return true
+ })
+}
+const group = (all: parts.SwarmParts, key: string) => all.groups.find(g => g.key === key && !g.deleted)!
+const update = (key: string, change: (data: parts.GroupData) => parts.GroupData) => (all: parts.SwarmParts): parts.PartWrite =>
+ ({ op: 'update', kind: 'group', key, seen: group(all, key).version, data: change(group(all, key).data!) })
+
+/** The signature as the `$group` pipeline gave it before (read/host.ts inputs, v1.3.86). */
+async function pipeline(tabId: string): Promise<string | null> {
+ const live = { tabId, deleted: false }, stamp = { key: true, changedAt: true } as const
+ const [head, members, agents, starts, machines, digests] = await Promise.all([
+ db.swarmHead.findUnique({ where: { id: `${tabId}:head` }, select: { changedAt: true } }),
+ db.swarmMember.findMany({ where: live, select: stamp }), db.swarmAgent.findMany({ where: live, select: stamp }),
+ db.swarmStart.findMany({ where: live, select: stamp }),
+ db.swarmMachine.findMany({ where: live, select: { key: true, hostChangedAt: true } }),
+ db.swarmGroup.aggregateRaw({ pipeline: [{ $match: { tabId, deleted: false } }, { $sort: { key: 1 } },
+ { $group: { _id: null, missing: { $sum: { $cond: [{ $ifNull: ['$hostDigest', false] }, 0, 1] } },
+ all: { $push: { $concat: [{ $ifNull: ['$hostDigest', ''] }, { $cond: ['$dormant', '1', '0'] }] } } } },
+ { $project: { _id: 0, missing: 1, all: { $reduce: { input: '$all', initialValue: '', in: { $concat: ['$$value', '$$this'] } } } } }] }),
+ ])
+ const groups = (digests as unknown as Array<{ missing: number; all: string }>)[0] ?? { missing: 0, all: '' }
+ if (!head || groups.missing) return null
+ const keyed = <T extends { key: string }>(rows: T[]) => rows.sort((a, b) => a.key < b.key ? -1 : a.key > b.key ? 1 : 0)
+ return createHash('sha256').update(JSON.stringify([head.changedAt, keyed(members), keyed(agents), keyed(starts), keyed(machines), groups.all]))
+ .digest('base64url').slice(0, 22)
+}
+
+/** The signature through the cache, as a host check reads it. */
+const cachedInputs = async (cache: SwarmCache, tabId: string) => hostInputs(db, await cache.at([tabId]), tabId)
+
+describe('the host signature\'s conversations through the patched cache', () => {
+ it('gives the pipeline\'s string after every kind of write, patching rather than loading again', { timeout: 120_000 }, async () => {
+ const { tabId, web } = await scene()
+ const delta = vi.fn(readDelta(db))
+ const cache = new SwarmCache(stamps, { delta })
+ const store = new SuperTabStore(db, Date.now, null, { readCache: cache })
+ let n = 0
+ const steps: Array<[string, () => Promise<unknown>]> = [
+ ['rename', () => write(tabId, update(web, g => ({ ...g, title: `T${++n}` })))],
+ ['sleep', () => write(tabId, update('extra-1', g => ({ ...g, dormant: true })))],
+ ['wake', () => write(tabId, update('extra-1', g => ({ ...g, dormant: false })))],
+ ['archive', () => write(tabId, update('extra-2', g => ({ ...g, archived: true })))],
+ ['agent leaves', () => write(tabId, all => { const g = all.groups.find(x => !x.deleted && x.data!.agentIds.length); return g ? update(g.key, d => ({ ...d, agentIds: d.agentIds.slice(1) }))(all) : null })],
+ ['new conversation', () => write(tabId, all => ({ op: 'insert', kind: 'group', key: `late-${++n}`, ordinal: 20_000 + n, data: { ...group(all, web).data!, id: `late-${n}`, assignment: null } }))],
+ ['conversation removed', () => write(tabId, all => ({ op: 'remove', kind: 'group', key: 'extra-3', seen: group(all, 'extra-3').version, data: group(all, 'extra-3').data }))],
+ ['a post', () => store.execute(owner, tabId, { operationId: `send-${randomUUID()}`, command: { type: 'message.send', conversationId: web, text: `m${++n}`, replyTo: null, recipientIds: [] } })],
+ ]
+ expect(await cachedInputs(cache, tabId)).toBe(await pipeline(tabId))
+ for (const [name, step] of steps) {
+ await step()
+ const expected = await pipeline(tabId)
+ expect(expected, name).not.toBeNull()
+ expect(await cachedInputs(cache, tabId), `${name}, patched`).toBe(expected)
+ expect(await hostInputs(db, DIRECT, tabId), `${name}, no cache`).toBe(expected)
+ }
+ // After the first load every refresh was a delta, not a read of the Swarm.
+ expect(delta.mock.calls.length).toBeGreaterThanOrEqual(steps.length - 1)
+ })
+
+ it('changes exactly when a host-relevant conversation changes: not for a rename or a post, for everything its rows read', { timeout: 60_000 }, async () => {
+ const { tabId, web } = await scene()
+ const cache = new SwarmCache(stamps, { delta: readDelta(db) })
+ const store = new SuperTabStore(db, Date.now, null, { readCache: cache })
+ const relevant: Array<[string, () => Promise<unknown>]> = [
+ ['put to sleep', () => write(tabId, update('extra-1', g => ({ ...g, dormant: true })))],
+ ['woken', () => write(tabId, update('extra-1', g => ({ ...g, dormant: false })))],
+ ['archived', () => write(tabId, update('extra-2', g => ({ ...g, archived: true })))],
+ ['an agent added', () => write(tabId, update('extra-4', g => ({ ...g, agentIds: [...g.agentIds, 'someone'] })))],
+ ['added', () => write(tabId, all => ({ op: 'insert', kind: 'group', key: 'added', ordinal: 30_000, data: { ...group(all, web).data!, id: 'added', assignment: null } }))],
+ ['removed', () => write(tabId, all => ({ op: 'remove', kind: 'group', key: 'extra-3', seen: group(all, 'extra-3').version, data: group(all, 'extra-3').data }))],
+ ]
+ const irrelevant: Array<[string, () => Promise<unknown>]> = [
+ ['renamed', () => write(tabId, update('extra-5', g => ({ ...g, title: 'Another name' })))],
+ ['a post', () => store.execute(owner, tabId, { operationId: `send-${randomUUID()}`, command: { type: 'message.send', conversationId: web, text: 'hello', replyTo: null, recipientIds: [] } })],
+ ]
+ let held = await cachedInputs(cache, tabId)
+ for (const [name, step] of relevant) {
+ await step()
+ const now = await cachedInputs(cache, tabId)
+ expect(now, name).not.toBe(held)
+ held = now
+ }
+ for (const [name, step] of irrelevant) {
+ await step()
+ expect(await cachedInputs(cache, tabId), name).toBe(held)
+ }
+ })
+
+ it('reads the Swarm\'s conversations once, from the index alone, and afterwards only the one that changed', async () => {
+ const { tabId, web } = await scene()
+ const cache = new SwarmCache(stamps, { delta: readDelta(db) })
+ const groupReads = (entries: Awaited<ReturnType<typeof profiled>>) => entries.filter(e => e.ns.endsWith('.SwarmGroup'))
+ const commands = (entries: Awaited<ReturnType<typeof profiled>>) => entries.filter(e => e.op !== 'getmore')
+ const sum = (entries: Awaited<ReturnType<typeof profiled>>, field: 'keysExamined' | 'docsExamined' | 'nreturned') => entries.reduce((n, e) => n + (e[field] ?? 0), 0)
+ let first: string | null = null
+ const cold = groupReads(await profiled(db, async () => { first = await cachedInputs(cache, tabId) }))
+ expect(first).not.toBeNull()
+ expect(commands(cold)).toHaveLength(1)
+ // Every conversation's key, digest and dormancy from the index: no document is read.
+ expect(sum(cold, 'keysExamined')).toBeGreaterThanOrEqual(EXTRA)
+ expect(sum(cold, 'docsExamined')).toBe(0)
+
+ const idle = groupReads(await profiled(db, async () => { expect(await cachedInputs(cache, tabId)).toBe(first) }))
+ expect(idle).toHaveLength(0)
+
+ await write(tabId, update(web, g => ({ ...g, archived: true })))
+ let next: string | null = null
+ const patched = groupReads(await profiled(db, async () => { next = await cachedInputs(cache, tabId) }))
+ expect(next).not.toBe(first)
+ expect(next).toBe(await pipeline(tabId))
+ // One read of the conversations changed since the last check, and it examines only them.
+ expect(commands(patched)).toHaveLength(1)
+ expect(sum(patched, 'nreturned')).toBe(1)
+ expect(sum(patched, 'docsExamined')).toBeLessThanOrEqual(2)
+ expect(sum(patched, 'keysExamined')).toBeLessThanOrEqual(2)
+ expect(reads(patched, 'SwarmGroup', ['changedAt'])).toHaveLength(1)
+ })
+
+ it('is null while a conversation has no digest, and the string again once it has one', async () => {
+ const { tabId } = await scene()
+ const cache = new SwarmCache(stamps, { delta: readDelta(db) })
+ expect(await cachedInputs(cache, tabId)).not.toBeNull()
+ // A conversation written before the digest existed, as such a part is stored.
+ await db.swarmGroup.updateMany({ where: { tabId, key: 'extra-7' }, data: { hostDigest: null, changedAt: new Date() } })
+ await stamps.bump(tabId, ['main'])
+ expect(await cachedInputs(cache, tabId)).toBeNull()
+ expect(await hostInputs(db, DIRECT, tabId)).toBeNull()
+ expect(await pipeline(tabId)).toBeNull()
+ await write(tabId, update('extra-7', g => ({ ...g, title: 'Rewritten' })))
+ expect(await cachedInputs(cache, tabId)).toBe(await pipeline(tabId))
+ expect(await cachedInputs(cache, tabId)).not.toBeNull()
+ })
+
+ it('a write committed after the load but stamped 20 s earlier is in the next patch', async () => {
+ const { tabId } = await scene()
+ const cache = new SwarmCache(stamps, { delta: readDelta(db) })
+ const before = await cachedInputs(cache, tabId)
+ await write(tabId, update('extra-6', g => ({ ...g, archived: true })), Date.now() - 20_000)
+ const after = await cachedInputs(cache, tabId)
+ expect(after).not.toBe(before)
+ expect(after).toBe(await pipeline(tabId))
+ })
+})
+
+describe('a host check', () => {
+ /** A Swarm with a host: a Lead and a worker on its machine, and EXTRA more conversations, each with a digest of its own. */
+ async function hosted() {
+ const machineId = `m-${randomUUID().slice(0, 8)}`
+ await db.machine.create({ data: { machineId, userId: owner.sub, apiKey: 'fixture-only', managerId: 'fixture', autonomousEnv: 'prod', authMode: 'remote' } })
+ const s = swarm(Date.now())
+ s.contribute(machineId, 'swarm')
+ s.add('lead-harness', 'lead', true, ownerActor, machineId, owner.sub)
+ const web = s.group('Web')
+ s.add('worker-harness', web, false, ownerActor, machineId, owner.sub)
+ const tabId = `tab-${randomUUID()}`
+ await writeLegacySwarm(db, { state: { ...structuredClone(s.state), id: tabId } as SuperTabState, revision: 10 })
+ const report = await migrateSwarm(db, tabId)
+ if (!report.ok) throw new Error(report.error)
+ const { id: _id, ...row } = (await db.swarmGroup.findFirst({ where: { tabId, key: web } }))!
+ await db.swarmGroup.createMany({ data: Array.from({ length: EXTRA }, (_, i) => {
+ const key = `more-${String(i).padStart(3, '0')}`
+ return { ...row, id: `${tabId}:${key}`, key, ordinal: 100 + i, hostDigest: `digest-${String(i).padStart(3, '0')}`,
+ agentIds: [], executionMachineId: null, data: { ...row.data as object, id: key, title: key, agentIds: [], assignment: null } }
+ }) })
+ await age(tabId)
+ return { tabId, machineId }
+ }
+
+ it('after one conversation changed reads that conversation, not every live one, and gives the rows\' etag as before', async () => {
+ const { tabId, machineId } = await hosted()
+ const cache = new SwarmCache(stamps, { delta: readDelta(db) })
+ const hosts = new HostReads(new SuperTabStore(db, Date.now, null, { readCache: cache }))
+ const first = await hosts.changes(owner, machineId, { full: true })
+ const etag = first.tabs.find(t => t.tabId === tabId)!.etag
+ // A conversation of the host's changes, as parts/commit.ts marks a write that concerns it.
+ await write(tabId, all => ({ op: 'update', kind: 'group', key: 'more-007', seen: group(all, 'more-007').version, data: { ...group(all, 'more-007').data!, archived: true },
+ host: { hostMachineIds: [machineId] } }))
+ let answer: Awaited<ReturnType<HostReads['changes']>> | undefined
+ const entries = (await profiled(db, async () => { answer = await hosts.changes(owner, machineId, { cursor: first.cursor, held: { [tabId]: etag } }) }))
+ .filter(e => e.ns.endsWith('.SwarmGroup'))
+ const tab = answer!.tabs.find(t => t.tabId === tabId)!
+ expect(tab.etag.slice(22)).toBe((await pipeline(tabId))!)
+ // The old check was a $group over every live conversation, planned on the live index (its key, digest and dormancy);
+ // the entry reads only what changed, through the changes index. (The host's rows read others of their own, not asserted here.)
+ expect(entries.filter(e => e.planSummary?.includes('hostDigest'))).toEqual([])
+ })
+})
diff --git a/backend/integration/superTabInbox.test.ts b/backend/integration/superTabInbox.test.ts
index 96a413b4e..6ac25260a 100644
--- a/backend/integration/superTabInbox.test.ts
+++ b/backend/integration/superTabInbox.test.ts
@@ -19,12 +19,15 @@ import { guardedWrite } from '../src/superTabs/parts/write.js'
import { SwarmReads } from '../src/superTabs/read/service.js'
import { HostReads } from '../src/superTabs/read/host.js'
import { buddyChatId } from '../src/superTabs/buddies.js'
-import { INBOX_READS, INBOX_WINDOW_MS, LATEST_CHATS, MAX_THREADS, PREVIEW_CHARS, resetInboxReads, THREAD_CHATS } from '../src/superTabs/read/inbox.js'
+import { INBOX_READS, INBOX_WINDOW_MS, LATEST_CHATS, MAX_THREADS, newestInThreads, perRange, PREVIEW_CHARS, resetInboxReads, SCAN_FIELDS, THREAD_CHATS } from '../src/superTabs/read/inbox.js'
import { SEED_SCAN } from '../src/superTabs/read/threadMarkSeeds.js'
import { stableId, type Author, type Conversation, type SuperTabState } from '../src/superTabs/model.js'
import { writeLegacySwarm } from '../scripts/swarm-parts/legacy.js'
import { NOW, outsider as outsiderActor, owner as ownerActor, peer as peerActor, swarm } from '../src/superTabs/read/fixture.test-support.js'
import { PROFILE_STAGES, profileEntry, type ProfileEntry } from './bench/profile.js'
+import { filterOf, profiled } from './plans.test-support.js'
+import { listNamedIndexes } from '../src/superTabs/parts/indexed.js'
+import type { Prisma } from '@prisma/client'
// A fresh loopback replica set, created here. This suite never reads DATABASE_URL or an existing database.
let replica: MongoMemoryReplSet
@@ -714,9 +717,10 @@ describe('what the inbox costs the database', () => {
reads.map(r => ` ${r.ns.split('.').at(-1)} ${r.op} ${r.planSummary ?? ''} docs=${r.docsExamined ?? 0}`).join('\n'))
// A fixed few reads whatever the chat count: the latest rows, the unread rows, their roots, the look-back for threads
// whose newer rows do not show the person (the read marks are another collection, and the person's thread marks of the
- // scanned chats one more indexed read). Nothing new to check for delivered copies here, so no read for them.
- expect(messageReads.length).toBeLessThanOrEqual(4)
- expect(reads.length).toBeLessThanOrEqual(7)
+ // scanned chats one more indexed read). Nothing new to check for delivered copies here, so no read for them. This scene's
+ // long threads pass the look-back's cap, so it reads a second time by union (inbox.ts newestInThreads, task #2148).
+ expect(messageReads.length).toBeLessThanOrEqual(5)
+ expect(reads.length).toBeLessThanOrEqual(8)
expect(rows.filter(r => r.ns.endsWith('.SuperTabThreadMark') && r.op !== 'getmore')).toHaveLength(1)
// Bounded by the caps, never by history: the long chat's rows are read only to its newest UNREAD_SCAN + 1, and each of its
// threads the person is not in only to MEMBER_SCAN rows before the mark (not its 301).
@@ -731,3 +735,99 @@ describe('what the inbox costs the database', () => {
expect(bytesFromDatabase).toBeLessThan(250_000)
}, 120_000)
})
+
+// Task #2148: Ken's profile showed these reads spending their time racing two plans per range, in a union no hint can reach.
+describe('reading many thread ranges at once', () => {
+ /** What one pipeline per thread answers, as these reads were written before: the answers the single read must keep. */
+ const oneByOne = (tabId: string, ranges: Array<{ conversationId: string; thread: string; sequence: Record<string, number> }>, limit: number, namedBy?: string) =>
+ perRange(db, ranges.map(r => [{ $match: { tabId, conversationId: r.conversationId, thread: r.thread, sequence: r.sequence } }, { $sort: { sequence: -1 } }, { $limit: limit },
+ ...namedBy ? [{ $match: { $or: [{ 'author.kind': 'person', 'author.id': namedBy }, { mentions: namedBy }] } }] : [],
+ { $project: { ...SCAN_FIELDS, conversationId: 1 } }] as Prisma.InputJsonValue[]))
+ /** Two chats, four threads: short, longer than any limit here, one nobody names the person in, and one with no row in its range. */
+ async function threads() {
+ const s = await scene(f => { f.group('Second') })
+ const second = (await store.readLoad(peer, s.tabId)).state.conversations.find(c => c.title === 'Second')!.id
+ const [shortRoot, longRoot, quietRoot, emptyRoot] = await insert(s.tabId, s.web, [1, 2, 3, 4].map(i => ({ text: `Root ${i}`, author: personAuthor(owner) })))
+ const [otherRoot] = await insert(s.tabId, second, [{ text: 'Other root', author: personAuthor(owner) }])
+ await insert(s.tabId, s.web, [
+ { text: 'Tagged', author: agentAuthor(s.worker), thread: shortRoot, mentions: [peer.sub] },
+ { text: 'Theirs', author: personAuthor(peer), thread: longRoot },
+ { text: 'Tagged early', author: agentAuthor(s.worker), thread: longRoot, mentions: [peer.sub] },
+ ...Array.from({ length: 30 }, (_, i) => ({ text: `Long ${i}`, author: agentAuthor(s.worker), thread: longRoot })),
+ { text: 'Tagged late', author: agentAuthor(s.worker), thread: longRoot, mentions: [peer.sub] },
+ ...Array.from({ length: 3 }, (_, i) => ({ text: `Quiet ${i}`, author: agentAuthor(s.worker), thread: quietRoot })),
+ ])
+ await insert(s.tabId, second, [{ text: 'Theirs there', author: personAuthor(peer), thread: otherRoot }, { text: 'Not theirs', author: agentAuthor(s.worker), thread: otherRoot }])
+ const top = (await db.superTabMessage.findFirstOrThrow({ where: { tabId: s.tabId, conversationId: s.web }, orderBy: { sequence: 'desc' } })).sequence, first = await sequenceOf(shortRoot!)
+ const web = (thread: string, sequence: Record<string, number>) => ({ conversationId: s.web, thread, sequence })
+ return { tabId: s.tabId, top, first, emptyRoot: emptyRoot!, ranges: (sequence: Record<string, number>) => [web(longRoot!, sequence), web(shortRoot!, sequence), web(quietRoot!, sequence),
+ web(emptyRoot!, sequence), { conversationId: second, thread: otherRoot!, sequence: { $gt: 0 } }] }
+ }
+
+ it('answers the rows one pipeline per thread answers, in the same order', async () => {
+ const f = await threads()
+ for (const sequence of [{ $lte: f.top }, { $gt: f.first }, { $lte: f.first + 10 }]) {
+ for (const limit of [1, 5, 10, 100]) {
+ expect(await newestInThreads(db, f.tabId, f.ranges(sequence), limit)).toEqual(await oneByOne(f.tabId, f.ranges(sequence), limit))
+ expect(await newestInThreads(db, f.tabId, f.ranges(sequence), limit, peer.sub)).toEqual(await oneByOne(f.tabId, f.ranges(sequence), limit, peer.sub))
+ }
+ }
+ }, 60_000)
+
+ it('names a person only within each thread\'s newest rows: an older tag or reply does not count', async () => {
+ const f = await threads()
+ const named = await newestInThreads(db, f.tabId, f.ranges({ $lte: f.top }), 5, peer.sub)
+ const texts = await db.superTabMessage.findMany({ where: { id: { in: named.map(m => m.id) } }, select: { text: true } })
+ expect(texts.map(m => m.text).sort()).toEqual(['Tagged', 'Tagged late', 'Theirs there'].sort())
+ }, 60_000)
+
+ it('cuts a thread to its limit in a batch under its cap, in the hinted read: a tag past the limit does not count', async () => {
+ const f = await threads()
+ await listNamedIndexes(db)
+ // The tag and the 6 rows after it: 7 rows, limit 5, and with four ranges that hold nothing the batch's cap is 7.
+ const tagged = await db.superTabMessage.findFirstOrThrow({ where: { tabId: f.tabId, text: 'Tagged early' } })
+ const [long, ...others] = f.ranges({ $gt: f.top })
+ const ranges = [{ ...long!, sequence: { $gte: tagged.sequence, $lte: tagged.sequence + 6 } }, ...others.map(r => ({ ...r, sequence: { $gt: 1_000_000 } }))]
+ let plain: Awaited<ReturnType<typeof newestInThreads>> = []
+ const entries = await profiled(db, async () => { plain = await newestInThreads(db, f.tabId, ranges, 5) })
+ const reads = entries.filter(e => e.ns.endsWith('.SuperTabMessage') && e.op !== 'getmore')
+ expect(reads).toHaveLength(1)
+ expect(reads[0]!.command?.hint).toEqual({ tabId: 1, conversationId: 1, thread: 1, sequence: 1 })
+ expect(plain.map(m => m.sequence)).toEqual([6, 5, 4, 3, 2].map(i => tagged.sequence + i))
+ expect(plain).toEqual(await oneByOne(f.tabId, ranges, 5))
+ expect(await newestInThreads(db, f.tabId, ranges, 5, peer.sub)).toEqual([])
+ expect(await oneByOne(f.tabId, ranges, 5, peer.sub)).toEqual([])
+ // A wider limit reaches the tag, so what cut it above is the limit.
+ expect((await newestInThreads(db, f.tabId, ranges, 7, peer.sub)).map(m => m.id)).toEqual([tagged.id])
+ }, 60_000)
+
+ it('answers nothing for no ranges, and a range with no rows is simply absent', async () => {
+ const f = await threads()
+ expect(await newestInThreads(db, f.tabId, [], 5)).toEqual([])
+ const empty = [{ conversationId: f.ranges({ $gt: 0 })[0]!.conversationId, thread: f.emptyRoot, sequence: { $gt: f.top } }]
+ expect(await newestInThreads(db, f.tabId, empty, 5)).toEqual([])
+ }, 60_000)
+
+ it('reads a batch in one operation hinted to the thread index, with no union to plan', async () => {
+ const f = await threads()
+ await listNamedIndexes(db)
+ // Limit 100 and few rows: the batch stays under its cap.
+ const entries = await profiled(db, () => newestInThreads(db, f.tabId, f.ranges({ $gt: f.first }), 100))
+ const reads = entries.filter(e => e.ns.endsWith('.SuperTabMessage') && e.op !== 'getmore')
+ expect(reads).toHaveLength(1)
+ expect(reads[0]!.command?.hint).toEqual({ tabId: 1, conversationId: 1, thread: 1, sequence: 1 })
+ // Every range is in its first filter (the profile keeps only that): nothing is left for a union to plan.
+ expect((filterOf(reads[0]!) as { $or: unknown[] }).$or).toHaveLength(5)
+ expect(reads[0]!.planSummary).toContain('thread: 1')
+ }, 60_000)
+
+ it('falls back to each thread\'s own limit when a batch has more rows than its cap, and reads no thread past it there', async () => {
+ const f = await threads()
+ await listNamedIndexes(db)
+ const entries = await profiled(db, () => newestInThreads(db, f.tabId, f.ranges({ $lte: f.top }), 1))
+ const reads = entries.filter(e => e.ns.endsWith('.SuperTabMessage') && e.op !== 'getmore')
+ expect(reads).toHaveLength(2)
+ expect(reads.filter(e => e.command?.hint)).toHaveLength(1)
+ expect(filterOf(reads.find(e => !e.command?.hint)!)).not.toHaveProperty('$or')
+ }, 60_000)
+})
diff --git a/backend/src/superTabs/cachedReads.test.ts b/backend/src/superTabs/cachedReads.test.ts
index 1688dba85..cc5f67fa1 100644
--- a/backend/src/superTabs/cachedReads.test.ts
+++ b/backend/src/superTabs/cachedReads.test.ts
@@ -172,7 +172,7 @@ describe('the delta a refresh reads', () => {
const aggregateRaw = vi.fn(async () => [])
const tx = Object.fromEntries(['swarmHead', 'swarmMember', 'swarmAgent', 'swarmGroup', 'swarmMachine', 'swarmStart', 'swarmProgress'].map(m => [m, { aggregateRaw }]))
const db = { $transaction: vi.fn(async (body: (tx: unknown) => Promise<unknown>) => body(tx)) }
- expect(await readDelta(db as never)('t', 5, ['group', 'start'])).toEqual({ parts: [], head: null })
+ expect(await readDelta(db as never)('t', 5, ['group', 'start'])).toEqual({ parts: [], head: null, hostDigests: new Map() })
expect(db.$transaction).toHaveBeenCalledTimes(1)
expect(aggregateRaw).toHaveBeenCalledTimes(2)
})
diff --git a/backend/src/superTabs/parts/load.ts b/backend/src/superTabs/parts/load.ts
index 079877ed2..c39d8e220 100644
--- a/backend/src/superTabs/parts/load.ts
+++ b/backend/src/superTabs/parts/load.ts
@@ -183,18 +183,25 @@ const changedPart = (kind: PartKind, row: Record<string, unknown>): Part => {
/** What changed in one Swarm after `since`, for the read cache to patch what it holds (cache.ts; spec 2026-11-02 Phase 1):
* the parts of `kinds`, and the head's own columns when the head is among them. */
-export interface SwarmDelta { parts: Part[]; head: HeadColumns | null }
+export interface SwarmDelta {
+ parts: Part[]
+ head: HeadColumns | null
+ /** Each changed group's stored host digest (null while it has none), by key: the host signature's entry patches from it
+ * (read/hostInputs.ts). A loader that does not carry it leaves it out, and that entry loads again. */
+ hostDigests?: ReadonlyMap<string, string | null>
+}
export async function loadDelta(tx: Transaction, tabId: string, since: number, kinds: readonly PartKind[]): Promise<SwarmDelta> {
- const parts: Part[] = []
+ const parts: Part[] = [], hostDigests = new Map<string, string | null>()
let head: HeadColumns | null = null
for (const [kind, rows] of await changedRows(tx, [tabId], new Date(since), kinds)) {
for (const row of rows) {
parts.push(changedPart(kind, row))
+ if (kind === 'group') hostDigests.set(row.key as string, typeof row.hostDigest === 'string' ? row.hostDigest : null)
if (kind === 'head') head = { groupCounter: Number(row.groupCounter), visibilityEpoch: Number(row.visibilityEpoch), sequenceBase: Number(row.sequenceBase) }
}
}
- return { parts, head }
+ return { parts, head, hostDigests }
}
/** The live starts of these agents (an agent is its machine and harness: write.ts extracted), whole, by the (tabId, agentId)
@@ -211,6 +218,8 @@ const COLLECTION = { head: 'SwarmHead', member: 'SwarmMember', agent: 'SwarmAgen
progress: 'SwarmProgress' } as const satisfies Record<PartKind, NamedCollection>
/** What a changed part is made of: its other fields are indexed copies of its data. */
const CHANGED_FIELDS = { tabId: 1, key: 1, version: 1, changedAt: 1, deleted: 1, ordinal: 1, data: 1, dormant: 1, readAt: 1,
+ // A group's stored host digest (no other kind has one): the host signature's entry patches from it.
+ hostDigest: 1,
// The head's own columns (no other kind has them): a patched base keeps them too.
groupCounter: 1, visibilityEpoch: 1, sequenceBase: 1 }
diff --git a/backend/src/superTabs/read/host.ts b/backend/src/superTabs/read/host.ts
index 4537442a0..9a56b7141 100644
--- a/backend/src/superTabs/read/host.ts
+++ b/backend/src/superTabs/read/host.ts
@@ -3,7 +3,6 @@
* that concerns it since the cursor (incremental). Grants and billing live outside the parts, so the host asks for a full
* read at start and at least every 60 s.
*/
-import { createHash } from 'node:crypto'
import type { Prisma, PrismaClient } from '@prisma/client'
import type { AuthUser } from '../../lib/ssoAuth.js'
import { machineAlive } from '../../lib/prisma.js'
@@ -21,12 +20,11 @@ import { plainHost } from '../parts/commit.js'
import { READ_FORMAT, type HostChanges, type HostChangesRequest } from './contract.js'
import { cursorFor, readCursor } from './cursor.js'
import { hostTab, type HostLookups } from './hostRows.js'
+import { hostInputs, ROW_HASH } from './hostInputs.js'
import { confidentialSessions } from '../confidential.js'
import { machineSwarms, swarmsIn } from '../swarmRows.js'
type Transaction = Prisma.TransactionClient
-/** The row hash's length at the front of an etag (read/hostEtag.ts); the input signature follows it. */
-const ROW_HASH = 22
/** A full read covers this many of the machine's Swarms, most recently changed first (was 100 full Swarms each second). */
const FULL_TABS = 500
@@ -73,7 +71,7 @@ export class HostReads {
const lastChangedAt = newest.get(tabId) ?? null, held = request.held?.[tabId]
// The etag is the rows' hash and then the signature of what they were built from. A matching signature answers
// 'unchanged' from version columns alone. Never on a full read: grants and billing live outside the parts.
- const inputs = await cached.get(tabId, 'host:inputs', 'main', () => this.inputs(tx, tabId))
+ const inputs = await cached.get(tabId, 'host:inputs', 'main', () => hostInputs(tx, cached, tabId))
if (!full && held && inputs && held.slice(ROW_HASH) === inputs) { tabs.push({ tabId, lastChangedAt, etag: held, unchanged: true as const }); continue }
const state = await this.tabState(tx, cached, tabId, machineId)
// Its Team account alone hosts a machine this Swarm records as one, whatever its account row says now (store.ts).
@@ -140,32 +138,6 @@ export class HostReads {
return parts.loadLight(tx, tabId, { groups: 'working', agentIds: agents.map(a => a.key), groupIds, projectIds, machines: false })
}
- /**
- * The signature of everything host rows are built from, read from version columns only (no part data): the head,
- * members, agents and starts by their data stamps, machines by their settings marker (not their readings), and
- * conversations by their host-fields digest and dormancy (a post leaves both). Null while a conversation has no digest
- * yet: a missing digest counts as changed.
- */
- private async inputs(tx: Transaction, tabId: string): Promise<string | null> {
- const live = { tabId, deleted: false }, stamp = { key: true, changedAt: true } as const
- const [head, members, agents, starts, machines, digests] = await Promise.all([
- tx.swarmHead.findUnique({ where: { id: `${tabId}:head` }, select: { changedAt: true } }),
- tx.swarmMember.findMany({ where: live, select: stamp }), tx.swarmAgent.findMany({ where: live, select: stamp }),
- tx.swarmStart.findMany({ where: live, select: stamp }),
- tx.swarmMachine.findMany({ where: live, select: { key: true, hostChangedAt: true } }),
- // Every conversation's digest (which carries its id) and dormancy, in key order, as one string: about 23 bytes each.
- tx.swarmGroup.aggregateRaw({ pipeline: [{ $match: { tabId, deleted: false } }, { $sort: { key: 1 } },
- { $group: { _id: null, missing: { $sum: { $cond: [{ $ifNull: ['$hostDigest', false] }, 0, 1] } },
- all: { $push: { $concat: [{ $ifNull: ['$hostDigest', ''] }, { $cond: ['$dormant', '1', '0'] }] } } } },
- { $project: { _id: 0, missing: 1, all: { $reduce: { input: '$all', initialValue: '', in: { $concat: ['$$value', '$$this'] } } } } }] }),
- ])
- const groups = (digests as unknown as Array<{ missing: number; all: string }>)[0] ?? { missing: 0, all: '' }
- if (!head || groups.missing) return null
- const keyed = <T extends { key: string }>(rows: T[]) => rows.sort((a, b) => a.key < b.key ? -1 : a.key > b.key ? 1 : 0)
- return createHash('sha256').update(JSON.stringify([head.changedAt, keyed(members), keyed(agents), keyed(starts), keyed(machines), groups.all]))
- .digest('base64url').slice(0, ROW_HASH)
- }
-
private async lookups(tx: Transaction, state: SuperTabState, teams: Array<{ machineId: string }>, grants: GrantReads): Promise<HostLookups> {
const people = state.people.length ? await tx.user.findMany({ where: { id: { in: state.people.map(p => p.id) }, autonomousEnv: state.environment }, select: { id: true, email: true } }) : []
const confidential = await confidentialSessions(tx, state.agents.map(a => a.id), state.id)
diff --git a/backend/src/superTabs/read/hostInputs.test.ts b/backend/src/superTabs/read/hostInputs.test.ts
new file mode 100644
index 000000000..eae3c543b
--- /dev/null
+++ b/backend/src/superTabs/read/hostInputs.test.ts
@@ -0,0 +1,80 @@
+import { describe, expect, it } from 'vitest'
+import { hostGroupsRule, hostGroupsSignature, type HostGroups } from './hostInputs.js'
+import type { Part } from '../parts/index.js'
+import type { SwarmDelta } from '../parts/load.js'
+
+const group = (key: string, version: number, extra: Partial<Part> = {}): Part =>
+ ({ kind: 'group', key, version, changedAt: version * 1_000, ordinal: version, data: { id: key }, ...extra }) as Part
+const delta = (parts: Part[], digests: Array<[string, string | null]> | null): SwarmDelta =>
+ ({ parts, head: null, ...digests ? { hostDigests: new Map(digests) } : {} })
+const held = (): HostGroups => ({
+ b: { digest: 'BBB', dormant: true },
+ a: { digest: 'AAA', dormant: false },
+ c: { digest: null, dormant: false },
+})
+
+describe('hostGroupsSignature', () => {
+ it('joins each conversation\'s digest and dormancy in key order, as the pipeline did, and counts the ones with no digest', () => {
+ expect(hostGroupsSignature(held())).toEqual({ missing: 1, all: 'AAA0BBB10' })
+ })
+
+ it('is empty for a Swarm with no conversation', () => {
+ expect(hostGroupsSignature({})).toEqual({ missing: 0, all: '' })
+ })
+})
+
+describe('hostGroupsRule', () => {
+ it('replaces a changed row with its new digest', () => {
+ const next = hostGroupsRule.apply(held(), delta([group('a', 4)], [['a', 'AAB']])) as HostGroups
+ expect(next.a).toEqual({ digest: 'AAB', dormant: false })
+ expect(hostGroupsSignature(next).all).toBe('AAB0BBB10')
+ })
+
+ it('keeps the rows that did not change as they were', () => {
+ const before = held()
+ const next = hostGroupsRule.apply(before, delta([group('a', 4)], [['a', 'AAB']])) as HostGroups
+ expect(next.b).toBe(before.b)
+ })
+
+ it('a conversation put to sleep, and woken, changes its flag', () => {
+ const asleep = hostGroupsRule.apply(held(), delta([group('a', 4, { dormant: true })], [['a', 'AAA']])) as HostGroups
+ expect(hostGroupsSignature(asleep).all).toBe('AAA1BBB10')
+ const woken = hostGroupsRule.apply(asleep, delta([group('a', 5)], [['a', 'AAA']])) as HostGroups
+ expect(hostGroupsSignature(woken).all).toBe('AAA0BBB10')
+ })
+
+ it('a tombstone removes the row, and the same key added again returns', () => {
+ const gone = hostGroupsRule.apply(held(), delta([group('b', 3, { deleted: true })], [['b', 'BBB']])) as HostGroups
+ expect(Object.keys(gone).sort()).toEqual(['a', 'c'])
+ const back = hostGroupsRule.apply(gone, delta([group('b', 4)], [['b', 'BBC']])) as HostGroups
+ expect(back.b).toEqual({ digest: 'BBC', dormant: false })
+ })
+
+ it('a new conversation is added', () => {
+ const next = hostGroupsRule.apply(held(), delta([group('d', 1)], [['d', 'DDD']])) as HostGroups
+ expect(hostGroupsSignature(next).all).toBe('AAA0BBB10DDD0')
+ })
+
+ it('a change the entry already holds, or a tombstone of a conversation not held, changes nothing', () => {
+ const before = held()
+ expect(hostGroupsRule.apply(before, delta([group('a', 3), group('b', 2, { dormant: true }), group('z', 9, { deleted: true })], [['a', 'AAA'], ['b', 'BBB'], ['z', 'XXX']]))).toBe(before)
+ })
+
+ it('a row that gains its digest is no longer missing', () => {
+ const next = hostGroupsRule.apply(held(), delta([group('c', 2)], [['c', 'CCC']])) as HostGroups
+ expect(hostGroupsSignature(next)).toEqual({ missing: 0, all: 'AAA0BBB1CCC0' })
+ })
+
+ it('a conversation changed with no stored digest is missing again', () => {
+ const next = hostGroupsRule.apply(held(), delta([group('a', 4)], [['a', null]])) as HostGroups
+ expect(hostGroupsSignature(next).missing).toBe(2)
+ })
+
+ it('a delta that carries no digests cannot be applied: the entry loads again', () => {
+ expect(hostGroupsRule.apply(held(), delta([group('a', 4)], null))).toBe('reload')
+ })
+
+ it('a delta with a conversation the loader gave no digest for cannot be applied either', () => {
+ expect(hostGroupsRule.apply(held(), delta([group('a', 4)], []))).toBe('reload')
+ })
+})
diff --git a/backend/src/superTabs/read/hostInputs.ts b/backend/src/superTabs/read/hostInputs.ts
new file mode 100644
index 000000000..aa04aaa01
--- /dev/null
+++ b/backend/src/superTabs/read/hostInputs.ts
@@ -0,0 +1,92 @@
+/**
+ * The signature of everything a host's rows are built from (read/host.ts), read from version columns only (no part data):
+ * the head, members, agents and starts by their data stamps, machines by their settings marker (not their readings), and
+ * conversations by their host-fields digest and dormancy (a post leaves both). Null while a conversation has no digest yet:
+ * a missing digest counts as changed.
+ *
+ * The conversations are the long part (1,715 on prod's busiest Swarm), so their digests are a read-cache entry patched from
+ * the Swarm's changes (cache.ts, cachePatch.ts) instead of a `$group` over every live conversation on every check
+ * (docs/plans/2026-11-02-host-groups-delta-plan.md).
+ */
+import { createHash } from 'node:crypto'
+import type { Prisma } from '@prisma/client'
+import type { CacheView } from '../cache.js'
+import type { PatchRule } from '../cachePatch.js'
+import { groupsBy } from '../parts/indexed.js'
+
+type Transaction = Prisma.TransactionClient
+/** The row hash's length at the front of an etag (read/hostEtag.ts); the input signature follows it. */
+export const ROW_HASH = 22
+
+/** What the signature reads of a live conversation: the digest of its host fields (null while it has none) and dormancy. */
+export interface HostGroupRow { digest: string | null; dormant: boolean }
+/** Every live conversation of a Swarm, by key. */
+export type HostGroups = Readonly<Record<string, HostGroupRow>>
+
+const byCodeUnit = (a: string, b: string) => a < b ? -1 : a > b ? 1 : 0
+
+/**
+ * The rows' digests and dormancy in key order as one string, about 23 bytes each, and how many rows have no digest. The
+ * string the `$group` pipeline built, byte for byte: a pod that still runs it and one that runs this agree on the etag.
+ */
+export function hostGroupsSignature(groups: HostGroups): { missing: number; all: string } {
+ let missing = 0, all = ''
+ for (const key of Object.keys(groups).sort(byCodeUnit)) {
+ const { digest, dormant } = groups[key]!
+ if (digest === null) missing++
+ all += (digest ?? '') + (dormant ? '1' : '0')
+ }
+ return { missing, all }
+}
+
+/**
+ * The `host:groups` entry: a changed conversation comes as its row is now, so applying one twice does nothing; a tombstone
+ * takes its row out. A delta that carries no digest for a changed conversation (a loader that does not read them) cannot be
+ * applied: the entry loads again.
+ */
+export const hostGroupsRule: PatchRule<HostGroups> = {
+ kinds: ['group'],
+ apply(held, delta) {
+ let next: Record<string, HostGroupRow> | null = null
+ for (const part of delta.parts) {
+ if (part.kind !== 'group') continue
+ const current = next ?? held
+ if (part.deleted) {
+ if (!Object.hasOwn(current, part.key)) continue
+ next ??= { ...held }
+ delete next[part.key]
+ continue
+ }
+ if (!delta.hostDigests?.has(part.key)) return 'reload'
+ const row: HostGroupRow = { digest: delta.hostDigests.get(part.key) ?? null, dormant: !!part.dormant }
+ const was = Object.hasOwn(current, part.key) ? current[part.key] : undefined
+ if (was && was.digest === row.digest && was.dormant === row.dormant) continue
+ next ??= { ...held }
+ next[part.key] = row
+ }
+ return next ?? held
+ },
+}
+
+/** Every live conversation's row, through the live index alone: it holds the key, digest and dormancy, so no document is read. */
+export async function loadHostGroups(tx: Transaction, tabId: string): Promise<HostGroups> {
+ const rows = await groupsBy(tx, 'live', [{ $match: { tabId, deleted: false } },
+ { $project: { _id: 0, key: 1, hostDigest: 1, dormant: 1 } }]) as Array<{ key: string; hostDigest?: string | null; dormant?: boolean }>
+ return Object.fromEntries(rows.map(row => [row.key, { digest: typeof row.hostDigest === 'string' ? row.hostDigest : null, dormant: !!row.dormant }]))
+}
+
+export async function hostInputs(tx: Transaction, view: CacheView, tabId: string): Promise<string | null> {
+ const live = { tabId, deleted: false }, stamp = { key: true, changedAt: true } as const
+ const [head, members, agents, starts, machines, held] = await Promise.all([
+ tx.swarmHead.findUnique({ where: { id: `${tabId}:head` }, select: { changedAt: true } }),
+ tx.swarmMember.findMany({ where: live, select: stamp }), tx.swarmAgent.findMany({ where: live, select: stamp }),
+ tx.swarmStart.findMany({ where: live, select: stamp }),
+ tx.swarmMachine.findMany({ where: live, select: { key: true, hostChangedAt: true } }),
+ view.get(tabId, 'host:groups', 'main', () => loadHostGroups(tx, tabId), hostGroupsRule),
+ ])
+ const groups = hostGroupsSignature(held)
+ if (!head || groups.missing) return null
+ const keyed = <T extends { key: string }>(rows: T[]) => rows.sort((a, b) => byCodeUnit(a.key, b.key))
+ return createHash('sha256').update(JSON.stringify([head.changedAt, keyed(members), keyed(agents), keyed(starts), keyed(machines), groups.all]))
+ .digest('base64url').slice(0, ROW_HASH)
+}
diff --git a/backend/src/superTabs/read/inbox.ts b/backend/src/superTabs/read/inbox.ts
index 7e84dad6b..832438680 100644
--- a/backend/src/superTabs/read/inbox.ts
+++ b/backend/src/superTabs/read/inbox.ts
@@ -29,6 +29,7 @@ import { threadSummaries } from '../threads.js'
import { sessionReadsRow } from '../buddySessions.js'
import { readMarks, readThreadMarkRows, type ThreadMarkRow } from '../readMarks.js'
import { readBy } from '../parts/indexed.js'
+import { logger } from '../../utils/logger.js'
import { ms } from '../parts/light.js'
import type { SuperTabStore } from '../store.js'
@@ -151,6 +152,44 @@ export async function perRange(db: PrismaClient, pipelines: Prisma.InputJsonValu
return rows
}
+/** One thread's rows in a sequence range. */
+export interface ThreadRange { conversationId: string; thread: string; sequence: Record<string, number> }
+
+/**
+ * Each range's newest `limit` rows, newest first, range by range, as `perRange` of each thread's own pipeline answers, or only those
+ * of them `namedBy` wrote or is tagged in. Ken's production profile (task #2148): a union plans every branch, racing the sequence
+ * index against the thread index, and a hint reaches only its first pipeline. So each batch is one read, hinted to the thread index:
+ * nothing to race and one plan, about 200 ms of planning down to 3 for 100 ranges. It reads every row of a range, not only its newest,
+ * so past a cap of a quarter of what the ranges' limits allow, a batch of long threads takes the union instead, which bounds each one.
+ */
+export async function newestInThreads(db: PrismaClient, tabId: string, ranges: ThreadRange[], limit: number, namedBy?: string): Promise<Row[]> {
+ const rows: Row[] = []
+ const named = namedBy === undefined ? [] : [{ $match: { $or: [{ 'author.kind': 'person', 'author.id': namedBy }, { mentions: namedBy }] } }]
+ for (let i = 0; i < ranges.length; i += CHATS_PER_READ) {
+ const batch = ranges.slice(i, i + CHATS_PER_READ)
+ const cap = Math.max(limit, Math.ceil(limit * batch.length / 4))
+ const newest = namedBy === undefined ? '$newest' : { $filter: { input: '$newest', as: 'm', cond: { $or: [
+ { $and: [{ $eq: ['$$m.author.kind', 'person'] }, { $eq: ['$$m.author.id', { $literal: namedBy }] }] },
+ { $in: [{ $literal: namedBy }, { $ifNull: ['$$m.mentions', []] }] }] } } }
+ const found = await readBy(db, 'SuperTabMessage', 'thread', [
+ { $match: { $or: batch.map(r => ({ tabId, conversationId: r.conversationId, thread: r.thread, sequence: r.sequence })) } },
+ { $project: { ...SCAN_FIELDS, conversationId: 1 } }, { $limit: cap + 1 },
+ { $group: { _id: { conversationId: '$conversationId', thread: '$thread' }, seen: { $sum: 1 },
+ newest: { $topN: { n: limit, sortBy: { sequence: -1 }, output: '$$ROOT' } } } },
+ { $project: { seen: 1, newest } }] as Prisma.InputJsonValue[]) as unknown as
+ Array<{ _id: { conversationId: string; thread: string }; seen: number; newest: Array<Record<string, unknown>> }>
+ if (found.reduce((seen, thread) => seen + Number(thread.seen), 0) > cap) {
+ logger.info('[inbox] a batch of thread ranges passed its cap and read by union', { ranges: batch.length, limit })
+ rows.push(...await perRange(db, batch.map(r => [{ $match: { tabId, conversationId: r.conversationId, thread: r.thread, sequence: r.sequence } },
+ { $sort: { sequence: -1 } }, { $limit: limit }, ...named, { $project: { ...SCAN_FIELDS, conversationId: 1 } }] as Prisma.InputJsonValue[])))
+ continue
+ }
+ const byThread = new Map(found.map(thread => [threadKey(thread._id.conversationId, thread._id.thread), thread.newest]))
+ for (const r of batch) rows.push(...(byThread.get(threadKey(r.conversationId, r.thread)) ?? []).map(row))
+ }
+ return rows
+}
+
/** Up to `limit` rows of one chat, newest first, in its (sequence, id) index range. */
const newestIn = (tabId: string, conversationId: string, sequence: Record<string, number>, limit: number, fields: Record<string, unknown>): Prisma.InputJsonValue[] =>
[{ $match: { tabId, conversationId, sequence } }, { $sort: { sequence: -1, _id: -1 } }, { $limit: limit }, { $project: { ...fields, conversationId: 1 } }] as Prisma.InputJsonValue[]
@@ -205,8 +244,7 @@ async function threadsIn(db: PrismaClient, tabId: string, state: SuperTabState,
const inIt = (m: Row) => own(m.author) || m.mentions.includes(personId)
const lagging = behind.slice(0, BEHIND_THREADS)
const [scanned, past] = await Promise.all([perRange(db, chats.map(g => newestIn(tabId, g.id, { $gt: marks.get(g.id)! }, UNREAD_SCAN + 1, SCAN_FIELDS))),
- perRange(db, lagging.map(r => [{ $match: { tabId, conversationId: r.conversationId, thread: r.threadId, sequence: { $gt: r.sequence } } },
- { $sort: { sequence: -1 } }, { $limit: UNREAD_SCAN + 1 }, { $project: { ...SCAN_FIELDS, conversationId: 1 } }]))])
+ newestInThreads(db, tabId, lagging.map(r => ({ conversationId: r.conversationId, thread: r.threadId, sequence: { $gt: r.sequence } })), UNREAD_SCAN + 1)])
const byChat = groupBy(scanned, m => m.conversationId)
const pastByThread = groupBy(past, m => threadKey(m.conversationId, m.thread!))
// A chat or thread whose newest UNREAD_SCAN rows do not reach its mark: older unread replies are not counted.
@@ -245,9 +283,7 @@ async function threadsIn(db: PrismaClient, tabId: string, state: SuperTabState,
// index range; only the rows that name them come back.
const unknown = withRoot.filter(c => !c.member)
if (unknown.length) {
- const named = await perRange(db, unknown.map(c => [{ $match: { tabId, conversationId: c.conversationId, thread: c.root, sequence: { $lte: c.mark } } },
- { $sort: { sequence: -1 } }, { $limit: MEMBER_SCAN },
- { $match: { $or: [{ 'author.kind': 'person', 'author.id': personId }, { mentions: personId }] } }, { $project: { ...SCAN_FIELDS, conversationId: 1 } }]))
+ const named = await newestInThreads(db, tabId, unknown.map(c => ({ conversationId: c.conversationId, thread: c.root, sequence: { $lte: c.mark } })), MEMBER_SCAN, personId)
const members = new Set((await readable(db, tabId, named, state)).map(m => threadKey(m.conversationId, m.thread!)))
for (const c of unknown) if (members.has(threadKey(c.conversationId, c.root))) c.member = true
}
diff --git a/backend/src/superTabs/read/threadMarkSeeds.ts b/backend/src/superTabs/read/threadMarkSeeds.ts
index a1e9f6eb8..8a1121719 100644
--- a/backend/src/superTabs/read/threadMarkSeeds.ts
+++ b/backend/src/superTabs/read/threadMarkSeeds.ts
@@ -21,7 +21,7 @@ import { logger } from '../../utils/logger.js'
import { stableId, type Author, type SuperTabState } from '../model.js'
import { seedThreadMark } from '../readMarks.js'
import { readBy } from '../parts/indexed.js'
-import { BEHIND_THREADS, groupBy, isReply, MEMBER_SCAN, perRange, readable, row, SCAN_FIELDS, UNREAD_SCAN, type Row } from './inbox.js'
+import { BEHIND_THREADS, groupBy, isReply, MEMBER_SCAN, newestInThreads, readable, row, SCAN_FIELDS, UNREAD_SCAN, type Row } from './inbox.js'
/** Rows of the range one chat read looks at, from the old mark upward. */
export const SEED_SCAN = 5 * UNREAD_SCAN
@@ -71,9 +71,7 @@ export async function seedThreadMarks(db: PrismaClient, input: SeedInput): Promi
// thread index range; only the rows that name them come back.
const unknown = withRoot.filter(thread => !member.has(thread))
if (unknown.length) {
- const named = await perRange(db, unknown.map(thread => [{ $match: { tabId, conversationId, thread, sequence: { $lte: old } } },
- { $sort: { sequence: -1 } }, { $limit: MEMBER_SCAN },
- { $match: { $or: [{ 'author.kind': 'person', 'author.id': personId }, { mentions: personId }] } }, { $project: { ...SCAN_FIELDS, conversationId: 1 } }]))
+ const named = await newestInThreads(db, tabId, unknown.map(thread => ({ conversationId, thread, sequence: { $lte: old } })), MEMBER_SCAN, personId)
for (const m of await readable(db, tabId, named, state)) member.add(m.thread!)
}
let marked = 0
diff --git a/backend/src/superTabs/store.ts b/backend/src/superTabs/store.ts
index 1a5fde4ca..b482d95d1 100644
--- a/backend/src/superTabs/store.ts
+++ b/backend/src/superTabs/store.ts
@@ -831,7 +831,11 @@ export class SuperTabStore {
const delivery = await tx.superTabDelivery.findFirst({ where: { id: command.deliveryId, tabId }, select: { conversationId: true } })
if (delivery) groupIds.push(delivery.conversationId)
}
- if (command.type === 'agent.remove' || command.type === 'agent.input' || command.type === 'agent.alert' || command.type === 'agent.rebind') agentIds.push(command.agentId)
+ if (command.type === 'agent.remove') agentIds.push(command.agentId)
+ // These only ask about the agent. A coordinator's project chat, always loaded, answers them, and every group it was ever in is
+ // what a read by agent gave: 1,668 on prod, growing with every task (Ken's profile, 2026-11-02, up to 1.66 s a read). Filtering
+ // that read by dormant would not do: its index has no dormant, so it examines them all, and removals need them (design D8).
+ if ((command.type === 'agent.input' || command.type === 'agent.alert' || command.type === 'agent.rebind') && !ledProject(state, command.agentId)) agentIds.push(command.agentId)
if (command.type === 'agent.remove') startAgentIds.push(command.agentId)
if (command.type === 'machine.remove') {
const pool = state.machines?.find(m => m.machineId === command.machineId)
diff --git a/backend/vitest.super-tabs.config.ts b/backend/vitest.super-tabs.config.ts
index 088962c28..fc5cb178f 100644
--- a/backend/vitest.super-tabs.config.ts
+++ b/backend/vitest.super-tabs.config.ts
@@ -3,10 +3,10 @@ import { defineConfig } from 'vitest/config'
export default defineConfig({ test: { include: ['integration/superTabs.test.ts', 'integration/superTabSharing.test.ts', 'integration/superTabGithub.test.ts', 'integration/superTabTeam.test.ts', 'integration/superTabWorkspaces.test.ts',
'integration/superTabRead.test.ts', 'integration/superTabReadMarks.test.ts', 'integration/superTabInbox.test.ts', 'integration/superTabFollowedThreads.test.ts', 'integration/superTabActivity.test.ts', 'integration/superTabLeaderboard.test.ts', 'integration/superTabMigration.test.ts', 'integration/superTabLiveness.test.ts',
'integration/superTabTidy.test.ts', 'integration/superTabProfile.test.ts', 'integration/superTabLegacyDrop.test.ts', 'integration/shareRaces.test.ts', 'integration/superTabIndexes.test.ts', 'integration/superTabRouteDecisions.test.ts',
- 'integration/superTabHotReads.test.ts', 'integration/swarmSweepStarts.test.ts', 'integration/superTabGroupPlans.test.ts', 'integration/superTabInvitationPlans.test.ts', 'integration/superTabCache.test.ts', 'integration/superTabCacheDelta.test.ts', 'integration/superTabMachineReports.test.ts',
+ 'integration/superTabHotReads.test.ts', 'integration/swarmSweepStarts.test.ts', 'integration/superTabGroupPlans.test.ts', 'integration/superTabInvitationPlans.test.ts', 'integration/superTabCache.test.ts', 'integration/superTabCacheDelta.test.ts', 'integration/superTabHostGroups.test.ts', 'integration/superTabMachineReports.test.ts',
'integration/superTabObserver.test.ts', 'integration/productUsageConflicts.test.ts', 'integration/productUsageReceipts.test.ts', 'integration/schemaRelease.test.ts', 'integration/superTabBuddies.test.ts', 'integration/superTabNotAvailable.test.ts',
'integration/superTabBuddyProbes.test.ts', 'integration/superTabBuddySessions.test.ts', 'integration/superTabSwarmBuddy.test.ts', 'integration/superTabCosts.test.ts', 'integration/harnessSharesList.test.ts', 'integration/atlasSnapshot.test.ts', 'integration/queryStats.test.ts',
'integration/superTabEvidence.test.ts', 'integration/superTabAttemptModels.test.ts', 'integration/superTabNamedReads.test.ts', 'integration/swarmSweepPlans.test.ts', 'integration/workerIndexPlans.test.ts', 'integration/superTabReflection.test.ts',
- 'integration/superTabMessagePages.test.ts', 'integration/superTabDeliveryCompaction.test.ts', 'integration/superTabNoticeDigest.test.ts', 'integration/superTabThreads.test.ts',
+ 'integration/superTabMessagePages.test.ts', 'integration/superTabDeliveryCompaction.test.ts', 'integration/superTabNoticeDigest.test.ts', 'integration/superTabCommandLoads.test.ts', 'integration/superTabThreads.test.ts',
'integration/superTabInstructions.test.ts', 'integration/superTabWorkflow.test.ts', 'integration/superTabDeletion.test.ts'], hookTimeout: 120000, testTimeout: 30000, setupFiles: ['integration/cacheOn.test-support.ts', 'src/lib/noRedis.test-support.ts'],
fileParallelism: false, env: { NODE_ENV: 'test', HARNESS_CREDENTIAL_ENCRYPTION_KEY: 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=' } } })
diff --git a/cli/src/superTabs/inCore/teamPurge.spec.ts b/cli/src/superTabs/inCore/teamPurge.spec.ts
index d421334bc..a6d940bc3 100644
--- a/cli/src/superTabs/inCore/teamPurge.spec.ts
+++ b/cli/src/superTabs/inCore/teamPurge.spec.ts
@@ -94,6 +94,20 @@ describe('a Buddy\'s private temporary folder when its session ends', () => {
rmSync(buddyTmpParent(), { recursive: true, force: true })
})
+ // Windows has no user id: Claude Code's temp folder there is under uid 0's name (agentTmp's own fallback).
+ it('finds a session\'s files where the platform has no user id', async () => {
+ const home = dir(), folder = join(dir(), 'buddy-no-uid'), SESSION = '11111111-1111-4111-8111-111111111111'
+ mkdirSync(folder)
+ const files = join(buddyTmpFolder(folder), 'claude-0', realpathSync(folder).replace(/[^A-Za-z0-9]/g, '-'), SESSION)
+ mkdirSync(files, { recursive: true })
+ const getuid = process.getuid
+ Object.defineProperty(process, 'getuid', { value: undefined, configurable: true, writable: true })
+ try { await scrubClaudeSession(home, target({ sessionId: SESSION, cwd: folder })) }
+ finally { Object.defineProperty(process, 'getuid', { value: getuid, configurable: true, writable: true }) }
+ expect(existsSync(files)).toBe(false)
+ rmSync(buddyTmpParent(), { recursive: true, force: true })
+ })
+
it('goes whole once its Buddy folder is gone, and another Buddy\'s stays', async () => {
const home = dir(), folder = join(dir(), 'buddy-one'), other = join(dir(), 'buddy-two')
mkdirSync(folder); mkdirSync(other)
@@ -328,6 +342,28 @@ describe('scrubbing a Claude Code session from the Team user\'s shared home', ()
})
})
+describe('a Claude Code config with part of it missing', () => {
+ it('takes the folder from its repository paths when it has no projects at all', async () => {
+ const home = dir(), folder = join(home, 'project-folder')
+ mkdirSync(join(home, '.claude'), { recursive: true })
+ writeFileSync(join(home, '.claude.json'), JSON.stringify({ githubRepoPaths: { 'acme/app': [folder, '/kept'] } }))
+ await scrubClaudeSession(home, target({ sessionId: null, cwd: folder }))
+ expect(JSON.parse(readFileSync(join(home, '.claude.json'), 'utf8'))).toEqual({ githubRepoPaths: { 'acme/app': ['/kept'] } })
+ })
+})
+
+describe('Windows, with the projects folder unreadable', () => {
+ // A listing that fails names only what the slug says, and the removal reports it unread, never as gone.
+ it('tries the one name the folder gives, and says what it could not read', async () => {
+ const home = dir(), told: string[] = []
+ mkdirSync(join(home, '.claude'), { recursive: true })
+ writeFileSync(join(home, '.claude', 'projects'), 'not a folder')
+ await scrubClaudeSession(home, target({ sessionId: null, cwd: 'C:\\Users\\team\\work' }), [], undefined, what => told.push(what), win32)
+ expect(existsSync(join(home, '.claude', 'projects'))).toBe(true)
+ expect(told).toEqual(['unread'])
+ })
+})
+
describe('the sweep for what a scrub left', () => {
// Re-check 2 of 68ec6f0: what the sweep found was named by its path, the Team home's and all. It names its kind.
it('finds the session or folder where Claude Code and Codex write its own data, and nowhere else, by its kind', async () => {
diff --git a/docs/plans/2026-11-02-host-groups-delta-plan.md b/docs/plans/2026-11-02-host-groups-delta-plan.md
new file mode 100644
index 000000000..83d953aaf
--- /dev/null
+++ b/docs/plans/2026-11-02-host-groups-delta-plan.md
@@ -0,0 +1,83 @@
+# Host checks stop reading every live group Implementation Plan
+
+> **For agentic workers:** Written by the writing-plans skill as the project's instructions adapt it (Engineer role, revision 23). The Engineer implements it task by task, test first, in this session; the pull request's Review step is the one independent review. No execution handoff.
+
+**Goal:** `HostReads` builds a Swarm's host signature from a cached, delta-patched list of its conversations' digests, so a check after a change reads only the conversations changed since the last one, and returns the same signature as today.
+
+**Architecture:** The conversations' part of the signature (`inputs()`, read/host.ts:157, a `$group` over every live group: 76 slow ops, 24 s, max 734 ms, planning 189 ms in Ken's 10:50-12:38 UTC profile) becomes a read-cache entry `host:groups` with a patch rule (cache.ts, cachePatch.ts), as the `parts` and `dormant` entries already are. A first load per Swarm and process (then every `DELTA_MAX_AGE_MS`, 10 min) projects `key, hostDigest, dormant` of every live group through the named `live` index (covered, no `$group`, no planning); after that each check applies the `loadDelta` rows. The signature string is built from it exactly as the pipeline built it.
+
+**Tech Stack:** TypeScript, Prisma on MongoDB (`aggregateRaw`), vitest; integration on `mongodb-memory-server`.
+
+**Spec:** [docs/superpowers/specs/2026-11-02-swarm-cache-delta-design.md](../superpowers/specs/2026-11-02-swarm-cache-delta-design.md) (the patch rules and the delta read this reuses); the task's brief (Ken's prod DB profile, item 3).
+
+## Global Constraints
+
+- Same answers as today: `hostInputs` returns the same string, byte for byte, as `inputs()` did for the same stored state, so etags a host holds stay valid across the deploy and old and new pods agree while they roll.
+- A conversation with no stored `hostDigest` still makes the signature `null` (a missing digest counts as changed).
+- No new index, no schema change, no production write. `GROUP_INDEX.live` already exists and is the covering index.
+- Swarm code stays out of `cli/src/core`. Optimizations ship on, with no switch.
+- Full Super Tab integration suite before Ready (`npm run test:super-tabs` in `backend/`).
+
+## Review Focus
+
+- A conversation put to sleep or woken (its `dormant` flag is in the signature) changes it exactly then: Task 1, Task 2.
+- A conversation that gains its digest after a pass found it missing leaves `null`: Task 1.
+- A tombstoned conversation leaves the signature; one added again under the same key returns: Task 1.
+- A rename or a post (not in `hostDigest`) leaves the signature unchanged: Task 2.
+- A write stamped 20 s before it committed is in the next patch (the cache's overlap): Task 2.
+- A delta without digests (a loader that does not carry them) reloads rather than guesses: Task 1.
+
+## Design
+
+- **Growth: what deletes it.** The entry holds one small row (key, 22-char digest, flag) per live conversation of a Swarm a host polls, about 100 bytes: 1,715 groups on prod's busiest Swarm is about 170 KB. It lives in the read cache, which bounds it (`CACHE_MAX_BYTES`, least recently used out; a tombstone removes its row; a Swarm's bump never leaves it past `maxAgeMs`/`DELTA_MAX_AGE_MS`). Nothing new is stored in Mongo.
+- **Concurrency: lock level.** None added. Group writes are unchanged (no write to the head or any shared document); a stored digest on the head was rejected because every conversation write would then contend on it. The patch reads under the cache's per-Swarm, per-stamp single flight.
+- **Read cost: only what changed.** Cold: one covered read of the Swarm's live conversations (key, digest, flag; no document fetch), once per process and Swarm, then at most every 10 minutes. Warm: `changes` since the entry's last read, less `OVERLAP_MS`, shared with the `parts` entry's delta when both are due (one read per Swarm and stamp). More than `DELTA_MAX_PARTS` (500) changed reloads in full.
+- **Scale: 10x and the first warning.** Cold load at 10x (17,000 conversations) is a covered read of ~17,000 keys, about 1.7 MB: the warning is `swarm cache` reload reason `age` and `parts` for `host:groups` (cacheStats) and the profile's `SwarmGroup` `aggregate` count by shape. The signature string is still built per miss in memory (a sort and a concatenation, microseconds at today's size).
+- **Compatibility.** The string and its hash input are unchanged, so old and new pods answer the same etag. A pod that predates `loadDelta`'s digests is not mixed in: the rule reloads a delta without them.
+- **Migration + rollback.** None: no stored data changes. Rolling back the release restores the `$group` pipeline; nothing to clean up.
+- **Audience.** N/A: the signature is built for the machine's owner's host check, as before; no new data reaches any reader.
+- **Identity.** N/A: nothing here uses a login.
+- **Failure.** A delta that fails or does not apply reloads the entry in full (cache.ts `patched`), with its reason counted; a 1-in-N background check against a full load (`verifyRate`) logs `swarm cache verify mismatch` and drops a drifted entry; the 10-minute full load is the net. Nothing is silently wrong for longer than that.
+- **Platforms.** Backend only.
+
+## File Structure
+
+- Create `backend/src/superTabs/read/hostInputs.ts`: `hostInputs()` (moved from `HostReads.inputs`), the `host:groups` loader, `hostGroupsRule`, `hostGroupsSignature`. One responsibility: the host signature.
+- Modify `backend/src/superTabs/read/host.ts`: `inputs` calls `hostInputs`; the pipeline goes.
+- Modify `backend/src/superTabs/parts/load.ts`: `SwarmDelta.hostDigests`, `CHANGED_FIELDS.hostDigest`.
+- Create `backend/src/superTabs/read/hostInputs.test.ts` (unit) and `backend/integration/superTabHostGroups.test.ts`; add the latter to `vitest.super-tabs.config.ts`.
+
+### Task 1: The rule and the signature
+
+**Files:**
+- Create: `backend/src/superTabs/read/hostInputs.ts`, `backend/src/superTabs/read/hostInputs.test.ts`
+- Modify: `backend/src/superTabs/parts/load.ts` (the delta carries digests)
+
+**Interfaces:**
+- Produces: `interface HostGroupRow { digest: string | null; dormant: boolean }`; `type HostGroups = Readonly<Record<string, HostGroupRow>>`; `hostGroupsRule: PatchRule<HostGroups>` (kinds `['group']`; a changed row comes as it is now, so a delta applied twice does nothing, and the cold load carries no version because the `live` index does not, which keeps the read covered); `hostGroupsSignature(groups: HostGroups): { missing: number; all: string }` (rows in key order, `digest + (dormant ? '1' : '0')`, `''` for a missing digest); `SwarmDelta.hostDigests?: ReadonlyMap<string, string | null>` (group key to stored digest).
+
+- [ ] **Step 1: Failing tests** in `hostInputs.test.ts`: (a) `hostGroupsSignature` of three rows given out of key order equals the old pipeline's string for the same rows (`'AAA'+'0'+'BBB'+'1'+...`) and counts `missing` for a null digest; (b) the rule replaces a changed row with its new digest, flips `dormant`, removes a tombstone, adds a new key, and returns the held object itself when the delta holds nothing new; (c) a delta with no `hostDigests` returns `'reload'`; (d) a row that gains its digest leaves `missing` at 0.
+- [ ] **Step 2: Run** `cd backend && npx vitest run src/superTabs/read/hostInputs.test.ts`; expect FAIL (module missing). Commit the failing test.
+- [ ] **Step 3: Implement** the types, rule and signature in `hostInputs.ts`, and in `parts/load.ts` add `hostDigest: 1` to `CHANGED_FIELDS` and collect `hostDigests` for group rows in `loadDelta` (null for a missing column).
+- [ ] **Step 4: Run** the unit test; expect PASS. Commit.
+
+### Task 2: `hostInputs()` through the cache, same answers, bounded read
+
+**Files:**
+- Create: `backend/integration/superTabHostGroups.test.ts`
+- Modify: `backend/src/superTabs/read/host.ts`, `backend/src/superTabs/read/hostInputs.ts`, `backend/vitest.super-tabs.config.ts`
+
+**Interfaces:**
+- Consumes: Task 1.
+- Produces: `hostInputs(tx: Transaction, view: CacheView, tabId: string): Promise<string | null>`, the string `HostReads.inputs` gave.
+
+- [ ] **Step 1: Failing integration test.** On a migrated `richSwarm` with a cache and `readDelta`: (a) *same answers*: after each of ~40 seeded random writes (rename, sleep, wake, agent leaves, assignment status, new group, tombstone, post), `hostInputs` through the cache equals `hostInputs` with `DIRECT`, and equals a reference built in the test from the old pipeline; (b) *exactly when relevant*: a rename and a message post leave it unchanged, a sleep, wake, status change, new group and tombstone each change it; (c) *bounded read*: under `profiledReads`, a check after one conversation changed reads `SwarmGroup` once with `docsExamined` and `returned` at most the changed count, against N on the cold read; no `$group` stage; (d) a group whose `hostDigest` is unset makes it `null`, and a write that sets it brings the string back; (e) the write stamped 20 s earlier is in the next patch.
+- [ ] **Step 2: Run** `cd backend && npx vitest run --config vitest.super-tabs.config.ts integration/superTabHostGroups.test.ts`; expect FAIL (`hostInputs` missing). Commit.
+- [ ] **Step 3: Implement** `hostInputs` and the `host:groups` loader (`groupsBy(tx, 'live', [{$match:{tabId, deleted:false}},{$project:{_id:0,key:1,hostDigest:1,dormant:1}}])`, read through `view.get(tabId, 'host:groups', 'main', load, hostGroupsRule)`); `HostReads.inputs` passes `cached` and calls it; delete the pipeline.
+- [ ] **Step 4: Run** the test; expect PASS. Commit.
+- [ ] **Step 5: Run the touched suites** (`superTabGroupPlans`, `superTabRead`, `superTabHotReads`, `superTabCache`, `superTabCacheDelta`) then the full `npm run test:super-tabs`; record red/green in a PR comment.
+
+### Task 3: Measure on production and open the pull request
+
+- [ ] **Step 1:** Count the `SwarmGroup` `$group` ops in the same window length before (v1.3.86) and after (the release that carries this) through the read-only production login only, from its environment file; record numbers in the PR.
+- [ ] **Step 2:** Open the pull request (body starts "Requested by [email protected] through Harness."), add the review step for Echo.
diff --git a/docs/plans/2026-11-02-inbox-thread-ranges-plan.md b/docs/plans/2026-11-02-inbox-thread-ranges-plan.md
new file mode 100644
index 000000000..e7954fc41
--- /dev/null
+++ b/docs/plans/2026-11-02-inbox-thread-ranges-plan.md
@@ -0,0 +1,73 @@
+# Inbox thread ranges: one hinted read instead of a planned union Implementation Plan
+
+> **Way of working:** by this project's instructions: each failing test committed before its code, then one independent review of the pull request.
+
+**Goal:** The inbox's and the mark-read seed's reads of up to 100 thread ranges stop racing two query plans per range, with the same answers.
+
+**Architecture:** A new `newestInThreads` reads one batch of thread ranges in one aggregate: a `$or` of the ranges, hinted to the thread index, each thread's newest rows kept by `$topN`. A batch that would read more rows than a cap falls back to today's union, which bounds each thread by its own limit.
+
+**Tech Stack:** TypeScript, Prisma `aggregateRaw` on MongoDB 8, Vitest integration tests with a real replica set.
+
+**Spec:** Task #2148, from Ken's production profile of 2026-11-02, 10:50-12:38 UTC.
+
+## What production showed (read-only login, 100 busiest threads of one Swarm)
+
+| | Union of 100 (today) | One hinted `$or` |
+|---|---|---|
+| Plans rejected | 100, one per range | 0 |
+| Planning time in explain | 200 ms (Ken's slowest: 201 ms) | 3 ms |
+| Execution time in explain | 112 ms | 30 ms |
+| Median time on the primary, less the round trip | about 265 ms | about 55 ms |
+| Rows read, rows answered | 1706, 175 | 1706, 175, the same rows |
+
+A hint on the aggregate reaches only its first pipeline, never a `$unionWith` branch, so the union can't be hinted. Naming `thread` in the `$sort` changes nothing: the planner drops a field the filter fixes. The largest thread in production has 69 rows, so the limit of 100 per thread does not bind today.
+
+## Global Constraints
+
+- PR body starts with: Requested by [email protected] through Harness.
+- No production write. No index is created: the thread index exists.
+- The read-only production login comes only from its environment file, and no ids or content are printed.
+- Full Super Tab integration suite before Ready.
+
+## Design
+
+- **Growth:** nothing is stored.
+- **Concurrency:** N/A because these are reads, outside any transaction, as before.
+- **Read cost:** one aggregate per 100 ranges, as today. It reads each thread's rows in its range, up to a cap for the batch of a quarter of what its ranges' limits allow (at least one limit), and then keeps each thread's newest rows. Only a batch over the cap pays again: at most 1.25 times what the union alone can read for 4 or more ranges, and up to twice for fewer.
+- **Scale:** at 10x thread length, more batches pass the cap and take the union, which costs what it costs today. The fallback log line (info level, one per batch) is the first sign, as a rate in the logs; nothing alerts on it.
+- **Compatibility:** N/A because no client or stored shape changes; the answers are the same rows in the same order.
+- **Migration and rollback:** none; reverting restores the union.
+- **Audience:** unchanged. Every row still goes through `readable`, and the caller has checked the person reads the chats.
+- **Identity:** the person's id is passed as a literal to the look-back's filter, as a value and never as a field path.
+- **Failure:** a missing index reads without the hint, as `readBy` does and says. An index removed since the last listing fails once and reads again.
+- **Platforms:** backend only.
+
+## Review Focus
+
+- The answers: a thread longer than its limit, a tag older than the limit, ranges bounded above and below, a batch over the cap.
+- Row order: the callers take `rows.slice(0, UNREAD_SCAN)` newest first.
+- The filter in `$filter` against the old `$match`: author and mentions.
+
+---
+
+### Task 1: Pin the answers and the read's shape
+
+**Files:**
+- Test: `backend/integration/superTabInbox.test.ts`
+
+**Interfaces:**
+- Consumes: `newestInThreads(db, tabId, ranges, limit, namedBy?)`, `perRange(db, pipelines)`.
+
+- [ ] **Step 1: Write the failing tests:** the same rows as one pipeline per thread, in every case above; one operation, hinted, with no `$unionWith`.
+- [ ] **Step 2: Run to verify they fail.** Run: `npx vitest run --config vitest.super-tabs.config.ts integration/superTabInbox.test.ts`. Expected: FAIL, `newestInThreads` does not exist.
+- [ ] **Step 3: Commit.**
+
+### Task 2: One hinted read per batch
+
+**Files:**
+- Modify: `backend/src/superTabs/read/inbox.ts`, `backend/src/superTabs/read/threadMarkSeeds.ts`
+
+- [ ] **Step 1: Implement `newestInThreads`,** and use it for the inbox's thread reads and the seed's look-back. `followedThreads.ts` is outside this task and keeps the union.
+- [ ] **Step 2: Run to verify they pass.** Same command. Expected: PASS.
+- [ ] **Step 3: Run the full suite.** Run: `npm run test:super-tabs`.
+- [ ] **Step 4: Commit.**
diff --git a/docs/plans/2026-11-02-load-no-dormant-of-coordinator-plan.md b/docs/plans/2026-11-02-load-no-dormant-of-coordinator-plan.md
new file mode 100644
index 000000000..354daf4e5
--- /dev/null
+++ b/docs/plans/2026-11-02-load-no-dormant-of-coordinator-plan.md
@@ -0,0 +1,72 @@
+# A command naming the PM never reads its dormant groups: Implementation Plan
+
+> Way of working: this project's instructions. The Engineer implements it test first, failing test committed before its code; Reviewer Rowan.
+
+**Goal:** `agent.input`, `agent.alert` and `agent.rebind` naming a project's coordinator read none of its dormant groups.
+
+**Architecture:** `store.ts moreNamed` names an agent, and `parts/load.ts loadActive` then reads every group it is in through `{tabId, agentIds}`. A coordinator is left out of that list for these three commands: its project chat, which is always loaded, already answers what they ask.
+
+**Tech Stack:** TypeScript, Prisma on MongoDB, vitest integration tests on a loopback replica set.
+
+**Spec:** Task #2147 (Ken's prod DB profile, 10:50-12:38 UTC, backend v1.3.86).
+
+## Why not `dormant: false` on the read
+
+Two reasons, both checked:
+- **It keeps the cost.** `{tabId, agentIds}` has no `dormant`, so the filter runs after the fetch. On prod, read-only `explain` for the agent with the most dormant groups: 84 groups returned, 1,752 documents still examined.
+- **It breaks callers.** `agent.remove`, `machine.remove` and `member.remove` take the agent out of every group it is in, dormant too (`docs/design/2026-10-26-swarm-state-parts.md`, D8). A worker's `agent.alert` may find its only group asleep: `currentGroup` and, for an agent with no stored project, `agentProjectId` read it.
+
+## What a coordinator's three commands read, checked
+
+- `currentGroup`: its active assignment, else its last open group it does not lead, else its project chat. A dormant group is no longer picked; the row goes to an active group or the project chat, where the coordinator reads.
+- `agentProjectId`: its stored project, else its first loaded group, and the project chat is always loaded.
+- `isConfidential`, `mayDrive`: a Lead is confidential by the project chat, and marked. `agent.input`'s other Swarms check is its own read.
+- `agent.rebind`, `loseTasks`: active work only.
+
+## Design
+
+- **Growth:** the read grew with every task the PM was ever in, and nothing deletes those. These three commands now read no more as tasks accumulate.
+- **Concurrency:** N/A: fewer parts loaded, and a commit compares only what was loaded.
+- **Read cost:** on prod, the agent with the most dormant groups: 1,668 of them, 9.5 MB whole (Ken's slow log: 676 KB light, up to 1.66 s). Now none for these commands. A removal still reads them, and removing a live PM is refused.
+- **Scale (10x):** unchanged for these commands. The metric that warns first: slow `SwarmGroup` reads on `{tabId, agentIds}`, from a removal or the watch check below.
+- **Compatibility:** server only; old and new clients see the same answers.
+- **Migration + rollback:** none. No index is created or dropped. Rolling back restores the read.
+- **Audience, identity:** unchanged: the same checks on the same agent.
+- **Failure:** if the project chat is not loaded, `ledProject` is false and the agent is named as before.
+- **Platforms:** server only.
+
+**Left open, out of this task's owned code:** `store.ts cachedState` (who may open a share) reads an agent's dormant groups by name through the read cache. It is cached until the Swarm changes; it is not `load.ts:63`.
+
+## Global Constraints
+
+- Removals keep reading dormant groups.
+- Prod is read only, through the read-only login from its environment file. No index is created or dropped.
+- Full Super Tab integration suite before review.
+- PR body starts: `Requested by [email protected] through Harness.`
+
+## Review Focus
+
+- A worker whose only group is asleep raising an alert: the row still goes to that group.
+- Removing an agent from the Swarm: it leaves its dormant groups too.
+- An agent that lost the coordinator role, with no project chat of its own: it is named as any other.
+- A coordinator's alert: its row lands in a group its readers can read, not nowhere.
+- A coordinator's terminal stays closed to whoever it was closed to.
+
+---
+
+### Task 1: The coordinator is left out
+
+**Files:**
+- Modify: `backend/src/superTabs/store.ts` (`moreNamed`)
+- Create: `backend/integration/superTabCommandLoads.test.ts`
+- Modify: `backend/vitest.super-tabs.config.ts`
+
+**Interfaces:**
+- Consumes: `ledProject(state, agentId)`.
+- Produces: nothing new.
+
+- [ ] **Step 1: Commit the failing test:** a Swarm with 1,000 dormant groups of the PM: `agent.alert`, `agent.rebind` and `agent.input` naming it read under 100 `SwarmGroup` rows. Guards: a worker's alert reaches its only dormant group; `agent.remove` of a worker leaves its dormant groups; the PM's alert row lands; its terminal stays closed.
+- [ ] **Step 2: Watch it fail** on the three reads.
+- [ ] **Step 3: Commit the code** that passes it.
+- [ ] **Step 4: Run** the new file, the Super Tab unit tests and the full Super Tab integration suite.
+- [ ] **Step 5: Open the pull request and the Review step** (Rowan), with the prod measures. The slow log after the release is Ken's: the read-only login cannot read it.