Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Stop nightly routing reports in project chats (#3058)

harness-for-autonomous[bot]7802e5756970
diff --git a/backend/integration/superTabBuddies.test.ts b/backend/integration/superTabBuddies.test.ts
index 6b75b993f..8f2e5cec0 100644
--- a/backend/integration/superTabBuddies.test.ts
+++ b/backend/integration/superTabBuddies.test.ts
@@ -332,7 +332,7 @@ describe('a Buddy through the store', () => {
await expect(f.store.costSummary(peer, f.tab, 7, agentId)).rejects.toThrow('A Buddy reads no routing')
})
- it('never reads the nightly routing reflection, which says what each engine and model cost: routing too', async () => {
+ it('receives no nightly report and still cannot read historical routing reports', async () => {
const f = await swarm()
const { agentId } = await buddy(f)
const group = String((await f.execute({ type: 'conversation.create', projectId: 'lead', title: 'Parser', brief: '', checks: [] })).result.conversationId)
@@ -340,11 +340,16 @@ describe('a Buddy through the store', () => {
await db.superTabRouteDecision.create({ data: { id: randomUUID(), tabId: f.tab, conversationId: group, assignmentId: randomUUID(), outcome: 'done',
createdAt: new Date(at), final: { kind: 'member' }, record: { assessment: { kind: 'parser' } }, run: { machineId: 'peer-mac', harnessId: 'h',
engine: 'claude', model: 'opus', effort: 'high', billing: 'subscription', startedAt: at, start: {}, endedAt: at + 60_000, end: {} } } })
- expect(await f.store.postReflection(f.tab, 'lead', '2026-10-30')).toBe('posted')
- const report = (await f.store.messages(peer, f.tab, 'lead', {})).messages.find(m => (m.source as { kind?: unknown } | null)?.kind === 'routing-reflection')
- expect(report?.text).toContain('- claude opus high, parser: 1 attempt, 0 shipped')
- expect((await f.store.messages(peer, f.tab, 'lead', {}, agentId)).messages.map(m => m.id)).not.toContain(report!.id)
- await expect(f.store.messages(peer, f.tab, 'lead', { messageId: report!.id }, agentId)).resolves.toMatchObject({ messages: [] })
+ expect(await f.store.postReflection(f.tab, 'lead', '2026-10-30')).toBe('empty')
+ expect((await f.store.messages(peer, f.tab, 'lead', {})).messages.some(m => (m.source as { kind?: unknown } | null)?.kind === 'routing-reflection')).toBe(false)
+ const report = await db.superTabMessage.create({ data: { id: randomUUID(), tabId: f.tab, conversationId: 'lead', sequence: 1,
+ author: { kind: 'agent', id: f.lead, name: 'Lead' }, recipientIds: [], mentions: [], text: 'Routing reflection: an old report.',
+ source: { kind: 'routing-reflection', projectId: 'lead', on: '2026-10-29' } } })
+ expect((await f.store.messages(peer, f.tab, 'lead', { messageId: report.id })).messages).toEqual([
+ expect.objectContaining({ id: report.id, text: report.text }),
+ ])
+ expect((await f.store.messages(peer, f.tab, 'lead', {}, agentId)).messages.map(m => m.id)).not.toContain(report.id)
+ await expect(f.store.messages(peer, f.tab, 'lead', { messageId: report.id }, agentId)).resolves.toMatchObject({ messages: [] })
})
it('never reads the routing proposals asked of its person: not in its decisions, nor as the PM\'s line when its person leaves one to the PM (review of #2276)', async () => {
diff --git a/backend/integration/superTabReflection.test.ts b/backend/integration/superTabReflection.test.ts
index 9e66b7ac5..26d55d866 100644
--- a/backend/integration/superTabReflection.test.ts
+++ b/backend/integration/superTabReflection.test.ts
@@ -12,11 +12,9 @@ import { SuperTabStore } from '../src/superTabs/store.js'
import type { Command } from '../src/superTabs/commands.js'
import type { TokensByModel } from '../src/superTabs/costs/tokens.js'
import { reflectNightly } from '../src/lib/routingReflection.js'
-import { proposalTitle } from '../src/superTabs/costs/reflection.js'
+import { proposalTitle, reflectionMessageId } from '../src/superTabs/costs/reflection.js'
import { poolRows } from './hostRows.test-support.js'
-// The nightly routing reflection (Scholes's roadmap, piece 2, 10-30): at 23:00 in the project's zone the worker posts what its
-// routed work cost and how it went, and asks its owner about routing changes. A fresh loopback replica set, created here.
let replica: MongoMemoryReplSet
let db: PrismaClient
const owner: AuthUser = { sub: '310000000000000000000001', email: '[email protected]', autonomousEnv: 'prod', role: 'user' }
@@ -113,82 +111,69 @@ async function swarm() {
}
describe('the nightly routing reflection', () => {
- it('posts a seeded day once in the project chat at 23:00, under the Lead, and asks the owner each proposal once', async () => {
+ it('never posts a report, and asks the owner each proposal once in two plain-language lines', async () => {
const f = await swarm()
await f.ship()
for (let i = 0; i < 3; i++) await f.stepUpFromGrid()
+ const messages = await db.superTabMessage.findMany({ where: { tabId: f.tab, conversationId: 'lead' } })
f.clock.now = NIGHT - 3 * HOUR
// Before 23:00 in Vietnam, nothing.
await reflectNightly(db, f.store, f.clock.now)
expect(await f.reports()).toEqual([])
f.clock.now = NIGHT
- // Two passes at once, as two workers in a rolling deploy: one report.
await Promise.all([reflectNightly(db, f.store, NIGHT), reflectNightly(db, f.store, NIGHT)])
await reflectNightly(db, f.store, NIGHT + 15 * MINUTE)
- const [report, ...more] = await f.reports()
- expect(more).toEqual([])
- expect(report!.author).toMatchObject({ kind: 'agent', id: f.lead })
- expect(report!.recipientIds).toEqual([])
- const lines = report!.text.split('\n')
- expect(lines).toContain('Last day: 7 attempts, 1 shipped. Last 7 days: 7 attempts, 1 shipped.')
- expect(lines).toContain('- claude opus high, parser: 1 attempt, 1 shipped, median 12 min to ship, $4.00 per shipped change, no first review')
- expect(lines).toContain(`- claude ${GRID} low, parser: 3 attempts, 0 shipped, no first review`)
- expect(lines).toContain('- claude sonnet medium, parser: 3 attempts, 0 shipped, no first review, 3 step-ups')
- expect(lines).toContain('1. Start parser work on a subscription model instead of the grid: about 3 hours to ship a week. '
- + '3 of 3 parser tasks started on the grid stepped up after their grid attempt failed. Only 3 tasks. Asked of the project owner.')
- // Counts only: no task title, machine or person.
- for (const secret of ['Secret', 'Worker Mac', 'test-host', owner.email, host.email]) expect(report!.text).not.toContain(secret)
+ expect(await f.reports()).toEqual([])
+ expect(await db.superTabMessage.findMany({ where: { tabId: f.tab, conversationId: 'lead' } })).toEqual(messages)
const decisions = await db.superTabDecision.findMany({ where: { tabId: f.tab } })
expect(decisions).toEqual([expect.objectContaining({ personId: owner.sub, askerAgentId: f.lead, conversationId: 'lead', assignmentId: null,
- question: 'Start parser work on a subscription model instead of the grid? Expected saving: about 3 hours to ship a week. '
- + '3 of 3 parser tasks started on the grid stepped up after their grid attempt failed. Only 3 tasks.',
+ question: 'Start parser work on a subscription model instead of the grid?\n'
+ + 'Why: 3 of 3 parser tasks needed another model after their grid attempt failed. Only 3 tasks. This could save about 3 hours to ship a week.',
options: ['Yes: the coordinator writes this into the Routing section of the instructions', 'No: keep routing as it is',
'Not yet: ask again when there is more evidence'],
// Three tasks are too few to recommend Yes (review of #2276).
recommended: 2, kind: 'routing-reflection' })])
- expect(await f.store.postReflection(f.tab, 'lead', '2026-10-30')).toBe('already')
+ for (const secret of ['Routing reflection:', 'Last day:', '\n- ', 'Secret', 'Worker Mac', 'test-host', owner.email, host.email]) {
+ expect(decisions[0]!.question).not.toContain(secret)
+ }
+ expect((await f.store.decisions(owner, f.tab, 'open')).decisions.map(d => d.id)).toContain(decisions[0]!.id)
+ expect(await f.store.postReflection(f.tab, 'lead', '2026-10-30')).toBe('empty')
// The next night, the same proposal still waits: it is not asked again.
- expect(await f.store.postReflection(f.tab, 'lead', '2026-10-31')).toBe('posted')
+ expect(await f.store.postReflection(f.tab, 'lead', '2026-10-31')).toBe('empty')
expect(await db.superTabDecision.count({ where: { tabId: f.tab } })).toBe(1)
- expect((await f.reports()).at(-1)!.text).toContain("Only 3 tasks. Waiting for the owner's answer since 2026-10-30.")
- // People read it; it is a notice for people, not a wake for the Lead.
- expect((await f.store.messages(host, f.tab, 'lead', {})).messages.map(m => m.id)).toContain(report!.id)
+ expect(await f.reports()).toEqual([])
+ expect((await f.store.costSummary(host, f.tab, 7, f.lead)).totals).toMatchObject({ attempts: 7, shipped: 1 })
})
- // Project instructions, phase 4 (decision 21): a Yes is a line the coordinator writes into the instructions. No approved
- // list, no nightly reminder to the coordinator; the reflection reads the instructions instead.
- it('asks an answered proposal again only after 14 days, posts no reminder of approved changes, and reports one the instructions hold', async () => {
+ it('suppresses answered and written proposals without posting approval reminders', async () => {
const f = await swarm()
await f.ship()
for (let i = 0; i < 3; i++) await f.stepUpFromGrid()
const GRID_TITLE = 'Start parser work on a subscription model instead of the grid'
- /** The report [days] after the first night, with an attempt in its last day. */
const night = async (on: string, days: number) => {
f.clock.now = NIGHT + days * DAY
await f.seed(f.group, { at: f.clock.now - HOUR })
- expect(await f.store.postReflection(f.tab, 'lead', on)).toBe('posted')
- return (await f.reports()).at(-1)!.text
+ const result = await f.store.postReflection(f.tab, 'lead', on)
+ expect(await f.reports()).toEqual([])
+ return result
}
- expect(await night('2026-10-30', 0)).toContain('Asked of the project owner.')
+ expect(await night('2026-10-30', 0)).toBe('posted')
const [first] = await db.superTabDecision.findMany({ where: { tabId: f.tab } })
await f.execute({ type: 'decision.answer', decisionId: first!.id, choice: 1 })
// The owner said No: the next night neither asks nor proposes it.
- expect(await night('2026-10-31', 1)).toContain('No change proposed.')
+ expect(await night('2026-10-31', 1)).toBe('empty')
expect(await db.superTabDecision.count({ where: { tabId: f.tab } })).toBe(1)
// Fifteen days after that answer it is asked again.
await db.superTabDecision.update({ where: { id: first!.id }, data: { answeredAt: new Date(NIGHT - 15 * DAY) } })
- expect(await night('2026-11-01', 2)).toContain(`${GRID_TITLE}: about 3 hours`)
+ expect(await night('2026-11-01', 2)).toBe('posted')
const again = (await db.superTabDecision.findMany({ where: { tabId: f.tab, OR: [{ answeredAt: null }, { answeredAt: { isSet: false } }] } }))
expect(again.map(d => proposalTitle(d.question))).toEqual([GRID_TITLE])
// The owner says Yes: no list of approved changes, and no reminder to the coordinator, that night or after.
f.clock.now = NIGHT + 2 * DAY + 2 * HOUR
await f.execute({ type: 'decision.answer', decisionId: again[0]!.id, choice: 0 })
const approved = await night('2026-11-02', 3)
- expect(approved).not.toContain('Approved changes')
- expect(approved).toContain('No change proposed.')
- // Until its line is in the instructions, each report says so: a Yes is never lost silently (final review of the branch).
+ expect(approved).toBe('empty')
const line = 'Parser work starts on a subscription model, not on the grid.'
- expect(approved.split('\n')).toContain(`Approved 2026-11-02; not in the instructions yet: ${line}`)
expect((await db.superTabMessage.findMany({ where: { tabId: f.tab, recipientIds: { has: f.lead } } }))
.filter(m => (m.source as { kind?: string } | null)?.kind === 'routing-reflection')).toEqual([])
expect((await db.superTabMessage.findMany({ where: { tabId: f.tab } })).filter(m => m.text.startsWith('Approved routing'))).toEqual([])
@@ -196,19 +181,15 @@ describe('the nightly routing reflection', () => {
const doc = await f.store.instructions(owner, f.tab, 'lead')
await f.execute({ type: 'instructions.save', projectId: 'lead', baseRevision: doc.revision, text: `${doc.text.trimEnd()}\n- ${line}\n`,
summary: `Routing: ${line}`, askedBy: owner.sub }, host, f.lead)
- // Approved lately and in the instructions: reported as there, not as missing.
const written = await night('2026-11-03', 4)
- expect(written).toContain(`In the instructions: ${line}`)
- expect(written).not.toContain('not in the instructions yet')
- // Answered over 14 days ago: the instructions alone keep it from being asked again, and the report says how the week ran.
+ expect(written).toBe('empty')
await db.superTabDecision.update({ where: { id: again[0]!.id }, data: { answeredAt: new Date(NIGHT - 28 * DAY) } })
const later = await night('2026-11-04', 5)
- expect(later).toContain(`In the instructions: ${line} The last 7 days ran 3 parser tasks on the grid.`)
- expect(later).toContain('No change proposed.')
+ expect(later).toBe('empty')
expect(await db.superTabDecision.count({ where: { tabId: f.tab, OR: [{ answeredAt: null }, { answeredAt: { isSet: false } }] } })).toBe(0)
})
- it('keeps at most three proposals open, says when one was not asked, and lets one unanswered for 7 days expire (review of #2276)', async () => {
+ it('keeps at most three proposals open without notices, and expires them after 7 unanswered days', async () => {
const f = await swarm()
await f.ship()
for (let i = 0; i < 3; i++) await f.stepUpFromGrid()
@@ -217,9 +198,9 @@ describe('the nightly routing reflection', () => {
recommended: 2, unblocks: 'If you say yes.', createdAt: new Date(NIGHT - 6 * DAY), kind: 'routing-reflection' })
await db.superTabDecision.createMany({ data: ['ui', 'docs', 'tests'].map(kind => open(`Stop routing ${kind} work to claude opus high`)) })
f.clock.now = NIGHT
- expect(await f.store.postReflection(f.tab, 'lead', '2026-10-30')).toBe('posted')
+ expect(await f.store.postReflection(f.tab, 'lead', '2026-10-30')).toBe('empty')
expect(await db.superTabDecision.count({ where: { tabId: f.tab } })).toBe(3)
- expect((await f.reports()).at(-1)!.text).toContain("Only 3 tasks. Not asked: at most 3 proposals wait for the owner's answer at once.")
+ expect(await f.reports()).toEqual([])
// Two days on, the three are 8 days old. The owner also has 17 other open decisions from the Lead: 20 with the three, the
// most one asker may hold. The three expire before they are counted, so the proposal is asked (delta review of #2276).
await db.superTabDecision.createMany({ data: Array.from({ length: 17 }, (_, i) => ({ id: randomUUID(), tabId: f.tab, batchId: 'plain', projectId: 'lead',
@@ -230,7 +211,7 @@ describe('the nightly routing reflection', () => {
expect(await f.store.postReflection(f.tab, 'lead', '2026-11-01')).toBe('posted')
const proposals = () => db.superTabDecision.findMany({ where: { tabId: f.tab, kind: 'routing-reflection' } })
expect((await proposals()).map(d => proposalTitle(d.question))).toEqual(['Start parser work on a subscription model instead of the grid'])
- expect((await f.reports()).at(-1)!.text).toContain('Only 3 tasks. Asked of the project owner.')
+ expect(await f.reports()).toEqual([])
// Unanswered for 8 days, it expires on a night with nothing to report too.
f.clock.now = NIGHT + 10 * DAY
expect(await f.store.postReflection(f.tab, 'lead', '2026-11-09')).toBe('empty')
@@ -246,24 +227,46 @@ describe('the nightly routing reflection', () => {
await f.ship()
// Two days later the evidence is still this week's, but the last day had no attempt.
f.clock.now = NIGHT + 2 * DAY
- const pass = await reflectNightly(db, f.store, f.clock.now)
+ await reflectNightly(db, f.store, f.clock.now)
expect(await f.reports()).toEqual([])
expect(await f.reports(quiet.conversationId)).toEqual([])
- expect(pass.posted).toBe(0)
expect(await f.store.postReflection(f.tab, 'lead', '2026-11-01')).toBe('empty')
// The quiet project has no evidence, so no pass reads it; asked directly, it has nothing to say.
expect(await f.store.postReflection(f.tab, quiet.projectId, '2026-11-01')).toBe('empty')
expect(await db.superTabDecision.count({ where: { tabId: f.tab } })).toBe(0)
})
- it('counts only its own project: never another project’s tasks, a private chat or the Leads conversation, and posts each where its people read it', async () => {
+ it('keeps active nights without a proposal silent in every Swarm, preserving costs and old reports', async () => {
+ for (const fixture of [await swarm(), await swarm()]) {
+ await fixture.ship()
+ const oldId = reflectionMessageId(fixture.tab, 'lead', '2026-10-29')
+ await db.superTabMessage.create({ data: {
+ id: oldId, tabId: fixture.tab, conversationId: 'lead', sequence: 1, author: { kind: 'agent', id: fixture.lead, name: 'Lead' },
+ text: 'Routing reflection: an old report.', recipientIds: [], mentions: [],
+ source: { kind: 'routing-reflection', projectId: 'lead', on: '2026-10-29' },
+ } })
+ const messages = await db.superTabMessage.findMany({ where: { tabId: fixture.tab } })
+ const costs = await fixture.store.costSummary(host, fixture.tab, 7, fixture.lead)
+ fixture.clock.now = NIGHT
+ expect(await fixture.store.postReflection(fixture.tab, 'lead', '2026-10-29')).toBe('already')
+ expect(await fixture.store.postReflection(fixture.tab, 'lead', '2026-10-30')).toBe('empty')
+ expect(await db.superTabMessage.findMany({ where: { tabId: fixture.tab } })).toEqual(messages)
+ expect(await db.superTabDecision.count({ where: { tabId: fixture.tab } })).toBe(0)
+ expect((await fixture.store.costSummary(host, fixture.tab, 7, fixture.lead)).totals).toEqual(costs.totals)
+ expect((await fixture.store.messages(owner, fixture.tab, 'lead', { messageId: oldId })).messages).toEqual([
+ expect.objectContaining({ id: oldId, text: 'Routing reflection: an old report.' }),
+ ])
+ }
+ })
+
+ it('asks only about its own project, never another project, a private chat or the Leads conversation', async () => {
const f = await swarm()
await f.ship()
// A private project of the owner's alone, with its own Lead and a stepped-up task.
const hidden = (await f.execute({ type: 'project.create', name: 'Hidden', visibility: 'private', brief: '' })).result as { projectId: string; conversationId: string }
const hiddenLead = String((await f.execute({ type: 'agent.add', conversationId: hidden.conversationId, machineId: 'lead-host', harnessId: randomUUID(),
name: 'Hidden Lead', configuration: config, lead: true })).result.agentId)
- await f.stepUpFromGrid(hidden.projectId, 'Hidden grid task')
+ for (let attempt = 0; attempt < 3; attempt++) await f.stepUpFromGrid(hidden.projectId, 'Hidden grid task')
await db.superTabModelEvidence.create({ data: { id: randomUUID(), tabId: f.tab, projectId: hidden.projectId, engine: 'claude', model: 'opus', kind: 'feature',
week: new Date(Date.UTC(2026, 9, 26)), updatedAt: new Date(f.clock.now) } })
// Attempts in a private chat and in the Leads conversation of this project's Swarm.
@@ -273,28 +276,28 @@ describe('the nightly routing reflection', () => {
f.clock.now = NIGHT
const pass = await reflectNightly(db, f.store, NIGHT)
expect(pass.failed).toBe(0)
- const [mine] = await f.reports(), [theirs] = await f.reports(hidden.conversationId)
- expect(mine!.text).toContain('Last day: 1 attempt, 1 shipped.')
- expect(mine!.text).not.toContain('step-up')
- expect(theirs!.text).toContain('Last day: 2 attempts, 0 shipped.')
- expect(theirs!.author).toMatchObject({ id: hiddenLead })
- // Someone outside the private project reads neither its chat nor its report; asking by another Swarm's id finds nothing.
+ expect(await f.reports()).toEqual([])
+ expect(await f.reports(hidden.conversationId)).toEqual([])
+ expect(await db.superTabDecision.findMany({ where: { tabId: f.tab } })).toEqual([expect.objectContaining({
+ projectId: hidden.projectId, conversationId: hidden.conversationId, personId: owner.sub, askerAgentId: hiddenLead,
+ question: expect.stringContaining('3 of 3 parser tasks'),
+ })])
await expect(f.store.messages(host, f.tab, hidden.conversationId, {})).rejects.toMatchObject({ code: 'NOT_FOUND' })
- await expect(f.store.messages(host, f.tab, hidden.conversationId, { messageId: theirs!.id })).rejects.toMatchObject({ code: 'NOT_FOUND' })
+ expect((await f.store.decisions(host, f.tab, 'open')).decisions).toEqual([])
const other = await swarm()
expect(await other.store.postReflection(other.tab, hidden.projectId, '2026-10-30')).toBe('skipped')
expect(await f.store.postReflection(f.tab, 'no-such-project', '2026-10-30')).toBe('skipped')
expect(await other.reports(hidden.conversationId)).toEqual([])
})
- it('tries a failed report again on the next pass, and the other projects still get theirs', async () => {
+ it('retries a failed reflection while other projects still receive their Decisions without reports', async () => {
const f = await swarm()
await f.ship()
// A second project of the same Swarm, with its own Lead and a day of routed work.
const web = (await f.execute({ type: 'project.create', name: 'Web', visibility: 'public', brief: '' })).result as { projectId: string; conversationId: string }
await f.execute({ type: 'agent.add', conversationId: web.conversationId, machineId: 'lead-host', harnessId: randomUUID(), name: 'Web Lead',
configuration: config, lead: true })
- await f.stepUpFromGrid(web.projectId, 'Web grid task')
+ for (let attempt = 0; attempt < 3; attempt++) await f.stepUpFromGrid(web.projectId, 'Web grid task')
await db.superTabModelEvidence.create({ data: { id: randomUUID(), tabId: f.tab, projectId: web.projectId, engine: 'claude', model: 'opus', kind: 'feature',
week: new Date(Date.UTC(2026, 9, 26)), updatedAt: new Date(f.clock.now) } })
f.clock.now = NIGHT
@@ -306,9 +309,10 @@ describe('the nightly routing reflection', () => {
const warned: unknown[] = []
expect(await reflectNightly(db, flaky, NIGHT, { warn: (...a: unknown[]) => { warned.push(a) } })).toMatchObject({ failed: 1 })
expect(await f.reports()).toEqual([])
- expect(await f.reports(web.conversationId)).toHaveLength(1)
+ expect(await f.reports(web.conversationId)).toEqual([])
+ expect(await db.superTabDecision.count({ where: { tabId: f.tab, projectId: web.projectId } })).toBe(1)
expect(warned).toHaveLength(1)
await reflectNightly(db, flaky, NIGHT + 15 * MINUTE)
- expect(await f.reports()).toHaveLength(1)
+ expect(await f.reports()).toEqual([])
})
})
diff --git a/backend/src/lib/routingReflection.test.ts b/backend/src/lib/routingReflection.test.ts
index 72e5dbd60..465d1c118 100644
--- a/backend/src/lib/routingReflection.test.ts
+++ b/backend/src/lib/routingReflection.test.ts
@@ -24,7 +24,7 @@ describe('when the nightly routing reflection is due', () => {
expect(reflectionDue({ digest: { time: '08:00', timeZone: 'Mars/Olympus' } }, utc(30, 16, 30))).toBe('2026-10-30')
})
- it('posts one report a project a local date', () => {
+ it('uses a stable nightly operation ID per project and local date', () => {
expect(reflectionMessageId('tab', 'p1', '2026-10-30')).toBe(reflectionMessageId('tab', 'p1', '2026-10-30'))
expect(reflectionMessageId('tab', 'p1', '2026-10-30')).not.toBe(reflectionMessageId('tab', 'p1', '2026-10-31'))
expect(reflectionMessageId('tab', 'p1', '2026-10-30')).not.toBe(reflectionMessageId('tab', 'p2', '2026-10-30'))
diff --git a/backend/src/lib/routingReflection.ts b/backend/src/lib/routingReflection.ts
index 5de5ac16d..af36d11ba 100644
--- a/backend/src/lib/routingReflection.ts
+++ b/backend/src/lib/routingReflection.ts
@@ -8,8 +8,8 @@ import { logger } from '../utils/logger.js'
/**
* The nightly routing reflection (Scholes's roadmap, piece 2, 2026-10-30): at 23:00 in each project's time zone, the singleton
- * worker posts in the project chat what its routed work cost and how it went, with at most three routing changes asked of
- * its owner (superTabs/costs/reflection.ts, store.ts postReflection). Nothing changes routing without the owner's answer.
+ * worker asks its owner about at most three routing changes as Decisions (superTabs/costs/reflection.ts, store.ts
+ * postReflection). It never posts reports in project chats; the coordinator reads the numbers with costs.
*
* - Which projects: those with evidence rows (superTabs/evidence/store.ts) from the last two weeks, grouped in Mongo by the
* evidence collection's week index; a project with no routed work has none and is never read. Then one head per Swarm,
diff --git a/backend/src/superTabs/costs/reflection.test.ts b/backend/src/superTabs/costs/reflection.test.ts
index 8a14aa6da..58e47c7a4 100644
--- a/backend/src/superTabs/costs/reflection.test.ts
+++ b/backend/src/superTabs/costs/reflection.test.ts
@@ -1,9 +1,8 @@
import { describe, expect, it } from 'vitest'
import type { Conversation } from '../model.js'
-import type { ModelKindEvidence } from '../evidence/store.js'
import type { RouteRun } from './attempts.js'
import type { CostRow } from './read.js'
-import { approvedLine, localDate, proposalQuestion, proposalTitle, routingLine, routingReflection, wilson, type FailedStart, type ReflectionInput } from './reflection.js'
+import { proposalQuestion, proposalTitle, routingLine, routingReflection, wilson, type FailedStart, type ReflectionInput } from './reflection.js'
const MINUTE = 60_000, HOUR = 60 * MINUTE, DAY = 24 * HOUR
const NOW = 10 * DAY
@@ -17,10 +16,6 @@ const row = (assignmentId: string, conversationId: string, kind: string, r: Rout
const group = (id: string, runs: Array<{ assignmentId: string; firstReview?: boolean }>, shipped?: { by: string; at: number }) => ({ id,
title: 'Secret plan for Acme', ledger: { runs: runs.map(r => ({ engine: 'claude', model: 'opus', at: 0, ...r })),
...shipped ? { shippedAt: shipped.at, shippedBy: { assignmentId: shipped.by } } : {} } }) as unknown as Conversation
-const evidence = (engine: string, model: string, kind: string, over: Partial<ModelKindEvidence>) => ({ engine, model, kind, attempts: 0, firstReviews: 0,
- firstReviewPassed: 0, reworkRounds: 0, shipped: 0, minutesToShipSum: 0, minutesToShipCount: 0, fixesAfterMerge: 0, tokensInput: 0, tokensOutput: 0,
- tokensCacheRead: 0, tokensCacheWrite: 0, tokensKnown: 0, apiCost: 0, costKnown: 0, costUnknown: 0, spend: 0, spendKnown: 0, firstReviewPassRate: null,
- averageMinutesToShip: null, costPerShipped: null, recent: [], ...over }) as ModelKindEvidence
/**
* A seeded week of one project:
@@ -53,8 +48,7 @@ function seededWeek(): ReflectionInput {
groups.push(group(g, [{ assignmentId: `d${i}`, firstReview: false }], { by: `d${i}b`, at: at(daysAgo) + 3 * HOUR }))
})
const failed: FailedStart = { engine: 'codex', model: 'gpt-6.1-sol', effort: 'high', at: at(0.2) }
- return { now: NOW, rows, groups, failedStarts: [failed, { ...failed, at: at(3) }], truncated: false, document: '# Project instructions\n\n## Routing\n\n- Opus builds.\n',
- evidence: [evidence('claude', 'opus', 'feature', { reworkRounds: 1, fixesAfterMerge: 1 }), evidence('codex', 'gpt-6.1-sol', 'ui', { reworkRounds: 2 })] }
+ return { now: NOW, rows, groups, failedStarts: [failed, { ...failed, at: at(3) }], document: '# Project instructions\n\n## Routing\n\n- Opus builds.\n' }
}
/** [n] more first reviews of a kind on a model, all as [passed], each shipped at $[cost]. */
function more(week: ReflectionInput, kind: string, engine: string, model: string, effort: string, n: number, passed: boolean) {
@@ -69,17 +63,9 @@ const PARSER = 'Route parser work to claude sonnet medium instead of claude opus
const DOCS = 'Start docs work on a subscription model instead of the grid'
describe('the nightly routing reflection', () => {
- it('reports a seeded week: what shipped, time to ship, cost per shipped change and first-review pass rate per engine, model, effort and kind, and rework per model', () => {
+ it('returns proposals only, without a report or a report renderer', () => {
const report = routingReflection(seededWeek())!
- const lines = report.text.split('\n')
- expect(lines).toContain('Last day: 2 attempts, 2 shipped. Last 7 days: 22 attempts, 14 shipped.')
- expect(lines).toContain('- claude opus high, parser: 5 attempts, 5 shipped, median 60 min to ship, $4.00 per shipped change, first review passed 4 of 5 (80%)')
- expect(lines).toContain('- claude sonnet medium, parser: 5 attempts, 5 shipped, median 60 min to ship, $2.00 per shipped change, first review passed 4 of 5 (80%)')
- expect(lines).toContain('- codex gpt-6.1-sol high, ui: 6 attempts, 1 shipped, median 60 min to ship, $12.00 per shipped change, first review passed 0 of 6 (0%)')
- expect(lines).toContain('- claude deepseek/deepseek-v4.1-flash-1017 low, docs: 3 attempts, 0 shipped, first review passed 0 of 3 (0%)')
- expect(lines).toContain('- claude sonnet medium, docs: 3 attempts, 3 shipped, median 60 min to ship, $2.00 per shipped change, no first review, 3 step-ups')
- expect(lines).toContain('Waste in the last 7 days: 2 failed starts (codex gpt-6.1-sol high: 2), 3 step-ups after a failed grid attempt '
- + '(docs: 3, 3 hours), 3 rework rounds (codex gpt-6.1-sol ui: 2, claude opus feature: 1), 1 fix after merge (claude opus feature: 1).')
+ expect(Object.keys(report)).toEqual(['proposals'])
})
it('proposes at most three changes, the largest saving first, each with its counts, its uncertainty and its API-equivalent saving', () => {
@@ -89,11 +75,9 @@ describe('the nightly routing reflection', () => {
saving: 'about $12.00 a week at API prices of attempts that failed their first review', dollars: 12, hours: 0, strong: false },
{ title: PARSER, why: 'Its first review passed 4 of 5 (80%) against 4 of 5 (80%), at $2.00 instead of $4.00 per shipped change at API prices. '
+ 'Small samples (5 and 5 first reviews): the difference may be chance.', saving: 'about $10.00 a week at API prices', dollars: 10, hours: 0, strong: false },
- { title: DOCS, why: '3 of 3 docs tasks started on the grid stepped up after their grid attempt failed. Only 3 tasks.',
+ { title: DOCS, why: '3 of 3 docs tasks needed another model after their grid attempt failed. Only 3 tasks.',
saving: 'about 3 hours to ship a week', dollars: 0, hours: 3, strong: false },
])
- expect(report.text).toContain('Proposed changes (advice: no rule or setting changes by itself):')
- expect(report.text).toContain(`2. ${PARSER}: about $10.00 a week at API prices. Its first review passed 4 of 5`)
})
// Project instructions, phase 4 (decision 21): a Yes is one fixed line the coordinator writes into the Routing section.
@@ -104,7 +88,7 @@ describe('the nightly routing reflection', () => {
expect(routingLine(docs!)).toBe('Docs work starts on a subscription model, not on the grid.')
const asked = proposalQuestion(ui!)
expect(asked.options[0]!.startsWith('Yes: the coordinator writes this into the Routing section of the instructions')).toBe(true)
- expect(asked).toEqual({ question: `${UI}? Expected saving: ${ui!.saving}. ${ui!.why}`,
+ expect(asked).toEqual({ question: `${UI}?\nWhy: ${ui!.why} This could save ${ui!.saving}.`,
options: [asked.options[0], 'No: keep routing as it is', 'Not yet: ask again when there is more evidence'],
recommended: 2, unblocks: expect.stringContaining(routingLine(ui!)) })
expect(asked.unblocks).not.toMatch(/reminded|30 days/)
@@ -140,54 +124,14 @@ describe('the nightly routing reflection', () => {
expect(routingReflection({ ...week, decided: ['Stop routing test work to codex gpt-6.1-sol xhigh'] })!.proposals.map(p => p.title)).toEqual([UI, PARSER, DOCS])
})
- // Project instructions, phase 4 (decision 21): the approved list, its nightly reminder and the recall of a lapsed approval
- // are gone; a proposal whose line the instructions hold is reported with how the last 7 days ran, never asked again.
it('reads the instructions instead of an approved list', () => {
const week = seededWeek()
const [ui, parser] = routingReflection(week)!.proposals
const document = `${week.document}- ${routingLine(ui!)}\n- ${routingLine(parser!)}\n`
const report = routingReflection({ ...week, document })!
expect(report.proposals.map(p => p.title)).toEqual([DOCS])
- expect(report.text.split('\n')).toEqual(expect.arrayContaining([
- 'In the instructions: Ui work no longer runs on codex gpt-6.1-sol high. The last 7 days ran 6 ui tasks on codex gpt-6.1-sol high.',
- 'In the instructions: Parser work runs on claude sonnet medium instead of claude opus high. The last 7 days ran 5 parser tasks on claude opus high.']))
- expect(report.text).not.toMatch(/Approved changes|reminded/)
// Without the line, the same change is proposed.
expect(routingReflection(week)!.proposals.map(p => p.title)).toEqual([UI, PARSER, DOCS])
- expect(routingReflection(week)!.text).not.toContain('In the instructions:')
- })
-
- // A Yes changes nothing by itself: until the coordinator writes its line, each report says it was approved and is not in the
- // instructions yet, read from the document (final review of the branch).
- it('lists an approved line not in the instructions yet, until it is there', () => {
- const week = seededWeek()
- const [ui] = routingReflection(week)!.proposals
- expect(approvedLine(proposalQuestion(ui!).unblocks)).toBe(routingLine(ui!))
- expect(approvedLine('If you say yes, the Lead is reminded each night.')).toBeNull()
- const approved = [{ line: routingLine(ui!), on: '2026-10-29' }, { line: 'Docs work runs on claude opus high instead of claude sonnet medium.', on: '2026-10-28', leftTo: 'PM' }]
- const report = routingReflection({ ...week, decided: [ui!.title], approved })!
- expect(report.text.split('\n')).toEqual(expect.arrayContaining([
- 'Approved 2026-10-29; not in the instructions yet: Ui work no longer runs on codex gpt-6.1-sol high.',
- 'Approved 2026-10-28 (left to the PM); not in the instructions yet: Docs work runs on claude opus high instead of claude sonnet medium.']))
- expect(report.proposals.map(p => p.title)).toEqual([PARSER, DOCS])
- const written = routingReflection({ ...week, decided: [ui!.title], approved: approved.slice(0, 1), document: `${week.document}- ${routingLine(ui!)}\n` })!
- expect(written.text).not.toContain('not in the instructions yet')
- expect(written.text).toContain(`In the instructions: ${routingLine(ui!)}`)
- })
-
- it('dates in the project\'s time zone, the default one for none or an unknown one', () => {
- const late = Date.UTC(2026, 9, 30, 18, 0)
- expect(localDate(late)).toBe('2026-10-31')
- expect(localDate(late, 'Europe/Berlin')).toBe('2026-10-30')
- expect(localDate(late, 'Mars/Olympus')).toBe('2026-10-31')
- })
-
- it('says after each proposal what happened to it', () => {
- const text = routingReflection(seededWeek())!.render(p => p.title === UI ? 'Asked of the project owner.' : null)
- expect(text).toContain(`1. ${UI}: about $12.00 a week at API prices of attempts that failed their first review. Its first review passed 0 of 6 (0%); `
- + 'even the top of its 95% range is 39%. Only 6 first reviews. Asked of the project owner.')
- expect(text.split('\n')).toContain('3. Start docs work on a subscription model instead of the grid: about 3 hours to ship a week. 3 of 3 docs tasks started on '
- + 'the grid stepped up after their grid attempt failed. Only 3 tasks.')
})
it('proposes nothing from too few first reviews, shipped changes or grid step-ups', () => {
@@ -195,7 +139,6 @@ describe('the nightly routing reflection', () => {
week.rows = week.rows.filter(r => !['p5', 'u6', 'd2', 'd2b'].includes(r.assignmentId))
week.groups = week.groups.map(g => g.id === 'g-u1' ? group('g-u1', [{ assignmentId: 'u1', firstReview: true }]) : g)
expect(routingReflection(week)!.proposals).toEqual([])
- expect(routingReflection(week)!.text).toContain('No change proposed.')
})
it('proposes no cheaper model on fewer than five first reviews, however well they went', () => {
@@ -215,11 +158,10 @@ describe('the nightly routing reflection', () => {
})
it('says only aggregates: no task title, assignment or conversation id', () => {
- const text = routingReflection(seededWeek())!.text
- for (const secret of ['Secret plan', 'g-p1', 'p1', 'u3']) expect(text).not.toContain(secret)
- })
-
- it('says when the newest attempts were all it could read', () => {
- expect(routingReflection({ ...seededWeek(), truncated: true })!.text).toContain('Only the newest 3000 attempts were read.')
+ for (const proposal of routingReflection(seededWeek())!.proposals) {
+ const question = proposalQuestion(proposal).question
+ expect(question.split('\n')).toHaveLength(2)
+ for (const secret of ['Secret plan', 'g-p1', 'p1', 'u3']) expect(question).not.toContain(secret)
+ }
})
})
diff --git a/backend/src/superTabs/costs/reflection.ts b/backend/src/superTabs/costs/reflection.ts
index cec455255..7c2e468dc 100644
--- a/backend/src/superTabs/costs/reflection.ts
+++ b/backend/src/superTabs/costs/reflection.ts
@@ -1,23 +1,13 @@
-/**
- * The nightly routing reflection (Scholes's roadmap, piece 2, 2026-10-30): what one project's routed work cost and how it went
- * in the last day and the last 7 days, the waste, and at most three routing changes for its owner to approve, each with its
- * expected saving. Pure: lib/routingReflection.ts reads the attempts (costs/read.ts costRows), failed starts and evidence, and
- * posts the text. Counts only: engines, models, efforts and kinds of work, never a task's title, a message or an id.
- * Nothing here changes routing: a proposal is advice asked of the owner, with one fixed Routing line (routingLine). A Yes has
- * the coordinator write that line into the project's instructions; the reflection reads them, and a proposal whose line is
- * there is reported with how the last 7 days ran, never asked again (project instructions, phase 4, decision 21). A line
- * approved lately that is not there yet is listed as approved and not in the instructions yet (final review).
- */
+/** Nightly routing proposals for owner Decisions, never a project-chat report. Counts only: engines, models, efforts and
+ * kinds of work, never task titles, messages or IDs. A Yes has the coordinator write the fixed routingLine into the
+ * instructions; proposals whose lines are already there or whose owners answered lately are not asked again. */
import { stableId, type Conversation } from '../model.js'
-import type { ModelKindEvidence } from '../evidence/store.js'
import { attemptCost } from './cost.js'
-import { MAX_ROWS, summarizeCosts, type CostRow, type CostSummaryRow } from './read.js'
+import { summarizeCosts, type CostRow, type CostSummaryRow } from './read.js'
const MINUTE = 60_000, DAY = 24 * 60 * MINUTE
/** At most this many proposals a night (brief). */
export const MAX_PROPOSALS = 3
-/** Rows of the per-model table the report lists; the rest are counted. */
-const MAX_LINES = 12
/** A cheaper model is proposed only with this much evidence on each side: fewer first reviews or shipped changes say too
* little (review of #2276: 3 and 2 gave confident proposals from a handful of tasks). */
const MIN_FIRST_REVIEWS = 5
@@ -50,18 +40,11 @@ export interface ReflectionInput {
groups: Conversation[]
/** The project's failed starts in the last 7 days. */
failedStarts: FailedStart[]
- /** The project's evidence since the week holding 7 days ago: rework rounds and fixes after merge. */
- evidence: ModelKindEvidence[]
- /** More attempts than one read takes: only the newest were read. */
- truncated: boolean
/** Titles of proposals the owner answered in the last REASK_DAYS days: not proposed again until then. */
decided?: readonly string[]
/** The project's instructions as they read now (the built-in template for a project without a document yet): a proposal
- * whose routing line (routingLine) is in them is reported, not proposed. */
+ * whose routing line (routingLine) is in them is not proposed. */
document: string
- /** The routing lines of proposals the owner said Yes to in the last REASK_DAYS days (approvedLine), with the local date of
- * the answer, and the coordinator's name when the owner left the answer to it: one not in [document] is listed. */
- approved?: ReadonlyArray<{ line: string; on: string; leftTo?: string }>
}
type RanOn = Pick<CostSummaryRow, 'engine' | 'model' | 'effort'>
@@ -69,25 +52,13 @@ type RanOn = Pick<CostSummaryRow, 'engine' | 'model' | 'effort'>
export type Change = { kind: 'cheaper'; taskKind: string; from: RanOn; to: RanOn } | { kind: 'stop'; taskKind: string; from: RanOn }
| { kind: 'grid'; taskKind: string }
/** A routing change for the owner: what, why, what it would save a week in dollars (API-equivalent) and hours to ship,
- * whether its evidence is strong enough to recommend Yes, and how many tasks of its kind the last 7 days ran the old way. */
-export interface Proposal { title: string; why: string; saving: string; dollars: number; hours: number; strong: boolean; change: Change
- ran: { tasks: number; on: string } }
-export interface Reflection {
- /** The report with no note on its proposals. */
- text: string
- proposals: Proposal[]
- /** The report with a note after each proposal: whether it was asked, and if not, why (store.ts postReflection). */
- render(note: (p: Proposal) => string | null): string
-}
+ * and whether its evidence is strong enough to recommend Yes. */
+export interface Proposal { title: string; why: string; saving: string; dollars: number; hours: number; strong: boolean; change: Change }
+export interface Reflection { proposals: Proposal[] }
/** Days an answered proposal is not proposed again. */
export const REASK_DAYS = 14
/** The zone of a project that set no digest: the owner's time zone is stored nowhere; the digest's is the one a project sets. */
export const DEFAULT_TIME_ZONE = 'Asia/Ho_Chi_Minh'
-/** The local date of [at] in [timeZone], or in the default zone for none or one this runtime does not know. */
-export function localDate(at: number, timeZone: string = DEFAULT_TIME_ZONE): string {
- const format = (zone: string) => new Intl.DateTimeFormat('en-CA', { timeZone: zone, year: 'numeric', month: '2-digit', day: '2-digit' }).format(at)
- try { return format(timeZone) } catch { return format(DEFAULT_TIME_ZONE) }
-}
const plural = (n: number, one: string, many = `${one}s`) => `${n} ${n === 1 ? one : many}`
const usd = (n: number) => `$${n.toFixed(2)}`
const percent = (rate: number) => `${Math.round(rate * 100)}%`
@@ -98,7 +69,7 @@ const capital = (text: string) => `${text.slice(0, 1).toUpperCase()}${text.slice
const cut = (text: string, max: number) => text.length <= max ? text : `${text.slice(0, max - 1)}…`
/** The one fixed Routing line a proposal carries: what a Yes has the coordinator write into the instructions, and what the
- * reflection looks for there. Models as the report names them. */
+ * reflection looks for there. */
export function routingLine(p: Pick<Proposal, 'change'>): string {
const c = p.change, kind = capital(c.taskKind)
return c.kind === 'cheaper' ? `${kind} work runs on ${ranOn(c.to)} instead of ${ranOn(c.from)}.`
@@ -108,14 +79,6 @@ export function routingLine(p: Pick<Proposal, 'change'>): string {
const passed = (r: CostSummaryRow) => `${r.firstReviewPassed} of ${r.firstReviews} (${percent(r.firstReviewPassRate!)})`
const atApi = 'a week at API prices'
-function line(r: CostSummaryRow): string {
- return `- ${ranOn(r)}, ${r.kind}: ${plural(r.attempts, 'attempt')}, ${r.shipped} shipped`
- + (r.medianMinutesToShip !== null ? `, median ${Math.round(r.medianMinutesToShip)} min to ship` : '')
- + (r.costPerShipped !== null ? `, ${usd(r.costPerShipped)} per shipped change` : '')
- + (r.firstReviews ? `, first review passed ${r.firstReviewPassed} of ${r.firstReviews} (${percent(r.firstReviewPassRate!)})` : ', no first review')
- + (r.stepUps ? `, ${plural(r.stepUps, 'step-up')}` : '')
-}
-
/** Per kind, the tasks whose grid attempt failed and stepped up: how many, and the grid attempts' hours and cost. Also how
* many tasks of the kind started on the grid. A step up is the attempt that carries it (decisions from before project instructions); the grid attempts
* before it in its task are what it replaced. */
@@ -167,7 +130,7 @@ function proposals(rows: CostSummaryRow[], grid: ReturnType<typeof gridStepUps>)
+ `${usd(dear.costPerShipped!)} per shipped change at API prices.`
+ (strong ? '' : ` Small samples (${dear.firstReviews} and ${cheap.firstReviews} first reviews): the difference may be chance.`),
saving: `about ${usd(dollars)} ${atApi}`, dollars, hours: 0, strong,
- change: { kind: 'cheaper', taskKind: kind, from: pick(dear), to: pick(cheap) }, ran: { tasks: dear.attempts, on: ranOn(dear) } })
+ change: { kind: 'cheaper', taskKind: kind, from: pick(dear), to: pick(cheap) } })
}
// A model whose first review clearly fails more often than it passes on a kind of work: even the top of its range is low.
for (const r of known) {
@@ -179,79 +142,37 @@ function proposals(rows: CostSummaryRow[], grid: ReturnType<typeof gridStepUps>)
+ (strong ? '' : ` Only ${r.firstReviews} first reviews.`),
saving: dollars ? `about ${usd(dollars)} ${atApi} of attempts that failed their first review`
: `${plural(failed, 'attempt')} a week that failed their first review`, dollars, hours: 0, strong,
- change: { kind: 'stop', taskKind: r.kind, from: pick(r) }, ran: { tasks: r.attempts, on: ranOn(r) } })
+ change: { kind: 'stop', taskKind: r.kind, from: pick(r) } })
}
// A kind whose tasks keep stepping up from the grid.
for (const [kind, k] of grid) {
if (kind === 'unspecified' || k.stepped < MIN_GRID_STEP_UPS || k.stepped * 2 < k.tasks) continue
const hours = tenths(k.hours), dollars = cents(k.dollars), strong = k.tasks >= STRONG_SAMPLE
found.push({ title: `Start ${kind} work on a subscription model instead of the grid`,
- why: `${k.stepped} of ${k.tasks} ${kind} tasks started on the grid stepped up after their grid attempt failed.`
+ why: `${k.stepped} of ${k.tasks} ${kind} tasks needed another model after their grid attempt failed.`
+ (strong ? '' : ` Only ${k.tasks} tasks.`),
saving: `about ${dollars ? `${usd(dollars)} at API prices and ` : ''}${hours} hours to ship a week`, dollars, hours, strong,
- change: { kind: 'grid', taskKind: kind }, ran: { tasks: k.tasks, on: 'the grid' } })
+ change: { kind: 'grid', taskKind: kind } })
}
return found.sort((a, b) => b.dollars - a.dollars || b.hours - a.hours)
}
const pick = (r: RanOn): RanOn => ({ engine: r.engine, model: r.model, effort: r.effort })
-function waste(input: ReflectionInput, grid: ReturnType<typeof gridStepUps>, stepUps: number): string {
- const failed = new Map<string, number>()
- for (const f of input.failedStarts) failed.set(ranOn(f), (failed.get(ranOn(f)) ?? 0) + 1)
- const stepped = [...grid].filter(([, k]) => k.stepped)
- const fromGrid = stepped.reduce((n, [, k]) => n + k.stepped, 0)
- /** A count over the evidence, with each model and kind that has some, the most first. */
- const byModel = (count: (m: ModelKindEvidence) => number, one: string, many?: string) => {
- const has = input.evidence.filter(m => count(m) > 0).sort((x, y) => count(y) - count(x))
- const total = has.reduce((n, m) => n + count(m), 0)
- return total ? [`${plural(total, one, many)} (${has.map(m => `${m.engine} ${m.model} ${m.kind}: ${count(m)}`).join(', ')})`] : []
- }
- const parts = [
- ...input.failedStarts.length ? [`${plural(input.failedStarts.length, 'failed start')} (${[...failed].sort((a, b) => b[1] - a[1])
- .map(([on, n]) => `${on}: ${n}`).join(', ')})`] : [],
- ...fromGrid ? [`${plural(fromGrid, 'step-up')} after a failed grid attempt (${stepped.map(([kind, k]) => `${kind}: ${k.stepped}, ${tenths(k.hours)} hours`).join('; ')})`] : [],
- ...stepUps > fromGrid ? [`${plural(stepUps - fromGrid, 'other step-up')}`] : [],
- ...byModel(m => m.reworkRounds, 'rework round'),
- ...byModel(m => m.fixesAfterMerge, 'fix after merge', 'fixes after merge'),
- ]
- return `Waste in the last 7 days: ${parts.length ? parts.join(', ') : 'none'}.`
-}
-
-/** The night's report, or null when the last day had no attempt and no failed start: then nothing is posted. At most
- * MAX_PROPOSALS proposals, the largest saving first, leaving out those the owner answered lately; one whose line the
- * instructions hold is reported with how the last 7 days ran instead. A reworded line is not recognized: proposed again. */
+/** At most MAX_PROPOSALS, the largest saving first, excluding recent answers and lines already in the instructions.
+ * Null when the last day had no attempt or failed start. */
export function routingReflection(input: ReflectionInput): Reflection | null {
const since = input.now - DAY
const dayRows = input.rows.filter(r => r.run.startedAt >= since)
if (!dayRows.length && !input.failedStarts.some(f => f.at >= since)) return null
- const week = summarizeCosts(input.rows, input.groups), day = summarizeCosts(dayRows, input.groups)
+ const week = summarizeCosts(input.rows, input.groups)
const grid = gridStepUps(input.rows, input.now)
const decided = new Set(input.decided ?? [])
const all = proposals(week.rows, grid)
- const written = all.filter(p => input.document.includes(routingLine(p)))
- // A Yes writes nothing by itself: the coordinator writes the line, and until it is in the instructions each report says so.
- const pending = (input.approved ?? []).filter((a, i, all) => !input.document.includes(a.line) && all.findIndex(b => b.line === a.line) === i)
- const proposed = all.filter(p => !written.includes(p) && !decided.has(p.title)).slice(0, MAX_PROPOSALS)
- const table = [...week.rows].sort((a, b) => b.attempts - a.attempts)
- const render = (note: (p: Proposal) => string | null) => [
- 'Routing reflection: what routed work cost and how it went. Counts only; routing changes only as the project owner approves.',
- `Last day: ${plural(day.totals.attempts, 'attempt')}, ${day.totals.shipped} shipped. Last 7 days: ${plural(week.totals.attempts, 'attempt')}, ${week.totals.shipped} shipped.`,
- 'By engine, model and effort, per kind of work (last 7 days; cost is what the tokens would cost on the API):',
- ...table.slice(0, MAX_LINES).map(line),
- ...table.length > MAX_LINES ? [`- and ${table.length - MAX_LINES} more`] : [],
- waste(input, grid, week.totals.stepUps),
- ...written.map(p => `In the instructions: ${routingLine(p)} The last 7 days ran ${plural(p.ran.tasks, `${p.change.taskKind} task`)} on ${p.ran.on}.`),
- ...pending.map(a => `Approved ${a.on}${a.leftTo ? ` (left to the ${a.leftTo})` : ''}; not in the instructions yet: ${a.line}`),
- ...proposed.length ? ['Proposed changes (advice: no rule or setting changes by itself):',
- ...proposed.map((p, i) => { const n = note(p); return `${i + 1}. ${p.title}: ${p.saving}. ${p.why}${n ? ` ${n}` : ''}` })]
- : ['No change proposed.'],
- ...input.truncated ? [`Only the newest ${MAX_ROWS} attempts were read.`] : [],
- ].join('\n')
- return { text: render(() => null), proposals: proposed, render }
+ return { proposals: all.filter(p => !input.document.includes(routingLine(p)) && !decided.has(p.title)).slice(0, MAX_PROPOSALS) }
}
-/** The id of a project's report for one local date: one a night, so a second pass or a second worker writes none. */
+/** Stable nightly Decision operation ID; also recognizes report rows posted before reports stopped. */
export const reflectionMessageId = (tabId: string, projectId: string, on: string): string => stableId('routing-reflection', tabId, projectId, on)
/** The answer that approves a proposal. */
@@ -262,18 +183,11 @@ const YES_WRITES = 'If you say yes, the coordinator writes into Routing, naming
* for an answer or answered lately is told by it (proposalTitle). It says plainly what Yes does: the coordinator writes the
* proposal's line into the Routing section of the instructions, naming the owner as who asked (project instructions, phase 4,
* decision 21; the template's PM section says so). */
-export const proposalQuestion = (p: Proposal) => ({ question: `${p.title}? Expected saving: ${p.saving}. ${p.why}`,
+export const proposalQuestion = (p: Proposal) => ({ question: `${p.title}?\nWhy: ${p.why} This could save ${p.saving}.`,
options: ['Yes: the coordinator writes this into the Routing section of the instructions', 'No: keep routing as it is',
'Not yet: ask again when there is more evidence'],
recommended: p.strong ? APPROVE : 2,
// A decision's unblocks is at most 300 characters (decisions.ts decisionChoices); only a model slug that long cuts the line.
unblocks: cut(`${YES_WRITES}${routingLine(p)}`, 300) })
-/** The routing line an asked proposal's unblocks names (proposalQuestion), or null for a decision asked before it named one.
- * A line cut to fit loses its ellipsis, so its start is still found in the instructions. */
-export function approvedLine(unblocks: string): string | null {
- if (!unblocks.startsWith(YES_WRITES)) return null
- const line = unblocks.slice(YES_WRITES.length)
- return line.endsWith('…') ? line.slice(0, -1) : line
-}
/** The proposal a decision asked, by the title its question starts with. */
export const proposalTitle = (question: string): string => question.slice(0, Math.max(0, question.indexOf('?')))
diff --git a/backend/src/superTabs/notices.test.ts b/backend/src/superTabs/notices.test.ts
index 3e61575d3..9a53f352c 100644
--- a/backend/src/superTabs/notices.test.ts
+++ b/backend/src/superTabs/notices.test.ts
@@ -87,7 +87,7 @@ describe('the notices the Swarm writes into a chat', () => {
// the Shipped line, now the task's own line (taskLine.ts; Quiet Swarm, Dee, 2026-11-02), people; a machine's removal ending,
// no longer said in any chat (anh Dee, 11-04, #2742), nobody; a task's own row under its requester's name, now only a person's
// own message (anh Dee, 11-04, 'bot messages', #2925), people.
- const retired = ['idle-stop', 'project-report', 'ship', 'machine-left', 'task-line']
+ const retired = ['idle-stop', 'project-report', 'ship', 'machine-left', 'task-line', 'routing-reflection']
expect(Object.keys(NOTICE_READERS).filter(kind => !written.has(kind) && !retired.includes(kind))).toEqual([])
expect(retired.filter(kind => written.has(kind))).toEqual([])
})
diff --git a/backend/src/superTabs/store.ts b/backend/src/superTabs/store.ts
index 213a72b15..f0b1c826c 100644
--- a/backend/src/superTabs/store.ts
+++ b/backend/src/superTabs/store.ts
@@ -89,7 +89,7 @@ import { endedHolds, HOLDING, startedHolds, type WorkHold } from '../github/hold
import { accrueLent, addLent, lentAgents, lentProjects } from './leaderboard/lent.js'
import { mayDrive, TERMINAL_ANSWER_MS, TERMINAL_EXPIRY_MS, TERMINAL_PICKUP_MS, TERMINAL_RECHECK_MS, type TerminalEffect } from './terminal.js'
import { costRows, failedStarts, summarizeCosts } from './costs/read.js'
-import { APPROVE, approvedLine, localDate, MAX_PROPOSALS, proposalQuestion, proposalTitle, REASK_DAYS, reflectionMessageId, routingReflection, type Proposal, type ReflectionInput } from './costs/reflection.js'
+import { MAX_PROPOSALS, proposalQuestion, proposalTitle, REASK_DAYS, reflectionMessageId, routingReflection, type ReflectionInput } from './costs/reflection.js'
import { readEvidence, recordEvidence } from './evidence/store.js'
import { closeAssignment, closeUnassignedStart, LIST_SIZE, newestConversations, newestDecisions, readDecision, recentDecisions, recordAttempts, timedDecisionOf, writeRouteEffects } from './routeDecisions.js'
import { answeredAgents, dropHeldTags, listMentions, markDone, mentionsWaiting, openTags, readMention, resolveByAnswer, resolveByReply, resolveWhenWorked, writeMentions } from './mentions.js'
@@ -4197,18 +4197,12 @@ export class SuperTabStore {
}
/**
- * Posts a project's nightly routing reflection for local date `on` (lib/routingReflection.ts) in its project chat under its
- * Lead's name, and asks each new proposal of the project owner as a decision from the Lead; nothing changes routing by
- * itself. Counts only the attempts and failed starts of the project's own task groups, never a private or Buddy chat, the
- * Leads conversation or another project. Reads first, outside any transaction (the week's attempts, failed starts, their
- * groups and evidence); then one short transaction re-checks, reads the project's reflection decisions and writes. The
- * report's id is the project's and date's, so a second pass or worker writes nothing ('already'). 'empty': no attempt and
- * no failed start in the last day. 'skipped': no such Swarm or project, a paused Swarm, or a project with no Lead.
- * Decisions (reviews of #2276): at most MAX_PROPOSALS open, each expiring after 7 days unanswered (every due night, a
- * report or not); one answered in the last REASK_DAYS days is not asked again. A Yes has the coordinator write the
- * proposal's line into the instructions (project instructions, phase 4, decision 21): the report reads the project's
- * document, the template at revision 0, and says how the last 7 days ran for a line it holds, and lists a line approved in
- * those days that it does not hold yet (final review). No approved list, no reminder.
+ * Asks the owner about fresh nightly routing proposals, never posting a project-chat report. The costs command keeps
+ * the numbers available to the coordinator. Existing report rows stay and their nightly IDs remain recognized.
+ * Reads only the project's task attempts, then rechecks the owner and open Decisions in a short transaction.
+ * 'posted' means fresh Decisions were written; 'empty' means nothing new to ask. At most MAX_PROPOSALS stay open;
+ * unanswered proposals expire after 7 days, and answers suppress another ask for REASK_DAYS. Instructions containing
+ * the proposed routing line suppress it too. No routing rule changes until the coordinator writes an approved line.
*/
async postReflection(tabId: string, projectId: string, on: string): Promise<'posted' | 'already' | 'empty' | 'skipped'> {
const id = reflectionMessageId(tabId, projectId, on)
@@ -4216,7 +4210,7 @@ export class SuperTabStore {
const read = await this.db.superTab.findUnique({ where: { id: tabId }, select: SWARM_ROW })
if (!read || read.deletedAt) return 'skipped'
const now = this.now(), since = now - 7 * DAY_MS
- const { rows, truncated } = await costRows(this.db, tabId, since)
+ const { rows } = await costRows(this.db, tabId, since)
const failed = await failedStarts(this.db, tabId, since)
const seen = (await loadRow(this.db, requireParts(read), { groupIds: [...new Set([...rows, ...failed].map(r => r.conversationId))] })).state
const at = reflectionPlace(seen, projectId)
@@ -4226,7 +4220,7 @@ export class SuperTabStore {
const revision = at.project.instructionsRevision ?? 0
const stored = revision ? await this.db.superTabInstructions.findUnique({ where: { id: instructionsId(tabId, projectId) }, select: { text: true } }) : null
const input: ReflectionInput = { now, rows: rows.filter(r => ids.has(r.conversationId)), groups: seen.conversations.filter(g => ids.has(g.id)),
- failedStarts: failed.filter(f => ids.has(f.conversationId)), evidence: (await readEvidence(this.db, tabId, since, [projectId])).models, truncated,
+ failedStarts: failed.filter(f => ids.has(f.conversationId)),
document: stored?.text ?? INSTRUCTIONS_TEMPLATE }
// A quiet night still expires the proposals left unanswered (delta review of #2276).
if (!routingReflection(input)) { await expireReflections(this.db, tabId, projectId, now); return 'empty' }
@@ -4243,34 +4237,19 @@ export class SuperTabStore {
await expireReflections(tx, tabId, projectId, now)
const asked = await reflectionDecisions(tx, tabId, projectId)
const open = asked.filter(d => !d.answeredAt)
- const zone = project.digest?.timeZone
const recent = asked.filter(d => d.answeredAt && d.answeredAt.getTime() >= now - REASK_DAYS * DAY_MS)
const decided = recent.map(d => proposalTitle(d.question))
- // A Yes writes nothing here: the report lists its line until the coordinator has written it into the instructions.
- const approved = recent.flatMap(d => {
- const line = d.choice === APPROVE ? approvedLine(d.unblocks) : null
- return line ? [{ line, on: localDate(d.answeredAt!.getTime(), zone), ...d.delegated ? { leftTo: coordinatorNameOf(project) } : {} }] : []
- })
- const report = routingReflection({ ...input, decided, approved })!
+ const report = routingReflection({ ...input, decided })!
const owner = state.people.find(p => p.id === project.ownerId)
const reads = !!owner && canReadGroup(state, { kind: 'person', userId: owner.id, email: owner.email }, chat)
- const waitingSince = new Map(open.map(d => [proposalTitle(d.question), localDate(d.createdAt.getTime(), zone)]))
+ const waiting = new Set(open.map(d => proposalTitle(d.question)))
const held = reads ? await tx.superTabDecision.count({ where: { tabId, personId: project.ownerId, askerAgentId: lead.id,
OR: [{ answeredAt: null }, { answeredAt: { isSet: false } }] } }) : 0
const room = Math.max(0, Math.min(MAX_PROPOSALS - open.length, MAX_OPEN_DECISIONS - held))
- const fresh = reads ? report.proposals.filter(p => !waitingSince.has(p.title)).slice(0, room) : []
- // Says of each proposal only what happened to it: asked, waiting, or why not (review of #2276).
- const note = (p: Proposal) => waitingSince.has(p.title) ? `Waiting for the owner's answer since ${waitingSince.get(p.title)}.`
- : fresh.includes(p) ? 'Asked of the project owner.' : !reads ? 'Not asked: the project owner does not read this chat.'
- : held + fresh.length >= MAX_OPEN_DECISIONS ? 'Not asked: the owner has too many open decisions.'
- : `Not asked: at most ${MAX_PROPOSALS} proposals wait for the owner's answer at once.`
- const author: Author = { kind: 'agent', id: lead.id, name: lead.name, displayName: agentDisplayName(state.agents, lead.id), membershipGeneration: lead.membershipGeneration }
- const rows: MessageDraft[] = [{ id, conversationId: chat.id, replyTo: null, recipients: [], author, text: report.render(note),
- source: { kind: 'routing-reflection', projectId, on } }]
- const sequences = numberMessages(state, rows, now)
- await this.save(tx, loaded, state, clock.next())
- await this.writeMessages(tx, tabId, state, rows, sequences)
- if (fresh.length) await writeDecisions(tx, tabId, askDecisions(state, chat, project.ownerId, fresh.map(proposalQuestion),
+ const fresh = reads ? report.proposals.filter(p => !waiting.has(p.title)).slice(0, room) : []
+ if (!fresh.length) return 'empty'
+ await this.save(tx, loaded, state, clock.next(), [{ kind: 'head', key: 'head' }])
+ await writeDecisions(tx, tabId, askDecisions(state, chat, project.ownerId, fresh.map(proposalQuestion),
{ operationId: id, askerAgentId: lead.id, assignmentId: null, kind: REFLECTION_DECISION }), now)
return 'posted'
})
diff --git a/docs/plans/2026-11-05-routing-reflection-plan.md b/docs/plans/2026-11-05-routing-reflection-plan.md
new file mode 100644
index 000000000..eb39abbbb
--- /dev/null
+++ b/docs/plans/2026-11-05-routing-reflection-plan.md
@@ -0,0 +1,53 @@
+# Quiet routing reflection implementation plan
+
+**Goal:** Stop nightly routing reports appearing in project chats while retaining costs and owner routing Decisions.
+
+**Architecture:** Keep the existing nightly calculation and decision lifecycle. Remove its project-chat message write; only fresh proposals create Decisions. The costs command continues reading the existing attempt ledger unchanged.
+
+**Tech Stack:** TypeScript, Prisma/MongoDB, Vitest.
+
+**Spec:** Swarm task #2997, “No routing report in project chats”, requested by Dee.
+
+**Execution:** Implement task-by-task, test first, following the project's Engineer workflow.
+
+## Global Constraints
+
+- No new settings; existing messages remain untouched.
+- Keep owner/audience checks, proposal limits, expiry, deduplication and approval semantics.
+- No production reads or writes, schema changes, new collections or changed query shapes. Remove the unused evidence read with the report renderer.
+- Run affected tests locally; backend unit and Super Tab integration suites are automatic PR CI evidence.
+- PR body starts with “Requested by dee through Harness.”
+
+## Design
+
+- Growth: removes new report messages; existing attempt and Decision retention remains unchanged.
+- Concurrency: existing short Swarm transaction and stable nightly Decision operation ID; retain serialization of Decision writes.
+- Read cost: existing bounded weekly evidence reads, no added queries.
+- Scale: unchanged 5,000 candidate and three-proposal bounds; existing warnings remain.
+- Compatibility: no client or command contract changes; existing reflection-message IDs remain recognized.
+- Migration + rollback: no migration; revert restores future posts without touching historical rows.
+- Audience: existing server-side project-owner access checks remain.
+- Identity: existing project Lead asks Decisions; no machine-owner login.
+- Failure: existing named worker warnings and bounded nightly retries remain.
+- Platforms: server-only; no platform-specific behavior.
+
+## Review Focus
+
+- Both proposal and no-proposal nights must leave all project-chat messages unchanged.
+- Multiple Swarms and private projects must retain isolation.
+- Owner Decisions must contain a short change and reason, not internal report tables.
+- Concurrent passes must not duplicate Decisions; expiry and recent-answer suppression must remain.
+- Existing posted reports and the costs command must remain readable.
+
+## Task 1: Quiet nightly reflection
+
+**Files:** `backend/src/superTabs/store.ts` (`postReflection`), `backend/src/superTabs/costs/reflection.ts`, `backend/src/lib/routingReflection.ts`, and their tests, including `backend/integration/superTabReflection.test.ts`.
+
+**Interfaces:** Preserve `postReflection` result keys and stable Decision IDs. `posted` counts fresh Decision writes, not chat posts; `empty` means no fresh proposal.
+
+- [x] Commit regression tests for no chat writes with/without proposals, owner wording, preserved costs/history, and lifecycle/isolation.
+- [x] Run focused tests and record the expected failures before production changes.
+- [x] Remove report-message creation and rendering from the store path; retain owner Decision writes and serialization.
+- [x] Simplify proposal wording to one change line and one reason line while retaining recognizable titles and approval routing lines.
+- [x] Run focused unit/integration tests and backend typecheck; record red/green evidence in the PR.
+- [ ] Mark ready for automatic component CI; request an independent review step, then report evidence through task status.