The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.
TypeScript 3d-modelingagent-frameworkagent-harnessai-agentscadclaude-codecodexcoding-agentsdeveloper-toolsesp32game-developmentmcp-servermulti-agentopen-hardwareopencodeopenharnesspcbroboticsterminaltmux
Merge batch 11-04 b1930: #3007 #3001 #3019 (#3020)
0xken15445815b09ca0
diff --git a/backend/integration/superTabBuddySessions.test.ts b/backend/integration/superTabBuddySessions.test.tsindex e5c05f34b..824354568 100644--- a/backend/integration/superTabBuddySessions.test.ts+++ b/backend/integration/superTabBuddySessions.test.ts@@ -219,7 +219,7 @@ describe('a Buddy thread session', () => {const harnessId = (await f.store.read(peer, f.tab)).state.agents.find(x => x.id === agentId)!.harnessIdconst before = await f.store.messages(peer, f.tab, f.chat, {})await expect(f.reply(b, 'No room', { newSession: true })).rejects.toMatchObject({ code: 'MACHINE_FULL',- message: 'Peer Mac is already running as many agents as it allows. End a session there, or try again when one finishes.' })+ message: 'Peer Mac is set to run at most 2 agents and runs 2. Its owner can change that in its settings, or try again when one finishes.' })const state = (await f.store.read(peer, f.tab)).stateexpect(state.conversations.find(c => c.id === f.chat)!.buddySessions!.map(e => e.thread)).toEqual([a])expect(state.machines!.find(m => m.machineId === 'peer-mac')!.maxAgents).toBe(2)diff --git a/backend/src/superTabs/commands.ts b/backend/src/superTabs/commands.tsindex 76dd95122..5b00172bf 100644--- a/backend/src/superTabs/commands.ts+++ b/backend/src/superTabs/commands.ts@@ -1,6 +1,6 @@import { applyPushOrigin, failPushUnknown, failTeamWritten, hasStep, mainAccountMembers, markTeamWritten, requireNameable, requireTeamWrittenAccount, requireTeamWrittenMachine, TEAM_WRITTEN_REASON, TEAM_WRITTEN_STOP, teamWrittenTask } from './teamWritten.js'import type { PushOrigin } from '../github/pushOrigin.js'-import { admitMachineStart, defaultProfileId, storedProfileId, BUDDY_WINDOWS_REFUSAL, refusesWindowsBuddy, buddyHomeFolder, buddyProjectsOn, SWARM_BUDDY_CAPABILITY, cleanupOverdue, dropIdleBuddyMachines, machineLimitReached, MACHINE_LIMIT_MESSAGE, NOT_SWARM_MACHINE, downSince, expireRemoval, expireRemovals, failTeamWork, machineContribution, machineOffline, machineSlots, contributedEngines, ownedMachine, productManagerOn, projectFolder, projectsOnMachine, requireTeamWork, teamMachineFor, teamAccountPool, teamPoolsOf, teamStartRequest, teamWorkCheck, movedToTeamAccount, type ContributedMachine } from './machines.js'+import { admitByRoom, admitMachineStart, defaultProfileId, storedProfileId, BUDDY_WINDOWS_REFUSAL, refusesWindowsBuddy, buddyHomeFolder, buddyProjectsOn, SWARM_BUDDY_CAPABILITY, cleanupOverdue, dropIdleBuddyMachines, machineLimitReached, MACHINE_LIMIT_MESSAGE, NOT_SWARM_MACHINE, downSince, expireRemoval, expireRemovals, failTeamWork, machineContribution, machineOffline, machineSlots, contributedEngines, ownedMachine, productManagerOn, projectFolder, projectsOnMachine, requireTeamWork, teamMachineFor, teamAccountPool, teamPoolsOf, teamStartRequest, teamWorkCheck, movedToTeamAccount, type ContributedMachine } from './machines.js'import { agentDisplayName, assignAgentDisplayNames } from './agentNames.js'import { keepUsernames, setUsername, swarmUsernames, usernameOf } from './usernames.js'import { setName, writtenNameOf } from './personNames.js'@@ -156,6 +156,8 @@ const MAX_REMOVED_MACHINES = 64export const commandSchema = z.discriminatedUnion('type', [z.object({ type: z.literal('machine.contribute'), machineId: id, name: title, cwd: path, scope: z.literal('swarm').optional(), engines: z.array(swarmEngine).min(1).max(2), maxAgents,+ // Its Agents setting (machines.ts agentLimit): an older client leaves it out.+ agents: z.enum(['memory', 'atMost']).optional(),ifMissing: z.literal(true).optional(), teamAccountOnly: z.literal(true).optional(),// With ifMissing: not if its person took it out (Swarm removedMachines). A Personal Swarm's own sync sends it (#2181).unlessRemoved: z.literal(true).optional() }).strict(),@@ -1214,9 +1216,7 @@ function applyCommandOnce(original: SuperTabState, actor: Actor, input: Command,}// Sessions have no count of their own (Scholes, 11-02): idle ones close themselves. The one limit is the machine's, which// its owner set to protect it (Lead, 11-02), said in words a person can act on. A Buddy-only machine has none (below).- if (!pool.buddyOnly && machineSlots(state, pool) >= pool.maxAgents) {- fail('MACHINE_FULL', `${pool.name} is already running as many agents as it allows. End a session there, or try again when one finishes.`)- }+ if (!pool.buddyOnly) admitByRoom(state, pool, machineSlots(state, pool), BUDDY_REQUIREMENTS.memoryBytes)const contributor = person(state, main.contributorId)// In the one Buddy chat a session works in one project its person reads, or, with none named, as the main Buddy does// (spec: The model). A project's own chat has its project.@@ -1314,7 +1314,7 @@ function applyCommandOnce(original: SuperTabState, actor: Actor, input: Command,try { admitted = admitMachineStart(state, request, context.now, undefined, context.teamMachines, context.otherWorkspaceMachines) } catch (error) {if (!(error instanceof SuperTabError && error.status < 500)) throw errorreturn error.code === 'MACHINE_PAUSED' ? `${machine} is paused` : error.code === 'MACHINE_OFFLINE' ? `${machine} is offline`- : error.code === 'MACHINE_FULL' ? `${machine} already runs as many agents as it takes`+ : error.code === 'MACHINE_FULL' ? (error.details?.reason === 'memory' ? `${machine} is out of free memory` : `${machine} already runs as many agents as it takes`): error.code === 'MACHINE_REMOVED' ? `${machine} is being removed from the Swarm` : `${machine} cannot take me right now`}if (!admitted) return 'my machine is no longer in the Swarm'@@ -1896,19 +1896,27 @@ function applyCommandOnce(original: SuperTabState, actor: Actor, input: Command,// So is their choice of their own account for teammates' work (machine.ownAccountWork), Desktop's join re-save// included; not over a removal, so a Mac removed and added again starts Team-account-only.const ownAccountChoice = state.machines.find(m => m.machineId === command.machineId && !m.buddyOnly && !m.removal)?.ownAccountChoice+ // Its Agents setting: a client that does not send one (older than #2860) keeps the stored one while the count is the same.+ const previous = state.machines.find(m => m.machineId === command.machineId && !m.buddyOnly)+ const agents = command.agents ?? (previous?.agents && previous.maxAgents === command.maxAgents ? previous.agents : undefined)state.machines = state.machines.filter(m => m.machineId !== command.machineId)state.machineCleanups = state.machineCleanups?.filter(c => c.machineId !== command.machineId)if (!state.machineCleanups?.length) delete state.machineCleanupsconst host = context.contribution?.hostMachineIdstate.machines.push({ machineId: command.machineId, name: mac?.name ?? context.contribution?.machineName ?? command.name, cwd: command.cwd, engines: [...new Set(command.engines)],- maxAgents: mac?.maxAgents ?? command.maxAgents, ...(command.scope ? { scope: command.scope } : {}),+ maxAgents: mac?.maxAgents ?? command.maxAgents, ...(mac ? (mac.agents ? { agents: mac.agents } : {}) : agents ? { agents } : {}),+ ...(command.scope ? { scope: command.scope } : {}),contributorId: actor.userId, membershipGeneration: contributor.generation, generation: operationId, paused,...(host ? { hostMachineId: host } : {}), ...(teamAccountOnly ? { teamAccountOnly: true as const } : {}), ...(projects && !host ? { projects } : {}),...(ownAccountChoice && !host ? { ownAccountChoice } : {}),...(othersToldWeek != null ? { othersToldWeek } : {}) })// A Mac given other slots or a new name gives its team account the same, and its pause; its own folder, engines and grant stay.const added = state.machines.at(-1)!- for (const team of teamPoolsOf(state, added)) if (!team.removal) Object.assign(team, { name: added.name, maxAgents: added.maxAgents, paused: added.paused })+ for (const team of teamPoolsOf(state, added)) if (!team.removal) {+ Object.assign(team, { name: added.name, maxAgents: added.maxAgents, paused: added.paused })+ if (added.agents) team.agents = added.agents+ else delete team.agents+ }break}case 'machine.pause': {diff --git a/backend/src/superTabs/machines.ts b/backend/src/superTabs/machines.tsindex a61ff6748..54cf49c38 100644--- a/backend/src/superTabs/machines.ts+++ b/backend/src/superTabs/machines.ts@@ -20,7 +20,12 @@ export interface ContributedMachine {cwd: stringscope?: 'swarm'engines: string[]+ /** The count its owner allows when [agents] is `atMost` (and for an older row, see agentLimit). Kept as a number every+ * client reads; with As memory allows it is only what the owner last saw, never a limit. */maxAgents: number+ /** Its Agents setting (Dee, 11-04: "remove the limit"): `memory`, as many as free memory allows, checked at each start;+ * `atMost`, its owner's own [maxAgents]. Absent on a row saved by an older client: agentLimit reads it. */+ agents?: 'memory' | 'atMost'paused: boolean/** `sessionKind` is how the reporting machine is signed in, stamped from its session by the store, never taken from* the report: `team` for a Mac's team account (lib/teamSession.ts), `computer` for any other. Absent on older readings.@@ -829,9 +834,87 @@ export function admitMachineStart(state: SuperTabState, request: AgentStartReque}// A Buddy-only machine is recorded with its Buddy's start, before its host ever read the Swarm: the start waits for it.if (machineOffline(pool, now) && !(pool.buddyOnly && !pool.observation)) fail('MACHINE_OFFLINE', 'Wait for a fresh reading from the contributed machine.')- if (!start && slots(pool) >= pool.maxAgents) fail('MACHINE_FULL', 'This machine has reached its contributed agent limit.')+ if (!start) admitByRoom(state, pool, slots(pool), request.requirements.memoryBytes)// A new start only: a grid model runs without the engine's subscription, and one already asked for is settled by its host.const unusable = !start && !local ? engineUnusable(pool, request.engine, now) : nullif (unusable) fail('ENGINE_UNAVAILABLE', unusable)return pool}++/** What one agent needs to start, as the CLI asks (capacity.ts START_REQUIREMENTS.memoryBytes). */+export const AGENT_MEMORY_BYTES = 2 * 1024 ** 3+/** The most agents any machine runs, as machine.contribute's own bound: a ceiling on growth, never a setting. */+export const MACHINE_AGENTS_CEILING = 1024+/** What joining suggested before 11-04 (CLI prepareContribution, Desktop superTabSuggestedAgentSlots): one agent per 4 GiB,+ * 1 to 16. Joining stored it as a limit; Mac 3 refused Milo's engineer at 16 (Dee, 11-04). */+export const joinSuggestion = (memoryTotalBytes: number): number =>+ Math.max(1, Math.min(16, Math.floor(memoryTotalBytes / (2 * AGENT_MEMORY_BYTES))))+/** The count its owner holds a machine to, or null when it takes agents as memory allows. A Buddy-only machine counts its+ * own Buddies (commands.ts). An older row says neither (a compatibility rule: review it once Desktop and the CLI send+ * `agents`, and remove it when no stored row lacks it): a stored number that is the old join suggestion (16, or the one its+ * measured memory gave) was never the owner's own limit; any other number is. Desktop's old As memory allows stored the+ * machine's maximum (one per 2 GiB), which free memory reaches first anyway. */+export function agentLimit(pool: Pick<ContributedMachine, 'maxAgents' | 'agents' | 'buddyOnly' | 'observation'>): number | null {+ if (pool.buddyOnly || pool.agents === 'atMost') return pool.maxAgents+ if (pool.agents === 'memory') return null+ if (pool.maxAgents === 16) return null+ const total = pool.observation?.resources?.memoryTotalBytes+ // Under 4 GiB the suggestion was its floor of 1, not a figure from memory.+ if (typeof total === 'number' && total >= 2 * AGENT_MEMORY_BYTES && pool.maxAgents === joinSuggestion(total)) return null+ return pool.maxAgents+}++/** Its measured free memory, from its last reading; null with none. */+export function freeMemoryBytes(pool: Pick<ContributedMachine, 'observation'>): number | null {+ const r = pool.observation?.resources+ return r && typeof r.memoryTotalBytes === 'number' && typeof r.memoryUsedBytes === 'number' ? Math.max(0, r.memoryTotalBytes - r.memoryUsedBytes) : null+}++/** A Mac and its Harness Team account: one machine, one memory (slotCounter counts their slots together too). */+function memoryPair(state: SuperTabState, pool: ContributedMachine): ContributedMachine[] {+ const mac = pool.hostMachineId ? state.machines?.find(m => m.machineId === pool.hostMachineId && m.contributorId === pool.contributorId) : pool+ return [...new Set([pool, mac, mac && teamPoolOf(state, mac)])].filter((m): m is ContributedMachine => !!m)+}++/** What a new start can use on [pool] as memory allows: the least free memory its Mac and Team account last read, less what+ * starts admitted there and not yet launched will take, which no reading shows yet (review of #3001: three 2 GiB starts+ * were all admitted on 5 GiB free). Null with no reading. */+export function startableMemoryBytes(state: SuperTabState, pool: ContributedMachine): number | null {+ const pair = memoryPair(state, pool)+ const readings = pair.map(freeMemoryBytes).filter((free): free is number => free !== null)+ if (!readings.length) return null+ const machines = new Set(pair.map(m => m.machineId))+ let held = 0+ for (const start of state.starts ?? []) {+ if (start.error || start.harnessId || !machines.has(start.request.machineId)) continue+ held += start.request.requirements?.memoryBytes || AGENT_MEMORY_BYTES+ }+ return Math.max(0, Math.min(...readings) - held)+}++/** How many more agents it takes now, for the route and Desktop: its owner's count less what runs, or, as memory allows,+ * one per AGENT_MEMORY_BYTES of startable memory (none with no reading). Its host checks free memory and disk again at+ * each launch. */+export function availableSlots(state: SuperTabState, pool: ContributedMachine, used: number): number {+ const limit = agentLimit(pool)+ const ceiling = Math.max(0, (limit ?? MACHINE_AGENTS_CEILING) - used)+ if (limit !== null) return ceiling+ const free = startableMemoryBytes(state, pool)+ return free === null ? 0 : Math.min(ceiling, Math.floor(free / AGENT_MEMORY_BYTES))+}++const gib = (bytes: number) => `${Math.round(bytes / 1024 ** 3 * 10) / 10} GB`+/** A new start's room on [pool], said in words its owner can act on: its own count, or its free memory. */+export function admitByRoom(state: SuperTabState, pool: ContributedMachine, used: number, memoryBytes: number) {+ const limit = agentLimit(pool)+ if (limit !== null) {+ if (used >= limit) fail('MACHINE_FULL', `${pool.name} is set to run at most ${limit} agent${limit === 1 ? '' : 's'} and runs ${used}. Its owner can change that in its settings, or try again when one finishes.`)+ return+ }+ if (used >= MACHINE_AGENTS_CEILING) fail('MACHINE_FULL', `${pool.name} already runs ${used} agents, the most any machine runs.`)+ const free = startableMemoryBytes(state, pool)+ if (free !== null && free < memoryBytes) {+ fail('MACHINE_FULL', `${pool.name} is out of free memory: ${gib(free)} free, and an agent needs ${gib(memoryBytes)}. Try again when an agent there finishes, or pick another machine.`,+ 409, { reason: 'memory' })+ }+}diff --git a/backend/src/superTabs/read/views.ts b/backend/src/superTabs/read/views.tsindex 28559849b..63f386370 100644--- a/backend/src/superTabs/read/views.ts+++ b/backend/src/superTabs/read/views.ts@@ -7,7 +7,7 @@ import { suspendedEngines } from '../engineHealth.js'import { leadChoices } from '../leadStart.js'import { isBuddyChat, isBuddyReader } from '../buddies.js'import { agentForBuddy, agentWithoutMachine, groupForBuddy, machineHiddenFrom, seenMachineIds, visibleState, withSeenRequirement, type Actor, type Agent, type Conversation, type SuperTabState } from '../model.js'-import { slotCounter, machineTeamWork, type ContributedMachine } from '../machines.js'+import { availableSlots, slotCounter, machineTeamWork, type ContributedMachine } from '../machines.js'import * as store from '../parts/index.js'import type { Part, SwarmParts } from '../parts/index.js'import { canReadGroup } from '../projects.js'@@ -137,7 +137,7 @@ function derive(state: SuperTabState, visible: SuperTabState, actor: Actor, now:const refusal = machineTeamWork(state, stored, now, extras.teamMachines, extras.otherWorkspaceMachines ?? [])const version = extras.machineVersions?.[pool.machineId]const suspended = suspendedEngines(stored, now)- return [pool.machineId, { availableSlots: Math.max(0, pool.maxAgents - slots(stored)),+ return [pool.machineId, { availableSlots: availableSlots(state, stored, slots(stored)),teamWork: refusal ? { allowed: false as const, ...refusal } : { allowed: true as const }, ...(version ? { clientVersion: version } : {}),...(Object.keys(suspended).length ? { suspendedEngines: suspended } : {}), ...(leads[pool.machineId] ? { lead: leads[pool.machineId] } : {}) }] as const}diff --git a/backend/src/superTabs/store.ts b/backend/src/superTabs/store.tsindex 3da2a6a55..28ce9ccc1 100644--- a/backend/src/superTabs/store.ts+++ b/backend/src/superTabs/store.ts@@ -1,6 +1,6 @@import { refusalLogFields, type DeclaredClient } from './refusalLog.js'import { writeReactionActivity, writeReplyActivity } from './activity.js'-import { accountMachineName, expireRemovals, followAccountName, LASTING_TEAM_REFUSALS, machineContribution, machineObservationSchema, slotCounter, machineTeamWork, preparableStart, readingOnly, refusableStart, recordAgentPresence, renamedBy, removalPreview, startPreparing, storedObservation, swarmGridRow, teamAccountRequired, teamWorkRefusal, ownAccountWork, ownAccountNote, type MachineObservation } from './machines.js'+import { accountMachineName, availableSlots, expireRemovals, followAccountName, LASTING_TEAM_REFUSALS, machineContribution, machineObservationSchema, slotCounter, machineTeamWork, preparableStart, readingOnly, refusableStart, recordAgentPresence, renamedBy, removalPreview, startPreparing, storedObservation, swarmGridRow, teamAccountRequired, teamWorkRefusal, ownAccountWork, ownAccountNote, type MachineObservation } from './machines.js'import { shownNameOf } from './personNames.js'import { readingClearsSuspensions, suspendedEngines } from './engineHealth.js'import { BUDDY_OWN_MACHINE_REFUSAL } from './buddyContract.js'@@ -1309,7 +1309,7 @@ export class SuperTabStore {// out for, and what Desktop can show on the machine.const refusal = machineTeamWork(state, stored, now, teamMachines, otherWorkspace)const suspended = suspendedEngines(stored, now)- return { ...pool, availableSlots: Math.max(0, pool.maxAgents - slots(stored)),+ return { ...pool, availableSlots: availableSlots(state, stored, slots(stored)),teamWork: refusal ? { allowed: false as const, ...refusal } : { allowed: true as const,// Its owner lets teammates' work run in their own account (machines.ts ownAccountWork): the route says so....(ownAccountWork(stored) ? { ownAccount: ownAccountNote(stored, shownNameOf(state, stored.contributorId)) } : {}) },diff --git a/backend/src/superTabs/teamRouting.test.ts b/backend/src/superTabs/teamRouting.test.tsindex 2d5f7736b..295cf696d 100644--- a/backend/src/superTabs/teamRouting.test.ts+++ b/backend/src/superTabs/teamRouting.test.ts@@ -1,6 +1,6 @@import { afterEach, beforeEach, describe, expect, it } from 'vitest'import { applyCommand, commandSchema, type Command, type CommandContext } from './commands.js'-import { admitMachineStart, machineObservationSchema, machineTeamWork, machineSlots, observedPrivacy, PRIVACY_FRESH_MS, removalPreview, teamAccountRequired, teamWorkRefusal, type ContributedMachine, type MachinePrivacy } from './machines.js'+import { admitMachineStart, availableSlots, machineObservationSchema, machineTeamWork, machineSlots, observedPrivacy, PRIVACY_FRESH_MS, removalPreview, teamAccountRequired, teamWorkRefusal, type ContributedMachine, type MachinePrivacy } from './machines.js'import { createSuperTab, SuperTabError, visibleState, type Actor, type AgentConfiguration, type AgentStart, type SuperTabState } from './model.js'import type { RouteDecision } from './routeRecord.js'@@ -722,3 +722,134 @@ describe('a machine added as its person joined', () => {})const commandSchemaOf = (command: unknown) => commandSchema.safeParse(command)++describe('machines take agents as memory allows (Dee, 11-04: "remove the limit")', () => {+ const GiB = 1024 ** 3+ /** A Mac with [freeGiB] free of [totalGiB], read just now. */+ const measured = (f: ReturnType<typeof swarm>, machineId: string, totalGiB: number, freeGiB: number) => {+ const m = f.machine(machineId)+ m.observation = { ...m.observation!, resources: { ...m.observation!.resources, memoryTotalBytes: totalGiB * GiB, memoryUsedBytes: (totalGiB - freeGiB) * GiB } }+ }+ const twoGiB = { memoryBytes: 2 * GiB, diskBytes: 0, maxCpuPercent: 100, tools: [], devices: [], quotaRequired: false }+ const start = (f: ReturnType<typeof swarm>, group: string, machineId = 'owner-mac') =>+ f.run({ type: 'agent.start', request: { ...f.request(group, machineId), requirements: twoGiB } }, owner, f.pooled(machineId))+ const fill = (f: ReturnType<typeof swarm>, machineId: string, count: number) => {+ for (let i = 0; i < count; i++) f.member(`running-${machineId}-${i}`, machineId, f.machine(machineId).contributorId)+ }++ it('joining stores no fixed count: a Mac added with the suggested 16 takes a 17th start while memory allows', () => {+ const f = swarm(), group = f.group()+ f.state.machines = f.state.machines!.filter(m => m.machineId !== 'owner-mac')+ f.run({ type: 'machine.contribute', machineId: 'owner-mac', name: 'Owner Mac', cwd: '/swarm', scope: 'swarm', engines: ['codex'], maxAgents: 16, agents: 'memory' } as Command,+ owner, { contribution: { machineOwnerId: 'owner', contributorId: 'owner', machineName: 'Owner Mac' } } as never)+ f.machine('owner-mac').observation = reading('computer')+ measured(f, 'owner-mac', 128, 60)+ expect(f.machine('owner-mac').agents).toBe('memory')+ fill(f, 'owner-mac', 16)+ expect(refusal(() => start(f, group))).toBeNull()+ })++ it('an older row holding the old join suggestion reads as memory allows', () => {+ const f = swarm(), group = f.group()+ // Mac 3 (Dee, 11-04): 16 stored when it joined, 16 running, plenty free.+ Object.assign(f.machine('owner-mac'), { maxAgents: 16 })+ measured(f, 'owner-mac', 128, 60)+ fill(f, 'owner-mac', 16)+ expect(refusal(() => start(f, group))).toBeNull()+ // A 32 GiB Mac whose join suggested 8 (one per 4 GiB): the same.+ const g = swarm(), other = g.group()+ Object.assign(g.machine('owner-mac'), { maxAgents: 8 })+ measured(g, 'owner-mac', 32, 20)+ fill(g, 'owner-mac', 8)+ expect(refusal(() => start(g, other))).toBeNull()+ })++ it('keeps an owner\'s own At most N, from a new client or an older one', () => {+ const f = swarm(), group = f.group()+ Object.assign(f.machine('owner-mac'), { maxAgents: 16, agents: 'atMost' })+ measured(f, 'owner-mac', 128, 60)+ fill(f, 'owner-mac', 16)+ const full = refusal(() => start(f, group))+ expect(full?.code).toBe('MACHINE_FULL')+ expect(full?.message).toBe('Owner Mac is set to run at most 16 agents and runs 16. Its owner can change that in its settings, or try again when one finishes.')+ // An older client's At most 3 on a 128 GiB Mac (suggestion 16, maximum 64) is the owner's.+ const g = swarm(), other = g.group()+ Object.assign(g.machine('owner-mac'), { maxAgents: 3 })+ measured(g, 'owner-mac', 128, 60)+ fill(g, 'owner-mac', 3)+ expect(refusal(() => start(g, other))?.code).toBe('MACHINE_FULL')+ })++ it('a start refused for memory says so plainly', () => {+ const f = swarm(), group = f.group()+ Object.assign(f.machine('owner-mac'), { maxAgents: 16 })+ measured(f, 'owner-mac', 128, 1.5)+ const short = refusal(() => start(f, group))+ expect(short?.code).toBe('MACHINE_FULL')+ expect(short?.message).toBe('Owner Mac is out of free memory: 1.5 GB free, and an agent needs 2 GB. Try again when an agent there finishes, or pick another machine.')+ expect(short?.message).not.toMatch(/limit/)+ })++ it('the route counts a machine\'s room by its free memory', () => {+ const f = swarm()+ Object.assign(f.machine('owner-mac'), { maxAgents: 16 })+ measured(f, 'owner-mac', 128, 21)+ fill(f, 'owner-mac', 16)+ // 16 running and 21 GiB free: ten more of 2 GiB, not the zero a count of 16 left.+ expect(availableSlots(f.state, f.machine('owner-mac'), machineSlots(f.state, f.machine('owner-mac')))).toBe(10)+ Object.assign(f.machine('owner-mac'), { agents: 'atMost', maxAgents: 18 })+ expect(availableSlots(f.state, f.machine('owner-mac'), machineSlots(f.state, f.machine('owner-mac')))).toBe(2)+ })++ // Review of #3001: the reading shows no memory for starts not launched yet.+ it('counts the memory of starts admitted and not yet launched', () => {+ const f = swarm(), group = f.group()+ Object.assign(f.machine('owner-mac'), { maxAgents: 16 })+ measured(f, 'owner-mac', 128, 5)+ const room = () => availableSlots(f.state, f.machine('owner-mac'), machineSlots(f.state, f.machine('owner-mac')))+ expect(room()).toBe(2)+ expect(refusal(() => start(f, group))).toBeNull()+ expect(room()).toBe(1)+ expect(refusal(() => start(f, group))).toBeNull()+ expect(room()).toBe(0)+ const third = refusal(() => start(f, group))+ expect(third?.message).toBe('Owner Mac is out of free memory: 1 GB free, and an agent needs 2 GB. Try again when an agent there finishes, or pick another machine.')+ expect(third?.details).toEqual({ reason: 'memory' })+ })++ it('counts a Mac and its Team account as one memory', () => {+ const f = swarm(), group = f.group()+ Object.assign(f.machine('mate-team'), { hostMachineId: 'mate-mac', name: 'MacBook Pro', cwd: '/team-swarm', maxAgents: 16 })+ Object.assign(f.machine('mate-mac'), { maxAgents: 16 })+ measured(f, 'mate-mac', 128, 3)+ measured(f, 'mate-team', 128, 3)+ const room = (id: string) => availableSlots(f.state, f.machine(id), machineSlots(f.state, f.machine(id)))+ expect([room('mate-mac'), room('mate-team')]).toEqual([1, 1])+ f.member('mine', 'mate-mac', 'mate', f.mine)+ expect(refusal(() => f.run({ type: 'agent.start', request: { ...f.request(group, 'mate-mac'), requirements: twoGiB } }, owner, f.pooled('mate-mac')))).toBeNull()+ // The one start holds the 2 GiB both readings show.+ expect([room('mate-mac'), room('mate-team')]).toEqual([0, 0])+ })++ it('keeps an owner\'s At most when an older client saves the machine again with the same count', () => {+ const f = swarm()+ const save = (extra: Record<string, unknown>) => f.run({ type: 'machine.contribute', machineId: 'owner-mac', name: 'Owner Mac', cwd: '/swarm', scope: 'swarm',+ engines: ['codex'], maxAgents: 16, ...extra } as Command, owner, { contribution: { machineOwnerId: 'owner', contributorId: 'owner', machineName: 'Owner Mac' } } as never)+ save({ agents: 'atMost' })+ save({})+ expect(f.machine('owner-mac').agents).toBe('atMost')+ // A new count from an older client is its own: the setting is read from the number again.+ f.run({ type: 'machine.contribute', machineId: 'owner-mac', name: 'Owner Mac', cwd: '/swarm', scope: 'swarm', engines: ['codex'], maxAgents: 6 } as Command,+ owner, { contribution: { machineOwnerId: 'owner', contributorId: 'owner', machineName: 'Owner Mac' } } as never)+ expect(f.machine('owner-mac').agents).toBeUndefined()+ })++ it('a Team account follows its Mac\'s setting', () => {+ const f = swarm()+ Object.assign(f.machine('mate-team'), { hostMachineId: 'mate-mac', name: 'MacBook Pro', cwd: '/team-swarm', maxAgents: 16, agents: 'atMost' })+ f.run({ type: 'machine.contribute', machineId: 'mate-mac', name: 'MacBook Pro', cwd: '/swarm', scope: 'swarm', engines: ['codex'], maxAgents: 16, agents: 'memory' } as Command,+ mate, { contribution: { machineOwnerId: 'mate', contributorId: 'mate', machineName: 'MacBook Pro' } } as never)+ expect(f.machine('mate-mac').agents).toBe('memory')+ expect(f.machine('mate-team').agents).toBe('memory')+ })+})diff --git a/cli/src/superTabs/host.spec.ts b/cli/src/superTabs/host.spec.tsindex 07871a8a3..1042f839f 100644--- a/cli/src/superTabs/host.spec.ts+++ b/cli/src/superTabs/host.spec.ts@@ -4525,13 +4525,20 @@ describe('closing the sessions the backend ends', () => {it('keeps a person\'s own session when a saved scrub names its id, and still removes the worktree\'s transcripts', async () => {const home = realpathSync(mkdtempSync(join(tmpdir(), 'buddy-home-'))), claude = join(home, '.claude'), worktrees = join(home, 'harnesses', 'worktrees')const slug = (cwd: string) => cwd.replace(/[^A-Za-z0-9]/g, '-')- const folder = join(worktrees, 'openharness', 'buddy-session'), mine = join(home, 'code', 'mine')+ // One worktree each, the second named `-2` as a start's is when its name is taken (start.ts): two agents never share a+ // folder, and two scrubs of one folder race each other's removal (a transcript read after the other scrub removed it fails+ // the scrub, which waits out its retry).+ const folders = { history: join(worktrees, 'openharness', 'buddy-session'), transcript: join(worktrees, 'openharness', 'buddy-session-2') }+ const mine = join(home, 'code', 'mine')// Theirs by a history line; theirs by a transcript in their own folder's transcript folder, with no history line.const byHistory = '11111111-1111-4111-8111-111111111111', byTranscript = '22222222-2222-4222-8222-222222222222'try {- mkdirSync(folder, { recursive: true })- for (const sub of ['paste-cache', 'todos', 'debug', `projects/${slug(folder)}`, `projects/${slug(mine)}`]) mkdirSync(join(claude, sub), { recursive: true })- writeFileSync(join(claude, 'projects', slug(folder), 'own.jsonl'), JSON.stringify({ cwd: folder }))+ for (const folder of Object.values(folders)) {+ mkdirSync(folder, { recursive: true })+ mkdirSync(join(claude, 'projects', slug(folder)), { recursive: true })+ writeFileSync(join(claude, 'projects', slug(folder), 'own.jsonl'), JSON.stringify({ cwd: folder }))+ }+ for (const sub of ['paste-cache', 'todos', 'debug', `projects/${slug(mine)}`]) mkdirSync(join(claude, sub), { recursive: true })writeFileSync(join(claude, 'projects', slug(mine), `${byTranscript}.jsonl`), JSON.stringify({ cwd: mine }))writeFileSync(join(claude, 'history.jsonl'), `${JSON.stringify({ sessionId: byHistory, project: mine, pastedContents: { 1: { contentHash: 'theirpaste' } } })}\n`)writeFileSync(join(claude, 'paste-cache', 'theirpaste.txt'), 'secret')@@ -4543,14 +4550,14 @@ describe('closing the sessions the backend ends', () => {const rows = ['history', 'transcript'].map(id => ended({ agent: { id, harnessId: `h-${id}`, machineId: 'machine-1' } }))const t = lifeFixture(rows, { claudeHome: home, worktrees, startReceipt: () => 'super-buddy-session' })t.host.stop()- const scrubs = { history: { agentId: 'history', sessionId: byHistory, cwd: folder, engine: 'claude', codexHome: null },- transcript: { agentId: 'transcript', sessionId: byTranscript, cwd: folder, engine: 'claude', codexHome: null } }+ const scrubs = { history: { agentId: 'history', sessionId: byHistory, cwd: folders.history, engine: 'claude', codexHome: null },+ transcript: { agentId: 'transcript', sessionId: byTranscript, cwd: folders.transcript, engine: 'claude', codexHome: null } }writeFileSync(hostJournalFile(t.dir, 'machine-1'), JSON.stringify({ ...t.saved(), scrubs }))t.core.agents.byAgent = vi.fn(() => undefined as never)const again = t.open({ claudeHome: home, worktrees, startReceipt: () => 'super-buddy-session' })await again.tick()await vi.waitFor(() => expect(t.saved().scrubs).toBeUndefined())- expect(existsSync(join(claude, 'projects', slug(folder)))).toBe(false)+ for (const folder of Object.values(folders)) expect(existsSync(join(claude, 'projects', slug(folder)))).toBe(false)expect(readFileSync(join(claude, 'history.jsonl'), 'utf8')).toContain(byHistory)expect(existsSync(join(claude, 'paste-cache', 'theirpaste.txt'))).toBe(true)expect(existsSync(join(claude, 'projects', slug(mine), `${byTranscript}.jsonl`))).toBe(true)diff --git a/cli/src/superTabs/machines.spec.ts b/cli/src/superTabs/machines.spec.tsindex e7f382751..70c2793e2 100644--- a/cli/src/superTabs/machines.spec.ts+++ b/cli/src/superTabs/machines.spec.ts@@ -929,6 +929,62 @@ describe('contributed machine executor', () => {expect(machinePosts(f).at(-1)).not.toHaveProperty('preparingStartIds')})+ // Issue #2989 and River on task #2890: a start dispatched here that never got an agent, failed by the Swarm ("did not+ // start it within 3 minutes"), held its memory on this machine until the ledger's launch deadline. Its host reads the+ // verdict once the start leaves its pending list, and ends the reservation at once.+ describe('that the backend fails before any agent joins', () => {+ const startPath = '/api/super-tabs/hosts/host/starts/tab/start'+ const settle = () => new Promise(resolve => setTimeout(resolve, 0))+ function failedEarly(status: () => unknown, ended: () => Promise<Record<string, unknown>> = async () => ({ ok: true, reservationId: 'creation', returned: true })) {+ const f = fixture()+ f.start.mockResolvedValue({ ...pendingCreation, state: 'unconfirmed' } as never)+ const original = f.backend.getMockImplementation()!+ f.backend.mockImplementation(async (core, method, path, body) => method === 'GET' && path === startPath ? status() : original(core, method, path, body))+ const usage = f.request.getMockImplementation()!+ f.request.mockImplementation(async (type, ...rest) => type === 'super_tabs' && (rest[0] as Record<string, unknown>)?.action === 'endCapacity'+ ? ended() : usage(type, ...rest as []))+ const ends = () => f.request.mock.calls.filter(c => c[0] === 'super_tabs' && (c[1] as Record<string, unknown>)?.action === 'endCapacity')+ return { f, ends, fail: () => { f.pool.starts = [] } }+ }++ it('ends its reservation on this machine once, as soon as the Swarm says it failed', async () => {+ const { f, ends, fail } = failedEarly(() => ({ error: 'Mac Auto did not start it within 3 minutes' }))+ await f.machine.tick()+ f.advance(); await f.machine.tick(); await settle()+ expect(ends()).toHaveLength(0)+ fail(); f.advance(); await f.machine.tick(); await settle()+ await vi.waitFor(() => expect(ends()).toEqual([['super_tabs', { action: 'endCapacity', creationId: 'creation' }, 10_000]]))+ expect(f.log.mock.calls.flat().join('\n')).toContain('Freed the memory of start start, which the Swarm failed: Mac Auto did not start it within 3 minutes')+ f.advance(); await f.machine.tick(); await settle()+ expect(ends()).toHaveLength(1)+ })++ it('leaves one that settled or got its agent to the ledger, and asks a few times when it cannot tell', async () => {+ for (const status of [{ harnessId: 'joined' }, { error: null }]) {+ const { f, ends, fail } = failedEarly(() => status)+ await f.machine.tick(); fail(); f.advance(); await f.machine.tick(); await settle()+ f.advance(); await f.machine.tick(); await settle()+ expect(ends()).toHaveLength(0)+ expect(f.backend.mock.calls.filter(c => c[2] === startPath)).toHaveLength(1)+ }+ const { f, ends, fail } = failedEarly(() => { throw new Error('backend down') })+ await f.machine.tick(); fail()+ for (let pass = 0; pass < 7; pass++) { f.advance(); await f.machine.tick(); await settle() }+ expect(f.backend.mock.calls.filter(c => c[2] === startPath)).toHaveLength(5)+ expect(ends()).toHaveLength(0)+ expect(f.log.mock.calls.flat().join('\n')).toContain('Could not free the memory of failed start start: backend down')+ })++ it('lets an older daemon that cannot end it keep the row for its launch deadline', async () => {+ const { f, ends, fail } = failedEarly(() => ({ error: 'Mac Auto did not start it within 3 minutes' }), async () => ({ error: 'UNKNOWN_ACTION' }))+ await f.machine.tick(); fail(); f.advance(); await f.machine.tick(); await settle()+ await vi.waitFor(() => expect(ends()).toHaveLength(1))+ f.advance(); await f.machine.tick(); await settle()+ expect(ends()).toHaveLength(1)+ expect(f.log.mock.calls.flat().join('\n')).toContain('Could not free the memory of failed start start: UNKNOWN_ACTION')+ })+ })+// Mac Auto's Harness Team account, 2026-10-29: the backend failed each start at 3 minutes while its Codex never recorded// its session, and the agent the machine launched for it stayed, idle, with no Swarm agent behind it.describe('that the backend then fails', () => {diff --git a/cli/src/superTabs/machines.ts b/cli/src/superTabs/machines.tsindex 5c144e899..705dd10bd 100644--- a/cli/src/superTabs/machines.ts+++ b/cli/src/superTabs/machines.ts@@ -237,6 +237,9 @@ export class ContributedMachines {private readonly heldSince = new Map<string, number>()/** Starts this host launched whose engine has not yet recorded its session: the agent each created, by start id. */private readonly unconfirmed = new Map<string, Unconfirmed>()+ /** Starts dispatched here whose creation has no agent yet, by start id ([failedEarly]). In memory only: after a restart the+ * ledger's launch deadline frees them (reservations.ts LAUNCH_DEADLINE_MS). */+ private readonly agentless = new Map<string, Agentless>()/** Removals cleaned and acknowledged by this host, and the one in progress per removal. */private readonly cleaned = new Set<string>()private readonly cleaning = new Map<string, Promise<void>>()@@ -497,6 +500,7 @@ export class ContributedMachines {for (const id of this.heldSince.keys()) if (!pendingIds.has(id)) this.heldSince.delete(id)this.loadUnconfirmed()for (const [id, launch] of this.unconfirmed) if (!pendingIds.has(id) && !launch.checking) void this.abandoned(id, launch)+ for (const [id, launch] of this.agentless) if (!pendingIds.has(id) && !launch.checking) void this.failedEarly(id, launch, where)for (const tab of this.published.keys()) if (!pools.some(row => row.tabId === tab)) this.published.delete(tab)for (const tab of this.lastSent.keys()) if (!pools.some(row => row.tabId === tab)) this.lastSent.delete(tab)for (const tab of this.readingFailures.keys()) if (!pools.some(row => row.tabId === tab)) this.readingFailures.delete(tab)@@ -723,6 +727,10 @@ export class ContributedMachines {if ((result.error === 'START_PENDING' || typeof result.refused === 'string') && identifier(result.harnessId) && identifier(result.creationId)) {this.remember(String(start.id), { path, harnessId: result.harnessId, creationId: result.creationId, tries: 0, checking: false })} else this.forget(String(start.id))+ // Dispatched here with a creation and no agent yet: should the Swarm fail it, [failedEarly] frees its memory at once.+ if (result.ok !== true && !identifier(result.harnessId) && identifier(result.creationId)) {+ this.agentless.set(String(start.id), { path, creationId: result.creationId, tries: this.agentless.get(String(start.id))?.tries ?? 0, checking: false })+ } else this.agentless.delete(String(start.id))const held = result.error === 'START_PENDING' && result.state === 'pending' ? object(result.held) : {}if (typeof held.detail !== 'string' || held.service === 'install') this.heldSince.delete(start.id)else {@@ -969,6 +977,30 @@ export class ContributedMachines {} finally { launch.checking = false }}+ /**+ * A start dispatched here with no agent that the backend no longer lists as pending: when the backend says it failed (its+ * machine did not start it within 3 minutes, say) and no agent joined, this machine's ledger ends its reservation at once+ * rather than at its launch deadline (River on task #2890, issue #2989). A status the backend cannot give now is asked+ * again on later passes, a few times; a daemon that cannot end it keeps the row until that deadline.+ */+ private async failedEarly(startId: string, launch: Agentless, where: Parameters<MachineDeps['open']>[0]): Promise<void> {+ launch.checking = true+ try {+ const status = object(await this.deps.backend(this.core, 'GET', launch.path))+ if (identifier(status.harnessId) || !status.error) { this.agentless.delete(startId); return }+ this.agentless.delete(startId)+ const local = await this.deps.open(where)+ try {+ const ended = object(await local.request('super_tabs', { action: 'endCapacity', creationId: launch.creationId }, 10_000))+ if (ended.ok === true) this.log(`Freed the memory of start ${startId}, which the Swarm failed: ${String(status.error)}`)+ else this.log(`Could not free the memory of failed start ${startId}: ${String(ended.error ?? 'no answer')}; its launch deadline frees it.`)+ } finally { local.close() }+ } catch (error) {+ if (++launch.tries >= ABANDONED_TRIES) this.agentless.delete(startId)+ this.log(`Could not free the memory of failed start ${startId}: ${error instanceof Error ? error.message : String(error)}`)+ } finally { launch.checking = false }+ }+/** Each running session's token counts by model, from the daemon's last usage reading (never a transcript read): the* Swarm prices each attempt by them (backend superTabs/costs). Counts only; the reading's folders and pull requests stay* here. A session with none read yet is left out. */@@ -1317,6 +1349,8 @@ const SWARM_GRID = /^swarm-[0-9a-f]{12}$//** A start this host launched whose engine has not recorded its session yet (ContributedMachines `abandoned`). */interface Unconfirmed { path: string; harnessId: string; creationId: string; tries: number; checking: boolean }+/** A start dispatched here whose creation has no agent yet (ContributedMachines `failedEarly`). */+interface Agentless { path: string; creationId: string; tries: number; checking: boolean }/** How many passes ask about an abandoned start before it is let go. */const ABANDONED_TRIES = 5diff --git a/cli/src/superTabs/reservations.spec.ts b/cli/src/superTabs/reservations.spec.tsindex 529521a10..a4a2eb6cc 100644--- a/cli/src/superTabs/reservations.spec.ts+++ b/cli/src/superTabs/reservations.spec.ts@@ -8,7 +8,7 @@ import { fakeCore } from '../testing/fakeCore.js'import { builtinSqlite } from '../lib/sqliteBuiltin.js'import type { CreationObservation } from '../lib/creationObservation.js'import type { MachineResources } from '../lib/machineResources.js'-import { CapacityReservations, START_DEADLINE_MS } from './reservations.js'+import { CapacityReservations, LAUNCH_DEADLINE_MS, START_DEADLINE_MS } from './reservations.js'const books: CapacityReservations[] = [], folders: string[] = []afterEach(() => { for (const book of books.splice(0)) book.stop(); for (const folder of folders.splice(0)) rmSync(folder, { recursive: true, force: true }) })@@ -29,7 +29,7 @@ function setup() {vi.mocked(core.agents.byAgent).mockImplementation(key => live.has(key) ? { agentId: key, registeredAt: live.get(key)! } as never : undefined)const deps = { resources: vi.fn(async () => resources),disk: vi.fn(async (cwd: string) => ({ freeBytes: 100, volume: cwd === '/second-volume' ? 'second' : 'first', path: cwd })),- sqlite: () => builtinSqlite() as never, now: () => clock.now, agentMemory: vi.fn(async () => new Map(measured)) }+ sqlite: () => builtinSqlite() as never, now: () => clock.now, agentMemory: vi.fn(async () => new Map(measured)), log: vi.fn() }const open = () => { const book = new CapacityReservations(core, 'machine-1', deps); books.push(book); return book }/** The start reserved as `n` launched as harness `agent-n` at this moment. */const launch = (n: number) => { native.set(id(n), { creationId: id(n), state: 'created', agentId: `agent-${n}` }); live.set(`agent-${n}`, clock.now) }@@ -49,7 +49,8 @@ it("counts a start's memory only during its start window, so settled agents do nfree(f, 7.85 * GiB)for (const n of [4, 5, 6]) { expect(await book.reserve(agentInput(n))).toMatchObject({ ok: true }); f.launch(n) }expect(await book.reserve(agentInput(7))).toMatchObject({ error: 'CAPACITY_UNAVAILABLE',- detail: 'Not enough free memory: 1.8 GiB free, 2 GiB needed. 6 GiB more is set aside for agents still starting.' })+ detail: 'Not enough free memory: 1.8 GiB free, 2 GiB needed. 6 GiB more is set aside for 3 agents still starting, the oldest '+ + 'reserved under a minute ago (start receipts fixture-creation-4, fixture-creation-5, fixture-creation-6).' })// The first of the three new agents settles: its own use is in free memory now, and its window no longer holds.f.clock.now += 5 * MINUTE - 1expect(await book.reserve(agentInput(7))).toMatchObject({ error: 'CAPACITY_UNAVAILABLE' })@@ -390,14 +391,76 @@ it('times the deadline from the last run that confirmed the reservation, so a reexpect(await book.reserve(agentInput(2))).toMatchObject({ ok: true })})-it.each(['pending', 'unconfirmed', null] as const)('keeps holding a %s creation past the deadline, as it may still launch', async state => {+// Issue #2989 (trunghieu, 2026-11-04): four dispatched 2 GiB starts that never got an agent, each failed by the Swarm ("did not+// start it within 3 minutes"), read pending or unconfirmed on the machine for good, and every later start there was refused+// with "8 GiB more is set aside for agents still starting", across a restart. Only a dispatched start has a creation, and+// the Swarm fails every dispatched start with no agent by its own start deadline.+it.each(['pending', 'unconfirmed', null] as const)('gives back a %s creation with no agent past the launch deadline, as the Swarm failed its start', async state => {const f = setup(), book = f.open()free(f, 3 * GiB)expect(await book.reserve(agentInput(1))).toMatchObject({ ok: true })f.native.set(id(1), state ? { creationId: id(1), state } : null)- f.clock.now += 7 * 24 * 60 * MINUTE+ // Still inside the Swarm's own wait for it (up to 15 minutes while its machine reports it preparing): it may launch.+ f.clock.now += LAUNCH_DEADLINE_MS - 1expect(await book.reserve(agentInput(2))).toMatchObject({ error: 'CAPACITY_UNAVAILABLE' })- expect(await book.observe(id(1))).toMatchObject({ released: false })+ expect(await book.observe(id(1))).toMatchObject({ released: false, harnessId: null })+ expect(f.deps.log).not.toHaveBeenCalled()+ f.clock.now += 1+ expect(await book.reserve(agentInput(2))).toMatchObject({ ok: true })+ expect(await book.observe(id(1))).toMatchObject({ error: 'CAPACITY_RECEIPT_MISSING' })+ expect(f.deps.log).toHaveBeenCalledTimes(1)+ expect(f.deps.log).toHaveBeenCalledWith(expect.stringContaining(`start receipt ${id(1)}`))+})++it('clears every stuck row on the first start after an update and restart, and still refuses a real shortage', async () => {+ const f = setup(), before = f.open()+ free(f, 16 * GiB)+ for (const n of [1, 2, 3, 4]) {+ expect(await before.reserve(agentInput(n))).toMatchObject({ ok: true })+ f.native.set(id(n), { creationId: id(n), state: n % 2 ? 'pending' : 'unconfirmed' })+ }+ before.stop()+ f.clock.now += 26 * 60 * MINUTE+ free(f, 8 * GiB)+ const after = f.open()+ expect(await after.reserve(agentInput(5))).toMatchObject({ ok: true })+ expect(f.deps.log).toHaveBeenCalledTimes(4)+ for (const n of [1, 2, 3, 4]) expect(await after.observe(id(n))).toMatchObject({ error: 'CAPACITY_RECEIPT_MISSING' })+ // Only the new start holds now: 8 GiB free fits three more 2 GiB starts and refuses the fourth.+ for (const n of [6, 7, 8]) expect(await after.reserve(agentInput(n))).toMatchObject({ ok: true })+ expect(await after.reserve(agentInput(9))).toMatchObject({ error: 'CAPACITY_UNAVAILABLE',+ detail: expect.stringMatching(/^Not enough free memory: 0 GiB free, 2 GiB needed\. 8 GiB more is set aside for 4 agents still starting/) })+})++// River on task #2890: a start the Swarm failed at 3 minutes held its memory for the whole launch deadline. Its host reads that+// verdict (machines.ts) and ends the reservation at once; the deadline stays the backstop for a verdict it never reads.+it('ends the reservation of a start the Swarm failed at once, whatever its creation reads, unless an agent launched', async () => {+ const f = setup(), book = f.open()+ free(f, 3 * GiB)+ expect(await book.reserve(agentInput(1))).toMatchObject({ ok: true })+ f.native.set(id(1), { creationId: id(1), state: 'unconfirmed' })+ expect(await book.reserve(agentInput(2))).toMatchObject({ error: 'CAPACITY_UNAVAILABLE' })+ expect(await book.ended(id(1))).toEqual({ ok: true, reservationId: id(1), returned: true })+ expect(f.deps.log).toHaveBeenCalledWith(expect.stringContaining(`start receipt ${id(1)}: the Swarm failed its start`))+ expect(await book.ended(id(1))).toEqual({ ok: true, reservationId: id(1), returned: false })+ expect(await book.reserve(agentInput(2))).toMatchObject({ ok: true })+ // One whose agent launched keeps its row and identity: its host stops that agent, and the stop releases it.+ free(f, 5 * GiB)+ expect(await book.reserve(agentInput(3))).toMatchObject({ ok: true })+ f.launch(3)+ expect(await book.ended(id(3))).toEqual({ ok: true, reservationId: id(3), returned: false })+ expect(await book.observe(id(3))).toMatchObject({ ok: true, harnessId: 'agent-3', released: false })+ expect(await book.ended('not a creation id')).toEqual({ error: 'INVALID_REQUEST' })+})++it('keeps a creation that launched after the launch deadline, with the identity it learns', async () => {+ const f = setup(), book = f.open()+ free(f, 3 * GiB)+ expect(await book.reserve(agentInput(1))).toMatchObject({ ok: true })+ f.clock.now += 7 * 24 * 60 * MINUTE+ f.launch(1)+ expect(await book.observe(id(1))).toMatchObject({ ok: true, harnessId: 'agent-1', released: false })+ expect(f.deps.log).not.toHaveBeenCalled()})it('keeps a launched creation that reads as missing past the deadline: its learned identity is not erased', async () => {diff --git a/cli/src/superTabs/reservations.ts b/cli/src/superTabs/reservations.tsindex 1b8cc9bfe..fd1159fd0 100644--- a/cli/src/superTabs/reservations.ts+++ b/cli/src/superTabs/reservations.ts@@ -27,6 +27,7 @@ export interface ReservationBook {reserve(input: Json, closed?: AbortSignal): Promise<Json>observe(creationId: unknown, closed?: AbortSignal): Promise<Json>giveBack(creationId: unknown, closed?: AbortSignal): Promise<Json>+ ended(creationId: unknown, closed?: AbortSignal): Promise<Json>creationOf(harnessId: string): string | nullstop(): void}@@ -37,6 +38,8 @@ interface Dependencies {now?: () => number/** Each live agent's measured memory, by harness id. */agentMemory?: () => Promise<Map<string, number>>+ /** Where a give-back is told: this machine's log (issue #2989: a held start was found only by reading the ledger). */+ log?: (line: string) => void}const MAX_ACTIVE = 256const MAX_FILE_BYTES = 32 * 1024 * 1024@@ -56,6 +59,20 @@ export const START_WINDOW_MS = 5 * 60_000* the start may still be waiting in the backend (a host asleep or down for longer), and its next run reserves again* against what is free then. Each run that confirms the row restarts its deadline. */export const START_DEADLINE_MS = 15 * 60_000+/** How long a start with a creation on its machine but no agent may hold its reservation. Only a dispatched start has a+ * creation, and the Swarm fails every dispatched start with no agent by its own deadline (backend liveness.ts: 3 minutes,+ * 15 while its machine reports it preparing). One its machine still reads as pending, unconfirmed or unreadable after+ * that was held for good: trunghieu's Team account refused every start for four such 2 GiB starts the Swarm had failed,+ * across a restart (issue #2989, 2026-11-04). The start window on top covers the reservation made just before dispatch. */+export const LAUNCH_DEADLINE_MS = START_DEADLINE_MS + START_WINDOW_MS+/** The starts a memory refusal counts, oldest first, so a held one can be found (issue #2989 named only the total). Members+ * read only the refusal's numbers (backend startFailure.ts); this rest reaches the start's Lead and the machine owner. */+const setAside = (rows: Reservation[], now: number) => {+ const oldest = [...rows].sort((a, b) => a.created_at - b.created_at), minutes = Math.floor((now - oldest[0]!.created_at) / 60_000)+ const age = minutes < 1 ? 'under a minute' : minutes === 1 ? '1 minute' : `${minutes} minutes`+ const named = oldest.slice(0, 3).map(row => row.creation_id).join(', ') + (oldest.length > 3 ? ` and ${oldest.length - 3} more` : '')+ return `${oldest.length === 1 ? '1 agent' : `${oldest.length} agents`} still starting, the oldest reserved ${age} ago (start receipts ${named})`+}const memoryOf = (core: CoreApi) => {const read = createHarnessResourcesReader(() => core.agents.live())return async () => new Map((await read()).agents.flatMap(row => row.memoryBytes === null ? [] : [[row.agentId, row.memoryBytes] as const]))@@ -63,7 +80,7 @@ const memoryOf = (core: CoreApi) => {/** All Super Tabs on this host share one ledger. SQLite serializes admission* across service restarts/overlap; no asynchronous work runs inside its write- * transaction. Unknown native outcomes keep their budget, without a timeout. */+ * transaction. Unknown native outcomes keep their budget until the Swarm's own start deadline has passed (LAUNCH_DEADLINE_MS). */export class CapacityReservations implements ReservationBook {private readonly db: Databaseprivate readonly file: string@@ -71,6 +88,7 @@ export class CapacityReservations implements ReservationBook {private stopped = falseprivate readonly now: () => numberprivate readonly agentMemory: () => Promise<Map<string, number>>+ private readonly log: (line: string) => voidconstructor(private readonly core: CoreApi, private readonly machineId: string,private readonly deps: Dependencies = { resources: readMachineResources, disk: readCapacityDisk,@@ -95,6 +113,7 @@ export class CapacityReservations implements ReservationBook {}this.now = deps.now ?? Date.nowthis.agentMemory = deps.agentMemory ?? memoryOf(core)+ this.log = deps.log ?? (line => console.log(`[super-tabs] ${line}`))this.identity = lstatSync(this.file)this.db = new Database(this.file)try {@@ -153,7 +172,7 @@ export class CapacityReservations implements ReservationBook {}private async reconcile(rows: Reservation[], closed?: AbortSignal): Promise<void> {- const deadline = this.now() - START_DEADLINE_MS+ const now = this.now(), deadline = now - START_DEADLINE_MS, launchDeadline = now - LAUNCH_DEADLINE_MSfor (let index = 0; index < rows.length; index += 8) {const observations = await Promise.all(rows.slice(index, index + 8).map(async row => {const observation = await this.native(row.creation_id)@@ -163,15 +182,21 @@ export class CapacityReservations implements ReservationBook {const released = observation?.state === 'failed' || observation?.state === 'cancelled'|| (!!harnessId && await this.core.agents.stopped?.(harnessId).catch(() => false) === true)|| (observation?.state === 'created' && await this.core.agents.gone?.(harnessId!).catch(() => false) === true)- // Only a confirmed missing creation: a pending, unconfirmed or unreadable one may still launch.- const expired = observation?.state === 'missing' && !harnessId- return { row, harnessId, released, expired }+ // A missing creation is given back at the start deadline, which a run that confirms it restarts: its start may+ // still be waiting in the backend. One that is pending, unconfirmed or unreadable was dispatched, and the Swarm+ // failed it by the launch deadline if no agent joined (issue #2989). A launched or ended creation keeps its row.+ const expired = !harnessId && !released+ ? observation?.state === 'missing' ? deadline : observation?.state === 'created' ? null : launchDeadline+ : null+ return { row, harnessId, released, expired, state: observation?.state ?? 'unreadable' }}))this.transaction(() => {- for (const { row, harnessId, released, expired } of observations) {+ for (const { row, harnessId, released, expired, state } of observations) {// The deadline is checked on the row as it is now: a run may have confirmed it again since it was read.- if (expired) this.db.prepare(`DELETE FROM reservations WHERE creation_id = ? AND fingerprint = ? AND released = 0- AND harness_id IS NULL AND created_at <= ?`).run(row.creation_id, row.fingerprint, deadline)+ const gone = expired === null ? null : this.db.prepare(`DELETE FROM reservations WHERE creation_id = ? AND fingerprint = ?+ AND released = 0 AND harness_id IS NULL AND created_at <= ?`).run(row.creation_id, row.fingerprint, expired) as { changes?: number }+ if (gone?.changes === 1) this.log(`Gave back the ${gibibytes(row.memory_bytes)} of memory reserved for start receipt ${row.creation_id}: `+ + `no agent ${Math.floor((now - row.created_at) / 60_000)} minutes after it reserved (its creation is ${state} here).`)// A later reconciliation may already have learned the native ID.// Missing/unavailable observations never erase that durable identity.this.db.prepare(`UPDATE reservations SET harness_id = COALESCE(harness_id, ?), released = ?@@ -220,6 +245,21 @@ export class CapacityReservations implements ReservationBook {}, closed)}+ /** Ends the reservation of a start the Swarm failed with no agent, which its host read from the Swarm (machines.ts): River on+ * task #2890, a start failed at 3 minutes held its memory for the whole launch deadline. Whatever its creation reads, as+ * the Swarm will not take its agent now; one that launched keeps its row, released when its host stops that agent. */+ async ended(creationId: unknown, closed?: AbortSignal): Promise<Json> {+ if (!creationIdValid(creationId)) return { error: 'INVALID_REQUEST' }+ this.check(closed)+ await this.reconcile(this.rows().filter(row => row.creation_id === creationId), closed)+ return this.transaction(() => {+ const row = this.db.prepare('SELECT memory_bytes FROM reservations WHERE creation_id = ? AND released = 0 AND harness_id IS NULL').get(creationId)+ const gone = this.db.prepare('DELETE FROM reservations WHERE creation_id = ? AND released = 0 AND harness_id IS NULL').run(creationId) as { changes?: number }+ if (gone?.changes === 1) this.log(`Gave back the ${gibibytes(Number(row?.memory_bytes ?? 0))} of memory reserved for start receipt ${creationId}: the Swarm failed its start.`)+ return { ok: true, reservationId: creationId, returned: gone?.changes === 1 }+ }, closed)+ }+/** The Swarm start that created this harness: every start confirms the harness its native receipt names before it* reports success, and a stopped one keeps its row. The core still checks the receipt itself. */creationOf(harnessId: string): string | null {@@ -278,7 +318,7 @@ export class CapacityReservations implements ReservationBook {const diskHeld = active.filter(row => row.volume === destination.volume && this.starting(row, now)).reduce((sum, row) => sum + row.disk_bytes, 0)if (memory > 0 && !finite(freeMemory)) return unavailable('The free memory on this machine could not be read.')if (memory > 0 && memory > freeMemory! - memoryHeld) return unavailable(memoryShort(freeMemory! - memoryHeld, memory)- + (memoryHeld > 0 ? ` ${gibibytes(memoryHeld)} more is set aside for agents still starting.` : ''))+ + (memoryHeld > 0 ? ` ${gibibytes(memoryHeld)} more is set aside for ${setAside(active.filter(row => this.held(row, measured, now) > 0), now)}.` : ''))if (disk > destination.freeBytes - diskHeld) return unavailable('Destination disk space is already in use or reserved for other agents. Choose another machine or free space before retrying.')this.db.prepare(`INSERT INTO reservations (creation_id, fingerprint, memory_bytes, disk_bytes, volume, created_at)VALUES (?, ?, ?, ?, ?, ?)`).run(creationId, fingerprint, memory, disk, destination.volume, now)diff --git a/cli/src/superTabs/service.spec.ts b/cli/src/superTabs/service.spec.tsindex 5079416e9..030f30302 100644--- a/cli/src/superTabs/service.spec.ts+++ b/cli/src/superTabs/service.spec.ts@@ -552,7 +552,7 @@ it('keeps host capacity separate from account authority and closes it on disconnconst reserve = vi.fn(async () => ({ ok: true, reservationId: 'native-creation-id' }))const observe = vi.fn(async () => ({ ok: true, released: true }))const giveBack = vi.fn(async () => ({ ok: true, reservationId: 'native-creation-id', returned: true }))- const stop = vi.fn(), factory = vi.fn(() => ({ reserve, observe, giveBack, creationOf: vi.fn(() => null), stop }))+ const stop = vi.fn(), factory = vi.fn(() => ({ reserve, observe, giveBack, ended: vi.fn(), creationOf: vi.fn(() => null), stop }))const service = superTabService(core, { reservations: factory })const closed = new AbortController().signalexpect(factory).not.toHaveBeenCalled()@@ -572,10 +572,25 @@ it('keeps host capacity separate from account authority and closes it on disconnservice.port.stop(); expect(stop).toHaveBeenCalledTimes(3)})+// Issue #2989: a start the Swarm failed gives its reservation back at once. Only this machine's own host, which read that+// verdict from the Swarm, may say so: a relayed caller with machine access could otherwise free memory still starting.+it('ends a failed start\'s reservation only for a caller on this machine', async () => {+ const core = fakeCore()+ const ended = vi.fn(async () => ({ ok: true, reservationId: 'native-creation-id', returned: true }))+ const factory = vi.fn(() => ({ reserve: vi.fn(), observe: vi.fn(), giveBack: vi.fn(), ended, creationOf: vi.fn(() => null), stop: vi.fn() }))+ const service = superTabService(core, { reservations: factory })+ const closed = new AbortController().signal+ expect(await service.requests.super_tabs({ action: 'endCapacity', creationId: 'native-creation-id' }, visitor, closed)).toEqual({ error: 'OWNER_REQUIRED' })+ expect(ended).not.toHaveBeenCalled()+ expect(await service.requests.super_tabs({ action: 'endCapacity', creationId: 'native-creation-id' }, { local: true, owner: false }, closed))+ .toMatchObject({ ok: true, returned: true })+ expect(ended).toHaveBeenCalledWith('native-creation-id', closed)+})+it('gives the host this machine\'s record of the Swarm starts, to close a session the backend ends', async () => {const core = fakeCore()const creationOf = vi.fn((harnessId: string) => harnessId === 'harness' ? 'super-0123456789abcdef' : null)- const factory = vi.fn(() => ({ reserve: vi.fn(), observe: vi.fn(), giveBack: vi.fn(), creationOf, stop: vi.fn() }))+ const factory = vi.fn(() => ({ reserve: vi.fn(), observe: vi.fn(), giveBack: vi.fn(), ended: vi.fn(), creationOf, stop: vi.fn() }))let options: SuperTabHostOptions = {}const host = vi.fn((_core: unknown, _machine: string, given: SuperTabHostOptions) => {options = givendiff --git a/cli/src/superTabs/service.ts b/cli/src/superTabs/service.tsindex de086f657..1e599839b 100644--- a/cli/src/superTabs/service.ts+++ b/cli/src/superTabs/service.ts@@ -420,6 +420,9 @@ export function superTabService(core: CoreApi, options: SuperTabServiceOptions =if (payload.action === 'reserveCapacity') return await reservations().reserve(payload, closed)if (payload.action === 'observeCapacity') return await reservations().observe(payload.creationId, closed)if (payload.action === 'returnCapacity') return await reservations().giveBack(payload.creationId, closed)+ // Only this machine's own host, which read the Swarm's verdict for its start (machines.ts): a relayed caller could+ // otherwise free memory a start still holds (issue #2989).+ if (payload.action === 'endCapacity') return asker.local ? await reservations().ended(payload.creationId, closed) : { error: 'OWNER_REQUIRED' }// A backend-issued context scopes provenance only. Existing machine// control is still required to open/write the native terminal itself.if (payload.action === 'prepareNative' || payload.action === 'renewNative') {diff --git a/docs/plans/2026-11-04-machine-agents-as-memory-allows-plan.md b/docs/plans/2026-11-04-machine-agents-as-memory-allows-plan.mdnew file mode 100644index 000000000..a0fbf23e9--- /dev/null+++ b/docs/plans/2026-11-04-machine-agents-as-memory-allows-plan.md@@ -0,0 +1,47 @@+# Machines take agents as memory allows — plan++> Written by the Swarm Engineer (Kiln5) for task #2860, by the project's way of working: plan first, each task test+> first, one review step on the pull request.++**Goal:** A machine's Agents setting is As memory allows unless its owner picked At most N. Joining no longer holds a+machine to the suggested 16; a start is refused only for real lack of free memory, and says so (Dee, 11-04: "remove the+limit"). Mac 3 refused Milo's engineer at 16 of 16.++**Architecture:** Backend only in this pull request (smallest safe change, River 11-04). `ContributedMachine.agents`+(`memory` | `atMost`) is the explicit setting; `machine.contribute` takes it optionally. A row without it (every row+today, and every join from today's clients) is read by `agentLimit`: a stored count equal to the old join suggestion+(16, or one per 4 GiB of the machine's measured memory) is As memory allows; any other number is the owner's own At most+N. Desktop's old As memory allows stored the machine's maximum (one per 2 GiB), which free memory reaches first. Read time, no migration. Admission+(`admitByRoom`) checks the owner's count or, as memory allows, the last reading's free memory against the start's memory+requirement; the host still checks free memory and disk at launch. `availableSlots` (route, plan, Desktop) counts room by+free memory (one per 2 GiB) when memory allows. A Team account copies its Mac's setting.++**Tech Stack:** TypeScript, zod, vitest (backend).++**Spec:** task #2860 brief.++## Global Constraints+- Old clients keep working: `maxAgents` stays a stored number; `agents` is optional on the command.+- Rollback: an older backend ignores `agents` and reads `maxAgents` as before.+- No database change (machines live in the Swarm state document).+- CLI `set-agents` and Desktop's settings dialog sending `agents` follow in a later pull request, after this backend ships+ (an older server refuses the unknown key).++## Design+- growth: one optional enum per machine row, removed with the row. concurrency: the Swarm state's existing write lock.+- read cost: computed from the row in memory. scale: unchanged per read. compatibility and migration: above.+- audience: only the owner's own session contributes (unchanged). identity: N/A, no login used.+- failure: a refused start says the machine is out of free memory, with the numbers. platforms: every host.++## Review Focus+- An owner's own At most N stays a limit; only the old join suggestion reads as memory allows.+- The memory refusal uses the reading's free memory, not the count.++## Task 1: Agents setting and memory admission+**Files:** `backend/src/superTabs/machines.ts`, `backend/src/superTabs/commands.ts`, `backend/src/superTabs/read/views.ts`,+`backend/src/superTabs/store.ts`, test `backend/src/superTabs/teamRouting.test.ts`.+**Interfaces:** `agentLimit(pool)`, `availableSlots(pool, used)`, `admitByRoom(pool, used, memoryBytes)`, `freeMemoryBytes(pool)`.+- [ ] Failing tests: join stores no fixed count; an old suggested 16 reads as memory allows; an owner's At most N is kept;+ a start refused for memory says so; the route counts room by free memory; a Team account follows its Mac.+- [ ] Watch them fail.+- [ ] Implement; run `npm test -- src/superTabs/teamRouting.test.ts` and the tests of the files that import the changed code.diff --git a/docs/plans/2026-11-04-stuck-start-reservations-plan.md b/docs/plans/2026-11-04-stuck-start-reservations-plan.mdnew file mode 100644index 000000000..f02c05199--- /dev/null+++ b/docs/plans/2026-11-04-stuck-start-reservations-plan.md@@ -0,0 +1,67 @@+# Stuck start reservations Implementation Plan++**Goal:** A machine stops counting memory and disk for Swarm starts that failed, timed out or never got an agent, so+trunghieu's Team account ("8 GiB more is set aside for agents still starting", issue #2989) admits starts again without+anyone touching it.++**Architecture:** The machine's capacity ledger (`cli/src/superTabs/reservations.ts`, `<dataDir>/super-tabs/capacity.sqlite`)+keeps one row per start. A row with no agent holds its whole budget while it is "starting". Today only a creation the+machine reports as `missing` is given back at the start deadline (15 minutes); one it reports `pending`, `unconfirmed` or+cannot read is held forever, and a restart does not clear it. The backend fails every dispatched start that has no agent+by its own start deadline (backend `liveness.ts`: 3 minutes, 15 while its machine reports it preparing), and only a+dispatched start has a native creation. So past that deadline plus a margin, an agentless row belongs to a start the+Swarm already failed: reconcile gives it back, logs it, and the next reserve (the first start after the update) clears+every old stuck row. A creation that reads `created` keeps its row and identity as today.++**Tech Stack:** TypeScript, node:sqlite ledger, vitest.++**Spec:** Task #2890 / issue #2989 (Ken's evidence: four dispatched 2 GiB starts with no harness, each failed by the+server "did not start it within 3 minutes").++## Global Constraints++- CLI only; Swarm code stays out of `cli/src/core`. No ledger schema change (no migration, old/new CLIs read the same file).+- Tests never touch the real `~/.harness`: the spec's fixture ledger lives in a `mkdtemp` folder.+- The memory refusal keeps its `Not enough free memory: X GiB free, Y GiB needed.` prefix: the backend reads only that+ prefix for members (backend `startFailure.ts`); the rest goes only to the start's Lead and the machine owner.++## Design++- growth: a give-back DELETEs the agentless row; rows with an agent are unchanged (released when it stops).+- concurrency: the delete runs in the ledger's existing `BEGIN IMMEDIATE` transaction and re-checks `harness_id IS NULL`+ and `created_at` on the row as it is then, so a run that confirmed it meanwhile keeps it.+- read cost: reconcile already reads every active row's creation status on reserve; nothing new is read.+- scale: at most 256 active rows (MAX_ACTIVE); stuck rows no longer accumulate toward it.+- compatibility: same file and schema; an older CLI reading the ledger after this one sees fewer rows.+- migration + rollback: none; rolling back restores the old hold-forever behavior for new rows only.+- audience: the extra refusal text (receipt ids, age) reaches only the start's Lead and the machine owner.+- identity: N/A because no account or login is involved.+- failure: each give-back is logged on the machine; the refusal names the starts still holding memory.+- platforms: all (the ledger is platform-neutral).++## Review Focus++- That no row whose creation is `created` (an agent that runs) is given back.+- That the bound (START_DEADLINE_MS + START_WINDOW_MS = 20 minutes after the reservation's last confirmation) is never+ shorter than the backend's own wait for a dispatched start.++## Task 1: Give back agentless rows past the launch deadline++**Files:** `cli/src/superTabs/reservations.ts`, `cli/src/superTabs/reservations.spec.ts`++**Interfaces:** new exported `LAUNCH_DEADLINE_MS`; optional `log` dependency on `CapacityReservations`.++- [ ] Failing tests: a dispatched start reading `pending`, `unconfirmed` or unreadable past the launch deadline is given+ back and the next start is admitted; one still inside it keeps holding; a ledger reopened (restart) with four stuck+ 2 GiB rows admits a start on its first reserve; a `created` row past the deadline keeps its identity; a real shortage+ is still refused.+- [ ] Code: reconcile expires agentless rows that are not `created`, `failed` or `cancelled` past `LAUNCH_DEADLINE_MS`+ (missing ones keep the existing 15-minute deadline), and logs each give-back.++## Task 2: The refusal names the starts holding memory++**Files:** same.++- [ ] Failing test: a memory refusal with memory set aside names how many starts hold it, their receipts and the oldest+ one's age, after the unchanged prefix.+- [ ] Code: build that text from the rows counted in `held`.