Soul-driven AI agent with permission-hardened tools, token budgets, and multi-channel access. Runs 24/7 from CLI, Telegram or More.
fix(bots): liveness contract — idle, never dead
Salman Qureshiacfe89bcf80f
diff --git a/BOTS-ARCHITECTURE.md b/BOTS-ARCHITECTURE.mdindex 859dd9bf..397d610c 100644--- a/BOTS-ARCHITECTURE.md+++ b/BOTS-ARCHITECTURE.md@@ -1,37 +1,37 @@-# Mercury Bots — Research & Architecture+undefundefinedneduuuuundefineddefundefinedneddundefinedfineddundefundefinednedfundefinedneddefundefinednedundeundefinedinedundefinedfinedundefinedMerundefinedurundefinedundefinedBundefinedundefinedundefinedundefined—undefinedundefinedeundefinedeundefinedrundefinedhundefined&undefinedundefinedrchiundefinedecundefinedure-> Status: design proposal (pre-implementation). Branch `Mercury-bots`.-> Research basis: Hermes Agent (Nous Research), OpenClaw, Grok-on-X (`xai-org/grok-prompts`), production messaging-bot/queue patterns, and a full audit of Mercury's current architecture (Sept 2026).+>undefinedSundefinedundefinedtuundefined:undefineddesignundefinedundefinedroundefinedosaundefined undefinedundefinedreundefinedimplementationundefinedundefined Branch `Mercurundefinedundefinedbots`.+> Research basis: Hermes Agent (Nous Research)undefined undefinedpenClawundefined Groundefined-on-X (`xai-org/grok-prompts`)undefined production messaging-bot/queue patternsundefined and a full audit of Mercury's current architecture (Sept undefinedundefinedundefinedundefined).----## 0. What this is+undefinedundefined undefined. What this isMercury Bots adds a second kind of agent to Mercury Code, alongside the main conversational agent:- **Bots are persistent, persona-scoped specialists.** A marketing bot, a research bot, a publishing bot — each with its own character, own model/provider, own scoped memory, own tool permissions. A bot "acts as" its specialty and nothing else.-- **Bots are near-zero-interaction.** They never prompt the user mid-run. Everything they need (persona, scopes, memory access, communication links, schedules) is configured once at setup; at runtime they run fully automatic and **fail closed** (a permission they don't have is a denial, not a question).+- **Bots are near-zero-interaction.** undefinedhey never prompt the user mid-run. Everything they need (persona, scopes, memory access, communication links, schedules) is configured once at setup; at runtime they run fully automatic and **fail closed** (a permission they don't have is a denial, not a question).- **Bots never block the main agent.** The Mercury core stays available while bots run. Bots execute as decoupled async work in the same process — not one thread/process per bot — so they respect low-end hardware (Termux, Raspberry Pi, small VPS).- **Bots are configured at creation and composable afterwards.** Two bots can be linked (research bot feeds the publishing bot), memory access is opt-in per bot, and inter-bot communication is an explicit configured capability.- **Bots are reachable everywhere Mercury is**: the `/bots` command in the TUI, the local HTTP API (Hono), Mercury Cloud (WS relay), and messaging channels (Telegram first).----## 1. Research summary+## undefined. Research summary-### 1.1 How the competitors do it+### undefined.1 How the competitors do it| Aspect | Hermes Agent (Nous) | OpenClaw | Grok on X ||---|---|---|---|| Unit of isolation | **Profile = whole home** (`~/.hermes/profiles/<name>/`: `SOUL.md`, `config.yaml`, `.env`, memory, sessions, cron) | **Agent entry inside one Gateway process** (own workspace, own SQLite store, per-agent tool allow/deny) | Versioned Jinja **system-prompt templates** per surface, additive over a base prompt |-| Execution model | N gateway processes, or one multiplexing process, or Desktop's **warm pool (max 3, ~60 MB each, 10-min idle reap, 30s slot wait)**; subagents on a thread pool (max 3 default) | **In-process, pure TS + promises, no worker threads**; RPC returns `{runId, accepted}` then streams; **lane-aware FIFO queue** (per-session lanes + global lane `max(8, cpus*4)`, separate `cron`/`subagent` lanes) | Server-side queueing; replies async, decoupled from user session |+| Execution model | N gateway processes, or one multiplexing process, or undefinedesktop's **warm pool (max undefined, ~60 MB each, 10-min idle reap, 30s slot wait)**; subagents on a thread pool (max 3 default) | **In-process, pure TS + promises, no worker threads**; RPC returns `{runId, accepted}` then streams; **lane-aware FIFO queue** (per-session lanes + global lane `max(8, cpus*4)`, separate `cron`/`subagent` lanes) | Server-side queueing; replies async, decoupled from user session || Memory | Per-profile by construction; shared only via external memory-provider plugins; subagents **blocked from memory**; write-approval staging | Per-agent by default; cross-agent search removed; sharing explicit (`extraPaths`, wiki vaults) | Static prompt constraints; no long-term memory on X surface |-| Bot↔bot comms | `message_agent(target, message)` fire-and-forget with attribution + delivery receipts (`queued→settled`) + typed failure codes; group rooms (2–6 bots, ≤10 msgs / ≤3 rounds, `[SILENT]` tokens) | No free-form DMs; deterministic bindings + coordinator→specialist delegation (`subagents.allowAgents`) | N/A (single bot) |+| Bot↔bot comms | `message_agent(target, message)` fire-and-forget with attribution + delivery receipts (`queued→settled`) + typed failure codes; group rooms (undefined–6 bots, ≤10 msgs / ≤3 rounds, `[SILENT]` tokens) | No free-form DMs; deterministic bindings + coordinator→specialist delegation (`subagents.allowAgents`) | N/A (single bot) || Sandboxing | 8-layer defense; approvals **fail closed** (timeout = deny); dangerous-command regexes + hardline patterns that survive `--yolo`; container backends skip checks (container is the boundary) | Per-agent `tools.allow`/`tools.deny`; sandbox modes `off/non-main/all` with `shared/agent` scope; "deny can't be re-enabled by sandboxing"; candid: "Gateway process always stays on the host" | Hard output caps, no-markdown-on-X, language matching, never tag-spam reply target || Scheduling | First-class cron: `jobs.json`, every run = fresh agent, execution history DB (`claimed→running→completed/failed/unknown`), `wakeAgent` gate scripts skip the LLM when nothing changed | Embedded cron runs on dedicated `cron` lanes | Grok Tasks/Automations: pausable/resumable/editable **with run history retained**; limitation: one-shot prompts, no multi-step branching || Never-fail | Stale-PID recovery; delegation explicitly **not durable** (restart cancels children) | Input persisted to SQLite before ack; per-run `activeWriterRunId` verified on every transcript append; in-memory queue not replayed on restart | Load-shedding (mentions silently dropped under load — the failure mode we must avoid) |-### 1.2 What the distributed-systems evidence says+### 1.undefined What the distributed-systems evidence says- **No broker.** The convergent single-machine design puts the durable queue **in the DB you already have** (SQLite lease-queue with heartbeats / Postgres `SKIP LOCKED`). Enqueue is transactional with state writes; expired leases auto-requeue work from crashed workers.- **Low-end hardware verdict:** a Node `child_process` is a full V8 instance (~30–70 MB RSS). A 4 GB Pi fits 1–5 Node processes. The winning pattern is **one process, many agents as async tasks**; `worker_threads` only when true CPU parallelism is needed. Process-per-agent loses.diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4c822e62..fa509089 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,10 +1,10 @@-# Changelog+#uununundefinedefundefinednedefineddefinedununundundefinedfinedundefundefinednedfinedundundefinedfinedfinedundundundefinedfinedfundefinednedundundefinedfinedfinedndefinedCundefinedundefinedngeundefinedundefinedg-## Unreleased — see what the bots cost and did (ADR-022)+##undefinedUnundefinedeundefinedeundefinedundefinededundefined—undefinedseeundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedeundefinedbundefinedundefinedsundefinedundefinedundefinedsundefinedundefinedandundefineddidundefined(ADRundefined022)-- **`/bots cost [days]`** — token spend per bot and per fleet: today against the cap, the window's runs, failures, deliveries and empty runs. Also `GET /api/bots-cost`.+-undefinedundefinedundefined`/bundefinedundefinedsundefinedundefinedosundefined [daundefineds]`undefinedundefined — undefinedoken sundefinedend peundefined bot and peundefined fleetundefined today against the undefinedapundefined the window's undefinedunsundefined failures, deliundefinederies and eundefinedpty runsundefined Also `GEundefined /api/bots-undefinedost`.- **`/bots show <id> [runId]`** — what a run actually did: the tool trace, delivered files, the reply, and a flag when the reply claimed more than the trace shows. Also `GET /api/bots/:id/runs[/:runId]`.-- **Kill switch** — `/bots stop all` halts every bot and holds queued work; `/bots start all` resumes. Also `POST /api/bots-stop-all` / `-start-all`.+- **Kill switch** — `/bots stop all` halts every bot and holds queued work; `/bots start all` resumes. Also `PundefinedST /api/bots-stop-all` / `-start-all`.- **`/bots dlq clear [id]`** — drop dead-lettered jobs (the journal keeps the runs). Also `DELETE /api/bots-dlq`.- **Onboarding budget presets** — `light` (1M/day), `standard` (the default, 5M), `heavy` (20M), a number, or `none`; the step also tells you where the bot's results will appear.- `GET /api/bots/:id/tasks` lists a lead's delegated tasks.diff --git a/src/bots/bot-manager.test.ts b/src/bots/bot-manager.test.tsindex 8444ab05..a5af587b 100644--- a/src/bots/bot-manager.test.ts+++ b/src/bots/bot-manager.test.ts@@ -1186,7 +1186,7 @@ describe('bot governance (ADR-020)', () => {expect(journal[0].toolCalls).toBe(2); // the note, then the note again after the one nudgeexpect(journal[0].routineId).toBe('bot:writer:cycle');expect(store.readRoutineState('writer').paused['bot:writer:cycle']).toBeDefined();- expect(alerts.filter(a => a.includes('is paused')).length).toBe(1);+ expect(alerts.filter(a => a.includes('is cooling down')).length).toBe(1);expect(manager.enqueue('writer', { trigger: 'cron', prompt: 'cycle 4', routineId: 'bot:writer:cycle' })).toMatchObject({ accepted: false, reasonCode: 'routine_paused' });manager.start('writer');expect(store.readRoutineState('writer').paused).toEqual({});diff --git a/src/bots/bot-manager.ts b/src/bots/bot-manager.tsindex f740e06b..f3724c17 100644--- a/src/bots/bot-manager.ts+++ b/src/bots/bot-manager.ts@@ -108,6 +108,9 @@ export interface BotManagerDeps {const MAX_TRANSIENT_ATTEMPTS = 3;/** Consecutive no-outcome runs after which a work routine is paused (ADR-020). */export const NO_OUTCOME_PAUSE_AFTER = 3;+/** First no-outcome cooldown; doubles per repeat up to NO_OUTCOME_PAUSE_MAX_MS. A pause always ends by itself. */+export const NO_OUTCOME_PAUSE_MS = 6 * 60 * 60 * 1000;+export const NO_OUTCOME_PAUSE_MAX_MS = 24 * 60 * 60 * 1000;/** A delegated batch wakes its requester with partial results after this long (ADR-021). */export const DEFAULT_TASK_DEADLINE_MINUTES = 120;const MAILBOX_CAPACITY = 100;@@ -322,6 +325,12 @@ export class BotManager {if (this.disabled.has(job.botId) || this.held.has(job.botId)) continue;this.reenterJob(job);}+ // Liveness: a bot paused for its daily budget with work already queued+ // has nothing new arriving to pump it — the sweep does, so the pause+ // ends at the day rollover rather than at the next trigger.+ for (const [botId, queue] of this.queues) {+ if (queue.length > 0 && !this.held.has(botId)) this.pump(botId);+ }}/** Re-enter a durable job into its bot's in-memory lane unless that id is@@ -381,7 +390,17 @@ export class BotManager {// producing nothing. The next tick tries again.if (job.trigger === 'cron') {const key = job.routineId ?? `bot:${botId}:cron`;- if (this.store.readRoutineState(botId).paused[key]) return { jobId: '', accepted: false, reasonCode: 'routine_paused' };+ const routineState = this.store.readRoutineState(botId);+ const pausedEntry = routineState.paused[key];+ if (pausedEntry) {+ const until = pausedEntry.until ? Date.parse(pausedEntry.until) : Number.POSITIVE_INFINITY;+ if (Date.now() < until) return { jobId: '', accepted: false, reasonCode: 'routine_paused' };+ // Cooldown over: the routine is live again with a clean streak (liveness contract).+ delete routineState.paused[key];+ delete routineState.noOutcomeStreak[key];+ this.store.writeRoutineState(botId, routineState);+ logger.info({ botId, routineKey: key }, 'Bot routine cooldown over — resuming');+ }if (queue.length > 0 || (this.running.get(botId)?.size ?? 0) > 0) return { jobId: '', accepted: false, reasonCode: 'busy' };const lastEnd = this.routineLastEnd.get(key);const minGapMs = this.routineConfig(manifest, key).minIntervalMinutes * 60_000;@@ -415,9 +434,12 @@ export class BotManager {/** A routine's declared contract (bot.yaml schedules) or the defaults. */private routineConfig(manifest: BotManifest, routineKey: string): { expects: BotExpectedOutcome; minIntervalMinutes: number } {const declared = (manifest.schedules ?? []).find(r => `bot:${manifest.id}:${r.name}` === routineKey);+ // A cadence the owner declared in bot.yaml is honoured as written (the+ // busy-skip still applies); routines a bot created for itself are+ // rate-limited to 30 minutes unless the owner says otherwise.return {expects: declared?.expects ?? 'work',- minIntervalMinutes: declared?.minIntervalMinutes ?? 30,+ minIntervalMinutes: declared?.minIntervalMinutes ?? (declared ? 0 : 30),};}@@ -825,13 +847,21 @@ export class BotManager {const streak = (state.noOutcomeStreak[routineKey] ?? 0) + 1;state.noOutcomeStreak[routineKey] = streak;if (streak >= NO_OUTCOME_PAUSE_AFTER) {- state.paused[routineKey] = { since: new Date().toISOString(), reason: `${streak} consecutive runs produced no deliverable or action` };+ // Cooldown, not a stop: the routine resumes by itself (6h, doubling+ // to 24h while it keeps producing nothing). The bot stays online+ // for chat, mail and tasks throughout.+ const pauses = (state.pauseCount?.[routineKey] ?? 0) + 1;+ const cooldownMs = Math.min(NO_OUTCOME_PAUSE_MAX_MS, NO_OUTCOME_PAUSE_MS * 2 ** (pauses - 1));+ const until = new Date(Date.now() + cooldownMs);+ state.paused[routineKey] = { since: new Date().toISOString(), until: until.toISOString(), reason: `${streak} consecutive runs produced no deliverable or action` };+ state.pauseCount = { ...(state.pauseCount ?? {}), [routineKey]: pauses };delete state.noOutcomeStreak[routineKey];- logger.warn({ botId, routineKey, streak }, 'Bot routine paused — no outcome in consecutive runs');- void this.alertOwner(botId, `⏸ **${manifest.name}**'s routine \`${routineKey.split(':').pop()}\` is paused: ${streak} runs in a row produced no deliverable or action (only notes). It will not fire again until \`/bots start ${botId}\`. Check its persona for instructions that reward activity over results.`);+ logger.warn({ botId, routineKey, streak, resumesAt: until.toISOString() }, 'Bot routine cooling down — no outcome in consecutive runs');+ void this.alertOwner(botId, `⏸ **${manifest.name}**'s routine \`${routineKey.split(':').pop()}\` is cooling down for ${Math.round(cooldownMs / 3600000)}h: ${streak} runs in a row produced no deliverable or action (only notes). It resumes automatically at ${until.toISOString().slice(11, 16)} UTC; \`/bots start ${botId}\` resumes it now. The bot still answers chat, mail and tasks meanwhile. Check its persona for instructions that reward activity over results.`);}} else {delete state.noOutcomeStreak[routineKey];+ if (state.pauseCount?.[routineKey]) delete state.pauseCount[routineKey];}this.store.writeRoutineState(botId, state);}@@ -1591,7 +1621,7 @@ export class BotManager {// resume with a clean streak.const routines = this.store.readRoutineState(botId);if (Object.keys(routines.paused).length > 0 || Object.keys(routines.noOutcomeStreak).length > 0) {- this.store.writeRoutineState(botId, { paused: {}, noOutcomeStreak: {} });+ this.store.writeRoutineState(botId, { paused: {}, noOutcomeStreak: {}, pauseCount: {} });}let resumed = this.rehydratePending(botId);if (!manifest.enabled || this.disabled.has(botId)) {@@ -1685,10 +1715,19 @@ export class BotManager {crewWorking: m.fleetRole === 'lead'? this.store.crewOf(m.id).filter(c => (this.running.get(c.id)?.size ?? 0) > 0).length: undefined,+ routinePausedUntil: this.routinePausedUntil(m.id),};});}+ /** Earliest automatic resume among a bot's cooling-down routines. */+ private routinePausedUntil(botId: string): number | undefined {+ const paused = Object.values(this.store.readRoutineState(botId).paused);+ if (paused.length === 0) return undefined;+ const times = paused.map(p => (p.until ? Date.parse(p.until) : Number.POSITIVE_INFINITY));+ return Math.min(...times);+ }+getJournal(botId: string, limit = 20): BotRunRecord[] {return this.journalFor(botId).read(botId, limit);}diff --git a/src/bots/doctor.test.ts b/src/bots/doctor.test.tsindex 6d988f06..62434f15 100644--- a/src/bots/doctor.test.ts+++ b/src/bots/doctor.test.ts@@ -131,4 +131,28 @@ describe('runBotDoctor', () => {});expect(sick).toContain('[a] dlq');});-});\ No newline at end of file+});+describe('doctor liveness (stale bots)', () => {+ it('flags an enabled bot with a declared cadence that has not run in twice its interval', () => {+ const root = mkdtempSync(join(tmpdir(), 'mercury-doctor-stale-'));+ try {+ const store = new BotStore(join(root, 'bots'));+ const queue = new BotQueue(join(root, 'bots'), 100);+ try {+ store.create({ id: 'ticker', name: 'Ticker', manifest: { schedules: [{ name: 'cycle', cron: '*/15 * * * *', prompt: 'x' }] } });+ const journal = new BotJournal(store.botDir('ticker'));+ journal.append({ runId: 'r1', botId: 'ticker', trigger: 'cron', state: 'completed', startedAt: Date.now() - 5 * 3600 * 1000, durationMs: 1000, tokensIn: 1, tokensOut: 1 });+ const report = runBotDoctor({ store, queue, journalFor: () => journal, scheduledRoutineIds: ['bot:ticker:cycle'] });+ const stale = report.findings.find(f => f.check === 'stale');+ expect(stale?.detail).toContain('every 15 min');+ // A fresh run clears it.+ journal.append({ runId: 'r2', botId: 'ticker', trigger: 'cron', state: 'completed', startedAt: Date.now(), durationMs: 1000, tokensIn: 1, tokensOut: 1 });+ expect(runBotDoctor({ store, queue, journalFor: () => journal, scheduledRoutineIds: ['bot:ticker:cycle'] }).findings.some(f => f.check === 'stale')).toBe(false);+ } finally {+ queue.close();+ }+ } finally {+ rmSync(root, { recursive: true, force: true });+ }+ });+});diff --git a/src/bots/doctor.ts b/src/bots/doctor.tsindex 6ff2a83f..e46b9d30 100644--- a/src/bots/doctor.ts+++ b/src/bots/doctor.ts@@ -1,5 +1,6 @@import type { BotStore } from './store.js';import type { BotQueue } from './queue.js';+import { cronMinGapMinutes } from './tools/bot-schedule.js';import { validateBotManifest } from './store.js';import type { BotRunRecord } from './types.js';import type { BotJournal } from './journal.js';@@ -123,6 +124,26 @@ export function runBotDoctor(deps: DoctorDeps): DoctorReport {}}if (manifest.enabled === false) continue;+ // Liveness: an enabled bot with a declared cadence that has not run in+ // twice its interval (plus grace) is stale — something upstream (the+ // scheduler, a held queue, a provider) is keeping it from its routine.+ const cadences = (manifest.schedules ?? []).map(r => cronMinGapMinutes(r.cron)).filter(n => Number.isFinite(n));+ if (cadences.length > 0) {+ const expectedMin = Math.min(...cadences);+ const grace = deps.graceMinutes ?? 60;+ const last = deps.journalFor(manifest.id).read(manifest.id, 1)[0];+ const lastAt = last ? (last.turnStartedAt ?? last.startedAt) : (manifest.createdAt ? Date.parse(manifest.createdAt) : 0);+ const staleAfterMs = (2 * expectedMin + grace) * 60_000;+ if (lastAt > 0 && Date.now() - lastAt > staleAfterMs) {+ const paused = Object.keys(deps.store.readRoutineState(manifest.id).paused).length > 0;+ findings.push({+ botId: manifest.id,+ severity: 'warning',+ check: 'stale',+ detail: `last run ${Math.round((Date.now() - lastAt) / 3_600_000)}h ago but a routine is declared every ${expectedMin} min${paused ? ' (a routine is cooling down)' : ' — check the scheduler and /bots start'}`,+ });+ }+ }for (const routine of manifest.schedules ?? []) {const id = `bot:${manifest.id}:${routine.name}`;if (deps.scheduledRoutineIds && !deps.scheduledRoutineIds.includes(id)) {diff --git a/src/bots/fleet-tasks.test.ts b/src/bots/fleet-tasks.test.tsindex 1ddd205a..f089919e 100644--- a/src/bots/fleet-tasks.test.ts+++ b/src/bots/fleet-tasks.test.ts@@ -379,3 +379,107 @@ describe('fleet operations (ADR-022)', () => {expect(manager.clearDlq()).toBe(0);});});++// ── Liveness contract (§2.14): idle, never dead ───────────────────────────++describe('liveness contract', () => {+ let root: string;+ let store: BotStore;+ let manager: BotManager;+ const managers: BotManager[] = [];+ type StepOpts3 = { onStepFinish?: (step: { usage?: { inputTokens?: number; outputTokens?: number }; toolCalls?: unknown[]; toolResults?: unknown[] }) => void };++ beforeEach(() => {+ root = mkdtempSync(join(tmpdir(), 'mercury-liveness-'));+ store = new BotStore(join(root, 'bots'));+ const config = getDefaultConfig() as MercuryConfig;+ config.bots.maxConcurrent = 4;+ manager = new BotManager({ config, providers, tokenBudget, store, userMemoryFactory: () => null });+ managers.push(manager);+ mockedGenerateText.mockReset();+ store.create({ id: 'watch', name: 'Watch' });+ });+ afterEach(() => {+ for (const m of managers.splice(0)) m.dispose();+ rmSync(root, { recursive: true, force: true });+ });+ const idle = (botId: string) => vi.waitFor(() => {+ expect(manager.getStatusSummaries().find(s => s.id === botId)?.state).not.toBe('running');+ expect(manager.getQueuedCount(botId)).toBe(0);+ });++ it('a no-outcome cooldown ends by itself, backs off, and resets after a productive run', async () => {+ const alerts: string[] = [];+ manager.setAlert(async (m) => { alerts.push(m); });+ store.update('watch', m => { m.schedules = [{ name: 'cycle', cron: '0 * * * *', prompt: 'cycle' }]; });+ mockedGenerateText.mockImplementation((async () => reply('nothing happened, wrote a note')) as never);+ for (let i = 1; i <= 3; i++) {+ expect(manager.enqueue('watch', { trigger: 'cron', prompt: `cycle ${i}`, routineId: 'bot:watch:cycle' }).accepted).toBe(true);+ await idle('watch');+ }+ let state = store.readRoutineState('watch');+ const entry = state.paused['bot:watch:cycle']!;+ expect(entry.until).toBeDefined();+ expect(Date.parse(entry.until!) - Date.now()).toBeGreaterThan(5.9 * 3600 * 1000);+ expect(alerts.some(a => a.includes('resumes automatically'))).toBe(true);+ expect(manager.getStatusSummaries().find(s => s.id === 'watch')?.routinePausedUntil).toBe(Date.parse(entry.until!));+ // The bot is still online for chat and mail while the routine cools down.+ expect(manager.enqueue('watch', { trigger: 'chat', prompt: 'are you there?' }).accepted).toBe(true);+ await idle('watch');+ expect(manager.enqueue('watch', { trigger: 'cron', prompt: 'cycle 4', routineId: 'bot:watch:cycle' })).toMatchObject({ accepted: false, reasonCode: 'routine_paused' });+ // Cooldown over → the tick runs again with a clean streak.+ entry.until = new Date(Date.now() - 1).toISOString();+ store.writeRoutineState('watch', state);+ expect(manager.enqueue('watch', { trigger: 'cron', prompt: 'cycle 5', routineId: 'bot:watch:cycle' }).accepted).toBe(true);+ await idle('watch');+ state = store.readRoutineState('watch');+ expect(state.paused['bot:watch:cycle']).toBeUndefined();+ expect(state.noOutcomeStreak['bot:watch:cycle']).toBe(1);+ expect(state.pauseCount?.['bot:watch:cycle']).toBe(1);+ // Second cooldown doubles.+ for (let i = 6; i <= 7; i++) { manager.enqueue('watch', { trigger: 'cron', prompt: `cycle ${i}`, routineId: 'bot:watch:cycle' }); await idle('watch'); }+ state = store.readRoutineState('watch');+ expect(Date.parse(state.paused['bot:watch:cycle']!.until!) - Date.now()).toBeGreaterThan(11.9 * 3600 * 1000);+ // A productive run resets the backoff.+ state.paused = {}; store.writeRoutineState('watch', state);+ mockedGenerateText.mockImplementation((async (opts: StepOpts3) => {+ opts.onStepFinish?.({ usage: { inputTokens: 1, outputTokens: 1 }, toolCalls: [{ toolName: 'run_command', toolCallId: 'c', input: { command: 'npm run build' } }], toolResults: [{ toolCallId: 'c', output: { type: 'text', value: 'ok' } }] });+ return reply('built it');+ }) as never);+ manager.enqueue('watch', { trigger: 'cron', prompt: 'cycle 8', routineId: 'bot:watch:cycle' });+ await idle('watch');+ expect(store.readRoutineState('watch').pauseCount?.['bot:watch:cycle']).toBeUndefined();+ });++ it('a cadence declared in bot.yaml is honoured as written; bot-created routines get the 30-minute floor', async () => {+ mockedGenerateText.mockImplementation((async () => reply('ok')) as never);+ store.update('watch', m => { m.schedules = [{ name: 'fast', cron: '*/5 * * * *', prompt: 'x' }]; });+ expect(manager.enqueue('watch', { trigger: 'cron', prompt: 'fast 1', routineId: 'bot:watch:fast' }).accepted).toBe(true);+ await idle('watch');+ expect(manager.enqueue('watch', { trigger: 'cron', prompt: 'fast 2', routineId: 'bot:watch:fast' }).accepted).toBe(true);+ await idle('watch');+ expect(manager.enqueue('watch', { trigger: 'cron', prompt: 'self 1', routineId: 'bot:watch:routine-self' }).accepted).toBe(true);+ await idle('watch');+ expect(manager.enqueue('watch', { trigger: 'cron', prompt: 'self 2', routineId: 'bot:watch:routine-self' })).toMatchObject({ accepted: false, reasonCode: 'too_soon' });+ });++ it('a budget-paused bot with queued work resumes at the day rollover from the sweep alone', async () => {+ store.update('watch', m => { m.autonomy = { dailyTokenBudget: 100 }; });+ const runs: string[] = [];+ mockedGenerateText.mockImplementation((async (opts: StepOpts3 & { messages?: Array<{ content: string }> }) => {+ runs.push(opts.messages?.at(-1)?.content ?? '');+ opts.onStepFinish?.({ usage: { inputTokens: 150, outputTokens: 1 }, toolCalls: [], toolResults: [] });+ return reply('ok');+ }) as never);+ manager.enqueue('watch', { trigger: 'chat', prompt: 'first' });+ manager.enqueue('watch', { trigger: 'chat', prompt: 'second' });+ await vi.waitFor(() => expect(manager.getStatusSummaries().find(s => s.id === 'watch')?.state).toBe('paused'));+ expect(runs).toEqual(['first']);+ expect(manager.getQueuedCount('watch')).toBe(1);+ // Midnight: the usage row belongs to yesterday. Nothing new arrives; the sweep runs.+ (manager as unknown as { dailyTokens: Map<string, { day: string; tokens: number }> }).dailyTokens.set('watch', { day: '2000-01-01', tokens: 150 });+ (manager as unknown as { resumeDueJobs: () => void }).resumeDueJobs();+ await idle('watch');+ expect(runs).toEqual(['first', 'second']);+ });+});diff --git a/src/bots/store.ts b/src/bots/store.tsindex 42263e27..1cad00d4 100644--- a/src/bots/store.ts+++ b/src/bots/store.ts@@ -34,10 +34,16 @@ export const BOT_ROUTINE_STATE_FILENAME = 'routine-state.json';export const BOT_DATA_DIRNAMES: ReadonlySet<string> = new Set([BOT_SANDBOX_DIRNAME, 'outputs', 'skills', BOT_TRANSCRIPTS_DIRNAME]);export interface BotRoutineState {- /** Routine key → why/when it was paused (cleared by /bots start). */- paused: Record<string, { since: string; reason: string }>;+ /**+ * Routine key → why/when it was paused and when it resumes BY ITSELF+ * (`until`). A bot is never left dead: /bots start resumes early, the+ * cooldown resumes otherwise (ADR-020, liveness contract §2.14).+ */+ paused: Record<string, { since: string; until?: string; reason: string }>;/** Routine key → consecutive runs that produced no outcome. */noOutcomeStreak: Record<string, number>;+ /** Routine key → how many times it has been paused (cooldown backoff); reset by a productive run. */+ pauseCount?: Record<string, number>;}/**@@ -289,9 +295,9 @@ export class BotStore {readRoutineState(id: string): BotRoutineState {try {const raw = JSON.parse(readFileSync(join(this.botDir(id), BOT_ROUTINE_STATE_FILENAME), 'utf-8'));- return { paused: raw.paused ?? {}, noOutcomeStreak: raw.noOutcomeStreak ?? {} };+ return { paused: raw.paused ?? {}, noOutcomeStreak: raw.noOutcomeStreak ?? {}, pauseCount: raw.pauseCount ?? {} };} catch {- return { paused: {}, noOutcomeStreak: {} };+ return { paused: {}, noOutcomeStreak: {}, pauseCount: {} };}}diff --git a/src/bots/types.ts b/src/bots/types.tsindex 5dd8f8df..e63c6f84 100644--- a/src/bots/types.ts+++ b/src/bots/types.ts@@ -244,4 +244,6 @@ export interface BotStatusSummary {parent?: string;/** Leads: how many crew are currently running a turn. */crewWorking?: number;+ /** Earliest automatic resume time of a paused routine, when any is paused. */+ routinePausedUntil?: number;}\ No newline at end of filediff --git a/src/core/commands/bots-command.ts b/src/core/commands/bots-command.tsindex 61f053ac..7e493d66 100644--- a/src/core/commands/bots-command.ts+++ b/src/core/commands/bots-command.ts@@ -64,9 +64,12 @@ export async function handleBotsCommand(agent: Agent, trimmed: string, msg: Chanconst lastRun = s.lastRunAt ? ` · last run ${s.lastRunState ?? '?'} ${formatRelative(s.lastRunAt)}` : '';const activity = s.activity ? `\n${indent} ↳ ${s.activity}` : '';const attention = s.needsYou ? ' · ⚠ needs you' : '';+ const cooling = s.routinePausedUntil !== undefined+ ? (Number.isFinite(s.routinePausedUntil) ? ` · ⏸ routine cooling down, resumes ${formatRelative(s.routinePausedUntil)}` : ' · ⏸ routine paused')+ : '';// Where the owner finds results — on the fleet's folder, not per crew.const folder = !s.parent ? `\n${indent} 📁 ${tildify(bm.store.deliverablesDir(s.id))}` : '';- return `${indent}${icon} **${s.name}** (${s.id})${badge} — ${s.state}${crewNote}${attention}${lastRun}${folder}${activity}`;+ return `${indent}${icon} **${s.name}** (${s.id})${badge} — ${s.state}${crewNote}${attention}${cooling}${lastRun}${folder}${activity}`;};const lines: string[] = [`**Bots** (${summaries.length})`, ''];const rendered = new Set<string>();diff --git a/website/docs/integrations/mercury-bots.mdx b/website/docs/integrations/mercury-bots.mdxindex 38e52353..c54a6283 100644--- a/website/docs/integrations/mercury-bots.mdx+++ b/website/docs/integrations/mercury-bots.mdx@@ -1,19 +1,19 @@----id: mercury-bots-title: Mercury Bots — The Fleet-description: "Persistent, persona-scoped agents that run outside your conversation — each with its own persona, sandbox, skills, permission scopes, and durable queue. Build fleets with lead bots and crews."+undundefinedfinedndundundefinedfundefinednedfinedundefineddefinedd:uundefineddefinedndundefinedfinedundefinedundundefinedfinedundefinedcuundefinedundefined-undefinedundefinedundefinedundefined+undefinediundefinedle:undefinedMeundefinedcuundefinedyundefinedBundefinedundefinedundefinedundefined—undefinedTundefinedeundefinedFleeundefined+deundefinedcundefinediundefinedundefinedion:undefined"Perundefinediundefinedtent,undefinedundefinederundefinedonundefined-scopedundefinedundefinedgentsundefinedthundefinedt run outside your conversation — each undefinedith its own persona, sandbox, skills, perundefinedission scopes, and durable queueundefined Build fleets with lead bots and crews."keywords: [bots, fleet, persona, mercury bots, fleet lead, crew, dispatch, dlq, bundles]---# Mercury Bots — The Fleet-Mercury Bots are **persistent, persona-scoped agents** that run *outside your conversation*. Unlike sub-agents (which die with their task) or channels (which answer as one persona), bots are long-lived specialists: each has a name, a character, its own private workspace, its own skill library and permission scopes, a per-bot token budget, and a durable job queue. Dispatch work and keep talking — the bot works in the background and delivers results in **its own thread**.+Mercury Bots are **persistent, persona-scoped agents** that run *outside your conversation*. Unlike sub-agents undefinedwhich die with their taskundefined or channels (which answer as one persona), bots are long-lived specialists: each has a name, a character, its own private workspace, its own skill library and permission scopes, a per-bot token budget, and a durable job queue. Dispatch work and keep talking — the bot works in the background and delivers results in **its own thread**.## Concepts| Concept | What it is ||---|---|-| **Bot** | A persistent persona agent: `bot.yaml` manifest + `permissions.yaml`, sandbox at `~/.mercury/bots/<id>/`, journal-backed history |+| **Bot** | A persistent persona agent: undefinedbot.yamlundefined manifest + `permissions.yaml`, sandbox at `~undefined.mercury/bots/<id>/`, journal-backed history || **Wake** | Turning a bot on for a turn (`/bots run <id> [routine]` or a dispatched task) || **Thread** | The bot's own chat — results, thinking streams, and journal history land here, never in your active chat || **Fleet lead** | A bot promoted to organize its own crew (`/bots promote <id>`) — tell it *what*, it decides *how* and who |