Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
feat([issue-12749]): track dirty ids and serialize only modified CoS agents on saveState
[._.]/ Adam Eivy136831987825
diff --git a/docs/STORAGE.md b/docs/STORAGE.mdindex 6813d0e6c0..f5e0761621 100644--- a/docs/STORAGE.md+++ b/docs/STORAGE.md@@ -159,7 +159,7 @@ The queues federate exactly as before: a full-sync peer advertises its working sDirect editing of the Markdown files and the file watcher are retired (decision 2026-10-20, [plan](./plans/2026-10-20-cos-task-queue-to-postgres.md)); there is no runtime fallback to either. **Migration:** db-migration 018 imports `data/TASKS.md` and `data/COS-TASKS.md` (at the location the install's CoS config named, else the defaults) and, on installs that ran the interim completed-task archive, folds `cos_task_archive` in and drops it; it verifies that every parsed task landed as a row before committing. db-migration 019, which runs only after 018 has committed, renames each file to `<name>.imported` (the recovery copy; nothing reads it) and drops the `userTasksFile` / `cosTasksFile` config keys. Input-derived, so no `data.reference` seed ships for either file. Backed up by the Postgres dump. Repository: `server/services/cosTaskRepository.js` (in-memory backend only under the dev/test escape hatch); store: `server/services/cosTaskStore.js`.-**CoS agent records** (`cos_agents`, `db-primary`, #12738): the agent working set — running agents plus finished ones `archiveStaleAgents()` has not yet moved to `data/cos/agents/<date>/` (retention `completedAgentRetentionMs`, 24 h by default) — is one row per agent, keyed by `agent_id`. The row's `agent` column is the record's exact JSON text (`json`, not `jsonb`, so bytes and key order round-trip and the merge-admission ownership compare stays exact); `task_id`, `status`, `started_at`, `completed_at` and `pid` are projections for queries, and `seq` keeps insertion order. `cosState` keeps its cached `state.agents` map, so callers are unchanged, and `saveState()` diffs it against each record's last stored text (`server/lib/cosAgentRows.js` `planAgentSave`) and writes only the changed rows in one transaction: a status or output update for one agent no longer rewrites every other agent, and the rest of `state.json` (config is in `config.json`) is rewritten only when it changed. `cos_agent_store.version` bumps with every write and a write names the version it was planned against; a stale cache (for example after a database restore) is refused with `COS_AGENTS_STALE` and reloads, rather than deleting rows it never saw. Safety readers (`readAgentsStateForSafetyCheck`, merge admission, the restore gate, abandoned-agent reconciliation) read the rows, never the cache, and an unreadable store is `trusted: false`. The sentinel sweep reads only `{ id, status }` (`listStoredAgentStatuses`). Older finished agents stay in the date-bucketed archive, reached by id through the agent index; they are never loaded wholesale. `state.json` keeps the small runtime remainder (running/paused flags, stats, Persistent Mind runtime, merge leases) behind `withStateLock`. **Intentionally machine-local — never federated.** **Migration:** db-migration 020 imports `state.json`'s `agents` map and verifies every record byte for byte (`agent::text` against its source serialization) before committing, so a failure stops the boot; db-migration 021, which runs only after 020 committed, re-verifies, keeps the original file as `state.json.imported` (the recovery copy; nothing reads it) and rewrites `state.json` without the key. Boot starts CoS before the database phase runs these migrations, so a read of a `state.json` that still carries the map waits (bounded, 2 min, then `503 COS_AGENTS_MIGRATING`) until the boot reports the migrations settled; otherwise an early read would cache an empty agent set and its save would drop the map before 020 imported it. Once the map is gone nothing waits. There is no runtime fallback: a `state.json` that still carries the map after that (a restored pre-migration file) is ignored with a warning and the key is dropped on the next save. Input-derived, so no `data.reference` seed. Stale `state.json.<pid>.<ms>.<uuid>.tmp` files left by a write that was killed are swept once per process on first load (`sweepStaleStateTempFiles`: other processes' files older than a minute). Backed up by the Postgres dump. Repository: `server/services/cosAgentRepository.js` (in-memory backend, persisted to `data/cos-agents.json`, only under the dev/test escape hatch).+**CoS agent records** (`cos_agents`, `db-primary`, #12738): the agent working set — running agents plus finished ones `archiveStaleAgents()` has not yet moved to `data/cos/agents/<date>/` (retention `completedAgentRetentionMs`, 24 h by default) — is one row per agent, keyed by `agent_id`. The row's `agent` column is the record's exact JSON text (`json`, not `jsonb`, so bytes and key order round-trip and the merge-admission ownership compare stays exact); `task_id`, `status`, `started_at`, `completed_at` and `pid` are projections for queries, and `seq` keeps insertion order. `cosState` keeps its cached `state.agents` map, so callers are unchanged, and `saveState()` diffs it against each record's last stored text (`server/lib/cosAgentRows.js` `planAgentSave`), with dirty-agent hints from hot mutation sites (`cosAgentLifecycle`, #12749) serializing only the changed records, and writes only the changed rows in one transaction: a status or output update for one agent no longer rewrites every other agent, and the rest of `state.json` (config is in `config.json`) is rewritten only when it changed. `cos_agent_store.version` bumps with every write and a write names the version it was planned against; a stale cache (for example after a database restore) is refused with `COS_AGENTS_STALE` and reloads, rather than deleting rows it never saw. Safety readers (`readAgentsStateForSafetyCheck`, merge admission, the restore gate, abandoned-agent reconciliation) read the rows, never the cache, and an unreadable store is `trusted: false`. The sentinel sweep reads only `{ id, status }` (`listStoredAgentStatuses`). Older finished agents stay in the date-bucketed archive, reached by id through the agent index; they are never loaded wholesale. `state.json` keeps the small runtime remainder (running/paused flags, stats, Persistent Mind runtime, merge leases) behind `withStateLock`. **Intentionally machine-local — never federated.** **Migration:** db-migration 020 imports `state.json`'s `agents` map and verifies every record byte for byte (`agent::text` against its source serialization) before committing, so a failure stops the boot; db-migration 021, which runs only after 020 committed, re-verifies, keeps the original file as `state.json.imported` (the recovery copy; nothing reads it) and rewrites `state.json` without the key. Boot starts CoS before the database phase runs these migrations, so a read of a `state.json` that still carries the map waits (bounded, 2 min, then `503 COS_AGENTS_MIGRATING`) until the boot reports the migrations settled; otherwise an early read would cache an empty agent set and its save would drop the map before 020 imported it. Once the map is gone nothing waits. There is no runtime fallback: a `state.json` that still carries the map after that (a restored pre-migration file) is ignored with a warning and the key is dropped on the next save. Input-derived, so no `data.reference` seed. Stale `state.json.<pid>.<ms>.<uuid>.tmp` files left by a write that was killed are swept once per process on first load (`sweepStaleStateTempFiles`: other processes' files older than a minute). Backed up by the Postgres dump. Repository: `server/services/cosAgentRepository.js` (in-memory backend, persisted to `data/cos-agents.json`, only under the dev/test escape hatch).**Private security assessments** reuse the existing Review Hub item store for report prose and CoS agent archives for local source/transcripts. They add no store format or database migration. The source inventory stays in run memory; interruption before report validation requires a new assessment. Assessment tasks and all their archive files are excluded from peer federation, and shared socket notifications carry no report prose. Reports follow existing local backup and Review Hub retention; temporary sandbox homes are removed after completion. See [the assessment design and research](research/2026-09-06-private-security-models.md).diff --git a/server/lib/README.md b/server/lib/README.mdindex 542db53c9a..dc353a9d4a 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -546,7 +546,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `taskGenerationOverrides.js` | `applyTaskGenerationOverrides(provider, metadata)` — the per-RUN `temperature` / `thinking` / `effort` a task form or schedule entry pins, layered onto a COPY of the provider record (the saved record is shared by every other run; the child-env composer turns the copy into OpenCode’s dynamic `agent.build` config downstream). Every value arrives untrusted — a COS-TASKS.md round-trip hands back strings — so each override applies only when it validates, and an empty temperature field is absent rather than `0`, which would otherwise pin greedy decoding on a task whose author just cleared the box. || `cosTaskPrompt.js` | The CoS task prompt/note split (#4153) — `metadata.prompt` is the full agent-facing payload, `metadata.context` the one-line human note. `isPromptPayload(value)` is the single discriminator (a newline means it's a prompt, not a note), shared by the store's write path and the on-disk migration so they can't classify the same task differently. `getTaskPrompt(task)` prefers `metadata.prompt` and falls back to a legacy `metadata.context`; `getTaskContextNote(task)` returns the note only on an already-split task; `taskContextBlock(task)` renders both as one block so prompt templates (including customized ones) never see the split; `splitTaskPromptFields(metadata)` routes a multi-line `context` to `prompt` at CREATE time only. Pure, dependency-free. || `cosTaskRows.js` | Row model for the `cos_tasks` queues: the stored form of a task (the old Markdown round-trip's normalization), the recent-completed working-set rule, fractional queue `position` assignment, and `planListSave` (turn "the whole list changed" into "only these rows changed"). Shared by the Postgres and in-memory repository backends so they cannot drift |-| `cosAgentRows.js` | Row model for `cos_agents`, the CoS agent working set: `agentRowColumns(agent)` (the task/status/start/completion/pid projections the queries read) and `planAgentSave({ persisted, next })` (turn "the whole agents map" into "only these rows changed" by comparing each record's JSON text with the one last stored). Shared by `cosState.saveState` and the repository backends |+| `cosAgentRows.js` | Row model for `cos_agents`, the CoS agent working set: `agentRowColumns(agent)` (the task/status/start/completion/pid projections the queries read) and `planAgentSave({ persisted, next, changedAgentIds })` (turn "the whole agents map" into "only these rows changed" by comparing each record's JSON text with the one last stored, serializing only dirty/added records when hinted). Shared by `cosState.saveState` and the repository backends || `taskPauseHold.js` | The user-PAUSE hold — pure metadata helpers shared by the writer (`pauseAgent`), the reader (`resumeAgent`), and the clearer (`updateTask`). `AGENT_PAUSED_CATEGORY` is the `blockedCategory` a pause stamps; `PAUSE_METADATA_KEYS` names every key it writes; `pauseMetadata({agentId, pausedAt, workspacePath, runId})` arms it and `clearedPauseMetadata()` releases it (undefined-valued, so `updateTask` deletes rather than serializing `"null"`); `isAgentPausedTask(task)` / `isResumablePausedTask(task, agentId)` answer whether a pause is still live and whether it is THIS agent's. `updateTask` clears the whole set whenever the task leaves `blocked`, so a revive by any path (dedupe, cooldown expiry, manual unblock) can't leave a task running under a stale pause. Also carries the cycle-safe RELEASE adapter (#3730) `agentManagement.js` registers at load — `resolvePausedTaskResume(task)` hands `updateTask` the paused run's resume pointer before the write (so any un-block resumes on the preserved worktree, not just the Resume dialog) and `retirePausedAgent(agentId, taskId, branchName)` retires the paused record after it. || `taskBlockCategories.js` | The `blockedCategory` vocabulary — which blocks are the system's to clear and which are a person's, read by the pause logic, the failure reaper, and the investigation auto-retry so they can't drift into three literal sets. `PAUSED_BLOCKED_CATEGORIES` (paused until something outside the task changes — keeps the resume pointer), `USER_DECISION_BLOCKED_CATEGORIES` (user intent / open decision — the reaper's exemption), `NON_AUTO_RETRY_BLOCK_CATEGORIES` (their union — a completed investigation must never revive these), `TIMED_COOLDOWN_BLOCKED_CATEGORIES` (a timer clears these — the cooldown sweeper revives them, so block reporters stay quiet), `PROVIDER_CONFIG_BLOCKED_CATEGORY` (the shared literal the agent-spawn stamp site uses, so the stamp cannot drift from the two sets it belongs to). Pure. || `taskRequeue.js` | The REQUEUE stamp (#3376) — pure metadata helpers for the one BACKWARD lifecycle step (`in_progress → pending`, performed by the orphan sweep and the retry-hold release). `REQUEUED_AT_KEY` / `LAST_SPAWNED_AT_KEY` name the two stamps; `isPostSpawnRequeue(pendingTask, inProgressTask)` answers whether the requeue happened strictly AFTER that spawn, which is how the federated merge tells a real requeue apart from an ordinary edit landing on a peer's stale `pending` copy. Returns false when either stamp is missing, so callers fall back to the lifecycle rank. |diff --git a/server/lib/cosAgentRows.js b/server/lib/cosAgentRows.jsindex 9b2cced3f6..369726d082 100644--- a/server/lib/cosAgentRows.js+++ b/server/lib/cosAgentRows.js@@ -29,21 +29,52 @@ export function agentRowColumns(agent) {/*** Diff the next agents map against the persisted serializations.*- * @param {{ persisted: Map<string, string>, next: object }} input+ * @param {{+ * persisted: Map<string, string>,+ * next: object,+ * changedAgentIds?: Iterable<string> | string,+ * serialize?: (agent: object) => string+ * }} input* `persisted` maps agent id to the JSON text last stored for it.+ * `changedAgentIds` is an optional hint of agent ids modified in memory.+ * When provided, only those ids plus any added/removed keys are serialized.+ * When omitted, all records are serialized to detect changes.* @returns {{ upserts: Array<{ id: string, body: string, agent: object }>, deletes: string[], persisted: Map<string, string> }}* `persisted` is what the store holds once the plan is applied.*/-export function planAgentSave({ persisted, next }) {+export function planAgentSave({ persisted, next, changedAgentIds, serialize = JSON.stringify }) {const upserts = [];const after = new Map();- for (const [id, agent] of Object.entries(next ?? {})) {- // An undefined entry is absent, exactly as JSON.stringify of the map drops it.+ const nextEntries = Object.entries(next ?? {});++ if (changedAgentIds == null) {+ for (const [id, agent] of nextEntries) {+ // An undefined entry is absent, exactly as JSON.stringify of the map drops it.+ if (agent === undefined) continue;+ const body = serialize(agent);+ after.set(id, body);+ if (persisted.get(id) !== body) upserts.push({ id, body, agent });+ }+ const deletes = [...persisted.keys()].filter((id) => !after.has(id));+ return { upserts, deletes, persisted: after };+ }++ const dirty = new Set(typeof changedAgentIds === 'string' ? [changedAgentIds] : changedAgentIds);+ for (const [id, agent] of nextEntries) {if (agent === undefined) continue;- const body = JSON.stringify(agent);- after.set(id, body);- if (persisted.get(id) !== body) upserts.push({ id, body, agent });+ const isDirty = dirty.has(id);+ const isPersisted = persisted.has(id);+ if (isDirty || !isPersisted) {+ const body = serialize(agent);+ after.set(id, body);+ if (persisted.get(id) !== body) {+ upserts.push({ id, body, agent });+ }+ } else {+ after.set(id, persisted.get(id));+ }}+const deletes = [...persisted.keys()].filter((id) => !after.has(id));return { upserts, deletes, persisted: after };}diff --git a/server/lib/cosAgentRows.test.js b/server/lib/cosAgentRows.test.jsnew file mode 100644index 0000000000..be02eec042--- /dev/null+++ b/server/lib/cosAgentRows.test.js@@ -0,0 +1,227 @@+import { describe, expect, it, vi } from 'vitest';+import { agentRowColumns, planAgentSave } from './cosAgentRows.js';++describe('agentRowColumns', () => {+ it('projects columns from a full agent record', () => {+ const cols = agentRowColumns({+ taskId: 'task-1',+ status: 'running',+ startedAt: '2026-10-20T18:00:00.000Z',+ completedAt: '2026-10-20T18:05:00.000Z',+ pid: 12345,+ });+ expect(cols).toEqual({+ taskId: 'task-1',+ status: 'running',+ startedAt: '2026-10-20T18:00:00.000Z',+ completedAt: '2026-10-20T18:05:00.000Z',+ pid: 12345,+ });+ });++ it('handles null, undefined, or missing properties cleanly', () => {+ expect(agentRowColumns(null)).toEqual({+ taskId: null,+ status: 'unknown',+ startedAt: null,+ completedAt: null,+ pid: null,+ });+ expect(agentRowColumns({ pid: -1, startedAt: 'invalid-date' })).toEqual({+ taskId: null,+ status: 'unknown',+ startedAt: null,+ completedAt: null,+ pid: null,+ });+ });+});++describe('planAgentSave', () => {+ const agent = (n, extra = {}) => ({+ id: `agent-${n}`,+ status: 'running',+ output: [`line ${n}`],+ ...extra,+ });++ describe('unhinted save (changedAgentIds omitted)', () => {+ it('serializes all records and returns upserts for new/changed and deletes for missing', () => {+ const a1 = agent(1);+ const a2 = agent(2);+ const serializer = vi.fn(JSON.stringify);++ const first = planAgentSave({+ persisted: new Map(),+ next: { 'agent-1': a1, 'agent-2': a2 },+ serialize: serializer,+ });++ expect(serializer).toHaveBeenCalledTimes(2);+ expect(first.upserts.map((u) => u.id)).toEqual(['agent-1', 'agent-2']);+ expect(first.deletes).toEqual([]);++ // Second save: agent-1 unchanged, agent-2 modified, agent-3 added+ serializer.mockClear();+ const a2Modified = { ...a2, status: 'completed' };+ const a3 = agent(3);++ const second = planAgentSave({+ persisted: first.persisted,+ next: { 'agent-1': a1, 'agent-2': a2Modified, 'agent-3': a3 },+ serialize: serializer,+ });++ // All 3 were serialized to diff+ expect(serializer).toHaveBeenCalledTimes(3);+ // Only modified/added are upserted+ expect(second.upserts.map((u) => u.id)).toEqual(['agent-2', 'agent-3']);+ expect(second.deletes).toEqual([]);+ });++ it('identifies deleted records when absent or undefined in next', () => {+ const persisted = new Map([+ ['agent-1', JSON.stringify(agent(1))],+ ['agent-2', JSON.stringify(agent(2))],+ ]);++ const plan = planAgentSave({+ persisted,+ next: { 'agent-1': agent(1), 'agent-2': undefined },+ });++ expect(plan.deletes).toEqual(['agent-2']);+ expect(plan.upserts).toEqual([]);+ expect([...plan.persisted.keys()]).toEqual(['agent-1']);+ });+ });++ describe('hinted save (changedAgentIds provided)', () => {+ it('serializes only the hinted records and avoids serializing untouched records', () => {+ const a1 = agent(1);+ const a2 = agent(2);+ const a3 = agent(3);+ const persisted = new Map([+ ['agent-1', JSON.stringify(a1)],+ ['agent-2', JSON.stringify(a2)],+ ['agent-3', JSON.stringify(a3)],+ ]);++ const serializer = vi.fn(JSON.stringify);+ const a2Updated = { ...a2, output: ['new output'] };++ const plan = planAgentSave({+ persisted,+ next: { 'agent-1': a1, 'agent-2': a2Updated, 'agent-3': a3 },+ changedAgentIds: ['agent-2'],+ serialize: serializer,+ });++ // ONLY agent-2 was serialized! agent-1 and agent-3 were NOT serialized.+ expect(serializer).toHaveBeenCalledTimes(1);+ expect(serializer).toHaveBeenCalledWith(a2Updated);+ expect(plan.upserts.map((u) => u.id)).toEqual(['agent-2']);+ expect(plan.deletes).toEqual([]);+ expect(plan.persisted.get('agent-1')).toBe(persisted.get('agent-1'));+ expect(plan.persisted.get('agent-2')).toBe(JSON.stringify(a2Updated));+ expect(plan.persisted.get('agent-3')).toBe(persisted.get('agent-3'));+ });++ it('serializes newly added records even if omitted from changedAgentIds', () => {+ const a1 = agent(1);+ const persisted = new Map([['agent-1', JSON.stringify(a1)]]);++ const serializer = vi.fn(JSON.stringify);+ const a2New = agent(2);++ const plan = planAgentSave({+ persisted,+ next: { 'agent-1': a1, 'agent-2': a2New },+ changedAgentIds: [], // hint omitted a2New+ serialize: serializer,+ });++ // agent-2 was serialized because it is a newly added key+ expect(serializer).toHaveBeenCalledTimes(1);+ expect(serializer).toHaveBeenCalledWith(a2New);+ expect(plan.upserts.map((u) => u.id)).toEqual(['agent-2']);+ expect(plan.persisted.has('agent-2')).toBe(true);+ });++ it('deletes removed records even if omitted from changedAgentIds', () => {+ const a1 = agent(1);+ const a2 = agent(2);+ const persisted = new Map([+ ['agent-1', JSON.stringify(a1)],+ ['agent-2', JSON.stringify(a2)],+ ]);++ const serializer = vi.fn(JSON.stringify);+ const plan = planAgentSave({+ persisted,+ next: { 'agent-1': a1 }, // agent-2 removed+ changedAgentIds: ['agent-1'],+ serialize: serializer,+ });++ expect(serializer).toHaveBeenCalledTimes(1); // only agent-1 serialized+ expect(plan.deletes).toEqual(['agent-2']);+ expect(plan.persisted.has('agent-2')).toBe(false);+ });++ it('handles changedAgentIds as Set, Array, or single string', () => {+ const a1 = agent(1);+ const persisted = new Map([['agent-1', JSON.stringify(a1)]]);+ const updated = { ...a1, status: 'completed' };++ const plan1 = planAgentSave({+ persisted,+ next: { 'agent-1': updated },+ changedAgentIds: new Set(['agent-1']),+ });+ expect(plan1.upserts).toHaveLength(1);++ const plan2 = planAgentSave({+ persisted,+ next: { 'agent-1': updated },+ changedAgentIds: 'agent-1',+ });+ expect(plan2.upserts).toHaveLength(1);+ });++ it('a hint that omits a really-changed record cannot lose that change on the next unhinted save', () => {+ const a1 = agent(1);+ const a2 = agent(2);+ const persisted = new Map([+ ['agent-1', JSON.stringify(a1)],+ ['agent-2', JSON.stringify(a2)],+ ]);++ // Both records change in memory+ const a1Updated = { ...a1, status: 'completed' };+ const a2Updated = { ...a2, status: 'completed' };+ const workingSet = { 'agent-1': a1Updated, 'agent-2': a2Updated };++ // Step 1: Hinted save that only includes agent-1+ const hintedPlan = planAgentSave({+ persisted,+ next: workingSet,+ changedAgentIds: ['agent-1'],+ });++ expect(hintedPlan.upserts.map((u) => u.id)).toEqual(['agent-1']);+ // Notice: agent-2 in hintedPlan.persisted MUST still hold its OLD serialization!+ expect(hintedPlan.persisted.get('agent-2')).toBe(persisted.get('agent-2'));++ // Step 2: Unhinted save against the persisted map returned by step 1+ const unhintedPlan = planAgentSave({+ persisted: hintedPlan.persisted,+ next: workingSet,+ });++ // The unhinted save detects agent-2 as changed because it wasn't saved in step 1!+ expect(unhintedPlan.upserts.map((u) => u.id)).toEqual(['agent-2']);+ expect(unhintedPlan.persisted.get('agent-2')).toBe(JSON.stringify(a2Updated));+ });+ });+});diff --git a/server/services/cosAgentLifecycle.js b/server/services/cosAgentLifecycle.jsindex b827c079af..c03d91dd55 100644--- a/server/services/cosAgentLifecycle.js+++ b/server/services/cosAgentLifecycle.js@@ -39,7 +39,7 @@ export async function registerAgent(agentId, taskId, metadata = {}) {};state.stats.agentsSpawned++;- await saveState(state);+ await saveState(state, { changedAgentIds: [agentId] });cosEvents.emit('agent:spawned', state.agents[agentId]);return state.agents[agentId];@@ -80,7 +80,7 @@ export async function updateAgent(agentId, updates) {} else {state.agents[agentId] = { ...state.agents[agentId], ...updates };}- await saveState(state);+ await saveState(state, { changedAgentIds: [agentId] });// Completion writes the archive before later completion hooks enrich an// agent's metadata (task summaries, scan reports, etc.). Mirror those@@ -238,7 +238,7 @@ async function updateCompletionReceipt(agentId, update) {const { output: _output, ...record } = updated;await atomicWrite(join(getAgentDir(agentId, date), 'metadata.json'), record);}- if (state.agents[agentId]) { state.agents[agentId] = updated; await saveState(state); }+ if (state.agents[agentId]) { state.agents[agentId] = updated; await saveState(state, { changedAgentIds: [agentId] }); }return true;}));}@@ -323,7 +323,7 @@ export async function completeAgent(agentId, result = {}) {state.stats.errors = (state.stats.errors || 0) + 1;}- await saveState(state);+ await saveState(state, { changedAgentIds: [agentId] });// `agent:completed` is intentionally emitted later, after the domain-usage// ledger is updated (see the recordDomainUsage block below, #1683). Do NOT// move it back here. `agent:updated` carries no scheduling side effect, so it@@ -385,7 +385,7 @@ export async function appendAgentOutput(agentId, line) {state.agents[agentId].output = state.agents[agentId].output.slice(-1000);}- await saveState(state);+ await saveState(state, { changedAgentIds: [agentId] });return state.agents[agentId];});@@ -412,7 +412,7 @@ export async function appendAgentOutputLines(agentId, lines) {if (state.agents[agentId].output.length > 1000) {state.agents[agentId].output = state.agents[agentId].output.slice(-1000);}- await saveState(state);+ await saveState(state, { changedAgentIds: [agentId] });return state.agents[agentId];});@@ -766,7 +766,7 @@ export async function sendBtwToAgent(agentId, message) {if (state.agents[agentId].btwMessages.length > 50) {state.agents[agentId].btwMessages = state.agents[agentId].btwMessages.slice(-50);}- await saveState(state);+ await saveState(state, { changedAgentIds: [agentId] });});cosEvents.emit('agent:btw', { agentId, message, timestamp });@@ -880,7 +880,7 @@ export async function cleanupZombieAgents() {}if (cleaned.length > 0) {- await saveState(state);+ await saveState(state, { changedAgentIds: cleaned });// Persist zombie-cleaned agents to date-bucketed dirs and update indexconst idx = await loadAgentIndex();@@ -945,7 +945,7 @@ export async function deleteAgent(agentId) {const wasRunning = state.agents[agentId]?.status === 'running';delete state.agents[agentId];- await saveState(state);+ await saveState(state, { changedAgentIds: [agentId] });// Persist reference removal before deleting bytes. Failed index writes// retain both the disk tree and its cached lookup for a safe retry.diff --git a/server/services/cosState.js b/server/services/cosState.jsindex fd99d93e80..bbc84b3866 100644--- a/server/services/cosState.js+++ b/server/services/cosState.js@@ -675,8 +675,11 @@ export async function readAgentsStateForSafetyCheck() {** `onStage` is called as each persistence stage begins (`prepare`, `serialize`,* `write`, `replace`, `dirSync`) for a caller's own slow-operation diagnostics.+ *+ * `changedAgentIds` optionally hints which agent records were modified, avoiding+ * serializing untouched agents in the working set (#12749).*/-export async function saveState(state, { onStage } = {}) {+export async function saveState(state, { onStage, changedAgentIds } = {}) {assertCurrentGeneration(state, stateRecovery);stampGeneration(state, stateRecovery);const span = stateLagProbe.begin('prepare');@@ -693,7 +696,11 @@ export async function saveState(state, { onStage } = {}) {// Agent records are rows: write only those whose serialization changed, so// one agent's update never rewrites another's (#12738).stage('agents');- const plan = planAgentSave({ persisted: agentStore.persisted, next: isPlainObject(state.agents) ? state.agents : {} });+ const plan = planAgentSave({+ persisted: agentStore.persisted,+ next: isPlainObject(state.agents) ? state.agents : {},+ changedAgentIds,+ });if (plan.upserts.length > 0 || plan.deletes.length > 0) {agentStore = { version: await applyAgentSave(plan, agentStore.version), persisted: plan.persisted };bytes += plan.upserts.reduce((sum, row) => sum + Buffer.byteLength(row.body), 0);diff --git a/server/services/cosState.test.js b/server/services/cosState.test.jsindex 2b98a204fb..bdde061301 100644--- a/server/services/cosState.test.js+++ b/server/services/cosState.test.js@@ -578,6 +578,81 @@ describe('agent records in cos_agents', () => {expect(Object.keys((await store.loadState()).agents)).toEqual([agent(2).id]);});+ it('serializes only hinted records on saveState with changedAgentIds', async () => {+ const store = await freshModule();+ const agents = Object.fromEntries(Array.from({ length: 5 }, (_, n) => [agent(n).id, agent(n)]));+ await seedAgents(agents);+ const state = await store.loadState();+ await store.saveState(state);++ state.agents[agent(2).id] = { ...state.agents[agent(2).id], output: ['hinted-line'] };+ const stringifySpy = vi.spyOn(JSON, 'stringify');++ try {+ await store.saveState(state, { changedAgentIds: [agent(2).id] });+ const serializedAgentIds = stringifySpy.mock.calls+ .map(([arg]) => arg?.id)+ .filter((id) => typeof id === 'string' && id.startsWith('agent-example-'));+ expect(serializedAgentIds).toEqual([agent(2).id]);+ } finally {+ stringifySpy.mockRestore();+ }++ const stored = await storedAgents();+ expect(stored[agent(2).id].output).toEqual(['hinted-line']);+ });++ it('preserves an omitted changed agent across a hinted save so the next unhinted save detects it', async () => {+ const store = await freshModule();+ await seedAgents({ [agent(1).id]: agent(1), [agent(2).id]: agent(2) });+ const state = await store.loadState();+ await store.saveState(state);++ // Both are modified in memory+ state.agents[agent(1).id] = { ...state.agents[agent(1).id], status: 'completed' };+ state.agents[agent(2).id] = { ...state.agents[agent(2).id], status: 'failed' };++ // Hinted save omits agent(2)+ await store.saveState(state, { changedAgentIds: [agent(1).id] });+ let stored = await storedAgents();+ expect(stored[agent(1).id].status).toBe('completed');+ expect(stored[agent(2).id].status).toBe('running'); // not saved yet!++ // Next save runs unhinted: detects the previously omitted agent(2) change+ await store.saveState(state);+ stored = await storedAgents();+ expect(stored[agent(2).id].status).toBe('failed');+ });++ it('scales efficiently with a synthetic 219-agent fixture: serializes only the hinted record', async () => {+ const store = await freshModule();+ const agents = Object.fromEntries(Array.from({ length: 219 }, (_, n) => [+ agent(n).id,+ agent(n, { output: Array.from({ length: 50 }, (_, i) => `line ${i} for agent ${n} with padding text`) }),+ ]));+ await seedAgents(agents);+ const state = await store.loadState();+ await store.saveState(state);++ const targetId = agent(42).id;+ state.agents[targetId] = { ...state.agents[targetId], output: ['new output line'] };++ const stringifySpy = vi.spyOn(JSON, 'stringify');+ try {+ await store.saveState(state, { changedAgentIds: [targetId] });+ const serializedAgentIds = stringifySpy.mock.calls+ .map(([arg]) => arg?.id)+ .filter((id) => typeof id === 'string' && id.startsWith('agent-example-'));+ // Only the single hinted agent was serialized, NOT all 219 agents+ expect(serializedAgentIds).toEqual([targetId]);+ } finally {+ stringifySpy.mockRestore();+ }++ const stored = await storedAgents();+ expect(stored[targetId].output).toEqual(['new output line']);+ });+it('sweeps stale state.json temp files from other processes, never its own or a fresh one', async () => {const uuid = '0123abcd-0123-4567-89ab-0123456789ab';const names = {