Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: preserve Persistent Mind attachments during screenshot cleanup
[._.]/ Adam Eivy91fe0c84d708
diff --git a/server/lib/README.md b/server/lib/README.mdindex 50b86e5821..7d731de177 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -79,6 +79,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `agentScratchPaths.js` | Runtime scratch PortOS itself writes into an agent worktree and never commits — `PUBLIC_REVIEW_INPUT_FILENAME` / `PUBLIC_REVIEW_PATCH_DIRNAME` (the public-review bundle materialized by `services/modelAbuseGuard.js`), the `AGENT_SCRATCH_PATHS` list, prefix-aware `matchesScratchRoot(path, roots)` and `isAgentScratchPath(path)`. `classifyWorktreeDirt` (`services/worktreeManager.js`) subtracts these for EVERY caller and uses the same matcher for its per-call `ignoredPaths`, so a worktree holding only scratch reads as clean end to end — before this, it was preserved by `removeWorktree`, held by `reapMergedWorktrees`, filed as a repo-state recovery task, and dispatched to a branch-reconcile coordinator, forever. Only STATIC names live here; a run-scoped name like `.agent-done-<agentId>` (`agentSentinel.js`) is passed per call instead. || `agentSentinel.js` | The `.agent-done` completion sentinel: `DONE_SENTINEL_NAME`, `doneSentinelName(agentId)` → the per-instance filename `.agent-done-<agentId>` (worktree-less agents share one workspace, so a shared name lets concurrent runs clobber — and finalize on — each other's signal), `doneSentinelCandidateNames(agentId)` → the canonical name plus the bounded one-character truncation recovery candidate for generated ids, `doneSentinelPath(workspacePath, agentId)` → the canonical path every producer resolves, `doneSentinelAgentId(filename)` → its inverse (`''` for the bare unscoped name, `null` for a non-sentinel), so the stale-sentinel sweep in `services/agentSentinelSweep.js` matches leftovers to runs the same way the producer named them, + pure `parseSentinelPayload(contents)` → `{ summary, payload }`. Back-compat — a plain-markdown sentinel yields `payload: null`; a JSON object yields its structured `payload` for a programmatic-I/O task type's `processTaskOutput` hook. `salvageSentinelPayload(contents)` (async) is the lenient second tier — runs `jsonExtract` over a fenced/prose-trailed/control-char-corrupted envelope so a less-capable model's near-valid output still surfaces its `payload` instead of being dropped. `extractSentinelPayloadFromTranscript(transcript, isPayload)` (async) is the third tier for programmatic-I/O types ONLY — scans the ANSI-stripped PTY transcript, newest balanced JSON block first, for a payload the model PRINTED instead of writing, and adopts it only if the owning hook's shape predicate accepts it (#3640). `PROGRAMMATIC_OUTPUT_COMPLETION_HEADING` names the briefing section that prints this run's absolute sentinel path — a task-type hook renders its prompt before spawn, so it points the agent at that section by name rather than at a filename it cannot know. || `persistentMind.js` | Pure persistent-CoS-mind runtime state: default-off durable shape, normalization, FIFO-message/self-wake selection, interrupted-wake requeue, accepted-route fingerprints/receipts, image-work rollback guard, bounded exponential backoff, watchdog staleness, and next-wake projection. Staleness alone never kills a turn: `services/persistentMindSupervisor.js` tracks each turn's in-flight provider calls and, before interrupting a stale heartbeat, checks liveness (an open call, and for Ollama-backed providers `/api/ps` still showing the model loaded); a live thought waits with no fixed cap, while a turn with no open call (e.g. after a process restart) or a gone provider is interrupted as before. |+| `persistentMindAttachmentOwnership.js` | Reserved screenshot filename and pending-marker ownership shared by Mind uploads, screenshot retention, and manual purge. || `persistentMindCapabilities.js` | Default-off persistent-mind action grants plus bounded typed CoS-task, private-Eidoverse, and self-cleanup contracts (app/provider/model/effort, PR completion policy, the optional exact provider/model task allowlist, and mind-owned context scopes). || `executionDurationKey.js` | Shared pure execution-duration key composers/parser, separator and absent-effort sentinel, and minimum execution sample threshold; preserves persisted keys for server writers and browser lookups. || `cosAgentListProjection.js` | What a CoS agent LIST response may carry. `toAgentListItem` / `toAgentListItems` drop the transcript and bound `metadata.taskDescription` to `AGENT_LIST_DESCRIPTION_CHARS` (2000, on a clause boundary via `textUtils.clampToCharLimit`), stamping `taskDescriptionTruncated` + the original `taskDescriptionLength` so the UI can hydrate the rest from `GET /api/cos/agents/:id`. That one field was 68% of the Agents tab payload — p50 is 75 characters but a pasted-prompt description runs past 50 KB, and `GET /api/cos/agents` sits on the first-paint critical path of every `/cos/*` tab and re-reads every 30s. Applied at the ROUTE layer only: `getAgents`/`getAgentsByDate` also feed the weekly digest and CoS reports, which need the whole text. |diff --git a/server/lib/index.js b/server/lib/index.jsindex 107639361b..c9cfdd8b28 100644--- a/server/lib/index.js+++ b/server/lib/index.js@@ -910,3 +910,4 @@ export * from './musicDesignerValidation.js';export * from './mediaJobPreparation.js';export * from './eventLoopLagProbe.js';+export * from './persistentMindAttachmentOwnership.js';diff --git a/server/lib/persistentMindAttachmentOwnership.js b/server/lib/persistentMindAttachmentOwnership.jsnew file mode 100644index 0000000000..2dcb94bbc6--- /dev/null+++ b/server/lib/persistentMindAttachmentOwnership.js@@ -0,0 +1,12 @@+// This namespace belongs to Mind even after its bounded metadata is pruned.+// Only the attachment lifecycle may decide that its bytes/markers can be removed.+export const MIND_ATTACHMENT_FILENAME_PREFIX = 'mind-';+export const PENDING_ATTACHMENT_MARKER_PREFIX = '.mind-pending-';+export const PENDING_ATTACHMENT_MARKER_PATTERN = /^\.mind-pending-([A-Za-z0-9_-]{1,128})$/;++export const isPersistentMindScreenshotEntry = (name) => {+ // Named purges can alias lowercase files on case-insensitive filesystems.+ const normalized = name.toLowerCase();+ return normalized.startsWith(MIND_ATTACHMENT_FILENAME_PREFIX)+ || normalized.startsWith(PENDING_ATTACHMENT_MARKER_PREFIX);+};diff --git a/server/services/dataManager.itemPurge.test.js b/server/services/dataManager.itemPurge.test.jsindex 72cd734590..91a364f13b 100644--- a/server/services/dataManager.itemPurge.test.js+++ b/server/services/dataManager.itemPurge.test.js@@ -99,6 +99,21 @@ describe('purgeCategory — category-scoped categories keep their behavior', ()expect(existsSync(dataPath('screenshots'))).toBe(true);});+ it('preserves Mind files during bulk purge and refuses explicit deletion of their reserved entries', async () => {+ const owned = ['mind-claimed-photo.png', 'mind-unindexed-photo.png', '.mind-pending-unindexed'];+ for (const name of owned) writeFileSync(dataPath('screenshots', name), 'owned');+ await purgeCategory('screenshots');+ expect(existsSync(dataPath('screenshots', 'index.json'))).toBe(false);+ for (const name of owned) {+ expect(existsSync(dataPath('screenshots', name))).toBe(true);+ await expect(purgeCategory('screenshots', { subPath: name }))+ .rejects.toMatchObject({ status: 409, code: 'SCREENSHOT_OWNED_BY_MIND' });+ await expect(purgeCategory('screenshots', { subPath: name.toUpperCase() }))+ .rejects.toMatchObject({ status: 409, code: 'SCREENSHOT_OWNED_BY_MIND' });+ expect(existsSync(dataPath('screenshots', name))).toBe(true);+ }+ });+it('removes a named subdirectory recursively', async () => {writeFileSync(dataPath('screenshots', 'account', 'inbox.json'), '{}');await purgeCategory('screenshots', { subPath: 'account' });diff --git a/server/services/dataManager.js b/server/services/dataManager.jsindex df981999f3..acafb3a31f 100644--- a/server/services/dataManager.js+++ b/server/services/dataManager.js@@ -7,6 +7,7 @@ import { execFile } from '../lib/childProcess.js';import { promisify } from 'util';import { parseFilesystemStats, sha256Text } from '../lib/fileCore.js';import { PATHS, ensureDir, isTopLevelEntryName, rmGuarded, writeFileGuarded } from '../lib/fileUtils.js';+import { isPersistentMindScreenshotEntry } from '../lib/persistentMindAttachmentOwnership.js';import { ServerError } from '../lib/errorHandler.js';import { withBackupAssetPublication } from '../lib/backupSnapshotBoundary.js';import {@@ -186,7 +187,7 @@ export const CATEGORIES = {'review': { label: 'Review', description: 'Review Hub todos, action requests and assessment reports', archivable: true, deletable: false },'rigging': { label: 'Rigging Clip Library', description: 'User-dropped animation-bearing GLB source files for retargeting — the only copy of assets you supplied', archivable: false, deletable: false },'runs': { label: 'AI Runs', description: 'Agent run logs and outputs', archivable: true, deletable: true, purgeScope: 'category' },- 'screenshots': { label: 'Screenshots', description: 'Task screenshots and images dropped into shell sessions — shell drops auto-delete after 7 days, others after 30 days unless an open task still uses them', archivable: true, deletable: true, purgeScope: 'category' },+ 'screenshots': { label: 'Screenshots', description: 'Task screenshots and images dropped into shell sessions — shell drops auto-delete after 7 days, others after 30 days unless an open task still uses them; Persistent Mind attachments are retained by Mind', archivable: true, deletable: true, purgeScope: 'category', purgeFilter: name => !isPersistentMindScreenshotEntry(name) },'settings': { label: 'Settings', description: 'Per-feature settings files', archivable: true, deletable: false },'sharing': { label: 'Peer Sync State', description: 'Peer-sync bookkeeping — purging forces a full resync and can resurrect deleted records', archivable: true, deletable: false },'spotify': { label: 'Spotify Sync', description: 'Machine-local Spotify sync cursor and cache — purging resets the cursor and can leave a gap in imported history', archivable: true, deletable: false },@@ -550,6 +551,11 @@ export async function purgeCategory(categoryKey, options = {}) {if (busy) throw new ServerError(busyReason, { status: 409, code: 'CATEGORY_BUSY' });if (wantsItem) {+ if (categoryKey === 'screenshots' && isPersistentMindScreenshotEntry(options.subPath)) {+ throw new ServerError('Persistent Mind owns this attachment or upload recovery marker; manage it through Mind', {+ status: 409, code: 'SCREENSHOT_OWNED_BY_MIND',+ });+ }const resolvedRoot = resolve(dirPath);const resolvedTarget = resolve(join(dirPath, options.subPath));// Boundary-aware containment check: use path.relative so a prefix likediff --git a/server/services/persistentMindAttachments.js b/server/services/persistentMindAttachments.jsindex 99a3366fa3..765ff65822 100644--- a/server/services/persistentMindAttachments.js+++ b/server/services/persistentMindAttachments.js@@ -29,11 +29,15 @@ import {import { backupSnapshotCutPending, withBackupAssetPublication } from '../lib/backupSnapshotBoundary.js';import { loadState, saveState, withStateLock } from './cosState.js';import { isUpdateInProgress } from './updateChecker.js';+import {+ MIND_ATTACHMENT_FILENAME_PREFIX,+ PENDING_ATTACHMENT_MARKER_PREFIX,+ PENDING_ATTACHMENT_MARKER_PATTERN,+} from '../lib/persistentMindAttachmentOwnership.js';-const nowIso = () => new Date().toISOString();+export { PENDING_ATTACHMENT_MARKER_PREFIX, PENDING_ATTACHMENT_MARKER_PATTERN } from '../lib/persistentMindAttachmentOwnership.js';-export const PENDING_ATTACHMENT_MARKER_PREFIX = '.mind-pending-';-export const PENDING_ATTACHMENT_MARKER_PATTERN = /^\.mind-pending-([A-Za-z0-9_-]{1,128})$/;+const nowIso = () => new Date().toISOString();export const attachmentFailure = (error, { code = 'INVALID_ATTACHMENT', status = 400 } = {}) => ({success: false,@@ -79,7 +83,7 @@ export const screenshotEntries = async () => readdir(PATHS.screenshots).then();export const removePendingAttachmentFiles = async (attachmentId, entries) => {- const prefix = `mind-${attachmentId}-`;+ const prefix = `${MIND_ATTACHMENT_FILENAME_PREFIX}${attachmentId}-`;const candidates = entries.filter((entry) => (entry.startsWith(prefix)&& PERSISTENT_MIND_IMAGE_EXTENSIONS.some((extension) => entry.toLowerCase().endsWith(extension))@@ -292,9 +296,9 @@ export async function createPersistentMindAttachment({ filename, data } = {}) {await mkdir(PATHS.screenshots, { recursive: true });await writeFileGuarded(pendingAttachmentMarkerPath(attachmentId), '', { flag: 'wx' });const saved = await saveImageUpload(PATHS.screenshots, {- filename: `mind-${attachmentId}-${originalName}`,+ filename: `${MIND_ATTACHMENT_FILENAME_PREFIX}${attachmentId}-${originalName}`,data,- }, { maxBytes: PERSISTENT_MIND_LIMITS.MAX_ATTACHMENT_BYTES, stagePrefix: `mind-${attachmentId}-` }).then(+ }, { maxBytes: PERSISTENT_MIND_LIMITS.MAX_ATTACHMENT_BYTES, stagePrefix: `${MIND_ATTACHMENT_FILENAME_PREFIX}${attachmentId}-` }).then((value) => value,async (error) => {await removeRejectedUpload(attachmentId);diff --git a/server/services/screenshotsGc.js b/server/services/screenshotsGc.jsindex d30d755b72..1aebea426c 100644--- a/server/services/screenshotsGc.js+++ b/server/services/screenshotsGc.js@@ -5,11 +5,11 @@* image drops (`shell-<id>-<name>`, pasted into a PTY once and never read* again), CoS task screenshots (`POST /api/screenshots`, referenced from a* task's `metadata.screenshots[]`), and vision-test / universe-describe inputs.- * Nothing else ever deletes them, so a photo handed to an agent once stayed on- * disk, in every backup, indefinitely.+ * Persistent Mind attachments share the directory but own their lifecycle.** Rules:* - `shell-*` files older than 7 days are removed — single-use PTY hand-offs.+ * - Mind attachments and pending markers are excluded regardless of age.* - Every other file older than 30 days is removed unless a CoS task that is* neither completed nor failed still lists it in `metadata.screenshots[]`.*@@ -22,6 +22,7 @@import { readdir, stat } from 'fs/promises';import { join, basename } from 'path';import { PATHS, unlinkGuarded } from '../lib/fileUtils.js';+import { isPersistentMindScreenshotEntry } from '../lib/persistentMindAttachmentOwnership.js';import { createSweepScheduler } from './sweepScheduler.js';import { getAllTasks } from './cosTaskStore.js';@@ -70,7 +71,7 @@ export async function _sweepScreenshots({readTasks = getAllTasks,} = {}) {const entries = await readdir(dir, { withFileTypes: true }).catch(() => []);- const files = entries.filter((e) => e.isFile()).map((e) => e.name);+ const files = entries.filter((e) => e.isFile() && !isPersistentMindScreenshotEntry(e.name)).map((e) => e.name);const result = { deleted: 0, keptReferenced: 0, keptYoung: 0 };if (files.length === 0) return result;diff --git a/server/services/screenshotsGc.mindOwnership.test.js b/server/services/screenshotsGc.mindOwnership.test.jsnew file mode 100644index 0000000000..6185ee3aa0--- /dev/null+++ b/server/services/screenshotsGc.mindOwnership.test.js@@ -0,0 +1,76 @@+import { afterAll, beforeEach, expect, it as vitestIt, vi } from 'vitest';+import { mkdir, readFile, readdir, rm, utimes, writeFile } from 'fs/promises';+import { basename, join } from 'path';+import { createTempDataRoot, makePathsProxy, ownTestBodies } from '../lib/mockPathsDataRoot.js';++const root = createTempDataRoot('screenshots-mind-ownership-');+const dir = join(root, 'screenshots');+const stateFile = join(root, 'state.json');+const { it, drain } = ownTestBodies(vitestIt);+const DAY = 24 * 60 * 60 * 1000;+const PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aD1sAAAAASUVORK5CYII=';++vi.mock('../lib/fileUtils.js', async (importOriginal) => ({+ ...makePathsProxy(await importOriginal(), { dataRoot: root }),+ resolveScreenshot: (name) => typeof name === 'string' && basename(name) === name ? join(dir, name) : null,+}));+vi.mock('./cosState.js', () => ({+ loadState: async () => JSON.parse(await readFile(stateFile, 'utf8')),+ saveState: async (state) => writeFile(stateFile, JSON.stringify(state)),+ withStateLock: (fn) => fn(),+}));+vi.mock('./updateChecker.js', () => ({ isUpdateInProgress: () => false }));+vi.mock('./cosTaskStore.js', () => ({ getAllTasks: async () => ({ user: { tasks: [] }, cos: { tasks: [] } }) }));++const { createPersistentMindAttachment, cleanupPersistentMindAttachments } = await import('./persistentMindAttachments.js');+const { _sweepScreenshots } = await import('./screenshotsGc.js');+const { purgeCategory } = await import('./dataManager.js');++beforeEach(async () => {+ await drain();+ await rm(dir, { recursive: true, force: true });+ await mkdir(dir, { recursive: true });+ await writeFile(stateFile, '{}');+});+afterAll(async () => {+ await drain();+ await rm(root, { recursive: true, force: true });+});++it('survives overlapping generic cleanup with unreadable Mind state and leaves expired uploads to the owner', async () => {+ const claimed = await createPersistentMindAttachment({ filename: 'claimed.png', data: PNG });+ const expired = await createPersistentMindAttachment({ filename: 'expired.png', data: PNG });+ expect(claimed.success).toBe(true);+ expect(expired.success).toBe(true);+ const state = JSON.parse(await readFile(stateFile, 'utf8'));+ state.persistentMind.pendingAttachments[0].claimedBy = 'completed-message';+ state.persistentMind.pendingAttachments[0].expiresAt = null;+ const now = Date.now();+ state.persistentMind.pendingAttachments[1].expiresAt = new Date(now - DAY).toISOString();+ await writeFile(join(dir, '.mind-pending-interrupted'), '');+ await writeFile(join(dir, 'mind-interrupted-.stage-image.png'), Buffer.from(PNG, 'base64'));+ await writeFile(join(dir, 'ordinary.png'), 'ordinary');+ for (const name of await readdir(dir)) {+ const old = new Date(now - 90 * DAY);+ await utimes(join(dir, name), old, old);+ }++ // A restore or temporarily unreadable state file cannot surrender ownership.+ await writeFile(stateFile, '{truncated');+ await expect(cleanupPersistentMindAttachments({ now })).rejects.toThrow();+ await Promise.all([_sweepScreenshots({ now }), purgeCategory('screenshots')]);+ expect((await readdir(dir)).sort()).toEqual([+ '.mind-pending-interrupted', claimed.attachment.filename, expired.attachment.filename,+ 'mind-interrupted-.stage-image.png',+ ].sort());++ await writeFile(stateFile, JSON.stringify(state));+ await expect(cleanupPersistentMindAttachments({ now })).resolves.toMatchObject({ success: true, removed: 2 });+ expect(await readdir(dir)).toEqual([claimed.attachment.filename]);++ // Claimed files remain durable after the bounded attachment index is pruned.+ await writeFile(stateFile, '{}');+ await Promise.all([_sweepScreenshots({ now }), purgeCategory('screenshots'), cleanupPersistentMindAttachments({ now })]);+ expect(await readdir(dir)).toEqual([claimed.attachment.filename]);+ await expect(cleanupPersistentMindAttachments({ now })).resolves.toMatchObject({ success: true, removed: 0 });+});diff --git a/server/services/screenshotsGc.test.js b/server/services/screenshotsGc.test.jsindex 4ee6cf4b8e..c2d7febc60 100644--- a/server/services/screenshotsGc.test.js+++ b/server/services/screenshotsGc.test.js@@ -63,6 +63,14 @@ describe('_sweepScreenshots', () => {expect(result).toEqual({ deleted: 2, keptReferenced: 1, keptYoung: 1 });});+ it('leaves old Mind attachments and upload recovery markers to their owning cleanup', async () => {+ const owned = ['mind-claimed-photo.png', 'mind-unindexed-photo.png', '.mind-pending-unindexed'];+ for (const name of [...owned, 'ordinary.png']) await put(name, 90);+ await _sweepScreenshots({ now: NOW, dir, readTasks: tasks([]) });+ for (const name of owned) expect(existsSync(join(dir, name))).toBe(true);+ expect(existsSync(join(dir, 'ordinary.png'))).toBe(false);+ });+it('deletes nothing and logs an error when the task store read throws', async () => {await put('shell-cccc3333-old.png', 30);await put('old.png', 90);