Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
test: drain detached settings ledger writes before a suite removes its temp data root
[._.]/ Adam Eivy61f89add3295
diff --git a/server/lib/README.md b/server/lib/README.mdindex 3692cf954e..3120005b55 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -392,7 +392,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `boundedStateMap.js` | `createBoundedStateMap({ maxSize, ttlMs })` — a `Map` that self-evicts so a per-key state cache can't grow one entry per key forever. Lazy TTL + LRU eviction on `get`/`set` (no timers); `get` doubles as the LRU touch. Used by the moltbook/moltworld rate-limiters to bound their per-agent state. || `createKeyCachedQueue.js` | Per-KEY serialized async work queue (sibling to `fileWriteQueue.js`'s single tail). `createKeyCachedQueue()` returns `queue(key, work)` that chains each `work` thunk onto the prior in-flight promise for that `key` — same-key work runs one-after-another (later sees earlier's committed result), different keys run concurrently. Self-pruning tail Map; `work` runs on both fulfil and reject so one failure can't stall the chain; carries `.clear()` for test reset. Used by the media-job completion hooks (writers-room / catalog / music-video scene-image attach) to serialize per-record. || `createNewestWinsGuard.js` | Newest-render-wins ordering guard for out-of-order async completions. `createNewestWinsGuard()` returns `{ isStale(key, at), mark(key, at), clear() }` — tracks the newest applied `queuedAt` per slot `key` so an older render completing after a newer one is dropped (`isStale` true) instead of clobbering the newer frame. ISO timestamps compare as strings; absent `at` is never stale. Used by `createMediaJobImageHook`'s opt-in guard and the catalog hook's portrait slot. |-| `fileWriteQueue.js` | Serialize read-modify-write cycles so they don't interleave. `createFileWriteQueue()` → single-tail `queue(fn)` for one shared file; `createRecordWriteQueue(assertId?)` → id-keyed `queueRecordWrite(id, fn)` where same-id cycles serialize and different-id cycles run in parallel (the queue the PG/file store facades use for `collectionStore.queueRecordWrite` parity); `createKeyedFileWriteQueue()` → `queueKeyedWrite(key, fn)` that collapses the pipeline stage stores' hand-rolled `Map` + per-key `createFileWriteQueue` factory (falsy keys → shared `'__unknown__'` tail; self-pruning via `createKeyCachedQueue`). |+| `fileWriteQueue.js` | Serialize read-modify-write cycles so they don't interleave. `createFileWriteQueue()` → single-tail `queue(fn)` for one shared file; `createRecordWriteQueue(assertId?)` → id-keyed `queueRecordWrite(id, fn)` where same-id cycles serialize and different-id cycles run in parallel (the queue the PG/file store facades use for `collectionStore.queueRecordWrite` parity); `createKeyedFileWriteQueue()` → `queueKeyedWrite(key, fn)` that collapses the pipeline stage stores' hand-rolled `Map` + per-key `createFileWriteQueue` factory (falsy keys → shared `'__unknown__'` tail; self-pruning via `createKeyCachedQueue`); `trackDetachedWrite(promise)` / `drainDetachedWrites()` → a `globalThis`-backed registry for bookkeeping a caller schedules off a queued turn and never awaits (the settings ledger row), so a teardown or shutdown can wait for it. || `forgeIssueCli.js` | Pure `gh`/`glab` issue-filing argument shapes and output parsing — `forgeLabelCreateArgs`, `forgeIssueCreateArgs`, `parseCreatedForgeIssue` — so the two CLIs' body-flag, label-attachment, and label-create disagreements have one definition. || `forgeIssueState.js` | `normalizeIssueState(state)` — collapse a forge issue/PR/MR state string to `open`/`closed`. GitHub reports `OPEN`/`CLOSED`; GitLab reports `opened`/`closed`/`locked`; anything unrecognized is treated as open so an unfamiliar state can never read as "already resolved". Shared by `layeredIntelligence/forgeFiler.js` and `services/blockedIssueReconcile.js`. || `imageClean.js` | `cleanImageBuffer(buf, { metadata, denoise })` (composable opt-in pipeline: lossless metadata/C2PA strip + optional median/sharpen denoise) · `stripPngMetadataChunks` / `stripPngC2PAChunk` (lossless PNG-chunk removers) · `compositeIgnoreZone(base, original, mask, { feather })` (preserve-region compositing: restore original pixels into a feathered mask over a diffused result) · `autoCleanGeneratedImage` (in-place clean for post-generation hook). HTTP route in `routes/imageClean.js` wraps `cleanImageBuffer` and appends a CPU light diffusion pass (`applyLightRegen` from `services/imageGen/regen.js`) for the `diffusion=light` SynthID-disruption step. |diff --git a/server/lib/fileWriteQueue.js b/server/lib/fileWriteQueue.jsindex b32a4c27fe..b5d90a499a 100644--- a/server/lib/fileWriteQueue.js+++ b/server/lib/fileWriteQueue.js@@ -107,3 +107,26 @@ export function createKeyedFileWriteQueue() {return queue(safeKey, fn); // callers see the real resolve/reject};}++/**+ * Detached writes — bookkeeping a caller schedules off a queued turn and does+ * not await (the settings ledger row, #12018) — still need one drain point:+ * a test teardown that removes the data root the write targets, or a shutdown.+ * The registry lives on `globalThis` so the module instances `vi.resetModules()`+ * creates share it; entries remove themselves on settle, fulfilled or rejected.+ */+const DETACHED_WRITES = Symbol.for('portos.detachedWrites');+const detachedWrites = globalThis[DETACHED_WRITES] ??= new Set();++/** Track a detached write's promise; the caller still owns its rejection. */+export function trackDetachedWrite(promise) {+ detachedWrites.add(promise);+ const release = () => detachedWrites.delete(promise);+ promise.then(release, release);+ return promise;+}++/** Await every detached write tracked so far, including ones scheduled meanwhile. */+export async function drainDetachedWrites() {+ while (detachedWrites.size) await Promise.allSettled([...detachedWrites]);+}diff --git a/server/services/settings.js b/server/services/settings.jsindex b4630c77dd..6c02dc7177 100644--- a/server/services/settings.js+++ b/server/services/settings.js@@ -2,7 +2,7 @@ import { hydratePrivateKeys, persistPrivateKeys } from './privateKeyStore.js';import { join } from 'path';import { EventEmitter } from 'events';import { safeJSONParse, PATHS, atomicWrite, tryReadFile, tryReadFileStrict } from '../lib/fileUtils.js';-import { createFileWriteQueue } from '../lib/fileWriteQueue.js';+import { createFileWriteQueue, drainDetachedWrites, trackDetachedWrite } from '../lib/fileWriteQueue.js';import { canonicalStringify, isPlainObject, POLLUTING_KEYS } from '../lib/objects.js';// `./userActions.js` is imported lazily at its single call site below, NOT here.// It reaches the DB schema/query layer, and settings.js is imported by nearly@@ -301,20 +301,20 @@ const save = async (settings, { actor = 'system', skipUserAction = false } = {},return cleaned;};-// Ledger writes in flight, so `flushSettingsLedger` can await them.-const pendingLedgerWrites = new Set();-// Record a settings change in the operator-action ledger (#5594) as detached// bookkeeping: the row is a Postgres write, and awaiting it inside the queued// save turn made a DB outage stall every later settings write for the pool// connect timeout and then reject a save whose file was already on disk — which// in turn skipped setPassword's session revocation (#12018). The ledger is an// audit trail, not part of the save's success contract, so the chain is not-// awaited by the caller and owns its own rejection.+// awaited by the caller and owns its own rejection. It is tracked so a drain+// point (`flushSettingsLedger`, the test runner's per-test teardown) can wait+// for it: a suite that removes its temp data root under an in-flight row would+// otherwise see the row recreate the directory and leak it.const recordSettingsChange = (actor, change) => {const happenedAt = new Date().toISOString();const changedKeys = change.keysChanged.join(',');- const write = import('./userActions.js')+ trackDetachedWrite(import('./userActions.js').then(({ recordUserAction }) => recordUserAction({type: 'settings.update',actor,@@ -331,15 +331,13 @@ const recordSettingsChange = (actor, change) => {})).catch((error) => {console.error(`❌ Settings ledger write failed for keys ${changedKeys}: ${error.message}`);- })- .finally(() => pendingLedgerWrites.delete(write));- pendingLedgerWrites.add(write);+ }));};// Await every ledger write scheduled so far. The writes are detached from the// save that scheduled them, so a caller that must observe the row (a test, a// shutdown drain) waits here rather than on the save's promise.-export const flushSettingsLedger = () => Promise.all([...pendingLedgerWrites]).then(() => undefined);+export const flushSettingsLedger = () => drainDetachedWrites();/*** `getSettings`, but reporting whether the snapshot came from a CORRUPT read.diff --git a/server/services/settings.test.js b/server/services/settings.test.jsindex d295406083..f3c411bba2 100644--- a/server/services/settings.test.js+++ b/server/services/settings.test.js@@ -297,8 +297,10 @@ describe('settings.js', () => {let current = {};tryReadFile.mockImplementation(async () => JSON.stringify(current));atomicWrite.mockImplementation(async (_path, content) => { current = JSON.parse(content); });- // A ledger write that never settles (Postgres pool waiting on a dead host).- recordUserAction.mockReturnValueOnce(new Promise(() => {}));+ // A ledger write that does not settle (Postgres pool waiting on a dead+ // host) until the test releases it, so the per-test drain can finish.+ let release;+ recordUserAction.mockReturnValueOnce(new Promise((resolve) => { release = resolve; }));await updateSettings({ first: 1 }, { actor: 'user' });await expect(updateSettings({ second: 2 }, { actor: 'user' })).resolves.toEqual({ first: 1, second: 2 });@@ -307,6 +309,7 @@ describe('settings.js', () => {// its resolved save); the first is still in flight.await new Promise((resolve) => setImmediate(resolve));expect(recordUserAction).toHaveBeenCalledTimes(2);+ release();});});diff --git a/server/vitest.setup.js b/server/vitest.setup.jsindex 4d524918d0..626ac52558 100644--- a/server/vitest.setup.js+++ b/server/vitest.setup.js@@ -50,9 +50,10 @@* suites (Option B) for working examples.*/-import { vi, afterAll } from 'vitest';+import { vi, afterAll, afterEach } from 'vitest';import { rmSync } from 'fs';import { mockNoPeers } from './lib/mockPathsDataRoot.js';+import { drainDetachedWrites } from './lib/fileWriteQueue.js';import './test/admissionSetup.js';// A per-file environment override must not silently bypass registry isolation.@@ -81,6 +82,12 @@ vi.mock('./services/instances.js', async (importOriginal) => {return mockNoPeers(actual);});+// A settings save schedules its operator-action ledger row off the queued+// turn (#12018); this runs after each test's own afterEach and before the+// suite's afterAll, so a teardown that removes the temp data root never races+// an in-flight row that would recreate the directory as a `test temp leak`.+afterEach(() => drainDetachedWrites());+// Wipe the shared git fixture templates (`lib/gitTestRepo.js`) after each// test file (#9000). Vitest isolates the module graph per file, so every file// that uses the helper builds its own template, and the helper's