Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: bound and diagnose Windows recycle smoke stages (#13251)
[._.]/ Adam Eivy059d6a602bf4
diff --git a/server/lib/README.md b/server/lib/README.mdindex c7a870f676..e3f51058ef 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -967,9 +967,8 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub- `mediaFileInfo.js`: `withMediaFileSize(kind, record)` / `withMediaFileSizes(items, kind?)` add current original-file byte sizes to bounded image/video pages; missing or unreadable files have no size.- `trashFiles.js`: `trashFiles(paths)` moves existing literal paths to the host OS Trash/Recycle Bin, guards test data, and propagates failures without permanent-delete fallback.-- `nativeTrash.js`: `moveToNativeTrash(paths)` recycles through OS facilities PortOS owns — one hidden, deadline-bound `osascript` (`NSFileManager.trashItemAtURL`, exported `MACOS_TRASH_SCRIPT`) per file on macOS so each gets Finder Put Back; one Windows PowerShell 5.1 `-EncodedCommand` run (shell32 `SHFileOperationW` with `FOF_ALLOWUNDO`, exported `WINDOWS_TRASH_SCRIPT`, paths in `PORTOS_TRASH_PATHS`, fixed-drive pre-flight) per batch; `trashXdg` per file on Linux. Refusals reject with a coded error (`TRASH_REFUSED` + per-file `report`, `TRASH_DEADLINE`, `TRASH_UNPARSABLE`, `TRASH_UNSUPPORTED_PLATFORM`); no local-directory or permanent-delete fallback.+- `nativeTrash.js`: `moveToNativeTrash(paths, { signal }?)` recycles through OS facilities PortOS owns — one hidden, deadline-bound `osascript` (`NSFileManager.trashItemAtURL`, exported `MACOS_TRASH_SCRIPT`) per file on macOS so each gets Finder Put Back; one Windows PowerShell 5.1 `-EncodedCommand` run (shell32 `SHFileOperationW` with `FOF_ALLOWUNDO`, exported `WINDOWS_TRASH_SCRIPT`, paths in `PORTOS_TRASH_PATHS`, fixed-drive pre-flight) per batch; `trashXdg` per file on Linux. Refusals reject with a coded error (`TRASH_REFUSED` + per-file `report`, `TRASH_DEADLINE`, `TRASH_ABORTED` for canceled macOS/Windows children, `TRASH_UNPARSABLE`, `TRASH_UNSUPPORTED_PLATFORM`); no local-directory or permanent-delete fallback.- `nativeTrashXdg.js`: `trashXdg(path)` moves one file into the freedesktop Trash 1.0 directory serving its volume on `node:fs` alone (original names, RFC 2396 `Path=`, `O_EXCL` info reservation, NAME_MAX-bounded names, mount identity from `/proc/self/mountinfo` with a `st_dev` fallback so a bind mount gets its own volume trash instead of an `EXDEV`, physical-parent mount discovery, an owned trash directory is tightened to 0700 and `TRASH_DIR_UNSAFE` refuses a symlinked or foreign-owned one or a volume root whose entries other users could rename, a sticky shared `.Trash` counts only when root or the caller owns it and falls back to `.Trash-<uid>` when it cannot hold the per-user directory, refuses instead of copying across devices); `_topDirTrash(topDir, uid)`, `_parseMountInfo(text)` and `_mountOf(realDir, mounts)` are test hooks.-- `nativeTrash.js`: `moveToNativeTrash(paths)` executes the pinned package native recycling helpers on macOS/Windows with hidden windows and a deadline; Linux uses the package XDG adapter. No local-directory or permanent-delete fallback.- `eventLoopLagProbe.js`: `createEventLoopLagProbe({ clock, intervalMs, toleranceMs, resources, attributeAboveMs })` opens phase-tracking spans that charge overdue time of one shared, span-scoped tick to the current phase, so a slow-operation diagnostic splits each phase into event-loop/host scheduling lag and asynchronous wait (#12329), and a span charged more than `attributeAboveMs` of lag carries `load` (process `cpuMs`, `gcMs`, `invCtx`, host `load1`/`cpus`) to tell a busy process from a starved one (#12852); `beginLoadSample(resources)`, `formatLoadAttribution(load)` and `systemResources` (the same counters for a caller without a span); `systemClock` (globals resolved at call time), `__createManualClock()` (deterministic `advance` = on-time timers vs `stall` = blocked loop, for tests), `roundMs`, and `phaseGroup(summary, name)` (a phase plus its `name.` sub-phases).- `pgJson.js`: text repair for values bound to PostgreSQL, which refuses a lone UTF-16 surrogate in `jsonb` and NUL in `jsonb`/`text` where every file store accepted both. `normalizePgString(s)` → `toWellFormed()` with NUL → U+FFFD (the same lossy repair a UTF-8 file write applied). `toPgJson(value)` → `JSON.stringify` that repairs string values and object keys; **use it, never bare `JSON.stringify`, for a parameter bound to a `json`/`jsonb` column or `::jsonb` cast** (`pgJsonConventions.test.js` enforces this). `normalizePgParams(params)` is applied by `db.js` `query()` and the `withTransaction` client: repairs string and array-element params (never unescapes `\^@`) and serializes plain-object params via `toPgJson`.- `errorLogFingerprint.js`: stderr-log triage primitives for `error-log-triage` (#12851). `groupStderrEntries(text)` strips the PM2 date prefix and folds stack/continuation lines under their head line; `fingerprintEntry(entry, { extraTerms })` redacts the entry (home paths, hosts, IPs, UUIDs, credentials, quoted record names, plus install literals such as the hostname and OS user) and reduces the head to a template (timestamps, ids, SHAs, filesystem paths and numbers become placeholders; API routes stay distinct) with a stable 16-hex fingerprint; `templateSearchExcerpt(template)` picks a tracker search phrase, or null when too generic. Pure; the collector and tracker dedupe live in `services/errorLogTriage.js`.diff --git a/server/lib/nativeTrash.js b/server/lib/nativeTrash.jsindex 4ef56b70bc..c560176270 100644--- a/server/lib/nativeTrash.js+++ b/server/lib/nativeTrash.js@@ -176,6 +176,7 @@ async function runReportingScript(file, args, options, label) {try {({ stdout } = await execFileAsync(file, args, { ...options, timeout: NATIVE_TRASH_DEADLINE_MS }));} catch (error) {+ if (options.signal?.aborted) throw codedError(`${label} recycle aborted`, 'TRASH_ABORTED');if (error.killed) throw codedError(`${label} recycle exceeded the ${NATIVE_TRASH_DEADLINE_MS}ms deadline`, 'TRASH_DEADLINE');throw Object.assign(codedError(`${label} recycle refused`, 'TRASH_REFUSED'), { exitCode: error.code, report: parseReport(error.stdout) });}@@ -184,11 +185,13 @@ async function runReportingScript(file, args, options, label) {return report;}-export async function moveToNativeTrash(paths) {+export async function moveToNativeTrash(paths, { signal } = {}) {+ const cancellation = signal ? { signal } : {};+ signal?.throwIfAborted();switch (process.platform) {case 'darwin':for (const path of paths) {- await runReportingScript('/usr/bin/osascript', ['-l', 'JavaScript', '-e', MACOS_TRASH_SCRIPT, path], {}, 'macOS');+ await runReportingScript('/usr/bin/osascript', ['-l', 'JavaScript', '-e', MACOS_TRASH_SCRIPT, path], cancellation, 'macOS');}return;case 'win32':@@ -196,7 +199,7 @@ export async function moveToNativeTrash(paths) {await runReportingScript(windowsPowerShell51Path(),['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-EncodedCommand', WINDOWS_ENCODED_SCRIPT],- { env: { ...process.env, PORTOS_TRASH_PATHS: paths.join('\n') } },+ { ...cancellation, env: { ...process.env, PORTOS_TRASH_PATHS: paths.join('\n') } },'Windows',);return;diff --git a/server/lib/nativeTrash.smoke.test.js b/server/lib/nativeTrash.smoke.test.jsindex a145a8f835..9e7ce62e7a 100644--- a/server/lib/nativeTrash.smoke.test.js+++ b/server/lib/nativeTrash.smoke.test.js@@ -1,53 +1,138 @@-// Opt-in real Recycle Bin check for the Windows CI job (issue #12325). The-// mocked suite in nativeTrash.test.js cannot notice a Windows update breaking-// the shipped windows-trash.exe, and a developer's real bin must stay untouched-// unless they set PORTOS_NATIVE_TRASH_SMOKE themselves.-import { afterEach, describe, expect, it } from 'vitest';-import { mkdtemp, readdir, readFile, rm, stat, unlink, writeFile } from 'node:fs/promises';-import { realpathSync } from 'node:fs';+// Opt-in real Recycle Bin check for the Windows CI job (issue #12325).+// A developer's real bin stays untouched unless PORTOS_NATIVE_TRASH_SMOKE is set.+import { afterEach, describe, expect, it, vi } from 'vitest';+import { mkdtemp, readdir, readFile, realpath, rm, stat, unlink, writeFile } from 'node:fs/promises';import { tmpdir } from 'node:os';import { join, parse } from 'node:path';-import { execFileSync } from './childProcess.js';+import { promisify } from 'node:util';+import { execFile } from './childProcess.js';import { moveToNativeTrash } from './nativeTrash.js';-// $I<id>: 8-byte version, 8-byte size, 8-byte FILETIME, then (version 2) a-// 4-byte path length and the UTF-16LE original path; version 1 is a padded path.+const execFileAsync = promisify(execFile);++// Keep stage budgets below the existing 30s test deadline, including setup.+// Diagnostics contain stage names/counts only, never SID or bin contents.+async function stage(name, budgetMs, run) {+ const started = performance.now();+ const controller = new AbortController();+ let timer;+ console.log(`♻️ Native trash smoke stage=${name} started budgetMs=${budgetMs}`);+ const deadline = new Promise((_, reject) => {+ timer = setTimeout(() => {+ controller.abort();+ reject(new Error(`Native trash smoke stage=${name} deadline after ${budgetMs}ms`));+ }, budgetMs);+ });+ try {+ return await Promise.race([run(controller.signal), deadline]);+ } catch (error) {+ // Native/fs errors can embed private paths. Keep only their bounded code.+ const code = /^[A-Z_0-9]+$/.test(String(error.code)) ? error.code : 'FAILED';+ throw new Error(`Native trash smoke stage=${name} ${controller.signal.aborted ? 'deadline' : code} after ${Math.round(performance.now() - started)}ms`);+ } finally {+ clearTimeout(timer);+ console.log(`♻️ Native trash smoke stage=${name} finished elapsedMs=${Math.round(performance.now() - started)}`);+ }+}++// $I<id>: version, size, FILETIME, then (v2) path length and UTF-16LE path;+// v1 stores a padded path. Names are random, so snapshot names before recycling+// and read only new records, stopping as soon as this run's unique path matches.const readOriginalPath = (buf) => (buf.readBigUInt64LE(0) >= 2n? buf.toString('utf16le', 28, 28 + (buf.readUInt32LE(24) - 1) * 2): buf.toString('utf16le', 24).replace(/\0.*$/s, ''));+async function findInfoFile(binDir, before, file, signal, { list = readdir, read = readFile } = {}) {+ const names = await list(binDir);+ let candidates = 0;+ for (const name of names) {+ signal.throwIfAborted();+ if (!name.startsWith('$I') || before.has(name)) continue;+ candidates++;+ const original = readOriginalPath(await read(join(binDir, name), { signal }));+ signal.throwIfAborted();+ if (original.toLowerCase() === file.toLowerCase()) return join(binDir, name);+ }+ console.log(`♻️ Native trash smoke bin-scan entries=${names.length} candidates=${candidates} matched=false`);+ throw new Error('No matching new Recycle Bin record');+}++// These controls run on every platform; they do not access the real bin.+describe('native trash smoke lifecycle controls', () => {+ it('names a stalled stage, aborts owned work, and clears its watchdog', async () => {+ vi.useFakeTimers();+ let signal;+ try {+ const pending = stage('bin-scan', 50, value => { signal = value; return new Promise(() => {}); });+ const assertion = expect(pending).rejects.toThrow('stage=bin-scan deadline');+ await vi.advanceTimersByTimeAsync(50);+ await assertion;+ expect(signal.aborted).toBe(true);+ expect(vi.getTimerCount()).toBe(0);+ } finally {+ vi.useRealTimers();+ }+ });++ it('reads only new records and stops at the matching unique path', async () => {+ const file = join('synthetic', 'unique-run', 'target.txt');+ const record = Buffer.alloc(28 + (file.length + 1) * 2);+ record.writeBigUInt64LE(2n);+ record.writeUInt32LE(file.length + 1, 24);+ record.write(file.toUpperCase(), 28, 'utf16le');+ const read = vi.fn().mockResolvedValue(record);+ const list = vi.fn().mockResolvedValue(['$Iold', '$Rold', '$Inew', '$Ilater']);+ await expect(findInfoFile('bin', new Set(['$Iold']), file, new AbortController().signal, { list, read }))+ .resolves.toBe(join('bin', '$Inew'));+ expect(read.mock.calls.map(([path]) => path)).toEqual([join('bin', '$Inew')]);+ });+});+describe.runIf(process.platform === 'win32' && process.env.PORTOS_NATIVE_TRASH_SMOKE)('native recycling adapter on a real Recycle Bin', () => {let dir;let binDir;let infoFile;afterEach(async () => {- // Remove only the pair this test created, then its temp directory.- if (infoFile) {- await unlink(infoFile).catch(() => {});- await unlink(join(binDir, `$R${parse(infoFile).base.slice(2)}`)).catch(() => {});- }- await rm(dir, { recursive: true, force: true });+ await stage('cleanup', 3000, async () => {+ // Remove only the pair this test proved belongs to it.+ if (infoFile) {+ await unlink(infoFile).catch(() => {});+ await unlink(join(binDir, `$R${parse(infoFile).base.slice(2)}`)).catch(() => {});+ }+ if (dir) await rm(dir, { recursive: true, force: true });+ });});it('recycles a temp file and leaves a $I record naming its original path', async () => {- // The shell records the long form; the runner's tmpdir is an 8.3 short path.- dir = await mkdtemp(join(realpathSync.native(tmpdir()), 'portos-native-trash-smoke-'));- const file = join(dir, 'smoke target.txt');- await writeFile(file, 'native trash smoke');- const sid = execFileSync('whoami', ['/user', '/fo', 'csv', '/nh'], { encoding: 'utf8', windowsHide: true })- .trim().split(',')[1].replaceAll('"', '');+ const file = await stage('setup', 2000, async signal => {+ // The shell records the long form; the runner's tmpdir is an 8.3 path.+ const root = await realpath(tmpdir());+ signal.throwIfAborted();+ dir = await mkdtemp(join(root, 'portos-native-trash-smoke-'));+ signal.throwIfAborted();+ const target = join(dir, 'smoke target.txt');+ await writeFile(target, 'native trash smoke', { signal });+ return target;+ });+ const sid = await stage('whoami', 3000, async signal => {+ const { stdout } = await execFileAsync('whoami', ['/user', '/fo', 'csv', '/nh'], { encoding: 'utf8', signal });+ const result = stdout.trim().split(',')[1]?.replaceAll('"', '');+ if (!/^S-\d+(?:-\d+)+$/.test(result)) throw new Error('Invalid SID');+ return result;+ });binDir = join(parse(file).root, '$Recycle.Bin', sid);-- await moveToNativeTrash([file]);-- await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' });- const seen = [];- for (const name of await readdir(binDir)) {- if (!name.startsWith('$I')) continue;- const original = readOriginalPath(await readFile(join(binDir, name)));- seen.push(original);- if (original.toLowerCase() === file.toLowerCase()) infoFile = join(binDir, name);- }- expect(infoFile, `a $I record naming ${file}; bin holds ${JSON.stringify(seen)}`).toBeDefined();+ const before = await stage('bin-snapshot', 3000, async () => new Set(await readdir(binDir).catch(error => {+ // The first recycle on a fresh volume creates its bin.+ if (error.code === 'ENOENT') return [];+ throw error;+ })));+ await stage('recycle', 15000, signal => moveToNativeTrash([file], { signal }));+ await stage('source-check', 1000, async () => {+ await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' });+ });+ await stage('bin-scan', 3000, async signal => {+ infoFile = await findInfoFile(binDir, before, file, signal);+ });+ expect(infoFile, 'a new $I record must name this run’s unique original path').toBeDefined();});});diff --git a/server/lib/nativeTrash.test.js b/server/lib/nativeTrash.test.jsindex ec430d49a6..d9c90cc770 100644--- a/server/lib/nativeTrash.test.js+++ b/server/lib/nativeTrash.test.js@@ -71,6 +71,18 @@ describe('Windows adapter', () => {expect(state.execFile).toHaveBeenCalledTimes(1);});+ it('forwards cancellation to the owned child and reports an abort without retry', async () => {+ restore = pinPlatform('win32');+ const controller = new AbortController();+ state.execFile.mockImplementation(async (...args) => {+ expect(args[2].signal).toBe(controller.signal);+ controller.abort();+ throw refusal({ name: 'AbortError', code: 'ABORT_ERR' });+ });+ await expect(moveToNativeTrash(PATHS, { signal: controller.signal })).rejects.toMatchObject({ code: 'TRASH_ABORTED' });+ expect(state.execFile).toHaveBeenCalledTimes(1);+ });+it('rejects unparsable output as a failure', async () => {restore = pinPlatform('win32');state.execFile.mockResolvedValue({ stdout: 'Add-Type : compile error', stderr: '' });