Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

fix: floating async calls no longer leave unowned rejections (#12978) (#13049)

[._.]/f1615d1da945
diff --git a/scripts/ci-test-plan.js b/scripts/ci-test-plan.js
index cecf01a93f..32824c2860 100644
--- a/scripts/ci-test-plan.js
+++ b/scripts/ci-test-plan.js
@@ -515,6 +515,7 @@ export const ALWAYS_RUN_TESTS = [
// test imports, with no import edge from that router to this guard.
'server/routeTestCoverageConventions.test.js',
'server/routeValidationConventions.test.js',
+ 'server/floatingAsyncConventions.test.js',
'server/timerCallbackConventions.test.js',
// Whole-tree scanner (#12815): any server file can bind a bare JSON.stringify to
// a jsonb parameter, with no import edge to this guard.
diff --git a/scripts/lib/testMemo.js b/scripts/lib/testMemo.js
index 922d9f1417..4f0e703639 100644
--- a/scripts/lib/testMemo.js
+++ b/scripts/lib/testMemo.js
@@ -112,6 +112,7 @@ export const TEST_MEMO_NEVER = Object.freeze({
'server/childListenerOrderConventions.test.js': 'scans the whole server tree',
'server/routeTestCoverageConventions.test.js': 'scans server/index.js mounts against every test',
'server/routeValidationConventions.test.js': 'scans every route module',
+ 'server/floatingAsyncConventions.test.js': 'scans the whole server tree',
'server/timerCallbackConventions.test.js': 'scans the whole server tree',
'client/src/lib/index.test.js': 'enumerates the client/src/lib directory against its README and barrel',
'client/src/hooks/index.test.js': 'enumerates the client/src/hooks directory against its README and barrel',
diff --git a/server/AGENTS.md b/server/AGENTS.md
index df4f826a34..2153db059a 100644
--- a/server/AGENTS.md
+++ b/server/AGENTS.md
@@ -26,6 +26,7 @@ These apply to Express/service/model code. Universal constraints (no try/catch o
## Timers
- **An `async` timer callback must own its rejections.** Inside `setTimeout(async …)` / `setInterval(async …)`, every `await` needs either an enclosing `try { … } catch` or a `.catch(…)` on the awaited chain. A timer fires outside the request lifecycle, so nothing holds the promise the callback returns and a rejection escapes as an unhandled rejection — fatal on Node >= 15. The CoS runner shipped that at boot: its orphan-cleanup timer awaited a state-file read/write, so one truncated `agents` file killed the runner seconds after start and PM2 restart-looped it against the same file (#5668). Log and continue in the catch unless the callback genuinely owns a failure verdict. `server/timerCallbackConventions.test.js` scans the whole server tree for this and carries no allowlist; the worked example is the watchdog in `server/services/mediaJobQueue/index.js`.
+- **A statement-level call to an `async` function must be `await`ed, end in `.catch(…)`, or start with `void`.** A bare `announceSelf(peer);` leaves a rejection with no owner, which becomes an `unhandledRejection` that `autoFixer` turns into an AI task for what was best-effort bookkeeping (#12978). `void` means "this function owns its rejections, fire-and-forget on purpose" — make that true first. `server/floatingAsyncConventions.test.js` resolves module-scope async functions (local and relatively imported) across the server tree and carries no allowlist.
## Child processes
diff --git a/server/floatingAsyncConventions.test.js b/server/floatingAsyncConventions.test.js
new file mode 100644
index 0000000000..289eeb85a6
--- /dev/null
+++ b/server/floatingAsyncConventions.test.js
@@ -0,0 +1,178 @@
+/**
+ * Repo-wide guard: a statement-level call to an `async` function must say who
+ * owns its rejection (#12978).
+ *
+ * announceSelf(peer); // ← bare: a rejection before its own catch is unowned
+ *
+ * Nobody holds the promise, so a rejection becomes an `unhandledRejection`.
+ * `setupProcessErrorHandlers` tags that critical, broadcasts it to the UI and
+ * `autoFixer` files an AI task for what was meant to be best-effort bookkeeping.
+ *
+ * ## The rule
+ *
+ * An `ExpressionStatement` that calls an `async` function must be `await`ed,
+ * end in `.catch(…)`, or start with `void`. `void` is the explicit spelling of
+ * "this function owns its rejections and is intentionally fire-and-forget" —
+ * confirm that is true before adding it.
+ *
+ * "Async function" means a module-scope `async` function declaration / `const f =
+ * async …` (nested helpers are not resolved — names shadow across scopes), or a name imported from a relative module whose same-named export
+ * is one of those. Calls through member access (`obj.run()`), namespace imports
+ * and re-exports are not resolved. A local binding that shadows an async name
+ * would be flagged; rename it.
+ *
+ * There is no allowlist: fix the call site.
+ */
+
+import { describe, it, expect } from 'vitest';
+import { execFileSync } from 'child_process';
+import { existsSync, readFileSync } from 'fs';
+import { dirname, join, resolve } from 'path';
+import { fileURLToPath } from 'url';
+import { parse } from '@babel/parser';
+
+const SERVER_ROOT = dirname(fileURLToPath(import.meta.url));
+
+const parseSource = (src) => parse(src, { sourceType: 'module', errorRecovery: true, allowReturnOutsideFunction: true });
+
+const isAsyncInit = (node) => (
+ (node?.type === 'ArrowFunctionExpression' || node?.type === 'FunctionExpression') && node.async === true
+);
+
+function walk(node, visit) {
+ if (!node || typeof node.type !== 'string') return;
+ visit(node);
+ for (const key of Object.keys(node)) {
+ if (key === 'loc' || key === 'extra') continue;
+ const child = node[key];
+ if (Array.isArray(child)) child.forEach((c) => walk(c, visit));
+ else if (child && typeof child === 'object') walk(child, visit);
+ }
+}
+
+/** Module-scope async functions declared in `ast`, plus which are exported. Nested ones are skipped: the same name is routinely a sync helper in a sibling scope. */
+function asyncDeclarations(ast) {
+ const local = new Set();
+ const exported = new Set();
+ const addDeclaration = (decl) => {
+ if (decl?.type === 'FunctionDeclaration' && decl.async && decl.id) local.add(decl.id.name);
+ if (decl?.type === 'VariableDeclaration') {
+ for (const d of decl.declarations) {
+ if (d.id.type === 'Identifier' && isAsyncInit(d.init)) local.add(d.id.name);
+ }
+ }
+ };
+ for (const stmt of ast.program.body) {
+ addDeclaration(stmt);
+ if (stmt.type === 'ExportNamedDeclaration') addDeclaration(stmt.declaration);
+ }
+ for (const stmt of ast.program.body) {
+ if (stmt.type !== 'ExportNamedDeclaration') continue;
+ const decl = stmt.declaration;
+ if (decl?.type === 'FunctionDeclaration' && decl.async) exported.add(decl.id.name);
+ if (decl?.type === 'VariableDeclaration') {
+ for (const d of decl.declarations) {
+ if (d.id.type === 'Identifier' && isAsyncInit(d.init)) exported.add(d.id.name);
+ }
+ }
+ for (const spec of stmt.specifiers || []) {
+ if (!stmt.source && local.has(spec.local.name)) exported.add(spec.exported.name || spec.exported.value);
+ }
+ }
+ return { local, exported };
+}
+
+const exportCache = new Map();
+
+function asyncExportsOf(absPath) {
+ if (!exportCache.has(absPath)) {
+ const candidates = [absPath, `${absPath}.js`, join(absPath, 'index.js')];
+ const file = candidates.find((p) => p.endsWith('.js') && existsSync(p) && !p.endsWith('/'));
+ exportCache.set(absPath, file ? asyncDeclarations(parseSource(readFileSync(file, 'utf8'))).exported : new Set());
+ }
+ return exportCache.get(absPath);
+}
+
+/**
+ * Lines of every bare `asyncFn();` statement in `src`. `dir` resolves relative
+ * imports; omit it to scan local declarations only.
+ */
+export function findFloatingAsyncCalls(src, dir = null) {
+ const ast = parseSource(src);
+ const asyncNames = new Set(asyncDeclarations(ast).local);
+ if (dir) {
+ for (const stmt of ast.program.body) {
+ if (stmt.type !== 'ImportDeclaration' || !stmt.source.value.startsWith('.')) continue;
+ const exported = asyncExportsOf(resolve(dir, stmt.source.value));
+ for (const spec of stmt.specifiers) {
+ if (spec.type === 'ImportSpecifier' && exported.has(spec.imported.name || spec.imported.value)) {
+ asyncNames.add(spec.local.name);
+ }
+ }
+ }
+ }
+ const hits = [];
+ walk(ast, (node) => {
+ if (node.type !== 'ExpressionStatement') return;
+ const call = node.expression;
+ if (call.type === 'CallExpression' && call.callee.type === 'Identifier' && asyncNames.has(call.callee.name)) {
+ hits.push(`line ${node.loc.start.line}: ${call.callee.name}()`);
+ }
+ });
+ return hits;
+}
+
+const trackedServerSources = () => execFileSync('git', ['ls-files', '*.js', '*.mjs', '*.cjs'], {
+ cwd: SERVER_ROOT,
+ encoding: 'utf8',
+ maxBuffer: 64 * 1024 * 1024,
+}).split('\n').filter((f) => f && !f.includes('.test.'));
+
+describe('async calls own their rejections (#12978)', () => {
+ it('scans the server tree', () => {
+ expect(trackedServerSources().length).toBeGreaterThan(200);
+ });
+
+ it('has no bare statement-level call to an async function', () => {
+ const violations = [];
+ for (const file of trackedServerSources()) {
+ const src = readFileSync(join(SERVER_ROOT, file), 'utf8');
+ for (const hit of findFloatingAsyncCalls(src, dirname(join(SERVER_ROOT, file)))) {
+ violations.push(`server/${file} ${hit}`);
+ }
+ }
+ expect(
+ violations,
+ 'These statements call an `async` function without `await`, `.catch(…)` or `void`, so a '
+ + 'rejection has no owner and becomes an unhandledRejection (which the auto-fixer turns into '
+ + 'an AI task). Fix: make the callee own its rejections and prefix the call with `void`, or '
+ + `append \`.catch(…)\`.\nOffenders:\n ${violations.join('\n ')}`,
+ ).toEqual([]);
+ });
+});
+
+describe('the floating-async recognizer', () => {
+ it('flags a bare call to a local async function, declaration or arrow', () => {
+ expect(findFloatingAsyncCalls('async function f() {}\nfunction g() { f(); }')).toEqual(['line 2: f()']);
+ expect(findFloatingAsyncCalls('const f = async () => {};\nf();')).toEqual(['line 2: f()']);
+ });
+
+ it('accepts await, void, a trailing .catch, and a returned or assigned promise', () => {
+ expect(findFloatingAsyncCalls(`
+ async function f() {}
+ async function g() { await f(); void f(); f().catch(() => {}); return f(); }
+ const p = f();
+ `)).toEqual([]);
+ });
+
+ it('ignores synchronous functions and member calls', () => {
+ expect(findFloatingAsyncCalls('function f() {}\nf();\nobj.run();')).toEqual([]);
+ });
+
+ it('resolves an async export imported from a relative module', () => {
+ // A sync export imported by name must not be flagged.
+ expect(findFloatingAsyncCalls("import { getAllowedCommands as sync } from './services/commands.js';\nsync();", SERVER_ROOT)).toEqual([]);
+ expect(findFloatingAsyncCalls("import { logAction as log } from './services/history.js';\nlog();", SERVER_ROOT))
+ .toEqual(['line 2: log()']);
+ });
+});
diff --git a/server/services/agentRunTracking.js b/server/services/agentRunTracking.js
index 4e53a034f7..da79d682ea 100644
--- a/server/services/agentRunTracking.js
+++ b/server/services/agentRunTracking.js
@@ -172,7 +172,7 @@ export async function completeAgentRun(runId, output, exitCode, duration, errorA
// `metadata` already carries the endTime stamped above, which bounds the
// transcript window. Owns its own error handling.
if (metadata.providerId && metadata.model) {
- recordCompletedRunUsage(metadata, output);
+ void recordCompletedRunUsage(metadata, output);
}
// Close the run's ledger entry (#4540). This is the terminal event for every
diff --git a/server/services/commands.js b/server/services/commands.js
index dc32af4442..2312b2fc35 100644
--- a/server/services/commands.js
+++ b/server/services/commands.js
@@ -13,6 +13,16 @@ export const STOP_GRACE_MS = 5000;
// child that ignores SIGTERM is still running and still ours to escalate.
const activeCommands = new Map();
+// History is best-effort bookkeeping beside the command result: a failed history
+// write must not become an unowned rejection from a spawn listener (#12978).
+async function logHistory(...args) {
+ try {
+ await logAction(...args);
+ } catch (err) {
+ console.error(`❌ Failed to log command history: ${err.message}`);
+ }
+}
+
// Stop an active entry: SIGTERM now, SIGKILL after STOP_GRACE_MS if the child
// is still alive. Idempotent — a second request during the grace window is
// absorbed rather than re-signalling or reporting the command as gone.
@@ -42,7 +52,7 @@ export function executeCommand(command, workspacePath, onData, onComplete, { tim
const trimmedForLog = (command || '').trim();
onComplete?.({ success: false, error: validation.error, exitCode: 1 });
if (trimmedForLog) {
- logAction('command', null, trimmedForLog.substring(0, 50), { command: trimmedForLog, workspacePath }, false, validation.error);
+ void logHistory('command', null, trimmedForLog.substring(0, 50), { command: trimmedForLog, workspacePath }, false, validation.error);
}
return null;
}
@@ -113,7 +123,7 @@ export function executeCommand(command, workspacePath, onData, onComplete, { tim
: entry.stopping ? `Stopped (${signal || `exit code ${code}`})`
: signal ? `Killed by ${signal}` : `Exit code ${code}`;
- logAction('command', null, command.substring(0, 50), {
+ void logHistory('command', null, command.substring(0, 50), {
command,
workspacePath,
exitCode: code,
@@ -137,7 +147,7 @@ export function executeCommand(command, workspacePath, onData, onComplete, { tim
child.on('error', (err) => {
if (settled) return;
settle();
- logAction('command', null, command.substring(0, 50), { command, workspacePath }, false, err.message);
+ void logHistory('command', null, command.substring(0, 50), { command, workspacePath }, false, err.message);
onComplete?.({
success: false,
error: err.message,
diff --git a/server/services/commands.test.js b/server/services/commands.test.js
index ba4502d86c..7d6d877d95 100644
--- a/server/services/commands.test.js
+++ b/server/services/commands.test.js
@@ -127,3 +127,27 @@ describe('executeCommand stop and timeout lifecycle (#12021)', () => {
expect(unbounded.kill).not.toHaveBeenCalled();
});
});
+
+describe('executeCommand history is best-effort (#12978)', () => {
+ it('logs a failed history write instead of leaving an unowned rejection', async () => {
+ spawnMock.mockReset();
+ logAction.mockReset();
+ logAction.mockRejectedValue(new Error('ENOSPC: no space left'));
+ const error = vi.spyOn(console, 'error').mockImplementation(() => {});
+ const unhandled = vi.fn();
+ process.on('unhandledRejection', unhandled);
+ try {
+ const child = createChild();
+ spawnMock.mockReturnValue(child);
+ const onComplete = vi.fn();
+ executeCommand('pwd', null, null, onComplete);
+ child.emit('close', 0, null);
+ await vi.waitFor(() => expect(error).toHaveBeenCalledWith(expect.stringContaining('Failed to log command history')));
+ expect(onComplete).toHaveBeenCalledWith(expect.objectContaining({ success: true }));
+ expect(unhandled).not.toHaveBeenCalled();
+ } finally {
+ process.off('unhandledRejection', unhandled);
+ error.mockRestore();
+ }
+ });
+});
diff --git a/server/services/fleetLlmUsage.js b/server/services/fleetLlmUsage.js
index a1591d82a9..1983cbf7bb 100644
--- a/server/services/fleetLlmUsage.js
+++ b/server/services/fleetLlmUsage.js
@@ -88,7 +88,7 @@ export function scheduleFleetHostUsagePersist() {
if (persistTimer) return;
persistTimer = setTimeout(() => {
persistTimer = null;
- writeLedger();
+ void writeLedger();
}, PERSIST_DEBOUNCE_MS);
// Never hold the process open for a usage report.
persistTimer.unref?.();
diff --git a/server/services/imageGen/fal.js b/server/services/imageGen/fal.js
index e54bea9226..295fdc7f8f 100644
--- a/server/services/imageGen/fal.js
+++ b/server/services/imageGen/fal.js
@@ -109,7 +109,7 @@ export const cancel = (jobId) => {
if (!entry) return false;
entry.aborted = true;
entry.controller.abort();
- cancelFalRequest(entry);
+ void cancelFalRequest(entry);
return true;
};
diff --git a/server/services/instances.js b/server/services/instances.js
index fa3163b173..80e84b0a1b 100644
--- a/server/services/instances.js
+++ b/server/services/instances.js
@@ -515,7 +515,7 @@ export async function addPeer({ address, port = DEFAULT_PEER_PORT, name, host, a
instanceEvents.emit('peers:updated', data.peers);
return entry;
});
- announceSelf(peer);
+ void announceSelf(peer);
return peer;
}
@@ -1114,7 +1114,7 @@ export async function probePeer(peer) {
const firstSighting = isFirstOnlineSighting(peer.id);
const transitioned = previousStatus !== 'online';
if (stored) {
- if (transitioned) announceSelf(stored);
+ if (transitioned) void announceSelf(stored);
if (transitioned || firstSighting) instanceEvents.emit('peer:online', stored);
}
}
@@ -1317,8 +1317,15 @@ export async function handleAnnounce({ address, port, instanceId, name, host })
return result;
}
+// Owns every rejection: callers in addPeer/probePeer fire it without awaiting.
async function announceSelf(peer) {
- const data = await loadData();
+ let data;
+ try {
+ data = await loadData();
+ } catch (err) {
+ console.log(`🌐 Announce skipped: instances data unreadable: ${err.message}`);
+ return;
+ }
if (!data.self) return;
const selfPort = parseInt(process.env.PORT, 10) || DEFAULT_PEER_PORT;
diff --git a/server/services/instances.test.js b/server/services/instances.test.js
index a54d0edd03..6d2e205333 100644
--- a/server/services/instances.test.js
+++ b/server/services/instances.test.js
@@ -211,6 +211,29 @@ describe('instances.js', () => {
expect(atomicWrite).toHaveBeenLastCalledWith(expect.any(String), expect.objectContaining({ peers: [secure, plain, classic] }));
});
+ it('logs and skips the self-announce when instances data becomes unreadable (#12978)', async () => {
+ readJSONFile.mockResolvedValue({ self: null, peers: [] });
+ const writesBefore = atomicWrite.mock.calls.length;
+ // addPeer persists first; only the fire-and-forget announce reads afterwards.
+ readJSONFile.mockImplementation(async () => {
+ if (atomicWrite.mock.calls.length > writesBefore) throw new Error('EIO: read failed');
+ return { self: null, peers: [] };
+ });
+ const log = vi.spyOn(console, 'log').mockImplementation(() => {});
+ const unhandled = vi.fn();
+ process.on('unhandledRejection', unhandled);
+ try {
+ await addPeer({ address: '192.0.2.30' });
+ // Suite may run fake timers: flush microtasks instead of sleeping.
+ for (let i = 0; i < 20; i++) await Promise.resolve();
+ expect(log).toHaveBeenCalledWith(expect.stringContaining('Announce skipped'));
+ expect(unhandled).not.toHaveBeenCalled();
+ } finally {
+ process.off('unhandledRejection', unhandled);
+ log.mockRestore();
+ }
+ });
+
it('should add a peer with correct defaults', async () => {
readJSONFile.mockResolvedValue({ self: null, peers: [] });
fetch.mockRejectedValue(new Error('not reachable'));
diff --git a/server/services/mediaJobQueue/index.js b/server/services/mediaJobQueue/index.js
index ebce65004f..d82c6d4b9c 100644
--- a/server/services/mediaJobQueue/index.js
+++ b/server/services/mediaJobQueue/index.js
@@ -992,10 +992,10 @@ export async function initMediaJobQueue({ deferCompletionRecovery = false } = {}
// never delete a file the job merely referenced (gallery image,
// prior render, etc). audioFilePath is the same kind of staged
// upload (a2v/voice/video jobs) — needs the same cleanup.
- safeUnlinkUpload(j.params?.uploadedTempPath);
- safeUnlinkUpload(j.params?.audioFilePath);
+ void safeUnlinkUpload(j.params?.uploadedTempPath);
+ void safeUnlinkUpload(j.params?.audioFilePath);
for (const p of normalizeTempPaths(j.params?.uploadedTempPaths)) {
- safeUnlinkUpload(p);
+ void safeUnlinkUpload(p);
}
} else if (j.status === 'queued') {
queue.push({ ...j, params: restoredParams(j) });
diff --git a/server/services/peerSocketRelay.js b/server/services/peerSocketRelay.js
index 771940b9bd..18729bfcfd 100644
--- a/server/services/peerSocketRelay.js
+++ b/server/services/peerSocketRelay.js
@@ -60,7 +60,7 @@ export function connectToPeer(peer) {
socket.emit('cos:subscribe');
// Fetch initial agent state via HTTP
- fetchPeerAgents(conn);
+ void fetchPeerAgents(conn);
});
socket.on('disconnect', () => {
diff --git a/server/services/usageReconciler.js b/server/services/usageReconciler.js
index 8b99dc0132..d60803fab1 100644
--- a/server/services/usageReconciler.js
+++ b/server/services/usageReconciler.js
@@ -921,7 +921,18 @@ export async function reconcileRunUsage(run, estimate, { home = homedir(), provi
* @param {{ home?: string }} [opts] `home` overrides the transcript root (tests)
* @returns {Promise<void>}
*/
-export async function recordCompletedRunUsage(metadata, output, { home = homedir(), providers = null } = {}) {
+export async function recordCompletedRunUsage(metadata, output, opts) {
+ // The synchronous estimate prefix in the body runs before its inner catch
+ // chain, so own the whole call here (#12978).
+ try {
+ await recordCompletedRunUsageUnguarded(metadata, output, opts);
+ } catch (err) {
+ maintenance.markCurrentUnsettled();
+ console.error(`❌ Failed to record usage: ${err.message}`);
+ }
+}
+
+async function recordCompletedRunUsageUnguarded(metadata, output, { home = homedir(), providers = null } = {}) {
if (!metadata?.providerId) return;
// OpenCode runs have no transcript store, but their captured stdout IS the
diff --git a/server/services/usageReconciler.test.js b/server/services/usageReconciler.test.js
index 3a34ffd6ab..9ea70d9dca 100644
--- a/server/services/usageReconciler.test.js
+++ b/server/services/usageReconciler.test.js
@@ -545,6 +545,18 @@ describe('recordCompletedRunUsage', () => {
expect(record.tokensIn).toBeGreaterThan(0);
});
+ it('does not reject when the synchronous estimate prefix throws (#12978)', async () => {
+ const error = vi.spyOn(console, 'error').mockImplementation(() => {});
+ try {
+ // A getter that throws stands in for any malformed metadata the prefix touches.
+ const metadata = { providerId: 'claude-code', get promptLength() { throw new Error('bad metadata'); } };
+ await expect(recordCompletedRunUsage(metadata, 'out')).resolves.toBeUndefined();
+ expect(error).toHaveBeenCalledWith(expect.stringContaining('Failed to record usage: bad metadata'));
+ } finally {
+ error.mockRestore();
+ }
+ });
+
it('skips a run with no providerId instead of creating an unknown bucket', async () => {
await recordCompletedRunUsage({ workspacePath: WORKSPACE, promptLength: 10 }, 'out');
expect(recordRunUsage).not.toHaveBeenCalled();
diff --git a/server/services/videoGen/fal.js b/server/services/videoGen/fal.js
index ed0d171122..7110bc0313 100644
--- a/server/services/videoGen/fal.js
+++ b/server/services/videoGen/fal.js
@@ -111,7 +111,7 @@ export const cancel = (jobId) => {
if (!entry) return false;
entry.aborted = true;
entry.controller.abort();
- cancelFalRequest(entry);
+ void cancelFalRequest(entry);
return true;
};
diff --git a/server/services/videoGen/runway.js b/server/services/videoGen/runway.js
index 20625f0bc0..4693e3cd04 100644
--- a/server/services/videoGen/runway.js
+++ b/server/services/videoGen/runway.js
@@ -74,7 +74,7 @@ export const cancel = (jobId) => {
if (!entry) return false;
entry.aborted = true;
entry.controller.abort();
- cancelRunwayTask(entry);
+ void cancelRunwayTask(entry);
return true;
};