Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: escalate a stopped operator command to SIGKILL and keep it tracked until close
[._.]/ Adam Eivy3c1498f3f4ae
diff --git a/server/lib/README.md b/server/lib/README.mdindex 1d09969544..e8f71285c4 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -431,7 +431,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `interactiveShellResolver.js` | `resolveInteractiveShell()` — picks the shell binary a PTY session runs (Shell page, agent TUI shells). On Windows the old `COMSPEC` default meant `cmd.exe`, where no command a user would type reaches another drive: `cd I:` silently PRINTS that drive's cwd and stays put, a bare `I:\` is "not recognized as a command", and `cd 'I:\path'` is a syntax error (no single-quote quoting) — so a Windows install with its repos on a second drive was stuck on `C:`. Resolves PORTOS_SHELL → newest `pwsh.exe` under `%ProgramFiles%\PowerShell\<major>` (majors enumerated and sorted numerically, so a future 8 needs no code change) → any `pwsh.exe` on PATH via `findCommandOnPath` (scoop/choco/Store shim) → `powershell.exe` 5.1 (ships with Windows) → `cmd.exe` last. PowerShell crosses drives with every form. On POSIX: PORTOS_SHELL → existing `SHELL` → `/bin/bash` `/usr/bin/bash` `/bin/sh` `/usr/bin/sh` then zsh if present → bare `bash`/`sh` via `findCommandOnPath` → bare `sh` last — never a hard-coded missing `/bin/zsh` (PM2/containers often lack `SHELL` and zsh; node-pty then `execvp`s and the Shell page shows only `execvp(3) failed`). Memoized; `_resetInteractiveShellCache()` for tests. `resolveInteractiveShellWith({platform, env, exists, readdir, findOnPath})` is the un-memoized injectable form for tests. || `openFolder.js` | `openFolderInSystemExplorer(localPath)` — cross-platform "open in Finder/Explorer/Nautilus" via detached spawn; child `error` handler prevents spawn failures from crashing the process. || `childProcess.js` | Drop-in `child_process` replacement that defaults every spawn to `windowsHide: true` — `spawn` / `spawnSync` / `fork` / `exec` / `execSync` / `execFile` / `execFileSync`, plus a `ChildProcess` re-export. **Server runtime code must import from here, never from `child_process` directly** (enforced by `childProcess.guards.test.js`): a console-less PM2 fork spawning a console child without `CREATE_NO_WINDOW` makes Windows hand the new console to Windows Terminal, which flashes a focus-stealing window. `exec`/`execFile` carry `util.promisify.custom` so `promisify(execFile)` still resolves to `{ stdout, stderr }`. An explicit `windowsHide: false` is respected. Background: `docs/WINDOWS_CONSOLE.md`. |-| `bufferedSpawn.js` | `bufferedSpawn(cmd, args, opts)` (structured non-throwing result) + `bufferedSpawnOrThrow` (throwing adapter), plus `killProcessTree`, `resolveWindowsExecutable`, `prepareWindowsSafeSpawn`, `prepareCliSpawn(command, args, env)` (composed resolve+wrap for a `spawn()`-safe pair), `needsShell`, `IS_WIN32`, `WIN_CMD_SHIMS`, `MAX_OUTPUT_BYTES`, `guardChildStdin(child)` (attach the no-op `stdin` `'error'` listener BEFORE any `stdin.write`/`stdin.end` — a child that dies before reading stdin makes the pipe emit `EPIPE`, and an unlistened stream `'error'` outside the Express request lifecycle takes the whole server process down; call it at every spawn that writes to a child's stdin) + `deliverChildStdin(child, payload, label)` (its other half: write-then-`end()` with a synchronously-throwing `write` caught, the pipe destroyed so a reading child sees EOF rather than hanging, and the failure logged so an empty-prompt run that exits 0 isn't filed as clean), `spawnFailureDetail(result, fallback)` (the most useful sentence from a failed result — prefers the spawn error, then a stderr line, then a stdout line that isn't bare JSON punctuation, because a `--json` CLI prints its payload to stdout even when it exits non-zero) — shared buffered-spawn machinery with capped stdout/stderr, timeout SIGTERM plus fire-and-forget SIGKILL escalation (`killGraceMs`, default 8s), Windows `.cmd`/`.bat` shim resolution, and `taskkill /T /F` tree-kill. `killProcessTree(child, signal, { processGroup })` takes the POSIX group down too when the child was spawned `detached` (Windows always tree-kills). A killable that is NOT a `ChildProcess` — a node-pty `IPty` TUI session — is killed through its own `.kill()`, with **no signal on Windows**: node-pty throws `Signals not supported on windows.` for any signal argument, so a signalled kill there killed nothing and threw past the caller. Used by `appBuilder.js`, `appUpdater.js`, the CoS agent spawners, and `setupScriptRunner.js`. |+| `bufferedSpawn.js` | `bufferedSpawn(cmd, args, opts)` (structured non-throwing result) + `bufferedSpawnOrThrow` (throwing adapter), plus `killProcessTree`, `terminateWithEscalation(child, { graceMs, label, processGroup })` (SIGTERM now, SIGKILL after the grace if the child is still running; returns a cancel for the caller's `close` handler — the one stop-escalation helper, used by the operator command runner's stop and by `bufferedSpawn`'s own timeout), `resolveWindowsExecutable`, `prepareWindowsSafeSpawn`, `prepareCliSpawn(command, args, env)` (composed resolve+wrap for a `spawn()`-safe pair), `needsShell`, `IS_WIN32`, `WIN_CMD_SHIMS`, `MAX_OUTPUT_BYTES`, `guardChildStdin(child)` (attach the no-op `stdin` `'error'` listener BEFORE any `stdin.write`/`stdin.end` — a child that dies before reading stdin makes the pipe emit `EPIPE`, and an unlistened stream `'error'` outside the Express request lifecycle takes the whole server process down; call it at every spawn that writes to a child's stdin) + `deliverChildStdin(child, payload, label)` (its other half: write-then-`end()` with a synchronously-throwing `write` caught, the pipe destroyed so a reading child sees EOF rather than hanging, and the failure logged so an empty-prompt run that exits 0 isn't filed as clean), `spawnFailureDetail(result, fallback)` (the most useful sentence from a failed result — prefers the spawn error, then a stderr line, then a stdout line that isn't bare JSON punctuation, because a `--json` CLI prints its payload to stdout even when it exits non-zero) — shared buffered-spawn machinery with capped stdout/stderr, timeout SIGTERM plus fire-and-forget SIGKILL escalation (`killGraceMs`, default 8s), Windows `.cmd`/`.bat` shim resolution, and `taskkill /T /F` tree-kill. `killProcessTree(child, signal, { processGroup })` takes the POSIX group down too when the child was spawned `detached` (Windows always tree-kills). A killable that is NOT a `ChildProcess` — a node-pty `IPty` TUI session — is killed through its own `.kill()`, with **no signal on Windows**: node-pty throws `Signals not supported on windows.` for any signal argument, so a signalled kill there killed nothing and threw past the caller. Used by `appBuilder.js`, `appUpdater.js`, the CoS agent spawners, and `setupScriptRunner.js`. || `setupScriptRunner.js` | `spawnSetupScript(envVars)` / `stopSetupScript(child)` / `SETUP_IMAGE_VIDEO_SCRIPT` — the one way to run `scripts/setup-image-video.sh` (shared by the Video Gen BYOV runtimes, the music engines and the MuScriptor venv). Runs it under `resolveBashBinary()` with a `toBashPath` script path, and on Windows presets the `PYTHON_BIN` the script would otherwise default to `python3` for. Cancel via `stopSetupScript`, which tree-kills so uv / pip / git die with bash. || `commandExists.js` | `commandExists(cmd, args = ['--version'], { timeoutMs = 5_000, env, cwd })` — does running `cmd args` succeed? A capability probe (spawn-based, with a deadline that settles even when the child ignores termination), not a PATH lookup like `processEnv.js`'s `whichFirst`; `env`/`cwd` let a caller check the exact child process configuration. Consolidates the two previously-private copies in `localLlm.js`/`ollamaManager.js`; callers probing a heavier CLI (e.g. `codeReview.js`'s reviewer-binary probe) pass a longer `timeoutMs`. Sibling `commandOutput(cmd, args, opts)` runs the same probe but returns the trimmed stdout (or `null` when it could not run / exited non-zero), so a caller can read a `--version` banner or a `models` listing without spawning the same child twice. || `runCliBounded.js` | `runCliBounded(cmd, args, { timeoutMs = 15000, ...spawnOptions })` — never-rejecting, never-hanging CLI runner resolving `{ code, stdout, stderr }`; kills a child still running at `timeoutMs` and resolves `code: -1` (stderr preserved, else `timed out`); pins PWD via `withSpawnCwdEnv`. Use it for every `gh`/`glab`/`git` call a scheduled job awaits (#11678). |diff --git a/server/lib/bufferedSpawn.js b/server/lib/bufferedSpawn.jsindex f8600b4e70..d6805595ab 100644--- a/server/lib/bufferedSpawn.js+++ b/server/lib/bufferedSpawn.js@@ -195,6 +195,40 @@ export function killProcessTree(child, signal = 'SIGTERM', { processGroup = fals}+/**+ * Stop a child gracefully with forced escalation: SIGTERM now, SIGKILL after+ * `graceMs` if the child has not exited. The tree's one SIGTERM→SIGKILL+ * pattern (the pm2 CLI runner and `bufferedSpawn`'s timeout path do the same+ * dance): a child that traps or ignores SIGTERM — an npm wrapper, docker+ * compose — must not outlive the caller's idea of "stopped" (#12021).+ *+ * Returns a cancel function for the caller's `close`/`error` handler so a+ * child that did exit on SIGTERM never gets a stray SIGKILL aimed at a pid+ * the OS may have reused. The timer is unref'd and never keeps the process+ * alive. On Windows `killProcessTree` is already `taskkill /F`, so there is+ * nothing to escalate to and the cancel is a no-op.+ *+ * @param {import('child_process').ChildProcess|{pid: number, kill?: Function}} child+ * @param {{ graceMs?: number, label?: string, processGroup?: boolean }} [opts]+ * @param {boolean} [isWin32] - platform override for tests+ * @returns {() => void} cancel the pending SIGKILL escalation+ */+export function terminateWithEscalation(child, { graceMs = 5000, label = 'child process', processGroup = false } = {}, isWin32 = IS_WIN32) {+ killProcessTree(child, 'SIGTERM', { processGroup }, isWin32);+ if (isWin32) return () => {};+ const timer = setTimeout(() => {+ if (child.exitCode !== null || child.signalCode !== null) return;+ console.warn(`⚠️ ${label} ignored SIGTERM for ${graceMs}ms — escalating to SIGKILL`);+ try {+ killProcessTree(child, 'SIGKILL', { processGroup }, isWin32);+ } catch (err) {+ console.error(`❌ ${label} SIGKILL escalation failed: ${err.message}`);+ }+ }, graceMs);+ timer.unref?.();+ return () => clearTimeout(timer);+}+/*** Spawn a command, buffer its (capped) stdout/stderr, enforce a timeout, and* resolve a structured result. NEVER rejects — every terminal condition@@ -245,20 +279,9 @@ export function bufferedSpawn(cmd, args, { cwd, env = process.env, timeoutMs, ki? setTimeout(() => {if (!settled) {settled = true;- killProcessTree(child, 'SIGTERM');- if (!IS_WIN32) {- const escalationTimer = setTimeout(() => {- try {- if (child.exitCode === null && child.signalCode === null) {- console.log(`⚠️ ${cmd} didn't exit on SIGTERM — escalating to SIGKILL`);- killProcessTree(child, 'SIGKILL');- }- } catch (err) {- console.error(`❌ ${cmd} SIGKILL escalation failed: ${err.message}`);- }- }, killGraceMs);- escalationTimer.unref?.();- }+ // Fire-and-forget: the timed-out result resolves now and the+ // escalation timer is unref'd, so no cancel is held here.+ terminateWithEscalation(child, { graceMs: killGraceMs, label: cmd });resolve({ success: false, code: -1, signal: null, stdout, stderr, timedOut: true });}}, timeoutMs)diff --git a/server/lib/bufferedSpawn.test.js b/server/lib/bufferedSpawn.test.jsindex 38cc3819e0..e74bca06b8 100644--- a/server/lib/bufferedSpawn.test.js+++ b/server/lib/bufferedSpawn.test.js@@ -1,5 +1,6 @@import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';import { EventEmitter } from 'events';+import { spawn as nodeSpawn } from 'node:child_process';import { ChildProcess } from './childProcess.js';import { mkdtemp, rm, writeFile } from 'fs/promises';import { join } from 'path';@@ -31,6 +32,7 @@ const {spawnFailureDetail,guardChildStdin,deliverChildStdin,+ terminateWithEscalation,} = await import('./bufferedSpawn.js');/**@@ -478,6 +480,61 @@ describe('prepareCliSpawn (composed resolve+wrap — the CoS spawn fix, #2243)',});});+describe.skipIf(IS_WIN32)('terminateWithEscalation (#12021)', () => {+ // Real children: the contract is that a process which traps SIGTERM still+ // dies, which a kill spy cannot prove.+ const spawnReal = (script) => new Promise((resolve) => {+ const child = nodeSpawn(process.execPath, ['-e', script], { stdio: ['ignore', 'pipe', 'ignore'] });+ child.stdout.once('data', () => resolve(child));+ });+ const closed = (child) => new Promise((resolve) => child.once('close', (code, signal) => resolve({ code, signal })));++ it('SIGKILLs a child that ignores SIGTERM once the grace elapses, and close fires', async () => {+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ const child = await spawnReal("process.on('SIGTERM', () => {}); setInterval(() => {}, 1000); process.stdout.write('ready');");+ const cancel = terminateWithEscalation(child, { graceMs: 300, label: 'stubborn child' });+ const result = await closed(child);+ cancel();+ expect(result.signal).toBe('SIGKILL');+ expect(warn).toHaveBeenCalledWith(expect.stringContaining('stubborn child ignored SIGTERM for 300ms'));+ warn.mockRestore();+ });++ it('leaves a child that honoured SIGTERM alone: cancel clears the escalation and no SIGKILL warning fires', async () => {+ vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] });+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ const child = makeFakeChild();+ const cancel = terminateWithEscalation(child, { graceMs: 300, label: 'polite child' });+ expect(child.kill).toHaveBeenCalledWith('SIGTERM');+ // The child exits inside the grace window; the caller's close handler cancels.+ child.signalCode = 'SIGTERM';+ cancel();+ vi.advanceTimersByTime(1000);+ expect(child.kill).toHaveBeenCalledTimes(1);+ expect(warn).not.toHaveBeenCalled();+ warn.mockRestore();+ });++ it('skips the SIGKILL timer when a child already exited before the grace elapsed (no stray signal at a reused pid)', () => {+ vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] });+ const child = makeFakeChild();+ terminateWithEscalation(child, { graceMs: 300 });+ child.exitCode = 0;+ vi.advanceTimersByTime(1000);+ expect(child.kill).toHaveBeenCalledTimes(1);+ });++ it('on Windows the tree kill is already forced, so there is nothing to escalate', () => {+ const child = makeFakeChild({ pid: 4242 });+ const taskkill = makeFakeChild();+ spawnMock.mockReturnValueOnce(taskkill);+ const cancel = terminateWithEscalation(child, { graceMs: 300 }, true);+ expect(spawnMock).toHaveBeenCalledWith('taskkill', ['/T', '/F', '/PID', '4242'], expect.anything());+ expect(cancel).toBeTypeOf('function');+ cancel();+ });+});+describe('bufferedSpawn — structured result', () => {it('resolves success on a clean (code 0) exit and captures stdout/stderr', async () => {const child = makeFakeChild();diff --git a/server/lib/validation.js b/server/lib/validation.jsindex ee73383467..702c4f7736 100644--- a/server/lib/validation.js+++ b/server/lib/validation.js@@ -2848,7 +2848,13 @@ export const calendarAutoConfigureBodySchema = z.object({email: optionalString,requestId: z.string().min(1).max(128).regex(/^[A-Za-z0-9_-]+$/).optional(),});-export const commandExecuteBodySchema = z.object({ command: nonEmptyString, workspacePath: nullishString });+// timeoutMs bounds an operator command run (#12021); absent means unbounded,+// the right default for an interactive run the operator is watching.+export const commandExecuteBodySchema = z.object({+ command: nonEmptyString,+ workspacePath: nullishString,+ timeoutMs: z.number().int().positive().max(24 * 60 * 60 * 1000).optional(),+});export const dataArchiveBodySchema = z.object({daysToKeep: z.preprocess((value) => (value == null ? undefined : value), z.coerce.number().finite().min(0).optional()),});diff --git a/server/routes/commands.js b/server/routes/commands.jsindex 4c635753fa..cf436103be 100644--- a/server/routes/commands.js+++ b/server/routes/commands.js@@ -14,7 +14,7 @@ const router = Router();// POST /api/commands/execute - Execute a commandrouter.post('/execute', requireHostControl, asyncHandler(async (req, res) => {- const { command, workspacePath } = validateRequest(commandExecuteBodySchema, req.body);+ const { command, workspacePath, timeoutMs } = validateRequest(commandExecuteBodySchema, req.body);// Validate workspacePath if provided: must exist, be a directory, and — after// symlinks are followed — resolve within an allowed root. Using realpath here@@ -57,7 +57,8 @@ router.post('/execute', requireHostControl, asyncHandler(async (req, res) => {},(result) => {if (commandId) broadcastToCommandSubscribers(`command:${commandId}:complete`, result);- }+ },+ { timeoutMs });if (!commandId) {diff --git a/server/routes/commands.test.js b/server/routes/commands.test.jsindex 457bf3548a..a7f1bfb03c 100644--- a/server/routes/commands.test.js+++ b/server/routes/commands.test.js@@ -86,6 +86,8 @@ function createChildProcess() {const child = new EventEmitter();child.stdout = new EventEmitter();child.stderr = new EventEmitter();+ child.exitCode = null;+ child.signalCode = null;child.kill = vi.fn();return child;}@@ -505,6 +507,42 @@ describe('commands routes', () => {expect(response.status).toBe(200);expect(response.body).toEqual({ stopped: true });expect(child.kill).toHaveBeenCalledWith('SIGTERM');++ // The child has not closed yet (it may be ignoring SIGTERM): a repeat+ // stop inside the grace window is absorbed, not reported as gone (#12021).+ const again = await request(app)+ .post(`/api/commands/${started.body.commandId}/stop`);+ expect(again.status).toBe(200);+ expect(again.body).toEqual({ stopped: true });+ expect(child.kill).toHaveBeenCalledTimes(1);++ child.emit('close', null, 'SIGTERM');+ const afterClose = await request(app)+ .post(`/api/commands/${started.body.commandId}/stop`);+ expect(afterClose.status).toBe(404);+ expect(afterClose.body.code).toBe('NOT_ACTIVE');+ expect(broadcastToCommandSubscribers).toHaveBeenCalledWith(+ `command:${started.body.commandId}:complete`,+ expect.objectContaining({ success: false, signal: 'SIGTERM', stopped: true, error: 'Stopped (SIGTERM)' }),+ );+ });++ it('accepts an optional per-command timeoutMs and rejects a non-positive one (#12021)', async () => {+ const child = createChildProcess();+ spawnMock.mockReturnValue(child);+ const { app } = createApp();++ const rejected = await request(app)+ .post('/api/commands/execute')+ .send({ command: 'pwd', timeoutMs: 0 });+ expect(rejected.status).toBe(400);+ expect(spawnMock).not.toHaveBeenCalled();++ const started = await request(app)+ .post('/api/commands/execute')+ .send({ command: 'pwd', timeoutMs: 60_000 });+ expect(started.status).toBe(202);+ child.emit('close', 0);});it('returns NOT_ACTIVE for an unknown command id', async () => {diff --git a/server/services/commands.js b/server/services/commands.jsindex 2ed2acb683..dc32af4442 100644--- a/server/services/commands.js+++ b/server/services/commands.js@@ -3,10 +3,25 @@ import { logAction } from './history.js';import { ALLOWED_COMMANDS, redactOutput, validateCommand } from '../lib/commandSecurity.js';import { safeChildProcessEnv } from '../lib/processEnv.js';import { withSpawnCwdEnv } from '../lib/spawnCwd.js';+import { terminateWithEscalation } from '../lib/bufferedSpawn.js';-// Track active commands+// SIGTERM grace before a stopped command is SIGKILLed (#12021).+export const STOP_GRACE_MS = 5000;++// commandId → { child, label, cancelEscalation, stopping, timedOut }. An entry lives+// until the child's `close`/`error` fires, never from a stop request: a+// child that ignores SIGTERM is still running and still ours to escalate.const activeCommands = new Map();+// Stop an active entry: SIGTERM now, SIGKILL after STOP_GRACE_MS if the child+// is still alive. Idempotent — a second request during the grace window is+// absorbed rather than re-signalling or reporting the command as gone.+function stopEntry(entry) {+ if (entry.stopping) return;+ entry.stopping = true;+ entry.cancelEscalation = terminateWithEscalation(entry.child, { graceMs: STOP_GRACE_MS, label: entry.label });+}+/*** Execute a shell command with safety checks*@@ -14,8 +29,12 @@ const activeCommands = new Map();* 1. Base command must be in allowlist* 2. Command cannot contain shell metacharacters that enable injection* 3. Command executed via spawn with shell:false where possible+ *+ * `timeoutMs` (optional, default none — the operator watches an interactive+ * run) bounds the run: at the deadline the child is stopped through the same+ * SIGTERM→SIGKILL path as a manual stop and the completion reports `timedOut`.*/-export function executeCommand(command, workspacePath, onData, onComplete) {+export function executeCommand(command, workspacePath, onData, onComplete, { timeoutMs = null } = {}) {const commandId = Date.now().toString(36) + Math.random().toString(36).substr(2);const validation = validateCommand(command);@@ -50,7 +69,22 @@ export function executeCommand(command, workspacePath, onData, onComplete) {shell: false});- activeCommands.set(commandId, child);+ const entry = { child, label: `command '${command.substring(0, 50)}'`, cancelEscalation: null, stopping: false, timedOut: false };+ activeCommands.set(commandId, entry);+ const timeoutTimer = timeoutMs > 0+ ? setTimeout(() => {+ entry.timedOut = true;+ console.warn(`⚠️ ${entry.label} exceeded ${timeoutMs}ms — stopping it`);+ stopEntry(entry);+ }, timeoutMs)+ : null;+ timeoutTimer?.unref?.();+ const settle = () => {+ settled = true;+ clearTimeout(timeoutTimer);+ entry.cancelEscalation?.();+ activeCommands.delete(commandId);+ };// Every copy of the output leaves this function redacted (#12019): each// streamed chunk on its own, and the accumulated buffer once more at close@@ -68,34 +102,41 @@ export function executeCommand(command, workspacePath, onData, onComplete) {onData?.(redactOutput(text), 'stderr');});- child.on('close', (code) => {+ child.on('close', (code, signal) => {if (settled) return;- settled = true;- activeCommands.delete(commandId);+ settle();const runtime = Date.now() - startTime;const success = code === 0;const redacted = redactOutput(output);+ const error = success ? null+ : entry.timedOut ? `Timed out after ${timeoutMs}ms`+ : entry.stopping ? `Stopped (${signal || `exit code ${code}`})`+ : signal ? `Killed by ${signal}` : `Exit code ${code}`;logAction('command', null, command.substring(0, 50), {command,workspacePath,exitCode: code,+ signal,runtime,output: redacted.substring(0, 10000) // Truncate to prevent huge history- }, success, success ? null : `Exit code ${code}`);+ }, success, error);onComplete?.({success,exitCode: code,+ signal,runtime,- output: redacted+ output: redacted,+ ...(error ? { error } : {}),+ ...(entry.stopping ? { stopped: true } : {}),+ ...(entry.timedOut ? { timedOut: true } : {}),});});child.on('error', (err) => {if (settled) return;- settled = true;- activeCommands.delete(commandId);+ settle();logAction('command', null, command.substring(0, 50), { command, workspacePath }, false, err.message);onComplete?.({success: false,@@ -108,16 +149,16 @@ export function executeCommand(command, workspacePath, onData, onComplete) {}/**- * Stop a running command+ * Stop a running command: SIGTERM, then SIGKILL after STOP_GRACE_MS if it is+ * still alive. Returns true while the command is ours to stop — including a+ * repeat request during the grace window, which is a no-op rather than+ * "not active" — and false once `close` has retired the entry.*/export function stopCommand(commandId) {- const child = activeCommands.get(commandId);- if (child) {- child.kill('SIGTERM');- activeCommands.delete(commandId);- return true;- }- return false;+ const entry = activeCommands.get(commandId);+ if (!entry) return false;+ stopEntry(entry);+ return true;}/**diff --git a/server/services/commands.test.js b/server/services/commands.test.jsnew file mode 100644index 0000000000..5c449a558b--- /dev/null+++ b/server/services/commands.test.js@@ -0,0 +1,120 @@+import { EventEmitter } from 'events';+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';++const spawnMock = vi.hoisted(() => vi.fn());+vi.mock('../lib/childProcess.js', async (importOriginal) => ({+ ...await importOriginal(),+ spawn: spawnMock,+}));+vi.mock('./history.js', () => ({ logAction: vi.fn() }));++import { logAction } from './history.js';+import { executeCommand, stopCommand, STOP_GRACE_MS } from './commands.js';++// A child that only dies when the caller says so: `kill` records the signal+// and the test decides when `close` fires, which is exactly the shape of an+// npm wrapper or docker compose ignoring SIGTERM (#12021).+function createChild() {+ const child = new EventEmitter();+ child.stdout = new EventEmitter();+ child.stderr = new EventEmitter();+ child.exitCode = null;+ child.signalCode = null;+ child.kill = vi.fn();+ return child;+}++describe('executeCommand stop and timeout lifecycle (#12021)', () => {+ let warn;+ beforeEach(() => {+ vi.useFakeTimers();+ warn = vi.spyOn(console, 'warn').mockImplementation(() => {});+ spawnMock.mockReset();+ logAction.mockReset();+ });+ afterEach(() => {+ warn.mockRestore();+ vi.useRealTimers();+ });++ it('keeps a stopped command active through the SIGTERM grace, escalates to SIGKILL, and retires it only on close', () => {+ const child = createChild();+ spawnMock.mockReturnValue(child);+ const onComplete = vi.fn();+ const id = executeCommand('pwd', null, null, onComplete);++ expect(stopCommand(id)).toBe(true);+ expect(child.kill).toHaveBeenCalledWith('SIGTERM');+ // A second stop during the grace window is a no-op, not "not active".+ expect(stopCommand(id)).toBe(true);+ expect(child.kill).toHaveBeenCalledTimes(1);++ vi.advanceTimersByTime(STOP_GRACE_MS - 1);+ expect(child.kill).toHaveBeenCalledTimes(1);+ vi.advanceTimersByTime(1);+ expect(child.kill).toHaveBeenLastCalledWith('SIGKILL');+ expect(warn).toHaveBeenCalledWith(expect.stringContaining('ignored SIGTERM'));+ expect(onComplete).not.toHaveBeenCalled();++ child.signalCode = 'SIGKILL';+ child.emit('close', null, 'SIGKILL');+ expect(onComplete).toHaveBeenCalledWith(expect.objectContaining({+ success: false, exitCode: null, signal: 'SIGKILL', stopped: true, error: 'Stopped (SIGKILL)',+ }));+ expect(logAction).toHaveBeenCalledWith('command', null, 'pwd', expect.objectContaining({ signal: 'SIGKILL' }), false, 'Stopped (SIGKILL)');+ expect(stopCommand(id)).toBe(false);+ });++ it('cancels the escalation when the child honours SIGTERM inside the grace window', () => {+ const child = createChild();+ spawnMock.mockReturnValue(child);+ const onComplete = vi.fn();+ const id = executeCommand('pwd', null, null, onComplete);++ stopCommand(id);+ child.signalCode = 'SIGTERM';+ child.emit('close', null, 'SIGTERM');+ vi.advanceTimersByTime(STOP_GRACE_MS * 2);+ expect(child.kill).toHaveBeenCalledTimes(1);+ expect(onComplete).toHaveBeenCalledTimes(1);+ expect(warn).not.toHaveBeenCalled();+ });++ it('bounds a run with timeoutMs through the same stop path and reports timedOut', () => {+ const child = createChild();+ spawnMock.mockReturnValue(child);+ const onComplete = vi.fn();+ executeCommand('pwd', null, null, onComplete, { timeoutMs: 1000 });++ vi.advanceTimersByTime(999);+ expect(child.kill).not.toHaveBeenCalled();+ vi.advanceTimersByTime(1);+ expect(child.kill).toHaveBeenCalledWith('SIGTERM');+ vi.advanceTimersByTime(STOP_GRACE_MS);+ expect(child.kill).toHaveBeenLastCalledWith('SIGKILL');++ child.emit('close', null, 'SIGKILL');+ expect(onComplete).toHaveBeenCalledWith(expect.objectContaining({+ success: false, timedOut: true, stopped: true, error: 'Timed out after 1000ms',+ }));+ });++ it('runs unbounded by default and clears a pending timeout when the command exits on its own', () => {+ const child = createChild();+ spawnMock.mockReturnValue(child);+ const onComplete = vi.fn();+ executeCommand('pwd', null, null, onComplete, { timeoutMs: 1000 });+ child.exitCode = 0;+ child.emit('close', 0, null);+ vi.advanceTimersByTime(5000);+ expect(child.kill).not.toHaveBeenCalled();+ expect(onComplete).toHaveBeenCalledWith(expect.objectContaining({ success: true, exitCode: 0 }));+ expect(onComplete.mock.calls[0][0]).not.toHaveProperty('error');++ const unbounded = createChild();+ spawnMock.mockReturnValue(unbounded);+ executeCommand('pwd', null, null, vi.fn());+ vi.advanceTimersByTime(60 * 60 * 1000);+ expect(unbounded.kill).not.toHaveBeenCalled();+ });+});