Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(autopilot): serve restart once per build, never another brain's service, non-blocking restart (#8019, from #8144)

garrytan3970883cdae3
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 927b2836bc..81181fb27a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,20 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**The serve restart after a self-upgrade now tries once per installed build, leaves alone a serve service installed for another brain, and no longer freezes the autopilot while it runs.** #7478 had the relaunched autopilot restart the serve its installed service runs when that serve was still on the old build. Three gaps came out of its review (#8013). A restart that came back unverified was tried again by every autopilot relaunch, because the attempt was only remembered in memory. On macOS, where the autopilot doesn't filter processes by `GBRAIN_HOME`, one brain's autopilot could restart the `com.gbrain.serve` installed for another brain on the same OS user. And the restart command ran synchronously for up to ten minutes, which stopped the daemon's timers and signal handling while `systemctl --user restart` waited for the old serve to stop.
+
+Now `serve-restart-attempt.json` under the brain's `.gbrain` directory remembers the build the restart was tried on, so each build gets one attempt across relaunches. The autopilot restarts the serve service only when its plist or unit starts this brain's own serve wrapper, and audits any other case as `skipped` / `manual`. The restart command runs asynchronously with a 120-second cap.
+
+### Itemized changes
+
+- `src/commands/upgrade-restart.ts`: the per-build attempt marker in `restartServeAfterSelfUpgrade`; the ownership check before `restartStaleServeService` collects processes; `hostExecutor` spawns asynchronously and kills a command past its timeout (120 s for the autopilot's serve restart, 600 s for `gbrain upgrade --restart-only` as before).
+- `src/core/serve-service.ts`: `serveServiceRunsWrapper` reads the launchd plist or the systemd unit and checks that it starts this brain's wrapper.
+- `src/core/upgrade-restart.ts`: `RestartExecutor.run` may return a promise, and `runRestartSteps` awaits each command.
+- Docs: the Restart section of `docs/guides/upgrades-auto-update.md` and the `upgrade-restart.ts` key-files entry.
+- Tests: `test/autopilot-serve-restart-8013.test.ts`.
+
## [0.60.699.0] - 2026-11-12
### The Ubicloud gate VMs run a current git
diff --git a/docs/architecture/key-files/commands-6.md b/docs/architecture/key-files/commands-6.md
index 6c73766325..4f14efb4fa 100644
--- a/docs/architecture/key-files/commands-6.md
+++ b/docs/architecture/key-files/commands-6.md
@@ -14,7 +14,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/commands/upgrade.ts` — self-update CLI. It sets `GBRAIN_UPGRADE_OWNS_MIGRATIONS=1` for the install step, so package postinstall (`scripts/postinstall.ts`) skips `apply-migrations`; post-upgrade owns them. An install step that fails or times out after `gbrain --version` already reports a newer version counts as installed. The run ends with a `Binary:` line (installed version, or upgrade failed) and a `Migrations:` line: complete; running, with the lock holder's host and pid when post-upgrade exits 75; or failed, with the recovery command (`--force-retry <v>` for wedged versions). `runPostUpgrade()` enumerates migrations from the TS registry (`src/commands/migrations/index.ts`) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally. After schema apply it prints the one-shot auto_chronicle default-on `[AGENT]` notice (`core/chronicle/upgrade-notice.ts`). Install methods are bun, bun-link and binary (a compiled binary is found by Bun's `/$bunfs/` entrypoint, so a release asset keeps its published name); anything else is `unknown` and prints manual steps. It never runs `clawhub update gbrain`, which would install whatever skill anyone publishes under that name (#8464; `test/upgrade.serial.test.ts`). Before the swap of a bun-link or package install it refuses (exit 78, nothing changed) a target whose Bun floor is above the host's Bun or unreadable (`--no-bun-floor-check` overrides, manual only; `src/core/bun-floor.ts`); after the swap it runs `gbrain --help` as a smoke that passes the startup Bun check and, on failure, exits 1 with per-method recovery (#5855). Everything the post-swap tail needs is imported statically, so the old process never loads new-release modules into its cached graph (#6604; the supervisor step constant lives in the import-free leaf `src/commands/movement-step.ts`; guard: `test/upgrade-post-swap-imports.test.ts`). A tag-pinned reinstall hint (`bunReinstallCommand`, `reinstallHintLines`) probes `git ls-remote --exit-code --tags … refs/tags/v<x>` before `bun remove -g gbrain`, prints a restore line, and on Windows prints one step per line; `planPinnedReinstall` probes the tag itself (published / not published / unknown) and falls back to the newest published `v<x>` tag (`latestPublishedVersion`) when the requested one is missing (#6662, D24; `test/upgrade-release-tag-hint.test.ts`). A verified version equal to the running one after the install step is no swap (#6828): it prints `No newer version was installed`, skips `last_upgrade`, `JUST_UPGRADED`, the cache reset and the smoke, and on a tag pin with no newer published tag the hint moves the install off the pin with `to_version: latest`, never a cached unpublished version (#6718). With `GBRAIN_SWAP_RECEIPT` set it writes a `SwapReceipt` (`installed` / `not_installed` / `installed_broken`) for the autopilot channel (`test/autopilot-unpublished-target-6828.test.ts`). Tests: `upgrade-migration-outcome.test.ts`, `upgrade-no-autopilot.serial.test.ts`, `upgrade-bun-link-arc.serial.test.ts`. `post-upgrade --json` (D2, `json: 'document'`): human lines go to stderr and stdout is one report `{status, apply_migrations:{exit_code}, schema, warnings[]}`; it calls `applyMigrations()` and still ends on any status apply-migrations returns (a failure is one envelope leading with the report). GBRA-175: `--channel stable|alpha` (`latest` is an alias of `alpha`; resolution order in the Release channels entry below): `resolveStableRelease` maps the commit `refs/tags/stable` peels to onto its `v<version>` tag through `git ls-remote`, and `installStableRelease` installs exactly that release (bun: remove then add `#v<version>`, restoring the previous release on failure; bun-link: fast-forward only; binary: `fetchReleaseByTag`), never downgrading. Before the swap `preflightHostRestart` refuses (`restart_refused`, exit 3) when a running process could not be restarted; after post-upgrade the new build runs `gbrain upgrade --restart-only` (`--no-restart` skips it). Tests: `upgrade-stable-channel.test.ts`. A global install pinned to a tag or commit (`bunGlobalExactTagPin(root, true)`) is replaced with the newest published release the same way instead of `bun update`, which keeps the pin. `parseChannelArg` and `resolveStableRelease` live in `src/core/stable-channel.ts` (re-exported).
- Release channels: `src/core/self-upgrade.ts` resolves the channel (`resolveReleaseChannel`: `--channel`, `GBRAIN_RELEASE_CHANNEL`, `self_upgrade.channel`, default `stable`; `latest` normalizes to `alpha`; an invalid value at the first present source is an error) and keeps the update cache (`update-check.<channel>.json`, tagged with the evidence policy; an older answer never overwrites a newer one) and the time-based snooze (`update-snoozed.<channel>.json`) per channel; `offeredVersion` never offers a downgrade except to move off a `known_bad` version. `src/core/stable-channel.ts` reads and strictly parses the evidence record (`stable.json` on `stable-record`; `requiredEvidenceKeys`: `soak_run`, plus `matrix_run` from `MATRIX_REQUIRED_FROM`), maps failures to `stable_missing` / `stable_invalid` / `stable_unreadable` with sanitized causes, and `resolveStableRelease` checks the `stable` and `v<version>` tags against the record commit. `src/commands/upgrade-channel.ts` plans an upgrade's target (`channelTarget`, `runUpgradeDryRun` for `gbrain upgrade --dry-run --json`) and refuses with the typed code. `src/commands/check-update.ts` `refreshUpdateCache` never reads a bare VERSION for stable and never falls back to alpha. `src/core/release-channel-notice.ts` prints the one-time `[gbrain notice release_channel]` line on stderr (marker file under `notices/`; not under `--json`, `serve` or `mcp`; only when no channel is set). `src/core/install-ref.ts` is the one ref every install surface names, linted by `scripts/check-install-ref.ts`. Tests: `test/release-channel-w1.serial.test.ts`, `test/upgrade-stable-channel.test.ts`, `test/install-ref.test.ts`. Docs: `docs/guides/upgrades-auto-update.md#stable-channel`.
-- `src/commands/upgrade-restart.ts` — GBRA-175: the host side of the upgrade restart. Real process sources (consumer heartbeat rows on Postgres, the supervisor PID file, the worker registry, `ps -U <uid> -ww -o pid=,ppid=,etime=,command=` with `/proc/<pid>/environ` GBRAIN_HOME matching on Linux; the install version above each entry point), the installed serve and autopilot services with the pid each manager runs (`hostServices`: `systemctl --user show -p MainPID`, `launchctl list`), `terminateIfUnchanged` (SIGTERM an MCP pid only while `ps` still shows the same server and start time), `restartRefusal` (ask_user + `--stop-mcp` when only MCP servers block, else the first manual step for the user), `preflightHostRestart` (pre-swap, every process stale) and `runHostRestart` (`gbrain upgrade --restart-only [--stop-mcp] [--dry-run] [--json]`: plan, run, verify for up to 180 s (360 s after an autopilot hand-off) with the warm-socket probe `probeCliSocket`). #7475: `restartServeAfterSelfUpgrade`, called each autopilot-daemon tick, restarts the serve the serve service runs once it is on an older build than the relaunched autopilot. It runs once per autopilot process, only in auto mode, in quiet hours and when the brain is idle (`computeAutopilotIdle`), and audits as `restart_serve`. #8776: `autopilotLauncher` (this brain's gbrain cron line from `crontab -l`, else `self_upgrade.autopilot_launcher external`) feeds the planner's hand-off; `upgradeRestartReadiness` is the preflight plan from the process table alone, which doctor's `self_upgrade_health` reads (local doctor only). `planServeRestart` reports a serve the service does not run, or one on another install, as `manual` and never restarts it. A serve counts as older only on evidence (a recorded build below this one, or no recorded build and a start before the install's `package.json` was written); a newer or unknown serve, or an install on disk that is not this build, is left alone. Tests: `upgrade-restart.test.ts`, `autopilot-serve-restart-7475.test.ts`.
+- `src/commands/upgrade-restart.ts` — GBRA-175: the host side of the upgrade restart. Real process sources (consumer heartbeat rows on Postgres, the supervisor PID file, the worker registry, `ps -U <uid> -ww -o pid=,ppid=,etime=,command=` with `/proc/<pid>/environ` GBRAIN_HOME matching on Linux; the install version above each entry point), the installed serve and autopilot services with the pid each manager runs (`hostServices`: `systemctl --user show -p MainPID`, `launchctl list`), `terminateIfUnchanged` (SIGTERM an MCP pid only while `ps` still shows the same server and start time), `restartRefusal` (ask_user + `--stop-mcp` when only MCP servers block, else the first manual step for the user), `preflightHostRestart` (pre-swap, every process stale) and `runHostRestart` (`gbrain upgrade --restart-only [--stop-mcp] [--dry-run] [--json]`: plan, run, verify for up to 180 s (360 s after an autopilot hand-off) with the warm-socket probe `probeCliSocket`). #7475: `restartServeAfterSelfUpgrade`, called each autopilot-daemon tick, restarts the serve the serve service runs once it is on an older build than the relaunched autopilot. It runs once per installed build (#8013: `serve-restart-attempt.json` survives autopilot relaunches), skips a serve service whose wrapper belongs to another brain (`serveServiceRunsWrapper`), runs the restart without blocking the daemon (`hostExecutor` spawns asynchronously, 120 s cap), only in auto mode, in quiet hours and when the brain is idle (`computeAutopilotIdle`), and audits as `restart_serve`. #8776: `autopilotLauncher` (this brain's gbrain cron line from `crontab -l`, else `self_upgrade.autopilot_launcher external`) feeds the planner's hand-off; `upgradeRestartReadiness` is the preflight plan from the process table alone, which doctor's `self_upgrade_health` reads (local doctor only). `planServeRestart` reports a serve the service does not run, or one on another install, as `manual` and never restarts it. A serve counts as older only on evidence (a recorded build below this one, or no recorded build and a start before the install's `package.json` was written); a newer or unknown serve, or an install on disk that is not this build, is left alone. Tests: `upgrade-restart.test.ts`, `autopilot-serve-restart-7475.test.ts`, `autopilot-serve-restart-8013.test.ts`.
- `src/core/upgrade-restart.ts` — GBRA-175: the pure half of the upgrade restart. `collectHostProcesses` (the process table is the authority: a heartbeat, PID-file or registry record whose pid `ps` does not show as gbrain is dropped; `gbrainEntry` skips wrappers such as tini, `bunx`, `bun x` and `bun run <bin>`, skips interpreter flags that take a value, resolves a relative entry against the process's `/proc/<pid>/cwd` and accepts a `src/cli.ts` whose package two levels up is named gbrain (#8723); a process's parent is its nearest gbrain ancestor, and a worker with none is refused naming its parent's pid and executable, never its command line), `planRestart` (a service restarts only processes under its manager's pid; `supervisorRestartArgv` keeps the launch options and PID file, read from `--pid-file <path>`, and drops a `--flag=value` token that is neither a global flag nor `--job-isolation=`, because the supervisor that holds it ignored it and this build's `=` gate refuses it (#7619); an autopilot under no installed service gets a `handoff` step when `ctx.autopilotLauncher` is known (#8776: SIGTERM while the pid is unchanged, then the launcher must bring autopilot back), else a manual step naming `gbrain autopilot --install`; anything else is a `ManualStep`), `runRestartSteps`, `restartProblems` (old pids gone, one build, roles and the supervised worker back, warm socket accepts) and `unverifiedBuilds`. A stale process whose `install_root` is another install than the one being upgraded gets a `replace_copy` step first when it is a Bun global copy (`bunGlobalCopyOf`: probe the tag, `bun remove -g` then `bun add -g` under its `BUN_INSTALL`, restore on failure), else a manual step from `copyFix`; after the restart `mismatchedBuilds` names any process back on another build, and the host's `mismatchFix` chains that replacement with `gbrain upgrade --restart-only`.
- `src/commands/watch.ts` — `gbrain watch`: the push transport. Reads turns from stdin as they arrive (`user:`/`assistant:` prefixes; unprefixed = user turn), keeps a rolling in-process window (`--window-turns`, default 4), calls `volunteerContext` per turn, streams pointers to stdout (`--json` for JSONL with turn attribution), logs `channel: 'watch'` events with a per-session id. Session dedupe feeds already-pushed slugs back as priorContext so the core's slug-only suppression dedupes. Blocks in the stdin iteration (interactive alive until Ctrl-C/Ctrl-D; piped ends at EOF) — deliberately NOT in DAEMON_COMMANDS; SIGINT closes the stream so teardown flows through finishCliTeardown. Per-turn resolution failures are fail-open. Registered in CLI_ONLY + CLI_ONLY_SELF_HELP + THIN_CLIENT_REFUSED_COMMANDS (thin clients use the `volunteer_context` MCP op). Pinned by `test/watch-command.test.ts`.
diff --git a/docs/guides/upgrades-auto-update.md b/docs/guides/upgrades-auto-update.md
index 956f07be63..86e02d416f 100644
--- a/docs/guides/upgrades-auto-update.md
+++ b/docs/guides/upgrades-auto-update.md
@@ -346,12 +346,14 @@ Once the relaunched autopilot runs the new build, it checks the serve that the
installed serve service runs (`com.gbrain.serve` under launchd,
`gbrain-serve.service` under systemd). If that serve is on an older build, the
autopilot restarts it through the service and verifies that it came back on
-the new build. It does this once per autopilot process, under the swap's own
+the new build. It tries once per installed build (`serve-restart-attempt.json`
+remembers the attempt across autopilot relaunches), under the swap's own
auto-mode, quiet-hours and idle gates, and records it in the self-upgrade
audit as `restart_serve` (`applied`, or `skipped` with `manual` or
`unverified`). It leaves alone a serve that the service does not run (one
-started by hand or from another install), the jobs supervisor and MCP
-servers. For those, doctor's `runtime_build_skew` fails while two builds run,
+started by hand or from another install), a serve service installed for
+another brain on the same OS user (its wrapper is under another
+`GBRAIN_HOME`), the jobs supervisor and MCP servers. For those, doctor's `runtime_build_skew` fails while two builds run,
and `gbrain upgrade --restart-only` is its fix.
<a id="restart-and-verify"></a>
diff --git a/src/commands/upgrade-restart.ts b/src/commands/upgrade-restart.ts
index b90a2a4322..135678a027 100644
--- a/src/commands/upgrade-restart.ts
+++ b/src/commands/upgrade-restart.ts
@@ -8,16 +8,16 @@
* onto the build already installed (doctor's `runtime_build_skew` fix);
* `--dry-run` prints the plan and changes nothing.
*/
-import { existsSync, readFileSync, readlinkSync, realpathSync, statSync } from 'node:fs';
+import { existsSync, mkdirSync, readFileSync, readlinkSync, realpathSync, statSync, writeFileSync } from 'node:fs';
import { homedir } from 'node:os';
import { delimiter, dirname, isAbsolute, join } from 'node:path';
-import { spawnSync } from '../core/spawn.ts';
-import { isThinClient, loadConfig, toEngineConfig } from '../core/config.ts';
+import { spawn, spawnSync } from '../core/spawn.ts';
+import { gbrainPath, isThinClient, loadConfig, toEngineConfig } from '../core/config.ts';
import { opError } from '../core/ops/contract.ts';
import type { Action } from '../core/agent-output.ts';
import { writeCliError } from '../cli/cli-error.ts';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
-import { SERVE_LAUNCHD_LABEL, SERVE_SYSTEMD_UNIT, serveServiceRestartArgv } from '../core/serve-service.ts';
+import { SERVE_LAUNCHD_LABEL, SERVE_SYSTEMD_UNIT, serveServiceRestartArgv, serveServiceRunsWrapper } from '../core/serve-service.ts';
import { autopilotLockPath, resolveAutopilotJob } from '../core/autopilot-paths.ts';
import { cronLineBelongsToBrain, plistPath, scriptWrapperPath, systemdUnitPath } from './autopilot/jobs.ts';
import { readSupervisorBuild, readSupervisorPid } from '../core/minions/supervisor-pid.ts';
@@ -195,12 +195,22 @@ async function hostSources(withHeartbeats = true): Promise<{ sources: ProcessSou
}
/** The real executor for restart steps; `pidFile` is the default supervisor PID file. */
-function hostExecutor(pidFile: string): RestartExecutor {
+/** Runs restart commands without blocking the event loop (the autopilot daemon keeps its timers and signals); a command still running after `timeoutMs` is killed. */
+export function hostExecutor(pidFile: string, timeoutMs = 600_000): RestartExecutor {
return {
- run: (argv, env) => {
- const r = spawnSync(argv[0]!, argv.slice(1), { encoding: 'utf8', timeout: 600_000, stdio: ['ignore', 'pipe', 'pipe'], env: env ? { ...process.env, ...env } : process.env });
- return { status: r.error ? null : r.status, output: `${r.stdout ?? ''}${r.stderr ?? ''}${r.error ? r.error.message : ''}` };
- },
+ run: (argv, env) => new Promise(resolve => {
+ let output = '';
+ let timedOut = false;
+ const child = spawn(argv[0]!, argv.slice(1), { stdio: ['ignore', 'pipe', 'pipe'], env: env ? { ...process.env, ...env } : process.env });
+ const timer = setTimeout(() => { timedOut = true; child.kill('SIGKILL'); }, timeoutMs);
+ child.stdout?.on('data', chunk => { output += chunk; });
+ child.stderr?.on('data', chunk => { output += chunk; });
+ child.on('error', error => { clearTimeout(timer); resolve({ status: null, output: `${output}${error.message}` }); });
+ child.on('close', code => {
+ clearTimeout(timer);
+ resolve({ status: timedOut ? null : code, output: timedOut ? `${output}${argv[0]} timed out after ${Math.round(timeoutMs / 1000)}s` : output });
+ });
+ }),
terminate: terminateIfUnchanged,
supervisorPid: file => { const status = readSupervisorPid(file ?? pidFile); return status.running ? status.pid : null; },
sleep: ms => new Promise(r => setTimeout(r, ms)),
@@ -390,6 +400,7 @@ export function planServeRestart(processes: readonly HostProcess[], serveService
export async function restartStaleServeService(): Promise<ServeRestartOutcome> {
const services = hostServices();
if (!services.serve) return { outcome: 'no_service' };
+ if (!serveServiceRunsWrapper(process.platform, homedir())) return { outcome: 'manual', detail: 'the serve service on this host starts another brain\'s serve wrapper (another GBRAIN_HOME), so this autopilot leaves it alone; run `gbrain upgrade --restart-only` with that brain\'s GBRAIN_HOME' };
const { sources, close, pidFile } = await hostSources();
try {
const collectServe = async () => (await collectHostProcesses(sources)).filter(p => p.role === 'serve');
@@ -403,7 +414,7 @@ export async function restartStaleServeService(): Promise<ServeRestartOutcome> {
const { probeCliSocket } = await import('../core/cli-socket/probe.ts');
const warmBefore = await probeCliSocket();
const startedMs = Date.now();
- const results = await runRestartSteps(plan, hostExecutor(pidFile));
+ const results = await runRestartSteps(plan, hostExecutor(pidFile, SERVE_RESTART_TIMEOUT_MS));
const verified = await verifyRestart(plan, startedMs, warmBefore, {
collect: collectServe, probeWarmSocket: () => probeCliSocket(),
sleep: ms => new Promise(r => setTimeout(r, ms)), now: () => Date.now(),
@@ -417,12 +428,18 @@ export async function restartStaleServeService(): Promise<ServeRestartOutcome> {
export type ServeRestartDeps = { state: { tried: boolean }; restart: () => Promise<ServeRestartOutcome> };
const SERVE_RESTART_DEPS: ServeRestartDeps = { state: { tried: false }, restart: restartStaleServeService };
+const SERVE_RESTART_TIMEOUT_MS = 120_000;
+const serveRestartMarkerPath = () => gbrainPath('serve-restart-attempt.json');
+
+function serveRestartAttemptedOnThisBuild(): boolean {
+ try { return (JSON.parse(readFileSync(serveRestartMarkerPath(), 'utf8')) as { version?: unknown }).version === VERSION; } catch { return false; }
+}
/**
* #7475: the autopilot daemon calls this each tick. In auto mode, under the self-upgrade swap's own quiet-hours and
* idle gates, the autopilot (relaunched on the new build after its swap) restarts the stale serve the serve service
- * runs, and audits the outcome as `restart_serve`. One attempt per autopilot process; the relaunch after the next
- * swap tries again. Never throws.
+ * runs, and audits the outcome as `restart_serve`. One attempt per build: `serve-restart-attempt.json` remembers it
+ * across autopilot relaunches (#8013), and the relaunch after the next swap tries again. Never throws.
*/
export async function restartServeAfterSelfUpgrade(isIdle: () => Promise<boolean>, deps: ServeRestartDeps = SERVE_RESTART_DEPS): Promise<void> {
if (deps.state.tried) return;
@@ -432,6 +449,9 @@ export async function restartServeAfterSelfUpgrade(isIdle: () => Promise<boolean
if (evaluateQuietHours(resolveQuietHoursWindow(cfg.self_upgrade?.quiet_hours), new Date()) === 'allow') return;
if (!await isIdle()) return;
deps.state.tried = true;
+ if (serveRestartAttemptedOnThisBuild()) return;
+ mkdirSync(dirname(serveRestartMarkerPath()), { recursive: true });
+ writeFileSync(serveRestartMarkerPath(), `${JSON.stringify({ version: VERSION, ts: new Date().toISOString() })}\n`);
const result = await deps.restart();
if (!('detail' in result)) return;
logSelfUpgrade({ channel: 'autopilot', action: 'restart_serve', current: VERSION, outcome: result.outcome === 'restarted' ? 'applied' : 'skipped', reason: result.outcome });
diff --git a/src/core/serve-service.ts b/src/core/serve-service.ts
index 9fd3058242..6f796cd981 100644
--- a/src/core/serve-service.ts
+++ b/src/core/serve-service.ts
@@ -57,6 +57,19 @@ export function serveServiceRestartArgv(service: { platform: string; home: strin
return null;
}
+/**
+ * True when the installed serve service starts `wrapper` (this brain's `wrapperPath()` by default). The launchd
+ * label and the systemd unit are one per OS user, so on a host with several brains the service may start another
+ * GBRAIN_HOME's wrapper; false then, and for a missing or unreadable service file.
+ */
+export function serveServiceRunsWrapper(platform: string, home: string, wrapper: string = wrapperPath()): boolean {
+ try {
+ if (platform === 'darwin') return readFileSync(launchdPlistPath(home), 'utf8').includes(`<string>${escapeXml(wrapper)}</string>`);
+ if (platform === 'linux') return readFileSync(systemdUnitPath(home), 'utf8').split('\n').includes(`ExecStart=${systemdArg(wrapper)}`);
+ } catch { /* no readable service file: not this brain's */ }
+ return false;
+}
+
// ---------------------------------------------------------------------------
// Target detection (pure — probe results are passed in)
// ---------------------------------------------------------------------------
diff --git a/src/core/upgrade-restart.ts b/src/core/upgrade-restart.ts
index 61784868f5..1158c06d51 100644
--- a/src/core/upgrade-restart.ts
+++ b/src/core/upgrade-restart.ts
@@ -357,8 +357,10 @@ export function planRestart(processes: readonly HostProcess[], ctx: RestartConte
return { expected: ctx.expected, current, stale, steps, manual };
}
+export interface RunResult { status: number | null; output: string }
+
export interface RestartExecutor {
- run: (argv: string[], env?: Record<string, string>) => { status: number | null; output: string };
+ run: (argv: string[], env?: Record<string, string>) => RunResult | Promise<RunResult>;
/** SIGTERM the MCP server or autopilot at pid only if it is still the process the plan saw: 'changed' when the pid now names another process. */
terminate: (pid: number, startedMs: number | null, role: 'mcp' | 'autopilot') => 'stopped' | 'gone' | 'changed';
/** The live supervisor's pid now (read from its PID file, the default one when null), or null. */
@@ -378,18 +380,18 @@ export async function runRestartSteps(plan: RestartPlan, exec: RestartExecutor):
const env = { BUN_INSTALL: step.bun_install };
const probe = ['git', 'ls-remote', '--exit-code', '--tags', 'https://github.com/garrytan/gbrain', `refs/tags/v${step.to}`];
const add = ['bun', 'add', '-g', `github:garrytan/gbrain#v${step.to}`];
- const probed = exec.run(probe);
+ const probed = await exec.run(probe);
if (probed.status !== 0) { results.push({ step, ok: false, detail: `v${step.to} has no release tag, so the copy at ${step.root} was left as it is (${failed(probe, probed)})` }); continue; }
- const removed = exec.run(['bun', 'remove', '-g', 'gbrain'], env);
- const added = removed.status === 0 ? exec.run(add, env) : removed;
+ const removed = await exec.run(['bun', 'remove', '-g', 'gbrain'], env);
+ const added = removed.status === 0 ? await exec.run(add, env) : removed;
if (added.status === 0) { results.push({ step, ok: true, detail: `replaced the Bun global copy at ${step.root} with v${step.to}` }); continue; }
- const restored = removed.status === 0 && step.from ? exec.run(['bun', 'add', '-g', `github:garrytan/gbrain#v${step.from}`], env).status === 0 : false;
+ const restored = removed.status === 0 && step.from ? (await exec.run(['bun', 'add', '-g', `github:garrytan/gbrain#v${step.from}`], env)).status === 0 : false;
results.push({ step, ok: false, detail: `could not replace the copy at ${step.root} (${failed(removed.status === 0 ? add : ['bun', 'remove', '-g', 'gbrain'], added)})${restored ? `; restored v${step.from}` : ''}` });
} else if (step.kind === 'service') {
- const r = exec.run(step.argv);
+ const r = await exec.run(step.argv);
results.push({ step, ok: r.status === 0, detail: r.status === 0 ? `ran \`${step.argv.join(' ')}\`` : failed(step.argv, r) });
} else if (step.kind === 'supervisor') {
- const stop = exec.run(step.stop);
+ const stop = await exec.run(step.stop);
if (stop.status !== 0) { results.push({ step, ok: false, detail: failed(step.stop, stop) }); continue; }
await exec.sleep(SUPERVISOR_RESPAWN_GRACE_MS);
const respawned = exec.supervisorPid(step.pid_file);
@@ -397,7 +399,7 @@ export async function runRestartSteps(plan: RestartPlan, exec: RestartExecutor):
results.push({ step, ok: true, detail: `stopped supervisor pid ${step.pid}; its service manager started pid ${respawned}` });
continue;
}
- const start = exec.run(step.start);
+ const start = await exec.run(step.start);
results.push({ step, ok: start.status === 0, detail: start.status === 0 ? `stopped supervisor pid ${step.pid} and started it again with \`${step.start.join(' ')}\`` : failed(step.start, start) });
} else if (step.kind === 'handoff') {
const outcome = exec.terminate(step.pid, step.started_ms, 'autopilot');
diff --git a/test/autopilot-serve-restart-8013.test.ts b/test/autopilot-serve-restart-8013.test.ts
new file mode 100644
index 0000000000..ffd3a9bd0b
--- /dev/null
+++ b/test/autopilot-serve-restart-8013.test.ts
@@ -0,0 +1,116 @@
+/**
+ * #8013: follow-ups to the #7475 serve restart. The attempt is remembered per build across autopilot relaunches, a
+ * serve service installed for another brain is never restarted, and the restart command runs without blocking the
+ * autopilot's event loop.
+ */
+import { afterAll, describe, expect, test } from 'bun:test';
+import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { hostExecutor, restartServeAfterSelfUpgrade, type ServeRestartOutcome } from '../src/commands/upgrade-restart.ts';
+import { readRecentSelfUpgrades } from '../src/core/audit/self-upgrade-audit.ts';
+import { launchdPlistPath, renderServeLaunchdPlist, renderServeSystemdUnit, serveServiceRunsWrapper, systemdUnitPath } from '../src/core/serve-service.ts';
+import { withEnv } from './helpers/with-env.ts';
+
+const dirs: string[] = [];
+afterAll(() => { for (const d of dirs) rmSync(d, { recursive: true, force: true }); });
+const tempDir = (prefix: string) => { const d = mkdtempSync(join(tmpdir(), prefix)); dirs.push(d); return d; };
+
+describe('serve restart attempt marker (#8013)', () => {
+ function brainHome(): { home: string; audit: string } {
+ const root = tempDir('gbrain-serve-restart-8013-');
+ const start = new Date().getUTCHours();
+ mkdirSync(join(root, '.gbrain'), { recursive: true });
+ writeFileSync(join(root, '.gbrain', 'config.json'), JSON.stringify({
+ engine: 'pglite', database_path: join(root, '.gbrain', 'brain.pglite'),
+ self_upgrade: { mode: 'auto', quiet_hours: { start, end: (start + 2) % 24, tz: 'UTC' } },
+ }));
+ return { home: root, audit: join(root, 'audit') };
+ }
+
+ test('a failed restart is not repeated by the next autopilot process on the same build', async () => {
+ const { home, audit } = brainHome();
+ let calls = 0;
+ const restart = async (): Promise<ServeRestartOutcome> => { calls++; return { outcome: 'unverified', detail: 'serve did not come back' }; };
+ await withEnv({ GBRAIN_HOME: home, GBRAIN_AUDIT_DIR: audit }, async () => {
+ await restartServeAfterSelfUpgrade(async () => true, { state: { tried: false }, restart });
+ await restartServeAfterSelfUpgrade(async () => true, { state: { tried: false }, restart });
+ expect(readRecentSelfUpgrades(1).filter(e => e.action === 'restart_serve')).toHaveLength(1);
+ });
+ expect(calls).toBe(1);
+ });
+
+ test('a marker left by another build does not hold the restart back', async () => {
+ const { home, audit } = brainHome();
+ writeFileSync(join(home, '.gbrain', 'serve-restart-attempt.json'), JSON.stringify({ version: '0.0.1.0', ts: '2026-01-01T00:00:00.000Z' }));
+ let calls = 0;
+ await withEnv({ GBRAIN_HOME: home, GBRAIN_AUDIT_DIR: audit }, () =>
+ restartServeAfterSelfUpgrade(async () => true, { state: { tried: false }, restart: async () => { calls++; return { outcome: 'current' }; } }));
+ expect(calls).toBe(1);
+ });
+
+ test('a busy brain leaves no marker, so a later idle tick still tries', async () => {
+ const { home, audit } = brainHome();
+ await withEnv({ GBRAIN_HOME: home, GBRAIN_AUDIT_DIR: audit }, () =>
+ restartServeAfterSelfUpgrade(async () => false, { state: { tried: false }, restart: async () => ({ outcome: 'current' }) }));
+ expect(existsSync(join(home, '.gbrain', 'serve-restart-attempt.json'))).toBe(false);
+ });
+});
+
+describe('serveServiceRunsWrapper (#8013)', () => {
+ const own = '/srv/brain-a/.gbrain/serve/gbrain-serve.sh';
+ const other = '/srv/brain-b/.gbrain/serve/gbrain-serve.sh';
+
+ test('launchd: true only for the plist that starts this brain\'s wrapper', () => {
+ const home = tempDir('gbrain-serve-owner-mac-');
+ mkdirSync(join(home, 'Library', 'LaunchAgents'), { recursive: true });
+ writeFileSync(launchdPlistPath(home), renderServeLaunchdPlist({ wrapperPath: own, home, logPath: '/tmp/l', errPath: '/tmp/e' }));
+ expect(serveServiceRunsWrapper('darwin', home, own)).toBe(true);
+ expect(serveServiceRunsWrapper('darwin', home, other)).toBe(false);
+ });
+
+ test('systemd: true only for the unit that starts this brain\'s wrapper, including a path with spaces', () => {
+ const home = tempDir('gbrain-serve-owner-linux-');
+ const spaced = '/srv/brain a/.gbrain/serve/gbrain-serve.sh';
+ mkdirSync(join(home, '.config', 'systemd', 'user'), { recursive: true });
+ writeFileSync(systemdUnitPath(home), renderServeSystemdUnit({ wrapperPath: spaced, logPath: '/tmp/l', errPath: '/tmp/e' }));
+ expect(serveServiceRunsWrapper('linux', home, spaced)).toBe(true);
+ expect(serveServiceRunsWrapper('linux', home, other)).toBe(false);
+ });
+
+ test('no service file, or another platform, is never this brain\'s', () => {
+ const home = tempDir('gbrain-serve-owner-none-');
+ expect(serveServiceRunsWrapper('darwin', home, own)).toBe(false);
+ expect(serveServiceRunsWrapper('linux', home, own)).toBe(false);
+ expect(serveServiceRunsWrapper('win32', home, own)).toBe(false);
+ });
+});
+
+describe('hostExecutor run (#8013)', () => {
+ test('a slow restart command leaves the event loop running', async () => {
+ let ticks = 0;
+ const timer = setInterval(() => { ticks++; }, 20);
+ try {
+ const r = await hostExecutor('/nonexistent/supervisor.pid').run(['sleep', '0.5']);
+ expect(r.status).toBe(0);
+ } finally {
+ clearInterval(timer);
+ }
+ expect(ticks).toBeGreaterThanOrEqual(10);
+ });
+
+ test('a command past its timeout is killed and reported as failed', async () => {
+ const started = Date.now();
+ const r = await hostExecutor('/nonexistent/supervisor.pid', 200).run(['sleep', '5']);
+ expect(r.status).toBeNull();
+ expect(r.output).toContain('timed out');
+ expect(Date.now() - started).toBeLessThan(3_000);
+ });
+
+ test('output and exit status come back as before', async () => {
+ const r = await hostExecutor('/nonexistent/supervisor.pid').run(['sh', '-c', 'echo out; echo err >&2; exit 3']);
+ expect(r.status).toBe(3);
+ expect(r.output).toContain('out');
+ expect(r.output).toContain('err');
+ });
+});